<commit_before>package pdf

import (
	"errors"
	"fmt"
	"strconv" )

// Implements:
// 	pdf.Object
//	bufio.Writer
type Indirect struct {
	fileBindings map[File]ObjectNumber
	// When not nil, sourceFile is a file this indirect object was read from.
	sourceFile      File
}

/*
pdf.Indirect is one of the most important and most complex
object types.  Whereas "direct" objects are rendered the same way on
every output stream, a pdf.Indirect is rendered differently
depending on which file it is associated with.  There is always an
underlying "direct object" that is written to one or more PDF files,
where it is assigned both an object and generation number.  When the
Serialize() method is invoked, a pdf.Indirect is rendered as an
indirect reference (for example, "10 1 R").  There are several
use-cases.

USE-CASE 1: A pdf.Indirect object is created, its Serialize() method
is invoked for one or more output files, then its Write() method is
invoked with its underlying direct object.  The PDF reference suggests
that a reference is sometimes written before the information required
for the object being referenced is available.  An example from the PDF
reference is using an indirect object reference for the length of the
stream before writing the stream, presumably so you can write the
stream dictionary at a moment when the length of the stream is
unknown.  The stream length is written as a separate object after the
stream is completed when the length is known with certainty.  This
particular example is somewhat contrived.  With the size of memory in
modern computers it's difficult to imagine a scenario where a stream
cannot be written to a buffer in its entirely and written as an atomic
stream object.  Nonetheless, a pdf.Indirect object supports this
programming style where object references are written to a file before
the objects they refer to have been completely defined.  We do *not*
support this model with streams, however, and *do* require streams to
be completed in memory before any portion of the strean is written to
a file.  A pdf.Indirect can be written before the direct object it
references has been defined.  A pdf.Indirect obtains and reserves an
object number whenever it is written to a file, whether or not the
object being referenced has yet been specified.  Eventually, the
Write() method must be called passing the object being referenced.
At that moment, the object being referenced is written to all files to
which the pdf.Indirect was written.  If the pdf.Indirect is
subsequently added to additional files, the Write()ed object must
also written to those files.  This, however, would require either
retaining a reference in memory indefinitely (bad) or reading it from
one of the files where it is known to exist (reading is not yet
implemented; it must be read as opposed to just copied because any
indirect references contained within it also need to be read and added
to the file accordingly).  For the time being, we elect not to keep
references in memory.  Until parsing is implemented, indirect objects
may be explicitly bound to files either (1) by using ObjectNumber()
prior to calling Write or (2) by calling pdf.File.AddObject().
Serialize()ing a pdf.Indirect to a new file after calling Write()
without an earlier call to Serialize() or ObjectNumber() will
generate an error.  The complete list of files that will contain the
refence must be known when Write() is called.

The call to ObjectNumber() is handled transparently and automatically
for forward references.  The client need not call it explicitly.  A
call to ObjectNumber() is required, however, for indirect objects that
are backward references.  An alternative way to obtain a backward
reference is using the return value from pdf.File.AddObject().  The
reference returned by pdf.File.AddObject() is bound only to one file.

USE-CASE 2: A pdf.Indirect is created based on a finished direct
object.  This is essentially the same as use-case 1.  An object is
constructed and Write() is called immediately.  Subsequent invocations
of Serialize() on files where it doesn't already exist cause it to be
added to that file.  As with USE-CASE 1, this requires either
retaining a reference in memory indefinitely (bad) or reading from one
of the files where it is known to exist (not yet implemented).  For
the time being, we elect not to keep references in memory. Until
parsing is implemented, so Serialize()ing a pdf.Indirect to a file
after calling Write() is an error.

USE-CASE 3: A pdf.Indirect is constructed when a token of the form "10
1 R" is read from file X.  The underlying direct object is unknown at
that moment, but it exists in X.  Since the object exists statically
in X, it is considered to have been finalized.  If the same object is
Serialize()'ed to file Y, then it should immediately be added to file
Y.  The objects contents are obtained from X.

POSSIBLE TEMPORARY BEHAVIOR: The referenced object is retained by the
pdf.Indirect so that it can be "dereferenced" or written to additional
files.  In future versions that are able to parse PDF files, the
object may be discarded from memory once written and read back from
disk if it is dererenced or written to another file.  Even better
would be a weak-reference to the object, but weak references are not
implemented in Go.

*/

// NewIndirect is the constructor for Indirect objects.  For
// convenience, if the files to which this indirect object should be
// bound are known at construction time, they may be provided as
// optional arguments.  Instead of providing these at construction,
// the client may call Indirect.ObjectNumber() after construction.
func NewIndirect(file... File) *Indirect {
	result := new(Indirect)
	result.fileBindings = make(map[File]ObjectNumber,5)
	result.sourceFile = nil

	for _,f := range file {
		result.ObjectNumber(f)
	}

	return result
}

func newIndirectWithNumber(objectNumber ObjectNumber, file File) *Indirect {
	result := new(Indirect)
	result.fileBindings = make(map[File]ObjectNumber,5)
	result.sourceFile = file
	result.fileBindings[file] = objectNumber
	return result
}

func (i *Indirect) Clone() Object {
	newIndirect := new(Indirect)
	newIndirect.fileBindings = i.fileBindings
	newIndirect.sourceFile = i.sourceFile
	return newIndirect
}

func (i *Indirect) Dereference() Object {
	if i.sourceFile == nil {
		panic (errors.New(`Attempt to deference an object with no known source`))
	}

	object,err := i.sourceFile.Object(i.ObjectNumber(i.sourceFile))
	if err != nil {
		panic (errors.New(fmt.Sprintf(`Unable to read object at %v`, i.ObjectNumber(i.sourceFile))))
	}
	// TODO:  Think about whether Dereference() is a good idea here.
	return object.Dereference()
}

// Serialize() writes a serial representation (as defined by the PDF
// specification) of the object to the Writer.  Indirect references
// are resolved and numbered as if they were being written to the
// optional File argument.  Having separate arguments for Writer and
// File allows writing an object to stdout, but using the indirect
// reference object numbers as if it were contained in a specific PDF
// file.
func (i *Indirect) Serialize(w Writer, file ...File) {
	if len(file) != 1 {
		panic("A single file parameter is required for pdf.Indirect.Serialize()")
	}
	if file[0].Closed() {
		panic("Attempt to Serialize to a closed file")
	}

	objectNumber := i.ObjectNumber(file[0])
	w.WriteString(strconv.FormatInt(int64(objectNumber.number), 10))
	w.WriteByte(' ')
	w.WriteString(strconv.FormatInt(int64(objectNumber.generation), 10))
	w.WriteString(" R")
}

// Write() writes the passed object as an indirect object (complete
// with an entry in the xref, an "obj" header, and an "endobj"
// trailer) to all files to which the Indirect object has been bound.
// Write() may be used to replace an existing object.
// Write() returns its Indirect object for constructions such as
//  a := NewIndirect(f).Write(object)
func (i *Indirect) Write(o Object) *Indirect{
	for file, objectNumber := range i.fileBindings {
		file.WriteObjectAt(objectNumber, o)
		if i.sourceFile == nil {
			i.sourceFile = file
		}
	}
	return i
}

// ObjectNumber() binds its object to the passed pdf.File object and
// returns an object number associated with that file.  Normally it is
// called automatically and transparently whenever an indirect
// reference is contained within some other object that is explicitly
// written to a file.  Client code may call ObjectNumber() explicitly
// if the client intends to write the object to a number of files
// using Indirect.Write().  that case, the caller should call
// ObjectNumber() one or more times *before* calling Write().
// Alternatively, client code may call File.Write(), which returns a
// *Indirect that may be used for backward references.  In the latter
// case, the reference will only be tied to only one file.
func (i *Indirect) ObjectNumber(f File) ObjectNumber {
	destObjectNumber,exists := i.fileBindings[f]
	if !exists {
		destObjectNumber = f.ReserveObjectNumber(i)
		i.fileBindings[f] = destObjectNumber
		// If the object was a pre-existing object, silently
		// add it to "file."
		if i.sourceFile != nil {
			sourceObjectNumber := i.fileBindings[i.sourceFile]
			o, err := i.sourceFile.Object(sourceObjectNumber)
			if err == nil {
				f.WriteObjectAt(destObjectNumber,o)
			}
		}
	}
	return destObjectNumber
}

func (i *Indirect) BoundToFile(f File) bool {
	_,exists := i.fileBindings[f]
	return exists
}
<commit_msg>Create maps in Indirect objects with an initial size of 1. Every new Indirect object is stored and retained in the in-memory copy of the xref as long as the file is open. Limiting the size to 1 minimizes the memory footprint.<commit_after>package pdf

import (
	"errors"
	"fmt"
	"strconv" )

// Implements:
// 	pdf.Object
//	bufio.Writer
type Indirect struct {
	fileBindings map[File]ObjectNumber
	// When not nil, sourceFile is a file this indirect object was read from.
	sourceFile      File
}

/*
pdf.Indirect is one of the most important and most complex
object types.  Whereas "direct" objects are rendered the same way on
every output stream, a pdf.Indirect is rendered differently
depending on which file it is associated with.  There is always an
underlying "direct object" that is written to one or more PDF files,
where it is assigned both an object and generation number.  When the
Serialize() method is invoked, a pdf.Indirect is rendered as an
indirect reference (for example, "10 1 R").  There are several
use-cases.

USE-CASE 1: A pdf.Indirect object is created, its Serialize() method
is invoked for one or more output files, then its Write() method is
invoked with its underlying direct object.  The PDF reference suggests
that a reference is sometimes written before the information required
for the object being referenced is available.  An example from the PDF
reference is using an indirect object reference for the length of the
stream before writing the stream, presumably so you can write the
stream dictionary at a moment when the length of the stream is
unknown.  The stream length is written as a separate object after the
stream is completed when the length is known with certainty.  This
particular example is somewhat contrived.  With the size of memory in
modern computers it's difficult to imagine a scenario where a stream
cannot be written to a buffer in its entirely and written as an atomic
stream object.  Nonetheless, a pdf.Indirect object supports this
programming style where object references are written to a file before
the objects they refer to have been completely defined.  We do *not*
support this model with streams, however, and *do* require streams to
be completed in memory before any portion of the strean is written to
a file.  A pdf.Indirect can be written before the direct object it
references has been defined.  A pdf.Indirect obtains and reserves an
object number whenever it is written to a file, whether or not the
object being referenced has yet been specified.  Eventually, the
Write() method must be called passing the object being referenced.
At that moment, the object being referenced is written to all files to
which the pdf.Indirect was written.  If the pdf.Indirect is
subsequently added to additional files, the Write()ed object must
also written to those files.  This, however, would require either
retaining a reference in memory indefinitely (bad) or reading it from
one of the files where it is known to exist (reading is not yet
implemented; it must be read as opposed to just copied because any
indirect references contained within it also need to be read and added
to the file accordingly).  For the time being, we elect not to keep
references in memory.  Until parsing is implemented, indirect objects
may be explicitly bound to files either (1) by using ObjectNumber()
prior to calling Write or (2) by calling pdf.File.AddObject().
Serialize()ing a pdf.Indirect to a new file after calling Write()
without an earlier call to Serialize() or ObjectNumber() will
generate an error.  The complete list of files that will contain the
refence must be known when Write() is called.

The call to ObjectNumber() is handled transparently and automatically
for forward references.  The client need not call it explicitly.  A
call to ObjectNumber() is required, however, for indirect objects that
are backward references.  An alternative way to obtain a backward
reference is using the return value from pdf.File.AddObject().  The
reference returned by pdf.File.AddObject() is bound only to one file.

USE-CASE 2: A pdf.Indirect is created based on a finished direct
object.  This is essentially the same as use-case 1.  An object is
constructed and Write() is called immediately.  Subsequent invocations
of Serialize() on files where it doesn't already exist cause it to be
added to that file.  As with USE-CASE 1, this requires either
retaining a reference in memory indefinitely (bad) or reading from one
of the files where it is known to exist (not yet implemented).  For
the time being, we elect not to keep references in memory. Until
parsing is implemented, so Serialize()ing a pdf.Indirect to a file
after calling Write() is an error.

USE-CASE 3: A pdf.Indirect is constructed when a token of the form "10
1 R" is read from file X.  The underlying direct object is unknown at
that moment, but it exists in X.  Since the object exists statically
in X, it is considered to have been finalized.  If the same object is
Serialize()'ed to file Y, then it should immediately be added to file
Y.  The objects contents are obtained from X.

POSSIBLE TEMPORARY BEHAVIOR: The referenced object is retained by the
pdf.Indirect so that it can be "dereferenced" or written to additional
files.  In future versions that are able to parse PDF files, the
object may be discarded from memory once written and read back from
disk if it is dererenced or written to another file.  Even better
would be a weak-reference to the object, but weak references are not
implemented in Go.

*/

// NewIndirect is the constructor for Indirect objects.  For
// convenience, if the files to which this indirect object should be
// bound are known at construction time, they may be provided as
// optional arguments.  Instead of providing these at construction,
// the client may call Indirect.ObjectNumber() after construction.
func NewIndirect(file... File) *Indirect {
	result := new(Indirect)
	result.fileBindings = make(map[File]ObjectNumber,1)
	result.sourceFile = nil

	for _,f := range file {
		result.ObjectNumber(f)
	}

	return result
}

func newIndirectWithNumber(objectNumber ObjectNumber, file File) *Indirect {
	result := new(Indirect)
	result.fileBindings = make(map[File]ObjectNumber,5)
	result.sourceFile = file
	result.fileBindings[file] = objectNumber
	return result
}

func (i *Indirect) Clone() Object {
	newIndirect := new(Indirect)
	newIndirect.fileBindings = i.fileBindings
	newIndirect.sourceFile = i.sourceFile
	return newIndirect
}

func (i *Indirect) Dereference() Object {
	if i.sourceFile == nil {
		panic (errors.New(`Attempt to deference an object with no known source`))
	}

	object,err := i.sourceFile.Object(i.ObjectNumber(i.sourceFile))
	if err != nil {
		panic (errors.New(fmt.Sprintf(`Unable to read object at %v`, i.ObjectNumber(i.sourceFile))))
	}
	// TODO:  Think about whether Dereference() is a good idea here.
	return object.Dereference()
}

// Serialize() writes a serial representation (as defined by the PDF
// specification) of the object to the Writer.  Indirect references
// are resolved and numbered as if they were being written to the
// optional File argument.  Having separate arguments for Writer and
// File allows writing an object to stdout, but using the indirect
// reference object numbers as if it were contained in a specific PDF
// file.
func (i *Indirect) Serialize(w Writer, file ...File) {
	if len(file) != 1 {
		panic("A single file parameter is required for pdf.Indirect.Serialize()")
	}
	if file[0].Closed() {
		panic("Attempt to Serialize to a closed file")
	}

	objectNumber := i.ObjectNumber(file[0])
	w.WriteString(strconv.FormatInt(int64(objectNumber.number), 10))
	w.WriteByte(' ')
	w.WriteString(strconv.FormatInt(int64(objectNumber.generation), 10))
	w.WriteString(" R")
}

// Write() writes the passed object as an indirect object (complete
// with an entry in the xref, an "obj" header, and an "endobj"
// trailer) to all files to which the Indirect object has been bound.
// Write() may be used to replace an existing object.
// Write() returns its Indirect object for constructions such as
//  a := NewIndirect(f).Write(object)
func (i *Indirect) Write(o Object) *Indirect{
	for file, objectNumber := range i.fileBindings {
		file.WriteObjectAt(objectNumber, o)
		if i.sourceFile == nil {
			i.sourceFile = file
		}
	}
	return i
}

// ObjectNumber() binds its object to the passed pdf.File object and
// returns an object number associated with that file.  Normally it is
// called automatically and transparently whenever an indirect
// reference is contained within some other object that is explicitly
// written to a file.  Client code may call ObjectNumber() explicitly
// if the client intends to write the object to a number of files
// using Indirect.Write().  that case, the caller should call
// ObjectNumber() one or more times *before* calling Write().
// Alternatively, client code may call File.Write(), which returns a
// *Indirect that may be used for backward references.  In the latter
// case, the reference will only be tied to only one file.
func (i *Indirect) ObjectNumber(f File) ObjectNumber {
	destObjectNumber,exists := i.fileBindings[f]
	if !exists {
		destObjectNumber = f.ReserveObjectNumber(i)
		i.fileBindings[f] = destObjectNumber
		// If the object was a pre-existing object, silently
		// add it to "file."
		if i.sourceFile != nil {
			sourceObjectNumber := i.fileBindings[i.sourceFile]
			o, err := i.sourceFile.Object(sourceObjectNumber)
			if err == nil {
				f.WriteObjectAt(destObjectNumber,o)
			}
		}
	}
	return destObjectNumber
}

func (i *Indirect) BoundToFile(f File) bool {
	_,exists := i.fileBindings[f]
	return exists
}
<|endoftext|>


<commit_before>// Program add_argument inserts a new argument into a function and all of it's callers
//
// Example:
//   $ add_argument -arg="foo int" -pos=$GOPATH/src/github.com/tmc/refactor_utils/test/original/z.go:#20 github.com/tmc/refactor_utils/test/original
package main

import (
	"flag"
	"fmt"
	"os"
)

type Options struct {
	position       string   // position
	argument       string   // argument to add
	args           []string // ssa FromArgs
	write          bool
	skipExists     bool   // skip if specified name and type are already present
	packageNameRe  string // package name regexp
	callgraphDepth int    // depth up the callgraph to make modifications
}

var options Options

func init() {
	flag.StringVar(&options.argument, "arg", "",
		"argument to add to the specified function")
	flag.StringVar(&options.position, "pos", "",
		"Filename and byte offset or extent of a syntax element about which to "+
			"query, e.g. foo.go:#123,#456, bar.go:#123.")
	flag.BoolVar(&options.write, "w", false,
		"write result to (source) file instead of stdout")
	flag.BoolVar(&options.skipExists, "skip-exists", true,
		"if an argument appears to exist already don't add it")
	flag.StringVar(&options.packageNameRe, "package-regexp", "",
		"regular expression that package names much match to be modified")
	flag.IntVar(&options.callgraphDepth, "depth", -1,
		"callgraph traversal limit (-1 for unlimited) ")
}

func main() {
	flag.Parse()
	options.args = flag.Args()
	if err := commandAddArgument(options); err != nil {
		fmt.Fprintf(os.Stderr, "Error: %s.\n", err)
		os.Exit(1)
	}
}
<commit_msg>Reflect current name in example invocation<commit_after>// Program add_argument inserts a new argument into a function and all of it's callers
//
// Example:
//   $ add_argument -arg="foo int" -pos=$GOPATH/src/github.com/tmc/srcutils/test/original/z.go:#20 github.com/tmc/srcutils/test/original
package main

import (
	"flag"
	"fmt"
	"os"
)

type Options struct {
	position       string   // position
	argument       string   // argument to add
	args           []string // ssa FromArgs
	write          bool
	skipExists     bool   // skip if specified name and type are already present
	packageNameRe  string // package name regexp
	callgraphDepth int    // depth up the callgraph to make modifications
}

var options Options

func init() {
	flag.StringVar(&options.argument, "arg", "",
		"argument to add to the specified function")
	flag.StringVar(&options.position, "pos", "",
		"Filename and byte offset or extent of a syntax element about which to "+
			"query, e.g. foo.go:#123,#456, bar.go:#123.")
	flag.BoolVar(&options.write, "w", false,
		"write result to (source) file instead of stdout")
	flag.BoolVar(&options.skipExists, "skip-exists", true,
		"if an argument appears to exist already don't add it")
	flag.StringVar(&options.packageNameRe, "package-regexp", "",
		"regular expression that package names much match to be modified")
	flag.IntVar(&options.callgraphDepth, "depth", -1,
		"callgraph traversal limit (-1 for unlimited) ")
}

func main() {
	flag.Parse()
	options.args = flag.Args()
	if err := commandAddArgument(options); err != nil {
		fmt.Fprintf(os.Stderr, "Error: %s.\n", err)
		os.Exit(1)
	}
}
<|endoftext|>


<commit_before>package main

import (
	"flag"
	"fmt"
	"github.com/rkoesters/goblin/lib/flagutil"
	"os"
	"path"
	"strings"
)

// BUG(rkoesters) -d option shouldn't have trailing slash.
var dflag = flag.Bool("d", false, "print directory compenent")

func main() {
	flagutil.Usage = "string [suffix]"
	flag.Parse()

	if flag.NArg() < 1 || flag.NArg() > 2 {
		flag.Usage()
		os.Exit(1)
	}

	dir, file := path.Split(flag.Arg(0))

	if *dflag {
		fmt.Println(dir)
		return
	}

	fmt.Println(strings.TrimSuffix(file, flag.Arg(1)))
}
<commit_msg>Fixed cmd/basename.<commit_after>package main

import (
	"flag"
	"fmt"
	"github.com/rkoesters/goblin/lib/flagutil"
	"os"
	"strings"
)

var dflag = flag.Bool("d", false, "print directory compenent")

func main() {
	flagutil.Usage = "string [suffix]"
	flag.Parse()

	if flag.NArg() < 1 || flag.NArg() > 2 {
		flag.Usage()
		os.Exit(1)
	}

	var dir, file string
	s := flag.Arg(0)
	n := strings.LastIndex(s, string(os.PathSeparator))
	if n < 0 {
		dir = "."
		file = s
	} else {
		dir = s[:n]
		file = s[n+1:]
	}

	if *dflag {
		fmt.Println(dir)
	} else {
		fmt.Println(strings.TrimSuffix(file, flag.Arg(1)))
	}
}
<|endoftext|>


<commit_before>/*
Copyright 2011 Google Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package main

import (
	"flag"
	"fmt"
	"log"
	"net/http"
	"os"
	"os/signal"
	"strings"
	"syscall"

	"camlistore.org/pkg/blobref"
	"camlistore.org/pkg/cacher"
	"camlistore.org/pkg/client"
	"camlistore.org/pkg/fs"

	"camlistore.org/third_party/code.google.com/p/rsc/fuse"
)

func usage() {
	fmt.Fprint(os.Stderr, "usage: cammount [opts] <mountpoint> [<root-blobref>|<share URL>]\n")
	flag.PrintDefaults()
	os.Exit(2)
}

func main() {
	// Scans the arg list and sets up flags
	debug := flag.Bool("debug", false, "print debugging messages.")
	client.AddFlags()
	flag.Parse()

	errorf := func(msg string, args ...interface{}) {
		fmt.Fprintf(os.Stderr, msg, args...)
		fmt.Fprint(os.Stderr, "\n")
		usage()
	}

	nargs := flag.NArg()
	if nargs < 1 || nargs > 2 {
		usage()
	}

	mountPoint := flag.Arg(0)
	var (
		cl    *client.Client
		root  *blobref.BlobRef // nil if only one arg
		camfs *fs.CamliFileSystem
	)
	if nargs == 2 {
		rootArg := flag.Arg(1)
		// not trying very hard since NewFromShareRoot will do it better with a regex
		if strings.HasPrefix(rootArg, "http://") ||
			strings.HasPrefix(rootArg, "https://") {
			if client.ExplicitServer() != "" {
				errorf("Can't use an explicit blobserver with a share URL; the blobserver is implicit from the share URL.")
			}
			var err error
			cl, root, err = client.NewFromShareRoot(rootArg)
			if err != nil {
				log.Fatal(err)
			}
		} else {
			cl = client.NewOrFail() // automatic from flags
			root = blobref.Parse(rootArg)
			if root == nil {
				log.Fatalf("Error parsing root blobref: %q\n", rootArg)
			}
			cl.SetHTTPClient(&http.Client{Transport: cl.TransportForConfig(nil)})
		}
	} else {
		cl = client.NewOrFail() // automatic from flags
		cl.SetHTTPClient(&http.Client{Transport: cl.TransportForConfig(nil)})
	}
	// TODO(mpl): probably needs the transport setup for trusted certs here.

	diskCacheFetcher, err := cacher.NewDiskCache(cl)
	if err != nil {
		log.Fatalf("Error setting up local disk cache: %v", err)
	}
	defer diskCacheFetcher.Clean()
	if root != nil {
		var err error
		camfs, err = fs.NewRootedCamliFileSystem(diskCacheFetcher, root)
		if err != nil {
			log.Fatalf("Error creating root with %v: %v", root, err)
		}
	} else {
		camfs = fs.NewCamliFileSystem(cl, diskCacheFetcher)
		log.Printf("starting with fs %#v", camfs)
	}

	if *debug {
		// TODO: set fs's logger
	}

	// This doesn't appear to work on OS X:
	sigc := make(chan os.Signal, 1)
	go func() {
		log.Fatalf("Signal %s received, shutting down.", <-sigc)
	}()
	signal.Notify(sigc, syscall.SIGQUIT, syscall.SIGTERM)

	conn, err := fuse.Mount(mountPoint)
	if err != nil {
		log.Fatalf("Mount: %v", err)
	}
	err = conn.Serve(camfs)
	if err != nil {
		log.Fatalf("Serve: %v", err)
	}
	log.Printf("fuse process ending.")
}
<commit_msg>cammount: rm TODO<commit_after>/*
Copyright 2011 Google Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package main

import (
	"flag"
	"fmt"
	"log"
	"net/http"
	"os"
	"os/signal"
	"strings"
	"syscall"

	"camlistore.org/pkg/blobref"
	"camlistore.org/pkg/cacher"
	"camlistore.org/pkg/client"
	"camlistore.org/pkg/fs"

	"camlistore.org/third_party/code.google.com/p/rsc/fuse"
)

func usage() {
	fmt.Fprint(os.Stderr, "usage: cammount [opts] <mountpoint> [<root-blobref>|<share URL>]\n")
	flag.PrintDefaults()
	os.Exit(2)
}

func main() {
	// Scans the arg list and sets up flags
	debug := flag.Bool("debug", false, "print debugging messages.")
	client.AddFlags()
	flag.Parse()

	errorf := func(msg string, args ...interface{}) {
		fmt.Fprintf(os.Stderr, msg, args...)
		fmt.Fprint(os.Stderr, "\n")
		usage()
	}

	nargs := flag.NArg()
	if nargs < 1 || nargs > 2 {
		usage()
	}

	mountPoint := flag.Arg(0)
	var (
		cl    *client.Client
		root  *blobref.BlobRef // nil if only one arg
		camfs *fs.CamliFileSystem
	)
	if nargs == 2 {
		rootArg := flag.Arg(1)
		// not trying very hard since NewFromShareRoot will do it better with a regex
		if strings.HasPrefix(rootArg, "http://") ||
			strings.HasPrefix(rootArg, "https://") {
			if client.ExplicitServer() != "" {
				errorf("Can't use an explicit blobserver with a share URL; the blobserver is implicit from the share URL.")
			}
			var err error
			cl, root, err = client.NewFromShareRoot(rootArg)
			if err != nil {
				log.Fatal(err)
			}
		} else {
			cl = client.NewOrFail() // automatic from flags
			root = blobref.Parse(rootArg)
			if root == nil {
				log.Fatalf("Error parsing root blobref: %q\n", rootArg)
			}
			cl.SetHTTPClient(&http.Client{Transport: cl.TransportForConfig(nil)})
		}
	} else {
		cl = client.NewOrFail() // automatic from flags
		cl.SetHTTPClient(&http.Client{Transport: cl.TransportForConfig(nil)})
	}

	diskCacheFetcher, err := cacher.NewDiskCache(cl)
	if err != nil {
		log.Fatalf("Error setting up local disk cache: %v", err)
	}
	defer diskCacheFetcher.Clean()
	if root != nil {
		var err error
		camfs, err = fs.NewRootedCamliFileSystem(diskCacheFetcher, root)
		if err != nil {
			log.Fatalf("Error creating root with %v: %v", root, err)
		}
	} else {
		camfs = fs.NewCamliFileSystem(cl, diskCacheFetcher)
		log.Printf("starting with fs %#v", camfs)
	}

	if *debug {
		// TODO: set fs's logger
	}

	// This doesn't appear to work on OS X:
	sigc := make(chan os.Signal, 1)
	go func() {
		log.Fatalf("Signal %s received, shutting down.", <-sigc)
	}()
	signal.Notify(sigc, syscall.SIGQUIT, syscall.SIGTERM)

	conn, err := fuse.Mount(mountPoint)
	if err != nil {
		log.Fatalf("Mount: %v", err)
	}
	err = conn.Serve(camfs)
	if err != nil {
		log.Fatalf("Serve: %v", err)
	}
	log.Printf("fuse process ending.")
}
<|endoftext|>


<commit_before>// Copyright 2015 ISRG.  All rights reserved
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.

package main

import (
	"crypto/x509"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"os"
	"path"
	"runtime"
	"sync"
	"sync/atomic"
	"time"

	"github.com/letsencrypt/boulder/Godeps/_workspace/src/github.com/cactus/go-statsd-client/statsd"
	"github.com/letsencrypt/boulder/Godeps/_workspace/src/github.com/codegangsta/cli"
	"github.com/letsencrypt/boulder/Godeps/_workspace/src/github.com/jmhodges/clock"
	gorp "github.com/letsencrypt/boulder/Godeps/_workspace/src/gopkg.in/gorp.v1"

	"github.com/letsencrypt/boulder/cmd"
	"github.com/letsencrypt/boulder/core"
	blog "github.com/letsencrypt/boulder/log"
	"github.com/letsencrypt/boulder/policy"
	"github.com/letsencrypt/boulder/sa"
)

const (
	good = "valid"
	bad  = "invalid"

	filenameLayout = "20060102"

	checkPeriod = time.Hour * 24 * 90

	batchSize = 1000
)

type report struct {
	begin     time.Time
	end       time.Time
	GoodCerts int64
	BadCerts  int64
	Entries   map[string]reportEntry
}

func (r *report) save(directory string) error {
	filename := path.Join(directory, fmt.Sprintf(
		"%s-%s-report.json",
		r.begin.Format(filenameLayout),
		r.end.Format(filenameLayout),
	))
	content, err := json.Marshal(r)
	if err != nil {
		return err
	}
	return ioutil.WriteFile(filename, content, os.ModePerm)
}

type reportEntry struct {
	Valid    bool     `json:"validity"`
	Problems []string `json:"problem,omitempty"`
}

type certChecker struct {
	pa           core.PolicyAuthority
	dbMap        *gorp.DbMap
	certs        chan core.Certificate
	issuedReport report
	clock        clock.Clock
}

func newChecker(saDbMap *gorp.DbMap, paDbMap *gorp.DbMap, enforceWhitelist bool) certChecker {
	pa, err := policy.NewPolicyAuthorityImpl(paDbMap, enforceWhitelist)
	cmd.FailOnError(err, "Failed to create PA")
	c := certChecker{
		pa:    pa,
		dbMap: saDbMap,
		certs: make(chan core.Certificate, batchSize),
		clock: clock.Default(),
	}
	c.issuedReport.Entries = make(map[string]reportEntry)
	return c
}

func (c *certChecker) getCerts() error {
	c.issuedReport.begin = c.clock.Now()
	c.issuedReport.end = c.issuedReport.begin.Add(-checkPeriod)

	var count int
	err := c.dbMap.SelectOne(
		&count,
		"SELECT count(*) FROM certificates WHERE issued >= :issued",
		map[string]interface{}{"issued": c.issuedReport.end},
	)
	if err != nil {
		return err
	}

	// Retrieve certs in batches of 1000 (the size of the certificate channel)
	// so that we don't eat unnecessary amounts of memory and avoid the 16MB MySQL
	// packet limit.
	// TODO(#701): This query needs to make better use of indexes
	for offset := 0; offset < count; {
		var certs []core.Certificate
		_, err = c.dbMap.Select(
			&certs,
			"SELECT * FROM certificates WHERE issued >= :issued ORDER BY issued ASC LIMIT :limit OFFSET :offset",
			map[string]interface{}{"issued": c.issuedReport.end, "limit": batchSize, "offset": offset},
		)
		if err != nil {
			return err
		}
		for _, cert := range certs {
			c.certs <- cert
		}
		offset += len(certs)
	}

	// Close channel so range operations won't block once the channel empties out
	close(c.certs)
	return nil
}

func (c *certChecker) processCerts(wg *sync.WaitGroup) {
	for cert := range c.certs {
		problems := c.checkCert(cert)
		valid := len(problems) == 0
		c.issuedReport.Entries[cert.Serial] = reportEntry{
			Valid:    valid,
			Problems: problems,
		}
		if !valid {
			atomic.AddInt64(&c.issuedReport.BadCerts, 1)
		} else {
			atomic.AddInt64(&c.issuedReport.GoodCerts, 1)
		}
	}
	wg.Done()
}

func (c *certChecker) checkCert(cert core.Certificate) (problems []string) {
	// Check digests match
	if cert.Digest != core.Fingerprint256(cert.DER) {
		problems = append(problems, "Stored digest doesn't match certificate digest")
	}

	// Parse certificate
	parsedCert, err := x509.ParseCertificate(cert.DER)
	if err != nil {
		problems = append(problems, fmt.Sprintf("Couldn't parse stored certificate: %s", err))
	} else {
		// Check stored serial is correct
		if core.SerialToString(parsedCert.SerialNumber) != cert.Serial {
			problems = append(problems, "Stored serial doesn't match certificate serial")
		}
		// Check we have the right expiration time
		if !parsedCert.NotAfter.Equal(cert.Expires) {
			problems = append(problems, "Stored expiration doesn't match certificate NotAfter")
		}
		// Check basic constraints are set
		if !parsedCert.BasicConstraintsValid {
			problems = append(problems, "Certificate doesn't have basic constraints set")
		}
		// Check the cert isn't able to sign other certificates
		if parsedCert.IsCA {
			problems = append(problems, "Certificate can sign other certificates")
		}
		// Check the cert has the correct validity period +/- an hour
		expiryPeriod := parsedCert.NotAfter.Sub(parsedCert.NotBefore)
		if expiryPeriod > checkPeriod {
			problems = append(problems, "Certificate has a validity period longer than 90 days")
		} else if expiryPeriod < checkPeriod {
			problems = append(problems, "Certificate has a validity period shorter than 90 days")
		}
		// Check that the PA is still willing to issue for each name in DNSNames + CommonName
		for _, name := range append(parsedCert.DNSNames, parsedCert.Subject.CommonName) {
			if err = c.pa.WillingToIssue(core.AcmeIdentifier{Type: core.IdentifierDNS, Value: name}); err != nil {
				problems = append(problems, fmt.Sprintf("Policy Authority isn't willing to issue for %s: %s", name, err))
			}
		}
		// Check the cert has the correct key usage extensions
		if !core.CmpExtKeyUsageSlice(parsedCert.ExtKeyUsage, []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth}) {
			problems = append(problems, "Certificate has incorrect key usage extensions")
		}
	}
	return problems
}

func main() {
	app := cmd.NewAppShell("cert-checker", "Checks validity of certificates issued in the last 90 days")
	app.App.Flags = append(app.App.Flags, cli.IntFlag{
		Name:  "workers",
		Value: runtime.NumCPU(),
		Usage: "The number of concurrent workers used to process certificates",
	}, cli.StringFlag{
		Name:  "report-dir-path",
		Usage: "The path to write a JSON report on the certificates checks to (if no path is provided the report will not be written out)",
	}, cli.StringFlag{
		Name:  "db-connect",
		Usage: "SQL URI if not provided in the configuration file",
	})

	app.Config = func(c *cli.Context, config cmd.Config) cmd.Config {
		config.CertChecker.ReportDirectoryPath = c.GlobalString("report-dir-path")

		if connect := c.GlobalString("db-connect"); connect != "" {
			config.CertChecker.DBConnect = connect
		}

		if workers := c.GlobalInt("workers"); workers != 0 {
			config.CertChecker.Workers = workers
		}

		return config
	}

	app.Action = func(c cmd.Config) {
		stats, err := statsd.NewClient(c.Statsd.Server, c.Statsd.Prefix)
		cmd.FailOnError(err, "Couldn't connect to statsd")

		auditlogger, err := blog.Dial(c.Syslog.Network, c.Syslog.Server, c.Syslog.Tag, stats)
		cmd.FailOnError(err, "Could not connect to Syslog")

		blog.SetAuditLogger(auditlogger)
		auditlogger.Info(app.VersionString())

		saDbMap, err := sa.NewDbMap(c.CertChecker.DBConnect)
		cmd.FailOnError(err, "Could not connect to database")

		paDbMap, err := sa.NewDbMap(c.PA.DBConnect)
		cmd.FailOnError(err, "Could not connect to policy database")

		checker := newChecker(saDbMap, paDbMap, c.PA.EnforcePolicyWhitelist)
		auditlogger.Info("# Getting certificates issued in the last 90 days")

		// Since we grab certificates in batches we don't want this to block, when it
		// is finished it will close the certificate channel which allows the range
		// loops in checker.processCerts to break
		go func() {
			err = checker.getCerts()
			cmd.FailOnError(err, "Batch retrieval of certificates failed")
		}()

		auditlogger.Info(fmt.Sprintf("# Processing certificates using %d workers", c.CertChecker.Workers))
		wg := new(sync.WaitGroup)
		for i := 0; i < c.CertChecker.Workers; i++ {
			wg.Add(1)
			go checker.processCerts(wg)
		}
		wg.Wait()
		auditlogger.Info(fmt.Sprintf(
			"# Finished processing certificates, sample: %d, good: %d, bad: %d",
			len(checker.issuedReport.Entries),
			checker.issuedReport.GoodCerts,
			checker.issuedReport.BadCerts,
		))
		err = checker.issuedReport.save(c.CertChecker.ReportDirectoryPath)
		cmd.FailOnError(err, "Couldn't save issued certificate report")
	}

	app.Run()
}
<commit_msg>Protect report entries map with mutex to prevent concurrent writes causing a bad map state<commit_after>// Copyright 2015 ISRG.  All rights reserved
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.

package main

import (
	"crypto/x509"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"os"
	"path"
	"runtime"
	"sync"
	"sync/atomic"
	"time"

	"github.com/letsencrypt/boulder/Godeps/_workspace/src/github.com/cactus/go-statsd-client/statsd"
	"github.com/letsencrypt/boulder/Godeps/_workspace/src/github.com/codegangsta/cli"
	"github.com/letsencrypt/boulder/Godeps/_workspace/src/github.com/jmhodges/clock"
	gorp "github.com/letsencrypt/boulder/Godeps/_workspace/src/gopkg.in/gorp.v1"

	"github.com/letsencrypt/boulder/cmd"
	"github.com/letsencrypt/boulder/core"
	blog "github.com/letsencrypt/boulder/log"
	"github.com/letsencrypt/boulder/policy"
	"github.com/letsencrypt/boulder/sa"
)

const (
	good = "valid"
	bad  = "invalid"

	filenameLayout = "20060102"

	checkPeriod = time.Hour * 24 * 90

	batchSize = 1000
)

type report struct {
	begin     time.Time
	end       time.Time
	GoodCerts int64
	BadCerts  int64
	Entries   map[string]reportEntry
}

func (r *report) save(directory string) error {
	filename := path.Join(directory, fmt.Sprintf(
		"%s-%s-report.json",
		r.begin.Format(filenameLayout),
		r.end.Format(filenameLayout),
	))
	content, err := json.Marshal(r)
	if err != nil {
		return err
	}
	return ioutil.WriteFile(filename, content, os.ModePerm)
}

type reportEntry struct {
	Valid    bool     `json:"validity"`
	Problems []string `json:"problem,omitempty"`
}

type certChecker struct {
	pa           core.PolicyAuthority
	dbMap        *gorp.DbMap
	certs        chan core.Certificate
	clock        clock.Clock
	rMu          *sync.Mutex
	issuedReport report
}

func newChecker(saDbMap *gorp.DbMap, paDbMap *gorp.DbMap, enforceWhitelist bool) certChecker {
	pa, err := policy.NewPolicyAuthorityImpl(paDbMap, enforceWhitelist)
	cmd.FailOnError(err, "Failed to create PA")
	c := certChecker{
		pa:    pa,
		dbMap: saDbMap,
		certs: make(chan core.Certificate, batchSize),
		clock: clock.Default(),
		rMu:   new(sync.Mutex),
	}
	c.issuedReport.Entries = make(map[string]reportEntry)

	return c
}

func (c *certChecker) getCerts() error {
	c.issuedReport.begin = c.clock.Now()
	c.issuedReport.end = c.issuedReport.begin.Add(-checkPeriod)

	var count int
	err := c.dbMap.SelectOne(
		&count,
		"SELECT count(*) FROM certificates WHERE issued >= :issued",
		map[string]interface{}{"issued": c.issuedReport.end},
	)
	if err != nil {
		return err
	}

	// Retrieve certs in batches of 1000 (the size of the certificate channel)
	// so that we don't eat unnecessary amounts of memory and avoid the 16MB MySQL
	// packet limit.
	// TODO(#701): This query needs to make better use of indexes
	for offset := 0; offset < count; {
		var certs []core.Certificate
		_, err = c.dbMap.Select(
			&certs,
			"SELECT * FROM certificates WHERE issued >= :issued ORDER BY issued ASC LIMIT :limit OFFSET :offset",
			map[string]interface{}{"issued": c.issuedReport.end, "limit": batchSize, "offset": offset},
		)
		if err != nil {
			return err
		}
		for _, cert := range certs {
			c.certs <- cert
		}
		offset += len(certs)
	}

	// Close channel so range operations won't block once the channel empties out
	close(c.certs)
	return nil
}

func (c *certChecker) processCerts(wg *sync.WaitGroup) {
	for cert := range c.certs {
		problems := c.checkCert(cert)
		valid := len(problems) == 0
		c.rMu.Lock()
		c.issuedReport.Entries[cert.Serial] = reportEntry{
			Valid:    valid,
			Problems: problems,
		}
		c.rMu.Unlock()
		if !valid {
			atomic.AddInt64(&c.issuedReport.BadCerts, 1)
		} else {
			atomic.AddInt64(&c.issuedReport.GoodCerts, 1)
		}
	}
	wg.Done()
}

func (c *certChecker) checkCert(cert core.Certificate) (problems []string) {
	// Check digests match
	if cert.Digest != core.Fingerprint256(cert.DER) {
		problems = append(problems, "Stored digest doesn't match certificate digest")
	}

	// Parse certificate
	parsedCert, err := x509.ParseCertificate(cert.DER)
	if err != nil {
		problems = append(problems, fmt.Sprintf("Couldn't parse stored certificate: %s", err))
	} else {
		// Check stored serial is correct
		if core.SerialToString(parsedCert.SerialNumber) != cert.Serial {
			problems = append(problems, "Stored serial doesn't match certificate serial")
		}
		// Check we have the right expiration time
		if !parsedCert.NotAfter.Equal(cert.Expires) {
			problems = append(problems, "Stored expiration doesn't match certificate NotAfter")
		}
		// Check basic constraints are set
		if !parsedCert.BasicConstraintsValid {
			problems = append(problems, "Certificate doesn't have basic constraints set")
		}
		// Check the cert isn't able to sign other certificates
		if parsedCert.IsCA {
			problems = append(problems, "Certificate can sign other certificates")
		}
		// Check the cert has the correct validity period +/- an hour
		expiryPeriod := parsedCert.NotAfter.Sub(parsedCert.NotBefore)
		if expiryPeriod > checkPeriod {
			problems = append(problems, "Certificate has a validity period longer than 90 days")
		} else if expiryPeriod < checkPeriod {
			problems = append(problems, "Certificate has a validity period shorter than 90 days")
		}
		// Check that the PA is still willing to issue for each name in DNSNames + CommonName
		for _, name := range append(parsedCert.DNSNames, parsedCert.Subject.CommonName) {
			if err = c.pa.WillingToIssue(core.AcmeIdentifier{Type: core.IdentifierDNS, Value: name}); err != nil {
				problems = append(problems, fmt.Sprintf("Policy Authority isn't willing to issue for %s: %s", name, err))
			}
		}
		// Check the cert has the correct key usage extensions
		if !core.CmpExtKeyUsageSlice(parsedCert.ExtKeyUsage, []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth, x509.ExtKeyUsageClientAuth}) {
			problems = append(problems, "Certificate has incorrect key usage extensions")
		}
	}
	return problems
}

func main() {
	app := cmd.NewAppShell("cert-checker", "Checks validity of certificates issued in the last 90 days")
	app.App.Flags = append(app.App.Flags, cli.IntFlag{
		Name:  "workers",
		Value: runtime.NumCPU(),
		Usage: "The number of concurrent workers used to process certificates",
	}, cli.StringFlag{
		Name:  "report-dir-path",
		Usage: "The path to write a JSON report on the certificates checks to (if no path is provided the report will not be written out)",
	}, cli.StringFlag{
		Name:  "db-connect",
		Usage: "SQL URI if not provided in the configuration file",
	})

	app.Config = func(c *cli.Context, config cmd.Config) cmd.Config {
		config.CertChecker.ReportDirectoryPath = c.GlobalString("report-dir-path")

		if connect := c.GlobalString("db-connect"); connect != "" {
			config.CertChecker.DBConnect = connect
		}

		if workers := c.GlobalInt("workers"); workers != 0 {
			config.CertChecker.Workers = workers
		}

		return config
	}

	app.Action = func(c cmd.Config) {
		stats, err := statsd.NewClient(c.Statsd.Server, c.Statsd.Prefix)
		cmd.FailOnError(err, "Couldn't connect to statsd")

		auditlogger, err := blog.Dial(c.Syslog.Network, c.Syslog.Server, c.Syslog.Tag, stats)
		cmd.FailOnError(err, "Could not connect to Syslog")

		blog.SetAuditLogger(auditlogger)
		auditlogger.Info(app.VersionString())

		saDbMap, err := sa.NewDbMap(c.CertChecker.DBConnect)
		cmd.FailOnError(err, "Could not connect to database")

		paDbMap, err := sa.NewDbMap(c.PA.DBConnect)
		cmd.FailOnError(err, "Could not connect to policy database")

		checker := newChecker(saDbMap, paDbMap, c.PA.EnforcePolicyWhitelist)
		auditlogger.Info("# Getting certificates issued in the last 90 days")

		// Since we grab certificates in batches we don't want this to block, when it
		// is finished it will close the certificate channel which allows the range
		// loops in checker.processCerts to break
		go func() {
			err = checker.getCerts()
			cmd.FailOnError(err, "Batch retrieval of certificates failed")
		}()

		auditlogger.Info(fmt.Sprintf("# Processing certificates using %d workers", c.CertChecker.Workers))
		wg := new(sync.WaitGroup)
		for i := 0; i < c.CertChecker.Workers; i++ {
			wg.Add(1)
			go checker.processCerts(wg)
		}
		wg.Wait()
		auditlogger.Info(fmt.Sprintf(
			"# Finished processing certificates, sample: %d, good: %d, bad: %d",
			len(checker.issuedReport.Entries),
			checker.issuedReport.GoodCerts,
			checker.issuedReport.BadCerts,
		))
		err = checker.issuedReport.save(c.CertChecker.ReportDirectoryPath)
		cmd.FailOnError(err, "Couldn't save issued certificate report")
	}

	app.Run()
}
<|endoftext|>


<commit_before>// Copyright 2015 The Go Authors.  All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

// Compilebench benchmarks the speed of the Go compiler.
//
// Usage:
//
//	compilebench [options]
//
// It times the compilation of various packages and prints results in
// the format used by package testing (and expected by golang.org/x/perf/cmd/benchstat).
//
// The options are:
//
//	-alloc
//		Report allocations.
//
//	-compile exe
//		Use exe as the path to the cmd/compile binary.
//
//	-compileflags 'list'
//		Pass the space-separated list of flags to the compilation.
//
//	-count n
//		Run each benchmark n times (default 1).
//
//	-cpuprofile file
//		Write a CPU profile of the compiler to file.
//
//	-memprofile file
//		Write a memory profile of the compiler to file.
//
//	-memprofilerate rate
//		Set runtime.MemProfileRate during compilation.
//
//	-obj
//		Report object file statistics.
//
//  -pkg
//		Benchmark compiling a single package.
//
//	-run regexp
//		Only run benchmarks with names matching regexp.
//
// Although -cpuprofile and -memprofile are intended to write a
// combined profile for all the executed benchmarks to file,
// today they write only the profile for the last benchmark executed.
//
// The default memory profiling rate is one profile sample per 512 kB
// allocated (see ``go doc runtime.MemProfileRate'').
// Lowering the rate (for example, -memprofilerate 64000) produces
// a more fine-grained and therefore accurate profile, but it also incurs
// execution cost. For benchmark comparisons, never use timings
// obtained with a low -memprofilerate option.
//
// Example
//
// Assuming the base version of the compiler has been saved with
// ``toolstash save,'' this sequence compares the old and new compiler:
//
//	compilebench -count 10 -compile $(toolstash -n compile) >old.txt
//	compilebench -count 10 >new.txt
//	benchstat old.txt new.txt
//
package main

import (
	"bytes"
	"flag"
	"fmt"
	"go/build"
	"io/ioutil"
	"log"
	"os"
	"os/exec"
	"path/filepath"
	"regexp"
	"runtime"
	"strconv"
	"strings"
	"time"
)

var (
	goroot   = runtime.GOROOT()
	compiler string
	runRE    *regexp.Regexp
	is6g     bool
)

var (
	flagAlloc          = flag.Bool("alloc", false, "report allocations")
	flagObj            = flag.Bool("obj", false, "report object file stats")
	flagCompiler       = flag.String("compile", "", "use `exe` as the cmd/compile binary")
	flagCompilerFlags  = flag.String("compileflags", "", "additional `flags` to pass to compile")
	flagRun            = flag.String("run", "", "run benchmarks matching `regexp`")
	flagCount          = flag.Int("count", 1, "run benchmarks `n` times")
	flagCpuprofile     = flag.String("cpuprofile", "", "write CPU profile to `file`")
	flagMemprofile     = flag.String("memprofile", "", "write memory profile to `file`")
	flagMemprofilerate = flag.Int64("memprofilerate", -1, "set memory profile `rate`")
	flagPackage        = flag.String("pkg", "", "if set, benchmark the package at path `pkg`")
	flagShort          = flag.Bool("short", false, "skip long-running benchmarks")
)

var tests = []struct {
	name string
	dir  string
	long bool
}{
	{"BenchmarkTemplate", "html/template", false},
	{"BenchmarkUnicode", "unicode", false},
	{"BenchmarkGoTypes", "go/types", false},
	{"BenchmarkCompiler", "cmd/compile/internal/gc", false},
	{"BenchmarkSSA", "cmd/compile/internal/ssa", false},
	{"BenchmarkFlate", "compress/flate", false},
	{"BenchmarkGoParser", "go/parser", false},
	{"BenchmarkReflect", "reflect", false},
	{"BenchmarkTar", "archive/tar", false},
	{"BenchmarkXML", "encoding/xml", false},
	{"BenchmarkStdCmd", "", true},
	{"BenchmarkHelloSize", "", false},
	{"BenchmarkCmdGoSize", "", true},
}

func usage() {
	fmt.Fprintf(os.Stderr, "usage: compilebench [options]\n")
	fmt.Fprintf(os.Stderr, "options:\n")
	flag.PrintDefaults()
	os.Exit(2)
}

func main() {
	log.SetFlags(0)
	log.SetPrefix("compilebench: ")
	flag.Usage = usage
	flag.Parse()
	if flag.NArg() != 0 {
		usage()
	}

	compiler = *flagCompiler
	if compiler == "" {
		out, err := exec.Command("go", "tool", "-n", "compile").CombinedOutput()
		if err != nil {
			out, err = exec.Command("go", "tool", "-n", "6g").CombinedOutput()
			is6g = true
			if err != nil {
				out, err = exec.Command("go", "tool", "-n", "compile").CombinedOutput()
				log.Fatalf("go tool -n compiler: %v\n%s", err, out)
			}
		}
		compiler = strings.TrimSpace(string(out))
	}

	if *flagRun != "" {
		r, err := regexp.Compile(*flagRun)
		if err != nil {
			log.Fatalf("invalid -run argument: %v", err)
		}
		runRE = r
	}

	for i := 0; i < *flagCount; i++ {
		if *flagPackage != "" {
			runBuild("BenchmarkPkg", *flagPackage, i)
			continue
		}
		for _, tt := range tests {
			if tt.long && *flagShort {
				continue
			}
			if runRE == nil || runRE.MatchString(tt.name) {
				runBuild(tt.name, tt.dir, i)
			}
		}
	}
}

func runCmd(name string, cmd *exec.Cmd) {
	start := time.Now()
	out, err := cmd.CombinedOutput()
	if err != nil {
		log.Printf("%v: %v\n%s", name, err, out)
		return
	}
	fmt.Printf("%s 1 %d ns/op\n", name, time.Since(start).Nanoseconds())
}

func runStdCmd() {
	args := []string{"build", "-a"}
	if *flagCompilerFlags != "" {
		args = append(args, "-gcflags", *flagCompilerFlags)
	}
	args = append(args, "std", "cmd")
	cmd := exec.Command("go", args...)
	cmd.Dir = filepath.Join(runtime.GOROOT(), "src")
	runCmd("BenchmarkStdCmd", cmd)
}

// path is either a path to a file ("$GOROOT/test/helloworld.go") or a package path ("cmd/go").
func runSize(name, path string) {
	cmd := exec.Command("go", "build", "-o", "_compilebenchout_", path)
	cmd.Stdout = os.Stderr
	cmd.Stderr = os.Stderr
	if err := cmd.Run(); err != nil {
		log.Print(err)
		return
	}
	defer os.Remove("_compilebenchout_")
	info, err := os.Stat("_compilebenchout_")
	if err != nil {
		log.Print(err)
		return
	}
	out, err := exec.Command("size", "_compilebenchout_").CombinedOutput()
	if err != nil {
		log.Printf("size: %v\n%s", err, out)
		return
	}
	lines := strings.Split(string(out), "\n")
	if len(lines) < 2 {
		log.Printf("not enough output from size: %s", out)
		return
	}
	f := strings.Fields(lines[1])
	if strings.HasPrefix(lines[0], "__TEXT") && len(f) >= 2 { // OS X
		fmt.Printf("%s 1 %s text-bytes %s data-bytes %v exe-bytes\n", name, f[0], f[1], info.Size())
	} else if strings.Contains(lines[0], "bss") && len(f) >= 3 {
		fmt.Printf("%s 1 %s text-bytes %s data-bytes %s bss-bytes %v exe-bytes\n", name, f[0], f[1], f[2], info.Size())
	}
}

func runBuild(name, dir string, count int) {
	switch name {
	case "BenchmarkStdCmd":
		runStdCmd()
		return
	case "BenchmarkCmdGoSize":
		runSize("BenchmarkCmdGoSize", "cmd/go")
		return
	case "BenchmarkHelloSize":
		runSize("BenchmarkHelloSize", filepath.Join(runtime.GOROOT(), "test/helloworld.go"))
		return
	}

	pkg, err := build.Import(dir, ".", 0)
	if err != nil {
		log.Print(err)
		return
	}
	args := []string{"-o", "_compilebench_.o"}
	if is6g {
		*flagMemprofilerate = -1
		*flagAlloc = false
		*flagCpuprofile = ""
		*flagMemprofile = ""
	}
	if *flagMemprofilerate >= 0 {
		args = append(args, "-memprofilerate", fmt.Sprint(*flagMemprofilerate))
	}
	args = append(args, strings.Fields(*flagCompilerFlags)...)
	if *flagAlloc || *flagCpuprofile != "" || *flagMemprofile != "" {
		if *flagAlloc || *flagMemprofile != "" {
			args = append(args, "-memprofile", "_compilebench_.memprof")
		}
		if *flagCpuprofile != "" {
			args = append(args, "-cpuprofile", "_compilebench_.cpuprof")
		}
	}
	args = append(args, pkg.GoFiles...)
	cmd := exec.Command(compiler, args...)
	cmd.Dir = pkg.Dir
	cmd.Stdout = os.Stderr
	cmd.Stderr = os.Stderr
	start := time.Now()
	err = cmd.Run()
	if err != nil {
		log.Printf("%v: %v", name, err)
		return
	}
	end := time.Now()

	var allocs, allocbytes int64
	if *flagAlloc || *flagMemprofile != "" {
		out, err := ioutil.ReadFile(pkg.Dir + "/_compilebench_.memprof")
		if err != nil {
			log.Print("cannot find memory profile after compilation")
		}
		for _, line := range strings.Split(string(out), "\n") {
			f := strings.Fields(line)
			if len(f) < 4 || f[0] != "#" || f[2] != "=" {
				continue
			}
			val, err := strconv.ParseInt(f[3], 0, 64)
			if err != nil {
				continue
			}
			switch f[1] {
			case "TotalAlloc":
				allocbytes = val
			case "Mallocs":
				allocs = val
			}
		}

		if *flagMemprofile != "" {
			if err := ioutil.WriteFile(*flagMemprofile, out, 0666); err != nil {
				log.Print(err)
			}
		}
		os.Remove(pkg.Dir + "/_compilebench_.memprof")
	}

	if *flagCpuprofile != "" {
		out, err := ioutil.ReadFile(pkg.Dir + "/_compilebench_.cpuprof")
		if err != nil {
			log.Print(err)
		}
		outpath := *flagCpuprofile
		if *flagCount != 1 {
			outpath = fmt.Sprintf("%s_%d", outpath, count)
		}
		if err := ioutil.WriteFile(outpath, out, 0666); err != nil {
			log.Print(err)
		}
		os.Remove(pkg.Dir + "/_compilebench_.cpuprof")
	}

	wallns := end.Sub(start).Nanoseconds()
	userns := cmd.ProcessState.UserTime().Nanoseconds()

	fmt.Printf("%s 1 %d ns/op %d user-ns/op", name, wallns, userns)
	if *flagAlloc {
		fmt.Printf(" %d B/op %d allocs/op", allocbytes, allocs)
	}

	opath := pkg.Dir + "/_compilebench_.o"
	if *flagObj {
		// TODO(josharian): object files are big; just read enough to find what we seek.
		data, err := ioutil.ReadFile(opath)
		if err != nil {
			log.Print(err)
		}
		// Find start of export data.
		i := bytes.Index(data, []byte("\n$$B\n")) + len("\n$$B\n")
		// Count bytes to end of export data.
		nexport := bytes.Index(data[i:], []byte("\n$$\n"))
		fmt.Printf(" %d object-bytes %d export-bytes", len(data), nexport)
	}
	fmt.Println()

	os.Remove(opath)
}
<commit_msg>cmd/compilebench: use go command in goroot, not environment<commit_after>// Copyright 2015 The Go Authors.  All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

// Compilebench benchmarks the speed of the Go compiler.
//
// Usage:
//
//	compilebench [options]
//
// It times the compilation of various packages and prints results in
// the format used by package testing (and expected by golang.org/x/perf/cmd/benchstat).
//
// The options are:
//
//	-alloc
//		Report allocations.
//
//	-compile exe
//		Use exe as the path to the cmd/compile binary.
//
//	-compileflags 'list'
//		Pass the space-separated list of flags to the compilation.
//
//	-count n
//		Run each benchmark n times (default 1).
//
//	-cpuprofile file
//		Write a CPU profile of the compiler to file.
//
//	-memprofile file
//		Write a memory profile of the compiler to file.
//
//	-memprofilerate rate
//		Set runtime.MemProfileRate during compilation.
//
//	-obj
//		Report object file statistics.
//
//  -pkg
//		Benchmark compiling a single package.
//
//	-run regexp
//		Only run benchmarks with names matching regexp.
//
// Although -cpuprofile and -memprofile are intended to write a
// combined profile for all the executed benchmarks to file,
// today they write only the profile for the last benchmark executed.
//
// The default memory profiling rate is one profile sample per 512 kB
// allocated (see ``go doc runtime.MemProfileRate'').
// Lowering the rate (for example, -memprofilerate 64000) produces
// a more fine-grained and therefore accurate profile, but it also incurs
// execution cost. For benchmark comparisons, never use timings
// obtained with a low -memprofilerate option.
//
// Example
//
// Assuming the base version of the compiler has been saved with
// ``toolstash save,'' this sequence compares the old and new compiler:
//
//	compilebench -count 10 -compile $(toolstash -n compile) >old.txt
//	compilebench -count 10 >new.txt
//	benchstat old.txt new.txt
//
package main

import (
	"bytes"
	"flag"
	"fmt"
	"go/build"
	"io/ioutil"
	"log"
	"os"
	"os/exec"
	"path/filepath"
	"regexp"
	"strconv"
	"strings"
	"time"
)

var (
	goroot   string
	compiler string
	runRE    *regexp.Regexp
	is6g     bool
)

var (
	flagGoCmd          = flag.String("go", "go", "path to \"go\" command")
	flagAlloc          = flag.Bool("alloc", false, "report allocations")
	flagObj            = flag.Bool("obj", false, "report object file stats")
	flagCompiler       = flag.String("compile", "", "use `exe` as the cmd/compile binary")
	flagCompilerFlags  = flag.String("compileflags", "", "additional `flags` to pass to compile")
	flagRun            = flag.String("run", "", "run benchmarks matching `regexp`")
	flagCount          = flag.Int("count", 1, "run benchmarks `n` times")
	flagCpuprofile     = flag.String("cpuprofile", "", "write CPU profile to `file`")
	flagMemprofile     = flag.String("memprofile", "", "write memory profile to `file`")
	flagMemprofilerate = flag.Int64("memprofilerate", -1, "set memory profile `rate`")
	flagPackage        = flag.String("pkg", "", "if set, benchmark the package at path `pkg`")
	flagShort          = flag.Bool("short", false, "skip long-running benchmarks")
)

var tests = []struct {
	name string
	dir  string
	long bool
}{
	{"BenchmarkTemplate", "html/template", false},
	{"BenchmarkUnicode", "unicode", false},
	{"BenchmarkGoTypes", "go/types", false},
	{"BenchmarkCompiler", "cmd/compile/internal/gc", false},
	{"BenchmarkSSA", "cmd/compile/internal/ssa", false},
	{"BenchmarkFlate", "compress/flate", false},
	{"BenchmarkGoParser", "go/parser", false},
	{"BenchmarkReflect", "reflect", false},
	{"BenchmarkTar", "archive/tar", false},
	{"BenchmarkXML", "encoding/xml", false},
	{"BenchmarkStdCmd", "", true},
	{"BenchmarkHelloSize", "", false},
	{"BenchmarkCmdGoSize", "", true},
}

func usage() {
	fmt.Fprintf(os.Stderr, "usage: compilebench [options]\n")
	fmt.Fprintf(os.Stderr, "options:\n")
	flag.PrintDefaults()
	os.Exit(2)
}

func main() {
	log.SetFlags(0)
	log.SetPrefix("compilebench: ")
	flag.Usage = usage
	flag.Parse()
	if flag.NArg() != 0 {
		usage()
	}

	s, err := exec.Command(*flagGoCmd, "env", "GOROOT").CombinedOutput()
	if err != nil {
		log.Fatalf("%s env GOROOT: %v", *flagGoCmd, err)
	}
	goroot = strings.TrimSpace(string(s))

	compiler = *flagCompiler
	if compiler == "" {
		out, err := exec.Command(*flagGoCmd, "tool", "-n", "compile").CombinedOutput()
		if err != nil {
			out, err = exec.Command(*flagGoCmd, "tool", "-n", "6g").CombinedOutput()
			is6g = true
			if err != nil {
				out, err = exec.Command(*flagGoCmd, "tool", "-n", "compile").CombinedOutput()
				log.Fatalf("go tool -n compiler: %v\n%s", err, out)
			}
		}
		compiler = strings.TrimSpace(string(out))
	}

	if *flagRun != "" {
		r, err := regexp.Compile(*flagRun)
		if err != nil {
			log.Fatalf("invalid -run argument: %v", err)
		}
		runRE = r
	}

	for i := 0; i < *flagCount; i++ {
		if *flagPackage != "" {
			runBuild("BenchmarkPkg", *flagPackage, i)
			continue
		}
		for _, tt := range tests {
			if tt.long && *flagShort {
				continue
			}
			if runRE == nil || runRE.MatchString(tt.name) {
				runBuild(tt.name, tt.dir, i)
			}
		}
	}
}

func runCmd(name string, cmd *exec.Cmd) {
	start := time.Now()
	out, err := cmd.CombinedOutput()
	if err != nil {
		log.Printf("%v: %v\n%s", name, err, out)
		return
	}
	fmt.Printf("%s 1 %d ns/op\n", name, time.Since(start).Nanoseconds())
}

func runStdCmd() {
	args := []string{"build", "-a"}
	if *flagCompilerFlags != "" {
		args = append(args, "-gcflags", *flagCompilerFlags)
	}
	args = append(args, "std", "cmd")
	cmd := exec.Command(*flagGoCmd, args...)
	cmd.Dir = filepath.Join(goroot, "src")
	runCmd("BenchmarkStdCmd", cmd)
}

// path is either a path to a file ("$GOROOT/test/helloworld.go") or a package path ("cmd/go").
func runSize(name, path string) {
	cmd := exec.Command(*flagGoCmd, "build", "-o", "_compilebenchout_", path)
	cmd.Stdout = os.Stderr
	cmd.Stderr = os.Stderr
	if err := cmd.Run(); err != nil {
		log.Print(err)
		return
	}
	defer os.Remove("_compilebenchout_")
	info, err := os.Stat("_compilebenchout_")
	if err != nil {
		log.Print(err)
		return
	}
	out, err := exec.Command("size", "_compilebenchout_").CombinedOutput()
	if err != nil {
		log.Printf("size: %v\n%s", err, out)
		return
	}
	lines := strings.Split(string(out), "\n")
	if len(lines) < 2 {
		log.Printf("not enough output from size: %s", out)
		return
	}
	f := strings.Fields(lines[1])
	if strings.HasPrefix(lines[0], "__TEXT") && len(f) >= 2 { // OS X
		fmt.Printf("%s 1 %s text-bytes %s data-bytes %v exe-bytes\n", name, f[0], f[1], info.Size())
	} else if strings.Contains(lines[0], "bss") && len(f) >= 3 {
		fmt.Printf("%s 1 %s text-bytes %s data-bytes %s bss-bytes %v exe-bytes\n", name, f[0], f[1], f[2], info.Size())
	}
}

func runBuild(name, dir string, count int) {
	switch name {
	case "BenchmarkStdCmd":
		runStdCmd()
		return
	case "BenchmarkCmdGoSize":
		runSize("BenchmarkCmdGoSize", "cmd/go")
		return
	case "BenchmarkHelloSize":
		runSize("BenchmarkHelloSize", filepath.Join(goroot, "test/helloworld.go"))
		return
	}

	pkg, err := build.Import(dir, ".", 0)
	if err != nil {
		log.Print(err)
		return
	}
	args := []string{"-o", "_compilebench_.o"}
	if is6g {
		*flagMemprofilerate = -1
		*flagAlloc = false
		*flagCpuprofile = ""
		*flagMemprofile = ""
	}
	if *flagMemprofilerate >= 0 {
		args = append(args, "-memprofilerate", fmt.Sprint(*flagMemprofilerate))
	}
	args = append(args, strings.Fields(*flagCompilerFlags)...)
	if *flagAlloc || *flagCpuprofile != "" || *flagMemprofile != "" {
		if *flagAlloc || *flagMemprofile != "" {
			args = append(args, "-memprofile", "_compilebench_.memprof")
		}
		if *flagCpuprofile != "" {
			args = append(args, "-cpuprofile", "_compilebench_.cpuprof")
		}
	}
	args = append(args, pkg.GoFiles...)
	cmd := exec.Command(compiler, args...)
	cmd.Dir = pkg.Dir
	cmd.Stdout = os.Stderr
	cmd.Stderr = os.Stderr
	start := time.Now()
	err = cmd.Run()
	if err != nil {
		log.Printf("%v: %v", name, err)
		return
	}
	end := time.Now()

	var allocs, allocbytes int64
	if *flagAlloc || *flagMemprofile != "" {
		out, err := ioutil.ReadFile(pkg.Dir + "/_compilebench_.memprof")
		if err != nil {
			log.Print("cannot find memory profile after compilation")
		}
		for _, line := range strings.Split(string(out), "\n") {
			f := strings.Fields(line)
			if len(f) < 4 || f[0] != "#" || f[2] != "=" {
				continue
			}
			val, err := strconv.ParseInt(f[3], 0, 64)
			if err != nil {
				continue
			}
			switch f[1] {
			case "TotalAlloc":
				allocbytes = val
			case "Mallocs":
				allocs = val
			}
		}

		if *flagMemprofile != "" {
			if err := ioutil.WriteFile(*flagMemprofile, out, 0666); err != nil {
				log.Print(err)
			}
		}
		os.Remove(pkg.Dir + "/_compilebench_.memprof")
	}

	if *flagCpuprofile != "" {
		out, err := ioutil.ReadFile(pkg.Dir + "/_compilebench_.cpuprof")
		if err != nil {
			log.Print(err)
		}
		outpath := *flagCpuprofile
		if *flagCount != 1 {
			outpath = fmt.Sprintf("%s_%d", outpath, count)
		}
		if err := ioutil.WriteFile(outpath, out, 0666); err != nil {
			log.Print(err)
		}
		os.Remove(pkg.Dir + "/_compilebench_.cpuprof")
	}

	wallns := end.Sub(start).Nanoseconds()
	userns := cmd.ProcessState.UserTime().Nanoseconds()

	fmt.Printf("%s 1 %d ns/op %d user-ns/op", name, wallns, userns)
	if *flagAlloc {
		fmt.Printf(" %d B/op %d allocs/op", allocbytes, allocs)
	}

	opath := pkg.Dir + "/_compilebench_.o"
	if *flagObj {
		// TODO(josharian): object files are big; just read enough to find what we seek.
		data, err := ioutil.ReadFile(opath)
		if err != nil {
			log.Print(err)
		}
		// Find start of export data.
		i := bytes.Index(data, []byte("\n$$B\n")) + len("\n$$B\n")
		// Count bytes to end of export data.
		nexport := bytes.Index(data[i:], []byte("\n$$\n"))
		fmt.Printf(" %d object-bytes %d export-bytes", len(data), nexport)
	}
	fmt.Println()

	os.Remove(opath)
}
<|endoftext|>


<commit_before>package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"strconv"
	"strings"
	"time"

	"github.com/pkg/errors"

	"github.com/datawire/ambassador/pkg/supervisor"
)

func (d *Daemon) interceptMessage() string {
	switch {
	case d.cluster == nil:
		return "Not connected (use 'edgectl connect' to connect to your cluster)"
	case d.trafficMgr == nil:
		return "Intercept unavailable: no traffic manager"
	case !d.trafficMgr.IsOkay():
		return "Connecting to traffic manager..."
	default:
		return ""
	}
}

// InterceptInfo tracks one intercept operation
type InterceptInfo struct {
	Name       string // Name of the intercept (user/logging)
	Namespace  string // Name in which to create the Intercept mapping
	Deployment string // Name of the deployment being intercepted
	Prefix     string // Prefix to intercept (default /)
	Patterns   map[string]string
	TargetHost string
	TargetPort int
}

// path returns the URL path for this intercept
func (ii *InterceptInfo) path() string {
	return fmt.Sprintf("intercept/%s/%s", ii.Namespace, ii.Deployment)
}

// Acquire an intercept from the traffic manager
func (ii *InterceptInfo) Acquire(_ *supervisor.Process, tm *TrafficManager) (int, error) {
	reqPatterns := make([]map[string]string, 0, len(ii.Patterns))
	for header, regex := range ii.Patterns {
		pattern := map[string]string{"name": header, "regex_match": regex}
		reqPatterns = append(reqPatterns, pattern)
	}
	request := map[string]interface{}{
		"name":     ii.Name,
		"patterns": reqPatterns,
	}
	reqData, err := json.Marshal(request)
	if err != nil {
		return 0, err
	}

	result, code, err := tm.request("POST", ii.path(), reqData)
	if err != nil {
		return 0, errors.Wrap(err, "acquire intercept")
	}
	if code == 404 {
		return 0, fmt.Errorf("deployment %q is not known to the traffic manager", ii.Deployment)
	}
	if !(200 <= code && code <= 299) {
		return 0, fmt.Errorf("acquire intercept: %s: %s", http.StatusText(code), result)
	}
	port, err := strconv.Atoi(result)
	if err != nil {
		return 0, errors.Wrapf(err, "bad port number from traffic manager: %q", result)
	}
	return port, nil
}

// Retain the given intercept. This likely needs to be called every
// five seconds or so.
func (ii *InterceptInfo) Retain(_ *supervisor.Process, tm *TrafficManager, port int) error {
	data := []byte(fmt.Sprintf("{\"port\": %d}", port))
	result, code, err := tm.request("POST", ii.path(), data)
	if err != nil {
		return errors.Wrap(err, "retain intercept")
	}
	if !(200 <= code && code <= 299) {
		return fmt.Errorf("retain intercept: %s: %s", http.StatusText(code), result)
	}
	return nil
}

// Release the given intercept.
func (ii *InterceptInfo) Release(_ *supervisor.Process, tm *TrafficManager, port int) error {
	data := []byte(fmt.Sprintf("%d", port))
	result, code, err := tm.request("DELETE", ii.path(), data)
	if err != nil {
		return errors.Wrap(err, "release intercept")
	}
	if !(200 <= code && code <= 299) {
		return fmt.Errorf("release intercept: %s: %s", http.StatusText(code), result)
	}
	return nil
}

// ListIntercepts lists active intercepts
func (d *Daemon) ListIntercepts(_ *supervisor.Process, out *Emitter) error {
	msg := d.interceptMessage()
	if msg != "" {
		out.Println(msg)
		out.Send("intercept", msg)
		return nil
	}
	for idx, cept := range d.intercepts {
		ii := cept.ii
		out.Printf("%4d. %s\n", idx+1, ii.Name)
		out.Send(fmt.Sprintf("local_intercept.%d", idx+1), ii.Name)
		key := "local_intercept." + ii.Name
		out.Printf("      Intercepting requests to %s when\n", ii.Deployment)
		out.Send(key, ii.Deployment)
		for k, v := range ii.Patterns {
			out.Printf("      - %s: %s\n", k, v)
			out.Send(key+"."+k, v)
		}
		out.Printf("      and redirecting them to %s:%d\n", ii.TargetHost, ii.TargetPort)
		out.Send(key+".host", ii.TargetHost)
		out.Send(key+".port", ii.TargetPort)
	}
	if len(d.intercepts) == 0 {
		out.Println("No intercepts")
	}
	return nil
}

// AddIntercept adds one intercept
func (d *Daemon) AddIntercept(p *supervisor.Process, out *Emitter, ii *InterceptInfo) error {
	msg := d.interceptMessage()
	if msg != "" {
		out.Println(msg)
		out.Send("intercept", msg)
		return nil
	}

	for _, cept := range d.intercepts {
		if cept.ii.Name == ii.Name {
			out.Printf("Intercept with name %q already exists\n", ii.Name)
			out.Send("failed", "intercept name exists")
			out.SendExit(1)
			return nil
		}
	}

	// Do we already have a namespace?
	if ii.Namespace == "" {
		// Nope. See if we have an interceptable that matches the name.

		matches := make([]InterceptInfo, 0)
		for _, deployment := range d.trafficMgr.interceptables {
			fields := strings.SplitN(deployment, "/", 2)

			appName := fields[0]
			appNamespace := d.cluster.namespace

			if len(fields) > 1 {
				appNamespace = fields[0]
				appName = fields[1]
			}

			if ii.Deployment == appName {
				// Abuse InterceptInfo rather than defining a new tuple type.
				matches = append(matches, InterceptInfo{"", appNamespace, appName, "", nil, "", 0})
			}
		}

		switch len(matches) {
		case 0:
			out.Printf("No interceptable deployment matching %s found", ii.Name)
			out.Send("failed", "no interceptable deployment matches")
			out.SendExit(1)
			return nil

		case 1:
			// Good to go.
			ii.Namespace = matches[0].Namespace
			out.Printf("Using deployment %s in namespace %s\n", ii.Name, ii.Namespace)

		default:
			out.Printf("Found more than one possible match:\n")

			for idx, match := range matches {
				out.Printf("%4d: %s in namespace %s\n", idx+1, match.Deployment, match.Namespace)
			}
			out.Send("failed", "multiple interceptable deployment matched")
			out.SendExit(1)
			return nil
		}
	}

	cept, err := MakeIntercept(p, out, d.trafficMgr, d.cluster, ii)
	if err != nil {
		out.Printf("Failed to establish intercept: %s\n", err)
		out.Send("failed", err.Error())
		out.SendExit(1)
		return nil
	}
	d.intercepts = append(d.intercepts, cept)
	out.Printf("Added intercept %q\n", ii.Name)
	return nil
}

// RemoveIntercept removes one intercept by name
func (d *Daemon) RemoveIntercept(p *supervisor.Process, out *Emitter, name string) error {
	msg := d.interceptMessage()
	for idx, cept := range d.intercepts {
		if cept.ii.Name == name {
			d.intercepts = append(d.intercepts[:idx], d.intercepts[idx+1:]...)

			out.Printf("Removed intercept %q\n", name)

			if err := cept.Close(); err != nil {
				out.Printf("Error while removing intercept: %v\n", err)
				out.Send("failed", err.Error())
				out.SendExit(1)
			}
			return nil
		}
	}
	if msg != "" {
		out.Println(msg)
		out.Send("intercept", msg)
		return nil
	}
	out.Printf("Intercept named %q not found\n", name)
	out.Send("failed", "not found")
	out.SendExit(1)
	return nil
}

// ClearIntercepts removes all intercepts
func (d *Daemon) ClearIntercepts(p *supervisor.Process) error {
	for _, cept := range d.intercepts {
		if err := cept.Close(); err != nil {
			p.Logf("Closing intercept %q: %v", cept.ii.Name, err)
		}
	}
	d.intercepts = d.intercepts[:0]
	return nil
}

// Intercept is a Resource handle that represents a live intercept
type Intercept struct {
	ii            *InterceptInfo
	tm            *TrafficManager
	cluster       *KCluster
	port          int
	crc           Resource
	mappingExists bool
	ResourceBase
}

// removeMapping drops an Intercept's mapping if needed (and possible).
func (cept *Intercept) removeMapping(p *supervisor.Process) error {
	var err error
	err = nil

	if cept.mappingExists {
		p.Logf("%v: Deleting mapping in namespace %v", cept.ii.Name, cept.ii.Namespace)
		delete := cept.cluster.GetKubectlCmd(p, "delete", "-n", cept.ii.Namespace, "mapping", fmt.Sprintf("%s-mapping", cept.ii.Name))
		err = delete.Run()
		p.Logf("%v: Deleted mapping in namespace %v", cept.ii.Name, cept.ii.Namespace)
	}

	if err != nil {
		return errors.Wrap(err, "Intercept: mapping could not be deleted")
	}

	return nil
}

type mappingMetadata struct {
	Name      string `json:"name"`
	Namespace string `json:"namespace"`
}

type mappingSpec struct {
	AmbassadorID []string          `json:"ambassador_id"`
	Prefix       string            `json:"prefix"`
	Service      string            `json:"service"`
	RegexHeaders map[string]string `json:"regex_headers"`
}

type interceptMapping struct {
	APIVersion string          `json:"apiVersion"`
	Kind       string          `json:"kind"`
	Metadata   mappingMetadata `json:"metadata"`
	Spec       mappingSpec     `json:"spec"`
}

// MakeIntercept acquires an intercept and returns a Resource handle
// for it
func MakeIntercept(p *supervisor.Process, out *Emitter, tm *TrafficManager, cluster *KCluster, ii *InterceptInfo) (*Intercept, error) {
	port, err := ii.Acquire(p, tm)
	if err != nil {
		return nil, err
	}

	cept := &Intercept{ii: ii, tm: tm, cluster: cluster, port: port}
	cept.mappingExists = false
	cept.doCheck = cept.check
	cept.doQuit = cept.quit
	cept.setup(p.Supervisor(), ii.Name)

	p.Logf("%s: Intercepting via port %v, using namespace %v", ii.Name, port, ii.Namespace)

	mapping := interceptMapping{
		APIVersion: "getambassador.io/v2",
		Kind:       "Mapping",
		Metadata: mappingMetadata{
			Name:      fmt.Sprintf("%s-mapping", ii.Name),
			Namespace: ii.Namespace,
		},
		Spec: mappingSpec{
			AmbassadorID: []string{fmt.Sprintf("intercept-%s", ii.Deployment)},
			Prefix:       ii.Prefix,
			Service:      fmt.Sprintf("telepresence-proxy.%s:%d", tm.namespace, port),
			RegexHeaders: ii.Patterns,
		},
	}

	manifest, err := json.MarshalIndent(&mapping, "", "  ")
	if err != nil {
		_ = cept.Close()
		return nil, errors.Wrap(err, "Intercept: mapping could not be constructed")
	}

	out.Printf("%s: applying intercept mapping in namespace %s\n", ii.Name, ii.Namespace)

	apply := cluster.GetKubectlCmdNoNamespace(p, "apply", "-f", "-")
	apply.Stdin = strings.NewReader(string(manifest))
	err = apply.Run()

	if err != nil {
		p.Logf("%v: Intercept could not apply mapping: %v", ii.Name, err)
		_ = cept.Close()
		return nil, errors.Wrap(err, "Intercept: kubectl apply")
	}

	cept.mappingExists = true

	sshCmd := []string{
		"ssh", "-C", "-N", "telepresence@localhost",
		"-oConnectTimeout=10", "-oExitOnForwardFailure=yes",
		"-oStrictHostKeyChecking=no", "-oUserKnownHostsFile=/dev/null",
		"-p", strconv.Itoa(tm.sshPort),
		"-R", fmt.Sprintf("%d:%s:%d", cept.port, ii.TargetHost, ii.TargetPort),
	}

	p.Logf("%s: starting SSH tunnel", ii.Name)
	out.Printf("%s: starting SSH tunnel\n", ii.Name)

	ssh, err := CheckedRetryingCommand(p, ii.Name+"-ssh", sshCmd, nil, nil, 5*time.Second)
	if err != nil {
		_ = cept.Close()
		return nil, err
	}

	cept.crc = ssh

	return cept, nil
}

func (cept *Intercept) check(p *supervisor.Process) error {
	return cept.ii.Retain(p, cept.tm, cept.port)
}

func (cept *Intercept) quit(p *supervisor.Process) error {
	cept.done = true

	p.Logf("cept.Quit removing %v", cept.ii.Name)

	cept.removeMapping(p)

	p.Logf("cept.Quit removed %v", cept.ii.Name)

	if cept.crc != nil {
		_ = cept.crc.Close()
	}

	p.Logf("cept.Quit releasing %v", cept.ii.Name)

	if err := cept.ii.Release(p, cept.tm, cept.port); err != nil {
		p.Log(err)
	}

	p.Logf("cept.Quit released %v", cept.ii.Name)

	return nil
}
<commit_msg>Intercept: Log error when removing Mapping<commit_after>package main

import (
	"encoding/json"
	"fmt"
	"net/http"
	"strconv"
	"strings"
	"time"

	"github.com/pkg/errors"

	"github.com/datawire/ambassador/pkg/supervisor"
)

func (d *Daemon) interceptMessage() string {
	switch {
	case d.cluster == nil:
		return "Not connected (use 'edgectl connect' to connect to your cluster)"
	case d.trafficMgr == nil:
		return "Intercept unavailable: no traffic manager"
	case !d.trafficMgr.IsOkay():
		return "Connecting to traffic manager..."
	default:
		return ""
	}
}

// InterceptInfo tracks one intercept operation
type InterceptInfo struct {
	Name       string // Name of the intercept (user/logging)
	Namespace  string // Name in which to create the Intercept mapping
	Deployment string // Name of the deployment being intercepted
	Prefix     string // Prefix to intercept (default /)
	Patterns   map[string]string
	TargetHost string
	TargetPort int
}

// path returns the URL path for this intercept
func (ii *InterceptInfo) path() string {
	return fmt.Sprintf("intercept/%s/%s", ii.Namespace, ii.Deployment)
}

// Acquire an intercept from the traffic manager
func (ii *InterceptInfo) Acquire(_ *supervisor.Process, tm *TrafficManager) (int, error) {
	reqPatterns := make([]map[string]string, 0, len(ii.Patterns))
	for header, regex := range ii.Patterns {
		pattern := map[string]string{"name": header, "regex_match": regex}
		reqPatterns = append(reqPatterns, pattern)
	}
	request := map[string]interface{}{
		"name":     ii.Name,
		"patterns": reqPatterns,
	}
	reqData, err := json.Marshal(request)
	if err != nil {
		return 0, err
	}

	result, code, err := tm.request("POST", ii.path(), reqData)
	if err != nil {
		return 0, errors.Wrap(err, "acquire intercept")
	}
	if code == 404 {
		return 0, fmt.Errorf("deployment %q is not known to the traffic manager", ii.Deployment)
	}
	if !(200 <= code && code <= 299) {
		return 0, fmt.Errorf("acquire intercept: %s: %s", http.StatusText(code), result)
	}
	port, err := strconv.Atoi(result)
	if err != nil {
		return 0, errors.Wrapf(err, "bad port number from traffic manager: %q", result)
	}
	return port, nil
}

// Retain the given intercept. This likely needs to be called every
// five seconds or so.
func (ii *InterceptInfo) Retain(_ *supervisor.Process, tm *TrafficManager, port int) error {
	data := []byte(fmt.Sprintf("{\"port\": %d}", port))
	result, code, err := tm.request("POST", ii.path(), data)
	if err != nil {
		return errors.Wrap(err, "retain intercept")
	}
	if !(200 <= code && code <= 299) {
		return fmt.Errorf("retain intercept: %s: %s", http.StatusText(code), result)
	}
	return nil
}

// Release the given intercept.
func (ii *InterceptInfo) Release(_ *supervisor.Process, tm *TrafficManager, port int) error {
	data := []byte(fmt.Sprintf("%d", port))
	result, code, err := tm.request("DELETE", ii.path(), data)
	if err != nil {
		return errors.Wrap(err, "release intercept")
	}
	if !(200 <= code && code <= 299) {
		return fmt.Errorf("release intercept: %s: %s", http.StatusText(code), result)
	}
	return nil
}

// ListIntercepts lists active intercepts
func (d *Daemon) ListIntercepts(_ *supervisor.Process, out *Emitter) error {
	msg := d.interceptMessage()
	if msg != "" {
		out.Println(msg)
		out.Send("intercept", msg)
		return nil
	}
	for idx, cept := range d.intercepts {
		ii := cept.ii
		out.Printf("%4d. %s\n", idx+1, ii.Name)
		out.Send(fmt.Sprintf("local_intercept.%d", idx+1), ii.Name)
		key := "local_intercept." + ii.Name
		out.Printf("      Intercepting requests to %s when\n", ii.Deployment)
		out.Send(key, ii.Deployment)
		for k, v := range ii.Patterns {
			out.Printf("      - %s: %s\n", k, v)
			out.Send(key+"."+k, v)
		}
		out.Printf("      and redirecting them to %s:%d\n", ii.TargetHost, ii.TargetPort)
		out.Send(key+".host", ii.TargetHost)
		out.Send(key+".port", ii.TargetPort)
	}
	if len(d.intercepts) == 0 {
		out.Println("No intercepts")
	}
	return nil
}

// AddIntercept adds one intercept
func (d *Daemon) AddIntercept(p *supervisor.Process, out *Emitter, ii *InterceptInfo) error {
	msg := d.interceptMessage()
	if msg != "" {
		out.Println(msg)
		out.Send("intercept", msg)
		return nil
	}

	for _, cept := range d.intercepts {
		if cept.ii.Name == ii.Name {
			out.Printf("Intercept with name %q already exists\n", ii.Name)
			out.Send("failed", "intercept name exists")
			out.SendExit(1)
			return nil
		}
	}

	// Do we already have a namespace?
	if ii.Namespace == "" {
		// Nope. See if we have an interceptable that matches the name.

		matches := make([]InterceptInfo, 0)
		for _, deployment := range d.trafficMgr.interceptables {
			fields := strings.SplitN(deployment, "/", 2)

			appName := fields[0]
			appNamespace := d.cluster.namespace

			if len(fields) > 1 {
				appNamespace = fields[0]
				appName = fields[1]
			}

			if ii.Deployment == appName {
				// Abuse InterceptInfo rather than defining a new tuple type.
				matches = append(matches, InterceptInfo{"", appNamespace, appName, "", nil, "", 0})
			}
		}

		switch len(matches) {
		case 0:
			out.Printf("No interceptable deployment matching %s found", ii.Name)
			out.Send("failed", "no interceptable deployment matches")
			out.SendExit(1)
			return nil

		case 1:
			// Good to go.
			ii.Namespace = matches[0].Namespace
			out.Printf("Using deployment %s in namespace %s\n", ii.Name, ii.Namespace)

		default:
			out.Printf("Found more than one possible match:\n")

			for idx, match := range matches {
				out.Printf("%4d: %s in namespace %s\n", idx+1, match.Deployment, match.Namespace)
			}
			out.Send("failed", "multiple interceptable deployment matched")
			out.SendExit(1)
			return nil
		}
	}

	cept, err := MakeIntercept(p, out, d.trafficMgr, d.cluster, ii)
	if err != nil {
		out.Printf("Failed to establish intercept: %s\n", err)
		out.Send("failed", err.Error())
		out.SendExit(1)
		return nil
	}
	d.intercepts = append(d.intercepts, cept)
	out.Printf("Added intercept %q\n", ii.Name)
	return nil
}

// RemoveIntercept removes one intercept by name
func (d *Daemon) RemoveIntercept(p *supervisor.Process, out *Emitter, name string) error {
	msg := d.interceptMessage()
	for idx, cept := range d.intercepts {
		if cept.ii.Name == name {
			d.intercepts = append(d.intercepts[:idx], d.intercepts[idx+1:]...)

			out.Printf("Removed intercept %q\n", name)

			if err := cept.Close(); err != nil {
				out.Printf("Error while removing intercept: %v\n", err)
				out.Send("failed", err.Error())
				out.SendExit(1)
			}
			return nil
		}
	}
	if msg != "" {
		out.Println(msg)
		out.Send("intercept", msg)
		return nil
	}
	out.Printf("Intercept named %q not found\n", name)
	out.Send("failed", "not found")
	out.SendExit(1)
	return nil
}

// ClearIntercepts removes all intercepts
func (d *Daemon) ClearIntercepts(p *supervisor.Process) error {
	for _, cept := range d.intercepts {
		if err := cept.Close(); err != nil {
			p.Logf("Closing intercept %q: %v", cept.ii.Name, err)
		}
	}
	d.intercepts = d.intercepts[:0]
	return nil
}

// Intercept is a Resource handle that represents a live intercept
type Intercept struct {
	ii            *InterceptInfo
	tm            *TrafficManager
	cluster       *KCluster
	port          int
	crc           Resource
	mappingExists bool
	ResourceBase
}

// removeMapping drops an Intercept's mapping if needed (and possible).
func (cept *Intercept) removeMapping(p *supervisor.Process) error {
	var err error
	err = nil

	if cept.mappingExists {
		p.Logf("%v: Deleting mapping in namespace %v", cept.ii.Name, cept.ii.Namespace)
		delete := cept.cluster.GetKubectlCmd(p, "delete", "-n", cept.ii.Namespace, "mapping", fmt.Sprintf("%s-mapping", cept.ii.Name))
		err = delete.Run()
		p.Logf("%v: Deleted mapping in namespace %v", cept.ii.Name, cept.ii.Namespace)
	}

	if err != nil {
		return errors.Wrap(err, "Intercept: mapping could not be deleted")
	}

	return nil
}

type mappingMetadata struct {
	Name      string `json:"name"`
	Namespace string `json:"namespace"`
}

type mappingSpec struct {
	AmbassadorID []string          `json:"ambassador_id"`
	Prefix       string            `json:"prefix"`
	Service      string            `json:"service"`
	RegexHeaders map[string]string `json:"regex_headers"`
}

type interceptMapping struct {
	APIVersion string          `json:"apiVersion"`
	Kind       string          `json:"kind"`
	Metadata   mappingMetadata `json:"metadata"`
	Spec       mappingSpec     `json:"spec"`
}

// MakeIntercept acquires an intercept and returns a Resource handle
// for it
func MakeIntercept(p *supervisor.Process, out *Emitter, tm *TrafficManager, cluster *KCluster, ii *InterceptInfo) (*Intercept, error) {
	port, err := ii.Acquire(p, tm)
	if err != nil {
		return nil, err
	}

	cept := &Intercept{ii: ii, tm: tm, cluster: cluster, port: port}
	cept.mappingExists = false
	cept.doCheck = cept.check
	cept.doQuit = cept.quit
	cept.setup(p.Supervisor(), ii.Name)

	p.Logf("%s: Intercepting via port %v, using namespace %v", ii.Name, port, ii.Namespace)

	mapping := interceptMapping{
		APIVersion: "getambassador.io/v2",
		Kind:       "Mapping",
		Metadata: mappingMetadata{
			Name:      fmt.Sprintf("%s-mapping", ii.Name),
			Namespace: ii.Namespace,
		},
		Spec: mappingSpec{
			AmbassadorID: []string{fmt.Sprintf("intercept-%s", ii.Deployment)},
			Prefix:       ii.Prefix,
			Service:      fmt.Sprintf("telepresence-proxy.%s:%d", tm.namespace, port),
			RegexHeaders: ii.Patterns,
		},
	}

	manifest, err := json.MarshalIndent(&mapping, "", "  ")
	if err != nil {
		_ = cept.Close()
		return nil, errors.Wrap(err, "Intercept: mapping could not be constructed")
	}

	out.Printf("%s: applying intercept mapping in namespace %s\n", ii.Name, ii.Namespace)

	apply := cluster.GetKubectlCmdNoNamespace(p, "apply", "-f", "-")
	apply.Stdin = strings.NewReader(string(manifest))
	err = apply.Run()

	if err != nil {
		p.Logf("%v: Intercept could not apply mapping: %v", ii.Name, err)
		_ = cept.Close()
		return nil, errors.Wrap(err, "Intercept: kubectl apply")
	}

	cept.mappingExists = true

	sshCmd := []string{
		"ssh", "-C", "-N", "telepresence@localhost",
		"-oConnectTimeout=10", "-oExitOnForwardFailure=yes",
		"-oStrictHostKeyChecking=no", "-oUserKnownHostsFile=/dev/null",
		"-p", strconv.Itoa(tm.sshPort),
		"-R", fmt.Sprintf("%d:%s:%d", cept.port, ii.TargetHost, ii.TargetPort),
	}

	p.Logf("%s: starting SSH tunnel", ii.Name)
	out.Printf("%s: starting SSH tunnel\n", ii.Name)

	ssh, err := CheckedRetryingCommand(p, ii.Name+"-ssh", sshCmd, nil, nil, 5*time.Second)
	if err != nil {
		_ = cept.Close()
		return nil, err
	}

	cept.crc = ssh

	return cept, nil
}

func (cept *Intercept) check(p *supervisor.Process) error {
	return cept.ii.Retain(p, cept.tm, cept.port)
}

func (cept *Intercept) quit(p *supervisor.Process) error {
	cept.done = true

	p.Logf("cept.Quit removing %v", cept.ii.Name)

	if err := cept.removeMapping(p); err != nil {
		p.Logf("cept.Quit failed to remove %v: %+v", cept.ii.Name, err)
	} else {
		p.Logf("cept.Quit removed %v", cept.ii.Name)
	}

	if cept.crc != nil {
		_ = cept.crc.Close()
	}

	p.Logf("cept.Quit releasing %v", cept.ii.Name)

	if err := cept.ii.Release(p, cept.tm, cept.port); err != nil {
		p.Log(err)
	}

	p.Logf("cept.Quit released %v", cept.ii.Name)

	return nil
}
<|endoftext|>