Among golang’s own tools is pprof, a performance analysis tool. The tool is implemented in two locations:

  • Runtime /pprof: a collector that collects running data for the pPROF visualization tool
  • Net/HTTP /pprof: Visualization analysis of prof data via an HTTP Server.

Golang provides a variety of built-in performance collectors that are responsible for collecting this performance data:

  • Goroutine: Stack traces of all current Goroutines. Go Routine analysis.
  • CPU: Stack returned by runtime. Analysis of the CPU.
  • Heap: A sampling of memory allocations of live objects. Memory analysis, heap allocation and memory leaks.
  • Allocation: a sampling of all past memory allocations. Memory analysis, object allocation statistics and memory leaks.
  • Thread: Stack traces that led to the creation of New OS Threads. OS thread creation.
  • Block: stack traces that led to blocking on primitives. Blocking analysis. Record the go routine blocking, waiting and synchronization, timer/channel communication, and other details.
  • Mutex: Stack traces of contended mutexes. Mutex analysis. Including all kinds of competitive situations.

Collect performance data in the APP

When we need to perform performance analysis on an app, the first question is how to collect performance data. In Golang, the performance data mentioned above can be collected in several ways:

Go the test way

The Go test method is a non-invasive way to collect performance data from your app, using the following command line:

go test -cpuprofile cpu.prof -memprofile mem.prof -bench .
Copy the code

The command line above is to initiate the Benchmark test because we have been contracted to use the Benchmark test method to test the performance of the app. But you can also use the Coverage test:

go test . -v -race -coverprofile=coverage.txt -covermode=atomic -timeout=20m -test.short -cpuprofile cpu.prof -memprofile mem.prof | tee coverage.log
go tool cover -html=coverage.txt -o cover.html
Copy the code

Either way, you can add command-line arguments to collect performance data.

The command line options available are:

  • -memprofile mem.prof
  • -memprofilerate 4096
  • -cpuprofile cpu.prof
  • -blockprofile block.prof
  • -blockprofilerate 4096
  • -mutexprofile mutex.prof
  • -mutexprofilefraction 1
  • -trace trace.out

SRC /testing/testing. Go – The go Programming Language

The web application class

Live Profile Download or Live Profiling can be done for continuously working Apps such as Webapp applications or system services.

The easiest way to do this is to embed the “net/ HTTP /pprof” package. You don’t have to do anything when you start the HTTP service using the standard DefaultServeMux method:

package main

import (
	"net/http"
	_ "net/http/pprof"
)

func main(a) {
	http.ListenAndServe(": 8080".nil)}Copy the code

The “net/ HTTP /pprof” package will automatically add a set of API endpoints, and the package’s init function will do this:

func init(a) {
	http.HandleFunc("/debug/pprof/", Index)
	http.HandleFunc("/debug/pprof/cmdline", Cmdline)
	http.HandleFunc("/debug/pprof/profile", Profile)
	http.HandleFunc("/debug/pprof/symbol", Symbol)
	http.HandleFunc("/debug/pprof/trace", Trace)
}
Copy the code

So your HTTP server automatically owns these endpoints. Then you can either access the endpoints directly from a browser or have Pprof use them for interactive analysis:

go tool pprof http://localhost:8080/debug/pprof/heap
> top
...
> exit
Copy the code

Further explanation will be dissected in subsequent chapters, see [] for details.

If you are not using the default DefaultServeMux, you will need to manually link endpoints to your MUX. For example, for GIN, it might look like this:

r := gin.Default()
r.GET("/debug/pprof/allocs", WrapH(pprof.Handler("allocs")))
r.GET("/debug/pprof/block", WrapH(pprof.Handler("block")))
r.GET("/debug/pprof/goroutine", WrapH(pprof.Handler("goroutine")))
r.GET("/debug/pprof/heap", WrapH(pprof.Handler("heap")))
r.GET("/debug/pprof/mutex", WrapH(pprof.Handler("mutex")))
r.GET("/debug/pprof/threadcreate", WrapH(pprof.Handler("threadcreate")))
r.Run(": 8080")

func WrapH(h http.Handler) gin.HandlerFunc {
	return func(c *gin.Context) {
		h.ServeHTTP(c.Writer, c.Request)
	}
}
Copy the code

General application

If your app is not a persistent Web service, you can manually insert profe-specific code into your app via the Runtime /pprof package. Then, after the run, take the generated cpu.prof or mem.prof data collection files to pprof for analysis.

With the aid ofpkg/profile

Here’s a simple application:

package main

import (
	"fmt"
	"github.com/pkg/profile"
)

func main(a){
	defer profile.Start(profile.ProfilePath(".")).Stop()
	a()
}

func a(a){
	for i:=0; i<10000; i++{
		fmt.Print(".")
	}
	fmt.Println()
}
Copy the code

It simply collects CPU data to produce a CPU.prof file that you can parse directly:

go tool pprof -http=:6060 cpu.prof
Copy the code

You can also collect other data by adding these calls at the beginning of the main function:

// CPUProfile enables cpu profiling. Note: Default is CPU
defer profile.Start(profile.CPUProfile).Stop()

// GoroutineProfile enables goroutine profiling.
// It returns all Goroutines alive when defer occurs.
defer profile.Start(profile.GoroutineProfile).Stop()

// BlockProfile enables block (contention) profiling.
defer profile.Start(profile.BlockProfile).Stop()

// ThreadcreationProfile enables thread creation profiling.
defer profile.Start(profile.ThreadcreationProfile).Stop()

// MemProfile changes which type of memory profiling to 
// profile the heap.
defer profile.Start(profile.MemProfile).Stop()

// MutexProfile enables mutex profiling.
defer profile.Start(profile.MutexProfile).Stop()
Copy the code

However, only one performance data call can be added at a time. If you want to add more than one at a time, do this call:

defer profile.Start(profile.MutexProfile, profile.MemProfile).Stop()
Copy the code

This method can continue to be extended because Start(…) Options Pattern is used.

However, if you want more customization features, you may have to code yourself through the Runtime/Pprof to support customization options.

Their own code

Here’s a simple demo that shows you how to customize code to collect performance data, and you can improve it further:

package main

import (
	"fmt"
	"github.com/hedzr/log"
	stdLog "log"
	"os"
	"runtime"
	"runtime/pprof"
	"sync"
)

func main(a) {

	ifcpuProfile ! ="" {
		defer enableCpuProfile(cpuProfile)()
	}
	ifmemProfile ! ="" {
		defer enableMemProfile(memProfile)()
	}

	var wg sync.WaitGroup
	wg.Add(1)
	go a(&wg)
	wg.Add(1)
	go b(&wg)
	wg.Wait()
}

func a(wg *sync.WaitGroup) {
	for i := 0; i < 10000; i++ {
		fmt.Print(".")
	}
	fmt.Println()
	wg.Done()
}

func b(wg *sync.WaitGroup) {
	for i := 0; i < 10000; i++ {
		fmt.Print("_")
	}
	fmt.Println()
	wg.Done()
}

var cpuProfile, memProfile string

func init(a) {
	stdLog.SetFlags(stdLog.LstdFlags | stdLog.Llongfile)
	cpuProfile, memProfile = "cpu.prof"."mem.prof"
}

//
// And review the pprof result in a web ui:
//
// go tool pprof -http=:8555 ./cpu.pprof
//
// Now you can open 'http://localhost:8555/ui' in a browser
//
func enableCpuProfile(cpuProfilePath string) (closer func(a)) {
	closer = func(a) {}
	ifcpuProfilePath ! ="" {
		f, err := os.Create(cpuProfilePath)
		iferr ! =nil {
			log.Fatal("could not create cpu profile: %v", err)
		}
		err = pprof.StartCPUProfile(f)
		iferr ! =nil {
			log.Fatal("error: %v", err)
		}
		closer = pprof.StopCPUProfile
	}
	runtime.SetBlockProfileRate(20)
	return
}

func enableMemProfile(memProfilePath string) (closer func(a)) {
	closer = func(a) {}
	ifmemProfilePath ! ="" {
		closer = func(a) {
			f, err := os.Create(memProfilePath)
			iferr ! =nil {
				log.Fatal("could not create memory profile: ", err)
			}
			defer f.Close()
			runtime.GC() // get up-to-date statistics
			iferr := pprof.WriteHeapProfile(f); err ! =nil {
				log.Fatal("could not write memory profile: ", err)
			}
		}
	}
	return
}

Copy the code

The above

cmdrintegration

In HedZR/CMDR (V1.7.46 +), we provide a complete add-on package pprof to simplify a simple solution for general application integration with Go Tool Pprof. If you are using CMDR for command-line application development, simply:

package main

import (
	"github.com/hedzr/cmdr"
	"github.com/hedzr/cmdr/plugin/pprof"
	"github.com/hedzr/log"
	"github.com/hedzr/logex/build"
	"gopkg.in/hedzr/errors.v2"
)

func main(a) { Entry() }

func Entry(a) {
	if err := cmdr.Exec(buildRootCmd(),
		cmdr.WithLogx(build.New(build.NewLoggerConfigWith(true."logrus"."debug"))), pprof.GetCmdrProfilingOptions(), ); err ! =nil {
		log.Fatalf("error occurs in app running: %+v\n", err)
	}
}

func buildRootCmd(a) (rootCmd *cmdr.RootCommand) {
	root := cmdr.Root(appName, cmdr.Version).
		Copyright(copyright, "hedzr").
		Description(desc, longDesc).
		Examples(examples)
	rootCmd = root.RootCommand()

	cmdr.NewBool(false).
		Titles("enable-ueh"."ueh").
		Description("Enables the unhandled exception handler?").
		AttachTo(root)

	//pprof.AttachToCmdr(root.RootCmdOpt())
	return
}
Copy the code

You can integrate the Pprof add-on as simply as Line 17 or explicitly attach it to root as Line 35 does. It gives you a set of command-line arguments, such as -ep and so on:

For such applications, profiling and pprof can simply be enabled:

app -ep
app --pprof
app --enable-profile
Copy the code

By default, it produces several performance data files such as cpu.prof, mem.prof, and so on. If you think it is necessary to change the categories of performance data to be collected, call:

pprof.GetCmdrProfilingOptions("cpu"."mem"."mutex"."block"."thread-create"."trace"."go-routine"),
// Or
pprof.AttachToCmdr(root.RootCmdOpt(), "cpu"."mem"."mutex"."block"."thread-create"."trace"."go-routine")
Copy the code

We certainly highly recommend that you use CMDR to simplify your profiling integration efforts, and not only to simplify it, but also to include more CLI helper features.

The visualization tool Pprof

There are generally two ways to start the PPROF visualization tool: run it directly, or write a short piece of code to start it.

Whichever way you get the Prof data, you can simply launch the visualization tool directly with it as the data to be analyzed:

go tool pprof -http=:6060 cpu.prof
Copy the code

Use the pprof command line tool

Pprof has a separate command-line tool that you can install like this:

go get -u github.com/google/pprof
Copy the code

This command takes the source code of pprof and compiles it into the command line execution file pprof and puts it in $GOPATH/bin, which requires you to have a full Golang compilation environment (which of course goes without saying).

However, this standalone command-line tool is not really any different from the Go Tool Pprof. So assuming we have access to prof data, the visualization tool can be launched in either of the following ways:

pprof -http=:6060 cpu.prof
go tool pprof -http=:6060 cpu.prof
Copy the code

Generate a report

In pprof interaction mode, reports can be generated using PDF commands.

Interact through a TTY terminal

If no -web or -http parameter is specified, pprof goes into interactive mode.

Viewing Real-time Data

If your app is HTTP service type and has embedded “net/ HTTP /pprof” package, then a series of endpoints (/debug/pprof/*) are available, and performance data can be downloaded directly from the corresponding endpoints and analyzed in interactive mode:

go tool pprof 'http://localhost:6060/debug/pprof/profile? seconds=30'
(pprof) top
Copy the code

But in practice you can often open these endpoints directly in a browser to view snapshots of performance data directly.

In interactive mode, these commands are commonly used:

q/exit/quit

Exit the interactive mode and return to the Shell cli.

tree

Displays a literal version of the call structure.

top

Displays the top seconds in time. You can add numbers, such as top5, top10, and you can add the argument -cum sort by cumulative time, which is often useful to help you show the sequence of calls.

list

The List command displays performance data for the specified function in the specified package.

peek

The peek command is similar to the list command, but you can specify an obscure keyword here, for example:

Viewing through a browser

When viewing CPU performance data in a browser, you can view a variety of views through pprof’s Web interface.

top view

This view is relatively simple, but due to the heavy weight of go core calls and system calls, you’ll need to be patient to weed them out before you find the topN of business logic.

If you want to better retrieve the topN, you can interactively execute the top10-cum, top5-cum commands directly, or reorder the top cum in the Web view, in which case the business code is usually prioritized.

Grpah view

Grpah views are arranged in order of APP execution, almost equivalent to a flowchart. The larger the font in this view, the more time it takes.

Flame Graph view

The flame chart is also arranged from top to bottom in order of execution. The width ratio of a function represents the proportion of its time. The wider the function, the more time it takes. However, due to the variety of program structures (such as Goroutine, TimerProc, etc.), such simple judgments are not exact. In large programs you need to carefully REFINE and REFINE to make valuable performance decisions.

Peek view

The Peek view lists time statistics as a text table that you can filter.

Source View

The Source View lists the Source code for function calls. For this view to display correctly, you need to use the Go Tool pprof version, which may not render the source code on the command line. In addition, you must add your app’s executable file when loading the view:

go tool pprof -http :6060 bin/mdx-cli ref/cpu.prof
Copy the code

Disassemble View

It’s similar to the Source View, except that it displays assembly code.

How do I analyze PPROF performance data

This is a difficult subject to explain.

In general, you can read most performance data literally.

How to filter and how to focus regular filtering can be done with pprof’s Web interface.

For a more detailed reference, Go directly to Profiling Go Programs – The Go Blog.

Locating bad code requires long debugging experience.

:end:

  • runtime/pprof – The Go Programming Language
  • net/http/pprof – The Go Programming Language
  • Profiling Go Programs – The Go Blog
  • Diagnostics – The Go Programming Language
  • Go: Profile Your Code Like a Master – by Ali Josie – The Startup – Medium
  • gperftools/gperftools: Main gperftools repository

The ORIGINAL POST: hedzr. Making. IO/golang/initial…