In software development, a project coming online is not the end. After going online, it is necessary to analyze the operation of the sample program and reconstruct the existing functions to make the program more efficient and stable. Golang’s toolkit includes pprof functionality, which makes it much easier to identify the parts of your application that take up a lot of memory and CPU. The addition of Uber’s fire map, visual display, makes it easier to analyze the application.

Net/HTTP /pprof and Runtime /pprof. Net/HTTP /pprof encapsulates runtime/pprof packages and exposes them using HTTP, as shown in the source code below:

Analyze web services using NET/HTTP /pprof

Pprof analyzes Web projects and is as simple as importing packages.

_ "net/http/pprof"Copy the code

Write a small Web server

package main

import (
    _  "net/http/pprof"
    "net/http"
    "time"
    "math/rand"
    "fmt"
)

var Count int64 = 0
func main() {
    go calCount()

    http.HandleFunc("/test".test)
    http.HandleFunc("/data", handlerData)

    err := http.ListenAndServe(": 9909", nil )
    iferr ! = nil { panic(err) } } func handlerData(w http.ResponseWriter, r *http.Request) { qUrl := r.URL fmt.Println(qUrl) fibRev := Fib() var fib uint64for i:= 0; i < 5000; i++ {
        fib = fibRev()
        fmt.Println("fib = ", fib)
    }
    str := RandomStr(RandomInt(100, 500))
    str =  fmt.Sprintf("Fib = %d; String = %s", fib, str)
    w.Write([]byte(str))
}

func test(w http.ResponseWriter, r *http.Request) {
    fibRev := Fib()
    var fib uint64
    index := Count
    arr := make([]uint64, index)
    var i int64
    for ; i < index; i++ {
        fib = fibRev()
        arr[i] = fib
        fmt.Println("fib = ", fib)
    }
    time.Sleep(time.Millisecond * 500)
    str :=  fmt.Sprintf("Fib = %v", arr)
    w.Write([]byte(str))
}

func Fib() func() uint64 {
    var x, y uint64 = 0, 1
    return func() uint64 {
        x, y = y, x + y
        return x
    }
}

var letterRunes = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890")
func RandomStr(num int) string {
    seed := time.Now().UnixNano()
    if seed <= 0 {
        seed = time.Now().UnixNano()
    }
    rand.Seed(seed)
    b := make([]rune, num)
    for i := range b {
        b[i] = letterRunes[rand.Intn(len(letterRunes))]
    }
    return string(b)
}

func RandomInt(min, max int) int {
    rand.Seed(time.Now().UnixNano())
    return rand.Intn(max - min + 1) + min
}

func calCount() {
    timeInterval := time.Tick(time.Second)

    for {
        select {
        case i := <- timeInterval:
            Count = int64(i.Second())
        }
    }
}Copy the code


The Web service listens on port 9909

The Web server has two HTTP methods: test: Fibonacci calculation based on the current number of seconds Data: Fibonacci calculation 5000 and returns a random string

To run the program, by visiting http://192.168.3.34:9909/debug/pprof/ to view information on the web version of the profiles

So these paths represent theta

/debug/pprof/profile: Accessing this link automatically does CPU profiling for 30 seconds and generates a file for download

/debug/pprof/block: Records the Goroutine blocking event. The default is to sample every blocking event that occurs.

/debug/pprof/goroutines: Records active goroutines. Only sample once at fetch time.

/debug/pprof/heap: Records the heap memory allocation. By default, the sample is taken once every 512K bytes allocated.

/debug/pprof/mutex: View the owners of contention mutex.

/ debug/pprof/threadcreate: system thread creation of records. Only sample once at fetch time.

In addition to golang providing me with more convenient methods for analysis, let’s use commands to access detailed information

We use WRK to access our two methods so that our service is running at high speed and the sampling results are more accurate

wrk -c 20 -t 5 -d3 m http://192.168.3.34:9909/data WRK - 20 - t 5 c-d3 m http://192.168.3.34:9909/testCopy the code

Analyze CPU usage

Analyze CPU usage using commands

Go tool pprof httpdemo http://192.168.3.34:9909/debug/pprof/profileCopy the code

By default, the Go runtime samples CPU usage at a frequency of 100 Hz. That’s 100 samples per second, or once every 10 milliseconds. Why use this frequency? Because 100 Hz is enough to generate useful data, but not enough to cause the system to stall. And 100 is easy to convert, for example, from total sampling counts to samples per second. In effect, the sample of CPU usage is a sample of the program counter on the stack of the current Goroutine.

The default sampling time is 30s. You can specify the sampling time by using the -seconds command. After sampling is complete, it enters the command line state:

You can type help to view the commands. Here are a few common commands

Run the top command. By default, the top command adds the top 10 CPU usage methods back. Of course, you can add a number after the command to specify the top number

The list command outputs relevant methods based on your re. Directly following the option O will print all methods. You can also specify a method name

For example, the handlerData method accounts for 74.81% of the CPU

Web command: Displayed in web page format: displays THE CPU usage in a more intuitive manner

Analyze memory usage

Similar to analyzing cpus using commands

Go tool pprof httpdemo http://192.168.3.34:9909/debug/pprof/heapCopy the code

By default, only current memory usage is sampled. Optionally, the alloc_objects command is used to sample memory from the start of the program

Go tool pprof - alloc_objects httpdemo http://192.168.3.34:9909/debug/pprof/heapCopy the code

As with CPU commands, top List Web. The difference here is that it shows memory usage. I’m not going to do that.

Install the go – the torch

An even more convenient tool is Uber’s Go-Torch

Installation is simple

go get github.com/uber/go-torch
cd $GOPATH/src/github.com/uber/go-torch
git clone https://github.com/brendangregg/FlameGraph.gitCopy the code

Then run a copy of flamegraph.pl under FlameGraph to /usr/local/bin

Flame map analysis CPU

Using the command

Go-torch -u http://192.168.3.34:9909 --seconds 60-f cpu.svgCopy the code

The cpu.svg file is generated in the current directory and opened with a browser

Get a more intuitive view of the application’s problems. The handlerData method is consuming too much CPU time. Then it’s time to analyze and optimize in the code.

Flame map analysis memory

Using the command

Go - the torch http://192.168.3.34:9909/debug/pprof/heap - colors mem-f mem.svgCopy the code

The cpu.svg file is generated in the current directory and opened with a browser

Analyze projects using Runtime /pprof

If your project is not a Web service, such as an RPC service, use Runtime /pprof. He provides a lot of methods, you can have a look at the source code

I’ve written a simple utility class. For call analysis



package profapp

import (
    "os"
    "rrnc_im/lib/zaplogger"
    "go.uber.org/zap"
    "runtime/pprof"
    "runtime"
)

func StartCpuProf() {
    f, err := os.Create("cpu.prof")
    iferr ! = nil { zaplogger.Error("create cpu profile file error: ", zap.Error(err))
        return
    }
    iferr := pprof.StartCPUProfile(f); err ! = nil { zaplogger.Error("can not start cpu profile, error: ", zap.Error(err))
        f.Close()
    }
}

func StopCpuProf() {
    pprof.StopCPUProfile()
}


//--------Mem
func ProfGc() {
    runtime.GC() // get up-to-date statistics
}

func SaveMemProf() {
    f, err := os.Create("mem.prof")
    iferr ! = nil { zaplogger.Error("create mem profile file error: ", zap.Error(err))
        return
    }

    iferr := pprof.WriteHeapProfile(f); err ! = nil { zaplogger.Error("could not write memory profile: ", zap.Error(err))
    }
    f.Close()
}

// goroutine block
func SaveBlockProfile() {
    f, err := os.Create("block.prof")
    iferr ! = nil { zaplogger.Error("create mem profile file error: ", zap.Error(err))
        return
    }

    if err := pprof.Lookup("block").WriteTo(f, 0); err ! = nil { zaplogger.Error("could not write block profile: ", zap.Error(err))
    }
    f.Close()
}Copy the code


You can call these methods within the methods that you want to analyze for example if I’m opening several methods using RPC



type TestProf struct {

}

func (*TestProf) StartCpuProAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {
    profapp.StartCpuProf()
    return nil
}

func (*TestProf) StopCpuProfAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {
    profapp.StopCpuProf()
    return nil
}


func (*TestProf) ProfGcAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {
    profapp.ProfGc()
    return nil
}

func (*TestProf) SaveMemAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {
    profapp.SaveMemProf()
    return nil
}

func (*TestProf) SaveBlockProfileAct(context.Context, *im_test.TestRequest, *im_test.TestRequest) error {
    profapp.SaveBlockProfile()
    return nil
}Copy the code


call



profTest.StartCpuProAct(context.TODO(), &im_test.TestRequest{})

    time.Sleep(time.Second * 30)
    profTest.StopCpuProfAct(context.TODO(), &im_test.TestRequest{})

    profTest.SaveMemAct(context.TODO(), &im_test.TestRequest{})
    profTest.SaveBlockProfileAct(context.TODO(), &im_test.TestRequest{})Copy the code


The idea is the same, the profile file will be exported in the current folder. Then use the flame map to analyze, you can’t specify the domain name, to specify the file

 go-torch  httpdemo cpu.prof 
 go-torch  httpdemo mem.profCopy the code