Starting from: https://studygolang.com/articles/14410

You might be wondering — especially if you’re new to Go — how to add monitoring to your microservices application. As anyone with a track record can tell you, monitoring is difficult. So what I’m telling you is that at least the basic surveillance doesn’t work that way. You do not need to start a Prometheus cluster for your simple application to get reports, and in fact, you do not even need additional services to add a simple output of your application statistics.

But what features of our application are we interested in? The Go Runtime package contains functions that interact with the Go runtime system — such as the scheduler and memory manager. This means we can access some internal applications:

Goroutines

Goroutine is a very lightweight thread prepared for us by Go’s scheduling manager. A typical problem that can occur in any code is called a “goroutines leak.” There are many reasons for this, such as forgetting to set the default HTTP request timeout, SQL timeout, lack of support for context packet cancellation, sending data to closed channels, and so on. When this problem occurs, a Goroutine may live indefinitely and never release the resources it uses.

We might be interested in runtime.numGoroutine () int, which is a very basic function that returns the number of goroutines that currently exist. We can reasonably confirm that we may be leaking Goroutines by simply printing this number and examining it over a period of time, and then investigate those issues.

Memory footprint

Memory footprint issues are common in the Go world. While most people tend to use efficient Pointers (more efficient than anything else in Node.js), a performance-related issue that often comes up is memory allocation. To demonstrate a simple, but inefficient way to reverse a string:

1package main 2 3import ( 4 "strings" 5 "testing" 6) 7 8func BenchmarkStringReverseBad(b *testing.B) { 9 b.ReportAllocs()1011 input := "A pessimist sees the difficulty in every opportunity; an optimist sees the opportunity in every difficulty."1213 for i := 0; i < b.N; i++ {14 words := strings.Split(input, " ")15 wordsReverse := make([]string, 0)16 for {17 word := words[len(words)-1:][0]18 wordsReverse = append(wordsReverse, word)19 words = words[:len(words)-1]20 if len(words) == 0 {21 break22 }23 }24 output := strings.Join(wordsReverse, " ")25 if output ! = "difficulty. every in opportunity the sees optimist an opportunity; every in difficulty the sees pessimist A" {26 b.Error("Unexpected result: " + output)27 }28 }29}3031func BenchmarkStringReverseBetter(b *testing.B) {32 b.ReportAllocs()3334 input := "A pessimist  sees the difficulty in every opportunity; an optimist sees the opportunity in every difficulty."3536 for i := 0; i < b.N; i++ {37 words := strings.Split(input, " ")38 for i := 0; i < len(words)/2; i++ {39 words[len(words)-1-i], words[i] = words[i], words[len(words)-1-i]40 }41 output := strings.Join(words, " ")42 if output ! = "difficulty. every in opportunity the sees optimist an opportunity; every in difficulty the sees pessimist A" {43 b.Error("Unexpected result: " + output)44 }45 }46}Copy the code

1package main 2 3import ( 4 "strings" 5 "testing" 6) 7 8func BenchmarkStringReverseBad(b *testing.B) { 9 b.ReportAllocs()1011 input := "A pessimist sees the difficulty in every opportunity; an optimist sees the opportunity in every difficulty."1213 for i := 0; i < b.N; i++ {14 words := strings.Split(input, " ")15 wordsReverse := make([]string, 0)16 for {17 word := words[len(words)-1:][0]18 wordsReverse = append(wordsReverse, word)19 words = words[:len(words)-1]20 if len(words) == 0 {21 break22 }23 }24 output := strings.Join(wordsReverse, " ")25 if output ! = "difficulty. every in opportunity the sees optimist an opportunity; every in difficulty the sees pessimist A" {26 b.Error("Unexpected result: " + output)27 }28 }29}3031func BenchmarkStringReverseBetter(b *testing.B) {32 b.ReportAllocs()3334 input := "A pessimist  sees the difficulty in every opportunity; an optimist sees the opportunity in every difficulty."3536 for i := 0; i < b.N; i++ {37 words := strings.Split(input, " ")38 for i := 0; i < len(words)/2; i++ {39 words[len(words)-1-i], words[i] = words[i], words[len(words)-1-i]40 }41 output := strings.Join(words, " ")42 if output ! = "difficulty. every in opportunity the sees optimist an opportunity; every in difficulty the sees pessimist A" {43 b.Error("Unexpected result: " + output)44 }45 }46}Copy the code

This bad function does an unnecessary allocation, namely:

  1. We create an empty slice to store the result string,

  2. We populate this slice (append allocating memory is necessary, but not optimal)

Thanks to the call to b.reportallocs (), the benchmark and associated output draw an accurate picture:

BenchmarkStringReverseBad-4              1413 ns/op             976 B/op          8 allocs/opBenchmarkStringReverseBetter-4            775 ns/op             480 B/op          3 allocs/opCopy the code

Due to the virtual memory implemented in Go, another aspect of memory allocation is garbage collection pauses or GC. A common expression for GC pauses is “stop the world,” noting that your application will stop responding completely during a GC pause. The Google team continues to improve GC performance, but less experienced developers will still face poor memory management in the future.

This Runtime package exposes the Runtime. ReadMemStats(m *MemStats) function used to populate a MemStats object. This object has a number of fields that are good indicators of memory allocation strategies and performance-related issues.

  • Alloc – Number of bytes currently allocated in the heap,

  • TotalAlloc – The maximum number of bytes allocated in the heap (not reduced),

  • Sys – Total memory obtained from the system,

  • Mallocs and Frees – Allocate, free, and live number of objects (mallocs-frees),

  • PauseTotalNs – Total GC pauses from application,

  • NumGC – The number of GC cycles completed

methods

Therefore, we started with the premise that we did not want to use external services to provide simple application monitoring. My goal is to print the collected metrics to the console every once in a while. We should start a Goroutine, get this data every X seconds, and print it to the console.

1package main 2 3import ( 4 "encoding/json" 5 "fmt" 6 "runtime" 7 "time" 8) 910type Monitor struct {11 Alloc,12 TotalAlloc,13 Sys,14 Mallocs,15 Frees,16 LiveObjects,17 PauseTotalNs uint641819 NumGC uint3220 NumGoroutine int21}2223func NewMonitor(duration int) {24 var m Monitor25 var rtm runtime.MemStats26 var interval = time.Duration(duration) * time.Second27 for {28 <-time.After(interval)2930 // Read full mem stats31 runtime.ReadMemStats(&rtm)3233 // Number of goroutines34 m.NumGoroutine = runtime.NumGoroutine()3536 // Misc memory stats37 m.Alloc = rtm.Alloc38 m.TotalAlloc = rtm.TotalAlloc39 m.Sys = rtm.Sys40 m.Mallocs = rtm.Mallocs41 m.Frees = rtm.Frees4243 // Live objects = Mallocs - Frees44 m.LiveObjects = m.Mallocs - m.Frees4546 // GC Stats47 m.PauseTotalNs =  rtm.PauseTotalNs48 m.NumGC = rtm.NumGC4950 // Just encode to json and print51 b, _ := json.Marshal(m)52 fmt.Println(string(b))53 }54}Copy the code

To use it, you can call it with Go NewMonitor(300), which prints your application metrics every 5 minutes. You can then examine these from the console or from the history log to see the behavior of the application. Any performance impact of adding it to your application is minimal.

                                                        
{

                                                        
"Alloc":1143448 , "TotalAlloc":1143448 , "Sys":5605624 , "Mallocs":8718 , "Frees":301 , "LiveObjects":8417 , "PauseTotalNs":0 , "NumGC":0 , "NumGoroutine":6

                                                        
}

                                                        

                                                        
{

                                                        
"Alloc":1144504 , "TotalAlloc":1144504 , "Sys":5605624 , "Mallocs":8727 , "Frees":301 , "LiveObjects":8426 , "PauseTotalNs":0 , "NumGC":0 , "NumGoroutine":5

                                                        
}

                                                        .
                                                            Copy the code

I think the output in the console is a useful insight into the problems you might run into in the near future.

Using expvar

Go actually has two built-in plug-ins that help us monitor the application in production. One of the built-in packages is expvar. This package provides a standardized interface for common variables, such as operation counters in the server. By default, these variables will be available on /debug/vars. Let’s store the metrics in EXPVAR.

A few minutes later, I registered the HTTP handler for EXPVAR, and I realized that the full MemStats structure was already on it. That’s great!

In addition to adding HTTP handlers, this package also logs the following variables:

  • cmdline os.Args

  • memstats runtime.Memstats

This package is sometimes used only to register its HTTP handler and the side effects of the above variables. To use this, link the package to your program: import _ “expvar”

Since the metrics are now exported, you just need to point to the monitoring system on the application and import the MEMSTATS output there. I know we still don’t have a Goroutine count, but this is easy to add. Import the expvar package and add the following lines:

1// The next line goes at the start of NewMonitor()2var goroutines = expvar.NewInt("num_goroutine")3// The next line goes after the runtime.NumGoroutine() call4goroutines.Set(int64(m.NumGoroutine))Copy the code

This “num_goroutine” field is now available in /debug/ Vars output, only after full memory statistics.

Beyond basic monitoring

Another powerful addition to the Go standard library is the NET/HTTP /pprof package. This package has many functions, but its main purpose is to provide run-time analysis data for the Go Pprof tool, which is bundled with the Go tool chain. Using it, you can further examine the operation of your application in production. If you want to learn more about Pprof and code optimization, you can check out my previous article:

  • Go program benchmark,

  • Go program benchmarking, Part 2

Also, if you want the Go program to continuously analyze, you can use a Google service, StackDriver Profiler. However, if you want to monitor your own infrastructure for whatever reason, Prometheus is probably the best choice. If you would like to see it, please enter your email below.

Look here…

It would be great if you could buy my book:

  • API Foundations in Go

  • 12 Factor Apps with Docker and Go

  • The SaaS Handbook(work in progress)

I guarantee you’ll learn a lot if you buy any of them. Purchasing copies enables me to write more about the same topic. Thank you for buying my book.

If you want to make an appointment with me for consulting/outsourcing services you can email me. I am good at apis, Go, Docker, VueJS, extension services, etc.

via: https://scene-si.org/2018/08/06/basic-monitoring-of-go-apps-with-the-runtime-package/

By Tit Petric

Translator: themoonbear

Proofreading: polaris1119

This article is originally compiled by GCTT and published by Go Chinese