[TOC]

GRPC request tracking

This is the first day of my participation in the More text Challenge. For details, see more text Challenge

preface

Let’s review the points we shared earlier:

  • GRPC introduction

Introduces the basic way to use your gRPC, framework, interaction and so on

  • GRPC certification

Share the gRPC four authentication methods of two important ways, interested can click to see oh

  • Openssl Certificate Generation Notes (go 1.15 or later)

The openSSL certificate generation has been sorted out and the key points have been highlighted and are worth a look

  • GRPC’s interceptor

The middleware in gRPC ecology is mainly to do the unified authentication work, and it is not necessary to write the authentication mode in each interface

In this article, there is a middleware to share gRPC’s various functions. If you are interested, please click on the link

go tool trace

Today we are going to talk about go request tracing, which means go Trace, and share Trace for several reasons:

  • Studying thetraceAfter that, you can practice it yourself and get a clear idea of the entire call stack
  • A lot of information can be captured by the tracker, as shown here

  • It can solve problems in our projects more quickly

We may all have such an experience, thinking clearly, the design of the program, can be very clear to explain the details, to the function of the increase can be flexible response.

But let us suddenly to modify others write a function or module, many times will be confused, this does not dare to move, that also dare not move, in the case of do not understand, doubt, must ask clear principle and logic, otherwise may not be online problems.

In this case, one of the most important reasons is how familiar you are with the current module/function and whether your thinking model is transferable.

To sum up, we should be in awe of the process, curious, persistent, and determined to solve the problem.

Where is it appropriate to use go Tool Trace and where is it not?

Don’t fit

  • Slow running functions, or to find out where most of the CPU time is being spent, what are the specialties, what are the tools for looking at CPU timego tool pprof

appropriate

  • Find out what the program is doing over time
  • go tool traceCan be achieved byview traceThe additional visualization capabilities provided by links are extremely helpful in diagnosing contention problems

Start writing a DEMO

GOMAXPROCS sets the maximum number of cpus that can be executed simultaneously, here we set it to 1

server.go

package main

import (
   "context"
   "fmt"
   "os"
   "runtime"
   "runtime/trace"
   "sync"
)

func main(a) {
   // Use GOMAXPROCS to set the maximum number of cpus that can be executed simultaneously to 1
   runtime.GOMAXPROCS(1)

   f, _ := os.Create("myTrace.dat")
   defer f.Close()


   // Start tracing, during which the trace is buffered and written to a file we specify
   _ = trace.Start(f)
   defer trace.Stop()

   // Let's define a task
   ctx, task := trace.NewTask(context.Background(), "customerTask")
   defer task.End()

   var wg sync.WaitGroup
   wg.Add(10)
   for i := 0; i < 10; i++ {
      // Start 10 coroutines to simulate doing tasks
      go func(num string) {
         defer wg.Done()

         / / tag num
         trace.WithRegion(ctx, num, func(a) {
            var sum, i int64
            // Simulate the task
            for ; i < 500000000; i++ {
               sum += i
            }
            fmt.Println(num, sum)
         })
      }(fmt.Sprintf("num_%02d", i))
   }
   wg.Wait()
}
Copy the code

Operation steps:

  • Compiling and running will produce the data files that we have defined
  • Go tool trace +myTrace.dat
  • The browser will pop uptracetheThe web pageAs shown in the following

tag instructions
View trace View the visual trace
Goroutine analysis Coroutines analysis
The Network blocking profile (⬇) Network congestion
Synchronization blocking profile (⬇) Synchronous blocking condition
The Syscall blocking profile (⬇) System call blocking
The Scheduler latency profile (⬇) Scheduling delay
User-defined tasks User-defined tasks
User-defined regions User-defined area
Minimum mutator utilization Minimum Mutator utilization

View trace

Visual Web tracking page

tag instructions
A time line It is used to display the time unit of execution, and the interval can be adjusted according to the time dimension by clicking the buttonYou can drag and drop the timeline across the interface
The heap Displays memory allocation and release during execution
coroutines Used to display each during executionGoroutineHow many coroutines are running during the run phase, which containsGCWaiting for,GCWaiting), operational (Runnable), in operation (Running) These three states.
OS threads Shows how many threads are running during execution, including those being calledSyscall(InSyscall), in operation (Running) These two states.
Virtual processor One row is displayed for each virtual processor, and the number of virtual processors generally defaults to the number of system cores.
Coroutines and events Shows what Goroutine is running on each virtual processor, and the wire behavior represents the event correlation.

You can useshift + ?Call out the help manual

Click the PROC color area

You can see what the processor does during this time, as shown in the figure

tag instructions
Start The start time
Wall Duration: The duration of the
Self Time The execution time
Start Stack Trace Stack information at the beginning
End Stack Trace Stack information at the end
Incoming flow The input stream
Outgoing flow The output stream
Preceding events Previous events
Following events Subsequent events
All connected All connected events

If we click on the Preceding Events below Event(s), we can see the execution time of each of the call stacks

How do I view user-defined tasks?

  • User-defined tasks
  • Click on the Count
  • goroutine view
  • Click on the color area

You can see the call stack, start time, end time, and how many coroutines the user-defined task has opened, and so on

User-defined tasks

Click on the Count

Click on the goroutine view

Click on the color area

You can view the specific actions performed during this period. The specific information is as follows

How to view user-defined areas?

The following is a similar way to view user-defined tasks

  • User-defined regions
  • Click on the specific Count

User-defined regions

Click on the specific Count

You can view the total time of the coroutine, network congestion, synchronization blocking, system call blocking, scheduling wait, garbage collection scan, garbage collection pause parameters information

When certain key Goroutines are blocked from running, there may be latency problems, probably for reasons that you can see

  • System call blocked;
  • Blocked by shared memory (channel/mutex, etc.)
  • The scheduler did not run the coroutine as often as we expected
  • Blocked by a Runtime system such as GC

As it happens, the tracing of the above reasons can be identified using go Tool Trace, which plays a good role in helping us track the problem and query the problem principle

Ok, that’s it for now, next time share the HTTP gateway of gRPC,

Technology is open, our mentality, should be more open. Embrace change, live in the sun, and strive to move forward.

I am nezha, welcome to like the collection, see you next time ~