Scheduler analysis

In China, the concurrent model of Go is called G-M-P model. However, it seems that there is no such definition in foreign countries after checking online. They prefer to directly call it Go Scheduler — Go Scheduler. However, g-m-p are important concepts in the Go scheduler. They are defined in the sys/ Runtime /runtime2.go file. Let’s see what they stand for:

  1. G for Goroutine, defined as struct G, which stores the state information of the Goroutine, such as the execution stack information of the Goroutine, the wait information of the Goroutine, the GC information of the variable and other information. Each time we create a Goroutine with the keyword go, it creates a corresponding G object at the bottom of the GO program.
  2. M for Machine, defined as struct M, corresponds to the working thread of the operating system and corresponds to the physical processor thread. It is responsible for the scheduling of tasks and is the entity that actually drives THE operation of G, but M is not responsible for the management of the state of G. When G needs to be switched, M will write the stack state of G back to G. In fact, prior to Go1.1, Go had only a G-M model, and there was no concept of P until Dmitry Vyukov published Scalable Go Scheduler Design Doc in 2012, The paper pointed out the problems with the THEN-G-M model — the use of globally unique locks and centralized state to protect goroutine-related operations, switching between Goroutines that could be overloaded, memory caching issues between each M, and the added overhead of preemptive threads blocking and non-blocking. Hence the introduction of Processors into runtime.
  3. P for Processor, defined in struct P, implements the logical Processor. It is responsible for providing the relevant context, managing the memory cache, and taking charge of the Goroutine task queue. P is the middle tier G and M, M and P binding first, and then M will continuously removed from the task queue of P G and resume execution (remove operation without lock, because there is no competition for resources problem), when the P task queue are processed, P from the global queue back to a G to perform (remove operation have lock, because it is likely to compete with other P), When the global queue also has no G, G is stolen from other PS to execute. When there is no more G to execute, M and P are unbound and go to sleep. The number of P defaults to the number of physical threads.

The Go scheduler is the focus of the concept, and G-M-P is an important part of the Go scheduler. P and M usually correspond. In simple terms, P manages a set of G and is responsible for mounting G to M. When an M runs the same M for a long time, the Runtime creates a new M, and the P where the blocking G is will mount the rest of the G to the new M. When the blocking G completes or ends, the old M will be reclaimed.

About the operation of the G, because M in G in the process of operation, will meet need a context switch – G to be executed when a switch, G the execution of the site have to be protected, so that next time be scheduling execution time on-site recovery, Go scheduler, the M’s stack and M required to register (SP, PC, etc.) save to G, You can do on-site protection. When the scheduler runs the G again, M restores the scene by accessing the registers saved in G and can continue execution from where the last interruption occurred.

The Go scheduler uses the m: N scheduling technique, which multiplexes or schedules m Goroutines to N OS threads. The scheduling of M is responsible for by the Go program runtime, and the scheduling of N is responsible for by the OS. This allows m scheduling to be completed in user mode without frequent switching between kernel mode and user mode. At the same time, the allocation and release of memory, file IO, etc., Go also through memory pool and Netpoll and other technologies, to minimize the call of kernel state.

The state of the G

Goroutine has different G states at successive stages of its life cycle. By analyzing the state of G, we can understand the scheduling of Goroutine. The runtime2.go file defines that G has the following states: Idle, Runnable, RUNNING, Syscall, Waiting, Dead, copystack. The scan, scanrunnable, scan running, scansyscall, and scanwaiting GC states are deprecated. Moribund_unused and enqueue_unused are deprecated:

  1. _Gidle for idle means that the goroutine has just been created and has not yet been initialized. Since it has a value of 0, the newly created G objects are all _Gidle. However, in the only two calls to the Runtime library, the created G is immediately assigned a value of _Gdead to avoid Trackbacks and stack scans before it is added to the GC view, because the G is not a true Goroutine until the necessary processing is done.
  2. _Grunnable for runnable, which means that the Goroutine is already running on the queue. In this case, the Goroutine has not yet executed user code, and the execution stack of M is not goroutine’s own.
  3. _Grunning for running, meaning that the Goroutine may be executing user code, the execution stack of M is already owned by the Goroutine, and object G is not in the run queue. This state value will be assigned to M and P, and then be set by M and P.
  4. _Gsyscall for system scall, which means that the Goroutine is executing a system call, not user code. Like _Grunning, Goroutine has an execution stack and is not in a run queue. This state value can only be set by the assigned M.
  5. _Gwaiting for waiting, meaning that the goroutine is blocked at run time, and it neither executes user code nor is on the run queue. It is recorded somewhere else, such as a channel wait queue, so that the Goroutine can be ready when it is needed, which is the underlying implementation of goroutine and channel. At this point, the stack is not owned by the G object, unless a pipe is reading or writing to the stack. Except in these types of cases, it is not safe to try to fetch the execution stack of a goroutine after it has entered _Gwaiting.
  6. _Gdead for dead, meaning that the goroutine is not currently in use, either because the goroutine has just been created, or because the goroutine has been executed and exited and placed in the release list. When a G completes and exits, and G is added to the release list, the execution stacks of G and G are both owned by M.
  7. _Gcopystack for copystack, meaning that the execution stack of the Goroutine has been moved. The Goroutine neither executes user code nor is on the run queue. When this state is _Grunning, the execution stack space is insufficient or too large, need to expand or GC occurs, is the execution stack expansion or contraction of the middle state.
  8. The _Gscan series, used to flag states being scanned by GC, generated by _Gscan=0x1000 plus enumerations of _GRunnable, _Grunning, _Gsyscall, and _Gwaiting, The advantage of this is that the status before Scan can be known directly through simple operations. When marked in this series of states, none of these Goroutines execute user code, and their stack is owned by the goroutine doing the GC. However, the _Gscanrunning state is a bit special. This flag prevents the running Goroutine from switching to another state and tells the G to scan its own stack. It is this clever approach that makes GC for the Go language very efficient.

From the states listed above, goroutine is efficient for both waiting and GC. However, when system call is called, inefficient system call business code will affect the operation performance of Go application. Fortunately, Go language has encapsulated many tools to replace inefficient system call. For example, network call seems to be system call, but Go has actually encapsulated Netpoll in the bottom layer. We should try to use these libraries to avoid system calls. Bad designs that lead to frequent copy stacks and designs that lead to frequent GC are some of the things we need to be aware of.

2020.02

tou.hwang

Read more:

  1. The Go scheduler
  2. Scheduling In Go : Part I – OS Scheduler
  3. Scheduling In Go : Part II – Go Scheduler
  4. Scheduling In Go : Part III – Concurrency

References:

  1. Scalable Go Scheduler Design Doc
  2. Go 1.13.1 source code

Although there is a reference to the following article, but I feel that the concept and view are not necessarily right, so I re-think and screen its content, and then make reference.

  1. Go scheduling model
  2. Go scheduling model
  3. In-depth Golang scheduler GMP model
  4. Go – Goroutine concurrency model