– Back-end morning reading class translation project chapter 4 –
– Translation: A-journey-with-go
Welcome to follow wechat official account:Back end morning reading class
This article describes the design concept and evolution of stack in Golang in detail. The reason why the initial Stack size from Segment Stack to Contiguous Stack increases from 8Kb to 2Kb is described.
ℹ️ article is based on Go 1.12.
Go provides a lightweight and intelligent coroutine management mechanism. Light because the coroutine stack initialization is only 2Kb, smart because the coroutine stack can automatically increase/decrease as we need it.
Stack size definition, we can find runtime/stack.go here:
// The minimum size of stack used by Go code
_StackMin = 2048
Copy the codeIt is important to note that it has been optimized in the following versions of time:
-
Go 1.2: Coroutine stack grows from 4Kb to 8Kb.
-
Go 1.4: Coroutine stack reduced from 8Kb to 2Kb.
Coroutine stack sizes vary mainly because of changes in stack allocation strategies. We’ll come back to this later in the article.
The default stack size is sometimes inadequate for the program we’re running. Go will automatically resize the stack.
Dynamic stack size
If Go can automatically increase the stack size, it also means that it can decide whether the stack size needs to be changed or not. Let’s look at an example to see how it works:
func main(a) {
a := 1
b := 2
r := max(a, b)
println(`max: `+strconv.Itoa(r))
}
func max(a int, b int) int {
if a >= b {
return a
}
return b
}
Copy the codeThis example just computes the largest of the two numbers. To see how Go manages coroutine stack allocation, we can look at the compilation process code for Go, using the command Go build-gcflags -s main.go. The output — I’ve only kept a few lines related to the stack — gives us some interesting information about what Go does:
"".main STEXT size=186 args=0x0 locals=0x70
0x0000 00000 (/go/src/main.go:5) TEXT "".main(SB),
ABIInternal, $1120[...].0x00b0 00176 (/go/src/main.go:5) CALL
runtime.morestack_noctxt(SB)
[...]
0x0000 00000 (/go/src/main.go:13) TEXT "".max(SB),
NOSPLIT|ABIInternal, $0- 24There are two instructions that involve changing the stack size: -call Runtime. morestack_noctxt: This method increases the stack size when needed. -nosplit: This instruction means that stack overflow detection is not required, he and the instruction//go:nosplit.
Copy the codeMorestack_noctxt, which calls the newStack method in Runtime /stack.go:
func newstack(a){[...].// Allocate a bigger segment and move the stack.
oldsize := gp.stack.hi - gp.stack.lo
newsize := oldsize * 2
if newsize > maxstacksize {
print("runtime: goroutine stack exceeds ", maxstacksize, "-byte limit\n")
throw("stack overflow")}// The goroutine must be executing in order to call newstack,
// so it must be Grunning (or Gscanrunning).
casgstatus(gp, _Grunning, _Gcopystack)
// The concurrent GC will not scan the stack while we are doing the copy since
// the gp is in a Gcopystack status.
copystack(gp, newsize, true)
if stackDebug >= 1 {
print("stack grow done\n")
}
casgstatus(gp, _Gcopystack, _Grunning)
}
Copy the codeThe stack size is first calculated based on the boundaries of GP.stack. hi and GP.stack. lo, which are Pointers to the head and tail of the stack.
type stack struct {
lo uintptr
hi uintptr
}
Copy the codeThe stack size is then multiplied by two, if it does not reach the maximum — the maximum depends on the system architecture.
// Max stack size is 1 GB on 64-bit, 250 MB on 32-bit.
// Using decimal instead of binary GB and MB because
// they look nicer in the stack overflow failure message.
if sys.PtrSize == 8 {
maxstacksize = 1000000000
} else {
maxstacksize = 250000000
}
Copy the codeNow that we know how it works, let’s write a simple example to verify the above. To debug, we need to set the stackDebug constant, which prints some debug messages in the newStack method above, run:
func main(a) {
var x [10]int
a(x)
}
//go:noinline
func a(x [10]int) {
println(`func a`)
var y [100]int
b(y)
}
//go:noinline
func b(x [100]int) {
println(`func b`)
var y [1000]int
c(y)
}
//go:noinline
func c(x [1000]int) {
println(`func c`)}Copy the codeThe go:noinline directive is used to avoid putting all methods on one line at compile time. If we put them all on one line, we will not see the dynamic stack growth at the beginning of each method.
Here is part of the debug log:
runtime: newstack sp=0xc00002e6d8 stack=[0xc00002e000.0xc00002e800]
stack grow done
func a
runtime: newstack sp=0xc000076888 stack=[0xc000076000.0xc000077000]
stack grow done
runtime: newstack sp=0xc00003f888 stack=[0xc00003e000.0xc000040000]
stack grow done
runtime: newstack sp=0xc000081888 stack=[0xc00007e000.0xc000082000]
stack grow done
func b
runtime: newstack sp=0xc0000859f8 stack=[0xc000082000.0xc00008a000]
func c
Copy the codeWe can see that the stack grows four times. In fact, the method starts by growing the stack to the size it needs. As we can see in the code, the stack boundary defines the stack size, so we can calculate the size of each newstack — newstack stack=[…] The directive provides a pointer to the current stack boundary:
runtime: newstack sp=0xc00002e6d8 stack=[0xc00002e000.0xc00002e800]
0xc00002e800 - 0xc00002e000 = 2048
runtime: newstack sp=0xc000076888 stack=[0xc000076000.0xc000077000]
0xc000077000 - 0xc000076000 = 4096
runtime: newstack sp=0xc00003f888 stack=[0xc00003e000.0xc000040000]
0xc000040000 - 0xc00003e000 = 8192
runtime: newstack sp=0xc000081888 stack=[0xc00007e000.0xc000082000]
0xc000082000 - 0xc00007e000 = 16384
runtime: newstack sp=0xc0000859f8 stack=[0xc000082000.0xc00008a000]
0xc00008a000 - 0xc000082000 = 32768
Copy the codeWe can see that Goroutine’s stack space starts at 2Kb at compile time and grows to its desired size at the start of the function until it meets the operating condition or reaches the system limit.
Stack allocation management
The dynamic stack allocation system is not the only reason that affects our application. However, how the stack is allocated can also have a big impact on your application. Let’s try to understand how it manages the stack with two full log traces. Let’s try to see how Go does stack management from the previous two stack growth traces:
runtime: newstack sp=0xc00002e6d8 stack=[0xc00002e000.0xc00002e800]
copystack gp=0xc000000300 [0xc00002e000 0xc00002e6e0 0xc00002e800] - > [0xc000076000 0xc000076ee0 0xc000077000] /4096
stackfree 0xc00002e000 2048
stack grow done
runtime: newstack sp=0xc000076888 stack=[0xc000076000.0xc000077000]
copystack gp=0xc000000300 [0xc000076000 0xc000076890 0xc000077000] - > [0xc00003e000 0xc00003f890 0xc000040000] /8192
stackfree 0xc000076000 4096
stack grow done
Copy the codeThe first instruction displays the address of the current stack, stack=[0xC00002E000, 0xC00002E800], and copies it to the new stack, which is twice its previous size. Copystack [0xC00002E000 […] 0xC00002E800] -> [0xC000076000 […] 0xC000077000], the length of 4096 bytes is the same as we saw above. The previous stack is then released: stackFree 0xC00002E000. We have drawn a diagram to help us understand the above logic:
The Copystack instruction copies the entire stack and moves all addresses to the new stack. We can easily see this in a short piece of code:
func main(a) {
var x [10]int
println(&x)
a(x)
println(&x)
}
Copy the codeThe printed address is
0xc00002e738[...].0xc000089f38
Copy the codeThe address 0xC00002e738 is included in the stack address we saw earlier stack=[0xC00002E000, 0xC00002E800], The same address 0xC000089F38 is also contained in the later stack stack=[0xC000082000, 0xC00008A000]. These two stack addresses are traced in debug mode above. This also proves that all values have indeed been moved from the old stack to the new stack.
Also, interestingly, the stack shrinks when garbage collection is triggered.
In our example, after the function call, there are no valid function calls on the stack other than the main function, so the stack is reduced when garbage collection starts. To demonstrate this problem, we can enforce garbage collection:
func main(a) {
var x [10]int
println(&x)
a(x)
runtime.GC()
println(&x)
}
Copy the codeThe Debug program displays the stack reduction log:
func c
shrinking stack 32768->16384
copystack gp=0xc000000300 [0xc000082000 0xc000089e60 0xc00008a000] - > [0xc00007e000 0xc000081e60 0xc000082000] /16384
Copy the codeAs we can see, the stack size has been reduced to half its original size, and the stack address stack=[0xC00007e000, 0xC000082000] has been reused. Also in runtime/stack.go — shrinkstack(), The default reduction function is to divide the current stack size by 2:
oldsize := gp.stack.hi - gp.stack.lo
newsize := oldsize / 2
Copy the codeContinuous stack vs. segmented stack
The strategy of copying the stack into a larger stack space is called a contiguous stack, as opposed to the segmented stack. Go moved to a continuous stack strategy in version 1.3. To see the difference, we can run the same example in Go 1.2. Also, we need to modify the stackDebug variable to show the Debug trace information. For this reason, since the Go 1.2 Runtime is written in C, we had to recompile the source code. Here is the result of the example:
func a
runtime: newstack framesize=0x3e90 argsize=0x320 sp=0x7f8875953848 stack=[0x7f8875952000.0x7f8875953fa0] - >new stack [0xc21001d000.0xc210021950]
func b
func c
runtime: oldstack gobuf={pc:0x400cff sp:0x7f8875953858 lr:0x0} cret=0x1 argsize=0x320
Copy the codeThe current stack stack=[0x7F8875952000, 0x7F8875953FA0] is 8Kb (8192 bytes + the size of the top of the stack), while the new stack is created with 18864 bytes (18768 bytes + the size of the top of the stack). 0x7F8875953FA0-0x7F8875952000 is not 8Kb, but 8096 bytes.
The memory size allocation logic is as follows:
// allocate new segment.
framesize += argsize;
framesize += StackExtra; // room for more functions, Stktop.
if(framesize < StackMin)
framesize = StackMin;
framesize += StackSystem;
Copy the codeStackExtra = 2048, StackMin = 8192, and StackSystem = 0 to 512.
So our new stack includes: 16016 (frame size) + 800 (arguments) + 2048 (StackExtra) + 0 (StackSystem).
Once all functions have been called, the new stack is released (log Runtime: oldstack). This behavior is one of the things that forced the Golang team to move to a continuous stack:
The current segmented stack mechanism has a “hot split” problem — if the stack is nearly full, a function call causes a new stack block to be allocated. When all function > number calls return, the new stack block is also reclaimed. If the same invocation occurs repeatedly and intensively, allocation/reclamation can result in significant overhead.
Docs.google.com/document/d/…
Because of this problem, Go 1.2 increases the minimum stack size to 8Kb. The stack size was then reduced back to 2Kb due to the implementation of a continuous stack.
Below is an illustration of a segmented stack:
conclusion
Go’s stack management is very efficient and easy to understand. Golang isn’t the only language that doesn’t opt for a segmented stack; the Rust language doesn’t opt for this option for the same reason.
For a more in-depth look at the stack, read Dave Cheney’s blog post on Redzone, and Bill Kennedy’s post explaining frames in the stack.
Read more: medium.com/a-journey-w…
Read more:
contiguous stack
We have a contiguous stack in Go 1.3
StackExtra
StackMin
StackSystem
talks about the redzone
frames in the stack
Welcome to follow wechat official account:Back end morning reading class
