What was the implementation of defer from the previous article golang? This is an extension of the last one, mainly introducing the low-level analysis of Panic and Recover. If you haven’t read the last one, you can read this one first. There are three parts to explain:
1 panic
2 defer panic
3 defer panic recover
Environment: Go Version GO1.12.5 Linux/AMD64
1 panic
Golang is divided into four categories:
- Captured by the compiler
- Direct manual panic
- Golang captured
- System captured
Captured by the compiler
/main.go:7:8: division by zero./main.go:7:8: division by zero
Direct manual panic
Sample code:
package main
func main() {
panic("Panic Error!!)}Copy the codeGo tool compile -s main.go > main.s
0x0034 00052 (main.go:4) CALL runtime.gopanic(SB)
Copy the codeLooking at the Gopanic (SB) implementation, first take a cursory look at what the code means. Some explanations are annotated in the code:
Func gopanic(e interface{}) {gp := getg() // Getg.... Var p_panic //_panic prototype p.arg = e // Save the panic parameter to the arg parameter p.link = gp._panic // bind p.link to the _panic of the current g. Gp._panic = (*_panic)(noescape(unsafe.Pointer(&p)))) // bind p to the header of g. atomic.Xadd(&runningPanicDefers, 1)for {
d := gp._defer
if d == nil {
break
}
if d.started {
ifd._panic ! = nil { d._panic.aborted =true
}
d._panic = nil
d.fn = nil
gp._defer = d.link
freedefer(d)
continue
}
d.started = trueD. _panic = (*_panic)(noescape(unsafe.Pointer(&p)))// bind p to the header of g. p.argp = unsafe.Pointer(getargp(0)) reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), Uint32 (d.iz)) // Call defer on g (if there is no defer function in the source program, the compiler generates one and binds it to G._defer) p.argp = nilifgp._defer ! = d { throw("bad defer entry in panic"// unchain d._panic = nil d.f = nil gp._defer = d.link PC := d.spc := unsafe.Pointer(d.sp) // must be Pointer so it is unsafe gets adjusted during stack copy freedefer(d)ifP.reovered {// ignore..... when talking about recover }} preprintpanics(gp._panic) // Loop to printpanic fatalpanic(gp._panic) // should notreturn
*(*int)(nil) = 0 // not reached
}
Copy the codeWe found that the prototype for panic is _panic. Let’s look at the definition:
type_panic struct { argp unsafe.Pointer // pointer to arguments of deferred call run during panic; cannot move - known to liblink arg interface{} // argument to panic link *_panic // link to earlier panic recovered bool // whether this panic is over aborted bool // the panic was aborted }Copy the codeFound is a structure type, inside the type we debug the code to explore the specific meaning. Let’s follow up the above source code example with GDB.
go build -o main gdb main
Enter the GDB interface and break to the panic function line as shown below:
gp._panic = (*_panic)(noescape(unsafe.Pointer(&p)))
Copy the codeIt turns out that the current GP definition has the _panc field as the linked header, and the _panic structure has the link field. It’s not hard to see how defer works: a linked list data structure is made up from goroutine._panic as a header and then _painc.link as a link. The reason why it is a linked list is because there may be panic in recover when it is recovered, for example, see the following code:
iferr := recover(); err ! = nil { panic("go on panic xitehip")}Copy the codeDeferd functions will also continue to have Panic. We’ll talk about recover in detail below. Gp._panic => p.link => Gp._panic (previous list header)
func reflectcall(argtype *_type, fn, arg unsafe.Pointer, argsize uint32, retoffset uint32)
Copy the codeThe unsafe.point (d.fin) argument is used to call the reflectCall. According to the definition of d := gp._defer in the source code, we know that the variable D is g._defer as mentioned above. In this example we didn’t use the defer keyword at all, so we didn’t call deferProc (SB) to generate defer. The only possibility is that the compiler generated a defer function for us and bound it to the header of g._defer. Continuing with the reflectCall function, see figure X below:
0x0000000000423025
/ / the runtime/asm_amd64 s TEXT, reflectcall (SB), NOSPLIT,$0-32 MOVLQZX argsize+24(FP), CX DISPATCH(Runtime ·call32, 32) DISPATCH(Runtime ·call64, 64)..... MOVQ$runtime· badreflectcall (SB), AX JMP AXCopy the code//runtime/asm_amd64.s
#define DISPATCH(NAME,MAXSIZE) \
CMPQ CX, $MAXSIZE; \
JA 3(PC); \
MOVQ $NAME(SB), AX; \
JMP AX
Copy the code//runtime/asm_amd64.s
#define CALLFN(NAME,MAXSIZE) \
TEXT NAME(SB), WRAPPER, $MAXSIZE- 32; \ NO_LOCAL_POINTERS; \ /* copy arguments to stack */ \ MOVQ argptr+16(FP), SI; \ MOVLQZX argsize+24(FP), CX; \ MOVQ SP, DI; \ REP; MOVSB; \ /* callfunction */ \
MOVQ f+8(FP), DX; \
PCDATA $PCDATA_StackMapIndex.$0; \
CALL (DX); \
/* copy return values back */ \
MOVQ argtype+0(FP), DX; \
MOVQ argptr+16(FP), DI; \
MOVLQZX argsize+24(FP), CX; \
MOVLQZX retoffset+28(FP), BX; \
MOVQ SP, SI; \
ADDQ BX, DI; \
ADDQ BX, SI; \
SUBQ BX, CX; \
CALL callRet<>(SB); \
RET
Copy the codeIs it a mess? What are these? Use GDB to trace to: Run to the following image:
TEXT callRet<>(SB), NOSPLIT, $320Copy the codeThe red box in RET is the return of the reflectCall. Hit the break point to ret. See the following figure:
0x423025
p.argp = nil
mov QWROD PTR [rsp+0x58],0x0
d._panic = nil
d.fn = nil
gp._defer = d.link
Copy the codeTo:
func fatalpanic(msgs *_panic)
Copy the codePrint it out and look at the implementation:
func fatalpanic(msgs *_panic) {
pc := getcallerpc()
sp := getcallersp()
gp := getg()
var docrash bool
systemstack(func() {
ifstartpanic_m() && msgs ! = nil { atomic.Xadd(&runningPanicDefers, -1) printpanics(msgs) } docrash = dopanic_m(gp, pc, sp) })if docrash {
crash()
}
systemstack(func() {
exit(2)
})
*(*int)(nil) = 0 // not reached
}
Copy the codeFocus on the following functions:
printpanics(msgs)
Copy the codeImplementation:
func printpanics(p *_panic) {
ifp.link ! = nil { printpanics(p.link)print("\t")}print("panic: ")
printany(p.arg)
if p.recovered {
print(" [recovered]")}print("\n")}Copy the codeIt turns out that this is a recursive call, starting at the head of the G. _panic list until the end of the list and then printing the panic message.
Golang captured
For example slice out of bounds, see the code below:
package main
import "fmt"
func main() {
arr := []int{1, 2}
arr[2] = 3
fmt.Println(arr)
}
Copy the codePanic: panic: Runtime error: index out of range compiles to assembly code: go tool compile -s main.go > main.s
0x003c 00060 (main.go:7) CALL runtime.panicindex(SB)
Copy the codeCall panicIndex (SB) to see its implementation:
func panicindex() {
if hasPrefix(funcname(findfunc(getcallerpc())), "runtime.") {
throw(string(indexError.(errorString)))
}
panicCheckMalloc(indexError)
panic(indexError)
}
Copy the codeFinally, panic(interface{}) is called, and then the manual panic execution process mentioned above is not repeated here.
System captured
For example, an assignment to a read-only memory area can cause panic
package main
import "fmt"
func main() {
var pi *int
*pi = 100
fmt.Printf("%v", *pi)
}
Copy the codePanic: Runtime error: invalid memory address or nil dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x488a53] goroutine 1 [running]: main.main() /server/gotest/src/hello/defer/main.go:7 +0x3a
Compilation into assembly code found no Gopanic entry. Since the final output of the panic stack is gopanic, it must have called gopanic, put a breakpoint on gopanic() and run directly to here, as shown below:
*pi = 100
In the red line: test BYTE PTR [ax], al Ax =0x0 When executing this statement, the CPU will enter the kernel state and save 0x488B1A into the register. The kernel state will send a message to the GO process. The GO processing function will change the content pointed by 0x488B1A into the function with the prior registration number when GO starts as the instruction entry, and return to the kernel state and execute the instruction of 0x488B1A -> registration function. I’m not going to go into the exact call chain here but panic, recover.
2 defer panic
2.1 the sample:
package main
import "fmt"
func main() {
defer fmt.Println("d1")
defer fmt.Println("d2")
panic("panic error")}Copy the codeD2 d1 Panic error
//runtime/panic.go
func gopanic(e interface{}) {
for{... // Execute the table header's DEFERd reflectCall (nil, unsafe.pointer (d.foon), deferArgs(d), uint32(d.iz), uint32(d.siz)) ... // Attach the next deferD to the table header}... Fatalpanic (gp._panic) // Run a recursive call to gP._panic on the panic list... }Copy the codeAs you can see from the code above, Gopanic first traverses the Deferd chain before the Panic chain, so panic Error is printed last.
2.2 the sample:
Line 14 Panic -> gopanic() -> ReflectCall -> line 12 defer -> reflectCall -> Line 8 defer -> Line 9 Panic -> gopanic -> ReflectCall -> Continue on the deferd chain at line 6 to defer -> fatalpanic(printpanics() recursively calls the G. _panic chain).
3 defer panic recover
Here is the recover execution process, first take a look at the following sample code:
package main
import "fmt"
func main() {
re()
fmt.Println("After recovery!")
}
func re() {
defer func() {
iferr := recover(); err ! = nil { fmt.Println("err:", err)
}
}()
panic("panic error1")}Copy the codeErr: Panic error1 After recovery!
Recover () catches exceptions and allows the program to proceed without exiting. In this example, there is an exception in re() and it is caught and the code below re() is executed to print ‘After recovery’.
Why would recover() jump to the output line?
From the assembly point of view: after executing re(), to ensure the continuation of the execution, first save the entry address of the next line, and then recover() and then retrieve it, and put it in the RIP instruction register so that the execution can proceed.
In re (a) in addition to deferd function has a panic () this line, it is obvious in its internal implementation will have related implementation, continue to analyze the realization of the recover and related implementation within the panic.
Gorecover (SB) : gorecover(SB) : gorecover(SB) : gorecover(SB) :
0x002a 00042 (main.go:13) CALL runtime.gorecover(SB)
Copy the codeAt the recover() interrupt point, we can see that goRecover (SB) is executed as follows:
func gorecover(argp uintptr) interface{} {
gp := getg()
p := gp._panic
ifp ! = nil && ! p.recovered && argp == uintptr(p.argp) { p.recovered =true
return p.arg
}
return nil
}
Copy the codeVar var var var var var var var var var var var var var var var var var var var var var var var var var var var var var var var var var var var In effect, it sets the current G. _panic flag to tell future programs that I have been captured.
The “recover()” out-of-place function returns reflectCall (nil, unsafe.pointer (d.fin), out-of-place (d) in the gopanic(interface{}) function mentioned above, Uint32 (d.iz), uint32(d.iz)) Execution continues on the next line. See the code below:
func gopanic(e interfac{}) { ....... Reflectcall (nil, unsafe.pointer (d.fish), deferArgs(d), uint32(D.fish), uint32(d.fish)) // Look down: p.argp = nilifgp._defer ! = d { throw("bad defer entry in panic"} // After defered, d._panic = nil d.f = nil gp._defer = D.link PC := d.pc //deferproc() sp := unsafe.Pointer(d.sp) // freedefer(d)ifP.reovered {// after the gorecover() function p.reovered ==true
atomic.Xadd(&runningPanicDefers, -1)
gp._panic = p.link
forgp._panic ! = nil && gp._panic.aborted { gp._panic = gp._panic.link }if gp._panic == nil { // must be doneSigcode0 = uintptr(sp) gp.sigcode1 = PC // PC restore function of stack. mcall(recovery) throw("recovery failed") // mcall should not return}... }Copy the codeTake a look at this line:
pc := d.pc
Copy the codeWhat is a PC? It is stored in the deferproc() function mentioned in the previous article, as shown in the code below:
func deferproc(siz int32, fn *funcval) { // arguments of fn follow fn
...
callerpc := getcallerpc()
d := newdefer(siz)
ifd._panic ! = nil { throw("deferproc: d.panic ! = nil after newdefer")
}
d.fn = fn
d.pc = callerpc
....
Copy the codeLet’s make a breakpoint at line 12 of the screenshot below to see what’s going on in the PC. Take a look at the instructions in the green box:
test eax, eax
0x4872d5
D. PC = callerpc
0x4872d5
test eax, eax
Gorecover (uintptr) == true
gp.sigcode1 = pc
Copy the codeAssign the return value of PC, which is the deferProc () function, to GP.sigcode1 in preparation for returning to normal flow.
mcall(recovery)
Copy the codeThe recovery function is implemented as follows:
func recovery(gp *g) {
// Info about defer passed in G struct.
sp := gp.sigcode0
pc := gp.sigcode1
// d's arguments need to be in the stack. if sp ! = 0 && (sp < gp.stack.lo || gp.stack.hi < sp) { print("recover: ", hex(sp), " not in [", hex(gp.stack.lo), ", ", hex(gp.stack.hi), "]\n") throw("bad recovery") } // Make the deferproc for this d return again, // this time returning 1. The calling function will // jump to the standard return epilogue. gp.sched.sp = sp gp.sched.pc = pc gp.sched.lr = 0 gp.sched.ret = 1 gogo(&gp.sched) }Copy the codeRecovery (*g) is mainly gP.sched assigned. Where PC is the return address of the current DeferProc function. Let’s take a look at the gogo(&gp.sched) function implementation. Since the gogo function is implemented in assembly, it is most convenient to use GDB trace as shown in the following code:
The TEXT runtime, gogo (SB), NOSPLIT,$16-8 MOVQ buf+0(FP), BX // gobuf MOVQ gobuf_g(BX), DX MOVQ 0(DX), CX // make sure g ! = nil get_tls(CX) MOVQ DX, g(CX) MOVQ gobuf_sp(BX), SP // restore SP MOVQ gobuf_ret(BX), AX MOVQ gobuf_ctxt(BX), DX MOVQ gobuf_bp(BX), BP MOVQ$0, gobuf_sp(BX) // clear to help garbage collector
MOVQ $0, gobuf_ret(BX)
MOVQ $0, gobuf_ctxt(BX)
MOVQ $0, gobuf_bp(BX)
MOVQ gobuf_pc(BX), BX
JMP BX
Copy the codeFocus on two lines of code:
MOVQ gobuf_ret(BX), AX
Copy the codeAX changes from a value to 1. The number of offsets in this instruction is gobuf_ret, where ret means return.
Let’s look at the last instruction:
JMP BX
Copy the codeTake a look at what BX really is:
0x4872d5
test eax,eax
test eax, eax
jne 0x4872f9
Copy the codeIt means if eax is not equal to 0 jump to this address otherwise go to the normal flow of the third line in the green box. Since eax is not equal to 0, it jumps to 0x4872f9 and tracks where the address points to, as shown below:
runtime.deferreturn()
Sp := getCallersp () sp is the sp of the caller. That is sp when you are about to call defer func() {. D.sp is the second defer on the call chain because the first DEFERd has been unchained. Obviously, these two are not equal, so return, exactly how the bottom layer of return returns the return address of re() is not traceable. Then execute to the lower entry address:
fmt.Println("After recovery!")
Copy the codeSee the code below to explain the whole process:
Deferproc () is stored in PC, the caller (re()) is stored in SP, and the defered function is added to the header of the list. Return0 (the return0 function sets ax to 0) =>go back to the code below test eax eax => since ax=0 continues to line 17 panic() =>gopanic() => call reflectcall(): execute the deferd function => Recovery (): Set recoverd flag to 1 => McAll (recovery) => GOGO (): set ax to 1. => Test eax: since ax=1 => jump to the deferReturn () function :callersp! = D.sp, d in d.sp here is the default _defer on G, so not wait => return get the return address of re() pop to rip => CPU executes its return value => output ‘After recovery’
. //defer function => deferProc 0x00000000004872d0 <+48>: call 0x426C00 < Runtime.deferProc > 0x00000000004872d5 <+53>:test eax,eax
0x00000000004872d7 <+55>: jne 0x4872f9 <main.re+89>
0x00000000004872d9 <+57>: jmp 0x4872db <main.re+59>
0x00000000004872db <+59>: lea rax,[rip+0x111be] # 0x4984a0
0x00000000004872e2 <+66>: mov QWORD PTR [rsp],rax
0x00000000004872e6 <+70>: lea rax,[rip+0x48643]
0x00000000004872ed <+77>: mov QWORD PTR [rsp+0x8],rax
//panic() => gopanic
0x00000000004872f2 <+82>: call 0x427880 <runtime.gopanic>
...
Copy the codeAt the heart of recover() are the assembly instructions generated by the defer function: the jump that distinguishes between normal flow and return flow. See assembly code above. The machine instructions are executed from the top down, and the normal process is to execute gopanic() generated by Panic () after deferProc. The process of getting the return value has to jump somewhere to get it, and golang’s designers put it in the deferReturn () function so they ended up there.
Leave a question how does the following code output and why?
package main
import "fmt"
func main() {
re()
fmt.Println("After recovery!")
}
func re() {
defer func() {
iferr := recover(); err ! = nil { fmt.Println("Recover again:", err)
}
}()
defer func() {
iferr := recover(); err ! = nil { switch v := err.(type) {
case string:
panic(string(v))
}
}
}()
panic("start panic")}Copy the codeReference: Go language panic/recover implementation