The introduction
In Go defer is often used to release resources, called before a function returns, and is often used to close file descriptors, close database connections, and unlock resources. Let’s dive into the Go language source code and look at the implementation of the defer keyword.
directory
- Defer the keyword
- Defer source analysis
- conclusion
- To learn more
1. Defer keyword
The defer statement in Go is typically used to do some cleanup or post-processing. The defer statements are executed in the order of the LIFO rules. Take a look at the code below and think about the values of the output when you use defer with return.
package main
import (
"fmt"
)
func main(a) {
fmt.Println("anonymousVarReturn return value is", anonymousVarReturn())
fmt.Println("anonymousVarReturn2 return value is", anonymousVarReturn2(1))
fmt.Println("anonymousVarReturn3 return value is", anonymousVarReturn3(1))
fmt.Println("namedVarReturn4 return value is", namedVarReturn4())
}
func anonymousVarReturn(a) int {
var i = 0
defer func(a) {
i++
fmt.Println("anonymousVarReturn defer, i is ", i)
}()
return 0
}
func anonymousVarReturn2(i int) int {
defer func(a) {
i++
fmt.Println("anonymousVarReturn2 defer, i is", i)
}()
return i
}
func anonymousVarReturn3(i int) int {
defer func(i int) {
i++
fmt.Println("anonymousVarReturn3 defer, i is", i)
}(i)
return i
}
func namedVarReturn4(a) (i int) {
defer func(i *int) {
*i++
fmt.Println("namedVarReturn4 defer, i is", *i)
}(&i)
return 1
}
Copy the codeExpand the output of relevant values below. The named return value has been modified by defer. Is it a little different from what you expected?
anonymousVarReturn defer, i is 1
anonymousVarReturn return value is 0
anonymousVarReturn2 defer, i is 2
anonymousVarReturn2 return value is 1
anonymousVarReturn3 defer, i is 2
anonymousVarReturn3 return value is 1
namedVarReturn4 defer, i is 2
namedVarReturn4 return value is 2
Copy the codeConfused, I searched the documents and found relevant instructions in go’s Defer, Panic, and Recover blogs.
The behavior of defer statements is straightforward and predictable. There are three simple rules: 1. A deferred function's arguments are evaluated when the defer statement is evaluated. 2. Deferred function calls are executed in Last In First Out order after the surrounding function returns. 3. Deferred functions may read and assign to the returning function's named return values.Copy the codeRule 3 illustrates the return problem of the code above. The defer method might read and assign to the returned named variable, so it can continue to operate on the same return variable after the return. If you look at the above methods anonymousVarReturn, anonymousVarReturn2, anonymousVarReturn3, and namedVarReturn4, you can see that the anonymous return value has not been modified by the defer method. Because the anonymous method return variable name is automatically created by Go, it will not be manipulated in defer.
2. Defer source analysis
Before the analysis of the source code need to understand Go assembly related knowledge, to understand the assembly content, need to read the following links to the document, read will have a deeper understanding of the assembly, and then look at the following assembly code will not be so confused.
Let’s briefly examine the assembly code associated with naming methods that return variables.
1 package main
2
3 import (
4 "fmt"
5 )
6
7 func main(a) {
8 fmt.Println("namedVarReturn4 return value is", namedVarReturn4())
9 }
10
11 func namedVarReturn4(a) (i int) {
12 defer func(i *int) {
13 *i++
14 fmt.Println("namedVarReturn4 defer, i is", *i)
15 }(&i)
16 return 1
17 }
Copy the codeUse the Go tool compile to output the relevant assembly code. First, see help to output the relevant command parameters.
go tool compile --help usage: compile [options] file.go... -n disable optimizations -s print assembly listing -l disable inlining...... Omit the partCopy the codeRun the following command to print the assembly code for the above code.
#An assembly instruction that outputs the entire code
go tool compile -N -l -S main.go
# go build -o test main.go
#Prints an assembly instruction for the main method
# go tool objdump -S -s main.main test
#Output the assembly instruction for the namedVarReturn4 method
# go tool objdump -S -s main.namedVarReturn4 test
Copy the codeHere is a snippet of the assembly instruction, looking at a few key lines of code.
0x0000 00000 (main.go:11) TEXT "".namedVarReturn4(SB), ABIInternal, $96- 8 -
0x0000 00000 (main.go:11) MOVQ (TLS), CX
0x0009 00009 (main.go:11) CMPQ SP, 16(CX)
0x000d 00013 (main.go:11) PCDATA $0, $2 -
0x000d 00013 (main.go:11) JLS 129
0x000f 00015 (main.go:11) PCDATA $0, $- 1
0x000f 00015 (main.go:11) SUBQ $96, SP
0x0013 00019 (main.go:11) MOVQ BP, 88(SP)
0x0018 00024 (main.go:11) LEAQ 88(SP), BP
0x001d 00029 (main.go:11) FUNCDATA $0, gclocals33cdeccccebe80329f1fdbee7f5874cb(SB)
0x001d 00029 (main.go:11) FUNCDATA $1, gclocals33cdeccccebe80329f1fdbee7f5874cb(SB)
0x001d 00029 (main.go:11) MOVQ $0."".i+104(SP)
0x0026 00038 (main.go:12) MOVL $8.""..autotmp_1+8(SP)
0x002e 00046 (main.go:12) LEAQ "". NamedVarReturn4. Func1 f. (SB), AX0x0035 00053 (main.go:12) MOVQ AX, ""..autotmp_1+32(SP)
0x003a 00058 (main.go:12) LEAQ "".i+104(SP), AX
0x003f 00063 (main.go:12) MOVQ AX, ""..autotmp_1+80(SP)
0x0044 00068 (main.go:12) LEAQ ""..autotmp_1+8(SP), AX
0x0049 00073 (main.go:12) MOVQ AX, (SP)
0x004d 00077 (main.go:12) PCDATA $1, $0
0x004d 00077 (main.go:12) CALL runtime.deferprocStack(SB) // the defer method is pushed
0x0052 00082 (main.go:15) TESTL AX, AX
0x0054 00084 (main.go:15) JNE 113
0x0056 00086 (main.go:15) JMP 88
0x0058 00088 (main.go:16) MOVQ $1."".i+104(SP) // Assign the I variable here
0x0061 00097 (main.go:16) XCHGL AX, AX
0x0062 00098 (main.go:16) CALL runtime.deferreturn(SB) // 这里执行 return
0x0067 00103 (main.go:16) MOVQ 88(SP), BP
0x006c 00108 (main.go:16) ADDQ $96, SP
0x0070 00112 (main.go:16) RET
0x0071 00113 (main.go:12) XCHGL AX, AX
0x0072 00114 (main.go:12) CALL runtime.deferreturn(SB)// Execute the defer method here
0x0077 00119 (main.go:15) MOVQ 88(SP), BP
0x007c 00124 (main.go:15) ADDQ $96, SP
0x0080 00128 (main.go:15) RET
0x0081 00129 (main.go:15) NOP
0x0081 00129 (main.go:11) PCDATA $1, $- 1
0x0081 00129 (main.go:11) PCDATA $0, $2 -
0x0081 00129 (main.go:11) CALL runtime.morestack_noctxt(SB)
0x0086 00134 (main.go:11) PCDATA $0, $- 1
0x0086 00134 (main.go:11) JMP 0... Omit the part"".namedVarReturn4.func1 STEXT size=298 args=0x8 locals=0x88 funcid=0x0
0x0000 00000 (main.go:12) TEXT "".namedVarReturn4.func1(SB), ABIInternal, $136- 8 -
0x0000 00000 (main.go:12) MOVQ (TLS), CX
0x0009 00009 (main.go:12) LEAQ - 8 -(SP), AX
0x000e 00014 (main.go:12) CMPQ AX, 16(CX)
0x0012 00018 (main.go:12) PCDATA $0, $2 -
0x0012 00018 (main.go:12) JLS 288
0x0018 00024 (main.go:12) PCDATA $0, $- 1
0x0018 00024 (main.go:12) SUBQ $136, SP
0x001f 00031 (main.go:12) MOVQ BP, 128(SP)
0x0027 00039 (main.go:12) LEAQ 128(SP), BP
0x002f 00047 (main.go:12) FUNCDATA $0, gclocals2d7c1615616d4cf40d01b3385155ed6e(SB)
0x002f 00047 (main.go:12) FUNCDATA $1, gclocals7985103f61d1dca6f16bfba926a2a610(SB)
0x002f 00047 (main.go:12) FUNCDATA $2."".namedVarReturn4.func1.stkobj(SB)
0x002f 00047 (main.go:13) MOVQ "".i+144(SP), AX
0x0037 00055 (main.go:13) TESTB AL, (AX)
0x0039 00057 (main.go:13) MOVQ "".i+144(SP), CX
0x0041 00065 (main.go:13) TESTB AL, (CX)
0x0043 00067 (main.go:13) MOVQ (AX), AX
0x0046 00070 (main.go:13) INCQ AX
0x0049 00073 (main.go:13) MOVQ AX, (CX)
0x004c 00076 (main.go:14) XORPS X0, X0
0x004f 00079 (main.go:14) MOVUPS X0, ""..autotmp_1+96(SP)
0x0054 00084 (main.go:14) MOVUPS X0, ""..autotmp_1+112(SP)
0x0059 00089 (main.go:14) LEAQ ""..autotmp_1+96(SP), AX
0x005e 00094 (main.go:14) MOVQ AX, ""..autotmp_3+64(SP)
0x0063 00099 (main.go:14) TESTB AL, (AX)
0x0065 00101 (main.go:14) LEAQ type.string(SB), AX
0x006c 00108 (main.go:14) MOVQ AX, ""..autotmp_1+96(SP)
0x0071 00113 (main.go:14) LEAQ ""..stmp_1(SB), AX
0x0078 00120 (main.go:14) MOVQ AX, ""..autotmp_1+104(SP)
0x007d 00125 (main.go:14) MOVQ "".i+144(SP), AX
0x0085 00133 (main.go:14) TESTB AL, (AX)
0x0087 00135 (main.go:14) MOVQ (AX), AX
0x008a 00138 (main.go:14) MOVQ AX, ""..autotmp_4+48(SP)
0x008f 00143 (main.go:14) MOVQ AX, (SP)
0x0093 00147 (main.go:14) PCDATA $1, $1
0x0093 00147 (main.go:14) CALL runtime.convT64(SB)
0x0098 00152 (main.go:14) MOVQ 8(SP), AX
0x009d 00157 (main.go:14) MOVQ AX, ""..autotmp_5+56(SP)
0x00a2 00162 (main.go:14) MOVQ ""..autotmp_3+64(SP), CX
0x00a7 00167 (main.go:14) TESTB AL, (CX)
0x00a9 00169 (main.go:14) LEAQ type.int(SB), DX
0x00b0 00176 (main.go:14) MOVQ DX, 16(CX)
0x00b4 00180 (main.go:14) LEAQ 24(CX), DI
0x00b8 00184 (main.go:14) PCDATA $0, $2 -
0x00b8 00184 (main.go:14) CMPL runtime.writeBarrier(SB), $0
0x00bf 00191 (main.go:14) NOP
0x00c0 00192 (main.go:14) JEQ 196
0x00c2 00194 (main.go:14) JMP 277
0x00c4 00196 (main.go:14) MOVQ AX, 24(CX)
0x00c8 00200 (main.go:14) JMP 202
0x00ca 00202 (main.go:14) PCDATA $0, $- 1
0x00ca 00202 (main.go:14) MOVQ ""..autotmp_3+64(SP), AX
0x00cf 00207 (main.go:14) TESTB AL, (AX)
0x00d1 00209 (main.go:14) JMP 211
0x00d3 00211 (main.go:14) MOVQ AX, ""..autotmp_2+72(SP)
0x00d8 00216 (main.go:14) MOVQ $2.""..autotmp_2+80(SP)
0x00e1 00225 (main.go:14) MOVQ $2.""..autotmp_2+88(SP)
0x00ea 00234 (main.go:14) MOVQ AX, (SP)
0x00ee 00238 (main.go:14) MOVQ $2.8(SP)
0x00f7 00247 (main.go:14) MOVQ $2.16(SP)
0x0100 00256 (main.go:14) PCDATA $1, $2
0x0100 00256 (main.go:14) CALL fmt.Println(SB)
0x0105 00261 (main.go:15) MOVQ 128(SP), BP
0x010d 00269 (main.go:15) ADDQ $136, SP
0x0114 00276 (main.go:15) RET
0x0115 00277 (main.go:14) PCDATA $0, $2 -
0x0115 00277 (main.go:14) CALL runtime.gcWriteBarrier(SB)
0x011a 00282 (main.go:14) JMP 202
0x011c 00284 (main.go:14) NOP
0x011c 00284 (main.go:12) PCDATA $1, $- 1
0x011c 00284 (main.go:12) PCDATA $0, $2 -
0x011c 00284 (main.go:12) NOP
0x0120 00288 (main.go:12) CALL runtime.morestack_noctxt(SB)
0x0125 00293 (main.go:12) PCDATA $0, $- 1
0x0125 00293 (main.go:12) JMP 0
Copy the codeAs you can see from the assembly instructions above, the process first calls deferprocStack to create defer, and then inserts the CALL Runtime.DeferReturn (SB) directive when the function returns (directive JMP 0).
Now that you know that defer is called in the flow through these two methods, and now that you know the name of the call, you can simply copy the Go source code and search globally for deferprocStack.
Here is the source code for deferprocStack
// src/runtime/panic.go
// deferprocStack queues a new deferred function with a defer record on the stack.
// The defer record must have its siz and fn fields initialized.
// All other fields can contain junk.
// The defer record must be immediately followed in memory by
// the arguments of the defer.
// Nosplit because the arguments on the stack won't be scanned
// until the defer record is spliced into the gp._defer list.
//go:nosplit
func deferprocStack(d *_defer) {
gp := getg()
ifgp.m.curg ! = gp {// go code on the system stack can't defer
throw("defer on system stack")}// siz and fn are already set.
// The other fields are junk on entry to deferprocStack and
// are initialized here.
d.started = false
d.heap = false
d.openDefer = false
d.sp = getcallersp()
d.pc = getcallerpc()
d.framepc = 0
d.varp = 0
// The lines below implement:
// d.panic = nil
// d.fd = nil
// d.link = gp._defer
// gp._defer = d
// But without write barriers. The first three are writes to
// the stack so they don't need a write barrier, and furthermore
// are to uninitialized memory, so they must not use a write barrier.
// The fourth write does not require a write barrier because we
// explicitly mark all the defer structures, so we don't need to
// keep track of pointers to them with a write barrier.* (*uintptr)(unsafe.Pointer(&d._panic)) = 0* (*uintptr)(unsafe.Pointer(&d.fd)) = 0* (*uintptr)(unsafe.Pointer(&d.link)) = uintptr(unsafe.Pointer(gp._defer))
*(*uintptr)(unsafe.Pointer(&gp._defer)) = uintptr(unsafe.Pointer(d))
return0()
// No code can go here - the C return register has
// been set and must not be clobbered.
}
Copy the codeHere is the source code for _defer passed in by the deferprocStack method
// src/runtime/runtime2.go
// A _defer holds an entry on the list of deferred calls.
// If you add a field here, add code to clear it in freedefer and deferProcStack
// This struct must match the code in cmd/compile/internal/gc/reflect.go:deferstruct
// and cmd/compile/internal/gc/ssa.go:(*state).call.
// Some defers will be allocated on the stack and some on the heap.
// All defers are logically part of the stack, so write barriers to
// initialize them are not required. All defers must be manually scanned,
// and for heap defers, marked.
type _defer struct {
siz int32 // includes both arguments and results
started bool
heap bool
// openDefer indicates that this _defer is for a frame with open-coded
// defers. We have only one defer record for the entire frame (which may
// currently have 0, 1, or more defers active).
openDefer bool
sp uintptr // sp at time of defer
pc uintptr // pc at time of defer
fn *funcval // can be nil for open-coded defers
_panic *_panic // panic that is running defer
link *_defer
// If openDefer is true, the fields below record values about the stack
// frame and associated function that has the open-coded defer(s). sp
// above will be the sp for the frame, and pc will be address of the
// deferreturn call in the function.
fd unsafe.Pointer // funcdata for the function associated with the frame
varp uintptr // value of varp for the stack frame
// framepc is the current pc associated with the stack frame. Together,
// with sp above (which is the sp associated with the stack frame),
// framepc/sp can be used as pc/sp pair to continue a stack trace via
// gentraceback().
framepc uintptr
}
Copy the codeHere is the source code for DeferReturn
// src/runtime/runtime2.go
// Run a deferred function if there is one.
// The compiler inserts a call to this at the end of any
// function which calls defer.
// If there is a deferred function, this will call Runtime ·jmpdefer,
// which will jump to the deferred function such that it appears
// to have been called by the caller of deferreturn at the point
// just before deferreturn was called. The effect is that deferreturn
// is called again and again until there are no more deferred functions.
//
// Declared as nosplit, because the function should not be preempted once we start
// modifying the caller's frame in order to reuse the frame to call the deferred
// function.
//
// The single argument isn't actually used - it just has its address
// taken so it can be matched against pending defers.
//go:nosplit
func deferreturn(arg0 uintptr) {
gp := getg()
d := gp._defer
if d == nil {
return
}
sp := getcallersp()
ifd.sp ! = sp {return
}
if d.openDefer {
done := runOpenDeferFrame(gp, d)
if! done { throw("unfinished open-coded defers in deferreturn")
}
gp._defer = d.link
freedefer(d)
return
}
// Moving arguments around.
//
// Everything called after this point must be recursively
// nosplit because the garbage collector won't know the form
// of the arguments until the jmpdefer can flip the PC over to
// fn.
switch d.siz {
case 0:
// Do nothing.
case sys.PtrSize:
*(*uintptr)(unsafe.Pointer(&arg0)) = *(*uintptr)(deferArgs(d))
default:
memmove(unsafe.Pointer(&arg0), deferArgs(d), uintptr(d.siz))
}
fn := d.fn
d.fn = nil
gp._defer = d.link
freedefer(d)
// If the defer function pointer is nil, force the seg fault to happen
// here rather than in jmpdefer. gentraceback() throws an error if it is
// called with a callback on an LR architecture and jmpdefer is on the
// stack, because the stack trace can be incorrect in that case - see
// issue #8153).
_ = fn.fn
jmpdefer(fn, uintptr(unsafe.Pointer(&arg0)))
}
Copy the codeThe jMPdefer method was last called in the deferReturn method above. Since this machine is 64-bit, it corresponds to the ASM_AMd64.s assembly directive. Here is the implementation of JMPdefer
//src/runtime/asm_amd64.s
// func jmpdefer(fv *funcval, argp uintptr)
// argp is a caller SP.
// called from deferreturn.
// 1. pop the caller
// 2. sub 5 bytes from the callers return
// 3. jmp to the argumentThe TEXT, the runtime jmpdefer (SB), NOSPLIT, $0- 16
MOVQ fv+0(FP), DX // fn
MOVQ argp+8(FP), BX // caller sp
LEAQ - 8 -(BX), SP // caller sp after CALL
MOVQ - 8 -(SP), BP // restore BP as if deferreturn returned (harmless if framepointers not in use)
SUBQ $5, (SP) // return to CALL again
MOVQ 0(DX), BX
JMP BX // but first run the deferred function
Copy the codeYou can get a sense of what this means with the source notes and related links to the documentation below. Interested can go to the relevant knowledge to further understanding.
3, summarize
Finally, I will summarize the process of defer.
- The compiler translates the defer statement as a call to the deferprocStack function.
- The deferprocStack function puts the _defer structure object into the _defer linked list of the current Goroutine.
- The compiler inserts a deferReturn call at the end of the function where defer is located, which recursively calls the function where the defer statement is located.
- Defer, Panic, and Recover
- Plan9 – Assembly – Full resolution
- A Quick Guide to Go’s Assembler
- Chapter I: Go Assembly
- Golang assembly
- The Go
- A Manual for the Plan 9 assembler
- Chinese version -Plan9 assembler manual
4. Learn more
Defer mechanism analysis in Golang wechat official account: DigitMagic Magic Number Laboratory