1. The description of the keyword defer
Defer is a deferred function. Each additional defer function pushes the corresponding _defer structure, and when executed, the pushback defer is executed first
1. There are three main scenarios that defer triggers
A “defer” statement invokes a function whose execution is deferred to the moment the surrounding function returns, either because the surrounding function executed a return statement, reached the end of its function body, or because the corresponding goroutine is panicking
1) When the main function return returns, before return is executed
2) When the execution of the main function ends
3) A panic has occurred in Goroutine
2. There are also three official rules for Defer
1) The parameters of the delay function were determined as soon as the defer statement appeared. This is because the required parameters were copied onto the stack when the defer statement appeared, and the main function changed the parameters later without affecting the values in defer
func testDerfer(a) {
i := 5
defer fmt.Println("defer",i)
i++
fmt.Println(i)
}
/* 6 defer 5 */
Copy the codeBut in the case of Pointers, we can see what the difference is:
func testDerfer(a) {
var i int
defer test(&i) // Copy the value address
i++
fmt.Println(&i,i)
}
func test(a *int) {
fmt.Println("defer",a,*a)
}
/*
0xc0000140e0 1
defer 0xc0000140e0 1
*/
Copy the code2) Deferred function execution is executed in last in first out order, i.e. Defer comes first and executes last
3) Deferred functions can operate on named return values of the main function
func testDerfer(a) (result int) {
i := 1
defer func(a) {
result++
}()
return i
}
// testDerfer returns 2
Copy the codeSecond, source code analysis
You can start by taking a look at the _defer structure fields, which are fields from version 1.2 without special version tags
type _defer struct {
siz int32 // includes both arguments and results
// Record how much space is occupied by the defer participation and the return value. This space is allocated directly after the defer structure to hold the parameters at registration and copy them to the caller parameters and return value space at execution
started bool // mark whether defer has been executed
heap bool // 1.13 Whether heap allocation is performed
openDefer bool // 1.14 indicates whether current defer is optimized for open coding
sp uintptr // Caller stack pointer, which allows the function to determine whether its registered defer has completed
pc uintptr // Deferporc return address
fn *funcval // Can be nil for open-coded defers registered defer function
_panic *_panic // Panic that is running defer is the structure that triggers the deferred call and may be empty;
link *_defer // Link to the previous defer structure to connect multiple defers
The following three main things are to find the unregistered defer functions and execute them in the correct order.
fd unsafe.Pointer // funcdata for the function associated with the frame
varp uintptr // value of varp for the stack frame
// framepc is the current pc associated with the stack frame. Together,
// with sp above (which is the sp associated with the stack frame),
// framepc/sp can be used as pc/sp pair to continue a stack trace via
// gentraceback().
framepc uintptr / / 1.14
}
Copy the codeLooking at the assembly code, you can see that in version 1.14, defer was handled in three ways
func (s *state) stmt(n *Node){...case ODEFER:
if Debug_defer > 0 {
var defertype string
if s.hasOpenDefers {
defertype = "open-coded"
} else if n.Esc == EscNever {
defertype = "stack-allocated"
} else {
defertype = "heap-allocated"
}
Warnl(n.Pos, "%s defer", defertype)
}
if s.hasOpenDefers { // Open source
s.openDeferRecord(n.Left)
} else {
d := callDefer / / heap allocation
if n.Esc == EscNever {
d = callDeferStack / / stack allocation
}
s.call(n.Left, d)
}
...
}
Copy the codeHeap allocation was used in version 1.12, stack allocation was added in version 1.13, and now open source has been added in version 1.14, so you can see heap allocation is the last resort. Now we’ll look at each of them.
1. Heap Allocation (Go 1.12)
- Register the defer handler with deferproc() when you declare the defer keyword and copy the corresponding _defer structure value to the heap.
The newly created defer structure is added to the header of the defer list, and the Goroutine’s defer pointer points to the header of the list, which is why defer is executed in reverse order.
// The registered function type is funcval, which is what defer executes
func deferproc(siz int32, fn *funcval){...// Insert the header
d.link = gp._defer
gp._defer = d
...
}
Copy the codeA funcval (closure) that does not capture a list (no variables used by external functions) is optimized at compile time to allocate a common funcval structure to the read-only data segment. But when funcval with the capture list allocates the size of the funcval structure in the heap, the variable used by the capture list will be stored in the heap, and the address of the variable used elsewhere will be used, so when the value changes, the reference will change at the same time
The main process is to get a SIz-sized _defer structure, which can be retrieved from the defer pool of the corresponding processor of the coroutine. If not, you need to allocate a siz-sized structure on the heap and place it in the head of the linked list.
- When the RET directive returns, insert deferReturn () and pull the head of the deferred list out for execution. When executed, the value is copied from the heap to the stack.
func deferreturn(arg0 uintptr){...// Retrieve the header
fn := d.fn
d.fn = nil
gp._defer = d.link
...
}
Copy the codeAnalyze the shortcomings of go1.12 defer
-
The _defer structure is heap allocation, and even if there is a pre-allocated defer pool, it needs to be fetched and released on the heap, with parameters copied back and forth across the stack
-
Register the defer information with the linked list, which itself is slow
2. Stack Allocation (Go 1.13)
- Register the defer handler with deferProc () when you declare the defer keyword, and register the defer structure on the stack into the defer linked list.
func deferprocStack(d *_defer) {
// Add variables and save the defer information to the local variable (d) area of the current function stack
gp := getg()
ifgp.m.curg ! = gp { throw("defer on system stack")
}
d.started = false
d.heap = false // stack allocation
d.openDefer = false
d.sp = getcallersp()
d.pc = getcallerpc()
d.framepc = 0
d.varp = 0* (*uintptr)(unsafe.Pointer(&d._panic)) = 0* (*uintptr)(unsafe.Pointer(&d.fd)) = 0* (*uintptr)(unsafe.Pointer(&d.link)) = uintptr(unsafe.Pointer(gp._defer))
*(*uintptr)(unsafe.Pointer(&gp._defer)) = uintptr(unsafe.Pointer(d))
return0()
}
Copy the code- Defer execution is still implemented through deferReturn, and the parameters are copied when the defer function executes, but not between stacks, but from local variable Spaces on the stack to parameter Spaces on the stack, improving performance by 30%
So the optimization point of 1.3: mainly reduce the heap allocation of defer information. However, defer1.2 is still used for both explicit and implicit defer processing (such as for loops)
3. Open coding (Go 1.14)
1) The use of open coding is conditional, and open source will only be enabled if the following three conditions are met
func walkstmt(n *Node) *Node {
case ODEFER:
Curfn.Func.SetHasDefer(true)
Curfn.Func.numDefers++
// The number of functions defer is less than or equal to eight
if Curfn.Func.numDefers > maxOpenDefers {
// Don't allow open-coded defers if there are more than
// 8 defers in the function, since we use a single
// byte to record active defers.
Curfn.Func.SetOpenCodedDeferDisallowed(true)}ifn.Esc ! = EscNever {// 2. The function defer keyword cannot be executed in the loop
// If n.Esc is not EscNever, then this defer occurs in a loop,
// so open-coded defers cannot be used in this function.
Curfn.Func.SetOpenCodedDeferDisallowed(true)}fallthrough
}
func buildssa(fn *Node, worker int) *ssa.Func {
if s.hasOpenDefers &&
// 3. The product of the function's 'return' statement and 'defer' statement is less than or equal to 15
s.curfn.Func.numReturns*s.curfn.Func.numDefers > 15 {
// Since we are generating defer calls at every exit for
// open-coded defers, skip doing open-coded defers if there are
// too many returns (especially if there are multiple defers).
// Open-coded defers are most important for improving performance
// for smaller functions (which don't have many returns).
s.hasOpenDefers = false}}Copy the codeDeferred bits and deferred records are the main constructs of open source:
func buildssa(fn *Node, worker int) *ssa.Func {
if s.hasOpenDefers {
// Create the deferBits variable and stack slot. deferBits is a
// bitmask showing which of the open-coded defers in this function
// have been activated.
// Initialize delay bits, 8 bits in total
deferBitsTemp := tempAt(src.NoXPos, s.curfn, types.Types[TUINT8])
s.deferBitsTemp = deferBitsTemp
// For this value, AuxInt is initialized to zero by default
startDeferBits := s.entryNewValue0(ssa.OpConst8, types.Types[TUINT8])
s.vars[&deferBitsVar] = startDeferBits
s.deferBitsAddr = s.addr(deferBitsTemp, false)
s.store(types.Types[TUINT8], s.deferBitsAddr, startDeferBits)
// Make sure that the deferBits stack slot is kept alive (for use
// by panics) and stores to deferBits are not eliminated, even if
// all checking code on deferBits in the function exit can be
// eliminated, because the defer statements were all
// unconditional.
s.vars[&memVar] = s.newValue1Apos(ssa.OpVarLive, types.TypeMem, deferBitsTemp, s.mem(), false)}}Copy the codeEach bit in the delay bit represents the corresponding bitdeferWhether the keyword needs to be executed. For example, if the last bit below is 1, it means that the defer bit will be executed. By using or operation, the first position of DF is 1, indicating that the bit can be executed. You can just judge during the execution and set it to 0 after judgment to avoid repeated execution
func (s *state) stmt(n *Node){...case ODEFER:
if Debug_defer > 0 {
var defertype string
if s.hasOpenDefers {
defertype = "open-coded"
} else if n.Esc == EscNever {
defertype = "stack-allocated"
} else {
defertype = "heap-allocated"
}
Warnl(n.Pos, "%s defer", defertype)
}
if s.hasOpenDefers { // Open source
s.openDeferRecord(n.Left) // Open source main handlers
} else {
d := callDefer / / heap allocation
if n.Esc == EscNever {
d = callDeferStack / / stack allocation
}
s.call(n.Left, d)
}
...
}
Copy the codeThe openDeferRecord mainly records the information that deferred is kept in the openDeferInfo structure.
type openDeferInfo struct {
// The ODEFER node representing the function call of the defer
n *Node
// If defer call is closure call, the address of the argtmp where the
// closure is stored.
closure *ssa.Value // The function called
// The node representing the argtmp where the closure is stored - used for
// function, method, or interface call, to store a closure that panic
// processing can use for this defer.
closureNode *Node
// If defer call is interface call, the address of the argtmp where the
// receiver is stored
rcvr *ssa.Value // The receiver of the method
// The node representing the argtmp where the receiver is stored
rcvrNode *Node
// The addresses of the argtmps where the evaluated arguments of the defer
// function call are stored.
argVals []*ssa.Value // The function's parameters are stored
// The nodes representing the argtmps where the args of the defer are stored
argNodes []*Node
}
Copy the code2) In the case of deferred execution, call the deferReturn function for execution. Define the parameters that defer requires as local variables, and call defer’s functions directly before the function returns
func deferreturn(arg0 uintptr) {
gp := getg()
d := gp._defer
if d == nil {
return
}
sp := getcallersp()
ifd.sp ! = sp {return
}
if d.openDefer {
done := runOpenDeferFrame(gp, d)
if! done { throw("unfinished open-coded defers in deferreturn")
}
gp._defer = d.link
freedefer(d)
return}... }Copy the codeGet the deferBits at execution time, and then decide which defer to execute
func runOpenDeferFrame(gp *g, d *_defer) bool {
done := true
fd := d.fd
// Skip the maxargsize
_, fd = readvarintUnsafe(fd)
deferBitsOffset, fd := readvarintUnsafe(fd)
nDefers, fd := readvarintUnsafe(fd)
deferBits := *(*uint8)(unsafe.Pointer(d.varp - uintptr(deferBitsOffset)))
for i := int(nDefers) - 1; i >= 0; i-- {
// read the funcdata info for this defer
var argWidth, closureOffset, nArgs uint32
argWidth, fd = readvarintUnsafe(fd)
closureOffset, fd = readvarintUnsafe(fd)
nArgs, fd = readvarintUnsafe(fd)
// Determine whether this bit will defer. If it is 1, it will; if it is 0, it will not
if deferBits&(1<<i) == 0 {
for j := uint32(0); j < nArgs; j++ {
_, fd = readvarintUnsafe(fd)
_, fd = readvarintUnsafe(fd)
_, fd = readvarintUnsafe(fd)
}
continue
}
closure := *(**funcval)(unsafe.Pointer(d.varp - uintptr(closureOffset)))
d.fn = closure
deferArgs := deferArgs(d)
// If there is an interface receiver or method receiver, it is
// described/included as the first arg.
for j := uint32(0); j < nArgs; j++ {
var argOffset, argLen, argCallOffset uint32
argOffset, fd = readvarintUnsafe(fd)
argLen, fd = readvarintUnsafe(fd)
argCallOffset, fd = readvarintUnsafe(fd)
memmove(unsafe.Pointer(uintptr(deferArgs)+uintptr(argCallOffset)),
unsafe.Pointer(d.varp-uintptr(argOffset)),
uintptr(argLen))
}
// Set the location to 0 to avoid repeated execution
deferBits = deferBits &^ (1 << i)
*(*uint8)(unsafe.Pointer(d.varp - uintptr(deferBitsOffset))) = deferBits
p := d._panic
// Call the defer function that needs to be executed
reflectcallSave(p, unsafe.Pointer(closure), deferArgs, argWidth)
ifp ! =nil && p.aborted {
break
}
d.fn = nil
// These args are just a copy, so can be cleared immediately
memclrNoHeapPointers(deferArgs, uintptr(argWidth))
ifd._panic ! =nil && d._panic.recovered {
done = deferBits == 0
break}}return done
}
Copy the codeThe disadvantages of 1.14 are: When a panic occurs during the execution of one of the defer functions, the deferred list will not be executed and the defer list will have to be executed. However, open coded defer is not registered with the list, so it needs to be implemented through stack scanning, so panic becomes slow. But panic is less common
This is how defer’s three implementations work