Hi, everyone, I’m Haohongfan.

This article focuses on some of the features of WaitGroup so that we can get to the bottom of WaitGroup. I won’t go into the basic usage of WaitGroup here. Compared to This is probably the easiest to understand Go Mutex source code, WaitGroup is much simpler.

Source analysis

Add()

Wait()

type WaitGroup struct {
	noCopy noCopy
	state1 [3]uint32
}
Copy the code

The WaitGroup underlying structure looks simple, but WaitGroup. State1 actually represents three fields: Counter, waiter, and sema.

  • Wg.add (N), wg.done ()
  • Waiter: The current number of WaitGroup waiting for the end of a WaitGroup task. This is the number of times wG.wait () has been called, so this value is usually 1.
  • Sema: Semaphore used to wake up the Wait() function.

Why put counter and waiter together?

This is to ensure the integrity of the WaitGroup state. For example, take a look at the source code below

// sync/waitgroup.go:L79 --> Add()
if v > 0 || w == 0 { // v => counter, w => waiter
    return
}
// ...
*statep = 0
for; w ! =0; w-- {
    runtime_Semrelease(semap, false.0)}Copy the code

When wg.counter <= 0 && WG.waiter! = 0, the waiters will be woken up for the waiting coroutine to continue running. However, the callers using WaitGroup usually operate concurrently. If the counter and the waiter are not obtained at the same time, the counter and the waiter may not match, causing the deadlock or the application to terminate the wait prematurely.

How do I get counter and Waiter?

For wG.state state changes, WaitGroup Add(), Wait() uses atomic calculations (to avoid lock contention). However, since atomic needs the user to ensure its 64-bit alignment, we set both counter and Waiter to a uint32 as a variable, which satisfies atomic requirements and ensures the integrity of obtaining waiter and Counter. But this leads to different ways of getting state on 32-bit and 64-bit machines. The diagram below:A quick explanation:

Because 64-bit alignment is guaranteed on 64-bit machines, fetch data with 64-bit alignment and fetch state1[0], which is itself 64-bit aligned. However, 64-bit alignment is not guaranteed on a 32-bit machine, since 32-bit machines are 4-byte aligned. If state[0] is used on a 64-bit machine, state[1] may cause atmoic usage errors.

So the first 32 bits of the 32-bit machine are empty, so that the next 64 bits naturally meet the 64 bit alignment, and the first 32 bits fit into semA just right. The early implementation of WaitGroup, SEMa, was separate from state1, resulting in four bytes wasted using WaitGroup, but since GO1.11 the structure has changed.

Why is Done missing from the flowchart?

Wg.add (1) is a + 1, wg.done () is a + 1, wg.done () is a + 1, wG.add (-1) is a + 1. Although we know that WG.add can pass negative numbers when used by WG.done, don’t use it.

Conditions for exiting waitGroup

There’s only one condition, WaitGroup. Counter is equal to 0

Special requirements for daily development

1. Control timeout/error control

Although WaitGroup can make the primary Goroutine wait for the child Goroutine to exit, WaitGroup encounters some special requirements, such as timeouts and error control, that are not well met and require special handling.

When A user purchases A certain goods on the e-commerce platform, he/she needs to obtain system A (equity system), System B (role system), system C (commodity system), and system D (XX system) in order to calculate the discount amount for the user. To improve program performance, multiple Goroutines may be issued to access these systems at the same time, inevitably using WaitGroup to wait for data to return, but there are some problems:

  1. When a system fails, how does a waiting Goroutine sense these errors?
  2. When a system responds too slowly, how does waiting Goroutine control access timeout?

None of these problems can be handled directly with WaitGroup. Using a channel directly in conjunction with WaitGroup to control timeouts and error returns is not easy to encapsulate and error-prone. We can use ErrGroup instead of WaitGroup.

The usage of ErrGroup is not explained here. golang.org/x/sync/errgroup

package main

import (
	"context"
	"fmt"
	"golang.org/x/sync/errgroup"
	"time"
)

func main(a) {
	ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
	defer cancel()
	errGroup, newCtx := errgroup.WithContext(ctx)

	done := make(chan struct{})
	go func(a) {
		for i := 0; i < 10; i++ {
			errGroup.Go(func(a) error {
				time.Sleep(time.Second * 10)
				return nil})}iferr := errGroup.Wait(); err ! =nil {
			fmt.Printf("do err:%v\n", err)
			return
		}
		done <- struct{} {}} ()select {
	case <-newCtx.Done():
		fmt.Printf("err:%v ", newCtx.Err())
		return
	case <-done:
	}
	fmt.Println("success")}Copy the code

2. Control Goroutine quantity

Scenario simulation: There are about 20-30 million data to process, and according to the server tests, the best performance is when 200 Goroutine processes are started. How to control?

When you have problems like this, you can’t just use WaitGroup. Make sure that all the data can be processed, but also that there are at most 200 Goroutines at a time. This problem requires WaitGroup to be used in conjunction with Channel.

package main

import (
	"fmt"
	"sync"
	"time"
)

func main(a) {
	var wg = sync.WaitGroup{}
	manyDataList := []int{1.2.3.4.5.6.7.8.9.10}
	ch := make(chan bool.3)
	for _, v := range manyDataList {
		wg.Add(1)
		go func(data int) {
			defer wg.Done()

			ch <- true
			fmt.Printf("go func: %d, time: %d\n", data, time.Now().Unix())
			time.Sleep(time.Second)
			<-ch
		}(v)
	}
	wg.Wait()
}
Copy the code

Use caution points

Using WaitGroup also cannot be replicated. I won’t go into specific examples. See “This is probably the easiest to understand Go Mutex source Code Analysis” for details.

That’s pretty much the end of the WaitGroup anatomy. If you want to talk to me, please leave a comment in the comments section.

Welcome to follow my official account: HHFCodeRV, learn and make progress together