More dry goods, public concern: Qiya cloud storage

Antecedents feed

Last time, we combed and analyzed the usage level and got the following tips:

  1. Sync.pool is designed to increase the reuse rate of temporary objects and reduce GC burden.
  2. Do not anticipate objects from pool. Get. They may be new or old (previously used and then Put in).
  3. You can’t make assumptions about the number of elements in the Pool, you can’t;
  4. Sync. Pool’s own Get and Put calls are concurrency safe,sync.NewPointer to the initialization function will be called concurrently, inside the security of only their own know;
  5. When you run out of an instance fetched from a Pool, be sure to call Put or the Pool won’t be able to reuse the instance. Normally this is done by defer;

Official opening statement:

A Pool is a set of temporary objects that may be individually saved and retrieved.

The principle of analyzing

Let’s take a closer look at sync.pool in terms of data structure and implementation logic.

The data structure

The Pool structure

The Sturct Pool structure is a structure for use by users. It is defined as:

type Pool struct {
	// Check whether the Pool is copied because the Pool does not want to be copied.
	// Once you have this field, you can use the Go Vet tool to detect problems at compile time;
	noCopy noCopy	
	
	// Array structure, corresponding to each P, the number is the same as the number of P;
	local     unsafe.Pointer 
	localSize uintptr        

	// Victim and victimSize will victimize local and localSize, respectively;
	// victim's purpose is to reduce performance jitter caused by cold start after GC and make allocation objects smoother;
	victim     unsafe.Pointer 
	victimSize uintptr      

	// The object initializes the constructor, using the method definition
	New func(a) interface{}}Copy the code

A few caveats:

  1. NoCopy to prevent copy from piling code, but this does not prevent compilation, only throughgo vetCheck out;
  2. local 和 localSizeThese two fields implement an array whose elements arepoolLocalStructure for managing temporary objects;
  3. victim 和 victimSizeThis is inpoolCleanupIt’s assigned in the flow, and what the assignment islocal 和 localSize. The victim mechanism changes the Pool clearing from one GC round to two GC rounds to improve object reuse rate and reduce jitter.
  4. Users can only assign New fields to define the object initialization construction behavior;

PoolLocal structure

This structure is the key structure for managing cache elements in a Pool. Pool. Local refers to an array of this type.

The array of structures in the Pool is allocated according to the number of P’s, and each P corresponds to one of these structures.

// Pool.local Specifies the type of the array element pointed to
type poolLocal struct {
	poolLocalInternal

	// Populate poolLocal to 128-byte alignment to avoid false Sharing performance issues
	pad [128 - unsafe.Sizeof(poolLocalInternal{})%128]byte
}

// Manage the internal structure of the cache, corresponding to each P, no lock operation
type poolLocalInternal struct {
    // Each P is private and does not require locking
    private interface{}
    // A double-linked list structure for attaching cache elements
    shared  poolChain
}
Copy the code

poolChain

We can take a look at the poolChain structure for a moment. This is purely a join, with only two Pointers, occupying 16 bytes of memory.

type poolChain struct {
	head *poolChainElt
	tail *poolChainElt
}
Copy the code

So the key is the list element, which is poolChainElt, which looks like this:

type poolChainElt struct {
	// Is essentially an array memory space, managed in ringbuffer mode;
	poolDequeue

	// List pointer
	next, prev *poolChainElt
}

type poolDequeue struct {
	headTail uint64

	// vals is a ring buffer of interface{} values stored in this
	// dequeue. The size of this must be a power of 2.
	vals []eface
}
Copy the code

PoolChainElt is an element point in a list that contains an array space (similar to ringBuffer), and the pool-managed cache objects are stored in poolDequeue (VALS []).

Get

func (p *Pool) Get(a) interface{} {
	// lock G on M (declare M cannot be preempted), return the ID of P bound to M
	// In the current scenario, it can also be considered that G is bound to P, because P cannot be preempted in this scenario, only P is preempted in system calls;
	l, pid := p.pin()
	// If the cached element can be fetched from private, it will be the fastest path;
	x := l.private
	l.private = nil
	if x == nil {
		// Get from the shared queue, Get from the shared queue, and post from the Put queue;
		x, _ = l.shared.popHead()
		if x == nil {
			// Try to fetch elements from the queue for other P's, or from the victim cache
			x = p.getSlow(pid)
		}
	}
	// the g-m lock is removed
	runtime_procUnpin()

	// The slowest path: field initialization, where there is no object in the Pool, can only be created on site;
	if x == nil&& p.New ! =nil {
		x = p.New()
	}
	// Return the object
	return x
}
Copy the code

The semantics of Get is to fetch an element from the Pool. The key here is that the element is cached in layers, from the fastest to the slowest. The fastest is to directly fetch the private field in the corresponding list of P, and the slowest is to call the New function to construct on the spot.

Try path:

  1. The currentThe corresponding Plocal.privateField;
  2. The currentThe corresponding PlocalBidirectional linked list;
  3. otherThe corresponding PlocalList;
  4. Element in the victim cache;
  5. NewField structure;

runtime_procPin

Runtime_procPin is a layer encapsulation of procPin, which is implemented as follows:

func procPin(a) int {
	_g_ := getg()
	mp := _g_.m

	mp.locks++
	return int(mp.p.ptr().id)
}
Copy the code

The purpose of the procPin function is for G to be preempted (that is, G will not be running on M). The core implementation here is the mp.lock ++ operation, which is evaluated in newStack if

if preempt {
	// The preemption flag has been set, but the execution right can be given only if the conditions are met;
	ifthisg.m.locks ! =0|| thisg.m.mallocing ! =0|| thisg.m.preemptoff ! =""|| thisg.m.p.ptr().status ! = _Prunning { gp.stackguard0 = gp.stack.lo + _StackGuard gogo(&gp.sched)// never return}}Copy the code

Pool.pinSlow

It is important to mention that this function does something very important, which is usually only entered when the Pool calls Get for the first time (note that this is the first Get call for each P, but only G on P can do this because of the allPoolsMu lock mutex).

func (p *Pool) pinSlow(a) (*poolLocal, int) {
	// g-m is unlocked first
	runtime_procUnpin()
	// The following logic is in the global lock allPoolsMu
	allPoolsMu.Lock()
	defer allPoolsMu.Unlock()
	// Get the id of the current g-m-p, P
	pid := runtime_procPin()
	s := p.localSize
	l := p.local
	if uintptr(pid) < s {
		return indexLocal(l, pid), pid
	}
	if p.local == nil {
		For the first time, pools need to register themselves in the allPools array
		allPools = append(allPools, p)
	}
	// Number of P's
	size := runtime.GOMAXPROCS(0)
	// The size of the local array is runtime.GOMAXPROCS(0)
	local := make([]poolLocal, size)
	atomic.StorePointer(&p.local, unsafe.Pointer(&local[0])) // store-release
	atomic.StoreUintptr(&p.localSize, uintptr(size))         // store-release
	return &local[pid], pid
}
Copy the code

PinSlow does the following things:

  1. For the first timePoolYou need to register yourselfallPoolsThe array;
  2. Pool.localAccording to the arrayruntime.GOMAXPROCS(0)If this is the default, then this is the number of P, which is the number of cpus;

runtime_procUnpin

This is the function corresponding to runtime_procPin, which declares that M can be preempted, with the field M.locks –.

func procUnpin(a) {
	_g_ := getg()
	_g_.m.locks--
}
Copy the code

Put

The Put method is very simple, because it’s post-processing, and the cleanup is done in the background at runtime, so it’s just putting the element in the queue.

// Put an element into the pool;
func (p *Pool) Put(x interface{}) {
	if x == nil {
		return
	}
	/ / G - M lock
	l, _ := p.pin()
	if l.private == nil {
		// Try to place it in the fastest position, which also corresponds to the order of Get requests;
 		l.private = x
		x = nil
	}
	ifx ! =nil {
		// Put it in the bidirectional list
		l.shared.pushHead(x)
	}
	// the g-m lock is removed
	runtime_procUnpin()
}
Copy the code

However, there is a small point to note that even Put calls p.pin(), so pool.local may also be created here.

runtime

The global variable

Each Pool structure is added to the global queue. In the SRC /sync/pool.go file, several global variables are defined:

var (
	/ / the mutex
	allPoolsMu Mutex

	// Pool array, all pools have registered addresses;
	allPools []*Pool

	// victim mechanism;
	oldPools []*Pool
)
Copy the code

The background process

init

Register the cleanup function at initialization time.

func init(a) {
	runtime_registerPoolCleanup(poolCleanup)
}
Copy the code

The poolCleanup function is called when gcStart calls the ClearPools () function at the start of the Golang GC. In other words, each GC round does a cleanup of all the pools.

poolCleanup

This is done periodically, registered with sync Package init, and executed by the Runtime in the background, to batch clean allPools elements.

func poolCleanup(a) {
	// Clean up the element victim on oldPools
	for _, p := range oldPools {
		p.victim = nil
		p.victimSize = 0
	}

	// Migrate local cache to victim;
	// There is no need for GC to empty all the Pool, and there is a victim below the bottom.
	for _, p := range allPools {
		p.victim = p.local
		p.victimSize = p.localSize
		p.local = nil
		p.localSize = 0
	}

	// Clear a wave of allPools
	oldPools, allPools = allPools, nil
}
Copy the code

Victim has changed the recovery action from one to two to be more resistant. Only the previous cache object is actually cleared each time. The current cache object is only moved to the recycle bin (victim).

Knowledge summary:

  1. This is called at the beginning of each GC roundpoolCleanupFunctions;
  2. Using a two-round cleanup process, the local cache and victim cache work together to protect against fluctuations.

Question to consider

The principle above has been dissected very clearly, now let’s consider some different questions:

1. What happens if Put is called instead of pool. Get?

There are no exceptions. The Pool can accept objects of any type from any source. You can call pool. Put even if it’s not pool. Get, and once you’ve done that, the Pool is no longer a single object element, but a grocery store.

Why?

  1. First of all,Put(x interface{})The interface does not make judgments or assertions about type X;
  2. Second,Pool.PutThere is no internal assertion or determination of the type, and there is no way to determine whether the element is from the interface of Get.

So, in the last post on Pool postures, after calling the pool. Get element, I had a line of type assertions that said something like this:

buffer := bufferPool.Get()
_ = buffer.(*[]byte)
Copy the code

This is important to note because the Sync.pool framework supports storing any type, which can essentially be a grocery store, so the types of objects that Get out and Put in are under the control of the business itself.

2. Pool.GetOut of the object, whyPool.PutPut back into the Pool Pool, in order not to become their own hate garbage?

First, pool. Get and pool. Put must be used together in terms of usage posture, and usually defer pool. Put is used to ensure that elements are released into the Pool.

Have you ever thought about why you suggest Get and Put go together? Will it become unrecyclable garbage if it is not matched?

Pool.get, pool.put, pool.put, pool.put, pool.get, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put, pool.put GC will release him.

As an extreme example, what happens if I make a change in the Pool posture, pool.get every time, and never call pool.put?

The answer is: nothing happens, the program runs as usual. Every time a Pool gets, a New function is executed to construct an object, and the Pool loses its essential function: reuse temporary objects. The very purpose of calling pool.put is to reuse objects.

3. Can the Pool itself be used after replication?

No, but you can do it. What do you mean?

If you copy a Pool in your code, your go Build will compile well, but it may cause internal leaks. The implementation of struct Pool explicitly states that copy is not allowed. Here is the official quote:

// A Pool must not be copied after first use.

Struct Pool has a Pool. NoCopy field that explicitly restricts you from copying, but this can only be checked by running Go Vet (so you must go VET static check before compiling your code to avoid a lot of problems).

$:~/pool$ go vet test_pool.go 
# command-line-arguments
./test_pool.go:26:20: assignment copies lock value to bufferPool2: sync.Pool contains sync.noCopy
Copy the code

Why Pool copy?

After the Copy, we have two reference sources for the objects in the same Pool. After the original Pool is empty, the object in the Copy Pool is not cleared, so that all the objects in the Pool are leaked. And the basis of lock-free design in a Pool is that multiple Goroutines will not operate on the same data structure, which cannot be guaranteed after a Pool copy. Similar to sync.waitgroup, the sync.cond first field uses noCopy, so these two structures cannot be copied.

Therefore, do not copy Pool, be sure to go vet before compiling code.

conclusion

To sum up the above points:

  1. Pool is to improve the reuse rate of temporary objects.
  2. Pool uses a two-tier recycle policy (local + victim) to avoid performance fluctuations.
  3. Pool is essentially a grocery store property that can store anything. Service users can control what items are put in the Pool and what types of items are expected to be taken out of the Pool.
  4. Pool The cache object in a Pool is also layered. The cache object is accessed from the hottest data to the coldest data.
  5. The Pool is concurrency safe, but has no lock structure. The principle is that each P is allocated a cache array (poolLocalInternalArray), so that the cache structure does not cause concurrency;
  6. Never copy a Pool. Explicitly forbid it. Otherwise, it will cause memory leaks and program concurrent logic errors.
  7. Used before the code is compiledgo vetDo static check, can reduce a lot of problems;
  8. The local array of the current Pool is passed to the victim array handle, and all the elements in the victim cache are cleared. In other words, after the victim mechanism is introduced, the object’s cache time becomes two GC cycles;
  9. Don’t make any assumptions about the objects in the Pool. There are two options: either return the memset object and then call itPool.Put, orPool.GetRemove the memset after use;

More dry goods, public concern: Qiya cloud storage