In the first talk of Go concurrent programming, let’s talk about simple concurrent access
Concurrent access problems are very common, such as multiple Goroutines operating concurrently on the same resource, eg: counters; Update the user’s account information at the same time; Second kill system; Write data concurrently to the same buffer and so on. If there is no mutex control, there will be some exceptions, such as the count of the counter is not accurate, the user account may be overdrawn, the second kill system oversold, the buffer data chaos, etc., the consequences are serious.
So how to solve these problems? Or you guessed it, Mutex, which in Go, is Mutex. Let’s take a closer look at the Mutex implementation mechanism and the basic use of Mutex in the Go standard library.
Ok, let’s first look at the implementation mechanism of mutex.
An implementation mechanism for mutex
Mutex is a basic means of concurrency control, which is a concurrency control mechanism established to avoid competition. In front of the concrete implementation principle of learning it, we need to understand a concept, is the critical area (part of the concurrent programming, if the program will be concurrent access or modify, so, in order to avoid unexpected results caused by concurrent access, this part of the program needs to be protected, that this part be protected program)
If a critical section is accessed synchronously by many threads, an access or operation error can occur, which is certainly not desirable. Therefore, we can use mutex to restrict critical sections to being held by only one thread at a time. When a critical section is held by one thread, other threads attempting to enter the critical section return a failure or wait. One of these waiting threads has no chance to continue holding the critical section until the holding thread exits.
You see, mutex is a good solution to the problem of resource competition. Some people also call mutex exclusive locks. In the Go library, it provides Mutex to implement the Mutex function. Mutex is the most widely used Synchronization primitives. While there is no strict definition of synchronization primitives, you can think of them as a basic data structure for solving concurrency problems.
Basic usage of Mutex
Before we look at Mutex, I want to give you one thing: the Locker interface. In the Go standard library, Package Sync provides a set of lock-related synchronization primitives. The Package also defines a Locker interface, which Mutex implements.
The Locker interface defines a set of methods for locking synchronization primitives:
type Locker interface {
Lock()
Unlock()
}
Copy the codeAs can be seen, the Lock interface method set defined by Go is very simple, that is, request Lock and Unlock methods, adhering to the concise style of Go language.
However, this interface is not used much in real projects because we tend to use specific synchronization primitives directly rather than through the interface. In simple terms, Mutex provides two methods: Lock and Unlock. Call the Lock method before entering a critical section, and call the Unlock method when exiting a critical section:
func(m *Mutex)Lock(a)
func(m *Mutex)Unlock(a)
Copy the codeWhen a Goroutine obtains ownership of the Lock by calling the Lock method, other Goroutines that request the Lock block the Lock call until the Lock is released and they acquire ownership of the Lock themselves.
Now, you might ask, why does it have to be locked? Don’t worry, LET me show you an example of a concurrent access scenario without locking to see what happens when implemented. In this example, we created 10 goroutines, incrementing a variable (count) continuously, and each goroutine was responsible for 100,000 increments, We expect the final count to be 10 * 100000 = 1,000,000 (one million).
package main
import (
"fmt"
"sync"
)
func main(a) {
var count = 0
// Use WaitGroup to wait for 10 goroutines to complete
var wg sync.WaitGroup
wg.Add(10)
for i := 0; i < 10; i++ {
go func(a) {
defer wg.Done()
// Increments the variable count 10 times
for j := 0; j < 100000; j++ {
count++
}
}()
}
// Wait for 10 goroutines to complete
wg.Wait()
fmt.Println(count)
return
}
Copy the codeIn this code, we use sync.waitGroup to wait for all goroutines to complete before printing the final result. Sync.waitgroup For now, all you need to know is that we use it to control waiting for a set of Goroutines to complete the task.
But, every time you run it, you’re going to get different results, and you’re basically not going to get the million results you want.
Why is that?
In fact, this is because count++ is not an atomic operation. It involves at least several steps, such as reading the current value of the variable count, incrementing it by one, and saving the result to count. Since the operation is not atomic, there can be concurrency problems.
For example, 10 Goroutines read a count of 9527 at the same time, each incrementing by one in its own logical way to 9528, and then write the result back to the count variable. But, in fact, we’re adding one to a total that should have been 10, and we’re swallowing a lot of counts. This is a common mistake for concurrent access to shared data.
// assembly code for the count++ operation
MOVQ "".count(SB), AX
LEAQ 1(AX), CX
MOVQ CX, "".count(SB)
Copy the codeThis problem is relatively easy for experienced developers to spot, but many times concurrency problems are hidden so deeply that even experienced people can’t easily find or Debug them. To solve this problem, Go provides a tool to detect problems with concurrent access to shared resources: Race Detector, which can help us automatically detect problems with data races. The Go Race Detector is based on Google’s C/C++ Sanitizers technology. The compiler detects all memory accesses and adds code to monitor access to these memory addresses (read or write). As the code runs, the Race Detector can monitor asynchronous access to shared variables and print a warning when a race occurs.
Let’s see how this tool works.
Adding the race parameter when compiling, testing, or running Go code makes it possible to detect concurrency problems. For example, in the example above, we can run with the race parameter to check for concurrency problems. If you go run-race counter.go, a warning message will be printed.
This warning will not only tell you that there is a concurrency problem, but it will also tell you which goroutine is writing to which variable on which row, and which goroutine is reading to which variable on which row. It is these concurrent read/write accesses that cause the data race.
While this tool is convenient to use, it does not detect data Race problems at compile time because it is implemented in such a way that it can only be detected when reading or writing to the actual address. Also, at runtime, it can only be detected after a data race is triggered, and if it happens not to be (for example, a data Race problem can only occur at midnight on February 14 or midnight on November 11), it cannot be detected.
Moreover, having race enabled applications deployed online can have a performance impact. Run the go tool compile-race -s counter. Go to see the code for the counter example:
. rel5+4 t=17 TLS+0
rel 43+4 t=8 runtime.racefuncenter+0
rel 50+4 t=16 type.int+0
rel 59+4 t=8 runtime.newobject+0
rel 78+4 t=8 runtime.racewrite+0
rel 97+4 t=16 type.sync.WaitGroup+0
rel 106+4 t=8 runtime.newobject+0
rel 134+4 t=8 runtime.racewriterange+0
rel 172+4 t=8 sync.(*WaitGroup).Add+0
rel 195+4 t=16 "". The main. Func1 · f +0
rel 225+4 t=8 runtime.newproc+0
rel 253+4 t=8 sync.(*WaitGroup).Wait+0
rel 267+4 t=8 runtime.raceread+0
rel 284+4 t=8 runtime.convT64+0
rel 304+4 t=16 type.int+0
rel 321+4 t=16 os.Stdout+0
rel 330+4 t=8 runtime.raceread+0
rel 337+4 t=16 os.Stdout+0
rel 344+4 t=16 go.itab.*os.File,io.Writer+0
rel 386+4 t=8 fmt.Fprintln+0.Copy the codeIn the compiled code, runtime.racefuncenter, Runtime. raceread, Runtime. racewrite, Runtime. racefuncexit and other methods to detect data race are added. With these inserted instructions, the Go Race Detector tool can successfully detect a Data Race problem.
In summary, this tool is implemented by inserting instructions at compile time and detecting concurrent reads and writes at run time to discover data race problems. Since this example has a data race problem, we need to find a way to solve it. This is where Mutex, the hero of the lesson, comes in and can easily eliminate the Data Race.
So how do we do that? Now, I will use this example to tell you the basic usage of Mutex.
As we know, the shared resource here is the count variable, the critical section is count++, and as long as we get the lock in front of the critical section and release the lock when we leave the critical section, we can solve the data race problem perfectly.
package main
import (
"fmt"
"sync"
)
func main(a) {
// Mutex protects counters
var mu sync.Mutex
// The counter value
var count = 0
// A helper variable to verify that all goroutines are complete
var wg sync.WaitGroup
wg.Add(10)
// Start 10 gourontine
for i := 0; i < 10; i++ {
go func(a) {
defer wg.Done()
// add up to 100,000 times
for j := 0; j < 100000; j++ {
mu.Lock()
count++
mu.Unlock()
}
}()
}
wg.Wait()
fmt.Println(count)
return
}
Copy the codeIf you run the program again, you’ll see that the data Race warning is gone and the system simply prints 1,000,000:
How about that? Is it very efficient to use Mutex? The results are amazing.
One thing to note here is that the zero value of Mutex is an unlocked state that has no goroutine waiting, so you don’t need any extra initialization, just declare a variable (like var mu sync.mutex).
What other uses does Mutex have?
In many cases, Mutex is embedded into other structs, as follows:
type Counter struct {
Mu sync.Mutex
Count uint64
}
Copy the codeWhen initializing an embedded struct, there is no need to initialize the Mutex field, and no null pointer or lock cannot be obtained because it is not initialized.
Sometimes, we can also use embedded fields. By embedding fields, you can call the Lock/Unlock method directly on the struct.
package main
import (
"fmt"
"sync"
)
func main(a) {
// Mutex protects counters
var counter Counter
// A helper variable to verify that all goroutines are complete
var wg sync.WaitGroup
wg.Add(10)
// Start 10 gourontine
for i := 0; i < 10; i++ {
go func(a) {
defer wg.Done()
// add up to 100,000 times
for j := 0; j < 100000; j++ {
counter.Lock()
counter.Count++
counter.Unlock()
}
}()
}
wg.Wait()
fmt.Println(counter.Count)
return
}
type Counter struct {
sync.Mutex
Count uint64
}
Copy the code** If the embedded struct has more than one field, we typically place Mutex on top of the field we want to control and use Spaces to separate the fields. ** Even if you don’t, the code will compile normally, but the logic is clearer and easier to maintain when written in this style.
You can even encapsulate lock acquisition, lock release, count increment as a method without exposing lock logic:
package main
import (
"fmt"
"sync"
)
func main(a) {
// Mutex protects counters
var counter Counter
// A helper variable to verify that all goroutines are complete
var wg sync.WaitGroup
wg.Add(10)
// Start 10 gourontine
for i := 0; i < 10; i++ {
go func(a) {
defer wg.Done()
// add up to 100,000 times
for j := 0; j < 100000; j++ {
counter.Incr() // The method protected by the lock
}
}()
}
wg.Wait()
fmt.Println(counter.Count)
return
}
// A thread-safe counter type
type Counter struct {
CounterType int
Name string
mu sync.Mutex
count uint64
}
// The increment method is internally protected by a mutex lock
func (c *Counter) Incr(a) {
c.mu.Lock()
defer c.mu.Unlock()
c.count++
}
// Get the value of the counter, also need lock protection
func (c *Counter) Count(a) uint64 {
c.mu.Lock()
defer c.mu.Unlock()
return c.count
}
Copy the codeconclusion
This article introduces the background of concurrency issues, the use and best practices of Mutex in the standard library, the detection of counter program problems through the Race Detector tool, and how to fix them. You already have an overview of the Mutex synchronization primitive. In the initial stages of project development, we may not have carefully considered the concurrency of the resource, because at the initial stage, we are not sure whether the resource will be shared. After more in-depth design, or new features are added, or code is refined, we need to consider the concurrency of shared resources. Of course, it’s even better if you can anticipate that resources will be shared and accessed at the beginning.
It doesn’t matter if you recognize concurrent access to a shared resource sooner or later. What matters is that once you recognize the problem, you address it with mutex.
For example, Docker Issues 37583, 35517, 32826, 30696, kubernetes Issues 72361, 71617, etc., were discovered later and repaired by Mutex.
A profound
You already know that if a Mutex lock has been acquired by a Goroutine, all the other Goroutines waiting have to wait. So, once the lock is released, which of the waiting Goroutines will get Mutex first?
1) When Mutex is in normal mode, if there is no new goroutine competing with the queue leader goroutine, the queue leader Goroutine wins. If there’s a new goroutine competing there’s a high probability that the new goroutine will get it. 2) When the leader goroutine fails to compete for a lock for 1ms, it puts Mutex into starvation mode. Once in starvation mode, lock ownership is transferred directly from the unlocking Goroutine to the leader goroutine, and the new goroutine goes directly to the back of the queue. 3) When a Goroutine acquires a lock, it switches the lock back to normal mode if it finds that it meets any of the following conditions: #1 it is last in the queue, and #2 it waits for the lock for less than 1ms.
In the process of cultivation, it is best to have someone to grow with you! Hello, everyone
I am code farmer impression, the more you do, the more you know…… See you next time ~
This article is published by OpenWrite, a blogging tool platform