Context: From the beginning to the master
preface
Hello, everyone, I’m Asong. Today I would like to share the Context package with you. After a year of settling down, I will start again from the source point of view based on Go1.17.1, but this time I will start from the beginning, because most beginner readers want to know how to use it before they care about how the source code is implemented.
I’m sure you’ll see code like this in your daily work:
func a1(ctx context ...){
b1(ctx)
}
func b1(ctx context ...){
c1(ctx)
}
func c1(ctx context ...)
Copy the code
Context is taken as the first argument (the official recommendation) and passed down, basically a project code is filled with context, but do you really know what it does and how it works? I remember when I first came into context, my colleagues were saying this is for concurrency control, you can set timeout, timeout will cancel and go down, fast return, and I just thought that if you just pass the context down in a function you can cancel and go back fast. I believe that most beginners have the same idea with me. In fact, this is a wrong idea. The cancellation mechanism is also notification mechanism, and pure pass-through will not work, for example, you write code like this:
func main(a) {
ctx,cancel := context.WithTimeout(context.Background(),10 * time.Second)
defer cancel()
go Monitor(ctx)
time.Sleep(20 * time.Second)
}
func Monitor(ctx context.Context) {
for {
fmt.Print("monitor")}}Copy the code
Even if the context is passed through, it doesn’t work if you don’t use it. So it is necessary to understand the use of context, this article will start from the use of context, step by step parsing Go language context package, let’s begin!!
context
The origin and function of packages
The context package was introduced to the standard library in go1.7:
Context can be used to pass context information between goroutines. The same context can be passed to functions running in different Goroutines. Context is safe for multiple Goroutines to use at the same time. You can create a context using background, TODO, and propagate the context between the function call chains, or you can replace it with a modified copy created WithDeadline, WithTimeout, WithCancel, or WithValue, which sounds a bit tricky, Context is used to synchronize requests for specific data between different Goroutines, cancellation signals, and request processing deadlines.
Gin, DATABASE/SQL, etc. All libraries support context, which makes concurrency control more convenient. Just create a context context in the server entry and pass it through.
context
The use of
createcontext
The context package provides two main ways to create a context:
context.Backgroud()
context.TODO()
These two functions are just aliases for each other. There is no difference between them.
context.Background
Is the default value for context from which all other contexts should derive.context.TODO
You should only use it when you are unsure which context you should use;
So in most cases, we use context.background as the starting context to pass down.
The above two methods create the root context, which does not have any function. The practice is to rely on the With functions provided by the context package to derive:
func WithCancel(parent Context) (ctx Context, cancel CancelFunc)
func WithDeadline(parent Context, deadline time.Time) (Context, CancelFunc)
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc)
func WithValue(parent Context, key, val interface{}) Context
Copy the code
Each of these functions is derived from the parent Context. Using these functions, we create a Context tree. Each node in the tree can have as many child nodes as possible.
You can derive anything you want from a parent Context, which is essentially a Context tree, where each node of the tree can have as many children as you want, and the node hierarchy can have as many children as you want, each child dependent on its parent, as shown in the figure above, We can derive four child contexts from context.background: Ctx1.0-cancel, CTx2.0-deadline, CTx3.0-timeout, and CTx4.0-withValue can also be derived as the parent context, even if the ctx1.0-cancel node is cancelled. It also does not affect the other three parent branches.
Creating the context method and the methods derived from the context are just that, so let’s take a look at how they’re used one by one.
WithValue
Carry data
In Python, we can use gEvent. Local. In Java, we can use ThreadLocal. In the Go language we can use the Context to pass, by using WithValue to create a Context with trace_id, and then pass it through, and print it in the log. Here’s an example:
const (
KEY = "trace_id"
)
func NewRequestID(a) string {
return strings.Replace(uuid.New().String(), "-"."".- 1)}func NewContextWithTraceID(a) context.Context {
ctx := context.WithValue(context.Background(), KEY,NewRequestID())
return ctx
}
func PrintLog(ctx context.Context, message string) {
fmt.Printf("%s|info|trace_id=%s|%s",time.Now().Format("The 2006-01-02 15:04:05") , GetContextValue(ctx, KEY), message)
}
func GetContextValue(ctx context.Context,k string) string{
v, ok := ctx.Value(k).(string)
if! ok{return ""
}
return v
}
func ProcessEnter(ctx context.Context) {
PrintLog(ctx, "Golang Dream Factory")}func main(a) {
ProcessEnter(NewContextWithTraceID())
}
Copy the code
Output result:
The 2021-10-31 15:13:25 | info | trace_id = 7572 e295351e478e91b1ba0fc37886c0 | Golang dreamworks Process finished with the exit code 0Copy the code
We create a CTX with trace_id based on context.background and pass it through the context tree. Any context derived from the CTX will get this value and when we finally print the log we can print the value from the CTX to the log. Currently, some RPC frameworks support Context, so trace_id is passed down more easily.
Four things to note when using withVaule:
-
You are not advised to use the context value to pass key parameters. Key parameters should be declared explicitly and should not be handled implicitly. It is better to use context values such as signature and trace_id.
-
Since a value is also in the form of a key or value, it is recommended that the built-in type be used for key to avoid conflicts caused by multiple packages using context at the same time.
-
In the example above, we get trace_id directly from the current CTX. In fact, we can also get the value from the parent context. To get the key-value pair, we first look in the current context. If it doesn’t find it looks for the value of that key from the parent context until it returns nil in some parent context or finds the value.
-
Context key and value are interface types. The type of this type cannot be determined at compile time, so it is not very safe. Therefore, do not forget to ensure the robustness of the program when asserting type.
Timeout control
Generally, robust programs need to set the timeout time to avoid resource consumption due to the long response time of the server. Therefore, some Web frameworks or RPC frameworks adopt withTimeout or withDeadline to do the timeout control. When a request reaches the timeout time set by us, it will be canceled in time and no further execution will be performed. WithTimeout and withDeadline work the same way, except that they pass different time parameters, and they both automatically cancel the Context by the time they pass in. Notice that they both return a cancelFunc method that cancels ahead of time. However, it is recommended to call cancelFunc after automatic cancellation to stop timing and reduce unnecessary waste of resources.
The difference between withTimeout and WithDeadline is that withTimeout takes the duration as an input parameter instead of a time object. It is the same to use either method, depending on business scenarios and personal habits, because essentially withTimout is also called WithDeadline inside.
Now let’s use an example to try out the timeout control. Now let’s simulate a request to write two examples:
- Subsequent execution terminates when the timeout period is reached
func main(a) {
HttpHandler()
}
func NewContextWithTimeout(a) (context.Context,context.CancelFunc) {
return context.WithTimeout(context.Background(), 3 * time.Second)
}
func HttpHandler(a) {
ctx, cancel := NewContextWithTimeout()
defer cancel()
deal(ctx)
}
func deal(ctx context.Context) {
for i:=0; i< 10; i++ {
time.Sleep(1*time.Second)
select {
case <- ctx.Done():
fmt.Println(ctx.Err())
return
default:
fmt.Printf("deal time is %d\n", i)
}
}
}
Copy the code
Output result:
deal time is 0
deal time is 1
context deadline exceeded
Copy the code
- Subsequent executions are terminated if the timeout period is not reached
func main(a) {
HttpHandler1()
}
func NewContextWithTimeout1(a) (context.Context,context.CancelFunc) {
return context.WithTimeout(context.Background(), 3 * time.Second)
}
func HttpHandler1(a) {
ctx, cancel := NewContextWithTimeout1()
defer cancel()
deal1(ctx, cancel)
}
func deal1(ctx context.Context, cancel context.CancelFunc) {
for i:=0; i< 10; i++ {
time.Sleep(1*time.Second)
select {
case <- ctx.Done():
fmt.Println(ctx.Err())
return
default:
fmt.Printf("deal time is %d\n", i)
cancel()
}
}
}
Copy the code
Output result:
deal time is 0
context canceled
Copy the code
It is relatively easy to use. It can be automatically cancelled over time or manually controlled. One of the pitfalls here is that the context in the call link that we pass through from the request entry carries a timeout, and if we want to have a separate Goroutine in there to do something else that doesn’t get canceled as the request ends, The passed context must be derived from either context.background or context.todo, and the rejection will not be as expected, as you can see in my previous article: Context is used incorrectly.
withCancel
Cancel the control
In daily business development, we often open multiple Gouroutines to fulfill a complex requirement. This can result in multiple Goroutines being opened in a single request and not being able to control them. And then we can use withCancel to generate a context and pass it to the different Goroutines, and when I want those goroutines to stop running, I can call Cancel to cancel.
Here’s an example:
func main(a) {
ctx,cancel := context.WithCancel(context.Background())
go Speak(ctx)
time.Sleep(10*time.Second)
cancel()
time.Sleep(1*time.Second)
}
func Speak(ctx context.Context) {
for range time.Tick(time.Second){
select {
case <- ctx.Done():
fmt.Println("I'm gonna shut up.")
return
default:
fmt.Println("balabalabalabala")}}}Copy the code
Running results:
balabalabalabala .... Balabalabalabala I'm going to shut upCopy the code
We create a background-based CTX with withCancel, and then launch a speech program that speaks every 1s. The main function cancels 10 seconds later, and speak exits when it detects a cancel signal.
The customContext
Context is essentially an interface, so we can customize the Context by implementing the Context. Generally, Web frameworks or RPC frameworks are implemented in this form. For example, gin framework Context has its own encapsulation layer, and the specific code and implementation are posted here. If you are interested, take a look at how gin.Context is implemented.
Source appreciation
Context is simply an interface that defines four methods:
type Context interface {
Deadline() (deadline time.Time, ok bool)
Done() <-chan struct{}
Err() error
Value(key interface{}) interface{}}Copy the code
Deadlne
Methods: whenContext
Automatically cancels or returns after the cancellation time has been cancelledDone
Methods: whenContext
To be canceled or arriveddeadline
Returns a closedchannel
Err
Methods: whenContext
Returns after being cancelled or closedcontext
Reason for cancellationValue
Method: Get the setkey
The value of the corresponding
This interface is implemented by three classes: emptyCtx, ValueCtx, and cancelCtx. It is written as an anonymous interface, allowing you to override any type that implements the interface.
Let’s take a look at the layers from creation to use.
Create a rootContext
The object created when we call context.Background, context.TODO is empty:
var (
background = new(emptyCtx)
todo = new(emptyCtx)
)
func Background(a) Context {
return background
}
func TODO(a) Context {
return todo
}
Copy the code
Background is the same as TODO. Background is usually used by main functions, initializers, and tests, and acts as the top-level context for incoming requests. TODO means that when it is not clear which Context to use or is not yet available, the code should use context.todo and then replace it later, just because the semantics are different.
emptyCtx
class
EmptyCtx is mainly used to create the root Context, and its implementation method is also an empty structure. The actual source code looks like this:
type emptyCtx int
func (*emptyCtx) Deadline(a) (deadline time.Time, ok bool) {
return
}
func (*emptyCtx) Done(a) <-chan struct{} {
return nil
}
func (*emptyCtx) Err(a) error {
return nil
}
func (*emptyCtx) Value(key interface{}) interface{} {
return nil
}
func (e *emptyCtx) String(a) string {
switch e {
case background:
return "context.Background"
case todo:
return "context.TODO"
}
return "unknown empty Context"
}
Copy the code
WithValue
The implementation of the
WithValue essentially calls the valueCtx class:
func WithValue(parent Context, key, val interface{}) Context {
if parent == nil {
panic("cannot create context from nil parent")}if key == nil {
panic("nil key")}if! reflectlite.TypeOf(key).Comparable() {panic("key is not comparable")}return &valueCtx{parent, key, val}
}
Copy the code
valueCtx
class
The purpose of valueCtx is to carry key-value pairs for the Context, and because it inherits the anonymous interface implementation, it inherits the parent Context, which is equivalent to embedding it into the Context
type valueCtx struct {
Context
key, val interface{}}Copy the code
The String method outputs the Context and the key-value pair it carries:
func (c *valueCtx) String(a) string {
return contextName(c.Context) + ".WithValue(type " +
reflectlite.TypeOf(c.key).String() +
", val " + stringify(c.val) + ")"
}
Copy the code
Implement the Value method to store key-value pairs:
func (c *valueCtx) Value(key interface{}) interface{} {
if c.key == key {
return c.val
}
return c.Context.Value(key)
}
Copy the code
Look at the picture to understand:
So we’re going to call the Value method in the Context and we’re going to call it all the way up to the final root node, and we’re going to return if we find key, and we’re going to return nil if we don’t find emptyCtx.
WithCancel
The implementation of the
Let’s take a look at the WithCancel entry function source code:
func WithCancel(parent Context) (ctx Context, cancel CancelFunc) {
if parent == nil {
panic("cannot create context from nil parent")
}
c := newCancelCtx(parent)
propagateCancel(parent, &c)
return &c, func(a) { c.cancel(true, Canceled) }
}
Copy the code
This function performs the following steps:
- To create a
cancelCtx
Object as a childcontext
- And then call
propagateCancel
Build a father and soncontext
The relationship between the parent and the parentcontext
When cancelled, soncontext
It will also be canceled. - Returns the child
context
Object and subtree cancellation functions
Let’s first examine the cancelCtx class.
cancelCtx
class
CancelCtx inherits the Context and also implements the interface canceler:
type cancelCtx struct {
Context
mu sync.Mutex // protects following fields
done atomic.Value // of chan struct{}, created lazily, closed by first cancel call
children map[canceler]struct{} // set to nil by the first cancel call
err error // set to non-nil by the first cancel call
}
Copy the code
Word short explanation:
mu
: is a mutex that guarantees concurrency security, socontext
It is concurrency safedone
: used to docontext
The cancel notification signal used in previous versionschan struct{}
Type, now usedatomic.Value
Do lock optimizationchildren
:key
Yes interface typecanceler
The purpose is to store the currentcanceler
The child node of the interface traverses the child node to send a cancellation signal when the root node cancelserror
: whencontext
Store cancellation information when canceling
The Done method is implemented here and returns a read-only channel so that we can wait externally for a notification through this blocked channel.
The specific code will not be posted. Let’s go back and look at how propagateCancel builds associations between parent and child contexts.
propagateCancel
methods
The code is a bit long and cumbersome to explain, but I’ll add comments to the code to make it a little more intuitive:
func propagateCancel(parent Context, child canceler) {
// If nil is returned, the current parent context is never cancelled, is an empty node, and is returned.
done := parent.Done()
if done == nil {
return // parent is never canceled
}
// Determine in advance whether a parent context is cancelled. If it is cancelled, there is no need to build the association.
// Cancel the current child node and return
select {
case <-done:
// parent is already canceled
child.cancel(false, parent.Err())
return
default:}// The purpose here is to find a context that can be "suspended" or "cancelled"
if p, ok := parentCancelCtx(parent); ok {
p.mu.Lock()
// A context that can be suspended or cancelled is found, but has been cancelled, so the child node is not needed either
// Call it off
ifp.err ! =nil {
child.cancel(false, p.err)
} else {
// Attach the current node to the parent node's childrn map, which can be cancelled layer by layer when calling cancel
if p.children == nil {
// Since the childer node also becomes the parent, the map structure needs to be initialized
p.children = make(map[canceler]struct{})
}
p.children[child] = struct{}{}
}
p.mu.Unlock()
} else {
// If no parent node is found to mount, then open a goroutine
atomic.AddInt32(&goroutines, +1)
go func(a) {
select {
case <-parent.Done():
child.cancel(false, parent.Err())
case <-child.Done():
}
}()
}
}
Copy the code
The real puzzle in this code is the if and else branches. Forget the code and just say why. Because we can customize the context ourselves, and when we plug the context into a structure, we can’t find a parent to cancel, so we have to start a new coroutine to listen on.
For this confusing recommendation read Rao Dada’s article: [In-depth Understanding of the Go Language Context](www.cnblogs.com/qcrao-2018/…
cancel
methods
Finally, let’s look at the implementation of the returned cancel method, which closes the Channel in the context and synchronizes the cancel signal to all its subcontexts:
func (c *cancelCtx) cancel(removeFromParent bool, err error) {
Context defines the default error:var Canceled = errors.New("context Canceled ").
if err == nil {
panic("context: internal error: missing cancel error")}// There is an error message indicating that the current node has been canceled
c.mu.Lock()
ifc.err ! =nil {
c.mu.Unlock()
return // already canceled
}
c.err = err
// Close the channel to notify other coroutines
d, _ := c.done.Load().(chan struct{})
if d == nil {
c.done.Store(closedchan)
} else {
close(d)
}
// The current node cancels down, traverses all of its children, and cancels
for child := range c.children {
// NOTE: acquiring the child's lock while holding parent's lock.
child.cancel(false, err)
}
// The node is empty
c.children = nil
c.mu.Unlock()
// Remove the current node from the parent node. True is passed only when called by the external parent node
// all others pass false, internal calls are excluded because c.dren = nil
if removeFromParent {
removeChild(c.Context, c)
}
}
Copy the code
At this point, the entire WithCancel method source code has been analyzed. From the source code, we can know that the Cancel method can be called repeatedly and is idempotent.
withDeadline
,WithTimeout
The implementation of the
First look at the WithTimeout method, which internally calls the WithDeadline method:
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) {
return WithDeadline(parent, time.Now().Add(timeout))
}
Copy the code
So let’s focus on how withDeadline is implemented:
func WithDeadline(parent Context, d time.Time) (Context, CancelFunc) {
// Cannot create a derived context for empty 'context'
if parent == nil {
panic("cannot create context from nil parent")}// When the parent context ends earlier than the time to be set, there is no need to deal with the child timer separately
if cur, ok := parent.Deadline(); ok && cur.Before(d) {
// The current deadline is already sooner than the new one.
return WithCancel(parent)
}
// Create a timerCtx object
c := &timerCtx{
cancelCtx: newCancelCtx(parent),
deadline: d,
}
// Attach the current node to the parent node
propagateCancel(parent, c)
// Get the expiration time
dur := time.Until(d)
// The current time has expired
if dur <= 0 {
c.cancel(true, DeadlineExceeded) // deadline has already passed
return c, func(a) { c.cancel(false, Canceled) }
}
c.mu.Lock()
defer c.mu.Unlock()
// If it is not cancelled, add a timer to cancel it periodically
if c.err == nil {
c.timer = time.AfterFunc(dur, func(a) {
c.cancel(true, DeadlineExceeded)
})
}
return c, func(a) { c.cancel(true, Canceled) }
}
Copy the code
The withDeadline method has one more timer than the withCancel method to call the cancel method. This cancel method is overridden in the timerCtx class, which is based on cancelCtx and has two more fields:
type timerCtx struct {
cancelCtx
timer *time.Timer // Under cancelCtx.mu.
deadline time.Time
}
Copy the code
TimerCtx cancel method, internal is also called cancelCtx cancel method to cancel:
func (c *timerCtx) cancel(removeFromParent bool, err error) {
CancelCtx cancels the child context by calling cancelCtx's cancel method
c.cancelCtx.cancel(false, err)
// Remove from the parent context and put it here
if removeFromParent {
// Remove this timerCtx from its parent cancelCtx's children.
removeChild(c.cancelCtx.Context, c)
}
// Stop the timer to release resources
c.mu.Lock()
ifc.timer ! =nil {
c.timer.Stop()
c.timer = nil
}
c.mu.Unlock()
}
Copy the code
How do you feel now that we’ve finished reading the source code?
context
The advantages and disadvantages of
The context package is designed to do concurrency control. This package has both advantages and disadvantages.
disadvantages
- Affect code aesthetics now basically all
web
The framework,RPC
Frameworks are implementedcontext
, which results in an argument to every function in our codecontext
Even if it is not necessary, it is a little ugly to pass along this parameter. context
Can carry values, but there is no limit, type and size are not limited, that is, there is no constraint, this is easy to abuse, the robustness of the program is difficult to guarantee; There is also the question of passagecontext
Carrying values is not as comfortable as passing them explicitly, and the readability becomes worse.- You can customize
context
Such risks are uncontrollable and lead to abuse. context
Cancel and auto cancel error returns are not friendly enough to customize errors, and it is difficult to troubleshoot problems that are difficult to troubleshoot.- In fact, the creation of derivative nodes is to create one linked list node after another, whose time complexity is O(n), and the efficiency of dropping will be low if there are too many nodes.
advantages
- use
context
Can do better concurrency control, can better managementgoroutine
Abuse. context
There is no limit to the carrier function, so we can transfer any data, it can be said that this is a double-edged sword- Online said
context
The package solutiongoroutine
thecancelation
Question, what do you think?
Refer to the article
PKG. Go. Dev/context @ go1… Studygolang.com/articles/21… Draveness. Me/golang/docs… www.cnblogs.com/qcrao-2018/… Segmentfault.com/a/119000003… www.flysnow.org/2017/05/12/…
conclusion
Context is a little ugly in use, but it can solve a lot of problems. Daily business development cannot do without the use of context, but do not use the wrong context, its cancellation also uses channel notification, so there should be listening code in the code to listen for cancellation signals. This is often the majority of beginners easy to ignore a point.
Examples have been uploaded to github: github.com/asong2020/G…
Well, that’s the end of this article. I amasong
And we’ll see you next time.
** Welcome to pay attention to the public account: [Golang Dream Factory]