Abstract

In the last article, we talked about how gRPC manages a connection from Client to Server.

We talked about gRPC having a Resolver, which resolves addresses; Has Balancer for load balancing.

In this article, we will look at gRPC’s design of Resolver and Balancer from a code point of view, and will go through the connection setup from start to finish.

1 DialContext

DialContext is an entry function used by the client to establish a connection. Let’s see what happens in this function:

func DialContext(ctx context.Context, target string, opts ... DialOption) (conn *ClientConn, err error) {

	// 1. Create ClientConn structure
	cc := &ClientConn{
		target:            target,
		...
	}
	
	// 2. Parse targetcc.parsedTarget = grpcutil.ParseTarget(cc.target, cc.dopts.copts.Dialer ! =nil)
	
	// 3. Find the appropriate resolverBuilder according to the resolved target
	resolverBuilder := cc.getResolver(cc.parsedTarget.Scheme)
	
	// create a Resolver
	rWrapper, err := newCCResolverWrapper(cc, resolverBuilder)
	
	/ / 5. Finished
	return cc, nil
}
Copy the code

Obviously, after leaving out a billion details, the process of establishing a connection is actually quite simple. Let’s go through it:

Since gRPC does not provide the functions of service registration and service discovery, developers need to write their own logic of service discovery: Resolver — parser.

After you have the result of the resolution, which is a list of IP addresses, you need to select the IP addresses, known as Balancer.

The rest is error handling, bottom-feeding strategies, etc., which are not covered in this article.

2 Obtaining a Resolver

Let’s start with Resolver.

cc.parsedTarget = grpcutil.ParseTarget(cc.target, cc.dopts.copts.Dialer ! =nil)
Copy the code

The logic of ParseTarget can be summed up in a simple sentence: get the address type of the target parameter passed in by the developer, and then find a Resolver that fits that type of address.

Resolver (Resolver, Resolver, Resolver, Resolver, Resolver, Resolver)

resolverBuilder := cc.getResolver(cc.parsedTarget.Scheme) func (cc *ClientConn) getResolver(scheme string) Resolver.Builder {// Check whether resolver for _ exists in the configuration. Scheme := range cc.dopts.resolvers {if scheme == rb.scheme () {return rb}} Return resolver.get (scheme)} // Func Get(scheme string) Builder {if b, ok := m[scheme]; ok { return b } return nil }Copy the code

From this we can assume that for each ResolverBuilder, pre-registration is required.

We found Resolver’s code, and sure enough, it registered itself in init().

Func init() {resolver.register (&passthroughBuilder{})} Func Register(b Builder) {m[b.cheme ()] = b}Copy the code

At this point, we have explored the registration and acquisition of Resolver.

3 Creating ResolverWrapper

Back in the ClientConn creation process, after obtaining the ResolverBuilder, proceed to the next step:

rWrapper, err := newCCResolverWrapper(cc, resolverBuilder)
Copy the code

In order to implement a plug-in Resolver, gRPC adopts the decorator pattern and creates a ResolverWrapper.

Let’s look at the details of creating a ResolverWrapper:

func newCCResolverWrapper(cc *ClientConn, rb resolver.Builder) (*ccResolverWrapper, error) {
	ccr := &ccResolverWrapper{
		cc:   cc,
		done: grpcsync.NewEvent(),
	}
  
  // From the Builder passed in, create a resolver and put it into a wrapper
  ccr.resolver, err = rb.Build(cc.parsedTarget, ccr, rbo)
 	return ccr, nil
}
Copy the code

Okay, so we can pause here.

Let’s stop and think about what we need to implement: To understand the decoupled Resolver and Balancer, we want to have an intermediate part that receives the addresses resolved by the Resolver and load balances them. So as we read through the rest of the code, we can take this question with us: What does the Resolver and Balancer communication look like?

Looking at the code above, the creation of ClientConn is complete. So we can assume that the rest of the logic is in rb.build (cc.parsedtarget, CCR, rBO).

4 Creating a Resolver

In fact, Build is not a defined method, it is an interface.

type Builder interface {
	Build(target Target, cc ClientConn, opts BuildOptions) (Resolver, error)
}
Copy the code

When creating a Resolver, we need to initialize the various states of the Resolver in the Build method. And because the Build method has a target parameter, we will need to resolve this target when we create the Resolver.

That is, when a Resolver is created, the domain name is resolved for the first time. And this parsing process is designed by the developers themselves.

The natural next step is to ask, what data structure should the parsed result be stored in, and how should the result be passed on?

Let’s take the simplest passthroughResolver:

func (*passthroughBuilder) Build(target resolver.Target, cc resolver.ClientConn, opts resolver.BuildOptions) (resolver.Resolver, error) {
	r := &passthroughResolver{
		target: target,
		cc:     cc,
	}
  // Create Resolver for the first time
	r.start()
	return r, nil
}

// For passthroughResolver, as its name suggests, the parameter is returned as a result
func (r *passthroughResolver) start(a) {
	r.cc.UpdateState(resolver.State{Addresses: []resolver.Address{{Addr: r.target.Endpoint}}})
}
Copy the code

For a Resolver, pass the resolved address into resolver.State and call r.c.updatestate.

So what is r.c.updatestate?

This is the ccResolverWrapper we mentioned above.

At this point the logic is clear: gRPC’s ClientConn resolves the domain name by calling ccResolverWrapper, and the resolution process is up to the developer. After parsing, the result of parsing is returned to ccResolverWrapper.

5 Balancer selection

We can therefore assume that in the ccResolverWrapper, the parsed result will be passed to the Balancer in some form.

Let’s move on:

func (ccr *ccResolverWrapper) UpdateState(s resolver.State){...// Save the latest state of Resolver resolution
	ccr.curState = s
  // Update the status
	ccr.poll(ccr.cc.updateResolverState(ccr.curState, nil))}Copy the code

Not to mention about poll method here is, the point we see CCR. Cc. UpdateResolverState (CCR) curState, nil) this part.

The cc in cCR. cc here is the ClientConn object we created.

In other words, the result of the Resolver resolution is eventually returned to ClientConn.

Note that in the case of the updateResolverState method, the logic is deep in the source code, mainly to handle various cases. Here I directly post the core part, so this part of the code you can understand as pseudo-code implementation, and the original code is different. If you want to see a concrete implementation, you can read the gRPC source code.

func (cc *ClientConn) updateResolverState(s resolver.State, err error) error {
  
  var newBalancerName string
  
  // If balancer is already configured, use balancer in the configuration
  ifcc.sc ! =nil&& cc.sc.lbConfig ! =nil {
    newBalancerName = cc.sc.lbConfig.name
  } 
  // Otherwise, iterate over the address in the result to determine which balancer should be used
  else {
    var isGRPCLB bool
    for _, a := range addrs {
      if a.Type == resolver.GRPCLB {
        isGRPCLB = true
        break}}if isGRPCLB {
      newBalancerName = grpclbName
    } else ifcc.sc ! =nil&& cc.sc.LB ! =nil {
      newBalancerName = *cc.sc.LB
    } else {
      newBalancerName = PickFirstBalancerName
    }
  }
  
  // Specific balancer logic
  cc.switchBalancer(newBalancerName)
  
  // Update Client status with balancerWrapper
  bw := cc.balancerWrapper
  uccsErr := bw.updateClientConnState(&balancer.ClientConnState{ResolverState: s, BalancerConfig: balCfg})

	return ret
}
Copy the code

Kangkang switchBalancer

func (cc *ClientConn) switchBalancer(name string){... builder := balancer.Get(name) cc.curBalancerName = builder.Name() cc.balancerWrapper = newCCBalancerWrapper(cc, builder, cc.balancerBuildOpts) }Copy the code

A sense of deja vu?

Yes, this part of the code is very similar to the ResolverWrapper creation process. Get the Builder Name, then get the Builder by Name, and then create the wrapper.

func newCCBalancerWrapper(cc *ClientConn, b balancer.Builder, bopts balancer.BuildOptions) 	*ccBalancerWrapper {
	ccb := &ccBalancerWrapper{
		cc:       cc,
		scBuffer: buffer.NewUnbounded(),
		done:     grpcsync.NewEvent(),
		subConns: make(map[*acBalancerWrapper]struct{})},go ccb.watcher()
	ccb.balancer = b.Build(ccb, bopts)
	return ccb
}
Copy the code

We will ignore the ccb. Watcher here, which is related to the state of the connection, and we will analyze it in the next article.

Likewise, the process of building a specific Balancer is up to the developer.

In the Balancer creation process, connection management is involved. We’ll do the same in the next paper. The main task for this article will be the Resolver/Balancer interaction.

After creating the corresponding BalancerWrapper, we come to the line bw. UpdateClientConnState.

Note that bw is the balancer we created above. So this is the real Balancer logic again.

However, this code will not be introduced in this article, gRPC for real HTTP/2 connection management logic is relatively complex, we will see in the next article.

6 summary

ResolverWrapper is created when a ClientConn is created. The ClientConn informs the ResolverWrapper to resolve the domain name.

At this point, ResolverWrapper will pass the request to the real Resolver, which will handle the resolution.

After parsing, the Resolver stores the result in ResolverWrapper, which returns the result to ClientConn.

When ClientConn finds that the result of the parse has changed, it notifies BalancerWrapper and re-balances the load. The BalancerWrapper will then tell the real Balancer to do the job and return the result to ClientConn.

Let’s draw a picture to show this process:

Write in the last

First of all, thank you for being here.

This is a pure source interpretation of the article, as a theoretical supplement to the previous article. It is recommended that the two articles be eaten together 🙂

If you have any questions during this process, please leave a message to me, or find me on the official account ** “Red Chicken fungus” **.

In the next article, I’ll introduce you to the nuts and bolts of Balancer, the underlying connection management of gRPC. Again, I’ll probably have an article on how to do it, and then an article on how to implement it, and I’ll see you in the next article.

Thanks again for reading!