Click “like” to see, form a habit, wechat search [three prince Aobing] pay attention to the programmer who likes to write feelings.

This article has been included in GitHub github.com/JavaFamily, there are a line of large factory interview complete test sites, information and my series of articles.

preface

In the next period of time, Aobin will take you to start the exciting Dubbo journey! Yes, IT is time to start writing the Dubbo series of articles. I have written an article on the evolution of the architecture before, which also explains the popularization and importance of micro-services. In the servitization scenario, the communication between services comes to mind.

This article aobing first takes you to have a general overview of the overall situation. Generally speaking, to be familiar with a framework, you must first know what it does, what pain points it can solve, what the core module is, and what the general operation process is.

If you get into the details right away, there is a 99.99% chance that a wave of DFS will be directly discouraged, so this article will take you through the introduction, overall layering, core components, and general call flow of Dubbo.

Not only that but I’m going to walk you through if you were to design an RPC framework what would you need to do? After this wave of operations you’ll see hey hey how did Dubbo design the way I thought it would? Great minds think alike!

I’ll also write a simple RPC framework implementation to give you an idea of how RPC works.

If you still don’t know what a Dubbo is after reading this article, I can quit.

Let’s talk about RPC first. I find that many students are not familiar with this concept, and some people compare RPC with HTTP. So let’s talk about RPC first.

What is the RPC

Remote Procedure Call (RPC) Remote Procedure Call (RPC) Remote Procedure Call

Think of those days when you were in college, and you were working on your big final assignment, and you were working on your library management system, and you were typing furiously on the keyboard, and you realized the module of borrowing books, returning books, and so on. The calls between the methods you realized were called local procedure calls.

If you tell me that your library system has been servitized and made remote calls, I can only tell you that you have something.

Simply speaking, a method call inside a local machine can be called a local procedure call, while a remote procedure call actually means that you call a method locally on a remote machine, which is called a remote procedure call.

Therefore, RPC object is a local procedure call, and how RPC calls remote methods can be HTTP or TCP based custom protocol.

So you’re not talking about RPC and HTTP in the same hierarchy.

And THE RPC framework is to achieve the same thing as that little assistant, the purpose is to let us use remote call as simple and convenient as local call, and solve some remote call will occur some problems, users with no perception, comfortable, rest assured, happy, it is good, I am also happy without trouble.

How to design an RPC framework

Now that we know what RPC is and the purpose of an RPC framework, let’s think about how you would design an RPC framework.

Service consumer

Let’s start by looking at what is needed from the consumer side (that is, the caller). First, the consumer is interface oriented, so it needs to know what interfaces are available to call, and that interfaces can be maintained through a common JAR package.

Now we know what interface can be called, but only interface ah, specific implementation how to come? This has to be fixed! Still need to come an agent kind so, let consumer adjust only, what matter did not worry about, my agent helps you fix.

By the way, you also need to tell the agent which method you are calling and what the parameter values are.

The proxy will do it for you, but it also needs to know which remote method it is calling, so there needs to be a registry from which the caller can know which service providers they can call. Generally, there are more than one provider.

So providers are usually clustered, and callers need to select a call through load balancing, which can be done through some policy such as same-room call priority etc.

Of course there needs to be a fault tolerance mechanism, after all this is a remote call, the network is unreliable, so there may be a retry or something.

We should also agree a protocol with the service provider, for example, we should use HTTP to communicate, that is, we should speak the same words, otherwise we may not understand.

Of course, serialization is necessary, after all, our local structure is “stereoscopic”, after serialization before transmission, so we need to agree on the serialization format.

And this process may also need to incorporate some Filter, for a wave of unified processing, such as call counting and so on.

This is all the framework needs to do to make consumers feel as if they are calling local methods.

Service provider

It is indisputable that the service provider must implement the corresponding interface.

Then you need to expose your interfaces, register yourself with the registry, and expose what you can offer.

Then a consumer request comes in and needs to be processed, and the provider needs to process the request using a protocol negotiated with the consumer, and then deserialize it.

The serialized request should be thrown into the thread pool for processing, a thread receives the request, finds the corresponding implementation call, and then returns the result.

The registry

In fact, we all mentioned the registry above, which is a kind of platform for people to expose their services and find out which services they can call.

Of course, you can also do configuration center, centralized configuration processing, dynamic change notification to subscribers.

Monitor the operational

In the face of numerous services, refined monitoring and convenient operation and maintenance are essential.

This is a lot of developers in the development of not aware, to you really online operation and maintenance of the time, if there is no good monitoring measures, fast operation and maintenance means, when the time is blind! At a loss what to do, waiting to be criticized!

Don’t ask me why I know the pain, I just know!

The subtotal

To recap, basically all an RPC framework needs to do is agree on a communication protocol, serialization format, some fault tolerance, load balancing policies, monitoring operations, and a registry!

Simple implementation of an RPC framework

Yes, simple implementation, above we thought a lot when thinking about how to design an RPC framework, that is the production environment use level functional requirements, this is a Demo, the purpose is to highlight the RPC framework key function – to implement remote calls.

So there’s nothing there, and I’m using pseudo code to show it, which is to remove some of the protective and restrictive code, because it looks like it’s too much to be intuitive, you need a bunch of try-catch stuff, so I cut it down a little bit, just to get to the point.

Let ‘s Do It!

First we define an interface and a simple implementation.

public interface AobingService {  
    String hello(String name);  
} 

public class AobingServiceImpl implements AobingService {  
    public String hello(String name) {  
        return "Yo man Hello, I am"+ name; }}Copy the code

Then we will implement the service provider’s ability to expose the service.

public class AobingRpcFramework { 
     public static void export(Object service, int port) throws Exception { 
          ServerSocket server = new ServerSocket(port);
          while(true) {
              Socket socket = server.accept();
              new Thread(new Runnable() {
                  // deserialize
                  ObjectInputStream input = new ObjectInputStream(socket.getInputStream()); 
                  String methodName = input.read(); // Read the method nameClass<? >[] parameterTypes = (Class<? >[]) input.readObject();// Parameter type
                  Object[] arguments = (Object[]) input.readObject(); / / parameters
                  Method method = service.getClass().getMethod(methodName, parameterTypes);  // Find a way
                  Object result = method.invoke(service, arguments); // Call the method
                  // Return the result
                  ObjectOutputStream output = newObjectOutputStream(socket.getOutputStream()); output.writeObject(result); }).start(); }}public static <T> T refer (Class<T> interfaceClass, String host, int port) throws Exception {
       return  (T) Proxy.newProxyInstance(interfaceClass.getClassLoader(), newClass<? >[] {interfaceClass},new InvocationHandler() {  
                public Object invoke(Object proxy, Method method, Object[] arguments) throws Throwable {  
                    Socket socket = new Socket(host, port);  // Specify the IP address and port of the provider
                    ObjectOutputStream output = new ObjectOutputStream(socket.getOutputStream()); 
                    output.write(method.getName());  // Pass the method name
                    output.writeObject(method.getParameterTypes());  // Pass the parameter type
                    output.writeObject(arguments);  // Pass the parameter value
                    ObjectInputStream input = new ObjectInputStream(socket.getInputStream());  
                    Object result = input.readObject();  // Read the result
                    returnresult; }}); }}Copy the code

Well, that’s it for the RPC framework, isn’t it simple? The caller passes the method name, parameter type, and parameter value, and the provider receives the parameter and returns the result of the call to the method. This is called a remote procedure call.

So let’s see how do we use it

        // The service provider only needs to expose the interface
       AobingService service = new AobingServiceImpl ();  
       AobingRpcFramework.export(service, 2333);  

       // The service caller only needs to set the dependency
       AobingService service = AobingRpcFramework.refer(AobingService.class, "127.0.0.1".2333);  
       service.hello(); 
Copy the code

It looks pretty good, but it’s pretty crude, and great for demo purposes!

Next up is Dubbo! Are food!

Introduction of Dubbo

Dubbo is a Java-based RPC framework that Alibaba opened source in 2011. It went dormant for a while, but other companies are still using Dubbo and extending it themselves, such as Dubbox from Dangdang and Dubbok from Kaola.com.

But alibaba resumed maintenance of Dubbo in 2017. In 2017, it was awarded the Top 3 Most Popular Open Source software in China 2017 by Open Source China.

In 2018, it merged with Dubbox, entered the Apache incubator, and graduated as an Apache Top-level project in 2019.

Currently, Dubbo community mainly maintains two major versions of 2.6.x and 2.7.x. 2.6.x is a stable version with bug fixes and a few feature enhancements.

While 2.7.x is the major development release, new features and optimizations are updated and added, and the release of 2.7.5 is considered by Dubbo to be a milestone release, which we will analyze later.

It implements interface-oriented proxy RPC invocation, and can cooperate with ZooKeeper and other components to realize service registration and discovery functions, and has load balancing and fault tolerance mechanisms.

Dubbo overall architecture

Let’s take a look at an image from the official website.

The role of each node in the diagram is explained in a warm way.

node The role that
Consumer The service consumer that needs to invoke the remote service
Registry The registry
Provider Service Provider
Container The container in which the service runs
Monitor The monitoring center

To recap the overall process, the service Provider starts up and registers its services with the registry.

The service Consumer starts to subscribe to the registry for the services it needs. The registry then notifies the Consumer of the Provider meta-information, and since the Consumer has obtained the Provider’s address from the registry, it can choose a Provider to invoke directly through load balancing.

The registry then pushes the changes to the service consumer if the service provider metadata changes.

Both service providers and consumers record the number and time of calls in memory and periodically send statistics to the monitoring center.

Some caveats

First of all, the registry and monitoring center are optional, you can not monitor, also do not register, directly write in the configuration file and then provide and consumer direct connection.

Then there are long connections between the registry, provider, and consumer, but not between the monitor, and the consumer calls the provider directly, without going through the registry.

Even if the registry and monitoring center go down, providers and consumers who are already functioning will not be affected because consumers have locally cached provider information.

Dubbo layered architecture

All in all, Dubbo is divided into three layers, and if each layer is further subdivided, there are ten layers. Don’t be afraid of ten layers, this C with you again, we first have a general impression, after the article C will take you further.

The three major layers are Business, RPC, Remoting, and the API layer and SPI layer.

Divided into three layers is actually the same as we know the meaning of network layering, only clear layers, clear boundaries of responsibility can be better expanded.

Dubbo is successful in dividing API layer and SPI layer. It adopts microkernel design and SPI extension, which enables access parties with special needs to customize the extension and do customized secondary development.

Let’s take a look at what each layer does.

  • Service, the business layer, is the business logic layer we develop.

  • Config, the configuration layer, mainly around ServiceConfig and ReferenceConfig, initialization of configuration information.

  • Proxy, Proxy layer, service provider, or consumer all generate a Proxy class to make the service interface transparent, and the Proxy layer makes remote calls and returns results.

  • Register, the registration layer, encapsulates service registration and discovery.

  • The Cluster, routing and Cluster fault tolerance layer is responsible for selecting nodes for specific calls, handling special call requirements and fault tolerance measures for remote call failures.

  • Monitor, the monitoring layer, monitors The Times and times of statistical calls.

  • Portocol, the remote call layer, encapsulates RPC calls and manages Invoker, which represents an abstract-encapsulated executor, more on that later.

  • Exchange, the information Exchange layer, encapsulates the request response model, synchronous to asynchronous.

  • Transport, the network Transport layer, abstracts the unified interface for network Transport, so that users can use Netty when they want and Mina when they want.

  • Serialize, the serialization layer, serializes data into a binary stream, and also deserializes it.

SPI

I’ll mention briefly that SPI (Service Provider Interface) is a Service discovery mechanism built into the JDK that completely decouples the Interface from the implementation. We only declare interfaces; the implementation class is selected in the configuration.

You define an interface and place a text file with the same name as the interface in the meta-INF /services directory. The contents of the file are the implementation classes of the interface, separated by newlines.

That’s how configuration determines which implementation to use!

Dubbo SPI also makes some improvements, but I’ll leave that for later.

Dubbo call procedure

I’ve already explained what each layer does, so let’s go through the process again to deepen your understanding of Dubbo.

Let’s start with the service provider and see how it works.

Service exposure process

First of all, Provider starts and encapsulates exposed interfaces into Invoker through Proxy component according to specific Protocol Protocol. Invoker is a very core component of Dubbo and represents an executable.

This is done by exposing a layer of your own bundle in the Registry, and then registering your Exporter to the Registry through Registry. This is the whole service exposure process.

The consumption process

Next we look at the consumer invocation process (the process of exposing the server is also shown in the figure, which is actually a fairly complete flow chart).

The consumer starts by pulling meta information from the service provider from the registry, and the invocation process also starts from the Proxy. After all, it requires a Proxy to be unaware.

Proxy holds an Invoker object. After invoking invoke, it is necessary to obtain the Invoker list of all callable remote services from Directory through Cluster. If certain routing rules are configured, For example, if an interface can only call a node, filter the Invoker list again.

The rest of the Invoker load balancing through LoadBalance to select a load balancing. And then through Filter to do some statistics, and then through the Client to do data transmission, such as using Netty to transmit.

Transmission requires protocol construction through Codec interface and serialization. Finally, it is sent to the corresponding service provider.

Once received, the service provider also processes the request through the Codec protocol, deserializes it, and then dumps the request to the thread pool for processing. A thread would find a corresponding Exporter based on the request, and finding an Exporter would be an Invoker, but there would be layer upon layer of filters that would eventually call the implementation class and return the result the way it was.

Complete the call!

conclusion

This time Ao Bing with you first understand what is RPC, and then planning a wave of RPC framework needs what components, and then use code to achieve a simple RPC framework.

Then I took you through the history of Dubbo, the overall architecture, the layered design architecture, and what each component does, and then I took you through the whole call process.

I’m too warm!

I will arrange several chapters for Dubbo in the near future. Finally, an interview version of Dubbo will be published. Let’s wait and see.

I’m Aobing, and the more you know, the more you don’t know, and I’ll see you next time!

Talented people’s [three lian] is the biggest power of aobing’s creation, if there are any mistakes and suggestions in this blog, welcome talented people to leave a message!

This article is constantly updated. You can search “Santaizi Aobing” on wechat and read it for the first time. Reply [Information] There are the interview materials and resume templates for first-line big factories prepared by me.