1.1 Introduction

Dubbo is a distributed services framework dedicated to providing high performance transparent RPC remote invocation solutions and providing SOA service governance solutions. This article aims to summarize the use and learning of Dubbo and explore the principle of source code in order to prepare the future can be used for reference in the work.

Since dubbo has many extensions for each layer, such as Redis and ZooKeeper options for the registry, Netty and MINA for the communication module, hession, Hession2 and Java serialization, etc., this paper cannot cover all aspects and focuses on the main flow. Choose ZooKeeper for registry (Client: Curator), Netty for communication, Dubbo for protocol, Hession2 for serialization, and Spring for container. Instead of copying the official documentation, this article explains the Dubbo framework in a step-by-step manner.

1.2 Why servitization

Why servitization?

With the business development, the application of the function and covered by the business bigger and bigger, cause complexity is higher and higher, with increasing amount of code, developers in release link will encounter conflict coordination and code release failure before and after the end, since the code in the process of development of bloated and had to bear larger burden reduce development efficiency, Without specific division of labor, each developer cannot be responsible for business modules. A single application contains different businesses. Problems of one business affect the normal services of other businesses. Therefore, will be split, and extract core services to solve the above problems, but also load balancing, service monitoring, high availability, isolation and degradation, routing strategy, improve the fault tolerance mechanism, serialization scheme choice, the choice of communication framework, developers of low-level details without awareness, service upgrade compatibility problems. Dubbo meets all of these requirements.

1.3 Dubbo

This section describes the evolution from zero basic service invocation to Dubbo. If you are familiar with the Dubbo function, skip it.

1) In the early stage of a simple service invocation, the caller and the server agree on the request parameter field and the request result field. The server starts a Tomcat + SpringMVC and listens to port 80. The caller initiates an HTTP request through httpClient, and the server returns JSON or XML data results. The caller takes the HTTP response result and parses the result data, and a service call ends. The problem is, over the course of a few months, there are more and more core services, more and more services are broken down, more and more urls are provided by the server in the configuration file, O&m is busy setting up load balancing devices and deploying new soft load services such as Nginx, LVS, and HaProxy. Even the change of internal domain name or URL path of each service will increase the labor cost of O&M personnel.

After tomcat is started, the server registers its service list with the registry, including the server IP address, port, and the unique identifier representing the service. For example, it is stored in the registry in the format /a_service/ip_port and /b_service/ip_port. After startup, the caller goes to the registry to find the address list of service A and subscribes /a_service. When the list of service A changes, the change message will be pushed to the caller. Next, the address list is obtained, and the caller creates multiple httpClient instances, each corresponding to a server IP_port, and initiates each call, randomly selecting one from the httpClient instance list, and initiating the call request. When a server goes down or the network fails, the caller will receive a notification message pushed by the registry and remove the httpClient corresponding to the failed IP_port from the list. When the server adds a new server, the caller also receives a notification message and creates a new HttpClient instance to join the HttpClient list. Thus, we added a registry cluster to join the registry client at the service side caller, which solved the problem raised by 1).

So the question is, right? Servers provided by servers have different hardware configurations and performance. Servers with higher weights need to receive more requests and implement load balancing based on polling. Therefore, load balancing policies are required.

3) Load balancing policy module the server will write the weight information into the registry, and the caller will select an instance from the HttpClient list according to their own or the load balancing policy suggested by the server, and then initiate the HTTP request. So the question is, right? Load balance while solved the problem of the equilibrium pressure, but if the service and network between the caller to flash off the request failed to do, if you can try again, but not for all the requests are open retry mechanism, some write requests such as accounts prepaid phone, certainly cannot try again many times, so need a cluster fault-tolerant module.

4) Cluster fault tolerance module selects different fault tolerance mechanisms for different services, such as failfast for non-idemidemtic write operations — failure is reported immediately; failover for quick response read operations — retry other servers. If the caller cannot tolerate the blocking of service invocation, it can also choose Failfast. You can select Failback for notification messages, which are periodically retransmitted when failures occur. You can select FailSafe for audit logs, which are ignored when failures occur. So the question is, right? This section describes how to solve the problems of call between different equipment rooms and read/write separation. When a server on a network segment fails, the server must be isolated immediately. So you need a routing policy module. The administrator adds routing rules, the caller subscribes to routing rule nodes, and the caller receives notification to modify local routing policies when changes occur.

So the question is, right? Adding these routing rules requires a management side, which is connected to the registry and can easily change weights, load balancing policies, and fault tolerance mechanisms

6) managementThe management terminal with a good Web interface.So the question is, right? Only management side, but can’t see the service invocation, unable to do capacity planning, such as the call number, response time, QPS, depend on the service, the service party a few units, the caller has several machines, to provide what services at the same time, I can’t jump every time the registry cluster command look, besides the above service statistics can see the command. Therefore, a monitoring center cluster is needed. The caller and the server regularly report the request and return data of the monitoring center service, and the monitoring center is displayed in a graph through calculation.

7) Monitoring center The caller and service introduce the monitoring center client. The client periodically reports service data to the monitoring center cluster. The monitoring center provides a Web interface for application administrators to log in and view the data. Since the monitoring center also provides statistical data collection services, it can also serve as a service provider to receive statistical reporting requests from common service providers and common callers. In code implementation, we can do a layer of interception between making a call and receiving the result. For example, monitorFilter records the time stamp before making a call, the time stamp when receiving the response result, and then calculates the time difference and sends it to the monitoring center.

So the question is, right? Httpclient is a service that needs to be configured with a registry, a monitoring center, a load balancing policy, a fault tolerance mechanism, and routing rules. The result is returned to parse the HttpClient encapsulated response object, in addition to the load balancing strategy, fault tolerant code encapsulation outside, this does not reduce the development cost but increased, if every service call is like a local call, servitization to the developer no perception of good. A proxy object is needed to encapsulate the underlying details and make communication details, routing, and load balancing invisible to the developer.

8) proxy objects The above said to the we need a service call to act as an agent, code should look like is such: listing 1. DemoService. JavapackageTest; public interface DemoService{String sayHello(String name); } in listing 2. DemoService $Proxy. Java

Public class DemoService$Proxy implements DemoService{
   Private java.lang.reflect.InvocationHandler handler;
   Public static java.lang.reflect.Method[] methods;
   Public java.lang.String interfaceName;
   Public DemoService$Proxy(String interfaceName,java.lang.reflect.InvocationHandlerarg1){
          this.handler=arg1;
          this.interfaceName=interfaceName;
   }

   Public java.lang.String sayHello(java.lang.String arg0){
          Object[]args=new Object[1];
          args[0]= arg0;
          Object ret =handler.invoke(interfaceName, methods[0],args);
                 return(java.lang.String)ret; }}Copy the code

DemoService$Proxy is a Proxy class, but it needs to pass some mandatory fields such as class name, method name, method parameter type, parameter value (this implementation is different from the Dubbo Proxy class, for example, the handler invoker attribute has interfaceName attribute). Handler hides all remote call details, including load balancing, routing, fault tolerance, and communication. Dynamic proxy can be implemented with the help of many open source libraries, such as Javassist, ASM, glib, and JDK. Which one should you choose? Of course, which one is better for performance and ease of use. On the one hand, it performs better than Glib. On the other hand, it can dynamically compile Java source code to generate bytecode. Asm requires framework developers to be familiar with the Class bytecode structure. Scheme comparison we can see about several dynamic proxy dubbo author’s blog – dynamic proxy scheme (javatar.iteye.com/blog/814426) on comparative performance. Question again? Callers can now use proxy objects to call remote services, regardless of the communication protocol, as an RPC framework, but what if the transport parameters are complex objects rather than primitive type parameters? This requires the introduction of serialization. Popular serialization protocols in the industry include Java serialization, hession, hession2, JSON serialization, Protobuf and Thrift. In addition, we need a high-performance communication framework, such as Netty and MINA.

9) Communication and serialization module has NIO communication framework, instead of HttpClient and Tomcat, performance has been greatly improved; NIO is not as easy to set read timeout times as BIO, so timeout management is essential (otherwise the heap runs out of memory). Second, the client-server establishes long connections. The server periodically scans all connections and closes idle connections. Third, to maintain a long connection, the client periodically sends heartbeat messages to the server to check the connection status in time. (If a FIN message fails to be sent due to network disconnection, the client does not know that the connection is closed. The server is changed from Tomcat to Netty, and receives the class name, method name, parameter and other data sent by the caller. Normally, reflection calls the final service code, but reflection is poor. We need to dynamically generate a Wrapper class for each service (by concatenating the source code, dynamically compiled with JavAssist) to avoid reflection, and the code looks like this: Listing wrapper.java

 Public class Wrapper0{    
    public Object invokeMethod(Object object, String method, Class[]parameterTypes,Object[]parameterValues)throwsjava.lang.reflect.InvocationTargetException{
com.test.DemoServiceImpl w; 
     try{ 
          w =(com.test.DemoServiceImpl)object;  
       }catch(Throwable e){ 
          throw new IllegalArgumentException(e);
}   
      try{if("sayHello"The equals (method) & ¶ meterTypes. Length = =1) {return w.sayHello((java.lang.String)parameterValues[0]); }}catch(Throwable e){   
         thrownewjava.lang.reflect.InvocationTargetException(e); }}Copy the code

The above code is somewhat different from the actual Dubbo code, so it’s written to make it easier to understand.

So the question is again? Before using HTTP protocol transmission, the client only needs to specify the path in the URL, spring MVC through the URL path to find the method to reflect the call. Now the server uses Netty, and the caller needs multiple services, and the server exposes multiple services and different services have different methods, how should the caller send data so that the server knows what services it needs? This requires a protocol that specifies what control message to send and what actions and responses the receiver will take when it receives the message.

10) protocol

Since it is a remote procedure call, the method name, class name, parameter type, parameter value and so on are required, so that the server can easily map to the specific method. In addition, I can control the server from returning results by passing some header information, such as message notifications.

Summary: These are the core components of Dubbo: divided by character

Provider (service provider) Consumer (service consumer) Monitor Center Registry Center Admin Web Console (used to modify routing, configuration, and registry)More detailed component diagrams: Broken down by function

  • Directory (responsible for the list of providers generated from the ZooKeeper center)
  • Router = router
  • Fault-tolerantstrategy
  • loadBalance
  • MonitorFilter (Monitor intercept)
  • ZookeeperClient (Zoookeeper client, we use ZooKeeper as an example)
  • Proxy (Proxy object)
  • NettyClient (we use Netty as the communication framework)
  • NettyServer (we use Netty as the communication framework)
  • Hession2Serialization(we chose Hession2 as the serialization scheme)

1.4 Dubbo SPI extension framework

Dubbo, as an open source RPC framework, has many functions, and different components have different solutions. Users will choose the right solution according to their own situation. For example, serialization, communication framework, registry have different options, load balancing, routing rules have different policies, Dubbo uses microkernel + plug-in extension system, to obtain excellent scalability. Dubbo did not use Spring, Guice and other IOC frameworks to manage, but used JDK SPI thinking to implement an IOC framework, reducing the dependence on external frameworks.

Dubbo extension framework features

  • Embedded in dubbo
  • Support for declaring the extension implementation through an SPI file (interfce must have an @SPI annotation) in the format extensionName=extensionClassName, which is similar to Spring’s beanName
  • Supported by configuring designated extensionName to choose corresponding implementation from SPI file ExtensionLoader. GetExtensionLoader (RegistryFactory. Class). GetExtension (” they “)

Similar to Spring’s beanfactory.getBean (” XXX “); ExtensionLoader is a core class that can be implemented by extension mechanisms, similar to Spring’s beanFactory, except that ExtensionLoader only houses all singleton implementations of a single type. The “bean” container that holds the Dubbo extension is made up of multiple ExtensionFactories.

  • Dependency injection is supported. The injection source (ExtensionFactory) can be customized, such as from SPI or from the Spring container. ExtensionFactory is also an extension that can be extended by itself. Set ${ExtName} (ExtName can be replaced by any extension name) to inject the corresponding ExtName of the related type.

  • Can be specified or dynamically generated Adaptive extended class, through the interface method in @ the Adaptive annotations to specify the value value as the key, from the configuration (mon. Com.alibaba.dubbo.com URL) to obtain the key corresponding extName value, Find the corresponding extension and then call the specific method to achieve method call adaptive

  • For extensions that have a constructor parameter of type interface, wrap the final extension implementation classes in order, such as ProtocolListenerWrapper–>ProtocolFilterWrapper –> DubboProtocol

  • You can add the @activate annotation to the same type of extension class name, and obtain the extension with the specified group and parameter name based on the @Activate attribute group and value

This article is shared with friends who need to brush the questions in the interview, I have specially arranged it. The technology inside is not clear by a few words. The answers to many questions are actually very simple, but the thinking and logic behind them are not simple. If you want to learn Java engineering, high performance and distributed, simple. Performance tuning, Spring, MyBatis, Netty source code, data structure, JVM, multithreading and so on, due to the space is limited, the following only show a small part of the interview questions, there is a need for a complete version of the friend can click a link jump to get,Link: Stamp here free download, get code: nuggets •