1. Distributed service framework design
A distributed service framework can generally be divided into the following parts,
(1) RPC base layer:
Including underlying communication frameworks such as NIO framework, communication protocols, serialization and deserialization protocols,
And in these parts of the encapsulation, shielding the underlying communication details and serialization way differences
(2) Service Release/consumption:
The service provider invokes the local interface implementation class through Java reflection according to the interface name, method name, parameter list and other information in the consumer request message.
The service consumer encapsulates the interface published by the service provider as a remote service invocation;
(3) Service invocation chain:
In the chain of responsibility of service invocation, related monitoring and extension, such as load balancing, service invocation performance statistics, call completion notification, can be accomplished through coding on the call chain aspect.
Failure retransmission and other functions
(4) Service Registry:
The registry is responsible for publishing and notifying services and supporting smooth online and offline services
(5) Service Governance Center:
Service Governance Center is a visual module that provides visual analysis and maintenance of services, including service health, invocation relationship and so on
The following uses Dubbo as an example to analyze the structure of the distributed services framework.
2.Dubbo’s design role
(1) System roles
Provider: exposes the service Provider of the service.
Consumer: Service Consumer that invokes the remote service.
Registry: A Registry where services are registered and discovered.
Monitor: monitors The Times and time of service invocation.
Container: service running Container.
(2) Call relationship
The service container is responsible for starting, loading, and running the service provider.
At startup, service providers register their services with the registry.
At startup, service consumers subscribe to the registry for the services they need.
The registry returns a list of service provider addresses to the consumer, and if there are changes, the registry pushes the change data to the consumer based on the long connection.
The service consumer, from the provider address list, selects one provider to call based on the soft load balancing algorithm. If the call fails, selects another one to call.
Service consumers and providers accumulate calls and call times in memory and regularly send statistics to the monitoring center every minute
3. How is Dubbo structured
The overall architecture of Dubbo is shown below:
Key points of design at each level in the framework hierarchical architecture:
Service interface layer: This layer is related to the actual business logic and designs corresponding interfaces and implementations according to the business of Service providers and Service consumers.
Configuration layer (Config) : external configuration interface, centering on ServiceConfig and ReferenceConfig, can directly new configuration classes, or generate configuration classes through Spring configuration parsing.
Proxy layer: transparent Proxy of service interface, generating client Stub and server Skeleton of service, centered on ServiceProxy, and ProxyFactory as extension interface.
Service Registration layer (Registry) : encapsulates the registration and discovery of service addresses. It is centered on service URLS and extends the interfaces to RegistryFactory, Registry and RegistryService. There may be no service registry, in which case the service provider directly exposes the service.
Cluster layer (Cluster) : encapsulates routing and load balancing of multiple providers, Bridges registries, centers on Invoker, and extends interfaces to Cluster, Directory, Router, and LoadBalance. Multiple service providers are combined into a single service provider to achieve transparency to the service consumer and interact with only one service provider.
Monitoring layer (Monitor) : Monitors the number and time of RPC calls. It centers on Statistics and extends interfaces to MonitorFactory, Monitor, and MonitorService.
Remote Invocation layer (Protocol) : Dispatcher RPC calls with Protocol, Invoker and Exporter interfaces centered on Invocation and Result. Protocol is the service domain, which is the main functional entry for Invoker exposure and references, and is responsible for the lifecycle management of Invoker. Invoker is the entity domain that is the core model of Dubbo, to which all other models rely or turn. It represents an executable to which invoke calls can be made. It may be a local implementation, a remote implementation, or a cluster implementation.
Exchange: Encapsulates the Request and Response mode, and turns it into async. It uses Request and Response as the center and uses Exchange channel, ExchangeClient and ExchangeServer as the expansion interface.
Network Transport Layer (Transport) : Abstract MINA and Netty as the unified interface, Message as the center, extended interfaces are Channel, Transporter, Client, Server and Codec.
Serialize: Reusable tools with Serialization, ObjectInput, ObjectOutput, and ThreadPool extensions.
4. The underlying implementation of Dubbo
(1) Agreement support
Dubbo supports a variety of protocols, as follows:
Dubbo Protocol Hessian protocol
HTTP RMI protocol
WebService agreement
Thrift protocol Memcached protocol Redis protocol
In the communication process, different service levels generally correspond to different quality of service, so it is very important to choose the appropriate protocol. You can choose based on the creation of your application. For example, the RMI protocol is generally restricted by firewalls. Therefore, do not use THE RMI protocol in the scenario of external and internal communication. Instead, use HTTP or Hessian.
(2) The Dubbo protocol is used by default
Number of connections: single connection
Connection mode: Long connection
Transport protocol: TCP
Transmission mode: NIO asynchronous transmission
Serialization: Hessian binary serialization
Scope of application: The incoming and outgoing parameter packets are small (less than 100K is recommended), there are more consumers than providers, a single consumer cannot overwhelm providers, try not to use the Dubbo protocol to transfer large files or large strings
Usage scenario: Regular remote service method calls
From the scope of application above, Dubbo is suitable for service invocations with a small amount of data and a large number of concurrent invocations, and where the number of consumer machines is much greater than the number of producer machines. It is not suitable for services with a large amount of data such as files, videos, etc., unless the request volume is very low.
(3) Dubbo source module diagram
Dubbo uses package structure to organize each module, each module and its relationship, as shown in the figure:
This can be organized using Dubbo’s code (managed using Maven) and compared to the above modules. A brief description of each package:
Dubo-common common logic module, including Util class and common model.
Dubbo -remoting is an implementation of the Dubbo protocol. If RPC uses RMI, you do not need to use this package.
Dubbo-rpc remote call module, which abstracts various protocols, and dynamic proxy, contains only one-to-one calls and does not care about cluster management.
Dubbo-cluster is a cluster module that disgusts multiple service providers as one provider, including load balancing, fault tolerance, and routing. The address list of the cluster can be statically configured or delivered by the registry.
Dubbo-registry registry module, clustering based on registry delivery addresses, and abstractions for various registries.
Dubo-monitor monitoring module, statistics service call times, call time, call chain tracking services.
Dubbo -config configuration module is the EXTERNAL API of Dubbo. Users use Dubbo through config to hide all details of Dubbo.
The dubbo-Container module is a Standalone container that starts with a simple Main load on Spring. As services usually do not require features of a Web container such as Tomcat/JBoss, there is no need to use a Web container to load the service.
5. Detailed process of service exposure and consumption
(1) The detailed process by which a service provider exposes a service
The service provider exposes the main process of the service:
The ServiceConfig class takes the ref(HelloWorldImpl), the actual class that provides the service, and uses ref to generate an instance of AbstractProxyInvoker using the getInvoker method of ProxyFactory.
This step completes the transformation of the specific service to Invoker. Next comes the transition from Invoker to Exporter.
The key to Dubbo handling service exposure is the transition from Invoker to MY Exporter (shown in red). Dubbo and RMI are typical implementations of Dubbo and RMI.
The realization of Dubbo
The turn of the Dubbo protocol’s Invoker to Exporter occurs in the DubboProtocol class’s export method, which mainly opens the socket listening service and receives various requests from the client. The communication details are implemented by Dubbo itself.
The RMI implementation
The RMI protocol’s Invoker turn to Exporter occurs in the RmiProtocol class’s export method,
It implements RMI services via Spring or Dubbo or JDK, and the communication details are handled by the UNDERLYING JDK, which saves a lot of work.
(2) The detailed process of service consumers’ consumption of a service
Main process of service consumption:
First, the init method of the ReferenceConfig class calls the refer method of Protocol to generate the Invoker instance (shown in red in the figure above), which is the key to service consumption.
Next, convert Invoker to the interface required by the client (for example, HelloWorld).
Recommend an exchange learning group: 697579751, which will share some veteran architects recorded video: Spring, MyBatis, Netty source code analysis, high concurrency, high performance, distributed, microservice architecture principle, JVM performance optimization these become the architect’s necessary knowledge system. You can also receive free learning resources, which have benefited a lot at present:
