How Dubbo connects heterogeneous microservices Thumbs up, thumbs up, thumbs up.

Follow my public account Solomon for more highlights

How does Dubbo connect heterogeneous microservices

Dubbo is an RPC service framework that provides a service programming model oriented to interface proxy. It also shields the underlying details of remote communication and has service governance functions, such as service discovery and traffic scheduling solutions.

Dubbo’s support for multi-protocol and multi-service discovery model is adopted to realize the interconnection between heterogeneous micro-service systems and solve communication problems in heterogeneous technical system scenarios. For example:

  • Smooth migration between heterogeneous architectures
  • Large-scale cross-region invocation
  • Address discovery and traffic scheduling in a multi-cluster deployment environment

1. Micro-service framework for contract broker

Dubbo is adopted as the basic framework for developing microservices, and an interface oriented programming model is implemented, which makes developing remote service calls just like developing local services, for example:

1.1 Definition of Contract

public interface GreetingsService {
    String say(String name);
}
Copy the code

1.2 Consumer invokes the service

@Reference
private GreetingService greetingService;// Reference the business proxy

public void say(String name) {
  greetingService.sayHi("Hello Xiao Ge on architecture!");// Remote access
}
Copy the code

1.3 Dubbo usage Layer architecture schematic diagram

2. Problems faced by isomorphic/heterogeneous microservice system

2.1 Definition of isomorphism and heterogeneity

  • Isomorphic microservice system
    • The internal microservices of the enterprise are all products developed and developed based on the same service framework, such as all Dubbo microservices supporting products.
  • Heterogeneous microservice system
    • Corporate microservices may be constructed using multiple different service frameworks, and the co-existence of multiple microservices with different technology stacks is very common in large organizations, such as Dubbo, Sofa- RPC, Brpc, etc

There are many reasons for this heterogeneity, whether it’s legacy systems, whether it’s technology stack migration, or whether it’s individual selection by different business units to meet their specific needs

2.2 Coexistence of heterogeneous microservices

Different systems usually use different RPC communication protocols and deploy independent registry clusters. Facing such multi-protocol and multi-registry cluster scenarios, how to achieve transparent address discovery and transparent RPC calls between each other?

  • With unified service registration and discovery products, different systems can use different driver SDK access
  • The communication part of asynchronous technology system uses PB to define the problem that public communication cannot communicate

2.3 Dubbo isomorphic system

The problem of multi-protocol, multi-registry clusters can also exist in homogeneous microservices, especially when the size of microservices within an organization grows to an order of magnitude.

  • Different services use different communication protocols, because different services face different business scenarios, which further lead to different data transmission characteristics. We need to adopt protocols that are more suitable for different business characteristics. For example, we might use Dubbo protocol for common business services, HTTP protocol for services interacting with FrontEnd, and gRPC protocol for services requiring streaming data transmission, etc.
  • Another common problem within the Dubbo system is that in a large-scale distributed deployment scenario, the microservice system will be deployed across regions and registries. In this case, the problems of address synchronization and traffic scheduling between multiple clusters will occur.

To sum up, ** both homogeneous and heterogeneous systems face the problem of multi-protocol communication and multi-registry cluster address discovery. **Dubbo is currently multi-protocol, multi-registry support, addressing the multi-protocol, multi-registry scenario within the Dubbo homogeneous system.

3. Dubbo multi-protocol, multi-registry mechanism

Two scenarios are used to explain the usage and working principles of Dubbo multi-protocol and multi-registry mechanisms.

More than 3.1 agreement

The above is a set of microservices developed using Dubbo, and the communication between services uses different protocols.

Application B, as a service provider, published 5 services, among which:

  • DemoService1 DemoService2throughdubboAgreement to release
  • DemoService3 DemoService4throughgRPCAgreement to release
  • DemoService0throughdubbo 、gRPCDual-protocol Publishing

As the consumer, application A uses dubbo to consume DemoService1 DemoService2 and gRPC to consume DemoService0.

As the consumer, application B uses gRPC to consume DemoService2 DemoService4 and Dubbo to consume DemoService0.

Here is the code configuration:

  1. Provider application B
<dubbo:service interface="org.apache.dubbo.samples.basic.api.DemoService1" protocol="dubbo"/>
<dubbo:service interface="org.apache.dubbo.samples.basic.api.DemoService2" protocol="dubbo"/>

<dubbo:service interface="org.apache.dubbo.samples.basic.api.DemoService3" protocol="grpc"/>
<dubbo:service interface="org.apache.dubbo.samples.basic.api.DemoService4" protocol="grpc"/>

<dubbo:service interface="org.apache.dubbo.samples.basic.api.DemoService0" protocol="dubbo, grpc"/>
Copy the code
  1. Application A on the consumer side
<dubbo:reference protocol="dubbo" interface="org.apache.dubbo.samples.basic.api.DemoService1"/>
<dubbo:reference protocol="dubbo" interface="org.apache.dubbo.samples.basic.api.DemoService2"/>

<dubbo:reference protocol="grpc" interface="org.apache.dubbo.samples.basic.api.DemoService0"/>
Copy the code
  1. Application C on the consumer side
<dubbo:reference protocol="grpc" interface="org.apache.dubbo.samples.basic.api.DemoService3"/>                                                                                     <dubbo:reference protocol="grpc" interface="org.apache.dubbo.samples.basic.api.DemoService4"/>

<dubbo:reference protocol="dubbo" interface="org.apache.dubbo.samples.basic.api.DemoService0"/>
Copy the code

3.1.1 Dubbo Multi-protocol support Status

Dubbo currently supports protocols such as Dubbo, REST, Thrift, gRPC, JsonRPC, and Hessian, which basically covers most of the mainstream RPC communication protocols in the industry. Support for these protocols is in the form of direct integration with the official Release implementation.

In addition to the above officially supported protocols, Dubbo’s flexible extension mechanism makes it easy to extend protocols for Dubbo, and developers can always add more protocol support to Dubbo, including their own protocol extensions.

3.1.2 Problems solved by multiple protocols

  • Seamlessly integrate the RPC framework into Dubbo’s service governance architecture.

    Through protocol extension, RPC protocol is incorporated into Dubbo service development system, so that Dubbo programming model, service discovery, traffic control and other capabilities can be reused.

  • Meet the call requirements of different scenarios.

    Each service may be developed to meet different business requirements, and the technology stack applied by the peripheral consumer end may be diverse. By enabling different communication protocols, the communication requirements of different scenarios can be optimized.

  • Implement migration between protocols.

    By supporting multiple protocols, you can quickly meet the requirements of intra-company protocol migration. For example, the Dubbo protocol is upgraded from its own protocol to the Dubbo protocol.

3.2 Multi-registry

When the service cluster scale is small, a centralized cluster deployment solution can solve our business problems well. However, with the growth of the application scale and the increase of user traffic, it is necessary to consider the introduction of trans-regional and multi-cluster deployment scheme for the business system. At this time, registry clusters closely related to the business system also face the selection of deployment scheme:

  1. Continue to maintain the globally shared registry cluster. The advantage of this architectural approach is simplicity; The disadvantages are that the registry cluster needs to store the full amount of address data, so the storage and push pressure will be great. In addition, some registry products (such as Zookeeper) may face stability and performance challenges in the scenario of cross-cluster network deployment.
  2. Each business cluster deploys a separate registry cluster. The advantage of a multi-registry cluster is that it can solve the problem of network availability across the cluster and reduce the storage and push pressure of the registry. The disadvantage is that the service framework (such as Dubbo, etc.) is required to be able to publish/listen to multiple registry clusters simultaneously.

Let’s take a closer look at Dubbo’s solution for a multi-registry cluster scenario.

The figure above shows two business clusters deployed in Beijing and Shanghai respectively. Each business cluster has its own independent registry cluster to solve the problem of transparent RPC communication between the services of the two business clusters.

  1. Service provider side, dual registry publishing
<dubbo:registry id="beijingRegistry" address="zookeeper://${zookeeper.address1}" default="false"/>                                                                           <dubbo:registry id="shanghaiRegistry" address="zookeeper://${zookeeper.address2}" />
                                                                                          
<dubbo:service interface="org.apache.dubbo.samples.multi.registry.api.HelloService" ref="helloService" registry="shanghaiRegistry,beijingRegistry"/>
<dubbo:service interface="org.apache.dubbo.samples.multi.registry.api.DemoService" ref="demoService" registry="shanghaiRegistry,beijingRegistry"/>
Copy the code
  1. Service consumer side, do single/dual registry subscription according to consumer demand
<dubbo:registry id="beijingRegistry" address="zookeeper://${zookeeper.address1}" default="false" preferred="true" weight="100"/>                                                                                         <dubbo:registry id="shanghaiRegistry" address="zookeeper://${zookeeper.address2}" default="true" weight="20"/>

<dubbo:reference interface="org.apache.dubbo.samples.multi.registry.api.DemoService"/>

<dubbo:reference  interface="org.apache.dubbo.samples.multi.registry.api.DemoService" registry="beijingRegistry, shanghaiRegistry"/>

<dubbo:reference interface="org.apache.dubbo.samples.multi.registry.api.HelloService" registry="beijingRegistry"/>

<dubbo:reference interface="org.apache.dubbo.samples.multi.registry.api.HelloService" registry="shanghaiRegistry,shanghaiRegistry"/>
Copy the code

3.2.1 Dubbo support for heterogeneous registry clusters

Although we do cluster multi-registry deployments, we usually deploy the same registry products, such as Zookeeper, Nacos; For registry migration scenarios, Dubbo is required to provide support for more registry products or, most importantly, to have good scalability. Dubbo currently supports the following registry implementations:

One important point to note here is that Dubbo’s service registration/discovery model is currently interface-grained, while starting with version 2.7.5, Dubbo has introduced an application-grained service registration/discovery model. On the one hand, this helps to optimize Dubbo’s current service discovery mechanism and increase service capacity; on the other hand, it is also very important for China Unicom’s micro-service system represented by SpringCloud (more on this in the next chapter). Stay tuned for more information on Application Granularity Service Discovery: Service Introspection, which we will supplement in upcoming articles and documents.

3.2.2 Traffic scheduling Problems Caused by multiple Subscriptions

With the introduction of multi-registry clustering, Dubbo added a layer of load balancing between registries in traffic location:

At the level of Cluster Invoker, we support the following location selection strategies (version 2.7.5+, please refer to the document for specific use) :

  • Assign priority

    <! -- Addresses from preferred= "true" registries will be selected preferentially and only Fallback to other registries if the center has no available addresses -->
<dubbo:registry address="zookeeper://${zookeeper.address1}" preferred="true" />
Copy the code

  • Zone with priority

    <! -- The location will match the zone key in the traffic, and the traffic will be preferentially sent to the address of the same zone.
<dubbo:registry address="zookeeper://${zookeeper.address1}" zone="beijing" />
Copy the code

  • Weight polling

    <! -- Addresses from Beijing and Shanghai clusters will be allocated 10:1 traffic -->
<dubbo:registry id="beijing" address="zookeeper://${zookeeper.address1}" weight="100" />
<dubbo:registry id="shanghai" address="zookeeper://${zookeeper.address2}" weight="10" />
Copy the code

  • By default, stick to can be used arbitrarily

3.2.3 Multiple Registries Scenario

  • Traffic in the same area is scheduled first

    To meet disaster recovery or service scalability requirements, services and applications are often deployed in multiple independent equipment rooms or regions. If each region has an independent registry cluster, traffic scheduling in the same region can solve latency and availability problems.

  • Registry Migration

    A company’s services may have been stored in one registry, such as Zookeeper, but at some point, for various reasons, when we move to another registry, the multi-registry model ensures a smooth migration.

  • Heterogeneous System Communication

    The services developed by different microservice systems are enclosed in their own service discovery systems, and the services of different systems can be discovered each other through a unified multi-registry model.

4. Make use of Dubbo Unicom heterogeneous micro-service system

Use Dubbo for interworking solutions. First of all, understand what kind of scenario unicom heterogeneous micro-service system is.

As shown in the figure above, some of our microservices can be built based on SpringCloud, gRPC, K8S or self-built systems, which are isolated from each other by default and cannot be connected. When we build a set of micro-service system based on Dubbo, we can make use of Dubbo’s multi-protocol and multi-service discovery model to achieve the interconnection between each micro-service system. Further, as shown by the orange arrow in the figure, relying on the Dubbo system as the bridge layer, we can also realize the connection between two heterogeneous microservice systems.

4.1 Protocol Migration in the Dubbo System (Coexistence)

Dubbo as a micro-service development framework and Dubbo as RPC protocol are two concepts, which can be completely separated. For example, the business system developed with Dubbo framework can use REST and gRPC communication. Specific protocols are most appropriate for business characteristics and technology planning.

At present, in the context of cloud native and Mesh, HTTP1/2 and gRPC protocols begin to receive more and more attention. On the one hand, it is natural that they do better in standardization, get more network equipment and infrastructure support, and have better versatility and penetration. For many enterprises that are willing to move to cloud native, moving to such protocols will undoubtedly help with future architecture upgrades.

The following figure illustrates an intermediate state of migration from the Dubbo protocol to the gRPC protocol in a Dubbo architecture.

  • The leftmost represents older applications that have not yet been migrated and that still consume and provide Dubbo protocol services during the migration process.
  • In the middle are applications that are being migrated. Some of them may be service providers that provide Dubbo protocol services to the older systems on the left. To provide gRPC service for the new system on the right; So they are both dual-protocol exposure services.
  • On the far right are newly developed or migrated applications that can communicate fully with the gRPC protocol.
  • Finally, after passing the intermediate state, we expect all applications to reach the state of the leftmost application to achieve complete gRPC protocol communication.

4.2 Spring Cloud System Migration to Dubbo System (Coexistence)

Due to the problem of the service discovery model between SpringCloud and Dubbo, the address exchange between the two systems requires the corresponding adaptation on the Dubbo side.

Some applications in the Dubbo system are the key nodes of the transparent connection between the two systems. Some service provider applications should be issued by dual protocols, and some consumer applications should be consumed by selected protocols. Since the old Spring Cloud architecture does not allow any changes, the key to connecting the two systems is the REST protocol. For Dubbo side applications:

  • Some applications may consume SpringCloud’s services using REST protocols;
  • Some applications may expose REST protocols for SpringCloud consumption;
  • Dubbo’s own system through their own protocol communication, here is more flexible, can be Dubbo, REST, gRPC, and so on any of them. If the REST protocol is selected, the connection to the SpringCloud architecture becomes more natural because the protocols are unified at both ends.

For applications that consume Spring Cloud services, configure the service:

<dubbo:reference interface ="xxx.SpringService" protocol="rest"/>
Copy the code

For applications that provide services for consumption on the Spring Cloud side, specify that the service is exposed to REST or dual-protocol exposure (because if the service is also called by applications within the new architecture) :

<dubbo:service interface="xxx.NewService" protocol="rest,dubbo"/>
Copy the code

Dubbo’s multi-protocol, multi-registry model provides the same flexibility for bidirectional migration, whether migrating from Dubbo to SpringCloud or SpringCloud to Dubbo.

4.3 Migrating a Self-Built System to a Dubbo System (Coexistence)

This scenario is somewhat similar to the SpringCloud migration described in the previous section, with the major difference being that the REST protocol is officially supported by Dubbo by default, while for existing proprietary communication protocols within the microservices architecture, You need to extend the Dubbo Protocol to provide support at the Protocol level. For details about how to extend the Protocol, see the following documents: juejin.cn/post/696423…

5. Summary and outlook

In order to realize the coexistence or migration between heterogeneous microservice systems, the key point is to get through the discovery of protocols and services between heterogeneous systems. Thanks to Dubbo’s support of multi-protocol and multi-registration model, Dubbo can easily become the middle layer of bridging heterogeneous microservice system.

yourLike and followIs the continuing power of the Solomon_ Shogo shell structure.

Hot historical Articles

  • 🔥Serverless Microservices elegant shutdown practices

  • 🔥 this algorithm can not understand! How are the 9 images presented

  • 🔥SpringBoot Mastery – Custom Condition annotations (Series 1)

  • 🔥Java is how to take off the beautiful woman’s clothes

  • 🔥 High-performance gateway was originally designed this way

  • 🔥REST FUL look still don’t understand, you play me!

  • How much 🔥Serverless affects programmers

  • 🔥 How does distributed transaction XID connect all microservices in tandem

  • 🔥 Microservices Distributed Transaction TCC core implementation

  • 🔥 hundreds of millions of traffic site performance optimization methodology steps

  • 🔥 microservice Nacos implements proximity access through CMDB to improve performance

  • 🔥 Micro service architecture DNS service registration and discovery mechanism

  • . More and more