If you look at the history of open source in China, you can’t really find another open source project that has attracted as much controversy and discussion as Dubbo.

On the one hand, in 2011, its open source filled the gap of RPC framework used in the production environment at that time, and was widely adopted upon release. On the other hand, it went through the process of stopping maintenance, restarting maintenance, donating to the Apache Foundation, and then graduating as a top project.

In the face of skeptical developers, how will Apache Dubbo continue to shine in the cloud native era?

This year marks the one-year anniversary of Dubbo’s graduation from the Apache Foundation, and it’s also an important year for Dubbo 3.0, the full embrace of cloud native. Open Source China and Apaceh Dubbo are co-curating a series of articles to review the development of the Apache Dubbo community. This series of articles covers Dubbo technology, community, and case studies, and will be posted weekly.

Leave a message on [Alibaba Cloud native public account] to tell the story of Apache Dubbo, the top ten students who like can receive an exclusive trophy cup sent by Dubbo; In addition, Apache Dubbo PMC@chickenlj randomly selected one lucky reader, the value of 260 yuan eye lamp. The drawing is next Wednesday.

Author’s brief introduction

Liu Jun, pseudonym Luturtle, GitHub account Chickenlj, Apache Dubbo PMC, the core developer of the project, witnessed the whole process of Dubbo restarting open source and graduating from the Apache Foundation. Now I am working in aliyun cloud native application Platform team, participating in service framework and micro-service related work, and mainly promoting Dubbo 3.0-Dubbo cloud native.

The beginning of the series: 3.0 fully rolled out, the first anniversary of ASF graduation

In the year since 2019, Dubbo has made great strides in both the community and the product, as well as in the development of Dubbo 3.0, Dubbo’s cloud-native version.

Community. There are two points that need to be highlighted: one is the further increase in the number of enterprise users who land and contribute to the community, and more than 200 medium-sized and large-scale companies have taken the initiative to get in touch with the community, such as Ctrip, ICBC, Guazi used car, Netlink Clearing, Zhongtong, etc.; Another subcommunity, represented by Dubbo-Go, is thriving.

Product technology evolution. Dubbo Java edition is available in 10 versions, with in-depth exploration of multiple languages, protocols, performance, service governance models, and more. Dubbo Go has been released in more than eight versions, and is already ahead of the Java version in some directions, while its functionality is largely aligned with the Java version.

It is worth mentioning that Alibaba is also actively promoting the implementation of Dubbo community version internally, and starting from this year, it will gradually replace its internal HSF framework with Dubbo. On the one hand, this will help Ali to return its rich service governance experience in HSF to the community, and on the other hand, the official landing of Ali will directly accelerate the development of Dubbo cloud native.

Under the tide of cloud native, 3.0 has been officially listed as the core goal of Dubbo product construction this year, involving the next generation RPC protocol, service governance model, cloud native infrastructure adaptation and many other aspects. Many of these areas have already been explored in 2.7, such as HTTP/ 2-based protocol support, application level service discovery, etc., which will be used as a foundation for further work. There will also be a detailed explanation of Dubbo 3.0’s Roadmap and technology solutions.

The first anniversary of Dubbo’s graduation

In July 2017, the Dubbo open Source project was reactivated and donated to the Apache Foundation in 2018. In May 2019, Dubbo officially graduated from the Apache Foundation incubator and became an Apache Top level project. Next, the paper introduces Dubbo’s achievements in the past year from three aspects: community, sub-community and product.

The community releases 24 versions a year and has over 300 contributors

If the initial reactivation was a project maintenance investment led by Alibaba, since Dubbo joined Apache, it has become a fully open foundation project led by the community and based on community contributions.

Today, this trend is becoming more pronounced. Internet and traditional enterprises, including Alibaba, Ctrip, ICBC, Guazi Used Car, Netlink Clearing and ZTO, have invested in the use of Dubbo and community code contribution. The Dubbo community is becoming very active and diverse.

In the past year, Dubbo community project has released a total of 24 releases with 27 committers/PMCS, among which 20% of contributors are from Alibaba and over 80% are developers or enthusiasts from different organizations.

Dubbo community has organized more than 10 offline meetup events, basically covering the cities where developers gather in China. Share speeches on more than 100 topics through offline or online live events, and explain in depth the latest developments, development of functional modules and recent planning of Dubbo community. Most of the keynote speeches are collected in the community, and Dubbo’s in-depth enterprises share their practical experience, among which typical representatives include Ctrip, INDUSTRIAL and Commercial Bank of China, Kora, And Credit Computing power.

According to GitHub statistics, the number of Dubbo stars has reached a new milestone, exceeding 30,000, an increase of nearly 5 times compared with the restart of open source. The number of contributors has grown from dozens to more than 300, of which more than 60 have been nominated as committers. Both the number of contributors and the percentage of committers have increased greatly. Dubbo Java has 65 releases.

While the above is a summary of the development of the Dubbo Java project community, the following sections describe the progress of the Dubbo Java product.

Dubbo Java iteration, currently maintaining 3 major versions

There are three major Dubbo Java releases currently maintained by the community: 2.5.x, 2.6.x, and 2.7.x.

2.7.x is the major development release of the community. In the past year, there have been 8 releases (2.7.0-2.7.7). Each release has some noteworthy feature or function upgrades, covering enhancements from programming model, service governance, performance and protocol.
The 2.6.x version is positioned as bugfix version. Three versions have been released in the past year, mainly fixing problems and security vulnerabilities without adding too many new features.
Version 2.5.x has announced EOF since early 2019, only security fixes; By the second half of the year, maintenance had stopped altogether.

Here is a brief hierarchical module diagram to review the evolution of Dubbo’s technical architecture over the past few years, from the perspectives of programming model, service governance, transport protocol, performance optimization, etc. :

Many of the above functions have been implemented by major manufacturers to solve specific business problems. We also look forward to more in-depth reviews of Dubbo practices from these vendors.

Dubbo- Go, in its fifth year of development, is keeping pace with Dubbo

In addition to Dubbo Java, there are also many excellent sub-projects (sub-communities) developed around Dubbo, including Dubbo-Spring-boot-Project, Dubbo-Go, etc. Here we will focus on the dubbo-Go sub-community.

Dubbo-go project was first built in May 2016 due to rain, and released in September of the same year. The timeline below clearly records the history of Dubbo-Go.

Adhering to the natural mission of “Bridge the Gap between Java and Go”, Dubbo-Go has entered its fifth year and also stepped out of its own unique development path:

The current version of v1.4.0 is aligned with version 2.6.x, the upcoming version will be aligned with v2.7.x [benchmark v2.7.5], and then the v1.6.0 version of Dubbo 3.x will be released.
Independently maintain the full set of implementation from the bottom hessian2 protocol library Dubo-Go-Hessian2, network library Getty to the top Dubbo standard Dubbo;
Separate TCP + Protobuf and gRPC + JSON communication schemes have also been developed [to be included in v1.5.0];
Interworking with Dubbo/gRPC/Spring Boot;
By connecting Opentracing and Promethus, Dubo-Go made its own unique exploration in the direction of observability and other microservices.
The trusted RPC call based on HTTPS has been implemented.
Has implemented its own unique microservices solution with Kubernetes as the registry;

Dubbo-go has developed from the go language implementation of Dubbo to the most powerful multi-language version of Dubbo. Its development cannot be separated from the powerful Dubbo-Go community behind it. In addition to the above characteristics of Dubo-Go, the following achievements have been achieved through cross-community cooperation:

Through cooperation with the MOSN community, Dubbo/ Dubbo-Go applications can access Dubbo Mesh based on MOSN at zero cost, realizing the coexistence of microservices and cloud native “dual-mode microservices”.
Working with the Sentinel community to fully access Sentinel’s downgrading and limiting scheme at Dubbo/ Dubbo-Go;
Working with the Apollo community to implement remote configuration delivery in Dubo-Go;
Work with the Nacos community to implement NACOS-based service discovery;

Dubo-go community’s main goals for Q2 2020 are:

Release v1.5.0 fully aligned with Dubbo 2.7.x;
Release v1.6.0 for Dubbo 3.0;
Continue your exploration in cloud native;
Continue to cooperate with brother communities to expand the scope of their use;
Production practice to promote in Ali Group, as well as more manufacturers landing.

The project (including sub-projects) has been produced by Ctrip, Doodle Intelligence, Ant Financial and other companies.

After alibaba group completes the integration of HSF and Dubbo this year, the project will also stand the test in alibaba Group’s Double eleven battlefield.

Cloud native Dubbo – Dubbo 3.0

3.0 is the code name for the next generation Dubbo architecture. Dubbo 3.0 was also widely discussed in the community a year ago when Reactive Stream was first explored. This time, in the context of cloud native, 3.0 represents a more comprehensive upgrade of the Dubbo architecture, involving the next generation RPC protocol, a new service governance model and cloud native infrastructure adaptation.

Alibaba is one of the main forces involved in the development and construction of Dubbo 3.0, an open source project that started at Alibaba and is now coming back to Alibaba.

Since last year, Alibaba has been gradually promoting the replacement of its internal HSF framework with Dubbo, by integrating Dubbo and HSF, and developing a version of Dubbo adapted to the cloud native architecture on this basis. Dubbo’s return to Alibaba is a good practice of embracing community, cloud origin and standardization.

The implementation of Alibaba’s internal Dubbo 3.0 is also a great benefit to the community. On the one hand, it will help Alibaba to feedback and export its rich service governance experience in HSF to the community, and on the other hand, it will directly promote the rapid evolution of Dubbo cloud native architecture. In addition to Alibaba, douyu, INDUSTRIAL and Commercial Bank of China, IQiyi, Douyu and other manufacturers are also participating in the construction of next-generation Dubbo 3.0.

There are three important Roadmap points for Dubbo 3.0 that will be explained separately in the following article:

Next generation RPC protocol. The new protocol will provide richer built-in semantics such as Stream and Flow Control, as well as better scalability and gateway friendliness.
Service discovery mechanisms based on application granularity. On the basis of considering the interface oriented ease of use and functionality of Dubbo, it solves the adaptation problem with Kubernetes Native Service and solves the performance bottleneck of address push in large-scale clusters.
Solutions for cloud native infrastructure. This involves the connection between Dubbo Service and infrastructure life cycle, the adaptation of Kubernetes Native Service, Service governance rules adapted to infrastructure scheduling, and solutions adapted to Service Mesh architecture, etc.

The following is a brief expansion along these three aspects.

Next Generation RPC protocol

The next generation of protocols will focus on HTTP/2, Reactive Stream, Flow Control, etc.

Reactive Stream: Reactive Stream introduces RPC to support richer communication semantics and API programming models, such as Request-Stream and Bi-stream.
**HTTP/2: ** Communication protocol built based on HTTP/2 has better versatility and penetration in the native scenario of micro-service cloud;
**Flow Control: ** Protocol built-in Flow Control mechanism, support similar to Reqctive Stream Request (n) Flow Control mechanism.

In terms of business scenarios, Dubbo, based on the new protocol, should support intelligent decision-making load balancing algorithm, be more friendly to Mesh and gateway, and provide multi-language implementation and interworking more easily at the framework level.

**Mesh: ** protocol is more friendly to Mesh penetration and distinguishes protocol header Metadata from RPC Payload, facilitating cooperation with the Mesh, including traffic control mechanism and application layer configuration negotiation.
** Protocol versatility: ** gives consideration to versatility and performance, supports protocol can run on various devices;
** Multilingual support: ** Such as improved cross-language service definition and serialization transport support through support for Protobuf.

Application level service governance

Interface orientation has always been the strength of the Dubbo framework. On the one hand, its ease of use, for developers to shield the existence of remote calls; On the other hand, interface-oriented address discovery and service governance bring more powerful capabilities, making the whole Dubbo governance system very powerful and flexible.

Given the benefits of interfaces, why explore application-oriented service governance patterns?

Sounds like a contradiction. Whether it’s interface oriented or application oriented is just a different way of looking at Dubbo. The transformation of “interface -> application” we are talking about is mainly reflected in service registration and discovery:

The new application-oriented model focuses on the second point, the registry’s data organization, shifting to an application-oriented/instance granularity. This solves two problems for us:

Align with microservice models such as Kubernetes Service at the Service discovery level;
The amount of data discovered by the service will decrease by an order of magnitude from “number of interfaces * number of instances” to “number of applications * number of instances”.

See the article Dubbo takes a big step toward Cloud Native: Application-level Service Discovery Parsing, and there will be more in-depth explanations of this mechanism and implementation later in this series.

Cloud native infrastructure

Cloud native brings comprehensive changes in underlying infrastructure, application development, deployment, and operation and maintenance:

infrastructure

Changes in infrastructure scheduling mechanism bring changes in operation and maintenance (life cycle) and service governance.
Service Discovery capability sinks. Kubernetes abstracts Native Service Discovery.

Service Mesh – Cloud native microservices solution

Mesh provides solutions for cross-language and SDK upgrades. Dubbo SDK collaborates with Mesh to achieve convenient adaptation of functions, protocols and service governance.
Mesh has not been rolled out on a large scale, and it is more suitable for applications that pay more attention to traffic control. The performance advantages of traditional SDK still exist, and the mixed part migration scenario may exist for a long time.

In terms of application scenarios, possible deployment environments for Dubbo include:

Instead of using Kubernetes Native Service, Kubernetes only serves as a container scheduling facility, and continues to use Dubbo’s self-built Service registration and discovery mechanism.
Dubbo is no longer concerned with Service registration. Dubbo Client is responsible for Service discovery and traffic allocation.
On the one hand, Dubbo SDK should adapt to the Mesh architecture and become RPC programming and communication scheme under the Mesh system. On the other hand, Dubbo and Mesh architecture should coexist for a long time to get through the service discovery and governance system.
Kubernetes on – and off-cloud hybrid deployment smooth migration support, including unified service discovery and network communication solution open.

In terms of Dubbo functions, support for cloud native infrastructure will be provided in the following aspects:

Life cycle: Dubbo is bound with Kubernetes scheduling mechanism to keep automatic alignment of service life cycle with Pod container life cycle;
Governance rules: Service governance rules are optimized in terms of rule body and rule format, for example, rule body is described by YAML, the direct dependence of filtering rules on IP is removed, and the CRD resources unique to rules are defined.
Service discovery: supports Service discovery of K8S Native Service, including DNS, API-Server, and xDS.
Mesh architecture collaboration: Build the next generation HTTP/ 2-based communication protocol, support xDS standardized data delivery.

A new generation of RPC protocols and an application-level service discovery model will be the foundation for this part.

Summary and Prospect

To kick off this series of articles, here is a brief summary and review of Dubbo’s achievements over the past year, including the development of the Dubbo community and product iterations. We’ll see more stories about landing from deep Dubbo users, the development of the Dubbo-Go sub-community, and more. More importantly, we’ve outlined the Roadmap for Dubbo – Dubbo 3.0 for the next generation of cloud native, with a Roadmap and Roadmap for Dubbo 3.0 to follow in this series.

Can Apache Dubbo, now ten years old, ride the wave again?