The author | Apache Dubbo source | alibaba cloud native public number

  • In 2011, alibaba B2B team decided to open source the project, and within a year, it gained a large number of users from different industries

  • In 2014, Dubbo took a break from updates due to team changes

  • Dubbo was rebooted as an open source in 2017

  • In 2019, Dubbo graduated from the Apache Foundation in just 15 months

  • In 2020, the integration of HSF and Dubbo began within Ali

  • Dubbo 3.0 Preview will be released in March 2021

  • …

Dubbo has been open source for 10 years, from 2011 to 2021. Ten years is a long time and full of changes for the surging cloud computing field, but Dubbo has not become a forgotten “front wave” in this era, but also ushered in the comprehensive upgrade of kernel cloud native technology stack in the cloud native era. This article is an interview with Beiwei, head of service framework at Alibaba Group, who is also in charge of HSF, Dubbo and Spring Cloud Alibaba. This article looks at how the community has evolved since he took over in 2017, and how Dubbo, now in its 10th year of open source, can take the cloud native path well.

In 2017, after I started to take over the Dubbo project, we announced that we would re-maintain the project, and received a lot of positive feedback. With the in-depth understanding of the project, I found that many large domestic manufacturers, even traditional state-owned enterprises, were widely using the project, so I realized my responsibility. With the Release of Dubbo 3.0 Preview coming in March, we have prepared a series of Dubbo 3.0 Preview ebooks (see ** at the end of this article for more information) ** and take this opportunity to talk about some of the most frequently asked questions since Dubbo’s open source reboot:

  • Less than two years after graduating from Apache, what have we done?
  • How does Ali integrate Dubbo and HSF? What’s the scale?
  • What do you think of Spring Cloud and gRPC?
  • What to look forward to in 3.0?

The fastest Apache project graduated in the same period, with comprehensive development of multiple languages

Since I was in charge of this project in 2017, some developers have been worried that Dubbo is just a KPI open source project led by Ali, and whether the project can sustain development or not. The Apache Foundation was a good choice, and it was this decision that made Dubbo the fastest of the five projects to graduate from the Apache Foundation during the same period: On May 21, 2019, Dubbo graduated from the Apache Foundation after just 15 months.

Donating Dubbo to Apache and making Dubbo a community-led project proved to be the right decision. According to the latest “2020 Open Source Digitization Report in Microservices” released by X-Lab, Dubbo ranks 693 globally in open source activity and fifth among microservices frameworks. The number of developers who have participated in community building has exceeded 10,000, and the number of code contributions from outside alibaba has exceeded the number of employees’ contributions.

Data source – Report on Open Source Digitization in Microservices 2020

Less than two years after the Dubbo project graduated from Apache, the entire community has 21 PMC members, 61 committers, and up to 370 contributors. In the past year, Dubbo has obtained JS, Python, Erlang, PHP and Go implementations from the community in terms of multi-language construction. Special thanks to km, Ctrip, LETxin and other developers for donating or submitting the vast majority of these multi-language versions of the code. Brings a wealth of multilingual support solutions to the community. Under the leadership of Yu Yu, Dubbo Go is one of the most active branches of the current multilingual versions, with a very active Go sub-community and a growing number of enterprise users. At present, Dubbo Go has equalled Dubbo Java 2.7 features in version 1.5. Currently working hand in hand with Java to map out cloud native in Dubbo 3.0.

Our definition of the big version of Dubbo 3 is cloud native, Ali-endorsed. The plan is divided into three release iterations, which are described in detail in the Dubbo 3 roadmap. Dubbo 3.0 is currently under development, with core features including the Dubbo3 protocol, application-level service discovery, and new routing rules. All of these features are expected to be implemented in the Go and Java versions at the same pace. The Preview release will be available in late March.

My experience in the community over the past two years has convinced me that it’s hard to sustain a core team of engineers with a pure open source mentality. Dubbo is being used by hundreds of thousands of businesses across the country. It’s not enough just to have a small team, and we want developers in the community to get more involved. For Dubbo, it doesn’t matter why a developer came in and contributed to the project in the first place. The important thing is that we want to keep the community open, even if it’s ok for individual engineers to participate in the community just to get a job later, which I think is normal, because contributing projects take up a lot of developers’ spare time, and we want this project to help people.

Dubbo and HSF fusion, Ali using Dubbo first

The second biggest question I face is: how do developers trust Alibaba if they don’t use Dubbo internally?

Those who are familiar with alibaba’s technology history may know that Taobao’s HSF project is also a middleware service framework, which overlapped with what Dubbo did. Dubbo was also criticized by many developers for not using Dubbo itself. This is also what we have been worrying about: ali’s internal self-research system, commercialized product technology and open source projects, the technical routes of the three parties have never been integrated.

With the development of Ali’s own system, the opportunity of integration finally arrived. In 2020, Ali Cloud put forward the “Trinity” concept, that is, “self-developed technology”, “open source project”, “commercial products” into a unified technology system, maximize the value of technology.

At present, HSF is based on Dubbo 3.0, and its internal features exist in the form of Dubbo plug-in. The optimization experience of HSF, such as high concurrency and high performance in large-scale internal scenarios of Ali Group, is applied to the Dubbo 3.0 core, realizing the unification of internal and external functions. Making these quality experiences available to communities and customers; On the other hand, Dubbo 3.0 cloud native features are further developed with the help of community development. Achieve an open and win-win situation with the community through the trinity.

My old master leader Lin hao, ali said bi, due to the HSF is designed and developed this year won the award for the outstanding engineers of China computer society, he referred to in the award-winning interview: as an engineer, a great sense of accomplishment from their own, has been the use of a wide range of technologies, and for the company’s business development can have a lot of support effect; The greater sense of achievement comes from the idea and realization of the technology behind their own can influence China’s major Internet companies and enterprises to embrace micro services. With the integration of Dubbo and HSF, we have demonstrated HSF’s capabilities in the whole open source system, enabling more people to build service-oriented systems by using Dubbo just like Alibaba used HSF before.

On November 11, 2020, Dubbo 3.0 has entered the stage of implementation in the group’s e-commerce business, and we will share some of our practical experience with you in the e-book.

What do you think of Spring Cloud and gRPC?

During Dubbo’s quiet years, many new service frameworks emerged. What is the relationship between Dubbo and Sping Cloud? Is one of the two choices enough? What’s the difference between Dubbo and gRPC?

How do Dubbo and Spring Cloud choose between them?

Dubbo has long been compared to Spring Cloud by developers, and the first thing that comes to mind when mentioning the two names is which one to choose, not how they work together. In my opinion, this is mainly a matter of technology selection and the user’s concern about the switching cost that comes with it. This is a misconception. It’s not an either-or relationship. Today Dubbo has become an important technology component in Spring Cloud Alibaba, and Dubbo services and Spring Cloud services can call each other perfectly. Going forward, Dubbo 3.0 further simplifies the deployment of service infrastructure in Dubbo and Spring Cloud hybrid scenarios. It is an indisputable fact that Spring Cloud has become a standard framework for Java development based on Spring, and a set of microservices common architecture pattern standards have been gradually abstracted based on a large amount of industry experience. The benefit of this standard is that it makes it very easy for developers to develop microservices software products and has become a “package” solution for developers supported by the entire Spring ecosystem.

Unlike Spring Cloud, scalability and flexibility were placed at the forefront of Dubbo’s design. Dubbo is easy to integrate with others, and others are easy to integrate with Dubbo. At the same time, Dubbo has been verified by a large number of users, and Ali has continued to practice products in the field of servitization. These are two things that Spring Cloud currently cannot do. With the renewal of Dubbo, its scene richness and stability have also been greatly improved, and it has been widely used in many head companies.

Going back to the concerns many developers have about technology selection: the two frameworks are not either. On the contrary, users can easily switch between the two frameworks and even work together perfectly in the future, thanks to the emergence of Spring Cloud Alibaba.

Spring Cloud has a strong international community. As an important member of the community, Alibaba also contributes to the implementation of Spring Cloud Alibaba, which is the most perfect and continuously updated implementation scheme in the whole Spring Cloud system.

Spring Cloud Alibaba emerges

Dubbo 2.7 now works well with the Spring Cloud. Through the integration of Dubbo in Spring Cloud Alibaba, Spring Cloud applications can invoke natively published Dubbo services, and Spring Cloud published Dubbo services can also be invoked by native Dubbo clients. This benefited from the experimental project of service introspection in 2.7 and the Spring Cloud side adaptation of Dubbo.

This experimental project has evolved into a native application-level service registry mechanism in the ongoing 3.0 major release. With this feature, future Spring Cloud applications and Dubbo applications will be better blended. Users can reuse the same set of service discovery, service configuration, and service management systems for Spring Cloud and Dubbo. Additional gateways for Dubbo and Spring will become a thing of the past. Users can switch between the two at zero cost, or choose a different framework depending on the scenario. You can even mix them in the same application. The conventional Proxy cluster in the Dubbo and Spring Cloud hybrid scenario is finally removed, and the architecture of the whole system is simpler and more stable. In version 3.0 of Dubbo, the team will continue to evolve the idea of application-level service registration, hoping that through this work, Spring Cloud Alibaba and Dubbo will achieve a high degree of unity in the model of registration data, reuse the same set of service registries, and further simplify the architecture in the mixed scenario.

In addition, our team is also actively developing the Spring Cloud Alibaba ecosystem. As one of the most influential Java teams in China, Ali middleware team has been paying close attention to the Spring project and improving ali’s middleware development experience through the encapsulation of Spring. The vast majority of applications in Alibaba’s e-commerce system have been booted.

When Spring Cloud first gained influence, it became a natural thing for us to take the initiative to integrate Alibaba open source components through Spring Cloud. Currently, Spring Cloud Alibaba has supported Nacos as a service registry and configuration center. Sentinel as a stream limiting component, Seata as a distributed transaction component, RocketMQ as a distributed messaging component, and of course Dubbo as an RPC component, completely replaced the Spring Cloud Netflix bucket, which had announced that it would stop updating. In addition, in order to accelerate the access of domestic engineers to Spring Initializr, the team also provided the ability to quickly generate Spring Cloud Alibaba applications through start.aliyun.com hosted on Alibaba Cloud.

It is no exaggeration to say that Spring Cloud Alibaba has become the de facto standard in the Spring Cloud framework, both in terms of GitHub project activity data and attention data.

What do you think of gRPC?

We’ve never shied away from gRPC, a respectable rival that is the de facto standard for communication protocols between cloud-native infrastructures.

But Dubbo has the advantage of being not just an RPC, but a service framework with strong governance capabilities. We think: Dubbo is gRPC with batteries.

One of the most valuable lessons we learned from gRPC was to reflect on the shortcomings of the protocol design in Dubbo 2 and focus on two important issues in the cloud native support space: multilingual support and gateway /Mesh parsing friendliness. In Dubbo 3.0, the new version of the protocol is the most important. In addition to solving the above two problems, the compatibility of gRPC protocol is also one of the design goals of the new protocol. GRPC has several obvious advantages: it is at the forefront of HTTP/2 protocol support, offers very rich multilingual library support, and seamlessly integrates with many of the Google-led cloud native infrastructures. GRPC and popular cloud native technologies such as Mesh have built a relatively perfect microservice technology stack, which seems to be a microservice solution based on gRPC.

But the gRPC framework itself focuses on RPC communication. In contrast, Dubbo provides a one-stop solution for microservices development and governance. Dubbo has a more user-friendly interface oriented service definition model and a more complete service discovery and governance mechanism. Dubbo 3 will also provide official Mesh solution support in the upcoming 3.0 planning, continuing to provide the community with an easy-to-use, one-stop solution.

Dubbo 3.0 outlook: Cloud native support

Many developers in the community have been waiting for version 3.0. Cloud native support is the main theme of Dubbo 3, with three iterations planned. It focuses on delivering key features such as cloud native friendly new-generation RPC protocol, application-level service registration discovery, K8s native service publishing, Mesh control surface xDS protocol docking and distributed service flexibility.

  • Version 3.0 focuses on application level service registration discovery, Tripe, and new routing rules.

  • Version 3.1 focuses on the release of K8s native service and xDS protocol connection of Mesh control plane.

  • Version 3.2 focuses on distributed service flexibility.

At present, application-level service discovery has been piloted in internal and some head user scenarios. As the project progresses, the team will release the details of functional implementation as soon as possible. With the release of Dubbo 3.0, we expect to deliver a new generation of services frameworks that are cloud native migration friendly and cloud native infrastructure friendly.

In the future, the overall development tone of Dubbo project will stick to the open source line of cooperation and pursue the line of higher quality and more perfect functions. At present, the top priority of community development is the evolution of Dubbo3.0. Dubbo 3.0 app registration discovery will be launched within Alibaba and on the open source side in September. This is not only the first step towards cloud native microservices for Dubbo, but also a prerequisite for docking K8s registration discovery and cross-framework RPC interworking.

On the application side, moving from interface-level registration discovery to application-level registration discovery can significantly reduce memory stress on registries and clients. On November 11, the cloud Native Service Governance rules will incorporate Dubbo’s best practices in large-scale, high-concurrency service governance over the years. The next generation of protocols based on Http2 / Protobuf will bring better ecology and overall support to Reactive, adaptive strategies and distributed load balancing covered by flexibility enhancements will lead to greater breakthroughs in performance and stability.

Back when Dubbo relaunched open source, the ecosystem was relatively weak. Today, Dubbo’s ecology is improving.

Dubbo rich extension implementation

For example, multilingual support has reached 6 and 30+ ecological subprojects. While actively integrating with Dubbo, we are actively integrating with third party open source projects like Spring Cloud Sleuth, Zipkin, Skywalking, Envoy, Tengine, and others.

In my mind, I hope to produce an official recommended Dubbo Stack, so that users don’t have to choose from the above. As for whether the Dubbo Stack will all come from Alibaba, I’m willing to let nature take its course. It’s still a matter of data, and we’ll recommend whichever component is most widely used in the production system. In general, the decision is up to the community and Dubbo users. Finally, thanks to 10 years of Dubbo users and community contributors, we’ve compiled a series of forward-looking articles on Dubbo’s Roadmap for 3.0, including a new generation of cloud-friendly RPC protocols, app-level service registration discovery, and K8s native service release. Welcome to reply to “Dubbo3” in the dialog box of “Alibaba Cloud native public account”.