2019 Aliyun Summit · Beijing Developers Special Session
The man of chivalry is for his country and people.
In Jin Yong’s novels, Guo Jing and Huang Rong are examples of “chivalry masters, serving the country and the people”. They helped the Song army defend Xiangyang for more than ten years by wearing cloth.
In the world of technology, there is no such thing as size.
But when a group of programmers move from a service company to a community of programmers, collaborating, improving, and using technology in an open source way with developers in the community, they are given greater responsibility and expectations.
The programmers of Ali Cloud intelligent middleware are working with the developers of the community to knock down the past and future of the domestic micro-service open source project.
The first Apache community top level project in the country that is not a Hadoop ecosystem
In 2016, the RocketMQ team implemented a low-latency storage solution for the first time in support of Singles Day, which survived the traffic test, with 99.996% latency falling below 10ms throughout the promotion period, achieving the stated goal of ensuring transaction stability.
For a distributed messaging engine with a nearly even read-write ratio, this technological breakthrough, even on a global scale, is absolutely laudable.
The RocketMQ team and community of developers
On the other hand, after three months of open source remodeling, the RocketMQ team is on the road to donating to the Apache Software Foundation, but it’s not an easy step.
“At that time, the open source atmosphere in China was not as active as it is now. After open source, many design documents, code quality and community building were not ideal. We always expect that domestic open source projects and open source communities can also play a remarkable value on the world open source stage, and hope that more open source projects “made in China” will become world-class open source projects.”
Ali Cloud intelligence senior technical expert Feng Jia recalled.
After nearly a year of efforts, on September 25, 2017, Apache Software Foundation officially announced that RocketMQ, an open source project donated by Alibaba to the Apache community, has officially graduated from the Apache community and become Apache Top Level Project (TLP). This is the first Apache community top level project in The country that is not part of the Hadoop ecosystem.
It’s worth noting that, according to pre-graduation calculations, 80 percent of RocketMQ’s new features and ecosystem integration came from community contributions.
A little over a year after graduation, RocketMQ has covered more than 60% of messaging scenarios in Internet finance and other fields, and has been used in more than a dozen industries including finance, power, logistics, gaming, e-commerce and shared travel. However, as the cloud computing, big data, such as artificial intelligence technology in the global scope was deepened, spawned such as IoT, chain blocks, AI, edge new application scenarios, such as the computing architecture on how to further evolve to better adapt to the new scene, good service over the next decade, this is the upcoming RocketMQ 5.0 to solve the problem.
Milestones in the messaging world
RocketMQ’s graduation from the Apache community coincides with another milestone in the messaging industry. OpenMessaging, the international standard for distributed messaging, has officially entered the Linux Foundation. This is the first international standard for distributed computing launched globally in China.
Messaging has become a key part of modern data-driven architecture, but there are still two major problems in messaging on a global scale:
First, the lack of vendor-neutral industry standards leads to high complexity and incompatibility of various message-oriented middleware, which in turn leads to product inefficiencies, confusion, and vendor lock-in for companies.
Second, the existing scheme framework is not well adapted to cloud architecture, that is, non-cloud native architecture, so it cannot effectively provide technical support for emerging business requirements such as big data, streaming computing and Internet of Things.
This is RocketMQ, during the process of open source developers and partners often noted that problem: “in the field of news, appeared on the market all kinds of different open source solutions, this led to higher user access and maintenance costs, in order to ensure the normal communication between each messaging engine, but also spend a lot of energy to do compatible.”
At this point, the need to establish a set of vendor-neutral, language-neutral de facto standards for messaging became a common aspiration among community members.
Jiang Jiangwei, Intelligent researcher of Ali Cloud & Dan Kohn, CEO of CNCF
After that, Alibaba launched the OpenMessaging project and invited Yahoo, Didi Chuxing and Streamlio to participate in it. A year later, more than 20 enterprises participated in the OpenMessaging open source standard community. Including Alibaba, Datapipeline, Didi Chuxing, Howhi Technology, JINGdong Mall, IFlytek, QingCloud, Streamlio, VIPKID, WeBank, Yahoo, China Mobile Suzhou R&d Center, etc. (in alphabetical order), It is also supported by top messaging open source vendors such as Apache RocketMQ and Apache Pulsar.
Compared to open source a distributed message project, a set of open source standards can be accepted by various vendors, for the entire domestic open source field, is a more milestone event.
From microservice framework to microservice ecology
Dubbo is a distributed service governance framework that alibaba opened source in 2012. It is one of the most influential and widely used open source service frameworks in China. In the most popular Chinese open source software awards launched by open source China in 2016, 2017 and 2018, it has entered the Top10 list for three consecutive years. In February 2019 Dubbo released 2.7.0, which will be used for the official graduation of the Apache Foundation. (Near Graduation)
The Dubbo team and the community developers
What impact does graduation from Apache Incubator have on the maintenance and development of the project, besides having a reputation?
“It’s an honor to graduate from an incubator, but it’s not the end, it’s the beginning of something else. It’s a bit like going to school. Graduation doesn’t mean the end of your learning, but the beginning of your greater social value. Graduation is also more of a rite of passage, meaning that the Dubbo team has met Apache’s requirements for a mature open source project and is beginning to develop independently.”
Ali Cloud intelligence senior technical expert North Wei in an interview with the media, answered.
From Dubbo’s home page
To date, Dubbo has earned 2.5W + STAR, ranking in the top 10 of all Java projects on GitHub, and an increasing number of enterprise users are choosing Dubbo as their microservices governance framework. However, with the gradual deepening of micro-services, the capabilities provided by Dubbo are gradually unable to meet the needs of various aspects of micro-services.
Alibaba cloud intelligent technology expert Wang Tao shared in a live broadcastDubbo is a microservices framework that helps developers quickly build high-performance microservices applications. However, in terms of API Gateway, fusing current limiting, distributed monitoring, distributed transactions and other aspects, there is a lack of a relatively complete solution around Dubbo, which is basically researched by each company or needs to investigate various open source frameworks outside for research and selection. A large amount of time and energy are spent on this. It doesn’t have a systematic solution.”
As a result, we made a further evolution, from a microservice framework to a microservice ecosystem.
By integrating with mature open source solutions, a complete Ecosystem of microservices is formed, forming the Dubbo Ecosystem, enabling developers to rapidly develop microservices applications without making too many changes to existing systems.
The Dubbo Ecosystem concept came about because of two microservices components that opened in the summer of 2018.
Midsummer night for techies
In the summer of 2018, the domestic open source field welcomed two new members.
As two important open source components in microservices and cloud native ecosystem, Nacos focuses on dynamic service discovery, configuration and service management in cloud native applications, while Sentinel focuses on stream limiting and degradation.
Both Nacos and Sentinel are generated from the core business scenarios of Alibaba in the past 10 years. Their open source technology is an effective supplement to the open source technology solutions in the field of micro services and meta-native. At the same time, they also emphasize the integration into the open source ecosystem. Also supports ecosystems such as SpringCloud and Kubenetes to enhance their vitality.
“Alibaba as early as in 2007 upgrading from IOE centralized application architecture for distributed architecture of service, the Internet is realized in a distributed environment, such as distributed services management, disaster data source switching, different live, plans, and current limiting rules scenarios such as configuration changes difficult problem, because in a large distributed system, You can’t just shut down an entire distributed system and do a software, hardware, or system upgrade.”
Ali Cloud intelligent senior technical expert Kun Yu in THE QCon scene shared.
Nacos Map
Compared with other open source solutions of service configuration center, Nacos started a little later, but in addition to the configuration center, it also provides dynamic service discovery, service sharing and management functions, and has better performance in large-scale scenarios, easier to use, and more flexible in distributed deployment.
Nacos supports a variety of startup modes, and users can choose to deploy each function module, such as registry and configuration center, separately or in combination according to the business scenario, which can fully support the growth of small startups to large enterprises and the evolution of the full life cycle of micro-services.
So far, more than 40 companies have deployed Nacos in production environments, with Tiger Teeth Being one of the first to introduce Nacos into production on a large scale.
Tiger teeth share the scene @ Guangzhou
“Tigress has been interested in Nacos since V0.2, and we have been involved in building the community, so to speak, early enterprise users. Before introducing Nacos, we also compared Spring Cloud Config Server, ZooKeeper and ectD. After an overall evaluation, based on the current situation of our micro-service system and business scenarios, we decided to use Nacos as a service registration and service discovery scheme in the transformation of services. We found that as the community version was updated and tigress became more practical, Nacos had more advantages than we found in our research.”
“Zhang Bo, head of the middleware team at Huya’s basic Security Department, shared at a developer event.
Coincidentally, on one side Nacos announced open source and was included in the CNCF Cloud native panorama, on the other side Netflix Eureka announced to stop open source investment under the Spring Cloud ecosystem, the game of the brave is full of variables, but in the view of the Nacos team, Nacos is just one of many open source projects at Alibaba. There will be more open source projects that feed back into the community and create an ecosystem, such as Sentinel, a lightweight streaming and downgrading component.
Aliware Open Source• Shenzhen station site
On July 29, 2018, AliwareOpen Source• Shenzhen site, which can only accommodate 400 people, more than 700 developers came to the site. Zijin, senior technical expert of Ali Cloud Intelligence, announced the open source of Sentinel, a lightweight stream limiting demotion component, at the scene.
Sentinel has gone through the test of double 11 in 10 years, covering all the core scenes of Ali, thus accumulating a large number of traffic integration scenes and production practices. Sentinel could not have come about without the concerted efforts of alibaba’s high availability architecture team.
“Capacity planning is one of the most important and challenging aspects in the preparation for Double 11. From the first year, the zero hour of Singles’ Day represented our highest ever business traffic, which is often dozens or even hundreds of times the daily traffic. Therefore, how to make a distributed site with continuous technical and business complexity to support the sudden traffic shock more smoothly is a problem we have been working on for the past 10 years.”
You Ji, senior technical expert of Ali Cloud intelligent high availability architecture team, shared after a Double 11 preparation.
In the past 10 years, capacity planning has gone through five stages including manual estimation, offline pressure measurement, online pressure measurement, full-link pressure measurement, full-link pressure measurement, isolation environment and elastic expansion. After the end of Double 11 in 2013, the birth of full-link pressure measurement solved the deterministic problem of capacity. As an epoch-making technology, the realization of full-link pressure measurement is a milestone event for the whole group.
In 2014, the HIGH Availability Architecture team won the Group CTO Award
Subsequently, based on the whole link pressure measurement as the core, to create a series of capacity planning related supporting ecology, while improving capacity, reduce the cost of the whole link, improve efficiency. As capacity planning technology continues to evolve, starting in 2018, the HIGH Availability Architecture team wanted to contribute these years of practice in a production environment to the community, and Sentinel became open source.
Just two months after open source, Sentinel has been included in the Cloud Native Panorama, located in the Choreography and Management module quadrant, along with AHAS, an application high availability service that provides automatic application architecture detection, fault injection high availability testing, and one-click application traffic limiting degradation.
Recently, the Spring-Cloud-Circuitbreaker – Sentinel module contributed by Sentinel was officially incorporated by the community into Spring Cloud Circuit Breaker. Sentinel has also joined the Spring Cloud Circuit Breaker club as one of Spring Cloud’s official mainstream recommended options.
Sentinel is not the only micro service solution recommended by Spring Cloud, but also Spring Cloud Alibaba.
In 2018, something big happened in China’s Java world.
Spring Cloud co-founder Spencer Gibb announced on the blog page of Spring Cloud website that Alibaba is open source Spring Cloud Alibaba and released the first preview version. This was later announced on Spring Cloud’s official Twitter account.
From Spring Cloud official Twitter
Perhaps influenced by Netflix’s reduction of open source investment in Spring Cloud, the popularity of Spring Cloud Alibaba after open source has exceeded the expectation of Alibaba’s senior technical expert Ji Wang.
In an interview with Open Source China, Hiwang said, “Spring Cloud Alibaba is a blessing for Java developers in China, making up for the limitations of Spring Cloud’s native implementation in large-scale cluster scenarios. At present, there are many implementations of Spring Cloud specification, such as Netflix has its own complete system, Consul support service registration and configuration management, ZooKeeper support service registration, etc. However, each implementation more or less has its own advantages and disadvantages, and most Spring Cloud users may find it difficult to realize the limitations of Netflix OSS and other implementations. Service discovery, distributed configuration, service invocation and fuse are not suitable for large-scale cluster scenarios. For example, we also have internal Eureka performance issues. “So we’re combining our super-scale clustering experience with the strong SpringCloud ecosystem to create some positive chemistry in the industry.”
After the summer, the open source craze continues
The benefit of efficiency is that we can focus our attention and time on things that require more creativity and are more fulfilling. For developers working in the engineering field, there is a strong sense of efficiency.
In September 2018, Ali opened source its widely used Java online diagnostics tool, Arthas. Perhaps hitting a pain point for developers looking for problems online, Arthas has racked up over 1m Stars and more than 40 resulting in open-source contributions just 147 days after its first post-open source Release.
Arthas Contributors
Not only do we see the popularity of Arthas among developers, but we also see a growing number of local developers who are using open source technology to accelerate their business, and we can’t help but imagine that in the future there will be more quality open source projects in China that will be noticed and loved by developers around the world.
In the field of technology, all milestones are achieved from a technical feelings.
Ali Cloud intelligent technology expert Duan Ling recalled: “Arthas started within Alibaba in 2015 when microservices were just starting to take off, and our team focused on getting The Spring Boot off the ground and improving development efficiency. On the other hand, I hope to improve the ability and efficiency of the technical team to troubleshoot problems online. At that time, after selection discussion, we chose to develop based on Greys, a Java open source online problem diagnosis tool, to provide a better application diagnosis experience.”
We’ve made a lot of improvements to the user experience: color UI, Web Console, Intranet one-click diagnostics, etc. Arthas has slowly become an essential tool for many of Alibaba’s technical colleagues to diagnose problems online. Although Arthas was well received within Ali, it was a tool for its own use. Open source is what we take, open source is what we use, so on September 28, 2018, we officially opened Arthas to help Java developers make diagnostics more efficient.
As more and more developers begin to use Arthas, a number of developer productivity tools have built Arthas into their products, enriching the way Arthas can be accessed and opened, such as the IDE plug-in Cloud Toolkit.
Time to 2019.
Ali Cloud intelligent senior development engineer Xuan Ok shared on the IntranetDistributed issues have always been a technical itch in application development. Not a pain point, because for a long time, the common strategy for dealing with distributed transactions has been to avoid them if you can. However, in today’s microservice architecture, the distributed transaction problem is becoming more and more unavoidable. There are many solutions, but they are either poor performance or highly intrusive and not easy to be implemented. In short, it was a bit of a bummer.
Fescar open source event live
This “discomfort” is centrally reflected in Fescar, an open source middleware for distributed transactions. When Qing Ming, senior development engineer of Ali Cloud Intelligence, announced in RocketMQ Meetup in January 2019 that the distributed transaction middleware Fescar was officially open source, Fescar received 3000+ stars and 58 issues in the community discussion within a week. The Fescar team then compiled and answered 13 questions from developers, such as how Fescar came to be, how it works, and how it differs from other open source distributed transaction solutions.
Alibaba’s middleware team launched TXC (Taobao Transaction Constructor) in 2014 and began to provide distributed Transaction services for applications within the group. In 2016, TXC after product transformation, as GTS (Global TransactionService) online Ali Cloud, became the industry at that time the only cloud on the distributed transaction product, in the form of Ali Cloud public cloud and proprietary cloud solutions, delivered to many external customers, and has been unanimously recognized by customers. In 2019, based on TXC And GTS technology accumulation, the middleware team launched the open source project Fescar (Fast & EaSy Commit And Rollback, Fescar) to build distributed transaction solutions together with the community.
TXC/GTS/Fescar is a unique solution to the problem of distributed transactions under microservices architecture. Fescar’s vision is to make the use of distributed transactions as simple and efficient as the use of local transactions. The ultimate goal is to make Fescar applicable to all distributed transaction scenarios.
Alibaba’s path to open source continues.
At the right time, ali Cloud Summit · Beijing developers special venue, Ali Cloud intelligent senior technical expert Li Sanhong announced that Alibaba Dragonwell, the long-term support version of Ali Open Source JDK, as the only Chinese enterprise on the JCP supreme Executive Committee, Will be more actively involved in the maintenance of the Java ecosystem.
“Dragonwell” means longwell, symbolizing Chinese tea culture.
The original link
This article is the original content of the cloud habitat community, shall not be reproduced without permission.