Introduction: Database, operating system and middleware are listed as three basic software. Whether we deposit or withdraw money in bank, or query health code, our daily applications and enterprise business cannot do without database. Arguably, without a database, it is difficult to build a digital base. Over the past 40 years, database technology has been innovating and iterating, which is no less than the huge leap from 1G to 5G in mobile communication technology. The emergence of cloud computing has ushered in the biggest inflection point in the global database market in 40 years. And this time, with the potential side of the Chinese manufacturers.

About the author:

Feifei Li, vice President of Alibaba Group, head of Alibaba Cloud database Product Division and chief database scientist of Dharma Academy. IEEE Fellow, ACM Distinguished Scientist

What determines the height of an iceberg is 80% of what you can’t see below the surface of the water.

If we compare enterprise business to an iceberg and data to the new oil of the digital age, then data is a “huge basement” buried beneath the water, with exciting dynamic potential. How to help users of thousands of lines and industries to save, use and manage data well, and empower enterprises with digital transformation needs the main body of data, the core basic software — database as a solid base.

Database, operating system and middleware are listed as three basic software, enterprise business and daily applications are inseparable from database. No matter in the bank deposits or withdrawals, or health code query, behind the database is inseparable. Arguably, without a database, it is difficult to build a digital base.

In the magic Quadrant assessment results of global cloud database 2021 just released by Gartner, the authoritative international organization, Ali Cloud database, as a representative of Chinese technology companies, has been ranked in the “LEADERS” quadrant for another year, which means that the comprehensive strength of Ali Cloud database has been firmly ranked in the first camp in the world.

This is not only the affirmation of the strength of Ali Cloud database, but also proves that Ali Cloud around the new track of cloud native database forward-looking layout and self-research innovation strategy is correct. In 2020, Ali Cloud entered the global cloud database “leader” quadrant for the first time, which is the first time that Chinese database entered the global top database ranks in 40 years. This year, AliYun is still the only Chinese company to enter the “leader” quadrant, and I am very proud to win recognition for Chinese databases on the international stage.

There are a total of 20 enterprises selected into Gartner’s Magic Quadrant of global cloud database this year, including 2 enterprises in China, accounting for 10%. This is not commensurate with China’s huge market space and industrial application prospects. In my opinion, more And more Chinese manufacturers will appear on this list in the future and continue to move into the “leader” quadrant.

My assessment comes as the emergence of cloud computing has brought the global database landscape to its biggest inflection point in 40 years. This time, The company will side with Chinese manufacturers.

The 40-year transformation of the database

The database is the most important cornerstone of the digital economy. People have felt the rapid changes brought by mobile communications, smart phones and AI, but the database seems to change little. However, in the eyes of database practitioners, over the past 40 years, database technology has been continuously innovating and iterating, which is no less than the huge leap from 1G to 5G of mobile communication technology.

At the beginning of 1980s, database system stepped into the center of information technology stage. Around 2000, the rise of big data technology; After 2010, cloud computing continued to heat up, and cloud native, distributed and other technologies continued to develop. Looking to the future, database technology will achieve a huge technological leap due to cloud computing.

As enterprise services become digital, online, and intelligent, enterprises face exponentially increasing massive storage requirements and challenges, including but not limited to those caused by service hotspots and unexpected traffic. Enterprises not only need to reduce costs and increase efficiency, but also need to conduct data analysis and insight, so as to generate intelligent decisions that can guide actions. Traditional commercial databases have been unable to meet and respond to rapidly growing business demands:

One: The system faces continuous expansion challenges. Traditional database is developed under the framework of Von Neumann, and its core feature is the tight coupling of computing and storage resources. If the resources required by the service exceed the capacity provided by the underlying system, the existing system needs to be flexibly expanded.

Two: there will always be a risk of system downtime. If the database that supports core online services is faulty, the service system cannot run properly. The database must be highly available to ensure that the database can switch between different servers, VMS, and availability zones and regions regardless of any underlying resource problems. This ensures that the upper-layer service systems are fault-free from errors or risks of underlying hardware resources.

Faced with these two core challenges, cloud native databases use virtualization technology to pool computing and storage resources and separate storage and computing. For example, in the old days, creating a separate database was like digging a well for each household. Today’s cloud-native database connects the water at the bottom of these Wells, ostensibly separate Wells, but the bottom of the Wells are connected to form an invisible pond that can be used on demand at any time.

In a word, from traditional database to cloud native database, the biggest technological change is resource pooling and resource decoupling, as well as the resulting flexibility, high availability, container deployment, intelligent operation and maintenance as the core capabilities. Behind this is the constant decoupling of technology, the deconstruction and reorganization of technology. By pooling and separating storage computing technology, it provides enterprises with uninterrupted, resilient, and highly available database services.

The essence of cloud computing is to efficiently pool resources by using distributed and virtualization technologies and transparently deploy applications in a centralized manner. The current mainstream development trend in the industry is to pool and decouple resources, based on cloud native and distributed technologies. The cloud native database 2.0 created by Ali Cloud — one-stop full-link data management and services, combines cloud native technology and distributed technology into one: The upper layer is distributed, and the lower layer is implemented in the way of cloud native. Each partition can enjoy the flexibility and high availability brought by cloud native, while the upper layer has the ability of horizontal expansion brought by distributed, which can solve the bottleneck problems caused by high concurrency. On this basis, create exclusive database and data warehouse system for different application scenarios and requirements, so as to make special library special; At the same time, HTAP, offline integration, warehouse integration, multi-module data processing and other core technologies are used to realize the integration of data processing and analysis, as well as the free flow of data in different application scenarios and systems. Finally, through the online data management platform to achieve one-stop full link data management and service.

For example, in a traditional architecture, an enterprise needs to deploy an IT system, deploy a database, and plan for system capacity and redundancy according to predictable business peaks. The problem and challenge with this model is high cost and low utilization because most of the time the system is idle and resources are misallocated. Resource decoupling of cloud native databases solves this problem nicely.

Last year, we rapidly expanded the hardware and software resources of more than 100,000 cloud servers for Dingding. However, the database system does not require complex and time-consuming expansion deployment like traditional database, but adopts the cloud native technology, hierarchical separation and elastic decoupling to meet business needs. In the cloud native database system, we can realize intelligent and automatic operation and maintenance process through AI and machine learning technology, quickly mobilize resources, match business needs, and expand resources as needed. Like water systems, users can control the amount of water in real time and flexibly. If demand goes up, turn on the faucet, turn it down when demand goes down. After the service peak, resources can be released quickly, greatly improving system resource usage efficiency. On the other hand, the ease and efficiency of operation and maintenance are also greatly improved for business systems.

Singles Day is one of the world’s largest superdigital projects, a training ground for Alibaba’s technology, but aliyun’s database has withstood the most rigorous tests. For example, at the peak of 580,000 Transactions Per Second at zero on November 11 in 2020, each transaction will have an action to open an order, which means millions of TPS(Transactions Per Second) in the database system. This kind of instantaneous explosion scenario, if not the use of cloud native technology, simply relying on traditional technology can not meet the requirements of high concurrency, flexibility, and high availability.

2021 is the year of popularization of digital transformation, and the business of thousands of lines and hundreds of industries is accelerating the process of going cloud. Even if a single customer has diversified complex services and rich business load characteristics, it is difficult for a single product and architecture to meet customer needs. Ali Cloud database fully embraces cloud native, and for the first time puts forward the concept of “one-stop full-link data management and service” from the perspective of customer scenarios, hoping to meet the diversified business demands of enterprises through products and product combinations.

In the cloud native database 2.0 era, Ali Cloud database provides a station covering data production and integration (DTS, DMS and DBS), real-time data processing and storage (RDS, PolarDB product series, Lindorm, Tair), data analysis and discovery (AnalyticDB product series), data development, management and intelligent operation and maintenance (DMS, DAS) Full link, full life cycle, full scene data management and services, through the fingertips, easy to use, safe and reliable cloud database, so that data seamless free flow.

Chinese database vendors with the trend

As an application-oriented technology, database is very important for first-mover advantage and ecological construction. If there is no breakthrough innovation in technology, it is very unlikely for latecomers to surpass it. In the cloud era, cloud computing has become a new track for the development of database, and Chinese database enterprises have ushered in an excellent period of opportunity for change.

According to Gartner, cloud database management systems (DBMSS) revenue will account for 50% of total DBMS market revenue by 2022, one year earlier than projected in 2020. The database continues to move towards the cloud, and the global database industry structure is accelerating reconstruction.

Cloud computing as a new scientific and technological service, database research and development, use, sales and other aspects have brought a complete change, but also let global manufacturers in the database track stand at the same starting line. China has a large number of digital users and many unique business scenarios. Database vendors that have not successfully dealt with double 11 traffic peak and “health code” scenarios will find it difficult to know how to solve practical business problems such as massive data and ultra-high concurrency transaction peak. Globally, Chinese manufacturers enjoy this opportunity. This time, Chinese vendors are riding the cloud, with the technology and innovation capabilities of foreign databases.

With ali cloud since research cloud PolarDB native database as an example, to launch in 2017, but today many functions, such as memory and computing and storage, three layer decoupled architecture implementation second level flexibility, the main writing, based on the column of memory pool stock index support HTAP etc is the global initiative or the industry leading technology, innovation has advanced foreign counterparts.

The Database Development Research Report (2021) released by The China Academy of Information and Communication Technology in June this year points out that China’s database industry has entered a period of significant development opportunities. According to the report, China’s database market will reach 24.1 billion yuan in 2020, accounting for 5.2 percent of the global market. It is estimated that by 2025, the total size of Database market in China will reach 68.8 billion yuan, with a compound annual growth rate (CAGR) of 23.4%, accounting for 12.3% of the global market.

The rapid development of digital economy inevitably promotes the rapid growth of Database market in China. It is the result of several generations of unremitting efforts that China’s database industry has been able to advance and retain the leadership position in Gartner’s Magic Quadrant. Database industry has achieved today’s results, and the country’s support for basic software products, as well as policy guidance and support is inseparable.

In the past, there were many objective reasons why Chinese database technology failed to open the market, but today, Chinese and overseas enterprises have begun to accept Chinese database technology on a large scale, and core applications such as finance and government affairs have begun to try to replace traditional database with cloud database on a large scale. With the implementation of the major strategy of self-reliance and self-improvement in science and technology, such comprehensive substitution will be a certain trend, and more and more enterprises will use Chinese database technology. Cloud native mixed with distributed integration, in offline integration, HTAP load query and processing, Internet of things, and multiple modules according to fusion processing and analysis, safe and reliable and privacy protection, intelligent operations and tuning, load, and machine learning and AI intelligent operator and application support, the adaptation of new hardware and optimization will be cloud native database key power and breakthrough of technical direction .

Today, there are more than 80 database manufacturers in China, many of which are highly concerned and sought after by the capital and become the new cutting-edge unicorn manufacturers. Although the personnel scale of the single domestic database enterprise is not enough to compete with the global giants, the industrial talents have formed a certain scale and echelon. This has laid a solid talent foundation for the steady development of China’s database industry. Ali Cloud database is also committed to cultivating technical talents in the field of database, establishing open source cloud native distributed database community and organization, writing cloud native database textbooks, hoping to attract more people to participate in, learn, contribute to database technology, and realize the prosperity of talent ecology.

In addition, through the joint innovation of industry-university-research cooperation, the number of papers published by Chinese manufacturers in VLDB, SIGMOD and ICDE has also shown an increasing trend year by year, accounting for about 23% in recent years. Some of the database technology innovations proposed in these papers have even been recognized by global peers. This is also one of the signs that Chinese database enterprises take advantage of the situation.

“Because of the time of heaven, the situation of the place, according to the interests of the people, the invincible, hit all.” In this era of great industrial change, a deep understanding and correct grasp of the situation we are facing is conducive to Chinese database manufacturers to see the direction and find the path of rapid growth. I believe that time and momentum will also be transformed into a driving force for the rapid development of China’s database industry, so that we have a proper place in such a key field of information technology.

The original link

This article is the original content of Aliyun and shall not be reproduced without permission.