Abstract: In the era of big data technology, enterprises are scrambling for digital transformation. Behind any successful project, there is no need for the support of data platform, let alone a large project worth 213.5 billion yuan. How robust, secure and stable should the data platform be?

Turnover exceeded $10 billion 00:02:05 turnover exceeded $66.6 billion 01:47:26 Turnover exceeded $100 billion 15:49:39 turnover exceeded $168.2 billion 22:28:37 turnover exceeded $200 billion


The 2018 Singles’ Day set a new record of 213.5 billion yuan


High-speed jump of the number, constantly refresh the state, cable in the code, flying fingers on the keyboard…

The new record is by no means a decade old or 2017

The records that product experts cling to and the records that technical experts are sworn to conquer

It is the record that the solution is calm, the record that the technology is refined, the record that the framework is stable, and the record that the system is robust

Data platform, create value calculated in 2004, in order to analysis the current situation from the big data, and predict the trend of the future, ali has the first big data warehouse, by 2008, taobao’s business and data volume has reached 2004, thousands of times, in order to make the data of production, reconstruct large data from the underlying technology is particularly important.

In the spring of 2010, the first version of SQL Engine, the predecessor of MaxCompute, went live, running on flying clusters of 30 machines at the time.

In 2011, the Flying Team began to explore the support of the group’s internal warehouse business, and ran the production operation of ladder 2 on 1500 machines in parallel, and achieved performance and stability comparable to Hadoop.

After the flying 5K project, MaxCompute entered the era of 5000 machines and cross-room scheduling, and could sort 100TB in 377 seconds.

At present, MaxCompute has become the main computing platform of Alibaba. 99% of alibaba group’s data storage and 95% of its computing power are used by MaxCompute. More than 5 million jobs run on the platform every day.

With the help of MaxCompute engine, from November 11, 2014, thousands of people have become the norm. As the secret weapon of large-scale computing behind project 11, MaxCompute withstood the test of massive cluster, massive data and high concurrency during Double 11.

MaxCompute can process 100PB of data in a single day in 2015, 180PB in 2016, 320PB in 2017, and more than 500PB in 2018, constantly pushing new limits and breaking new records. At the same time, MaxCompute stably supports the online flow peak of 120,000 transactions /s in the e-commerce mixing unit, stably carries 45% of the shopping guide flow, and successfully completes the target of 1200 DB business mixing, providing strong guarantee and smooth support for the peak value of double 11 transactions.

During Singles day, in addition to Tmall, Taobao, Ali Group, MaxCompute also provides powerful computing services for finance, payment, new retail and other industries.

In the past few years, through the combination of big data and artificial intelligence algorithm, MaxCompute has become the most important part of ET brain. ET city brain automatically commands city intersection signal lights, and has been successfully applied to traffic governance in Malaysia. ET Industrial Brain drives factories to achieve better yield and is committed to helping manufacturers increase yield by 1%.

From Digital Ali to digital City, MaxCompute can carry eB-level data storage capacity and become the world’s first 100TB Bigbench big data computing platform. It covers more than a dozen countries and regions at home and abroad on public cloud, and more than 100+ sets are deployed on private cloud.

MaxCompute is not just a single engine, but a complete service for the enterprise. Over the past 10 years, MaxCompute has provided data capabilities to a wide range of cloud enterprises while steadily supporting all data services of Alibaba Group. MaxCompute based on the construction of stable, secure, scalable and efficient interaction with the application system of the data center, has become a number of industries under the customer’s digital transformation of the first choice.

From online to offline, from manufacturing to Internet e-commerce, from domestic to overseas, MaxCompute computing capacity is extending to all walks of life, greatly reducing the cost of social computing.

With each challenge, MaxCompute stands at a new starting point, with no noise and no ambition.

Author: Jin Heng