Introduction: As a widely used Serverless DataBase, Tablestore can provide economic billing mode, greatly reduce business costs, and at the same time, it has extreme flexible service capability and completely zero operation and maintenance characteristics, which can bring users more smooth experience.

The author | li xin

With the rapid development of Internet technology in the past decade, more and more industries have joined the Matrix of The Internet, which has brought more rich and complex business scene requirements, which is undoubtedly a huge challenge to the performance of data application system.

Relational database MySQL is the most widely used database product in application systems, with powerful data query and strong transaction processing capabilities. In today’s cloud era, application systems are gradually evolving to be built based on cloud native Serverless architecture, because of its advantages of low cost and high flexibility. However, there are still some obvious deficiencies in mysql-based data storage under Serverless architecture:

1. Poor elastic expansion capability. An important feature of the Serverless scenario is the significant peaks and troughs of application load. When faced with the traffic peak, the DBA needs to manually expand the cluster to prevent the cluster from being overwhelmed. When traffic is at a low ebb, the cluster needs to be scaled down to save costs.
2. High o&M complexity. To build MySQL, you need to purchase clusters, install services, and manage connections. After services are online, they need to pay attention to data security, service availability, and response time. As a result, they spend more time on cluster O&M and cannot focus on service research and development.
3. The high cost. Generally, DBAs need to estimate the service scale to set the initial database capacity in advance. If the number of service requests does not reach the estimated value, resources in the cluster will remain idle, resulting in resource waste.

Serverless DataBase

MySQL supports the relational model and its strong transaction characteristics, so that it has a very important position in the application system, is one of the storage components that can not be completely replaced. However, if you blindly rely on MySQL, the application system cannot be completely Serverless and cannot enjoy the extreme flexibility brought by Serverless.

Within Ali, we have some new architectural practices. The data that requires strong transaction processing still uses relational table storage, while for the non-strong transaction table data storage, we have designed a Serverless table storage with extreme flexibility.

Our design requirements for the Serverless database product include the following features:

• ** fully elastic. ** Can automatically expand and shrink flexibly according to the application load, which provides users with a more economical billing mode and a smoother experience.
• ** Charge by volume. ** The cost of using a Serverless database is mainly due to the cost of computing and storage. Users only need to pay for storage units and response units actually generated by services, saving costs.
• ** Zero operation and maintenance. ** out-of-the-box, no need to manage capacity, water level, software upgrade, kernel optimization and other operations and maintenance matters, truly let r & D focus on business development.

The Serverless architecture is widely used in many business scenarios. For example, in its core e-commerce business, Century Lianhua Group realizes full cloud operation and gradually transforms its business into the middle-stage mode with full Serverless architecture in view of the pain points such as difficult budget of resources and difficult system deployment encountered in self-built IDC room.

Century Lianhua Group adopted the scheme of function calculation + API gateway + Tablestore, which easily supported the big promotion activities such as 6.18 and Double 11. Among them, table storage Tablestore, as the core storage in Serverlesss architecture on cloud of Century Lianhua e-commerce system, has the advantages of extreme flexibility, free operation and maintenance and low cost.

Table storage Tablestore introduction

Table storage Tablestore in 2009 ali Cloud was established at the beginning of the project research and development, based on the bottom feitian platform built from scratch, is a multi-model, multi-engine Serverless table storage. It has exported more than 30 regions at home and abroad on the public cloud, with a scale of 15,000 servers and 200PB storage, which is the underlying core storage of many commercial products of Ali Cloud.

At the same time, it has been exported to finance, energy, power, logistics, medical, government and enterprise industries offline, serving public cloud 1000+ enterprise customers and 500+ offline projects.

Table Storage Tablestore integrates HBase and ElasticSearch, provides extreme flexibility, free o&M, and out-of-the-box features, supports flexible storage from GB to PB, and insensitive expansion of 100,000-level TPS service capabilities. As a one-stop structured data storage platform integrating storage, search and analysis, it not only supports massive table data but also provides rich data retrieval and analysis capabilities.

The overall structure of Tablestore Tablestore is shown in the figure below:

Table storage provides a variety of data models, including wide table model (Widecolumn), message model (Timeline) and Timeseries model (Timeseries).

• The wide table model mainly carries table structured data storage, such as e-commerce order data.
• The message model mainly hosts message data stores, such as IM/Feeds messages.
• Timing model mainly carries timing data storage, such as timing data of Internet of Things devices.

Here we will take the e-commerce order scene as an example, take you to experience the tableStore-based wide table model to build a Serverless order storage system.

Tablestore experience

1. Preparation

Before you can experience the extreme flexibility of Tablestore, you need to prepare the following steps:

(1) Create an Ali Cloud account and obtain the AK of the Ali Cloud account. (Cloud account AK is the key to access all cloud services including Tablestore, and AK is required to access Tablestore service in the future).

(2) Download and start the command line tool Tablestore CLI provided by Tablestore. The command line tool provides some simple instructions to manage table storage service.

First, configure the connection key through the config command and open the Tablestore service through the enable_service command:

config –id accessKeyID –key accessKeySecretenable_service

Create an instance with create_instance:

create_instance -d “order storage” -n serverless-db -r cn-hangzhou

Instance is equivalent to the concept of MySQL database. After creating an instance, you do not need to worry about the water level of the physical machine cluster where the instance resides. You only need to focus on developing service logic. At the same time, the read/write and storage on the instance are charged by volume. If there is no read/write and no storage, there is no actual cost.

At this point, a Serverless DataBase that supports GB to PB storage, has no concurrency restrictions, zero operation and maintenance, and is fully resilient has been created.

2, create table

Widecolumn is a Schema-free data table. Different from the relational database MySQL, the Widecolumn data table only needs to define the primary key structure, and does not need to define the attribute column structure.

For example, the table structure of an order table order is shown in the following figure (sliding) :

Create an order table with a wide table model, specify the primary key ID of the order table instead of the attribute column.

create_instance -d “order storage” -n serverless-db -r cn-hangzhou

After the create command is executed, an order width table is successfully created. The newly created order width table initializes one data partition.

As the order data volume increases or the number of visits increases, the wide table model will split and expand into multiple data partitions according to the distribution range of the first primary key (namely, the order ID in the above data model) and evenly distribute to multiple physical machines to support larger data scale (TB or PB) and read/write throughput (more than 100,000 TPS). The entire extension process is completely automated by the server without human intervention.

3. Data import

The simulation generated 1 million sample order data and imported them into the ORDER table in batches through the import command. The write speed of a single data partition can reach tens of thousands of rows /s, and the write throughput can be further improved as the partition expands.

import -i orderDataFile -l 1000000

Current speed is: 10000 rows/s. Total succeed count 10000, failed count 0.Current speed is: 12600 rows/s. Total succeed count 22600, failed count 0……. Current speed is: 9200 rows/s. Total succeed count 1000000, failed count 0.Import finished, total count is 1000000, failed 0 rows.

4. Order inquiry

Use the get command to query the wide table model for a single row by order number (ID) and get a single row of order data. The GET command can only perform a single row query based on rowKey.

Example of querying an order:

get –pk ‘[“0000005be2b43dd134eae18ebe079774”]’

+———————————-+——-+——–+———+————-+—————+——–+——–+——— -+——–+———+——-+——-+——–+————+| order_id | cId | cName | hasPaid | oId | orderTime | pBrand | pCount | pId | pName | pPrice | pType | sId | sName | totalPrice |+———————————-+——-+——–+———+————-+—————+——–+——–+——– – + — — — — — — — — + + — — — — — — — — — — — — — — — — + + — — — — — — — — — — — — – + — — — — — — — — — — — — + | 0000005 be2b43dd134eae18ebe079774 | c0015 | | | false on Friday O0035062633 | 3 | | 1507519847532 | millet p0005003 6 2299.21 | | | millet mobile phone | s0017 seven | | sale zheng 6897.63 |+———————————-+——-+——–+———+————-+—————+——–+——–+——– –+——–+———+——-+——-+——–+————+

5. Order retrieval and statistics

Order scenarios often rely on multi-condition combination filtering, in which case it is necessary to rely on the multi-index feature of Tablestore. Multiple index is a table data index provided by Tablestore similar to Elasticsearch. It supports rich query methods and data aggregation capabilities, and can create indexes on multiple columns. Unlike MySQL’s federated indexes, multivariate indexes can be queried based on any combination of fields and will not be matched by the leftmost prefix of multiple columns.

For example, we build indexes on fields such as ID, pName and totalPrice respectively, and use data structures such as inverted index, word segmentation and BKDTree to provide query capabilities such as accurate query, full-text search and range query. In addition, multivariate indexes support field grouping, multi-field sorting, and statistical aggregation capabilities.

Create a multivariate index on a wide table with create_search_index to speed up queries.

create_search_index -t order -n order_index{ “IndexSetting”: null, “FieldSchemas”: [{ “FieldName”: “id”, “FieldType”: “KEYWORD”, “Index”: true, “EnableSortAndAgg”: true, “Store”: true },{ “FieldName”: “pName”, “FieldType”: “TEXT”, “Index”: true, “EnableSortAndAgg”: false, “Store”: true },{ “FieldName”: “totalPrice”, “FieldType”: “DOUBLE”, “Index”: true, “EnableSortAndAgg”: true, “Store”: true } …

// Other fields

]}

Tablestore supports SQL query ability, compatible with MySQL query syntax, and tries to retain the use of relational database habits. SQL can automatically select indexes and perform query acceleration. Through the query acceleration of multivariate indexes, it also has the ability of millisecond delay query under the scale of ten billion data.

Order retrieval according to sName, pBrand and pName three field conditions:

Select * FROM ‘order’ WHERE sName = “sbrand” and pBrand = “sbrand” AND pBrand = “sbrand” LIMIT 3;

+———————————-+——-+——–+———+————-+—————+——–+——–+——— -+———+——–+——-+—————+——-+——–+————+| id | cId | cName | hasPaid | oId | orderTime | pBrand | pCount | pId | pName | pPrice | pType | payTime | sId | sName | totalPrice |+———————————-+——-+——–+———+————-+—————+——–+——–+——– –+———+——–+——-+—————+——-+——–+————+| 00001c760c04126da067e90409467c4e | c0022 | Zhao a elimination | true | o0009999792 | 3 | | 1494976931954 | millet p0005004 | red rice 5 s 499.01 cell phone | | 1494977189780 | | s0005 | | sale Friday 1497.03 |+———————————-+——-+——–+———+————-+—————+——–+——–+——– –+———+——–+——-+—————+——-+——–+————+| 0000d89f46952ac03da71a33c8e83eef | c0012 | Away money 2 | | false o0024862442 | 2 | | 1502415559707 | millet p0005004 | red rice 5 s | | 499.01 phone | null | s0015 | sales | 998.02 on Friday |+———————————-+——-+——–+———+————-+—————+——–+——–+——– –+———+——–+——-+—————+——-+——–+————+| 0000f560b62779285e86947f8e8d0e4c | c0008 | Von eight elimination | | false o0000826505 | 1 | | 1490386088808 | millet p0005004 | red rice 5 s | | phone | null | 499.01 s0015 | sales | 499.01 on Friday |+———————————-+——-+——–+———+————-+—————+——–+——–+——– –+———+——–+——-+—————+——-+——–+————+

Count the order quantity of each brand in all orders:

select pBrand,count(*) from `order` group bypBrand;

+ + — — — — — — — — — — — — — — — — — — + | pBrand | count (*) | + — — — — — — — – + — — — — — — — — — — + | vivo | 162539 | + — — — — — — — – + — — — — — — — — — — + | | 304252 lenovo | + — — — — — — — – + — — — — — — — — — — + | oppo | 242513 | + — — — — — — — – + — — — — — — — — — — + | | 96153 | apple + + — — — — — — — — — — — — — — — — — — + | | millet 194543 |+——–+———-+

conclusion

As a widely used Serverless DataBase, Tablestore provides economic billing mode and can greatly reduce business costs. The above order scenario is an example. Under the order data level of 100 million and the average read and write volume of 2000TPS, the cost of using tables to store Tablestore is less than 400 yuan/month. At the same time, Tablestore has extreme elastic service ability and completely zero operation and maintenance characteristics, which can bring users a more silky experience.

The original link

This article is the original content of Aliyun and shall not be reproduced without permission.