Abstract: huawei cloud new generation financial distributed database GaussDB(for openGauss) has officially launched a number of major new kernel features such as Ustore storage engine and DCF high availability component based on Paxos protocol.
In the digital era, the technology iteration and updating is more rapid than ever before, and the database technology also bursts out a huge development tension in the tide of technology, and the kernel technology as the soul of database technology has also realized a transition development.
Huawei cloud database has always attached importance to the development of kernel technology, has rich experience in database kernel development, and continues to build a series of highly competitive kernel features. Huawei’s new-generation financial distributed database GaussDB(for openGauss) has launched several major kernel features, including the Ustore storage engine and DCF high availability component based on Paxos. For enterprise users to create a stable high-performance storage engine, high availability of core capabilities, innovation technology as the starting point, to further promote the digital transformation of enterprises.
Ustore storage engine innovation, the performance is still stable under frequent data updates
Ustore Storage engine, also known as in-place Update storage engine. In-place Update is a new storage mode for GaussDB(for openGauss) kernel. The GaussDB(foropenGauss) kernel currently uses the Append Update mode as the row engine. Append updates work well for business additions, deletions, and HOT(Heap only tuple) updates, but garbage collection is not efficient for non-HOT Update scenarios across data pages.
The Ustore engine stores the “valid data” of the latest version and the “junk data” of the previous version separately. The “valid data” of the latest version is stored on the data page and a separate UNDO space is created for unified management of the “junk data” of the previous version. Therefore, the data space does not expand due to frequent updates. Recycling is more efficient. Ustore engine adopts numA-aware UNDO subsystem design, which makes UNDO subsystem can be effectively extended on multi-core platform. At the same time, the multi-version index technology is used to solve the index cleaning problem and effectively improve the efficiency of storage space recycling.
Figure 1: Ustore data storage diagram: Data in the original page, undo through pointer management separately
In conclusion, compared with the original garbage data cleaning mechanism, Ustore data storage can bring higher performance, higher efficiency, more full space utilization, more stable operation of the overall system, and adapt to more business scenarios and workloads.
In terms of application practice, THE Ustore storage engine has been POC tested in some customers’ sites, showing stable and reliable database service ability and achieving the expected effect.
High availability DCF component innovation based on Paxos protocol ensures strong data consistency and high availability
In addition to the Ustore storage engine, GaussDB(for openGauss) also introduces the DCF high availability component based on the Paxos protocol. This component enhances GaussDB(for openGauss) in high availability while ensuring data consistency. Include:
(1) Get rid of third-party arbitration components through self-arbitration and majority selection of master, greatly shorten the RTO time, and can prevent the split brain under any failure of double master;
(2) Support the multi-cluster deployment mode of synchronous and asynchronous node deployment;
(3) Improve the log replication efficiency between the active and standby nodes to improve the maximum throughput capacity of the system.
Figure 2: Paxos principle diagram
With the DCF high availability component of GaussDB(for openGauss), users can not only avoid the risk of system cracking, but also improve availability.
In the future, GaussDB(for openGauss) will continue to build more intelligent, secure and efficient technical capabilities in key features such as Ustore storage engine and Paxos protocol optional master based on AI autonomous technology, and start from the most urgent problems and long-term needs of thousands of industries database applications. We will continue to make breakthroughs in core technologies to create more advanced and better database services for customers.
【 Major event preview 】
On August 18, the 12th China Database Technology Conference (DTCC2021) will be held in Beijing with the theme of “Digital Future”. Huawei Cloud GaussDB will explain GaussDB series databases and core capabilities, and share best practices in GaussDB industry. Product UGO GaussDB, moreover, the migration tool is currently in beta, welcome to huawei cloud website free experience > > www.huaweicloud.com/product/ugo…
Click to follow, the first time to learn about Huawei cloud fresh technology ~