Abstract: Huawei Cloud NoSQL database architect Wenlong Yu was invited to attend the 13th China System Architects Conference (SACC2021) and delivered an important speech, sharing the architecture design concept of GaussDB(for Redis) and the core competitiveness of the product to build business agility with innovative technology. Drive digital transformation of enterprises.
From May 20 to 22, the 13th China System Architects Conference (SAC 2021) with the theme of “Digital Transformation, Architecture Remodeling” was webcast on cloud. Wenlong Yu, NoSQL database architect of Huawei Cloud, was invited to attend and delivered an important speech, sharing the architecture design concept of GaussDB(for Redis) and the core competitiveness of its products, so as to build business agility with innovative technologies and drive enterprise digital transformation.
As one of the most popular NoSQL databases in the industry, Redis has advantages in performance, data structure diversity and other aspects. However, with the acceleration of enterprise digitization, the disadvantages of open source Redis in AOF expansion, snapshot performance, master-slave disconnection, high cost, low capacity and other aspects are becoming more and more obvious, which further gives rise to the technological innovation of Redis ecology.
GaussDB(for Redis) is a Redis protocol compliant cloud native database developed by Huawei cloud database team. The database uses the computing and storage separation architecture to overcome the memory limitation of open-source Redis and can be easily expanded to PB storage.
In GaussDB(for Redis) computing and storage separation architecture model, the computing layer provides hot data cache, and the storage layer is responsible for displacing full data. Internal interconnect via RDMA high-speed network, algorithmic prediction of user access rules, automatic hot and cold data exchange, the ultimate performance improvement.
The underlying core part of the architecture is based on huawei’s powerful and widely used self-developed distributed storage system DFV, which realizes a set of Share Everything’s cloud native architecture, giving full play to the advantages of cloud native elastic scaling and resource sharing. Perfect avoid open source Redis master-slave stacking, master-slave inconsistent, fork jitter, memory utilization only 50%, large key block, gossip cluster management and other problems.
Based on this architecture, GaussDB(for Redis) provides five competitive capability advantages: strong consistency, high availability, hot and cold separation, elastic scaling, and high performance.
L strong consistent. Data replication is a matter of storage, so specialist tasks are left to specialist teams. Using distributed storage DFV, GaussDB(for Redis) is the first in the industry to implement strong consistency between three replicas and six replicas. Under the strong consistent architecture, users no longer need to worry about extreme problems such as data loss, inconsistency and OOM caused by the accumulation of open source Redis, nor do they need to worry about business errors, such as inconsistency of counters, stream limiter, access statistics and hash field.
L High availability. The data replication of single sharding and cluster of open source Redis adopts the master-slave architecture, resulting in a cluster of N nodes. If a pair of master-slave (i.e., 2 nodes) is lost at the same time, the whole cluster will be unavailable. In GaussDB(for Redis), all data can be seen and shared by each compute node. Therefore, for an instance of N nodes, at most N-1 node exceptions can be tolerated, truly achieving ultra-high availability.
L Separation of heat and cold. To make full use of data value, GaussDB(forRedis) separates hot and cold data. Hot data is stored in memory and cold data is stored in disk. At the same time, automatic LRU removes cold data and real-time prediction algorithm loads hot data. In this way, hot and cold data consistency is strong, the interaction logic is simple, and business friendly.
L Elastic expansion. When the amount of data expands to a certain scale, capacity expansion is often considered a high-risk and difficult operation. GaussDB(for Redis) decouples the capacity expansion of computing resources from storage resources based on the cloud native architecture. Computing layer capacity expansion can be completed in seconds by modifying the route mapping without any data relocation. The storage layer is a co-built super data lake with huge capacity and excellent dynamic capacity expansion. In addition, based on the 64MB fine-grained data partition, the entire expansion process is almost indifferent to the upper-layer database services. Therefore, GaussDB(forRedis) can easily support large-scale expansion of services in the later period, and implement computing/storage hierarchical expansion and purchase on demand.
L High-performance. GaussDB(forRedis) is 2 to 3 times better than other cloud vendors in terms of throughput and latency due to multi-read and write operations and no waste of resources on standby nodes. In addition, GaussDB(for Redis) benefits from multi-threaded architecture, high compression ratio, and high memory utilization. Compared with open source Redis with the same memory, GaussDB performs better in throughput and latency.
At present, more and more enterprises choose to cloud their core business to this architecture. Moreover, the core capabilities of products built based on this architecture greatly promote the development of enterprise business and accelerate the digital transformation and upgrading of enterprises. Huawei Cloud expects to have more enterprise customers in the future. Based on the core capabilities of GaussDB(for Redis), we will jointly promote industrial development and create new opportunities in the win-win era.
Click to follow, the first time to learn about Huawei cloud fresh technology ~