Overall architecture and basic processes of Codis

  • Basic concept: CODIS and Redis Cluster are sliced cluster solutions

  • Codis architecture

    • Codis Server: redeveloped Redis instance, add additional data structure, support data migration operation, handle data read and write requests \

    • Codis Proxy: receives client requests and forwards them to the proxy \ between coDIS Server client and server

    • Zookeeper: Saves cluster metadata (data location information and CODIS proxy information) \

    • Codis Dashboard and CoDIS FE: together constitute a cluster management tool \

      • The CODIS Dashboard is responsible for cluster management, including adding and deleting CODIS Server, CODIS Proxy and data migration

      • Codis FE is responsible for providing the Dashboard’s Web interface for cluster management directly from the Web interface

  • How does Codis handle requests \

    • Use the CODIS Dashboard to set access addresses for the CODIS Server and CODIS Proxy so that the proxy and server can establish connections

    • The client (the tripartite package implementation interface responsible for network requests) directly establishes a connection with the CODIS proxy (a Redis-enabled RESP interaction protocol that can be switched from singletons to CODIS)

    • After receiving the request, coDIS Proxy will query the mapping relationship between ZooKeeper request data and CODIS Server, and forward the request to the corresponding CODIS server for processing \

    • The result is returned to the proxy, which then returns to the client

  • Concerns of slicing cluster:

    • The data distribution
    • Cluster expansion and data migration
    • Client Compatibility
    • Reliability assurance

\

Key technical principles of Codis

The data distribution

  • Yes The mapping is performed through logical slots

    • A Codis cluster has 1024 slots numbered from 0 to 1023

    • Slots are manually allocated to coDIS servers, with a portion of slots (optionally allocated automatically) \ on each server

    • The CRC32 algorithm is used to compute the hash value of the data key (64-bit int) and modulo the hash value to 1024 (to get Slot numbers) \

    • Query ZooKeeper. Obtain the server ID based on slot number

  • routing

    • The mapping between Slot and CODIS server is called a data routing table (routing table for short)

    • After the routing table is allocated on the CODIS Dashboard, the Dashboard sends the routing table to the CODIS Proxy and saves the routing table in Zookeeper

    • Codis-proxy will cache the routing table locally, when it receives the client request, directly query the local routing table, you can complete the correct request forward \

  • Difference between Codis and Redis Cluster \

    • The routing table in Codis is allocated and modified by us via the Codis Dashboard and is stored in the Zookeeper cluster

    • The modified route is sent to proxy (the intermediate layer responsible for forwarding) \

    • In a Redis Cluster, the data routing table is transmitted between each instance, and eventually a copy is saved on each instance (there is no intermediate layer and nodes need to be passed to each other) \

    • Redis Cluster consumes more Cluster network resources

Cluster expansion and data migration

  • Expansion directions: Add coDIS Server and CODIS proxy

  • Increase the codis server \

    • Start the new CODIS Server and add it to the cluster \

    • Migrate some data to the new server\

  • Increase the codis proxy \

    • When a proxy is added, Zookeeper will have the latest access list \

    • The client can also read the proxy access list from Zookeeper and send requests to the newly added proxy\

  • Data migration process (incremental migration)

    • On the source server, Codis randomly selects data from the Slot to be migrated and sends it to the destination server
    • After receiving the target ACK, the source server deletes the local data
    • Repeat the above migration process until all data in the Slot to be migrated has been migrated

  • You can select the migration mode

    • Synchronization: In the process of sending data from the source server to the destination server, the source server is blocked (potentially) and cannot process the new request operation \

    • Asynchronous: After the source server sends data to the destination server, it can process other requests. After the ack message is sent to the destination server, the local data is deleted

      • During migration, data is read-only to ensure data consistency.)

      • For Bigkey, asynchronous migration uses the split instruction mode to migrate, avoiding big data migration \

      • If the migration fails, data consistency is damaged. Therefore, an expiration time is set on the target server. If the migration fails, the server will be deleted after the expiration

      • Slotsmgrttagslot-async Number of batch migration keys

Client Compatibility

  • With Redis singleton, the client can interact with the instance and read and write data as long as it complies with RESP protocol
  • Codis Proxy connects directly to the client, coDIS Proxy is compatible with single instance client (guaranteed compatibility) \

Reliability assurance

  • The more components there are, the more potential risk points there are

  • codis server\

    • It’s essentially an instance of Redis

    • Redis’ master-slave replication mechanism and sentinel mechanism are both available on COdis Server

    • Codis configures slave libraries for each server, and uses the sentry mechanism to monitor. When a failure occurs, the master and slave libraries can be switched, thus ensuring the reliability of the server

  • Zookeeper\

    • Use multiple instances to hold data

  • codis proxy\

    • The stability is guaranteed by Zookeeper

  • Codis Dashboard and CoDIS FE

    • They mainly provide configuration management and manual operation of administrators, the load pressure is not large, so their reliability can be guaranteed without additional

\

How to expand the cluster and migrate data?

  • In terms of stability and maturity, Codis has been used relatively early and has been in mature production deployment in the industry

  • From the perspective of business application client compatibility, clients connected to a single instance can directly connect to coDIS Proxy, while clients originally connected to a single instance need to develop a new function \ if they want to connect to Redis Cluster

  • Codis Server was developed based on open source Redis 3.2.8, so Codis does not support new commands and data types in subsequent open source versions of Redis, and some commands \ are not supported

  • From the data migration performance dimension, Codis can support asynchronous migration, which has less impact on the cluster’s performance in processing normal requests than synchronous migration

\

conclusion

  • Codis composition

    • codis server

    • codis proxy\

    • Zookeeper Codis also supports etCD or local file systems for storing metadata information \

    • Codis Dashboard and CoDIS FE \

  • Multiple sets of COdis can be used to isolate services