Redis Cluster In-depth and Practice (Continued)

These reviews

The last article based on redis distributed lock implementation wrote a distributed lock based on Redis implementation. Distributed environment, will not use a single point of Redis, to achieve high availability and disaster recovery, at least is redis master slave. Redis single thread work, a physical machine only runs one Redis instance is too waste, redis single machine is obviously a hidden trouble of single point of failure. Memory resources are often limited, so it is not practical to continuously expand memory vertically. Therefore, it is necessary to cooperate with multiple hosts to provide services for horizontal scalable expansion, that is, multiple Redis instances run cooperatively in distributed mode.

In the previous article Redis Cluster in-depth and practice introduced Redis Cluster related content, before specially spent time on the Redis official website to see the Redis Cluster related documents and implementation. This article is the sequel of that article, because the author recently in the research of redis master switch to redis cluster scheme, will talk about the redis cluster scheme selection and redis cluster practice.

Redis clustering can be implemented in the following ways:

Client sharding, such as Redis’s Java client Jedis, is also supported, using consistent hashes
Proxy-based sharding, such as CODIS and Twemproxy
Route query, Redis-cluster

Let’s introduce these schemes respectively.

Client Sharding

Redis Sharding is a multi-Redis instance Cluster method commonly used in the industry before The emergence of Redis Cluster. The main idea is to use hash algorithm to hash the key of Redis data. Through hash function, the specific key will be mapped to the specific Redis node. Java Redis client drives Jedis and supports Redis Sharding function, namely ShardedJedis and ShardedJedisPool combined with cache pool.

Redis Sentinel provides Redis monitoring and failover functions in active/standby mode to achieve high system availability. When the primary Redis goes down, the secondary Redis takes over, ascends to the primary Redis, and continues to provide services. The active and standby nodes form a Redis node, which ensures the high availability of the node through automatic failover.

The advantage of client-side Sharding technology is that it is very simple. The Redis instances of the server are independent and unrelated to each other. Each Redis instance runs like a single server, which is very easy to linear expansion and has strong flexibility of the system.

The disadvantages of client Sharding are also obvious. As Sharding processing is placed on the client side, operation and maintenance will be challenged when the scale is further expanded. Client Sharding does not support dynamic addition and deletion of nodes. When the server Redis instance group topology changes, each client needs to update and adjust. Connections cannot be shared, and as applications scale up, resource waste constrains optimization.

Proxy-based sharding

The client sends the request to a proxy component, which parses the client’s data, forwards the request to the correct node, and finally returns the result to the client.

The features of this mode are as follows:

Transparent access, business procedures do not care about the back-end Redis instance, low switching cost.
The Proxy logic is isolated from the storage logic.
The proxy layer has one more forwarding, which degrades the performance.

The simple structure diagram is as follows:

The main components are Twemproxy and Codis.

Twemproxy

Twemproxy, also known as Nutcraker, is a Redis and memcache proxy server application developed by Twtter. As an efficient cache server, Redis has great application value. However, when the amount of user data increases, it is necessary to run multiple Redis instances. At this time, there is an urgent need for a tool to manage multiple Redis instances in a unified manner to avoid the inconvenience and maintenance brought by managing all connections in each client. Twemproxy is born for this purpose.

Twemproxy has the following characteristics:

fast
lightweight
Maintains a permanent server connection
Supports automatic deletion of failed nodes. You can set the time at which the node is reconnected, and you can set how many times the node is removed after being connected
Supports setting hashtags. The HashTag allows you to map keys of the same type to the same instance.
Reduce the number of direct connections with Redis and keep the long connection with Redis. You can set the number of each connection between the agent and the background Redis
It has its own consistent hash algorithm, which can automatically fragment data to multiple redis instances at the back end. Supports multiple hash algorithms, including setting the weight of back-end instances. Currently, Redis supports the following hash algorithms: One_at_a_time, MD5, CRc16, CRc32, fNV1_64, FNV1a_64, FNV1_32, fNV1a_32, Hsieh, Murmur, Jenkins.
Supports RedIS Pipelining Request, which connects multiple reIDS Pipelining requests to Redis.
Support status monitoring; You can set the status monitoring IP address and port, access IP and port can get a JSON format of status information string; You can set the interval for refreshing monitoring information.

The official website of TwemProxy describes the above features. TwemProxy can be used to access TwemProxy in the same way as the Redis client. Twitter, however, has long since given up updating Its TwemProxy. Twemproxy’s biggest pain point is that it cannot be smoothly expanded/shrunk. Another pain point of Twemproxy is that the operation and maintenance is not friendly, and there is even no control panel.

Codis

Codis is an open source Redis cluster solution for Pea Pod. Redis is a distributed Redis solution. For upper layer applications, there is no significant difference between connecting to Codis Proxy and connecting to native Redis Server. Upper layer applications can be used as stand-alone Redis. Non-stop data migration and other work, all behind everything, for the front of the client is transparent, you can simply think that behind the connection is an infinite memory Redis service.

The latest release of Codis is CODIS-3.2, and CODIS-Server is based on Redis-3.2.8. There are the following components:

Codis Server: Based on redis-3.2.8 branch development. Additional data structures have been added to support slot-related operations and data migration instructions.
Codis Proxy: The Redis Proxy service that the client connects to and implements the Redis protocol. Except for some commands that are not supported (unsupported command list), there is no difference between native Redis (like Twemproxy).
Codis Dashboard: a cluster management tool that supports adding, deleting, and data migration of CODIS-proxy and CODIS-Server. When the cluster status changes, coDIS-Dashboard maintains the status consistency of all CODIS-proxies in the cluster. For a service cluster, only zero or one CoDIS-Dashboard can be deployed at a time. All changes to the cluster must be made via codis-Dashboard.
Codis Admin: command line tool for cluster management. It can be used to control coDIS-Proxy and CODIS-Dashboard status and access external storage.
Codis FE: cluster management interface. Multiple cluster instances can share the same front-end display page; Manage the back-end CODIS-Dashboard list through the configuration file. The configuration file can be automatically updated.
Storage: Provides external Storage for the cluster status. Provides the concept of Namespace. Different clusters are organized according to different product names. At present, only Zookeeper, Etcd, Fs three implementations are provided, but the abstract interface can be self-extended.

As for the specific installation and use, see the official website of CodisLabs, not involved here.

Codis features:

Codis supports more diverse commands and basically supports redis commands.
Migration cost is low, migration to CODIS is not so troublesome, as long as the use of redis command within the scope of CODIS support, as long as you modify the configuration can access.
Codis provides a more user-friendly o&M tool that provides a Web graphical interface to manage clusters.
Supports multi-core CPU, twemProxy can only single-core
Group division is supported. One master and multiple slaves can be set in the group. Sentinel monitors the master and slave of Redis, and automatically switches the slave to the master when the master goes down

Routing query

Redis Cluster is a server Sharding technology available in version 3.0. Instead of using a consistent hash, the Redis Cluster uses the concept of slots, which are divided into 16,384 slots. The request is sent to any node, and the node that receives the request sends the query request to the correct node for execution. When a client-side key is not assigned to a node, as with a single Redis instance, when a client-side key is not assigned to a node, Redis returns a redirect to the correct node, sort of like a 302 redirect from a browser page.

If a node fails, its slots fail, and the whole cluster fails. In order to increase the accessibility of the cluster, the official recommended solution is to configure nodes as a master slave structure, that is, a master node with n slave nodes. At this time, if the master node fails, the Redis Cluster will choose a slave node to become the master node according to the election algorithm, and the whole Cluster will continue to provide services.

Features:

Without a central architecture, it supports dynamic capacity expansion and is transparent to services
Capable of Sentinel monitoring and automatic Failover
The client does not need to connect to all nodes in the cluster, but to any available node in the cluster
High performance, the client directly connects to redis service, eliminating the loss of proxy

The disadvantage is that operation and maintenance is also very complex, data migration requires manual intervention, can only use 0 database, does not support batch operation, distributed logic and storage module coupling, etc.

The model selection finally determines redis Cluster. The main reasons are high performance and decentralized support for scaling. For data migration in operation and maintenance, there is no particularly mature solution in the industry. Redis Cluster is provided by Redis official, and we look forward to perfect support from Redis official in the future.

The installation

It is recommended that a cluster needs at least six nodes, that is, three active nodes and three standby nodes. The configuration files of the six nodes are the same. You only need to change the port number.

port 7000
cluster-enabled yes  Enable cluster mode
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
Copy the code

After startup, you can see the following log.

[82462] 26 Nov 11:56:55. 329 * No cluster configuration found, I ‘m 97 a3a64667477371c4479320d683e4c8db5858b1

Since no nodes.conf exists, each instance is assigned an ID when it starts. In order to have a unique name in the clustered environment, the ID will be used permanently. Each instance holds an ID used by other nodes, rather than by IP and port. The IP and port may change, but the unique node ID will not change until the node dies.

We have now started six instances of Redis and need to create clusters by writing some meaningful configuration information to each node. Redis Cluster’s command line tool, Redis-trib, uses Ruby programs to execute special commands on instances, making it easy to create new clusters, check or reshard existing clusters, etc.

./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001\127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005Copy the code

— Replicas 1 adds a slave to each master.

Configuration JedisClusterConfig

@Configuration
public class JedisClusterConfig {
    private static Logger logger = LoggerFactory.getLogger(JedisClusterConfig.class);

    @Value("${redis.cluster.nodes}")
    private String clusterNodes;

    @Value("${redis.cluster.timeout}")
    private int timeout;

    @Value("${redis.cluster.max-redirects}")
    private int redirects;
    
    @Autowired
    private JedisPoolConfig jedisPoolConfig;

    @Bean
    public RedisClusterConfiguration getClusterConfiguration(a) {
        Map<String, Object> source = new HashMap();

        source.put("spring.redis.cluster.nodes", clusterNodes);
        logger.info("clusterNodes: {}", clusterNodes);
        source.put("spring.redis.cluster.max-redirects", redirects);
        return new RedisClusterConfiguration(new MapPropertySource("RedisClusterConfiguration", source));
    }

    @Bean
    public JedisConnectionFactory getConnectionFactory(a) {
        JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(getClusterConfiguration());
        jedisConnectionFactory.setTimeout(timeout);
        return jedisConnectionFactory;
    }

    @Bean
    public JedisClusterConnection getJedisClusterConnection(a) {
        return (JedisClusterConnection) getConnectionFactory().getConnection();
    }

    @Bean
    public RedisTemplate getRedisTemplate(a) {
        RedisTemplate clusterTemplate = new RedisTemplate();
        clusterTemplate.setConnectionFactory(getConnectionFactory());
        clusterTemplate.setKeySerializer(new StringRedisSerializer());
        clusterTemplate.setDefaultSerializer(new GenericJackson2JsonRedisSerializer());
        returnclusterTemplate; }}Copy the code

Passwords can be configured. Cluster is not very password friendly. Requirepass and MasterAuth need to be set if a password is set for a cluster, otherwise you will encounter authorization issues when a master/slave switch occurs.

Configure redis cluster

redis:
  cluster:
    enabled: true
    timeout: 2000
    max-redirects: 8
    nodes: 127.0. 01.: 7000127.00 0.1:7001
Copy the code

The nodes and timeout period of redis Cluster are configured.

Using RedisTemplate

@RunWith(SpringRunner.class)
@SpringBootTest
public class RedisConfigTest {
    @Autowired
    RedisTemplate redisTemplate;
    
    @Test
    public void clusterTest(a) {
      redisTemplate.opsForValue().set("foo"."bar");
      System.out.println(redisTemplate.opsForValue().get("foo")); }}Copy the code

The usage is very simple, injection RedisTemplate can be operated, RedisTemplate usage is rich, you can consult.

conclusion

This paper mainly talks about the selection of REDis cluster, there are three main types: client sharding, proxy based sharding and routing query. For the first two ways, respectively, a brief introduction, finally choose redis Cluster program officially provided by Redis, and practice. Although the official version has not been released for a long time and there are not many successful practice cases, the overall design of Redis Cluster is relatively simple, and most operations can be carried out according to a single point of operation process. I use the Jedis client to support JedisCluster is also better, it is also very convenient to use. In fact, there is a pressure data, we will fill in later.

Subscribe to the latest articles, welcome to follow my official account

reference

What should Redis cluster solution do?
cluster-tutorial
twemproxy
CodisLabs