Redis

Introduction to Redis

Nosql basic concepts

In order to solve a series of problems such as high concurrency, high availability, high scalability, big data storage and database solutions, is NoSql.

NoSql, called non-relational database, its full name is Not only SQL. It can not replace the relational database, only as a good supplement to the relational database.

No classification

  • Typical applications: Content caching, which is mainly used to handle high access loads of large amounts of data. Data model: a series of key-value pairs Advantage: quick query Disadvantage: Lack of structure of stored data

  • Column storage Database products: Cassandra, HBase, and Riak Typical application: Distributed file system Data model: Stores data in a column cluster. Advantages: Fast search speed, strong scalability, and easy distributed expansion Disadvantages: Limited functions

  • Document database products: CouchDB and MongoDB Typical application: Web application (similar to key-value, Value is structured) Data model: a series of key-value pairs Advantages: not strict data structure Disadvantages: Low query performance and lack of unified query syntax

  • Graph database related database: Neo4J, InfoGrid, Infinite Graph Typical application: Social network data model: Graph structure Advantage: Graph structure correlation algorithm. Disadvantages: Need to calculate the whole graph to get the result, not easy to do distributed cluster scheme.

Basic concepts of Redis

Redis is a high performance key value database developed in C language. Redis can store data through several key value types. Key value type: String Character type MAP Hash type List List type SET Set type sortedSet Type of an ordered set

The development history

In 2008 Merzia, an Italian start-up, launched LLOOGG, a Real-Time web statistics system based on MySQL. But it wasn’t long before Salvatore Sanfilippo, the company’s founder, became frustrated with MySQL’s performance. He decided to create a database for LLOOGG himself, which he developed in 2009: Redis. But Salvatore Sanfilippo wasn’t satisfied with just using Redis for LLOOGG. Instead, they wanted more people to use it. So in the same year, Salvatore Sanfilippo made Redis open source. He began working with Pieter Noordhuis, another major Redis contributor, to continue development of Redis to this day.

Salvatore Sanfilippo didn’t expect that in just a few years, Redis would have a huge user base. Hacker News published a database usage survey in 2012, which found that nearly 12 percent of companies were using Redis. In China, such as Sina Weibo, Jiepang.com, Zhihu, and in foreign countries, such as GitHub, Stack Overflow, Flickr, etc., are all Redis users.

VMware has sponsored the development of Redis since 2010, and Salvatore Sanfilippo and Pieter Noordhuis joined VMware in March and May, respectively, to develop Redis full-time.

Application scenarios

Caching (data queries, short links, news content, product content, and so on). (most used) Session separation in distributed cluster architectures. A chat room’s list of online friends. Task queues. (Seckill, Snap, 12306, etc.) Website visit statistics. Data expiration processing (to the millisecond)

2 Redis installation

download

Website address: http://redis.io/ download address: http://download.redis.io/releases/redis-3.0.0.tar.gz

The installation

# SFTP Upload the installation package to Linux
# decompressionThe tar - ZXVF redis. 3.0.0. Tar. GzInstall the C environment
sudo apt-get install gcc-c++
# compiler
cdRedis - 3.0.0 make# installation
make install PREFIX = /usr/local/redis
# check
cd /usr/local/redis
ls
Copy the code

Redis start

The front-end start

  • Front-end start command./redis-server
  • The shutdown of front-end startup forces the shutdownctrl+cNormally closed./redis-cli shutdown

Once the client is closed, the Redis service will also stop

The back-end start

  • Copy the redis. Conf file from the redis source package to the bin directoryCp/root/redis - 3.0.0 / redis. Conf
  • Modify the redis.conf file to change daemonize to yesvim redis.conf
  • Start Redis using back-end commands./redis-server redis.conf
  • Check whether the startup is successfulps -aux | grep redis
  • Disabling the backend startup mode to forcibly disable:Kill -9 Indicates the process IDNormal shutdown:./redis-cli shutdown

3 Redis client

Built-in Client

  • Start the/redis-cli -h 127.0.0.1 -p 6379-h Specifies the IP address of the redis server to be accessed. -p Specifies the port of the Redis server to be accessed./redis-cli Uses the default IP address 127.0.0.1 and port 6379
  • Close CTRL + C 127.0.0.1:6379>quit

Graphical interface client

Redis-desktop-manager opens as follows:

Select database mode: Select with the subscript of the database, you can select the specified database. The subscript starts from 0.

127.0.0.1:6379> select 15
OK
127.0.0.1:6379[15]>
Copy the code

Jedis client

introduce

Redis not only uses commands to operate, now basically mainstream languages have client support, such as Java, C, C#, C++, PHP, Node.js, Go and so on.

Jedis, Redisson, Jredis, jjDBC-redis, Jedis, Redisson, Jedis, jjDBC-redis, Jedis, Redisson, Jedis, jjDBC-redis, Jedis, JjdbC-redis, Jedis, Redisson, Jedis, Jedis The most commonly used in enterprises is Jedis, and we will focus on studying Jedis.

Jedis also hosted on the lot, address: https://github.com/xetorthio/jedis

Engineering structures,

  • Add jar commons-pool2-2.3.jar jedis-2.7.0.jar
Singleton connection redis

Connect to Redis using a connection pool

Spring integration jedisPool

Add the Spring jar package to configure the Spring configuration file applicationContext.xml

<?xml version="1.0" encoding="UTF-8"? >
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mvc="http://www.springframework.org/schema/mvc"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
    xsi:schemaLocation="Http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.2.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.2.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.2.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-3.2.xsd">

    <! -- Connection pool configuration -->
    <bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">
        <! -- Max connections -->
        <property name="maxTotal" value="30" />
        <! -- Maximum number of free connections -->
        <property name="maxIdle" value="10" />
        <! -- The maximum number of connections released each time -->
        <property name="numTestsPerEvictionRun" value="1024" />
        <! -- Release connection scan interval (ms) -->
        <property name="timeBetweenEvictionRunsMillis" value="30000" />
        <! -- Connection minimum idle time -->
        <property name="minEvictableIdleTimeMillis" value="1800000" />
        <! -- How long after the connection is idle, release when the idle time > this value and idle connections > the maximum number of idle connections -->
        <property name="softMinEvictableIdleTimeMillis" value="10000" />
        <! -- Maximum number of milliseconds to wait for a connection, less than zero: block indefinite time, default -- 1 -->
        <property name="maxWaitMillis" value="1500" />
        <! -- Check the validity of the connection when obtaining it. Default is false -->
        <property name="testOnBorrow" value="false" />
        <! -- Check validity when idle, default false -->
        <property name="testWhileIdle" value="true" />
        <! -- whether to block when connections are exhausted, false to report exceptions, true to block until timeout, default true -->
        <property name="blockWhenExhausted" value="false" />
    </bean>

    <! -- Redis standalone connection pool -->
    <bean id="jedisPool" class="redis.clients.jedis.JedisPool"
        destroy-method="close">
        <constructor-arg name="poolConfig" ref="jedisPoolConfig" />
        <constructor-arg name="host" value="192.168.242.130" />
        <constructor-arg name="port" value="6379" />
    </bean>
</beans>
Copy the code

The test code

@Test
    public void testJedisPool(a) {
        JedisPool pool = (JedisPool) applicationContext.getBean("jedisPool");
        Jedis jedis = null;
        try {
            jedis = pool.getResource();

            jedis.set("name"."lisi");
            String name = jedis.get("name");
            System.out.println(name);
        } catch (Exception ex) {
            ex.printStackTrace();
        } finally {
            if(jedis ! =null) {
                // Close the connectionjedis.close(); }}}Copy the code

4 Data Types

Type String

Set key value

127.0.0.1:6379 >set test 123
OK
Copy the code

Get the key value

127.0.0.1:6379 > gettest
"123"
Copy the code

Value and assign getSet key value

127.0.0.1:6379 > getsettest 321
"123"127.0.0.1:6379 > gettest
"321"
Copy the code

Mset key value [key value… mget key [key…]

127.0.0.1:6379> mset k1 v1 k2 v2 k3 v3
OK
127.0.0.1:6379> mget k1 k2
1) "v1"
2) "v2"
Copy the code

Delete the del key

127.0.0.1:6379 > deltest
(integer1)Copy the code

Numerical increase or decrease

  • Increment number When the string stored is an integer, Redis provides a useful command, INCR, that increments the current key value and returns the incremented value. Syntax: incr key 
    127.0.0.1:6379 >setNum 1 OK 127.0.0.1:6379> incr num (integer) 2
127.0.0.1:6379> incr num
(integer) 3
127.0.0.1:6379> incr num
(integer4)Copy the code
  • Increments the specified integer incrby key increment 
    127.0.0.1:6379> incrby num 2
(integer) 8
127.0.0.1:6379> incrby num 2
(integer10)Copy the code
  • Decr key 
    127.0.0.1:6379 > decr num (integer) 9
127.0.0.1:6379> decr num
(integer8)Copy the code
  • Decreases the specified value decryby Key Decrement 
    127.0.0.1:6379> decrby num 2
(integer) 6
127.0.0.1:6379> decrby num 2
(integer4)Copy the code

APPEND appends the value to the end of the key value. If the key does not exist, SET the value of the key to value. That is, SET key value. The return value is the total length of the appended string. Syntax: append key value

127.0.0.1:6379 >setSTR hello OK 127.0.0.1:6379> append STR"world"
(integer) 10 127.0.0.1:6379> get STR"helloworld"
Copy the code

STRLEN returns the length of the key value, or 0 if no key exists. Syntax: strlen key

127.0.0.1:6379 > strlen STR (integer10)Copy the code

application

  • The product number and order number of the auto-increment primary key are generated using the increasing number feature of String

Hash Hash type

The problem with using strings

Assume that a User object is stored in Redis in the form of JSON serialization. The User object has id, username, password, age, name and other attributes. The storage process is as follows: Save and update: User object JSON (string) Redis

What should I do if I update only the age attribute in business and do not update the other attributes? If the above method is still used, resources will be wasted during transmission and processing. The following hash can solve this problem well.

introduce

A hash, called a hash type, provides a mapping of fields to field values. The field value can only be a string. Other types such as hash type and collection type are not supported. As follows:

The command

The HSET assignment does not distinguish between an insert and an update. The HSET assignment returns 1 when an insert is performed and 0 when an update is performed.

  • Syntax for setting one field value at a time: hset key field value 
    127.0.0.1:6379> hset user username zhangsan
(integer1)Copy the code
  • Hmset key field value [field value…] 
    127.0.0.1:6379> hmset user age 20 username lisi
OK
Copy the code
  • Assign if the field does not exist, similar to hset, except that if the field does exist, this command does nothing. Syntax: hsetnx key field value 
    127.0.0.1:6379> hsetnx user age 30
(integer) 0
Copy the code

The values

  • Syntax for getting one field value at a time: hget key field 
    127.0.0.1:6379 > hget user username"lisi"
Copy the code
  • Hmget key field [field…] 
    127.0.0.1:6379> hmget user age username
1) "20"
2) "lisi"
Copy the code
  • Syntax for getting all field values: hgetall Key 
    127.0.0.1:6379 > hgetall user 1)"username"
2) "lisi"
3) "age"
4) "20"
Copy the code

Delete fields You can delete one or more fields, and the return value is the number of fields that were deleted. Hdel key field [field…]

127.0.0.1:6379> hdel user age
(integer) 1
127.0.0.1:6379> hdel user age username
(integer1)Copy the code

Add number syntax: hincrby Key field increment

127.0.0.1:6379> hincrby user age 2
(integer2)Copy the code

Syntax for determining whether a field exists: hexists key field

127.0.0.1:6379> hexists user age
(integer1)Copy the code

Syntax for obtaining only the field name or value: hkeys key hVALS key

127.0.0.1:6379 > hkeys user 1)"age"
Copy the code

Syntax for obtaining the number of fields: hlen key

127.0.0.1:6379 > hlen user (integer1)Copy the code

The application stores product information

127.0.0.1:6379 > hlen user (integer1)Copy the code

Obtaining commodity information

1001 1) 127.0.0.1:6379 > hgetall items:"id"
2) "3"
3) "name"
4) "apple"
5) "price"
6) "5.00"
Copy the code

List the type

Difference between ArrayList and LinkedList

Arraylist uses arrays to store data. It is fast to query and slow to add and delete data

Linkedlist uses a two-way Linkedlist to store data. It is fast to add and delete data, slow to query data, but also fast to query data on both ends of the Linkedlist.

The list of Redis is stored in linked lists, so the operation of the list data type of Redis is operated on the data at both ends of the list.

The command

Add elements to both ends of the list

  • Add element syntax to left side of list: lpush key value [value…]
127.0.0.1:6379> lpush list:1 12 3 (integer) 3
Copy the code
  • Add element syntax to right side of list: rpush key value [value…]
127.0.0.1:6379> rpush list:1 4 5 6
(integer6)Copy the code

The LRANGE command is one of the most common commands for list types. To retrieve a fragment of a list, all elements between start and stop (including elements at both ends) are returned. The index starts at 0. The index can be negative, for example, “-1” represents an element on the last edge.

Syntax: lrange key start stop

127.0.0.1:6379> lrange list:1 0 2
1) "3"
2) "2"
3) "1"127.0.0.1:6379> lrange list:1 0-1 1)"3"
2) "2"
3) "1"
4) "4"
5) "5"
6) "6"
Copy the code

The LPOP command pops an element from the left of the list in two steps:

  • The first step is to remove the element to the left of the list from the list
  • The second step is to return the value of the removed element. Syntax: LPOP key RPOP key 
    127.0.0.1:6379 > lpop list: 1"3"127.0.0.1:6379 > rpop list: 1"6"
Copy the code

Syntax for getting the number of elements in a list: llen key

1 (127.0.0.1:6379 > llen list:integer4)Copy the code

LREM deletes the first count of elements whose value is value and returns the actual number of deleted elements. The command is executed differently depending on the count value:

  • When count>0, LREM deletes from the left side of the list.
  • When count<0, LREM deletes from behind the list.
  • When count=0, LREM deletes all elements with value.

Syntax: LREM key count value

Gets/sets the element value of the specified index

  • Get the element value syntax for the specified index: lindex key index 
    127.0.0.1:6379 > lindex list: 1. 2"4"
Copy the code
  • Sets the element value syntax for the specified index: lset key index value 
    127.0.0.1:6379> lset list:1 2 2
OK
Copy the code

Only the range specified in the list is reserved: ltrim key start stop

127.0.0.1:6379> lrange list:1 0-1 1)"2"
2) "1"
3) "2"
4) "5"127.0.0.1:6379> ltrim list:1 0 2 OK 127.0.0.1:6379> lrange list:1 0-1 1)"2"
2) "1"
3) "2"
Copy the code

Inserting an element into a list This command first searches the list left to right for the element with the value pivot, and then determines whether to insert value BEFORE or AFTER the element, depending on whether the second argument is BEFORE or AFTER. Grammar: linsert key before | after the pivot value

127.0.0.1:6379> lrange list:1 0-1 1)"2"
2) "1"
3) "2"
127.0.0.1:6379> linsert list:1 after 1 9
(integer) 4 127.0.0.1:6379> lrange list:1 0-1 1)"2"
2) "1"
3) "9"
4) "2"
Copy the code

Syntax for moving elements from one list to another: rpoplpush source destination

127.0.0.1:6379> lrange list:1 0-1 1)"2"
2) "1"
3) "9"
4) "2"Newlist 127.0.0.1:6379 > rpoplpush list: 1"2"127.0.0.1:6379> lrange newList 0-1 1)"2"127.0.0.1:6379> lrange list:1 0-1 1)"2"
2) "1"
3) "9"
Copy the code

The user publishes the product review, converts the comment information into JSON and stores it in the list. The user queries the list of comments on the page and retrieves json data from Redis to display on the page.

Define the item review list. Key: the item review key whose id is 1001.

The Set type

Collection type: Unordered, non-repeatable List type: ordered, repeatable

The command

Sadd key member [member…]

127.0.0.1:6379 > saddset a b c
(integer) 3 127.0.0.1:6379 > saddset a
(integer) 0
Copy the code

Srem key member [member…]

127.0.0.1:6379 > sremset c
(integer1)Copy the code

Get the syntax for all elements in the collection: smembers key

127.0.0.1:6379 > smembersset
1) "b"
2) "a"
Copy the code

Sismember key member

127.0.0.1:6379 > sismemberset a
(integer127.0.0.1) 1:6379 > sismemberset h
(integer) 0
Copy the code

Operation command

The difference set operation of sets A-B is the set of elements that belong to A and not to B

127.0.0.1:6379 > saddsetA 1 2 3
(integer) 3 127.0.0.1:6379 > saddsetB 2 3 4
(integer) 3 127.0.0.1:6379 > sdiffsetA setB 
1) "1"127.0.0.1:6379 > sdiffsetB setA
1) "4"
Copy the code

The intersection operation of the set belongs to the set of elements A and B

127.0.0.1:6379 > sintersetA setB
1) "2"
2) "3"
Copy the code

The union of sets is the set of the elements of A or B

127.0.0.1:6379 > sunionsetA setB
1) "1"
2) "2"
3) "3"
4) "4"
Copy the code

Syntax for getting the number of elements in a collection: scard key

127.0.0.1:6379 > smemberssetA
1) "1"
2) "2"
3) "3"127.0.0.1:6379 > scardsetA
(integer) 3
Copy the code

Eject an element from the collection Note: Since the collection is unordered, all SPOP commands will eject a random element from the collection. Syntax: spop key

127.0.0.1:6379 > spopsetA
"2"
Copy the code

Sortedset type

Sortedset or zset Sortedset is an ordered set, sortable, but unique. The difference between a Sortedset and a set is that you add a score to the elements in the set and sort by that score.

The command

Add elements

Adds an element and its fraction to an ordered set, replacing the original fraction if the element already exists. The return value is the number of new elements added to the collection, excluding pre-existing elements. Zadd key score member [score member…

127.0.0.1:6379> zadd scoreboard 80 zhangsan 89 lisi 94 wangwu
(integer) 3
127.0.0.1:6379> zadd scoreboard 97 lisi
(integer) 0
Copy the code
Get element scores

Syntax: zscore key member

127.0.0.1:6379 > zscore "lisi"97"
Copy the code
Remove elements

Removes one or more members of the ordered set key. Nonexistent members are ignored. An error is returned when key exists but is not of an ordered set type.

Zrem key member [member…]

127.0.0.1:6379> zrem scoreboard lisi
(integer1)Copy the code
Gets a list of elements ranked in a range

Gets a list of elements ranked in a range

  • Zrange key start stop [withscores] zrange key start stop [withscores] 
    127.0.0.1:6379> zrange scoreboard 0 2
1) "zhangsan"
2) "wangwu"
Copy the code
  • Zrevrange key start stop [withscores] 
    127.0.0.1:6379> zrevrange scoreboard 0 2
1) "wangwu"
2) "zhangsan"
Copy the code

    You can add the withscores parameter ·· 127.0.0.1:6379> zrevrange scoreboard 0 2 withscores to the end of the command to obtain the scores of the elements

    1. “wangwu”
    2. “94”
    3. “zhangsan”
    4. “80”…
Gets the ranking of elements
  • From small to large grammar: zrank key member
127.0.0.1:6379> zrank scoreboard zhangsan
(integer) 0
Copy the code
  • From large to small syntax: zrevrank key member
127.0.0.1:6379> zrevrank scoreboard zhangsan
(integer1)Copy the code
Gets the element of the specified score range

Zrangebyscore key min Max [withscores] [limit offset count]

127.0.0.1:6379> ZRANGEBYSCORE 90 97 WITHSCORES 1)"wangwu"
2) "94"
3) "lisi"
4) "97"
127.0.0.1:6379> ZRANGEBYSCORE scoreboard 70 100 limit1 2, 1)"wangwu"
2) "lisi"
Copy the code
Increments the score of an element

The return value is the changed score syntax: zincrby Key Increment Member

127.0.0.1:6379> ZINCRBY scoreboard 4 lisi 
"101"Copy the code
Gets the number of elements in the collection

Syntax: zcard key

127.0.0.1:6379 > zcard "(integer) 3
Copy the code
Gets the number of elements in the specified range of fractions

Syntax: zcount key min Max

127.0.0.1:6379> zcount scoreboard 80 90
(integer1)Copy the code
Remove elements by rank range

Syntax: zremrangebyrank key start stop

127.0.0.1:6379> zremrangebyrank scoreboard 0 1integer) 2 127.0.0.1:6379> zrange scoreboard 0-1 1)"wangwu"
Copy the code
Remove elements according to the score range

Syntax: zremrangebyscore key min Max

127.0.0.1:6379> zadd scoreboard 84 zhangsan    
(integer) 1
127.0.0.1:6379> ZREMRANGEBYSCORE scoreboard 80 100
(integer1)Copy the code
application
Merchandise sales list

We define a sorted list (set) with Key items as sellsort and score as sales volume.

Write commodity sales volume:

  • ZADD Items: Sellsort 9 1001 10 1002 192.168.101.3:7007> ZADD Items: Sellsort 9 1001 10 1002

  • Item number 1001 sales + 1 192.168.101.3:7001> ZINCRBY items: Sellsort 1 1001

  • Top 10 items: 192.168.101.3:7001> ZRANGE items:sellsort 0 9 withscores

Keys command

Common commands

  • Keys returns all keys that satisfy the given pattern redis 127.0.0.1:6379> keys myList *

    1. “mylist”
    2. “mylist5”
    3. “mylist6”
    4. “mylist7”
    5. “mylist8”

  • Example for confirming the existence of a key: According to the result, there is no HongWan key in the database. Redis 127.0.0.1:6379> exists HongWan (integer) 0 redis 127.0.0.1:6379> exists age (integer) 1 redis 127.0.0.1:6379 >

  • Del Deletes a key redis 127.0.0.1:6379> del age (integer) 1 redis 127.0.0.1:6379> exists age (integer) 0

  • Rename Rename key Example: age successfully renamed to age_new redis 127.0.0.1:6379[1]> keys *

    1. “age” redis 127.0.0.1:6379[1]> rename age age_new OK redis 127.0.0.1:6379[1]> keys *
    2. “Age_new redis 127.0.0.1:6379 [1] >

  • Type An example of the type of the return value: 127.0.0.1:6379> type addr String redis 127.0.0.1:6379> type myzset2 zset redis 127.0.0.1:6379> Type mylist list redis 127.0.0.1:6379>

Set the lifetime of the key

In actual use, Redis is more used as a cache. However, the cached data generally needs to set the survival time, that is, the data will be destroyed after expiration.
EXPIRE key seconds Set key lifetime (unit: second) Specifies the number of seconds after the key is automatically deleted
TTL key Check the remaining lifetime of the key
PERSIST key Clear life time
PEXPIRE key milliseconds The unit is milliseconds

Example:

192.168.101.3:7002 >set test1 settest192.168.101.3:7002> gettestTo obtaintestThe value of the"1"192.168.101.3:7002 > EXPIREtest5 settestSurvival time of 5 seconds (integer) 1 192.168.101.3:7002 > TTLtestTo viewtestThe generation time of the birth is 1 second. Delete (integer) 1 192.168.101.3:7002 > TTLtest
(integer) - 2 192.168.101.3:7002 > gettestTo obtaintestThe value of, has been deleted (nil)Copy the code

Redis persistence scheme

Rdb way

Redis uses snapshots to persist data to disks.

Set the conditions for persistent snapshots

Modify the conditions of persistent snapshots in redis.conf:

The directory where persistent files are stored

In redis. Conf, you can specify the directory to store persistent files

The problem of Rdb

Once Redis is turned off illegally, the data after the last persistence is lost.

If the data isn’t important, don’t care. If data cannot be lost, use aOF.

Aof way

Redis does not persist in this way by default. In Aof mode, the redIS database is operated once, and the operation records are stored in Aof persistence file.

  • Step 1: Enable aOF persistence. Change appendOnly in redis. Conf to yes to enable aof persistence.

  • Aof files are stored in the same directory as RDB files. Aof Specifies the name of the file store

When using AOF and RDB, if Redis is restarted, the data is loaded from the AOF file.

Master/slave replication of Redis

What is master-slave replication

Persistence ensures that data will not be lost even if the REDis service is restarted. After the Redis service is restarted, persistent data will be restored to the memory. However, when the redis server’s hard disk is damaged, data may be lost.

  • Data in the primary Redis has two replicas, namely, redis1 and redis2. Even if one Redis server goes down, the other two Redis services can continue to provide services.
  • Data in the master REDis is synchronized with data on the slave REDis in real time. When data is written by the master Redis, it is copied to the two slave Redis services through the master/slave replication mechanism.
  • There is only one primary redis, and there can be multiple secondary Redis.
  • Master/slave replication does not block the master, and the master can continue processing client requests while data is synchronized
  • A Redis can be both master and slave, as shown below:

Primary/secondary replication Settings

Host configuration

Don’t need to configure

From the machine configuration

  • Step 1: Make a copy of the slave machinecp bin/ bin2 -r
  • Step 2: Modify the redis.conf syntax of the slave machine: slaveof masterip masterport slaveof 192.168.242.137 6379
  • Step 3: Change the slave port address to 6380
  • Step 4: Clear persistent files on the slave machinerm -rf appendonly.aof dump.rdb
  • Step 5: Start the slave machine./redis-server redis.conf
  • Step 6: Start the 6380 client./redis-cli -p 6380

Note: Once a add, delete, or change operation is performed on the host, the slave machine will synchronize data to the slave machine and the slave machine cannot perform write operations

Redis cluster

Redis – cluster architecture diagram

Architectural details: (1) All Redis nodes are ping-pong with each other, and binary protocols are used internally to optimize transmission speed and bandwidth; (2) The failure of nodes takes effect only when more than half of the nodes in the cluster detect failure; (3) the client is directly connected to the Redis nodes, without the need for an intermediate proxy layer. (4) Redis-cluster maps all physical nodes to [0-16383]slot. The cluster is responsible for the maintenance of node<->slot<->value in the redis cluster 16384 hash slots, when a key-value needs to be placed in the Redis cluster, Redis first uses crC16 algorithm to calculate a result for key, and then calculates the remainder of the result for 16384. In this way, each key will correspond to a hash slot numbered between 0 and 16383, and Redis will map hash slots to different nodes roughly equally based on the number of nodes.

Redis-cluster voting is fault-tolerant

(1) All the master nodes in the cluster vote. If more than half of the master nodes communicate with one master node for more than (cluster-node-timeout), the master node is considered to be down. (2): When does the entire cluster become unavailable (Cluster_state :fail)?

  • If any master in the cluster fails and no slave exists in the current master state, the cluster enters the Fail state. If the mapping between slots [0-16383] of the cluster is incomplete, the slot fails.
  • If more than half of the masters in a cluster fail, the cluster enters the Fail state regardless of whether there are slaves.

Install Ruby

The cluster management tool (Redis-trib.rb) is written using the Ruby scripting language.

  • Install ruby
sudo apt-get install ruby
Copy the code
  • Upload redis-3.0.0.gem to Linux
  • Install the Ruby and Redis interfacesThe gem install redis - 3.0.0. Gem
  • Copy the following files from the SRC directory under the redis-3.0.0 package to redis/redis-cluster/
cd /usr/local/redis/
mkdir redis-cluster
cd/ root/redis - 3.0.0 / SRC/cp redis - trib. Rb/usr /local/redis/redis-cluster
Copy the code

Set up the cluster

A cluster requires at least three hosts. If each host is configured with a slave host, at least six hosts are required. Port design: 7001-7006

  1. Make a copy of machine 7001cp bin ./redis-cluster/7001 -r
  2. If persistent files exist, delete themrm -rf appendonly.aof dump.rdb
  3. Set cluster parameters and modify redis.conf
  4. Modify the port
  5. Copy machines 7002-7006
cp 7001/ 7002-r
cp 7001/ 7003-r
cp 7001/ 7004-r
cp 7001/ 7005-r
cp 7001/ 7006-r
Copy the code
  1. Modify the machine port 7002-7006
  2. Create the start-all.sh file
cd 7001
./redis-server redis.conf
cd.cd 7002
./redis-server redis.conf
cd.cd 7003
./redis-server redis.conf
cd.cd 7004
./redis-server redis.conf
cd.cd 7005
./redis-server redis.conf
cd.cd 7006
./redis-server redis.conf
cd.Copy the code
  1. Modifying file Permissions
chmod u+x start-all.sh
Copy the code
  1. Execute the file and start the six machines
./start-all.sh
Copy the code
  1. Create the cluster./redis-trib.rb create --replicas 1 192.168.126.128:7001 192.168.126.128:7002 192.168.126.128:7003 192.168.126.128:7004 192.168.126.128:7005 192.168.126.128:7006

Connect the cluster

root@ubuntu:/usr/local/redis/redis-cluster/7001# ./redis-cli -p 7001 -c

-c Specifies the cluster connection

Viewing Cluster Information

  • Viewing Cluster Information

    192.168.126.128:7002> cluster info Cluster_state: OK Cluster_SLOts_assigned :16384 Cluster_SLOts_OK :16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:2 cluster_stats_messages_sent:260 cluster_stats_messages_received:260Copy the code

  • Viewing cluster Nodes

    192.168.126.128:7002 > cluster nodes 3 a15e73dacb512745156535ae7f959acf65ae12e 192.168.126.128:7005 slave 23e173cdc0b7673dc28cae70efaabbc41308bfdc 0 1531452321139 5 connected 2a58a53a5b10f7bd91af04128a6ed439d534c0ee 192.168.126.128: master 7001-0, 1531452322145 1 connected d0808388485dd08f1a2ecdfe3d2b213742d0050d 0-5460 192.168.126.128:7004 slave 2 a58a53a5b10f7bd91af04128a6ed439d534c0ee 0 1531452318117 4 connected 23 e173cdc0b7673dc28cae70efaabbc41308bfdc 192.168.126.128:7002 myself, master - 0 0 2 connected. 5461-10922 2af2312acc56552f9f73470f90d9a51973fc74 d3 192.168.126.128:7006 slave faf92cfdbd12e1b27b270fb0798e67017f4d0b 0 78 1531452320132 6 connected 78 faf92cfdbd12e1b27b270fb0798e67017f4d0b 192.168.126.128: master 7007-0 1531452319123 3 connected. 10923-16383Copy the code

Jedis connects clusters

@Test
    public void jedisCluster(a) {
        / / create jedisCluster
        Set<HostAndPort> nodes = new HashSet<>();
        nodes.add(new HostAndPort("192.168.242.137".7001));
        nodes.add(new HostAndPort("192.168.242.137".7002));
        nodes.add(new HostAndPort("192.168.242.137".7003));
        nodes.add(new HostAndPort("192.168.242.137".7004));
        nodes.add(new HostAndPort("192.168.242.137".7005));
        nodes.add(new HostAndPort("192.168.242.137".7006));
        nodes.add(new HostAndPort("192.168.242.137".7007));

        JedisCluster cluster = new JedisCluster(nodes);
    
        cluster.set("s4"."444");
    
        String result = cluster.get("s4");
        System.out.println(result);
    
        cluster.close();
    }
Copy the code

Using the spring

Configuration applicationContext. XML

<! -- Connection pool configuration -->
<bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">
    <! -- Max connections -->
    <property name="maxTotal" value="30" />
    <! -- Maximum number of free connections -->
    <property name="maxIdle" value="10" />
    <! -- The maximum number of connections released each time -->
    <property name="numTestsPerEvictionRun" value="1024" />
    <! -- Release connection scan interval (ms) -->
    <property name="timeBetweenEvictionRunsMillis" value="30000" />
    <! -- Connection minimum idle time -->
    <property name="minEvictableIdleTimeMillis" value="1800000" />
    <! -- How long after the connection is idle, release when the idle time > this value and idle connections > the maximum number of idle connections -->
    <property name="softMinEvictableIdleTimeMillis" value="10000" />
    <! -- Maximum number of milliseconds to wait for a connection, less than zero: block indefinite time, default -- 1 -->
    <property name="maxWaitMillis" value="1500" />
    <! -- Check the validity of the connection when obtaining it. Default is false -->
    <property name="testOnBorrow" value="true" />
    <! -- Check validity when idle, default false -->
    <property name="testWhileIdle" value="true" />
    <! -- whether to block when connections are exhausted, false to report exceptions, true to block until timeout, default true -->
    <property name="blockWhenExhausted" value="false" />
</bean>
<! -- Redis cluster -->
<bean id="jedisCluster" class="redis.clients.jedis.JedisCluster">
    <constructor-arg index="0">
        <set>
            <bean class="redis.clients.jedis.HostAndPort">
                <constructor-arg index="0" value="192.168.101.3"></constructor-arg>
                <constructor-arg index="1" value="7001"></constructor-arg>
            </bean>
            <bean class="redis.clients.jedis.HostAndPort">
                <constructor-arg index="0" value="192.168.101.3"></constructor-arg>
                <constructor-arg index="1" value="7002"></constructor-arg>
            </bean>
            <bean class="redis.clients.jedis.HostAndPort">
                <constructor-arg index="0" value="192.168.101.3"></constructor-arg>
                <constructor-arg index="1" value="7003"></constructor-arg>
            </bean>
            <bean class="redis.clients.jedis.HostAndPort">
                <constructor-arg index="0" value="192.168.101.3"></constructor-arg>
                <constructor-arg index="1" value="7004"></constructor-arg>
            </bean>
            <bean class="redis.clients.jedis.HostAndPort">
                <constructor-arg index="0" value="192.168.101.3"></constructor-arg>
                <constructor-arg index="1" value="7005"></constructor-arg>
            </bean>
            <bean class="redis.clients.jedis.HostAndPort">
                <constructor-arg index="0" value="192.168.101.3"></constructor-arg>
                <constructor-arg index="1" value="7006"></constructor-arg>
            </bean>
        </set>
    </constructor-arg>
    <constructor-arg index="1" ref="jedisPoolConfig"></constructor-arg>
</bean>
Copy the code

The test code

private ApplicationContext applicationContext;
    @Before
    public void init(a) {
        applicationContext = new ClassPathXmlApplicationContext(
                "classpath:applicationContext.xml");
    }

    / / redis cluster
    @Test
    public void testJedisCluster(a) {
        JedisCluster jedisCluster = (JedisCluster) applicationContext
                .getBean("jedisCluster");

        jedisCluster.set("name"."zhangsan");
        String value = jedisCluster.get("name");
        System.out.println(value);
    }
Copy the code