Wechat search public number “code road mark”, point attention not lost!
From the beginning of this article, I will start the topic of Redis and gradually sort out some key points of Redis knowledge and share them. The following points have been sorted out so far. Today, I will share the first article, which is mainly about some basic knowledge points of Redis. Through this article, I will explore and improve the knowledge system of Redis.
What is Redis?
Redis is one of the most popular NoSQL databases available today. Redis is an open source, network-enabled, memory-based, and optionally persistent key-value pair storage database written in ANSI C with multiple data structures. Support distributed, theoretically can be infinitely scalable; Key-value Storage system; Open source, ANSI C language, BSD protocol compliance, network support, memory – and persistence-based logging, key-value database, and multiple language apis.
Compared with other database types, Redis has the following characteristics: C/S communication model, single-process single-thread model, rich data types, atomic operation, persistence, high concurrency read and write, support lua script.
Due to the excellent performance of Redis, major manufacturers around the world are embracing and using it, such as Github, Twitter, Alibaba, Baidu, and some major manufacturers continue to develop and customize the open source version on the basis of in-depth. Of course, Redis is so widely used that it has become a must-have skill for us developers, and it can’t be ignored in daily work and interviews.
Redis data types and operation instructions
Redis commonly used data types include String, Hash, List, Set and SortedSet. Generally speaking, skilled use of these data types and operation instructions can meet the vast majority of our work scenarios. In addition, Redis also supports some advanced types such as HyperLogLog, Geo, Pub/Sub, etc.
This article is a brief summary of the five commonly used data types and directives. For directives that are not listed, go to redis. IO /commands#.
String
It is a binary-safe string, meaning that it can store not only strings, but also images, videos, and other types, with a maximum length of 512 MEgabytes. The following lists common String directives and their usage examples.
| instruction | use |
|---|---|
| SET key value | Sets the value of the specified key |
| GET key | Gets the value of the specified key. |
| GETSET key value | Sets the value of the given key to value and returns the old value of the key. |
| MGET key1 [key2..] | Gets the values of all (one or more) of the given keys. |
| SETEX key seconds value | Associate the value value with the key and set the expiration time of the key to seconds. |
| SETNX key value | Set the key value only if the key does not exist. |
| MSET key value [key value…] | Set one or more key-value pairs at the same time. |
| INCR key | Increment the value of the number stored in the key by one. |
| INCRBY key increment | Add the value stored in the key to the given increment. |
| DECR key | Subtract the number stored in the key by one. |
| DECRBY key decrement | Values stored by key minus the given decrement value. |
| DEL key [key1..] | Deletes the given key and returns the number of deleted keys |
The String type is a compact key-value structure and one of the most commonly used types in development, similar to the Map type in Java. In addition, Redis provides increment and decrement instructions for numeric types, which can be used to achieve counters, transmitters and other functions.
Hash
This type is a map consisting of fields and associated values, where both fields and values are strings. The Hash type is similar to that of table records in a relational database. Key is the primary key, field is the column name, and value is the corresponding column value. Using Hash, you can describe multiple features of an object. The Hash operation commands are as follows:
| instruction | use |
|---|---|
| HDEL key field1 [field2] | Deletes one or more hash table fields |
| HEXISTS key field | Check whether the specified field in the hash table key exists. |
| HGET key field | Gets the value of the specified field stored in the hash table. |
| HGETALL key | Gets all the fields and values of the specified key in the hash table |
| HINCRBY key field increment | Add increment to the integer value of the specified field in the hash table key. |
| HINCRBYFLOAT key field increment | Increments the floating point value of the specified field in the hash table key. |
| HKEYS key | Gets all fields in the hash table |
| HLEN key | Gets the number of fields in the hash table |
| HMGET key field1 [field2] | Gets the values of all given fields |
| HMSET key field1 value1 [field2 value2 ] | Set multiple field-value pairs into the hash key at the same time. |
| HSET key field value | Set the value of field in hash table key to value. |
| HSETNX key field value | Set the value of the hash table field only if the field field does not exist. |
| HVALS key | Gets all values in the hash table. |
| HSCAN key cursor [MATCH pattern] [COUNT count] | Iterates key-value pairs in a hash table. |
The Hash type is closer to object-oriented concepts. Objects are identified by keys, and objects are described in multiple dimensions as “field-value” key-value pairs. Each Hash can store 2^32-1 key-value pairs, which is about 4 billion.
List
This type is an insertion sequentially sorted collection of string elements, based on a double-linked list implementation. A list can contain up to 2^32-1 elements. The operation command for List is as follows:
| instruction | use |
|---|---|
| BLPOP key1 [key2 ] timeout | Removes and retrieves the first element of the list. If there are no elements in the list, the list is blocked until the wait times out or an eject element is found. |
| RPUSH key value1 [value2] | Adds one or more values to the list |
| RPUSHX key value | Add a value to an existing list |
| BRPOP key1 [key2 ] timeout | Removes and retrieves the last element of the list. If there are no elements in the list, the list is blocked until the wait times out or an eject element is found. |
| LINDEX key index | Gets the elements in the list by index |
| LLEN key | Get the list length |
| LPOP key | Removes and gets the first element of the list |
| LINSERT key BEFORE | AFTER pivot value |
| LPUSH key value1 [value2] | Inserts one or more values into the list header |
| LPUSHX key value | Inserts a value into the head of an existing list |
| LRANGE key start stop | Gets the elements in the specified range of the list |
| LREM key count value | Remove list elements |
| LSET key index value | Set the value of a list element by index |
| LTRIM key start stop | To trim a list, that is, to keep only elements within a specified range, and to remove all elements that are not within a specified range. |
| RPOP key | Removes the last element in the list, returning the removed element. |
| RPOPLPUSH source destination | Removes the last element of the list and adds it to another list and returns |
| BRPOPLPUSH source destination timeout | Pops a value from a list, inserts the pop-up element into another list and returns it; If the list has no elements, it blocks until the wait times out or a popup element is found. |
List is implemented using a bidirectional linked List, similar to a queue structure, with the head of the queue on the left and the tail on the right. So, RPUSH is adding elements at the end of the queue, while LPOP is pulling elements at the end. Based on this feature, we can use a List as an asynchronous queue. As shown below:
Set
A Set is an unordered collection. The main difference between a Set and a List is that the elements in a Set are unordered and unique. The lower level of the Set type is implemented through a hash table with the following commands:
| instruction | use |
|---|---|
| SADD key member1 [member2] | Adds one or more members to a collection |
| SCARD key | Gets the number of members of the collection |
| SDIFF key1 [key2] | Returns the difference between the first collection and the others. |
| SDIFFSTORE destination key1 [key2] | Returns the difference set for a given collection and stores it in destination |
| SINTER key1 [key2] | Returns the intersection of all sets given |
| SINTERSTORE destination key1 [key2] | Returns the intersection of all the given collections and stores them in destination |
| SISMEMBER key member | Check whether the member element is a member of the collection key |
| SMEMBERS key | Returns all members of the collection |
| SMOVE source destination member | Move the member element from the source collection to the Destination collection |
| SPOP key | Removes and returns a random element from the collection |
| SRANDMEMBER key [count] | Returns one or more random numbers in a collection |
| SREM key member1 [member2] | Removes one or more members of a collection |
| SUNION key1 [key2] | Returns the union of all given sets |
| SUNIONSTORE destination key1 [key2] | The union of all given collections is stored in the Destination collection |
| SSCAN key cursor [MATCH pattern] [COUNT count] | Iterate over the elements in the collection |
The Set type is mainly used in some scenarios, such as social scenarios. Through the operation of intersection, union and difference sets, the Set type can be used to find social relations such as common friends, common concerns and common preferences.
SortedSet
SortedSet is an ordered set type. Each element is associated with a double fractional weight that sorts the members of the set from smallest to largest. Like the Set type, its underlying implementation is through a hash table. Sorted Set common commands and their descriptions are shown in the following table:
| instruction | use |
|---|---|
| ZADD key score1 member1 [score2 member2] | Adds one or more members to an ordered collection, or updates the scores of existing members |
| ZCARD key | Gets the number of members of an ordered collection |
| ZCOUNT key min max | Computes the number of members in an ordered set with a specified interval fraction |
| ZINCRBY key increment member | Increment the score of a specified member in an ordered set |
| ZRANGE key start stop [WITHSCORES] | Returns an ordered collection of members within a specified interval by indexing the interval |
| ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT] | Returns an ordered set of members within a specified interval by a fraction |
| ZSCORE key member | Returns the score value of a member in an ordered set |
SortedSet is characterized by unique and ordered internal elements, which can be maintained by the operation of the data through the above instructions. We can use it to achieve the functions of leaderboards, delay queues and so on.
Pub/Sub
Redis publish subscription (PUB/SUB) is a message communication pattern: the sender (PUB) sends messages and the subscriber (sub) receives messages, enabling a 1:N message queue. As shown below:
| instruction | role |
|---|---|
| PUBLISH channel message | Sends the message to the specified channel. |
| SUBSCRIBE channel [channel …] | Subscribe to a given channel or channels of information. |
| UNSUBSCRIBE [channel [channel …]] | To unsubscribe from a given channel. |
Why is Redis so fast?
This is a common interview question and we need to know how big Redis is before answering it. Redis official articleHow fast is Redis?”, which shows that the performance of Redis is affected by network bandwidth, CPU, physical machine/virtual machine, data size, number of client connections and other factors. The figure below is from this article. The data shows how Redis QPS is affected by the number of connections in the high-end configuration (the horizontal axis is the number of connections, and the vertical axis is QPS). When the number of connections is less than 5000, QPS can reach 100,000 +.
- Redis is a pure memory database, generally simple access operation, thread occupation time is short, time is mainly spent on IO.
- Excellent data structure: In order to pursue high performance, the underlying data structure of Redis is specially designed with simple structure and operation time complexity of O(1).
- Single thread model: single thread is used in request command processing, which ensures atomicity of each operation and avoids unnecessary context switching and competition between threads.
- I/O request processing is efficient using non-blocking multiplexing I/O technology. (More on multiplexing later)
Why is Redis single threaded?
Let’s take a look at the official answer from Redis:
It’s not very frequent that CPU becomes your bottleneck with Redis, as usually Redis is either memory or network bound. For instance, using pipelining Redis running on an average Linux system can deliver even 1 million requests per second, so if your application mainly uses O(N) or O(log(N)) commands, it is hardly going to use too much CPU.
It is not common for a CPU to become a bottleneck in Redis, which is usually memory or network limited. For example, on a normal Linux system using Pipeline, Redis can handle a million requests per second, and if your application mainly uses O(N) or O(log(N)) commands, it hardly takes up much CPU.
To sum up: single thread is fast enough, why multithread! Hahaha…
In fact, Redis is a single thread is not serious, the so-called single thread is Redis to handle network requests using a single thread. Due to the use of non-blocking, multiplexed I/O technology, the use of a single thread is sufficient for almost pure memory with no latency operations. However, this is not to say that there is only one thread in Redis, it has other threads to do the auxiliary work, such as: data expiration policy, memory flushing mechanism, master slave replication, etc.
One of Redis’s strengths is that it has a single-threaded mechanism that allows it to be fast. But now that our servers tend to have 8, 16, or 32 cores at every turn, Redis can’t make full use of hardware resources with just one thread. To provide hardware resource utilization, we can deploy multiple instances of Redis on a single machine or use virtualization /Docker technology to reallocate physical resources.
Redis data expiration policy and memory flushing mechanism
Redis has excellent performance, in large part due to its use of memory for data storage and the lean design of its internal data structures by Redis developers. The amount of available memory in Redis is not only limited by physical memory space, but also by system parameters. Maxmemory represents the maximum number of Redis bytes:
# In short... if you have slaves attached it is suggested that you set a lower # limit for maxmemory so that there is some free RAM on the system for slave # output buffers (but this is not needed if the policy is 'noeviction'). # # maxmemory <bytes> maxmemory 268435456Copy the codeThe available memory resources of Redis are limited. In order to make full use of the memory resources and provide high performance service, Redis has developed a set of perfect data expiration processing strategy and memory elimination mechanism. The target of the data expiration policy is the data whose expiration time is set and has expired. Memory flushing mechanism refers to the guarantee mechanism adopted to ensure availability when Redis runs out of memory.
For expired data, Redis does not simply use the periodic deletion mode, but adopts the “periodic deletion + lazy deletion strategy”. Here’s why:
- Scheduled deletion: A timer is used to monitor the key. When the key expires, it is automatically deleted. Although memory is released in time, it consumes CPU resources. In the case of large concurrent requests, the CPU spends time processing the request, not deleting the key, so this strategy is not used.
- Periodically clean + lazy delete: By default, Redis checks every 100ms for expired keys and deletes expired keys. It should be noted that Redis does not check all keys every 100ms, but randomly selects and checks them (if all keys are checked every 100ms, Redis will not be stuck). Therefore, if you only use the periodic deletion policy, many keys will not be deleted. Here, lazy delete comes in handy. When we get a key, Redis checks to see if the key has expired and deletes it if it has expired.
If you think about it, the “clean regularly + lazy delete strategy” is not foolproof. If the key is not deleted on a regular basis, and then the key is not requested in a timely manner, then the lazy deletion will not take effect either. This will cause Redis memory to get higher and higher, where the memory flushing mechanism comes into play. Redis provides the following six memory flushing mechanisms:
- Volatile – lRU: Selects the least recently used expires data from a set with an expiration date (server.db[I].expires)
- Volatile – TTL: Selects expired data from a set (server.db[I].expires) to be discarded
- Volatile -random: Selects any data from a set with an expiration date (server.db[I].expires) to be discarded
- Allkeys-lru: Culls the least recently used data from the dataset (server.db[I].dict)
- Allkeys-random: Random selection of data from a dataset (server.db[I].dict)
- No-enviction: forbids the expulsion of data. New write operations will report an error
From the description above, the policy for volatile-* applies to data sets that are set to expire, even though it is useless if all data is written without expiration. The all-* policy applies to all data sets. Deleting unexpired data may affect production security. No-enviction will cause Redis to be unable to provide new write services, which is very unfriendly.
Therefore, we need to carefully evaluate whether to set expiration times for data and choose an appropriate memory flushing strategy during development. In our company, we have strict regulations on the use of Redis: all data must be set to expire, Redis cannot be used as a persistent database, and all data in Redis must be rebuildable.
We can set the appropriate memory flushing mechanism for Redis through the configuration file:
Maxmemory-policy volatile- LRU is the least recently used for expired SettingsCopy the codeconclusion
The content of this article is more theoretical basic knowledge, there are also some high-frequency questions in the interview, in fact, the foundation of Redis is far more than these, through the following article slowly supplement!
If you find it useful, please share it with a friend who needs it! Focus on not getting lost, wechat search public number “code road mark”!