Overview of Redis

Redis is a key-value memory caching middleware that most programmers have used in their projects. We can also use memory to implement the cache, but with Redis, the cache function can be unified into a component, easy to reuse expansion.

At the bottom, Redis uses IO multiplexing techniques like SELECT, epoll, etc. It can guarantee the throughput better. Redis also uses a single thread to process requests, avoiding the additional cost of thread switching and lock contention.

In addition, Redis also optimized some data structures, so the performance of Redis is very good, the official test report is that a single machine can support about 10W /s QPS.

Redis communication protocol

Redis is based on THE TCP long connection C/S architecture, using text serialization protocol, and like HTTP, is also a request a response, the client receives the response and then continue to request.

Of course, it is also possible to send multiple requests and then respond to all the results at once, which is called pipeline technology.

Redis text serialization protocol is relatively simple, through some standard format to parse text, roughly as follows:

  • \r\n indicates the end of parsing
  • It is a simple character string starting with +
  • Errors, starting with “-“
  • The value is an integer starting with a colon
  • Large string type starting with “$”
  • Array type, starting with *

For example, the client sends the command to the server:

SET key value

Will be resolved to:

*3\r\n$3\r\nSET\r\n$3\r\nkey\r\n$5\r\nvalue\r\n

The command above can be seen as:

*< number of bytes > CR LF $< number of bytes > CR LF < number of bytes > CR LF... $< number of bytes of parameter N > CR LF < data of parameter N > CR LFCopy the code

The response from the server can be of many types, usually determined by the first byte of the response data:

The first byte of the status reply is "+" the first byte of the error reply is "-" the first byte of the integer reply is ":" The first byte of the bulk reply is "$". The first byte of multi Bulk Reply is "*".Copy the code

For example, the status return of the response is as follows:

+OK

Redis data structure

Redis supports five data types to help developers make better use of the cache. The bottom layer is made up of six data structures.

Five data types

String: The string type is the most basic data type in Redis. For example, after a set name “hello” operation, a string is returned when a get name operation is performed, and the counterpoint operation is supported. Typically, one key can store 512MB of value.

Hash: The hash type is mainly used to store objects. Generally, if we have a whole object to store, which contains multiple fields, we can use hash to store the object, because Redis provides extraction and setting of these fields, reducing the developer’s secondary processing, such as serialization and deserialization operations.

List: A simple list of strings that allows us to push, pop, and support a range of list elements from both ends. You can view it as a two-way list.

Set: A set is a combination of values that are not repeated. It provides operations such as intersection, union, difference, etc. Set operations can be used to find common friends.

Sorted set: an ordered set that provides sorting capabilities based on the above set. Sorted by a score attribute.

Six underlying data structures

The above data types are actually implemented by corresponding data structures at the bottom of Redis, which are carefully designed by Redis and can greatly improve processing efficiency.

Simple dynamic strings: Redis is written in C, where string types are more primitive, such as the \0 character terminator. Therefore, Redis implements its own string type, such as string length, pre-allocated memory, dynamic expansion and other features, which also ensures processing security.

Linked list: A double-ended linked list with prev, next Pointers to the front and back nodes, len attributes, and multiple types of values.

Dictionary: The key-value mapping operation is realized through the hash algorithm, and the hash conflict is solved by the chain address method. The general time complexity can reach O(1).

Skip list: A multi-level ordered linked list, where each level is an ordered extraction of the next level, reducing the number of searches, somewhat like an ordered binary tree search.

Set of integers: An ordered set of integers with no repeating elements.

Ziplist (ziplist) : a piece of contiguous memory that has been specially encoded to effectively save memory.

Quick lists: Organize Ziplist as a bidirectional linked list. Due to the internal continuity of Ziplist, it can reduce the memory fragmentation problem of linked lists and improve memory utilization.

Redis’ elimination strategy

The elimination strategy of Redis mainly consists of LRU elimination, TTL elimination and random elimination.

  • LRU obsolescence: the least recently used obsolescence
  • TTL weeding out: The earlier the expiration date, the earlier the weeding out.
  • Random elimination: Elimination using a random algorithm.

Since Redis may or may not set an expiration date on a key, the elimination strategy needs to be further refined:

  • Volatile – LRU: LRU elimination is implemented for keys with an expiration date. Volatile – LRU elimination is not implemented for keys with an expiration date.
  • Volatile – TTL: TTL is used only for keys whose expiration time is set.
  • Volatile -random: Random elimination is performed only for keys whose expiration time is set.
  • Allkeys-lru: lru elimination for allkeys
  • Allkeys-random: performs a random elimination strategy for allkeys
  • No-enviction: the elimination policy is not executed. If a write operation occurs, an error message is displayed. Read requests can continue.

In the Redis configuration file redis.conf, we can set the elimination policy:

Maxmemory 300MB maxmemory-policy volatile- lRUCopy the code

Redis usage scenarios

Redis can be used in many different ways, but the most common is data caching. But because it provides multiple data types, we can also develop other scenarios, such as:

  • Sorted set: As mentioned earlier, sorted sets are sorted for each write and do not contain duplicate values, so we can take the unique identity of the user, such as userId as key, score as score, and then proceedZADDOperation to get the leaderboard.
  • Check-in: Check-in is usually only two states, signed and unsigned. It’s just like 0 and 1, so redissetbit,getbitThis kind of counterpoint is suitable for check-in scenarios.
  • Counting: Redis is a single-threaded operation, and counting functions such as likes and followers can be handed over to Redis to avoid concurrency contention issues. Of course, you also have to consider persistence.

About Distributed Locking

Sometimes we may use Redis as an auxiliary use of distributed locks, responding to redis operations to determine whether the lock is currently available.

However, such a solution would have a single-node bottleneck, and if Redis went down, the lock would become unusable.

Some of you might say that Redis also has its own high availability solution. But in fact, redis high availability scheme is not suitable for distributed lock applications, there will be the risk of multiple nodes acquiring the lock at the same time.

If you really need a more rigorous distributed lock, you still need to use distributed coordination solutions such as ZooKeeper or ETCD to ensure strong consistency.

Redis uses attention points

Cache avalanche and penetration

Redis provides high performance for our programs by caching redundant data. But be aware that once the cache fails, a flood of requests will overwhelm the system, a cache avalanche.

In addition to cache avalanche, there is also the possibility of cache penetration. For example, each time a different data is accessed, the request will still fall behind.

To prevent a cache avalanche, we can control the request by joining the message queue and slowly digesting it. Alternatively, you can directly enable the traffic limiting function to control the traffic within a reasonable range.

For cache penetration, we can create a black and white list of malicious requests and reject them outright. If it is a normal request, you can temporarily cache the filtered results, even if the value is NULL.

Data concurrency problem

Since Redis exists as a component, our application communication can actually be considered distributed, which means there are cache and back-end data consistency issues.

A common practice is to remove the cache key when new data arrives and wait for the next query to refill the cache.

The reason why Redis is not allowed to update data is to prevent multiple update actions from happening at the same time. Due to network reasons, the later update may reach Redis one step earlier than the previous update, which will be different from the original process. Therefore, only delete action is taken, and nothing else is done.

However, even if the deletion of the key scheme has a certain probability of the same as the above situation, really serious, generally set the timing expiration time, so that the data at most during this period of time is inconsistent.

conclusion

The use of Redis is very simple, but in fact involves a lot of knowledge, especially need to pay attention to its concurrent data consistency problem. The more we know about Redis, the better we can master it. We can understand more details by ourselves. I hope this article can help you, thank you!

Interested friends can search the public account “Read new technology”, pay attention to more pushed articles.

Thank you for your support!

Read new technology, read more new knowledge.