Redis Key naming strategy

Redis is a cache database in k-V form. Each Object to be cached needs a unique Key to identify it. However, in our daily development, it is common for a company or department to share a Redis cluster. Therefore, this can cause Key conflicts, causing data overwriting problems (even in the same department, there may be different developers using the same Key).

Name it according to the business name – Not recommended

Similar businesses are common in a project, so if different developers use the same Key, it can cause bugs in the system. This problem can also occur if different project teams are working together on a Redis cluster. Such as:

The set name qinyi 127.0.0.1:6379

OK

# Different businesses in the same project use the same key (co-developed by different developers)

Lpush Name Qinyi Imooc

(error) WRONGTYPE Operation against a key holding the wrong kind of value

127.0.0.1:6379

Precede the business name with the project name – not recommended

In this way, Key conflicts caused by different projects can be avoided. However, different services in the same project cannot be avoided. Such as:

# a and B are prefixed with the name of the project as Key when using Redis

127.0.0.1:6379 set a – name qinyi

OK

127.0.0.1:6379 set b -name imooc

OK

It is still not possible to avoid different businesses in the same project using keys of the same name

Lpush A-name Qinyi Imooc

(error) WRONGTYPE Operation against a key holding the wrong kind of value

127.0.0.1:6379

Key = Project name – Service name – Timestamp – Suggestion

This way of naming the Key is better than the previous way, adding a timestamp identifier. The implication here is that the probability of defining the same timestamp with different developers is very, very low. The precision of the timestamp can be selected according to the actual service situation, for example, day or hour. Such as:

# Key add day level timestamp to avoid Key conflicts

127.0.0.1:6379 set a – qinyi name – 20190314

OK

127.0.0.1:6379 lpush A-name -20190315 Qinyi Imooc

(integer) 2

127.0.0.1:6379

Setting the expiration time

When working with Redis, it is necessary to set an expiration date for the Key in most cases. Here’s why:

As a cache, speed up the execution of the program. However, cached objects are likely to change frequently and do not reside in Redis for long periods

When Redis memory usage exceeds a threshold (which can be customized), Redis’ cache flushing policy is triggered and the LRU process is very slow. The performance of Redis is greatly affected

Therefore, when saving KV, it is best to set the expiration time. Such as:

The set name qinyi 127.0.0.1:6379

OK

127.0.0.1:6379 expire 30 name

(integer) 1

127.0.0.1:6379 TTL name

(integer) 25

127.0.0.1:6379

Something to know about Redis setting expiration times (including deletion of expiration keys here) :

How does Redis save the expiration key

Redis key expiration deletion policy

Data structure selection

There are five data structures commonly used by Redis: String, hash, list, set, and zset(sorted set). As you can see, using string in most scenarios can solve the problem. However, this is not necessarily the optimal choice. The following is a brief description of their applicable scenarios:

String: single cache result, not related to other KV

Hash: An Object contains many attributes that need to be stored separately. Note: Do not use strings in this case, because strings take up more memory

List: An Object contains a lot of data that can be repeated and ordered

Set: An Object contains a lot of data. It does not require the data to be ordered, but it does not allow repetition

Zset: An Object contains a lot of data, and the data itself contains a weight value, which can be used to sort

In addition, Redis provides some special data structures for specific scenarios. For example, Geo and HyperLogLog

Geo: For LBS related applications, please refer to Redis Geo

HyperLogLog: indicates the cardinality statistics that allow errors. For details, see Redis HyperLogLog

Choice of persistence mechanism

Redis provides two persistence mechanisms: RDB and AOF.

RDB: Writes a snapshot of data in memory to a disk at a specified interval. The actual operation is to fork a sub-process to write data to a temporary file. After the data is written to a temporary file, the original file is replaced and stored in binary compression

AOF: Logs record all write and delete operations processed by the server. Query operations are not recorded. You can open the file to view detailed operation records

The advantages of these two methods of persistence are not explained; conclusions can be drawn from their implementations. Below, analyze the disadvantages of each of them.

The disadvantage of RDB

RDB is not a good choice if you want to ensure high availability of data, i.e. avoid data loss as much as possible. If the system breaks down before scheduled persistence, data that has not been written to disk will be lost

Because the RDB is forked to assist in data persistence, a large data set can cause the entire server to go out of service for hundreds of milliseconds, or even a second

The disadvantage of AOF

AOF files are usually larger than RDB files for the same amount of data. RDB can recover big data faster than AOF

Due to different synchronization strategies, AOF tends to run slower than RDB

Summary: The criteria for choosing between these two options is whether the system is willing to sacrifice some performance in exchange for higher cache consistency (AOF), or whether the system is willing to sacrifice some performance in exchange for higher cache consistency (AOF) by not enabling backup during frequent write operations.

Do not use Redis to store single Key big data

The big data mentioned here does not mean that Redis is not used to store more data, but that a single KV is very large. The core reason for this is that Redis is a single-process, single-thread application. This working mode avoids the time consuming of thread switching in the operation process, and does not need to consider thread safety. It does not need to acquire locks, release locks and other operations, which is very simple and direct. But:

Redis single thread working mechanism, can not use multi-core parallel processing. If you encounter a large amount of I/O operations, the situation of waiting for I/O will increase. Even if Redis uses IO multiplexing technology at the bottom, the EVENT handle of I/O will occupy resources

Redis persistence requires data to be saved, and the large amount of data in KV will also affect the implementation of persistence

​