preface

Redis is a database of key-value pairs. The five commonly used data types are string, hash, list, set, and zset.

Redis is used as a cache for two main purposes: high performance and high concurrency, because memory naturally supports high concurrency

Application scenarios

Distributed lock (String)

Setnx key Value, if the key does not exist, sets the value of the key to value and returns 1. If the given key already exists, setnx does nothing and returns 0.

When setnx returns 1, the lock is acquired. After the operation, del key means the lock is released. If setnx returns 0, the lock has failed to be acquired

Counter (string)

For example, the number of times each question was browsed

setKey 0incr key // incr readcount::{post ID} Get key // get readcount::{post id} Obtains the number of readingsCopy the code

Distributed Globally Unique ID (string)

There are many ways to implement distributed global unique ID, and only redis is introduced here

Each time a userId is obtained, increment the userId by 1

Obtain the maximum number of userids directly, cache them locally and slowly accumulate them. When the maximum number of userids is nearly reached, obtain another number. If a user service is down, at most a small number of userids are not used

setUserId 0incr usrId // Returns 1incrby userId 1000 // returns 10001Copy the code

Message Queue (List)

I’m just going to go in and out of the list

If there are any elements in the list, the list will be ejected. No elements will be blocked until the wait times out or an ejected element is found. Rpush key valueblPOP key value 10Copy the code

List of Sina /Twitter users’ messages

Let’s say xiaobian Li follows two microblog a and B, and A sends a microblog (numbered 100), then run the following command

lpush msg::li 100Copy the code

B sent a microblog (numbered 200) and executed the following command:

lpush msg::li 200Copy the code

If you want to get the last 10 messages, you can execute the following command (the most recent message must be at the far left of the list) :

[start,stop] : lrange MSG ::li 0 9Copy the code

Lucky Draw (SET)

Sadd key {userId} # draw count winners and remove count winners from the draw, Srandmember Key count is not removed from the raffleCopy the code

Implement “like”, “check-in”, “like” and other functions (set)

Sadd like::8001 1001 srem like::8001 1001 srem like::8001 1001 sismember like::8001 1001 Like ::8001 # scard like::8001Copy the code

Implement focus model, people you may know (SET)

Someone seven cares about

sevenSub -> {qing, mic, james}

Aoyama cares about people

qingSub->{seven,jack,mic,james}

People Mic cares about

MicSub->{seven,james,qing,jack,tom}

Sinter sevenSub qingSub -> {mic, James}# the person I am following is also concerned about him. Otherwise return 0sismember micSub qingsismember jamesSub qing# someone I might know. Sdiffstore sevenMayKnow qingSub sevenSub -> {seven, Jack}Copy the code

E-commerce commodity Screening (SET)

Each item is stored with a static list of labels such as brand, size, processor, and memory

Brand :: Lenovosadd Brand :: Lenovosadd screenSize::15.6 Savior y700P - 001 mechanical revolution Z2AIRsadd processor: : i7 savior y700P - 001 mechanical revolution X8TIPlus # brand is lenovo, the screen size is 15.6, Sinter Brand :: Lenovo screenSize::15.6 Processor :: I7 -> Rescuer Y700P-001Copy the code

Ranking Edition (ZSET)

Redis zSet is built for leaderboards, friend lists, de-duplication, history and other business needs

# zrevrange ranking 0 2 withscores # zrevrange ranking 0 2 withscoresCopy the code

Expiry policies

Periodically delete

Redis puts each key with an expiration date into a separate dictionary that redis periodically iterates over to delete expired keys.

Periodic Deletion policy

Redis defaults to ten expiration scans per second (100ms once). Instead of traversing all keys in an expired dictionary, expiration scans employ a simple greedy strategy.

Random 20 keys from expired dictionary;

Delete expired keys from the 20 keys.

If more than a quarter of keys are out of date, repeat Step 1.

Lazy to delete

In addition to periodic traversal, it also uses an lazy policy to remove expired keys. The lazy policy means that when a client accesses the key, Redis checks the expiration date of the key and deletes it immediately without giving you anything back.

Regular deletion is centralized processing, lazy deletion is scattered processing.

Why do you choose the two strategies of periodic deletion and lazy deletion?

Delete it if it expires. If you put 100,000 keys in redis, all of which are set to expire, and you check 100,000 keys every few hundred milliseconds, the Redis will basically die, and the CPU will be overloaded with expired keys

However, the problem is that regular deletion may result in many expired keys not being deleted when the time comes. What can be done? So it’s lazy deletion. That is, when you get a key, Redis checks, is that key expired if it’s expired? If it’s out of date it’s deleted at this point and it doesn’t give you anything back.

It is not that the key is deleted when the time is up, but that Redis lazily checks it again when you query the key

By combining the above two methods, the expired key is guaranteed to be killed.

Therefore, using the above two strategies, it is not difficult to explain the following phenomenon: data clearly expired, but still occupy memory

Memory flushing strategy

This problem may be encountered by some friends, why is the data in Redis missing?

Because Redis puts data into memory, and memory is limited. For example, Redis only uses 10 gigabytes of data. What if you write 20 gigabytes of data into it? Of course it kills 10 gigabytes, and then it keeps 10 gigabytes. What data does that kill? What data is retained? Of course, it is to eliminate the infrequently used data, keep the commonly used data

Redis provides the following memory flushing strategies:

  1. Noeviction will not continue to service write requests (DEL requests can continue to service), read requests can continue to service. This ensures that data will not be lost, but the online business will not continue. This is the default elimination strategy.
  2. Volatile – LRU attempts to eliminate keys with expiration dates, and the least used keys are preferred. Keys that do not have an expiration date are not obsolete, which ensures that data that needs to be persisted is not suddenly lost. (This is the most used)
  3. Volatile – TTL is the same as above, except that the TTL is not the LRU, but the TTL of the remaining life of the key.
  4. Volatile -random is the same as above, except that the eliminated key is a random key from the expired key set.
  5. Allkeys-lru differs from volatile lRU in that the key object to be eliminated is the entire key set, not just the expired key set. This means that keys that are not set to expire will also be eliminated.
  6. Allkeys-random is the same as above, but the elimination strategy is random key. Allkeys-random is the same as above, but the elimination strategy is random key.

Persistence strategy

Redis data is stored in memory. If Redis goes down, all data will be lost, so persistence mechanisms must be provided.

Redis has two persistence mechanisms, the first is snapshot (RDB) and the second is AOF logging. A snapshot is a full backup, and AOF logs are continuous incremental backups. A snapshot is a binary serialized form of memory data and is very compact in storage, whereas an AOF log is an instruction log text of memory data modifications. AOF logs will become extremely large in the long-term running process. When the database restarts, AOF logs need to be loaded for command replay, which will take an extremely long time. So you need to do regular AOF rewrites to slim down the AOF log.

RDB is forked by the main Redis process to enable the sub-process to perform disk IO operations to carry out RDB persistence. AOF logs store the sequential instruction sequence of the Redis server, and AOF logs only record the instruction records for memory modification. That is, RDB records data and AOF records instructions

How to choose BETWEEN RDB and AOF?

  1. Don’t just use RDB, because that will cause you to lose a lot of data, because RDB is to back up data at intervals
  2. Don’t just use AOF, because that has two problems. First, cold standby through AOF is not as fast as RDB recovery. Second, RDB is more robust by simply generating snapshots each time, avoiding the bugs of complex backup and recovery mechanisms such as AOF
  3. Restoring memory state with RDB loses a lot of data, and replaying AOP logs is slow. Redis4.0 introduced hybrid persistence to address this problem. Store the contents of the RDB file with the incremental AOF log file. Here the AOF log is no longer the full log, but rather the incremental AOF log that occurs between the beginning of persistence and the end of persistence, which is usually small. Therefore, when Redis restarts, the contents of RDB can be loaded first, and then the incremental AOF log can completely replace the previous full AOF file replay, so the restart efficiency is greatly improved.

Cache avalanche and cache penetration

What is cache avalanche?

Suppose you have a system, the peak request for 5000 times per second, 4000 times gone cache, only 1000 fell on to the database, a database of 1000 per second concurrency is a normal index, can work normally, but if the cache is down, 5000 times a second request will all fall on the database, the database immediately died, Since the database can handle up to 2000 requests per second, if the DBA restarts the database, it will immediately be killed by new requests. This is a cache avalanche.

How to resolve cache avalanche

Ex ante: Redis high availability, master slave + Sentinel, Redis Cluster, avoid total crash

Issue: Local EhCache + Hystrix stream limiting & degrade to avoid MySQL being killed

After the fact: Redis persistence, fast recovery of cached data

What is cache penetration?

If the client sends 5000 requests per second, 4000 of them are malicious attacks by hackers, even if they are not found in the database. For example, if the user ID is positive and the hacker constructs a negative user ID,

If the hacker keeps sending those 4,000 requests every second, the cache won’t work and the database will be killed quickly.

How to resolve cache penetration

Data that cannot be queried is stored in the cache and value is null, for example, set-999.

In short, a cache avalanche is when the cache fails, all requests hit the database, and the database is instantly killed. Cache penetration is the query of a certain data does not exist, and the data cannot be found from the storage layer is not written to the cache, which causes the non-existent data to be queried every time the storage layer, losing the significance of cache