Redis was released in 2009 and is over 10 years old today. As an essential skill, there are endless topics to talk about. Any point in this article can be expanded to complete a medium-sized article.

When you’re communicating or interviewing, you need to use the most accurate language to describe the situation, so this is for you.

Redis ability:

  • 1 0W/s QPS (Redis-Benchmark)
  • 1W + Long link (netstat/ss)
  • The most complex Zset 6kW data takes 5ms on average to write 1K /s and read 5k/s
  • Persistence (RDB)

#1. Basic overview

To learn a new language, it is important to master its basic data structures and the apis for these data structures. Redis’s data structures are like a language.

#Redis data structure

There are 5 commonly used and 10 in total. During the interview, you can usually answer 5 kinds of questions, but the other 5 kinds are extra points.

  • Stringstring
  • HashThe dictionary
  • ListThe list of
  • SetA collection of
  • ZSetOrdered set. Performance Reference:”How Good is Redis’s Zset?” opens New Window.
  • PubsubPublish and subscribe (not recommended, many pits)
  • BitmapThe bitmap
  • GEOLocation (limited use, people nearby)
  • StreamFlow (5.0) (very similar to Kafka)
  • HyperloglogBase statistics

The agreement of Redis

Redis is a text protocol

  • RESPEnd with CRLF (\r\n)
  • RESP3(Enabled by Redis6 to increase client cache)

Redis underlying data structure

Small and large data volumes are often different, focusing on the main structure of large data volumes.

  • String-sds
  • Hash-(ziplist , dict)
  • Set-(intset,dict)
  • List-(ziplist,quicklist)
  • ZSet– (ziplist + skiptable jump table)
  • Stream-(Radix tree radix number)

In Java, you can refer to implementations like ConcurrentSkipListMap.

In Java, an ordered Set is called a TreeSet, but it is implemented with a red-black tree.

Redis persistence

In the production environment, only RDB mode is used.

  • RDB
  • AOF(Similar to Binglog Row mode)
  • Mixed mode: RDB+AOF

O (n) instructions

  • keys *
  • hgetall
  • smembers
  • sunion
  • .

If the set size is uncertain, use SCAN HSCAN SSCAN ZSCAN instead. In addition, dangerous commands like keys are best masked by the RENAME command.

Performance optimization

  • unlinkDelete key -> asynchrony to avoid blocking
  • pipelineBatch transfer, reduce network RTT -> Reduce frequent network interaction
  • Multi-valued instruction (MSET, HMSET) -> Reduces frequent network interactions
  • Turn off theaof– > avoid io_wait

scaling

  • lua
  • redis-module

The Module pattern is less well known and is relatively low-level development.

2. Troubleshooting

  • The monitor instructionThe command output is displayed. You can use grep to filter
  • keyspace-eventsSubscribe to events for certain keys. For example, an event to delete a piece of data, the underlying implementation is based on PubSub
  • slow logAs the name implies, full query, very useful
  • --bigkeysStart the Redis large Key health check. This is done in scan mode without worrying about blocking
  • memory usage key,memory statsinstruction
  • infoInstruction, concerninstantaneous_ops_per_sec,used_memory_human,connected_clients
  • redis-rdb-toolsRDB offline analysis

3. Elimination strategy

If you are applying for the position of REDis DBA, you will not be able to answer this question.

  1. Passive delete (delete and return NIL only if you get it is lazy delete)
  2. Active delete (run once for 100ms, random delete lasts for 25ms, similar to Cron)
  3. -> Memory usage exceeds maxMemory, triggering an active cleanup policy

For the third case, there are eight strategies. Note that Redis already has LFU.

  1. The defaultvolatile-lruLook up least recently used data from the set expired data set
  2. volatile-ttlDelete keys with short remaining time from a set of expired data
  3. volatile-randomSelect any data from the set to expire for obsolescence
  4. volatile-lfuRemove recently infrequently used data from an expired data set
  5. allkeys-lru
  6. allkeys-lfu
  7. allkeys-randomThe data that is used least frequently will be eliminated first
  8. no-enviction

If maxMemory is not set,Redis will use memory until the operating system’s Oom-killer is triggered.

4. Cluster mode

  1. stand-alone
  2. Single machine multi-instance
  3. Primary/secondary (1+n)
  4. Master/Slave (1+ N) & Sentry (3 or cardinal number)
  5. Redis Cluster (recommended, but with restrictions)

Redis Cluster is recommended for Internet, outsourcing and project at will.

On a large scale

  • twemproxy
  • codis
  • Developed based on Netty Redis protocol
  • Management platform: CacheCloud

5. Redis FAQ

Redis usage scenarios

  • Cache (cache consistency cache penetration cache breakdown cache avalanche)
  • Distributed Lock (Redlock)
  • Distributed current limiting
  • Session

API example:

  • Zset leaderboard, sort
  • Bitmap user check-in and online status
  • Geo people nearby
  • Stream A message flow similar to Kafka
  • Number of HYPERloglog ACCESS IP addresses per day

Cache consistency

Why is there a consistency problem?

  • Write. Cache and database are two different components, and as long as double-writes are involved, there is a possibility that only one write will succeed, resulting in data inconsistency.
  • The update. Updates are similar in that two different components need to be updated.
  • Read. Read Ensure that the information read from the cache is up to date and consistent with that in the database.
  • Delete it. When deleting database records, how do I delete data from the cache?

Recommended use: Cache Aside Pattern

Read requests:

  • Read cache first and then DB

Change operation:

  • Work on the database first, then eliminate the cache

When complex transactions and rollback operations are involved, you can place elimination in finally.

Problem: Cache flush failed! (Probability is very low, timed compensation)

Cache breakdown

Impact, slight.

A large number of requests under high traffic read a failed Key -> Redis Miss -> penetrate DB

Solution: Use distributed locks where only the first thread to acquire the lock requests the database and then inserts the cache

The cache to penetrate

Impact, average.

Access a non-existent Key (malicious attack) -> Redis Miss -> penetrate to DB

Solution:

  1. Set a Null value for the corresponding Key and place it in the cache
  2. BloomFilter predetermines

Cache avalanche

Impact: Serious.

A large number of keys fail at the same time | 2.Redis down -> Redis Miss -> pressure calls DB

Solution:

  1. Add a relative random number to the expiration time
  2. Ensure high availability of Redis

A distributed lock

Redis distributed locks are not that simple. Redlock by Redisson is recommended. The most basic instruction is setnx.

setnx-> SET key value [EX seconds|PX milliseconds|KEEPTTL] [NX|XX] [GET]
Copy the code

Key points of distributed lock:

  • atomic
  • The lock timeout
  • A deadlock
  • Read-write lock
  • failover

The simplest Redis distributed lock code (not rigorous).

Java side code emulates lock and unlock.

public String lock(String key, int timeOutSecond) { for (; ;) { String stamp = String.valueOf(System.nanoTime()); boolean exist = redisTemplate.opsForValue().setIfAbsent(key, stamp, timeOutSecond, TimeUnit.SECONDS); if (exist) { return stamp; } } } public void unlock(String key, String stamp) { redisTemplate.execute(script, Arrays.asList(key), stamp); }Copy the code

The lua script unlock.

local stamp = ARGV[1]
local key = KEYS[1]
local current = redis.call("GET",key)
if stamp == current then
    redis.call("DEL",key)
    return "OK"
end
Copy the code

6. Use Redis

Common Java clients

  • Lettuce SpringBoot default, Netty based event driven model
  • Jedis’s legacy client, which uses Commons -pool for thread pool development
  • Redisson very rich distributed data structures, including locks, distributed maps, etc. Extensive use of Lua scripts ️

Use standard

There is no one-size-fits-all specification. More:

  • Use connection pooling and do not frequently create closed client connections
  • The message size is less than 10kb and can be compressed using snappy and msgpack
  • Avoid large and hot keys
  • Do not use the O(n) instruction
  • Do not use the Zrange command without range
  • Not using database (easy to overwrite data)
  • No advanced data structures (use the basic 5)
  • Transactions are not used
  • Disable the monitor for a long time

springboot cache redis

  • Pay more attention to standardization when using
  • The cache layer is too abstract, so if you need to manipulate the underlying data structure, use redisTemplate directly

Redis is multithreading?

It depends on which stage. The data manipulation phase is always single threaded, even with Redis6.

End

Good luck! If you can help, please feel free to praise.

You can follow my B station account →→→→B station account

Study communication groups →→→→ →Communication group