This is the second day of my participation in Gwen Challenge

preface

I believe that you have used Redis more or less in your daily work. Thanks to the rich data structure of Redis, it is more and more widely used in enterprise projects and plays an increasingly important role.

As Redis is used more and more in the enterprise, there is also a growing number of developers who are not experienced in complex/specific businesses and do not have a deep understanding of Redis commands, perhaps even as far as object caching

So the purpose of this article is to take you through the following two milestones:

  • To understandRedis common commands
  • How to do it in an enterprise projectjolt

Common commands

String

Common operations

  • set key value: Stores string key-value pairs
  • mset key value [value ...]: Batch stores string key-value pairs
  • setnx key value: Stores a nonexistent string key-value pair
  • get key: Gets a string key
  • mget key [key ...]: Gets string keys in batches
  • del key [key ...]: Deletes a key
  • expire key seconds: Set the expiration time of a key in seconds

Atoms to add and subtract

  • incr key: Adds the numeric value stored in the key to +1
  • decr key: Stores the numeric value -1 in the key
  • incrby key increment: Adds the value stored in the key to increment
  • decrby key decrement: Decrement the values stored in the key

Hash

Common operations

  • hset key field value: Stores the value of a hash table key
  • hsetnx key field value: Stores the key value of a nonexistent hash table key
  • hmset key field value [field value ...]: Stores multiple key-value pairs in a hash table key
  • hget key field: Retrieves the field key corresponding to the hash table key
  • hmget key field [field ...]: Obtains multiple field keys in the hash table key in batches
  • hdel key field [field ...]: Deletes the field key in the hash table key
  • hlen key: Returns the number of fields in the hash table key
  • hgetall key: Returns all the keys in the hash table key

Atoms to add and subtract

  • hincrby key field incrementIncrement increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment

List

  • lpush key value/[value ...]: Inserts one or more values value into the head of the key list(Far left)
  • rpush key value/[value ...]: Inserts one or more values value at the end of the key list(Far right)
  • lpop key: Removes and returns the head element of the key list
  • rpop key: Removes and returns the last element of the key list
  • lrange key start stop: returns the element in the list key within the range specified by the offsets start and stop
  • blpop key [key ...] timeout: if be in,Within a specified timeIf no element is popped, a nil and is returnedThe waiting timeIf timeout=0, block and wait. Instead, return a list of two elements, the first being the key of the popout element, and the second being the popout element from the head of the key list
  • brpop key [key ...] timeout: if be in,Within a specified timeIf no element is popped, a nil and is returnedThe waiting timeIf timeout=0, block and wait. Instead, return a list of two elements, the first being the key of the popout element, and the second being the popout element from the end of the key column

Set

Common operations

  • sadd key value: Stores an element to the set key, ignoring the element’s existence
  • srem key value: Removes elements from the set key
  • smembers key: Gets all elements in the collection key
  • srandmember key [count]: Selects count elements from the set key. Elements are not removed from the key
  • spop key [count]: Selects count elements from the set key and removes the elements from the key

Arithmetic operations

  • sinter key [key ...]: Intersection operation
  • sinterstore destination key [key ...]: Intersection operation, store the result ina new set destination
  • sunion key [key ...]: Union operation
  • sunionstore destination key [key ...]: union operation, store the result ina new collection destination
  • sdiff key [key ...]: Difference set operation
  • sdiffstore destination key [key ...]: difference set operation, store the result ina new collection destination

ZSet- Ordered set

Common operations

  • zadd key score member [[score member]...]: Adds elements with scores to an ordered set
  • zrem key member [member ...]: Removes elements from an ordered collection
  • zscore key member: Returns the score of member in the ordered set key
  • zincrby key increment memberAdd increment to member of the ordered set
  • zcard key: Returns the number of elements in the ordered set key
  • zrange key start stop [withscores]Gets the elements of the ordered set key from start to stop subscripts
  • zrevrange key start stop [withscores]: Gets the elements of the ordered set key from start to stop subscripts in reverse order

Set operations

  • zunionstore destkey numkeys key [key ...]: Union calculation
  • zinterstore destkey numkeys key [key ...]: Intersection calculation

Other commands

  • keys: Full traversal, used to list allSatisfies certain regular string rulesWhen the amount of redis data is large, the performance is poor, so avoid using the key

  • scan: Progressive traversal,scan cursor [MATCH pattern] [COUNT count]

The scan parameter provides three parameters:

  • If you want to walk over a full amount of data, you must start with a 0 cursor
  • The second: match: indicates a regular match
  • Third: count: the number of keys traversed at a timeReference value, the number of low-level traversals is not necessarily, the number of results may not match

Application scenarios

String

Single value cache

  • set key value: Sets the single-value cache
  • get key: Gets the single-value cache

Object caching

  • Set student value(json format)Set:A complete objectThe cache
  • MSET student:name aoteman student:age 18Set:Object partial dataCache, in fact, is a batch set key, a key is a field in an object
  • MGET student:name student:age: getObject partial dataCache is to obtain keys in batches

  • The first way:set student value, is relatively simple, but updating is relatively complex
  • The second way:MSET student:name aoteman student:age 18If some data of an object is frequently modified, you can use this command.For example, you only need to modify the activity price of the goods. There is no need to take out the whole item object, convert it into an object, update it, and then set it back, but this command willCreate lots of keys, is also a disadvantage

Simple distributed lock

  • setnx lock trueIf 0 is returned, the setting fails. If 1 is returned, the setting succeeds

  • del lock: Deletes/releases locks

  • set lock true ex 10 nx: Set the distributed locktimeoutTo prevent unexpected program termination from causing deadlocks

counter

  • incr key: counter +1
  • get key: Gets the counter value

Application scenario: set the key to article:{read/like/… }:count:{id};}:count:{id}

Distributed system global serial number

  • incrby key value:incrby student 1000Add 1000 to student where key = student

Application scenario: distributed system global serial number, each key value plus value, get the return value, in the local to allocate, in the local memory to the ID +1, if you use incR command, each generation to request redis, Redis pressure will be very large

The global sequence number generated in this way is discontinuous, and if there are multiple machines, other solutions are required to be continuous

Distributed session

Spring + Redis implements session sharing

Hash

Object caching

  • hmset user {userId}:name aoteman {userId}:age 18Set:objectThe cache
  • hmget user {userId}:name {userId}:age: Gets the object cache

One drawback of this application scenario is the amount of data. Assuming that there are tens of millions of records in the user table, a user key will be used to obtain the data, especially in the retrieval process, which will be very time-consuming. Moreover, Redis is a single-thread model, which will greatly affect the throughput

Shopping cart model in e-commerce

  • Add goods:Hset cart:{user id} {merchandise id} {merchandise number}

  • Shopping cart quantity increased:Hincrby cart:{user id} {item ID} {item number}

  • Query shopping cart quantity:Hlen cart:{user ID}

  • Delete an item from the shopping cart:Hdel Cart :{user id} {commodity ID}

  • Get all items in shopping cart:Hgetall Cart :{user ID}

Pros and cons of Hash

advantages

  1. For example, the key of a user can manage all users
  2. For an object of the same size, use Hash over StringThe operation consumes less memory
  3. For an object of the same size, use Hash over StringMore space saving

disadvantages

  1. The Hash expiration function cannot be used on the inner field, but only on the outer key. That is, once the key expires, all data under the key takes effect

  2. The Redis cluster architecture is not suitable for large-scale use. Suppose there are tens of millions of data under the user’s key, then according to hash routing, this key can only be allocated to one machine, that is, tens of millions of data are on one Redis, and all access is on this Redis. The data skewering scenario occurs. This case can be resolved with segmented storage

List

The data structure

In distributed/multi-machine environments, redis is a good solution for unified data structures

  • Stack:lpush + lpop= First in, last out (FILO)

  • Queue:lpush + rpop= First in, first out (FIFO)

Since Dijia has been removed and returned, the rightmost end is now AISi

  • Blocking MQ:lpush + brpop, similar to consumers in message queues

Push message flows such as weibo and wechat official account

Push message flow example

Weibo ABAB followed the news of A and B on weibo/public account in the same way:

  • A sends A microblog message with the message ID of 100:Lpush MSG :{微博ABAB ID} {article ID}
  • B sent a microblog message with an article ID of 101:Lpush MSG :{微博ABAB ID} {article ID}
  • ABAB log in to check the latest weibo news:Lrange MSG :{微博ABAB ID} 0 4: Get the latest 5 news/the latest news is on the far left

The message stream push method of Weibo is to push articles in the message list of all followers who follow it, and followers take messages in order

However, this approach has a fatal disadvantage: the number of microblog fans is too large. Assuming that a public account has tens of millions of fans, it is necessary to push tens of millions of lists every time an article is published, which is bound to cause performance bottlenecks. Therefore, another method is needed: pull method

A sends A microblog and puts the message into A message queue specially belonging to A. The fan account takes the initiative to pull this queue when logging in, and then stores it locally, as shown in the picture below:

Set

Various raffle scenes

  • sadd key {userId}: Click to enter the draw to join the collection
  • smembers key: View all users participating in the lottery
  • srandmember key [count]/spop key [count]: Draw count winners

The number of winners only needs to change the count size, such as: 1 first prize, 2 second prizes, 5 third prizes, spop key [count] command to delete the winners from the collection

Like on wechat, Weibo and Douyin(Same for collection and label)

  • Thumb up:Sadd like:{article ID} {user ID}
  • Unlike:Srem like:{article ID} {user ID}
  • Check if the user has clicked “like” :Sismember like:{article ID} {user ID}:Tiktok’s “like” heart is highlightedThis can be done with this command
  • Get a list of likes:Smembers like:{message ID}
  • Get the number of likes:Scard like:{message ID}

Set operations

  • sinter setA setB setC– > {C} :intersection
  • sunion setA setB setC– > {A, B, C, D, E} :And set
  • sdiff setA setB setC– > {A} :SetA set as the benchmark, and setB, setC set of the union to do the difference set

Set operation implementationSocial concern model

Examples of social attention models

  • Tailuo :Set -> {aisi, dijia}

  • Other Ultraman concerns: AISi :Set -> {Tailuo, Dijia, Leiou, Gaiya}

  • Dijia :Set -> {aisi, Saiwen, Zuofei}

Sinter Tailuo :Set AISi :Set -> {dijia}

Sdiff aisi:Set Tailuo :Set -> {leiou, gaiya}

Set operation implementationscreening

The characteristics of the product into a screening condition, a screening condition is a Set, finally take the intersection, can achieve screening

  • Sadd brand:huawei product ID
  • Sadd Brand: Xiaomi product ID
  • Sadd phoneType:picture commodity ID
  • Sadd ROM :512 commodity ID
  • .

sinter brand:huawei phoneType:picture rom:512 …. -> {commodity ID set}

Redis is obviously not the best way to do this kind of product search. This example is intended to show that the Set data structure has natural advantages for the operation Set scenario

ZSet

List:Top search rankings, music rankings, news rankings, and more can be implemented using ZSET

  • Click on news:Zincrby new:{date} 1 {message ID}
  • Top 10 on display day:Zrevrange new:{date} 0 9 withscores
  • 7 days search list calculation:Zunionstore new:{date start - end} 7 new:{date} new:{date +1} new:{date +2}...
  • Show the top 10 of the seven days:Zrevrange new:{date start - end} 0 9 withscores

SCAN

Redis paging

Suppose there are now so many keys, as shown below:

We set count to 3 and find no match, because count is the number of keys iterated at a time. That is, we select 3 keys from the total number of keys to iterate the match. If there is no match, we return empty array, which returns 2

The cursor is 0 for the first traversal, and the first integer in the result is returned as the cursor for the next traversal. All keys are scanned until cursor is 0

This is the Redis page, but it doesn’t match exactly the way mysql does

High energy warning

Although scan can be pagination, it is not perfect. If there are key changes (increase, delete, modify) during scan, the traversal effect may encounter the following problems, which need to be considered during development

  1. The newly added key may not be traversed
  2. I iterate over the repeated keys

Because the storage structure of Redis is stored in the form of K-V, similar to HashMap, the key is located to a subscript or bucket through the Hash function. In case of Hash conflict or collision, the subscript is stored in a way similar to a linked list. This subscript is the cursor of scan

  • If a key is added during scan and the bucket bits calculated by the hash key have already been scanned, the scan will not be performed again. In this case, the new key may not be traversed

  • If keys are added during scan, capacity expansion occurs, and the rehash mechanism is implemented after capacity expansion, the original subscripts will change, and duplicate keys will be iterated

conclusion

Through this article’s introduction to Redis common commands in different data structures, common enterprise-level scenarios, and their respective considerations, you should know the following blocks:

  • What different data structures should haveCommon commands
  • What are the common Redis commandsEnterprise-level application scenarios

Redis scene is definitely not only so, here I can think of only the writing in the above scenario, eventually need people to understand, see the essence through the phenomena, think in the field of responsibility, have what business can use Redis rich data structure optimization, welcome comments area under discussion, let me open mind, by the way

Phase to recommend

Starting from scratch to learn Redis series (a) | Redis environment set up

Finally, if you feel confused about the article, please leave a comment immediately. If you think I have something, please like 👍, follow ❤️ and share 👥, because this will be my motivation to output more high-quality articles, thank you!!

If you want to get books related to Redis, you can follow the wechat official account Java Encyclopedia and enter Redis

Finally, thank you for your support. See you next time!