list

  • Data storage requirements: Stores multiple data and distinguishes the sequence of data entering the storage space
  • Required storage structure: One storage space stores multiple data, and the sequence of data can be reflected
  • List type: save multiple data, the underlying use of == bidirectional linked list storage structure == implementation

  • Sequential tables are fast to query data, but the order cannot change from beginning to end. Inserting data from the middle is slow, and adding and deleting operations are slow
  • Linked lists are fast to insert and delete and retain the original physical order. When inserting or deleting an element, you only need to change the pointer pointer.
  • A bidirectional list has two Pointers, one to the previous node and one to the next.

Basic operations on list data

  • Add/modify data

Lpush key value1 [value2]……

Rpush key value1 [value2]……

  • To get the data

lrange key start stop

lindex key index llen key

  • Get and remove data

lpop key

rpop key

127.0.0.1:6379> lpush list1 one 
(integer) 1
127.0.0.1:6379> lpush list1 two
(integer) 2
127.0.0.1:6379> lpush list1 three
(integer) 3
127.0.0.1:6379> lrange list1 0 2
1) "three"
2) "two"
3) "one"

127.0.0.1:6379> rpush list2 one two three
(integer) 3
127.0.0.1:6379> lrange list2 0 2
1) "one"
2) "two"
3) "three"127.0.0.1:6379> lrange list2 0-1# Same as python's list
1) "one"
2) "two"
3) "three"127.0.0.1:6379 > lpush list2 0Insert one or more values into the head of the list
(integer) 4
127.0.0.1:6379> rpush list2 4		Insert one or more values to the end of the list
(integer) 5
127.0.0.1:6379> lrange list2 0 -1
1) "0"
2) "one"
3) "two"
4) "three"
5) "4"127.0.0.1:6379 > lindex list2 0Get a value from a list by subscript
"one"127.0.0.1:6379 > lindex list2 2"three"127.0.0.1:6379 > llen list1Return the length of the list
(integer) 3

127.0.0.1:6379> lpop list2		Remove the first element of the list
"0"127.0.0.1:6379 > rpop list2Remove the last element of the list
"4"
127.0.0.1:6379> lrange list2 0 -1
1) "one"
2) "two"
3) "three"

Copy the code

List data extension operation

  • Obtain and remove data within a specified time

blpop key1 [key2] timeout

brpop key1 [key2] timeout brpoplpush source destination timeout

127.0.0.1:6379> lpush list5 a b
(integer) 2
127.0.0.1:6379> lpop list5
"b"127.0.0.1:6379 > lpop list5"a"
127.0.0.1:6379> lpop list5
(nil)
127.0.0.1:6379> blpop list5 10
(nil)
(10.10s)

127.0.0.1:6379> lpush list5 d
(integer) 1
127.0.0.1:6379> blpop list5 10
1) "list5"
2) "d"
(5.73s)

127.0.0.1:6379> rpush list5 e
(integer) 1
127.0.0.1:6379> lrange list5 0 -1
1) "e"
127.0.0.1:6379> brpop list5 5
1) "list5"
2) "e"
127.0.0.1:6379> lrange list5 0 -1
(empty array)

Copy the code
  • Shear data

ltrim key start stop

127.0.0.1:6379> rpush mylist a b c d
(integer) 4 127.0.0.1:6379> lrange myList 0-1 1)"a"
2) "b"
3) "c"
4) "d"127.0.0.1:6379> ltrim myList 12The list has been changed by subscripting the specified lengthOK 127.0.0.1:6379> lrange myList 0-1Only the truncated elements are left
1) "b"
2) "c"
Copy the code
  • Remove the last element of the list and add it to the new list

rpoplpush key key2

127.0.0.1:6379> rpush mylist1 a b c d
(integer) 4
127.0.0.1:6379> lrange mylist1 0 -1
1) "a"
2) "b"
3) "c"
4) "d"127.0.0.1:6379 > rpoplpush mylist1 mylist2# # Remove the last element of the list and move it to the new list
"d"
127.0.0.1:6379> lrange mylist1 0 -1
1) "a"
2) "b"
3) "c"127.0.0.1:6379> lrange myList2 0-1The value does exist in the target list
1) "d"

Copy the code
  • The update operation replaces the index value in the list with another value

Lset key subscript New value

127.0.0.1:6379 > exists the listCheck whether the list exists
(integer) 0 127.0.0.1:6379> lset list 0 MAOIf the list does not exist, an error will be reported when we try to update it
(error) ERR no such key
127.0.0.1:6379> lpush list mao
(integer) 1 127.0.0.1:6379> lrange list 0-1 1)"mao"
127.0.0.1:6379> lset list 0 zhu		Update the value of the current subscript if it existsOK 127.0.0.1:6379> lrange list 0-1 1)"zhu"
127.0.0.1:6379> lset list 1 gou		If there is no subscript, we will get an error if we try to update it
(error) ERR index out of range

Copy the code
  • Inserts a specific value before or after an element in the list

insert key before/after value new-value

127.0.0.1:6379 > rpush mylist'hello'
(integer) 1
127.0.0.1:6379> rpush mylist 'world'
(integer) 2
127.0.0.1:6379> linsert mylist before 'world' 'niu'
(integer) 3 127.0.0.1:6379> lrange myList 0-1 1)"hello"
2) "niu"
3) "world"127.0.0.1:6379 > linsert mylist after'world' 'zhu'
(integer) 4 127.0.0.1:6379> lrange myList 0-1 1)"hello"
2) "niu"
3) "world"
4) "zhu"
Copy the code

Application Scenario When you click “like” in wechat moments, the information about the friend you like is displayed in the order of “like”. If you cancel the “like”, the information about the friend will be removed

The solution

  • Remove specified data

lrem key count value

  • Redis is used for data control with sequence of operations
127.0.0.1:6379> rpush friend a b c d e		A, B, C, D, e
(integer) 5 127.0.0.1:6379> lrange friend 0-1 1)"a"
2) "b"
3) "c"
4) "d"
5) "e"
127.0.0.1:6379> lrem friend 1 d		# remove d
(integer) 1 127.0.0.1:6379> lrange friend 0-1 1)"a"
2) "b"
3) "c"
4) "e"
127.0.0.1:6379> rpush friend d		
(integer) 5 127.0.0.1:6379> lrange friend 0-1 1)"a"
2) "b"
3) "c"
4) "e"
5) "d"

127.0.0.1:6379> lpush letter a b a c d a o a u e a
(integer) 11 127.0.0.1:6379> Lrange letter 0-1 1)"a"
 2) "e"
 3) "u"
 4) "a"
 5) "o"
 6) "a"
 7) "d"
 8) "c"
 9) "a"
10) "b"
11) "a"
127.0.0.1:6379> lrem letter 3 a		# delete the first three as
(integer) 3
127.0.0.1:6379> lrange letter 0 -1
1) "e"
2) "u"
3) "o"
4) "d"
5) "c"
6) "a"
7) "b"
8) "a"
Copy the code

Precautions for data operations of the list type

  • The data stored in the list is of type string. The total capacity of the data is limited, up to 2^32- 1 elements (4294967295).
  • A list has the concept of an index, but data is usually queued and queued, or pushed and unloaded on a stack
  • Get all data End Index set to -1
  • Lists can be paginated, with the first page usually coming from the list and the second page and more loaded from the database

List application scenarios

  • In Twitter and Sina Weibo, the following list of individual users should be displayed in the order of their following, and the list of fans should list the recently followed fans in the front
  • How do news and information sites present the latest news or information in chronological order?
  • During enterprise operation, the system generates a large amount of operation data. How to ensure that operation logs of multiple servers are output in a uniform sequence?

The solution

  • List-dependent data has sequential characteristics for information management
  • The queue model is used to solve the problem of multi-channel information summary merging
  • Use the stack model to solve the problem of latest messages
127.0.0.1:6379> rpush logs A1.. (integer1 127.0.0.1:6379> Rpush logs A1... (integer127.0.0.1:6379> rpush logs B1.. (integer) 2
127.0.0.1:6379> rpush logs b1...
(integer127.0.0.1:6379> rpush logs C1.. (integer) 4
127.0.0.1:6379> rpush logs c1...
(integer127.0.0.1:6379> lrange logs 0-1 1)"a1.."
2) "b1.."
3) "a1..."
4) "c1.."
5) "b1..."
6) "c1..."

Copy the code

set

  • New storage requirements: Store large amounts of data to provide greater efficiency in queries
  • Required storage structure: can save a large amount of data, efficient internal storage mechanism, easy to query
  • Set: Identical to the hash storage structure, stores only keys, not nil, and values are not allowed to be duplicated

Basic operations on data of type set

  • Add data

sadd key member1 [member2]

  • Get all data

smembers key

  • Delete the data

srem key member1 [member2]

127.0.0.1:6379 > sadd myset xiaotianAdd elements to the set
(integer1 127.0.0.1:6379> sadd myset lovexiaotian (integer) 1
127.0.0.1:6379> sadd myset maomao
(integer) 1
127.0.0.1:6379> smembers myset		Set to view all values of the specified set
1) "maomao"
2) "xiaotian"
3) "lovexiaotian"
127.0.0.1:6379> srem myset maomao
(integer) 1
127.0.0.1:6379> smembers myset
1) "xiaotian"
2) "lovexiaotian"
Copy the code
  • Get aggregate data

scard key

  • Determines whether the collection contains the specified data

sismember key member

127.0.0.1:6379 > scard myset (integer) 2
127.0.0.1:6379> sismember myset xiaotian	Check if a value is in the set
(integer) 1 127.0.1:6379 > sismember myset zhuzhu (integer) 0
Copy the code

Business scenario Each user will set three hobbies when using Weibo for the first time. However, in order to increase the user’s activity and interest points, users must gradually become interested in other information categories and increase customer retention. How to achieve this?

Business analysis

  • The system analyzes the latest or hottest information items of each classification and organizes them into set sets
  • Select some of the information at random
  • Organize the whole information set to be displayed with the hot information in the user’s concern information classification

The solution

  • Gets a specified amount of data randomly from the collection

srandmember key [count]

  • Randomly retrieves some data from the collection and moves it out of the collection

spop key [count]

127.0.0.1:6379 > sadd news youxiAdd elements to the set
(integer) 1
127.0.0.1:6379> sadd news tiyu
(integer) 1
127.0.0.1:6379> sadd news yule
(integer) 1
127.0.0.1:6379> sadd news kexue
(integer) 1
127.0.0.1:6379> srandmember news 1		Select an element at random
1) "yule"
127.0.0.1:6379> srandmember news 1
1) "youxi"
127.0.0.1:6379> srandmember news 1
1) "kexue"127.0.0.1:6379 > scard news,integer) 4
127.0.0.1:6379> srandmember news 3		# # Select a specified number of elements at random
1) "kexue"
2) "tiyu"
3) "yule"
127.0.0.1:6379> srandmember news 3
1) "youxi"
2) "kexue"
3) "yule"127.0.0.1:6379 > spop news# Randomly remove some elements from the set set
"kexue"127.0.0.1:6379 > smembers news 1)"youxi"
2) "tiyu"
3) "yule"127.0.0.1:6379 > spop news 2# a random number of elements in a set
1) "tiyu"
2) "youxi"127.0.0.1:6379 > smembers news 1)"yule"

Copy the code
  • Redis is applied to random recommendation information retrieval, such as hot song list recommendation, hot news recommendation, hot selling travel routes, APP recommendation, up main recommendation, etc

The business scenario

  • In order to promote the communication between users and ensure the improvement of the single rate of business, Mai Mai needs to let each user have a large number of friends. In fact, new employees do not have more friends in the workplace. How to quickly accumulate more friends for users?
  • In order to increase user popularity and improve user retention, Sina Weibo users need to follow more people so as to obtain more information or hot topics. How to improve the total amount of users following others?
  • QQ new users are entering the network at a younger and younger age. These users’ circle of friends is very small and often concentrated in a school or even a class. How to help users quickly accumulate friends and bring more activity to users?
  • Wechat public account is one of the channels of wechat information circulation. Increasing the public account concerned by users has become a way to improve user activity. How to help users accumulate more public accounts concerned?
  • In order to increase meituan takeout to a single volume, it must help users to explore their food needs. How to recommend the most suitable food to users?

The solution

  • Find the intersection, union and difference sets of two sets

    sinter key1 [key2]

sunion key1 [key2] sdiff key1 [key2]

  • Find the intersection, union, and difference sets of two sets and store them in the specified set

    sinterstore destination key1 [key2]

sunionstore destination key1 [key2] sdiffstore destination key1 [key2]

  • Moves the specified data from the original collection to the target collection

    smove source destination member

127.0.0.1:6379 > sadd user1 MAO# add user 1's friends
(integer1 127.0.0.1:6379> sadd user1 zhu (integer1 127.0.0.1:6379> sadd user1 niu (integer) 1
127.0.0.1:6379> sadd user2 zhu		# add user 2's friends
(integer) 1
127.0.0.1:6379> sadd user2 gou
(integer) 1
127.0.0.1:6379> sinter user1 user2		# Mutual friends meet
1) "zhu"127.0.0.1:6379 > sunion user1, user2# and set
1) "mao"
2) "gou"
3) "niu"
4) "zhu"127.0.0.1:6379 > sdiff user1, user2# difference set
1) "mao"	
2) "niu"127.0.0.1:6379 > sdiff user2 user1# difference set
1) "gou"
127.0.0.1:6379> sinterstore user3 user1 user2	Store the intersection of user 1 and user 2 to user 3
(integer) 1
127.0.0.1:6379> smembers user3
1) "zhu"
127.0.0.1:6379> sunionstore user4 user1 user2	Store the union of users 1 and 2 to user 4
(integer) 4
127.0.0.1:6379> smembers user4
1) "mao"
2) "gou"
3) "niu"
4) "zhu"
127.0.0.1:6379> sdiffstore user5 user1 user2	Store the difference between user 1 and user 2 to user 5
(integer) 2
127.0.0.1:6379> smembers user5
1) "mao"
2) "niu"
127.0.0.1:6379> smove user2 user1 gou	Move the element from user2 to user1
(integer) 1
127.0.0.1:6379> smembers user1
1) "mao"
2) "gou"
3) "niu"
4) "zhu"127.0.0.1:6379 > smembers user2 1)"zhu"

Copy the code
  • Redis is applied to associative search, second-degree associative search and deep associative search of similar information
  • Show common concern (once)
  • Show mutual friends (once)
  • Starting from user A, obtain the friend information list of user B (once)
  • Starting from user A, get the shopping list of friend user B (second degree)
  • Starting from user A, get the game recharge list of friend user B (second degree)

Precautions for data operations of the set type

  • The set type does not allow duplicate data. If the added data already exists in the set, only one copy is retained
  • Set has the same storage structure as hash, but it does not enable space for storing values in the hash

Application scenario of set type

Service Scenario 1

There are 12,000 employees in the group, and there are more than 700 roles, more than 3,000 business operations and more than 23,000 kinds of data in the internal OA system. Each employee has one or more roles. How to verify the permissions of business operations quickly?

The solution

  • It relies on the non-repeating feature of set data, and relies on the hash feature of set to store structural features for data filtering and fast query
  • Obtain all user roles based on the user ID
  • Obtain a user based on all roles == All operation rights == Add the user to the set
  • Obtain users based on all roles of users == Select all data == Add all data to set Total
127.0.0.1:6379> sadd RID :001 getall getByIdAdd role permissions
(integer2 127.0.0.1:6379> sadd RID :002 getCount getall insert (integer) 3 127.0.0.1:6379> sunionStore uid:007 RID :002# merge employee privileges with uid 007
(integer) 4
127.0.0.1:6379> smembers uid:007
1) "insert"
2) "getById"
3) "getCount"
4) "getall"
127.0.0.1:6379> sismember uid:007 insert
(integer1)Copy the code
  • Redis is used for merging non-duplicate data of the same type

Service Scenario 2

  • The company promoted its new website, counted the PV (traffic),UV (independent visitors) and IP (independent IP) of the website.
  • PV: the number of visits to the website can be improved by refreshing the page
  • UV: the number of times the website is accessed by different users. The number of visits can be counted by cookie. The SAME user can switch IP address, and UV remains unchanged
  • IP: indicates the total number of times a website is accessed by different IP addresses. The number of visits can be counted by IP address. The IP address remains unchanged when different users access a website from the same IP address

The solution

  • Using the data of set set to remove the feature, record all kinds of access data
  • Create string data and use INCR to count daily visits (PV)
  • Set model is established to record different number of cookies (UV)
  • Establish set model to record different IP numbers (IP)
127.0.0.1:6379 > sadd ips 192.168.188.10# an independent IP access
(integer) 1
127.0.0.1:6379> sadd ips 192.168.188.20		# second independent IP access
(integer) 1
127.0.0.1:6379> sadd ips 192.168.188.10		The same IP access is not added
(integer) 0
127.0.0.1:6379> scard ips
(integer2)Copy the code
  • Redis is used for fast deduplication of the same type of data

Service Scenario 3

  • The blacklist
    • Information websites pursue high traffic, but because of the value of their information, they are often easy to be used by lawbreakers.

Quick access to information, individual special industry website information through crawler to obtain analysis, can be converted into trade secrets for sale. For example, third-party train ticket, air ticket, hotel ticket purchasing software, e-commerce brush comments, brush praise. + At the same time, the pseudo traffic brought by crawlers will also give operators an illusion and produce wrong decisions, so as to effectively avoid the repeated crawling of websites by crawlers as a basic problem to be considered by every website. After discriminating crawler users based on technology, such users need to be effectively shielded, which is a typical application of == blacklist ==. + PS: Not to say that crawlers do destructive work, some small sites need crawlers to drive some traffic to them.

  • White list
    • For applications with higher security, blacklists cannot solve security problems. In this case, you need to configure accessible user groups and rely on whitelists for more stringent access authentication

The solution

  • Set up problem user discovery and identification rules based on business strategy
  • The blacklist of users that meet the rule is periodically updated and added to the set
  • After the user’s behavior information reaches the blacklist, it is compared with the blacklist to confirm the behavior direction
  • Blacklist filter IP addresses: Information sources applied to open visitors’ access rights
  • Blacklist Filter device information: Applies to the information sources that restrict access to the device
  • Blacklist filtering users: Applies to information sources based on access permissions

sorted_set

  • New storage requirements: Data sorting facilitates the efficient presentation of data and provides a way to sort data according to its characteristics
  • Required storage structure: A new storage model that can hold sortable data
  • Sorted_set type: Adds sortable fields to the storage structure of a set

Sorted_set Basic operation for data of type

  • Add data

    zadd key score1 member1 [score2 member2]

  • Get all data

    zrange key start stop [WITHSCORES]

zrevrange key start stop [WITHSCORES]

  • Delete the data

    zrem key member [member …]

127.0.0.1:6379> zadd scores 100 maomao 60 xiaotian 94 zhuzhu 47 pancheng	# Add four students and grades
(integer) 4 127.0.0.1:6379> zrange scores 0-1Get all data from small to large
1) "pancheng"
2) "xiaotian"
3) "zhuzhu"
4) "maomao"127.0.0.1:6379> zrange scores 0-1 withscores# withscores retrieves the name and score
1) "pancheng"
2) "47"
3) "xiaotian"
4) "60"
5) "zhuzhu"
6) "94"
7) "maomao"
8) "100"127.0.0.1:6379> zrevrange scores 0-1 withscoresReverse sort from large to small
1) "maomao"
2) "100"
3) "zhuzhu"
4) "94"
5) "xiaotian"
6) "60"
7) "pancheng"
8) "47"127.0.0.1:6379 > zrem scores pancheng# delete data
(integer) 1 127.0.0.1:6379> zrevrange scores 0-1 withscores 1)"maomao"
2) "100"
3) "zhuzhu"
4) "94"
5) "xiaotian"
6) "60"
Copy the code
  • Get data by conditions

    zrangebyscore key min max [WITHSCORES] [LIMIT]

zrevrangebyscore key max min [WITHSCORES]

  • Conditional deletion of data

    zremrangebyrank key start stop

zremrangebyscore key min max

127.0.0.1:6379> zrangebyscore scores 50 100 withscores	# From small to big
1) "xiaotian"
2) "60"
3) "pancheng"
4) "88"
5) "zhuzhu"
6) "94"
7) "maomao"
8) "100"

127.0.0.1:6379> zrevrangebyscore scores 100 50 withscores		# From big to small
1) "maomao"
2) "100"
3) "zhuzhu"
4) "94"
5) "pancheng"
6) "88"
7) "xiaotian"
8) "60"127.0.0.1:6379> zrange scores 0-1 withscores 1)"xiaotian"
2) "60"
3) "pancheng"
4) "88"
5) "zhuzhu"
6) "94"
7) "maomao"
8) "100"
127.0.0.1:6379> zremrangebyrank scores 0 1		Delete by subscript
(integer) 2 127.0.0.1:6379> zrange scores 0-1 withscores 1)"zhuzhu"
2) "94"
3) "maomao"
4) "100"127.0.0.1:6379> zrange scores 0-1 withscores 1)"xiaotian"
2) "66"
3) "pancheng"
4) "88"
5) "zhuzhu"
6) "94"
7) "maomao"
8) "100"
127.0.0.1:6379> zremrangebyscore scores 70 90	Delete according to conditions
(integer) 1 127.0.0.1:6379> zrange scores 0-1 withscores 1)"xiaotian"
2) "66"
3) "zhuzhu"
4) "94"
5) "maomao"
6) "100"
Copy the code

== note: ==

  • Min and Max are used to define the criteria for a search query
  • Start and stop are used to limit the scope of the query, apply to the index, and start and end the index
  • Offset and count are used to define the query scope and the start position and total amount of data
Copy the code
  • Get aggregate data

    zcard key

zcount key min max

  • Set intersection and union operation

    zinterstore destination numkeys key [key …]

zunionstore destination numkeys key [key …]

127.0.0.1:6379> zadd s1 50 aa 60 bb 70 cc
(integer) 3
127.0.0.1:6379> zadd s2 60 aa 40 bb 90 dd
(integer) 3
127.0.0.1:6379> zadd s3 70 aa 20 bb 100 dd
(integer) 3
127.0.0.1:6379> zinterstore ss 3 s1 s2 s3
(integer) 2 127.0.0.1:6379> zrange ss 0-1 withscores 1)"bb"
2) "120"
3) "aa"
4) "180"
127.0.0.1:6379> zinterstore sss 3 s1 s2 s3 aggregate max		# maximize
(integer) 2 127.0.0.1:6379> zrange SSS 0-1 withscores 1)"bb"
2) "60"
3) "aa"
4) "70"

Copy the code

Extended operation for data of type sorTED_set

The business scenario

  • TOP10 websites of various resources (movies, songs, documents, e-commerce, games, etc.)
  • Game activity statistics
  • Game friend intimacy

Business analysis

  • Establish a ranking basis for all participating resources

The solution

  • Get the index of the data (ranking)

    zrank key member

zrevrank key member

  • Score value acquisition and modification

    zscore key member

zincrby key increment member

127.0.0.1:6379> Zadd Music 800 suis 750 Ikura 700 ACA# Streams of three singers
(integer) 3
127.0.0.1:6379> zrank music suis	# From small to large
(integer) 2
127.0.0.1:6379> zrank music aca
(integer) 0
127.0.0.1:6379> zrank music ikura
(integer) 1
127.0.0.1:6379> zrevrank music suis		# View the rankings from large to small
(integer) 0 127.0.0.1:6379> zscore music ACAGet the number of plays
"700"127.0.0.1:6379> zincrby music 10 ACA# increase
"710"

Copy the code
  • Redis is applied to the ranking corresponding to the counter combination sorting function

Precautions for sorTED_set data operations

  • Score stores 64 bits of data. If it is an integer, the value ranges from -9007199254740992 to 9007199254740992
  • Score can also hold data that is a == double value ==, based on the characteristics of a double floating-point number, precision may be lost when used

Hou carefully

  • Sorted_set storage is still based on the ==set structure ==, so the data == cannot repeat ==, if the same data is added repeatedly, the score value will be reversed

Compound == overwrite ==, preserving the result of the last modification

127.0.0.1:6379> zadd test1 11 aa
(integer) 1 127.0.0.1:6379> zrange test1 0-1 withscores 1)"aa"
2) "11"
127.0.0.1:6379> zadd test1 22 aa
(integer) 0		# return failure but the value still changes127.0.0.1:6379> zrange test1 0-1 withscores 1)"aa"
2) "22"127.0.0.1:6379> zadd test1integer) 0 127.0.0.1:6379> zrange test1 0-1 withscores 1)"aa"
2) "33"

Copy the code

Application scenario of sorTED_set

Application Scenario 1

  • Basic service + value-added service website will set up the trial of each member, so that users can fully experience the advantages of membership. For example, watching movies and trying VIP games

VIP experience, cloud disk download experience VIP, data view experience VIP. After the VIP experience expires, manage such information effectively. Even for official VIP users, there are corresponding management methods.

  • The website will be open for voting and discussion on a regular basis. How to effectively manage such expiration information.

The solution

  • For the task processing limited by time line, the processing time is recorded as score value, and the sorting function is used to distinguish the processing sequence

  • Record a time to process, process the corresponding task when it expires, remove the record in Redis, and record a time to process

  • When a new task is added, determine and update the current time of the next task to be processed

  • To improve sorted_set performance, tasks are usually stored into several sorted_sets based on features. For example, within an hour, within a day, within a week, within a month, within a season, within a year, etc., the operation is promoted step by step and several tasks to be operated are included in the queue to be processed within an hour

  • Obtain the current system time

    time

127.0.0.1:6379> zadd ts 1509802345 UID :001 1509802390 UID :002 151038284 UID :003 (integer) 3 127.0.0.1:6379> zrange ts 0-1 withscores 1)"uid:003"
2) "151038284"
3) "uid:001"
4) "1509802345"
5) "uid:002"
6) "1509802390"
127.0.0.1:6379> zrem ts uid:003
(integer) 1 127.0.0.1:6379> zrange ts 0-1 withscores 1)"uid:001"
2) "1509802345"
3) "uid:002"
4) "1509802390"127.0.0.1:6379> zadd ts 1510382841 UID :003 (integer) 1 127.0.0.1:6379> zrange ts 0-1 withscores 1)"uid:001"
2) "1509802345"
3) "uid:002"
4) "1509802390"
5) "uid:003"
6) "1510382841"127.0.0.1:6379 > time# timestamp
1) "1618647056"
2) "262066"127.0.0.1:6379 > time 1)"1618647057"
2) "779953"127.0.0.1:6379 > time 1)"1618647059"
2) "242339"
Copy the code
  • Redis is used to manage the execution sequence of scheduled tasks or task expiration

Application Scenario 2

  • Task/message weight setting application

When tasks or messages are to be processed and a task queue or message queue is formed, priority should be guaranteed for high-priority tasks and how to achieve task weight management.

The solution

  • For the task with weight, the task with high weight should be processed first, and score recording weight can be used to set the weight of multi-conditional task
  • If there are too many weight conditions, the sorting score value needs to be processed to ensure that score value can be compatible with 2 or more conditions, such as foreign trade orders have priority over domestic orders, president orders have priority over employees’ orders, and manager orders have priority over employees’ orders
  • Due to the limited length of score, truncation of data is required, especially if the time is set to hour or minute level (after conversion).
  • First set the order category, then set the order initiating role category, the overall score length must be uniform, not enough bits to complement 0. The first collation must not start with 0
    • For example, foreign trade 101, Domestic 102, manager 004, employee 008.
    • Score of foreign trade order under staff is 101008 (priority)
    • The domestic single score under the manager was 102004
127.0.0.1:6379> zadd order 102004 order:id:1
(integer) 1 127.0.0.1:6379> zadd ORDER 101008 ORDER :id:2 (integer) 1 127.0.0.1:6379> Zrevrange order 0-1 withscores# Order priority
1) "order:id:1"
2) "102004"
3) "order:id:2"
4) "101008"127.0.0.1:6379> zrange order 0-1 1)"order:id:2"
2) "order:id:1"
Copy the code
  • Redis is used for instant task/message queue execution management

Data type practice cases

Practice Case 1

The business scenarioSemantic recognition and automatic dialogue in the field of artificial intelligence will be an important technology in the future service robot answering call system. Baidu has developed user evaluation semantic recognition service, which is open to enterprises for free trial, and trained Baidu’s own model. The maximum number of calls that a user can make per minute is 10

The solution

  • Design counters that record the number of calls to control the number of business executions. The user ID is used as the key and the number of times used as the value
  • Gets the number before calling to determine if the limit number has been exceeded
    • Up to a maximum of times, each call counts +1
    • Business call failed, count -1
  • Set the life cycle of the counter to the specified period, such as 1 second/minute, automatically empty the number of times used within the period

Limit when you reach 10

127.0.0.1:6379> get uid:00415
(nil)
127.0.0.1:6379> setex uid:00415 60 1
OK
127.0.0.1:6379> get uid:00415
"1"00415 (127.0.0.1:6379 > incr uid:integer) 2
127.0.0.1:6379> incr uid:00415
(integer) 3
127.0.0.1:6379> get uid:00415
"3"
127.0.0.1:6379> incrby uid:00415 7
(integer) 10
127.0.0.1:6379> get uid:00415
(nil)
Copy the code

Solution improvement

  • Cancel the determination of the maximum value and use the incR operation over the maximum value to throw an exception instead of judging whether the maximum value is greater than each time
  • If it’s nil,
    • If yes, set it to Max- times
    • If not, count +1
    • Business call failed, count -1
  • If the + operation exceeds the upper limit, the usage reaches the upper limit

127.0.0.1:6379> get uid:00415
(nil)
127.0.0.1:6379> setex uid:00415 60 9223372036854775797
OK
127.0.0.1:6379> get uid:00415	Set a maximum value of -10
"9223372036854775797"00415 (127.0.0.1:6379 > incr uid:integer127.0.0.1:6379> incrby uid:00415 9 (integer) 9223372036854775807 127.0.0.1:6379> incr UID :00415# increase the value again and tell the user that the value is out of range
(error) ERR increment or decrement would overflow
Copy the code

Practice Case 2

In the process of using wechat, when wechat receives a message, the most recently received message will be placed at the top by default. When multiple friends and subscribed numbers send messages at the same time, the order will be continuously changed. You can also set important sessions to the top. Once the user goes offline and opens wechat again, what order should messages be displayed?

The solution

  • The data dependent on the list is sequential in nature for message management, using the list structure as a stack
  • Create separate lists for top and normal sessions to manage separately
  • When a user message is received in a list, add the sender’s ID from one side of the list (set here to the left)
  • If multiple messages with the same ID are repeatedly pushed to the stack, the message id corresponding to the current ID is deleted regardless of whether there is a message corresponding to the current ID
  • When the message is pushed, the top session list is pushed first, and then the general session list is pushed. The list completed by the push will clear all data
  • The number of messages, namely the number of conversations between wechat users, is recorded separately with the idea of a counter and updated synchronously with the list operation
127.0.0.1:6379> lrem 100 1 200
(integer) 0
127.0.0.1:6379> lpush 100 200
(integer) 1
127.0.0.1:6379> lrem 100 1 300
(integer) 0
127.0.0.1:6379> lpush 100 300
(integer) 2
127.0.0.1:6379> lrem 100 1 400
(integer0 127.0.0.1:6379> lpush 100 400 (integer) 3
127.0.0.1:6379> lrem 100 1 200
(integer) 1
127.0.0.1:6379> lpush 100 200
(integer) 3
127.0.0.1:6379> lrem 100 1 300
(integer) 1
127.0.0.1:6379> lpush 100 300
(integer) 3 127.0.0.1:6379> lrange 100 0-1The order in which messages are received last
1) "300"
2) "200"
3) "400"
Copy the code
  • Redis is used for time-sequential data manipulation, regardless of time