Click “like” to see, form a habit, the public account search [dime technology] pay attention to more original technical articles. This article has been included in GitHub org_Hejianhui /JavaStudy.

Five common data structures

String structure

Common string operations

SET key value 	// Store string key-value pairs
MSET key value [key value ...] 	// Batch store string key-value pairs
SETNX key value 	// Store a nonexistent string key-value pair
GET key 	// Get a string key
MGET key [key ...]	// Get string keys in batches
DEL key [key ...] 	// Delete a key
EXPIRE key seconds // Set the expiration time of a key in seconds
Copy the code

Atoms to add and subtract

INCR key // Add one to the number stored in the key
DECR key // Subtract the number stored in the key by 1
INCRBY key increment // Add increment to the value stored in key
DECRBY key decrement // Subtract decrement from the values stored in key
Copy the code

String Application Scenario

  • Single value cache
  • Object caching
  • A distributed lock
  • counter
  • Web cluster Session sharing
  • Distributed system global serial number
  1. Single value cache
SET key value
Get key
Copy the code
  1. Object caching
SET user:1Value (DATA in JSON format) MSET user:1:name yijiaoqian user:1:balance 1888
MGET user:1:name user:1:balance 
Copy the code
  1. A distributed lock
SETNX product:10001  true // If 1 is returned, the lock is successfully obtained
SETNX product:10001  true // Failed to obtain the lock if 0 is returned. Performing service operations... DEL product:10001	// Release the lock

SET product:10001 true  ex  10  nx	// Prevent unexpected program terminations from causing deadlocks

Copy the code
  1. counter
INCR article:readcount:{article id} GET article:readcount:{article id}Copy the code
  1. Web cluster Session sharing

Spring Session + Redis implements Sessio sharing

  1. Distributed system global serial number
INCRBY orderId 1000	// Redis generates serial numbers in batches to improve performance
Copy the code

Hash structure

Common Hash Operations

HSET key field value 	// Store a hash table key
HSETNX key field value 	// Store the key of a nonexistent hash table key
HMSET key field  value [field value ...] // Store multiple key-value pairs in a hash table key
HGET key field 	// Get the field key corresponding to the hash table key
HMGET key field [field ...] // Select * from key
HDEL key field [field ...] 	// Delete the field key from the hash table
HLEN key	// Return the number of fields in the hash table key
HGETALL key	// Return all the keys in the hash table key

HINCRBY key field increment Increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment increment
Copy the code

Hash Application Scenarios

  • Object storage
HMSET user {userId}:name  yijiaoqian {userId}:balance  1888
HMSET user 1:name yijiaoqian 1:balance 1888
HMGET user 1:name 1:balance  
Copy the code
  • E-commerce shopping cart
  1. The key is the user ID
  2. The commodity ID is Field
  3. The quantity of goods is value

Shopping cart operation:

  1. Add item: hset Cart :1001 10088 1
  2. Add quantity: Hincrby Cart :1001 10088 1
  3. Total number of goods: Hlen CART :1001
  4. Delete item: hdel Cart :1001 10088
  5. Get all cart items: HGEtall Cart :1001

Advantages and disadvantages of Hash structure

Advantages:

  1. Similar data classification and integration storage, convenient data management
  2. It consumes less memory and CPU than string operations
  3. More space efficient than string storage

Disadvantages:

  1. Expiration can’t be used on fields, only on keys
  2. Redis cluster architecture is not suitable for large-scale use

The List structure

List Common Operations

LPUSH key value [value ...] // Insert one or more values into the head of the key list (leftmost)
RPUSH key value [value ...]	 // Insert one or more values value at the end of the key list (rightmost)
LPOP key	// Remove and return the head element of the key list
RPOP key	// Remove and return the last element of the key list
LRANGE key start stop	// Returns the element in the list key within the range specified by the offsets start and stopBLPOP key [key ...]  timeoutIf there is no element in the list, block and wait for timeout seconds. If timeout=0, block and waitBRPOP key [key ...]  timeoutIf there is no element in the list, block and wait for timeout seconds. If timeout=0, block and wait
Copy the code

List Application Scenarios

  • Common data structures:
  1. Stack = LPUSH + LPOP (FILO)
  2. Queue = LPUSH + RPOP (FIFO)
  3. Blocking MQ = LPUSH + BRPOP

  • Message flow of Weibo and wechat official account

A dime of money paid attention to Lei Jun, Ma Yun and other big V1) Rebus tweets with the ID10018LPUSH MSG :{dime-id}10018
2) Jack Ma posted a message on Weibo with the ID10086LPUSH MSG :{dime-id}10086
3LRANGE MSG :{dime -ID}0  4
Copy the code

The Set structure

Set common operations

SADD key member [member ...]	// Add an element to the set key. If the element exists, ignore it. If the key does not exist, create a new element
SREM key member [member ...]	// Remove the element from the set key
SMEMBERS key	// Get all the elements in the set key
SCARD key	// Get the number of elements in the set key
SISMEMBER key member	// Check whether the member element exists in the set key
SRANDMEMBER key [count]	// Select count from the set key
SPOP key [count]	// Select count from the set key and delete the element from the set key
Copy the code

Set operation

SINTER key [key ...] // Intersection operation
SINTERSTORE destination key [key ..] // Store the intersection results ina new collection destination
SUNION key [key ..] // Union operation
SUNIONSTORE destination key [key ...]	// Store the union result in the new collection destination
SDIFF key [key ...] // The difference set operation
SDIFFSTORE destination key [key ...]	// Store the difference set result ina new collection destination
Copy the code

Set Application Scenarios

  • Wechat lucky draw small program

1. Click SADD key {userID} 2 to join the collection. SRANDMEMBER key [count]/ SPOP key [count]Copy the code
  • Like, favorites and tags on wechat and Weibo

SADD like:{message ID} {user ID} 2. Unlike: SREM like:{message ID} {user ID} 3. SISMEMBER like:{message ID} {user ID} 4. SMEMBERS like:{message ID} 5 SCARD like:{message ID}Copy the code
  • Set operations

SINTER set1 set2 set3 -> { c } 	/ / intersection
SUNION set1 set2 set3 -> { a,b,c,d,e }	/ / and set
SDIFF set1 set2 set3  -> { a }	/ / difference set
Copy the code
  • Set operation to achieve the micro blog and Wechat attention model

1) People concerned by Zhang SAN:zhangsanSet-> {lisi, wangwu}2) People who pay attention to dime: yijiaoqianSet--> {zhangsan, zhaoliu, lisi, wangwu}3) People li Si cares about:lisiSet-> {zhangsan, yijiaoqian, zhaoliu, wangwu, xunyu)5) THE people I care about are also concerned about him (10 cents): SISMEMBER lisiSet yijiaoqian 6) People I may know: SDIFF yijiaoqianSet zhangsanSet->(zhangsan, zhaoliu}Copy the code
  • Set operation to achieve e-commerce commodity screening

SADD brand:huawei  P40
SADD brand:xiaomi  mi-10
SADD brand:iPhone iphone12
SADD os:android P40 mi-10
SADD cpu:brand:intel P40 mi-10
SADD ram:8G P40 mi-10Iphone12 SINTER OS: Android CPU: Brand: Intel RAM :8G > {P40, mi-10}

Copy the code

ZSet ordered set structure

ZSet Common operations

ZADD key score member [score member]... // add ZREM key member [member...] ZINCRBY key increment Member ZINCRBY key increment Member ZINCRBY key increment member ZINCRBY key increment member ZRANGE key start stop [WITHSCORES] ZRANGE key start stop [WITHSCORES] ZREVRANGE key start stop [WITHSCORES] ZREVRANGE key start stop [WITHSCORES] ZREVRANGE key start stop [WITHSCORESCopy the code

ZSet set operation

ZUNIONSTORE destkey numkeys key [key ...] ZINTERSTORE destkey numkeys key [key... // Calculate the intersectionCopy the code

ZSet application scenarios

  • ZSet set operation to achieve the leaderboard

ZINCRBY hotNews:20201221 1. Improve the Regulations on juvenile delinquency 2. ZREVRANGE hotNews:20201221 0 9 WITHSCORES 3. 7 days search list calculation: ZUNIONSTORE hotNews:20201215-20201221 7 hotNews:20201215 hotNews:20201216... ZREVRANGE hotNews:20201215-20201221 0 9 WITHSCORESCopy the code

Single threading and high performance of Redis

Is Redis single threaded?

Single-threading of Redis mainly means that the network IO and key-value pair reading and writing of Redis are completed by one thread, which is also the main process of Redis to provide key-value storage service externally. But other Redis functions, such as persistence, asynchronous deletion, and cluster data synchronization, are actually performed by additional threads.

Why is Redis single thread so fast?

Because all its data are in memory, all the operations are memory level operations, and single thread to avoid multi-thread switching performance loss problem, because Redis is a single thread, so be careful to use Redis instructions, for those time-consuming instructions (such as keys), must be careful to use, Not being careful can cause Redis to get stuck.

How does Redis single thread handle so many concurrent client connections?

IO multiplexing for Redis: Redis uses ePoll for I/O multiplexing to queue connection information and events, which in turn are placed into file event dispatchers, which distribute events to event handlers.

# check the maximum number of connections supported by Redis, modifiable in the redis. Conf file, # maxClients10000
127.0. 01.:6379> CONFIG GET maxclients
    ##1) "maxclients"# #2) "10000"
Copy the code

Other Advanced Commands

Keys: full traversal key

The key is used to list all the keys that meet the specified regular string rules. Avoid the key when the redis data volume is large and the performance is poor.

127.0. 01.:6379> set codehole1 a
OK
127.0. 01.:6379> set codehole2 b
OK
127.0. 01.:6379> set codehole3 c
OK
127.0. 01.:6379> set code1hole a
OK
127.0. 01.:6379> set code2hole b
OK
127.0. 01.:6379> set code3hole c
OK
127.0. 01.:6379> keys *
1) "codehole1"
2) "codehole3"
3) "codehole2"
4) "code3hole"
5) "code1hole"
6) "code2hole"
127.0. 01.:6379> keys codehole*
1) "codehole1"
2) "codehole3"
3) "codehole2"
127.0. 01.:6379> keys code*hole
1) "code3hole"
2) "code1hole"
3) "code2hole"
Copy the code

Scan: Progressively traverses the key

SCAN cursor [MATCH pattern] [COUNT count]
Copy the code

The scan parameter provides three parameters:

  • First argument cursor integer value (hash bucket index value)
  • The second is the regular pattern for key
  • The third is the number of keys traversed at one time (reference value, the number of underlying traversals is not necessarily the number of results that meet the conditions).

The cursor value is 0 for the first traversal, and then the first integer value in the returned result is used as the cursor for the next traversal. Iterate until the returned cursor value is 0.

Note: However, scan is not perfect. If there are key changes (add, delete, modify) during scan, the traversal effect may encounter the following problems: The newly added keys may not be traversed, and repeated keys may be traversed, which means that scan cannot guarantee the complete traversal of all keys, which we need to consider during development.

Info: Displays the running information of the Redis service

Divided into 9 blocks, each block has a very large number of parameters:

  • Server Indicates the environment parameters of the Server
  • Clients Client information
  • Memory Statistics about the running Memory of the server
  • Persistence Persistence information
  • Stats General statistics
  • Replication Information about primary and secondary Replication
  • CPU CPU usage
  • Cluster Cluster information
  • KeySpace Indicates the number of key-value pairs

Core Attribute Description:

connected_clients:2# number of clients instantaneous_OPs_per_sec instantaneous_OPs_per_sec789Used_memory:929864The amount of memory allocated by Redis (byteUsed_memory_human:908.07K # Redis Total memory allocated (Kb, human is displayed in units) used_memory_rss_human:2.28M # The size of memory requested from the operating system (Mb) (this value is usually greater than used_memory, because Redis memory allocation policy can generate memory fragmentation)929864# Redis memory consumption peak (byte)
used_memory_peak_human:908.07K # redis memory consumption peak (KB)0The maximum available memory value set in the configurationbyte), the default0, unlimited maxMemory_human :0B # configuration maxMemory_policy: Noeviction # When maxMemory is reachedCopy the code

GitHub Org_Hejianhui /JavaStudy GitHub Hejianhui /JavaStudy