❝
First praise after look, annual salary million ~~~ official document translation summary, if there is a mistake please correct
❞
Redis (Database) usually obtains data from the redis(database) service when an application needs it:
+-------------+ +----------+
| | ------- GET user:1234 -------> | |
| Application | | Database |
| | <---- username = Alice ------- | |
+-------------+ +----------+
Copy the codeWhen using client caching, the application puts the results of hot queries directly into the application’s local memory, and then accesses those queries directly from the local memory, without needing to revisit the Redis service:
+-------------+ +----------+
| | | |
| Application | ( No chat needed ) | Database |
| | | |
+-------------+ +----------+
| Local cache | | | | user:1234 = | | username | | Alice | +-------------+ Copy the codeThe effect on “hot key” is very obvious, greatly reducing the pressure on the Redis server
Advantages of client caching:
- The latency is very small, eliminating the network overhead of accessing redis services
- The number of queries received by the Redis service is much smaller, so that Redis needs fewer nodes to achieve the same number of concurrent requests (fewer redis nodes are needed when using client caching, while keeping the application with the same QPS/TPS)
Two big difficult problem
The first question is how do you invalidate an outdated client cache
Sometimes this problem may not be a problem, and in some scenarios it is only necessary to set the maximum TTL of the client cache. More complex scenarios can also use redis pub/sub to notify stale keys of expiration. However, pub/sub costs a lot. Usually, every update of the cache requires PUBLISH to push outdated keys to the queue, which usually wastes more CPU slices of Redis-server. Then, each application client needs to subscribe to the queue. Even clients that do not cache the key will receive outdated keys pushed by Redis-server
Ps: The title of the official website says two big questions, I only saw this one, the other question did not say what (⊙ O ⊙)…
Redis client cache implementation
Redis uses Tracking to notify clients of cache failures and which clients to notify. It has two modes:
- By default, redis-server remembers the key accessed by each client and notifies the client if the key is invalid. There is a space cost (remember that each key cached by the client consumes memory), but only clients with corresponding keys are notified of the cache
Client 1 -> Server: CLIENT TRACKING ON
Client 1 -> Server: GET foo
(The server remembers that Client 1 may have the key "foo" cached)
(Client 1 may remember the value of "foo" inside its local memory)
Client 2 -> Server: SET foo SomeOtherValue
Server -> Client 1: INVALIDATE "foo" Copy the code- The client can enable tracking, which is not enabled at the start of the connection
- When tracking is enabled, until the connection is closed redis-server records the keys requested by each client.
- When a key has been modified by some client, is out of date, or has exceeded the memory limit and is deprecated, all clients that have enabled tracking and requested the key will receive an invalidation message.
- After receiving the message, the client deletes the corresponding key to prevent the outdated data from being accessed
On the surface, this looks perfect, but think about how much information the server needs to store for the lifetime of the connection if there are 10K clients. So Redis limits memory usage and CPU usage in the following ways
- The “Invalidation Table” is used to record the list of clients that may hold the key for each key. When the key in the Table is changed, the list of clients used by the key pair is read, the clients are notified, and the key is deleted from the Table. The table has a capacity limit. When the table is full and another key needs to be recorded, the oldest key will be removed from the table and the new key will be added. When the old key is removed, each client will be forcibly notified of the failure of the key (even if the key has not been modified), so as to: Later, the key was modified and the corresponding client could not be found to notify, thus causing outdated data access problems
- The invalid list stores the unique numeric ID of each client, and if the client is disconnected, the information is progressively GC
- Invalid lists are shared by all databases, and there is no database numbers to distinguish between them, so when one of database2
fooWhen the key is changed, it is cached in databasE3fooThe client of the key will also be notified of the failure. To reduce memory usage and implementation complexity
Broadcasting modeEach client cache key is usually not remembered, so this method does not waste any memory in Redis-server. Instead, clients subscribe to prefixed keys (object:oruser:), and will be notified each time the key prefix matches
Dual connection mode – Used for failure notification
Redis 6 new protocol connection RESP3, in the same connection can be used to do data query and receive failed key notification messages. However, many client implementations prefer to use two separate connections: one for data queries and one for receiving redis-server failures. The reason for doing this is that when tracking is enabled on a client, it can specify another connection to receive the failed key information by the client ID. If there is a failed key information, it can be redirected to the specific connection. All data connections from the same client can be redirected to the specific connection. This works well when the client implements connection pooling. This dual connection mode also works with RESP2, which only supports this mode because its connections are not as reusable as RESP3
A complete example of how RRESP2 works: Enable Trackinng to redirect to a dedicated connection for invalid information, query for a key, and then receive notification that the key is invalid when it is modified
To start, the client opens the first Connection, Connection 1, as a dedicated fail-message Connection, gets the Connection ID and subscribing to a special channel via pub/sub:
(Connection 1 -- a dedicated Connection to receive failure information)CLIENT ID
4:SUBSCRIBE __redis__:invalidate
* 3$9 subscribe $20 __redis__:invalidate : 1.Copy the codeTracking is now enabled for Datalink 2:
(Connection 2 -- Data Connection)CLIENT TRACKING on REDIRECT 4
+OK
GET foo
$3 bar Copy the codeConnection 2 May cache foo -> bar in local memory
A connection from another client changes the value of fookey:
(Connections to other clients)SET foo bar
+OK
Copy the codeNext, the invalidation message connection will receive the invalidation key foo
(Connection 1 -- a dedicated Connection to receive failure information)* 3$7
message
$20
__redis__:invalidate * 1$3 foo Copy the codeAfter receiving the key, the client checks whether the key exists in its local memory and disables it
Note: Pub/Sub transmits information as an array, with a key as an element. So that a message can be sent when a batch of keys expires
It is also important to note that RESP2’s use of pub/sub is purely a trick to reuse older client implementations. However, not all clients receive messages with invalid keys. Only the connections specified by the REDIRECT use pub/ SUB to receive messages
When using RESP3, invalid messages are pushed within the same connection or redirected to a dedicated connection (see the RESP3 specification for details)
What should “tracking” track
As you can see, by default, the client does not need to tell the server which keys it has cached. Every key queried by the client is tracked by the server because it may be cached by the client
The advantage is obvious: the client does not need to explicitly tell the cache key. However, as long as there is write traffic on the server, the corresponding client failure is triggered. The server obviously assumes that the client must cache access to the key, so:
- This is more efficient when clients tend to cache using a strategy of flushing out old keys
- The server will be forced to retain more data about the client key
- The client will receive a useless invalid message about its uncached object
Specifies the key to be cached by the client
Specify the “OPTIN” option when enabling Tracking
CLIENT TRACKING on REDIRECT 1234 OPTIN
Copy the codeIn this mode, the CLIENT does not cache any keys by default unless the CLIENT CACHING YES command is sent. The next command of this command is cached:
CLIENT CACHING YES
+OK
GET foo
"bar"
Copy the codeThe next command will cache all commands if it is a transaction “MULTI”, as well as lua scripts. This reduces the number of CLIENT CACHING YES commands, which is an optimization
Broadcasting mode
Having covered the first mode of client caching so far (default: Redis-server keeps track of the keys that each client may cache and notifies those clients when a key fails), let’s move on to the second mode. This mode does not consume any memory for the redis-server, but sends more unwanted messages to the client. The main behavior of this mode is:
- CLIENT TRACKING on REDIRECT 10 BCAST PREFIX object: PREFIX User: When TRACKING is enabled, add multiple prefixes to the BCAST option to enable the broadcast mode
- The redis-server does not store any data in the invalid list, but instead uses the data structure of the key prefix (user:) and the list of clients corresponding to the prefix to record the clients corresponding to each prefix
+--------+
| client |
+--------+
+-------+ +--------+
| user: | +----> | client |
+-------+ +--------+ +--------+ | client | +--------+ Copy the code- When a key is changed, all client lists that match all prefixes of the key are notified
- The more prefixes registered by the Redis-server, the more serious the CPU consumption becomes. But the more prefixes, the more accurate the client match is likely to be, and the less useless notifications the client receives
- When a key is matched with several or a single prefix, only one notification is created and sent to all clients with the corresponding prefix. Creating only one notification saves CPU consumption
NOLOOP options
A redis-CLI has cached keyfoo and wants to modify the key, directly modifying keyfoo in redis-server. Redis-server then notifies the client that Foo is invalid. There will be no problem in this case. But if redis-cli modifs the key in redis-server and modifs the value in local memory at the same time, redis-server notifies foo that it is invalid, isn’t that a bit redundant?
So Tracking provides a “NOLOOP” option that can be used in both default mode and broadcast mode. “When this option is enabled, redis-server does not notify clients that change keys of invalid keys.”
Race condition – Dual – link mode is unsafe
“D” indicates data connection and “I” indicates failure notification connection
Unsafe scenario:
[D] client -> server: GET foo
[I] server -> client: Invalidate foo (somebody else touched it)
[D] server -> client: "bar" (the reply of "GET foo")
Copy the codeThe D connection sends the command GET foo to read foo, and before receiving a response from GET Foo, redis has already sent a response, and the response is on its way back, redis-server foo is modified by another client. Connection I, on the same client as D, received a message notifying (and processing) that Foo was invalid, and then the response from GET Foo, on the same client as D, came back, obviously with stale data, because Redis responded to GET Foo before Foo was updated by other clients
The process after modification:
Client cache:set `foo` `caching-in-progress`
[D] client-> server: GET foo.
[I] server -> client: Invalidate foo (somebody else touched it)
Client cache: removes' foo 'from local memory[D] server -> client: "bar" (the reply of "GET foo")
Client cache: discovery without 'foo' does not proceedset `bar` Copy the codeA set key is allowed in local memory if:
- The key already exists and is being updated. allow
- If the key does not exist, it may be invalid, or the key has never been cached. In this case, only set key Caching-in-progress is allowed.
When using RESP3’s single link mode, where a connection is used for both data transfer and invalid notifications, there is no such problem, because in a single link scenario, all data and notifications are ordered (ordering means blocking, which is a problem with single connections, right?).
What happens when you disconnect unexpectedly
When the only invalidation notification connection is disconnected, isn’t the local cache of the corresponding client unable to update in real time? The following actions can avoid this problem:
- When the invalidation notification is disconnected, the client forcibly clears all local caches
- When using “RESP2+Pub/Sub” or “RESP3”, invalid channels will be pinged periodically. If ping times out, the connection will be closed and the cache cleared
What should be cached?
In general:
- Frequently updated keys should not be cached. The more frequently updated keys are, the more frequently they are notified to clients
- Try to cache the hot key, the effect is obvious
- Cache keys should be used as often as possible and updated at a reasonable frequency (the lower the better, reasonable means not too high). Update frequency should be considered
INCRTo count
Client cache obsolescence can be borrowed from LRU/LFU
Other Matters needing attention
- The best way to request a key is to get the TTL of the key back as well, and set a similar TTL in local memory
- If the key does not have a TTL, set the TTL to a maximum value to prevent access problems caused by incorrect connections
- The client memory usage must be limited, and the old key should be expelled when a new key cache is available
Limit server memory usage
Be sure to set a maximum value for the server’s invalid list, record the maximum number of keys, or use broadcast mode without using any memory. When broadcast mode is not used, the memory occupied by the server and the number of keys tracked are proportional to the number of clients corresponding to those keys