Suppose we have a business where some data obtained by users comes from third-party interfaces. To avoid frequent requests for third-party interfaces, we often add a layer of caching, which must be time-sensitive. Assuming that the structure we want to store is a hash (without the atomic operation of String ‘SET anotherkey “will expire in a minute” EX 60’), we both want to batch it into the cache, Make sure each key has an expiration date (in case the key never expires), and transactions are a good choice.
To ensure the atomicity of consecutive operations, most databases have transaction support, and Redis is no exception. But it’s not quite the same as a relational database.
Each transaction operation has begin, COMMIT, and ROLLBACK. Begin indicates the start of the transaction, COMMIT indicates the commit, and ROLLBACK indicates the rollback of the transaction. It looks roughly like this
begin();try { command1(); command2(); .... commit();} catch(Exception e) { rollback();}
Copy the codeRedis looks similar in form, divided into three phases
- Start transaction (multi)
- Command to join the queue (Business operation)
- Exec or discard a transaction
> multiOK> incr starQUEUED> incr starQUEUED> exec(integer) 1(integer) 2
Copy the codeThe above instructions demonstrate a complete transaction process. All instructions are not executed before exec, but cached in a transaction queue of the server. Once the server receives exec instructions, it disassemps and executes the whole transaction queue, and returns the running results of all instructions at one time after execution.
Redis transactions can execute multiple commands at once and are essentially a collection of commands. All commands in a transaction are serialized, executed sequentially and not inserted by other commands, without any insertion.
A series of commands can be executed in a queue in a one-time, sequential and exclusive manner (the main function of Redis transaction is actually to connect multiple commands to prevent other commands from jumping the queue)
That’s what the official document says
Transactions can execute more than one command at a time with two important guarantees:
A transaction is a single isolated operation: all commands in the transaction are serialized and executed sequentially. The transaction will not be interrupted by command requests from other clients during execution. A transaction is an atomic operation: all or none of the commands in a transaction are executed
This atomic operation, unlike the atomicity of the relational DB, does not guarantee atomicity completely, as will be explained later.
Several commands for Redis transactions
Soul strike! Are non-atomic Redis transactions also called transactions?
The MULTI command is used to start a transaction and always returns OK.
MULTI after execution, the client can send any number of commands to the server. These commands are not executed immediately, but are placed in a queue. When the EXEC command is invoked, all commands in the queue will be executed.
On the other hand, by calling DISCARD, the client can empty the transaction queue and abandon the transaction execution.
Nonsense not to say, direct operation to see the results of better understanding ~
Smooth sailing
Normal execution (batch processing can be very cool, each successful operation will take what they need, each other)
Soul strike! Are non-atomic Redis transactions also called transactions?
Discard a transaction (discard an operation to discard a transaction; previous operations do not count)
Soul strike! Are non-atomic Redis transactions also called transactions?
Consider this question: Suppose we have a key with an expiration date, and the key expires in a transaction operation. Will the exec succeed?
Errors in transactions
The Redis transaction is for data security operations, we may encounter the following two errors:
- The enqueue command may fail before the transaction executes EXEC. For example, commands can produce syntax errors (wrong number of arguments, wrong name of arguments, and so on), or other more serious errors, such as running out of memory (if the server has set the maximum memory limit with maxMemory).
- The command may fail after the EXEC call. For example, a command in a transaction might handle the wrong type of key, such as a list command over a string key, and so on.
Redis adopts different processing strategies for the above two kinds of errors. For errors occurring before EXEC execution, the server will record the failure of command joining and when the client invokes EXEC command, Reject execution and automatically abandon the transaction. (Prior to Redis 2.6.5, this was done by checking the return value of QUEUED: if the command QUEUED, it was QUEUED successfully; Otherwise, you fail to make the team)
There is no special treatment for errors that occur after EXEC command execution: even if one or some of the commands in the transaction fail during execution, other commands in the transaction continue to execute.
All operations will not succeed after exec if an error is reported in one operation record.
Soul strike! Are non-atomic Redis transactions also called transactions?
(in the example, k1 is set to String, decr k1 can be placed in the operation queue, because the statement error can only be detected at execution time, other correct statements will be executed normally)
Soul strike! Are non-atomic Redis transactions also called transactions?
Why doesn’t Redis support rollback
If you have experience with relational databases, the idea that Redis does not roll back when a transaction fails, but continues to execute the remaining commands may strike you as a little strange.
Here’s the official boast:
Redis command will only failed because of wrong grammar (and these problems cannot be found in the team), or command on the wrong type of button above: that is to say, from a practical point of view, failure command is caused by programming errors, and these errors should be in the process of development was found, and should not appear in a production environment. Because rollback support is not required, Redis can be kept simple and fast internally.
There is an argument that Redis is buggy in the way it handles transactions, but it is important to note that in general, rolling back does not solve the problem caused by programming errors. For example, if you intend to add 1 to the value of a key by INCR, but accidentally add 2 to it, or if you perform INCR on the wrong type of key, rollback does not handle these situations.
Since there is no mechanism to avoid programmer errors, and such errors usually do not occur in a production environment, Redis chooses the simpler, faster rollback free approach to transactions.
Soul strike! Are non-atomic Redis transactions also called transactions?
Transactions with Watch
The WATCH command is used to monitor any number of keys before a transaction begins: when the EXEC command is invoked to execute a transaction, if any of the monitored keys have been modified by other clients, the entire transaction will be interrupted and no longer executed, returning a failure.
The WATCH command can be called multiple times. Monitoring of keys takes effect from after WATCH execution until EXEC is called.
Users can also monitor any number of keys in a single WATCH command, like this:
redis> WATCH key1 key2 key3 OK
Copy the codeWhen EXEC is invoked, monitoring of all keys is removed, regardless of whether the transaction executes successfully. In addition, when the client disconnects, the client’s monitoring of the key is also removed.
Let’s take a simple example. I use Watch to monitor my account balance (I have 100 pocket money a week) and make normal consumption
Soul strike! Are non-atomic Redis transactions also called transactions?
But this card, also bound to my daughter-in-law’s Alipay, if WHEN I consume, she also consume, what will happen?
Sleepy I went downstairs to 711 to buy a pack of cigarettes, bought a bottle of water, at this time my daughter-in-law in the supermarket directly brush 100, at this time the balance is insufficient I still pick chewing gum.
Soul strike! Are non-atomic Redis transactions also called transactions?
When I went to pay my bill, I found my credit card failed (transaction interrupted)
Soul strike! Are non-atomic Redis transactions also called transactions?
You may not understand the use of watch, let’s take a look, if the same scenario, we do not have a watch balance, the transaction will not fail, the debit card becomes negative, it is not in line with business!
Soul strike! Are non-atomic Redis transactions also called transactions?
You can manually unmonitor all keys using the UNWATCH command with no arguments. For transactions that require multiple key changes, sometimes the program needs to lock multiple keys at the same time and then check whether the current values of these keys match the program’s requirements. When the value does not meet the requirement, the UNWATCH command can be used to cancel the current monitoring of the key, abandon the transaction halfway, and wait for the transaction’s next attempt.
The watch directive, similar to an optimistic lock, does not execute the entire transaction queue if the key value has been changed by another client, such as a list that has been pushed/popped by another client. (Of course, Redis can also be used to achieve distributed lock to ensure security, belonging to pessimistic lock)
The watch command monitors multiple keys prior to transaction execution, and if any key value changes after watch, the exec command will abandon the transaction and return a Null response notifying the caller that the transaction failed.
Pessimistic locking
Pessimistic locks, as the name implies, are Pessimistic. Each time I fetch the data, I think someone else will change it, so I Lock the data each time I fetch it, so that someone else will try to fetch it and block it until it gets the Lock. Traditional relational database inside used a lot of this locking mechanism, such as row lock, table lock, read lock, write lock, etc., are in the operation before the first lock
Optimistic, * * * * the lock
Optimistic Lock, as the name implies, is very Optimistic. Every time I go to get data, I think that others will not modify it, so I will not Lock it. But when UPDATING, I will judge whether others have updated the data during this period, and I can use the version number and other mechanisms. Optimistic locking is suitable for multi-read applications to improve throughput. Optimistic locking policy: Commit version must be greater than record current version to perform update
The implementation principle of the WATCH command
The watched_keys dictionary is stored in the server.h/redisDb structure type that represents the database. The keys of the dictionary are the monitored keys of the database, and the value of the dictionary is a linked list of all the clients that are monitoring the key, as shown below.
Soul strike! Are non-atomic Redis transactions also called transactions?
typedef struct redisDb { dict *dict; /* The keyspace for this DB */ dict *expires; /* Timeout of keys with a timeout set */ dict *blocking_keys; /* Keys with clients waiting for data (BLPOP)*/ dict *ready_keys; /* Blocked keys that received a PUSH */ dict *watched_keys; /* WATCHED keys for MULTI/EXEC CAS */ int id; /* Database ID */ long long avg_ttl; /* Average TTL, just for stats */ unsigned long expires_cursor; /* Cursor of the active expire cycle. */ list *defrag_later; /* List of key names to attempt to defrag one by one, gradually. */} redisDb; list *watched_keys; /* Keys WATCHED for MULTI/EXEC CAS */Copy the codeThe WATCH command associates the current client with the key to monitor in watched_keys.
For example, if the current client is Client99, the watched_keys shown earlier will be changed to look like this when the client executes WATCH key2 key3:
Soul strike! Are non-atomic Redis transactions also called transactions?
With the watched_keys dictionary, if a program wants to check whether a key is being monitored, it simply checks to see if the key is in the dictionary. If the program wants to get all the clients that monitor a key, it simply retrieves the value of the key (a linked list) and iterates through the list.
After the successful execution of any command to modify the database key space (such as FLUSHDB, SET, DEL, LPUSH, SADD, and so on), The multi-. c/touchWatchedKey function is called — it goes to the watched_keys dictionary to see if any clients are watching for keys that have been ordered to change, and if so, The program turns on the REDIS_DIRTY_CAS option on all clients that monitor this/these modified keys:
Soul strike! Are non-atomic Redis transactions also called transactions?
Void multiCommand(client *c) {if (c->flags & CLIENT_MULTI) {addReplyError(c,"MULTI calls can not be nested"); return; } / / open the transaction FLAG c - > flags | = CLIENT_MULTI; addReply(c,shared.ok); }/* "Touch" a key, so that if this key is being WATCHed by some client the * next EXEC will fail. */void touchWatchedKey(redisDb *db, robj *key) { list *clients; listIter li; listNode *ln; DictSize (db->watched_keys) == 0) return; DictFetchValue (db->watched_keys, key); if (! clients) return; // Traverse all clients and open their CLIENT_DIRTY_CAS flag listRewind(Clients,&li); while((ln = listNext(&li))) { client *c = listNodeValue(ln); c->flags |= CLIENT_DIRTY_CAS; }}Copy the codeWhen the client sends the EXEC command to trigger the transaction, the server checks the status of the client:
- If the CLIENT_DIRTY_CAS option is enabled on the client, at least one of the keys monitored by the client has been modified and the security of the transaction has been compromised. The server aborts the transaction and returns an empty reply to the client indicating that the transaction failed.
- If the CLIENT_DIRTY_CAS option is not turned on, then all monitor keys are secure and the server is officially executing the transaction.
Summary:
Three stages
- Open: Starts a transaction with MULTI
- Enqueue: To enqueue multiple commands into a transaction that are not executed immediately but are placed in a transaction queue waiting to be executed
- Execution: transactions are triggered by the EXEC command
Three features
- Separate isolated operations: All commands in a transaction are serialized and executed sequentially. The transaction will not be interrupted by command requests from other clients during execution.
- There is no concept of isolation level: commands in the queue are not actually executed until the transaction is committed, because no instructions are actually executed until the transaction is committed, so there is no headache of “in-transaction queries seeing updates in the transaction and out-of-transaction queries not”
- No guarantee of atomicity: Redis if a command fails in the same transaction, subsequent commands are still executed without rollback
In traditional relational databases, ACID properties are often used to verify the security of transaction functionality. Redis transactions guarantee consistency (C) and isolation (I), but not atomicity (A) and persistence (D).
The last
Redis transaction needs to go through a network read and write before sending each instruction to the transaction cache queue. When there are too many instructions in a transaction, the network IO time required will also increase linearly. Therefore, the client of Redis is usually used together with pipeline when executing transactions, which can compress multiple IO operations into a single IO operation.