This is the 24th day of my participation in Gwen Challenge

Daily sentence

Only to be thoroughly tempered, good steel.

Redis transaction function in detail

MULTI, EXEC, DISCARD, and WATCH commands are the foundation of Redis transaction functionality.

Redis transactions allow a set of commands to be executed in a single step and guarantee two important things:

The distinction does not support transaction rollback

  • Redis serializes all commands in a transaction and executes them sequentially.

  • (atomic) It is not possible for Redis to insert a request from another client in the middle of a Redis transaction.

    • In a Redis transaction, Redis either executes all of the commands or nothing. Therefore, Redis transactions can guarantee atomicity. The EXEC command triggers the execution of all commands in a transaction.

      • When a client is executing a transaction, if it disconnects from the Redis server before invoking the MULTI command, nothing in the transaction will be performed.

      • Conversely, if it disconnects from the Redis server only after invoking the EXEC command, all operations in the transaction are performed.

  • (Isolation) This ensures that Redis executes these commands as a separate isolated operation.

  • (Persistence) When Redis uses an increment only File (AOF: Append-only File), Redis can ensure that a single write(2) system call is used so that the transaction is written to disk.

However, if the Redis server goes down, or the system administrator stops the Redis server process in some way, it is likely that Redis only performed part of the operation in the transaction.

Redis will check the above state on reboot and exit with an error message.

The above increment file can be fixed using the redis-check-aof tool, which will remove incomplete transactions from the above file so that the Redis server can be started again.

Starting with version 2.2, in addition to these two guarantees, Redis is able to provide additional guarantees in the form of optimistic locks, which are very similar to CAS: Check And Set operations. Redis’s optimistic locking is described later in this article.

Relevant command

MULTI (Start transaction)

Used to mark the beginning of a transaction block. Redis queues subsequent commands one by one before executing the command sequence atomically using the EXEC command.

This command is executed in the following format:

MULTI
Copy the code

The return value from this command is a simple string, always OK.

EXEC (executes transactions)

The transaction executes all previously queued commands and then restores the normal connection state.

When using the WATCH command, the EXEC command will execute the commands in the transaction only if the monitored key has not been modified, which takes advantage of the checkre-set (CAS) mechanism.

This command is executed in the following format:

EXEC
Copy the code

The return value of this command is an array in which each element is the return value of each command in the atomized transaction. When using the WATCH command, the EXEC command returns a Null value if the transaction execution is aborted.

DISCARD (Clear execution queue)

Clear all commands previously queued in a transaction, and then restore normal connection status.

If the WATCH command is used, the DISCARD command unmonitors all keys that are currently connected to the monitor.

This command is executed in the following format:

DISCARD
Copy the code

The return value from this command is a simple string, always OK.

WATCH

This command is used to set the given key to be monitored when a transaction needs to be executed conditionally.

This command is executed in the following format:

WATCH key [key ...]
Copy the code

The return value from this command is a simple string, always OK.

The time complexity is always O(1) for each key.

UNWATCH

Clears all keys previously monitored for a transaction.

If you invoke EXEC or DISCARD, there is no need to invoke the UNWATCH command manually. This command is executed in the following format:

UNWATCH
Copy the code

The return value from this command is a simple string, always OK. The time is always O(1).

Method of use

  • Enter a Redis transaction using the MULTI command. The return value of this command is always OK. At this point, the user can issue multiple Redis commands. Redis queues these commands instead of executing them.

  • Once the EXEC command is invoked, Redis executes all commands in the transaction.

  • Instead, calling the DISCARD command clears the transaction queue and exits the transaction.

The following example atomizes the values of the foo and bar keys:

  • As you can see from the session above, the return value of the EXEC command is an array in which each element is the return value of each command in the transaction, in the same order as the order in which the commands were issued.

  • When a Redis connection is in the context of a MULTI request, the return value of all commands issued through the connection is a QUEUE string (from the Redis protocol perspective, the return value is sent as a Status Reply). When the EXEC command is invoked, Redis simply schedules the execution of the commands in the transaction queue.

Internal transaction error

Two types of command errors may be encountered during the running of a transaction:

  1. A command may fail when queued. Therefore, it is possible for a transaction to fail before the EXEC command is invoked. For example, the command might have syntax errors (wrong number of arguments, wrong command name, and so on), or it might have some critical conditions (for example, if you use the maxMemory directive to configure a memory limit for the Redis server, you might have an out-of-memory condition).

  2. After the EXEC command is invoked, a command in a transaction may fail to execute. For example, we perform the wrong type of operation on a key (for example, perform a List operation on a String key).

The first type of error can be detected using Redis clients, which can check the return value of the queued command before calling the EXEC command: if the return value of the command is a QUEUE string, the command has been queued correctly; Otherwise, Redis will return an error.

If an error occurs while putting a command on the queue, most clients will abort the transaction and discard it.

Starting with Redis version 2.6.5, the server remembers errors that occur during transaction accumulation commands. Redis then rejects the transaction and returns an error message after running the EXEC command. Finally, Redis automatically discards the transaction.

  • Prior to version 2.6.5, if the above error occurred, Redis would still run the offending transaction after the EXEC command was invoked by the client, executing the command that was successfully queued, regardless of the previous error.

  • Starting with version 2.6.5, Redis will adopt the new behavior described earlier when it encounters the above error, making it easy to mix transactions and pipes. In this case, the client can send the entire transaction to the Redis server at once and read all the returned values again at a later time.

In contrast, Redis does not do anything special for transaction errors that occur after the EXEC command is invoked: during a transaction, even if one command fails, all other commands continue to execute.

This behavior is clearer at the protocol level. In the following example, a command will fail while a transaction is running, even if the command is syntactically correct:

The EXEC command in the example above returns a batch string containing two elements, an OK code and an -err error message. The client selects an appropriate method to provide error information to the user based on its library

Note that even if a command fails, all other commands in the transaction queue are still executed — Redis does not stop executing the commands in the transaction.

Here’s another example, again using the Telnet communication protocol, to see how syntax errors in a command are reported to the user as soon as possible:

This time, Redis did not queue the INCR command at all due to a syntax error.

Why doesn’t Redis support rollback?

If you have a background in relational databases, you will realize that while Redis commands may fail during a transaction, Redis still executes the remaining commands in the transaction and does not perform a rollback, which you may find strange.

  • The command will fail only if the Redis command is called with syntax errors (which Redis can detect while the command is placed on a transaction queue), or if an operation is performed on a key that does not match its data type: In practice, this means that Redis commands fail only because of program errors, which are most likely to be discovered during program development, but rarely in production.

  • Redis has been simplified internally to ensure faster performance, as Redis does not require transaction rollback capabilities.

A common objection to this behavior of Redis transactions is that the program may be buggy.

However, you should note that transaction rollback does not resolve any programming errors. For example, if a query increments the value of a key by 2 instead of 1, or increments the wrong key, the transaction rollback mechanism does not solve these programmatic problems.

Note that no one can fix the programmer’s own errors, which can cause Redis commands to fail. Because these errors are less likely to make it into production, we developed Redis in a simpler and faster way, without error rollback.

Discards command queues

The DISCARD command can be used to abort a transaction. In this case, no commands in the transaction are executed and the Redis connection is restored to a normal state.

Optimistic locking is implemented through CAS operations

Redis uses the WATCH command to implement the check and set (CAS) behavior of a transaction.

The keys that are parameters to the WATCH command are monitored by Redis, which can detect their changes. Before the EXEC command is executed, if Redis detects that at least one key has been changed, the entire transaction is aborted, and the EXEC command returns a Null value to alert the user that the transaction failed.

For example, suppose we need to automatically increment the value of a key by 1 (assuming Redis doesn’t have the INCR command).

The pseudo-code for the first attempt might look like this:

val = GET mykey
val = val + 1
SET mykey $val
Copy the code

If we have only one Redis client that performs the above pseudocode operation within a specified period of time, then the pseudocode will work reliably. If more than one client tries to increment the value of this key at about the same time, a race state will occur. For example, both client -A and client -B read the old value of this key (e.g., 10). Both clients increment the value of the key to 11, and then use the SET command to SET the new value of the key to 11.

Therefore, the final value of this key is 11, not 12.

Now, we can solve the above problem perfectly with the WATCH command, as shown below:

WATCH mykey
val = GET mykey
val = val + 1
MULTI
SET mykey $val
EXEC
Copy the code

From the above pseudocode, the transaction will fail if there is a race state and another client modifies the result of the val variable in the time between our invocation of the WATCH command and the EXEC command.

We just need to repeat the operation of the above pseudo-code and hope that there will not be a race state this time. This form of locking is called optimistic locking, and it is a very powerful lock. In many use cases, multiple clients may access different keys, so conflicts are unlikely — that is, there is usually no need to repeat the operation of the pseudo-code described above.

WATCH Command description

So what does the WATCH command actually do? This command causes the EXEC command to run transactions only if certain conditions are met: we require Redis to execute transactions only if all monitored keys have not been modified.

However, the same client may modify these keys within the transaction without aborting the transaction. Otherwise, Redis will not enter the transaction at all. (Note that if you use the WATCH command to monitor a volatile key, and then Redis expires the key after you monitor it, the EXEC command will still work.)

The WATCH command can be called multiple times. Simply put, all WATCH commands monitor the corresponding key as soon as it is invoked until the EXEC command is invoked. You can use any number of keys as command arguments in a single WATCH command.

When the EXEC command is invoked, all keys go to an unmonitored state and Redis does not care if the transaction is aborted. When a client connection is closed, all keys are also left unmonitored.

You can also use the UNWATCH command (no arguments required) to clear all monitored keys. This command is sometimes useful when we apply optimistic locks to certain keys.

Since we may need to run a transaction to modify these keys, we may not want to proceed after reading the current contents of these keys, using the UNWATCH command to clear all monitored keys. After running the UNWATCH command, the Redis connection is free to run new transactions again.

How do I use the WATCH command to implement the ZPOP operation?

This article provides an example of how to use the WATCH command to create a new atomization operation (Redis does not natively support this atomization operation), using the ZPOP operation as an example. This command pops the element with the lowest score from an ordered set in an atomized fashion. The following source code is the simplest implementation:

WATCH zset
element = ZRANGE zset 0 0
MULTI
ZREM zset element
EXEC
Copy the code

If the EXEC command in the pseudocode fails (for example, returning a Null value), then we simply need to repeat the operation.

Redis scripts and transactions

Redis scripts are also transactional by definition. So what you can do with Redis transactions can also be done with Redis scripts, which are usually simpler and faster.

Redis only introduced scripting features in version 2.6, and transactions have been around for a long time, so the current version has two seemingly repetitive features.

It is unlikely that support for transaction features will be removed anytime soon. Because users can still avoid the race state without resorting to Redis scripts, it is semantically appropriate. For another, more important reason, the Redis transaction feature has minimal implementation complexity.

However, it is unlikely that we will see an entire user base exclusively using Redis scripts for quite some time. If this happens, then we may discard, or even eventually remove, Redis transactions.