“This is the 17th day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

Transaction is an important function of database. A transaction is a series of operations that read and write data. Transactions execute with proprietary property guarantees including Atomicity, Consistency, Isolation and Durability (ACID property). These attributes include requirements for both the results of the transaction and the changes in the data state of the database before and after the transaction.

So, can Redis guarantee ACID properties completely? After all, if some attributes are not guaranteed in some scenarios, they may lead to data errors, so it is important to know how Redis supports these attributes and prepare strategies in advance.

Next, we will first understand the specific requirements of the ACID property for transaction execution. With this knowledge base, we can accurately determine whether Redis transaction mechanism can guarantee the ACID property.

Requirements for the transaction ACID property

First, atomicity. The atomicity requirement is clear: multiple operations in a transaction must complete, or none of them must complete. Atomicity is also one of the most valued properties when business applications use transactions.

Let me give you an example. If the user buys two items A and B in one order, the database needs to subtract the inventory for both items. If only the inventory of one item is deducted, then the inventory of the other item must be wrong after the order is completed.

The second attribute is consistency. This is easy to understand, meaning that the data in the database is consistent before and after the transaction is executed.

The third property is isolation. It requires that while the database is performing a transaction, other operations cannot access the data being accessed by the transaction.

Let me explain this to you with the example of the user ordering. Assume that the existing inventory of goods A and B is 5 and 10 respectively, and the quantity of user X’s order to A and B is 3 and 6 respectively. If the transaction does not have isolation, when user X orders the transaction, user Y also buys 5 pieces of B at a time, which, when combined with the 6 pieces of B purchased by X, exceeds the total value of B, which does not meet the business requirements.

The last property is persistence. After a transaction is executed by the database, changes to the data are persisted. When the database is restarted, the value of the data must be the modified value.

With the ACID property in mind, let’s take a look at how Redis implements transactions.

How does Redis implement transactions?

The transaction execution process consists of three steps, and Redis provides MULTI and EXEC commands to complete these three steps. So let’s analyze it.

First, the client uses a command to explicitly indicate the start of a transaction. In Redis, this command is MULTI.

In the second step, the client sends the specific operations (such as adding, deleting, or modifying data) to the server. These operations are data read and write commands provided by Redis, such as GET and SET. However, while these commands are sent by the client to the server, the Redis instance simply stores these commands in a command queue and does not execute them immediately.

In step 3, the client sends the commit transaction command to the server, and the database actually performs the specific operation sent in step 2. The EXEC command provided by Redis performs the transaction commit. After the server receives the EXEC command, all commands in the command queue are actually executed.

The following code shows the process of executing a transaction using MULTI and EXEC.

127.0.0.1:6379> MULTI OK 127.0.0.1:6379> DECR a:stock QUEUED # DECR b:stock QUEUED (integer) 4 2) (integer) 9Copy the code

We assume that the initial values of the a:stock and B :stock keys are 5 and 10. The two DECR commands executed after the MULTI command, which decate the values of the a:stock and b:stock keys by one, return QUEUED, indicating that the operations are temporarily stored in the command queue and have not yet been executed. When the EXEC command is executed, you can see that 4 and 9 are returned, indicating that both DECR commands have been successfully executed.

Ok, by using MULTI and EXEC commands, we can execute multiple operations together, but does this meet the ACID property required by the transaction? Next, let’s make a concrete analysis.

What attributes are guaranteed by Redis’s transaction mechanism?

Atomicity is the most important attribute of transaction operation, so we first analyze whether Redis transaction mechanism can guarantee atomicity.

atomic

If the transaction executes without any errors, MULTI and EXEC can be used together to ensure that multiple operations complete. But is atomicity guaranteed if transaction execution goes wrong? We need to look at it in three different ways.

In the first case, before the EXEC command is executed, the operation command sent by the client has inherent errors (such as syntax errors, using non-existent commands), which are identified by the Redis instance when the command is queued.

In this case, Redis reports an error and logs the error when the queue is ordered. At this point, we can continue to submit commands. After the EXEC command is executed, Redis will reject all submitted command operations and return the result that the transaction failed. In this way, all commands in the transaction will no longer be executed, ensuring atomicity.

Let’s take a look at a small example of a transaction that failed because of an error when the transaction operation was enqueued.

127.0.0.1:6379> MULTI OK # send the first operation in the transaction, but Redis does not support this command, Error message 127.0.0.1:6379> PUT a:stock 5 (error) ERR unknown command 'PUT', with args beginning with: 'a:stock', '5', # send the second operation in the transaction, this operation is correct command, Redis QUEUED the command 127.0.0.1:6379> DECR b:stock QUEUED Redis rejects 127.0.0.1:6379> EXEC (error) EXECABORT Transaction discarded because of previous errors.Copy the code

In this case, the transaction contains a PUT command that Redis does not support, so when the PUT command is queued, Redis will report an error. Although there is a valid DECR command in the transaction, the entire transaction is abandoned after the EXEC command is executed at the end.

Let’s look at the second case.

Unlike the first case, when the transaction operation is enqueued, the data types of the command and operation do not match, but the Redis instance does not detect an error. However, after executing the EXEC command, Redis will report an error when it actually executes these transactions. However, it is important to note that Redis will report an error for an incorrect command, but it will still execute the correct command. In this case, the atomicity of the transaction cannot be guaranteed.

Let me give you a little example. The LPOP command in the transaction operates on String data, and does not report an error when enqueued. However, EXEC does not report an error when executed. The LPOP command itself did not execute successfully, but the DECR command in the transaction did.

127.0.0.1:6379> MULTI OK 127.0.0.1:6379> LPOP a:stock QUEUED # DECR b:stock QUEUED 127.0.0.1:6379> EXEC 1) (Error) WRONGTYPE Operation against a key holding the wrong kind of value 2) (integer) 8Copy the code

See here, you may have a doubt, traditional database (MySQL, for example) in executing a transaction, will provide the rollback mechanism, when a transaction to perform when an error occurs, all operations in the transaction will cancel, have modified data will be restored to the transaction before the execution of the state, so, in the case of just now, if the command of the actual execution times wrong, Is it possible to use a rollback mechanism to recover the original data?

In fact, there is no rollback mechanism in Redis. Although Redis provides the DISCARD command, this command can only be used to actively abandon transaction execution, temporarily empty the command queue, not rollback effect.

How do I use the DISCARD command? Let’s look at the code below.

127.0.0.1:6379> GET a:stock "4" 127.0.0.1:6379> MULTI OK 127.0.0.1:6379> DISCARD OK # DISCARD a:stock = 1 DECR a:stock = 1 127.0.0.1:6379> GET a:stock "4"Copy the code

In this example, the value of the a:stock key starts with 4, and then we perform a transaction that wants to subtract 1 from the value of a:stock. However, at the end of the transaction, we execute the DISCARD command, so the transaction is abandoned. If we look at the value of a:stock again, we see that it is still 4.

Finally, let’s look at the third case where the Redis instance fails while executing the EXEC command of the transaction, causing the transaction to fail.

In this case, if Redis has AOF logging enabled, only part of the transactions will be logged to the AOF log. We need to check the AOF log file using the redis-check-aof tool, which removes completed transactions from the AOF file. In this way, after we use AOF to restore the instance, the transaction will not be executed again, thus ensuring atomicity.

Of course, if the AOF log is not enabled, the data will not be recovered after the instance restarts, which is not atomic.

Now that you know Redis guarantees atomicity for transactions, let’s briefly summarize:

  • An error will be reported when the command enters the team, and transaction execution will be abandoned to ensure atomicity;
  • No error was reported when the command entered the team, but the actual execution reported error, does not guarantee atomicity;
  • If AOF logging is enabled, atomicity can be guaranteed.

Next, let’s learn about the guarantee of consistency properties.

consistency

The consistency assurance of a transaction can be affected by bad commands, instance failures. Therefore, according to the occurrence of command failure and instance failure, we can divide it into three cases.

Situation 1: command entry error

In this case, the transaction itself is abandoned, so database consistency can be guaranteed.

Case 2: No error is reported when the command is entered, but an error is reported when it is executed

In this case, the incorrect command will not be executed, and the correct command will be executed without changing the consistency of the database.

Case 3: The instance fails when the EXEC command is executed

In this case, the instance will be restarted after failure, which depends on the way of data recovery. We will discuss the case according to whether the instance has RDB or AOF enabled.

If we do not enable RDB or AOF, then after the instance fails and restarts, the data is lost and the database is consistent.

If we use the RDB snapshot, since the RDB snapshot is not executed at transaction execution time, the results of the transaction command operation will not be saved to the RDB snapshot and the data in the database will be consistent when the RDB snapshot is used for recovery.

If we use THE AOF log and the instance fails before the transaction is logged to the AOF log, the database data recovered using the AOF log is consistent. If only part of the operation is logged to the AOF, we can use redis-check-aof to clear the completed operation in the transaction, and the database will be consistent after recovery.

So, in summary, Redis transactions guarantee consistency properties in the event of command execution errors or Redis failures. Next, let’s look at isolation.

Isolation,

The isolation of a transaction is guaranteed to be affected by concurrent operations that are executed with the transaction. Transaction execution can be divided into two stages: command enqueue (before EXEC command execution) and command actual execution (after EXEC command execution). Therefore, we divide these two stages into two situations to analyze:

  1. Concurrent operations are executed before the EXEC command. At this time, the isolation must be guaranteed by the WATCH mechanism, otherwise the isolation cannot be guaranteed.
  2. When concurrent operations are executed after the EXEC command, isolation is guaranteed.

So let’s look at the first case. When a transaction’s EXEC command has not been executed, the command operations of the transaction are transient in the command queue. At this point, if there are other concurrent operations, we need to see if the transaction uses the WATCH mechanism.

The function of the WATCH mechanism is to monitor the value changes of one or more keys before the transaction execution. When the transaction invokes the EXEC command to execute, the WATCH mechanism will first check whether the monitored keys are modified by other clients. If modified, transaction execution is abandoned to avoid breaking the isolation of the transaction. The client can then execute the transaction again, and if there are no concurrent modifications to the transaction data, the transaction can execute normally and isolation is guaranteed.

The specific implementation of the WATCH mechanism is realized by the WATCH command. I will give you an example. You can see the following figure to further understand the use of the WATCH command.

Let me explain to you what’s in the picture below.

At T1, client X sends the WATCH command to the instance. After receiving the WATCH command, the instance starts monitoring the change in the value of a:stock.

Then, at T2, client X sends MULTI and DECR commands to the instance, which temporarily stores the DECR command to the command queue.

At T3, client Y also sends a DECR command to the instance to modify the value of a:stock. The instance executes the command when it receives it.

At T4, the instance receives the EXEC command from client X, but the WATCH mechanism of the instance finds that A :stock has been modified and will abandon transaction execution. In this way, the isolation of transactions can be guaranteed.

Of course, without the WATCH mechanism, concurrent operations performed before EXEC commands would read and write data. In addition, when executing EXEC commands, the data to be manipulated by transactions has been changed. In this case, Redis does not make transactions isolated from other operations, so isolation is not guaranteed. The graph below shows what happens without the WATCH mechanic, so you can take a look.

At t2, the EXEC command sent by client X has not been executed, but the DECR command of client Y has been executed. At this point, the value of A :stock will be modified, so the isolation of transactions initiated by X cannot be guaranteed.

This is the case where the concurrent operation is executed before the EXEC command. Let me talk about the second case where the concurrent operation is received and executed by the server after the EXEC command.

Because Redis uses a single thread to execute commands, and Redis guarantees that all commands in the command queue are executed first after the EXEC command is executed. So, in this case, concurrent operations do not break the isolation of the transaction, as shown below:

Finally, let’s examine the persistence property guarantee of Redis transactions.

persistence

Since Redis is an in-memory database, whether or not data is persisted depends entirely on the Redis persistence configuration.

If Redis does not use RDB or AOF, the persistence properties of the transaction are definitely not guaranteed. If Redis uses RDB mode, then the data modified by the transaction is not guaranteed to persist if the instance goes down after a transaction is executed but before the next RDB snapshot is taken.

If Redis uses AOF mode, the persistence properties of the transaction are not guaranteed because the three AOF configuration options no, Everysec, and always will have data loss.

So, regardless of Redis’ persistence model, the persistence properties of a transaction are not guaranteed.

summary

In this lesson, we learned about transaction implementation in Redis. Redis supports transactions through four commands: MULTI, EXEC, DISCARD and WATCH. The functions of these four commands are summarized in the table below, which you can see again.

The ACID property of a transaction is a fundamental requirement for us to operate correctly using transactions. From the analysis of this lesson, we learned that Redis transaction mechanism can guarantee consistency and isolation, but cannot guarantee persistence. However, because Redis itself is an in-memory database, persistence is not a required attribute, and we are more concerned with atomicity, consistency, and isolation.

The atomicity case is more complicated. Only when the command syntax used in the transaction is wrong, the atomicity is not guaranteed. In other cases, the transaction can be atomically executed.

So, I give you a little advice: strictly follow the Redis command specifications for the development of the program, and through code review to ensure that the command is correct. In this way, the transaction mechanism of Redis can be applied in practice to ensure the correct execution of multiple operations.

Each lesson asking

As usual, I have a quick question for you. If a Redis instance fails while executing a transaction and Redis uses the RDB mechanism, can the atomicity of the transaction be guaranteed?

Welcome to write down your thoughts and answers in the comments area, and we will exchange and discuss together. If you find today’s content helpful, you are welcome to share it with your friends and colleagues. I’ll see you next time.