1. Transaction and locking mechanisms

1.1. Transaction Definition

  • A Redis transaction is a separate isolated operation: all commands in the transaction are serialized and executed sequentially. The transaction will not be interrupted by command requests from other clients during execution.
  • The main purpose of Redis transactions is to concatenate multiple commands to prevent other commands from jumping the queue.

1.2. Multi, Exec, discard

  • MultiCommand: transaction starts.
  • The entered commands are entered in the command queue, but are not executed until the command is enteredExecAfter that, Redis will execute the commands in the previous command queue.
  • You can pass it in the teamdiscardTo give up the team.

Case description

#Examples of successful execution127.0.0.1:6379> multi OK 127.0.0.1:6379(TX)> set k1 v1 QUEUED 127.0.0.1:6379(TX)> set k2 v2 QUEUED 127.0.0.1:6379(TX)> set k2 v2 QUEUED 127.0.0.1:6379(TX)> Set k3 v3 QUEUED 127.0.0.1:6379(TX)> exec 1) OK 2) OK 3) OK 127.0.0.1:6379>

#Case of abandonment

127.0.0.1:6379> multi
OK
127.0.0.1:6379(TX)> set k4 v4
QUEUED
127.0.0.1:6379(TX)> discard
OK
127.0.0.1:6379> 
Copy the code

1.3. Error handling in transactions

1. If an error is reported on one of the commands, the entire queue is cancelled.

Code instructions

127.0.0.1:6379> multi
OK
127.0.0.1:6379(TX)> set k1 v1
QUEUED
127.0.0.1:6379(TX)> set k2
(error) ERR wrong number of arguments for 'set' command
127.0.0.1:6379(TX)> 
127.0.0.1:6379(TX)> set k3 v3
QUEUED
127.0.0.1:6379(TX)> exec
(error) EXECABORT Transaction discarded because of previous errors.
127.0.0.1:6379> keys *
(empty array)

Copy the code

2. If an error occurs in a command, only the command with the error is not executed. All other commands are executed without rollback.

Code instructions

127.0.0.1:6379> multi OK 127.0.0.1:6379(TX)> 127.0.0.1:6379(TX)> set K1 v1 QUEUED 127.0.0.1:6379(TX)> set K2 v2 QUEUED 127.0.0.1:6379(TX)> incr k1 QUEUED 127.0.0.1:6379(TX)> exec 1) OK 2) OK 3) (error) ERR value is not an integer or out of  rangeCopy the code

1.4. Why are transactions needed?

Case description

  • A request to reduce the amount by 8000
  • A request to reduce the amount by 5000
  • A request to reduce the amount by 1000

In the case of multiple threads, a thread requesting 8000, a thread requesting 5000, and a thread requesting 1000 May read 10000, respectively. And then executing them separately would result in the final total being -4000. So we need to make it a transaction and have it execute in order.

1.5. Lock mechanism

1.5.1 Pessimistic Lock

  • Pessimistic LockAs the name suggests, it’s pessimistic. Every time you go to get your data, you think someone else is going to change it.So it locks up every time we pick up dataSo that someone else who wants to take the data will block it until it gets the lock.
  • Traditional relational databases use many of these locking mechanisms, such asRow locks, table locks, etc., read locks, write locksEtc., are locked before doing the operation.

1.5.2 Optimistic Locking

  • Optimistic Lock,As the name suggests, they’re optimistic, and every time they go to the data, they assume that no one else is going to change it,Therefore, it will not be locked, but when updating, it will determine whether others have updated the data during this period, using the version number and other mechanisms.
  • Optimistic locking applies toRead moreApplication types,This improves throughput. Redis uses this check-and-set mechanism to implement transactions.

1.6. WATCH and UNWATCH

1.6.1, WATCH

  • In the implementationmultiBefore, executewatch key1 [key2], you can monitor one (or more) keys.
  • If this (or these) key is changed by another command before the transaction executes, the transaction will be interrupted.

An example

#The first connection window
127.0.0.1:6379> watch k1
OK
127.0.0.1:6379> 
127.0.0.1:6379> multi
OK
127.0.0.1:6379(TX)> get k1
QUEUED
127.0.0.1:6379(TX)> set k3 v3
QUEUED


#The second connection window modifies K1
127.0.0.1:6379> append k1 1
(integer) 3


#The first connection window performs the transaction
127.0.0.1:6379> watch k1
OK
127.0.0.1:6379> 
127.0.0.1:6379> multi
OK
127.0.0.1:6379(TX)> get k1
QUEUED
127.0.0.1:6379(TX)> set k3 v3
QUEUED
127.0.0.1:6379(TX)> exec
(nil)
Copy the code

1.6.2, UNWATCH

  • Unmonitor all keys with the WATCH command.
  • If the EXEC or DISCARD command is executed first after the WATCH command, there is no need to execute UNWATCH.

1.7. Redis transaction features

  1. Separate isolated operations: All commands in a transaction are serialized and executed sequentially. The transaction will not be interrupted by command requests from other clients during execution.
  2. There is no concept of isolation level: no command in the queue is actually executed until it commits, because no instruction is actually executed until the transaction commits
  3. Atomicity is not guaranteed: if a command fails in a transaction, subsequent commands are still executed without rollback

2. Persistence

Redis provides two different forms of persistence.

  • RDB
  • AOF

2.1 and RDB

2.1.1 What is RDB?

Writes a Snapshot of an in-memory data set to disk at a specified time interval. A Snapshot is restored by reading the Snapshot file directly into memory.

2.1.2 How is backup performed?

  • Redis is created separately (forkA subprocess for persistence writes data to a temporary file until the persistence process is completeThe temporary file replaces the last persisted file.
  • During the whole process, the main process does not perform any I/O operations, which ensures high performance. If large-scale data recovery is required and data integrity is not sensitive, the RDB mode is more efficient than the AOF mode.
  • The downside of RDB is that data can be lost after the last persistence.

2.1.3, Fork

  • Fork copies the same process as the current one. All data values of the new process (variables, environment variables, program counters, etc.) are the same as those of the original process, but are a new process and a child of the original process
  • In Linux, fork() produces a child that is exactly the same as the parent, but will later exec the system call. For efficiency, Linux introduced the copy-on-write technique.
  • Generally, the parent process and child process share the same physical memory. Only when the contents of each segment of the process space change, the parent process copies the contents to the child process.

2.1.4. Related Configuration

#Name of the configuration file
dbfilename dump.rdb


#The configuration file placement path, namely redis: /usr/local/bin
dir ./


#Snapshot retention policy: The first configuration means a 3600 second interval in which all data will be persisted to hard disks if more than one data is changed
# save 3600 1
# save 300 100
# save 60 10000


#Compare save and BGSave
#Save: Save only, other ignore, all block. Manually save the Settings. Is not recommended.
#Bgsave: Redis takes snapshots asynchronously in the background and responds to client requests.



#If Redis cannot write to disk, stop Redis writes. Yes.
stop-writes-on-bgsave-error yes


#You can set whether to compress snapshots stored in disks. If so, Redis uses the LZF algorithm for compression. Recommend yes
rdbcompression yes



#After storing the snapshot, you can also have Redis use the CRC64 algorithm to validate the data, but this increases the performance cost by about 10%. Yes is recommended
rdbchecksum yes

Copy the code

2.1.5 RDB backup

  • Run the config get dir command to query the RDB file directory

  • Copy the *. RDB file somewhere else

  • The recovery of RDB

    • Close the Redis
    • Copy the backup file to the working directory firstcp dump2.rdb dump.rdb
    • Start Redis and backup data is loaded directly

2.1.6 advantages,

  • Suitable for large-scale data recovery
  • Data integrity and consistency requirements are not high more suitable for use
  • Save disk space
  • Fast recovery

2.1.7 and disadvantages

  • Although Redis uses copy-on-write for forking, it can be a performance drain if the data is large.
  • Backups are made at regular intervals in the backup cycle, so if Redis unexpectedly goes down, all changes since the last snapshot will be lost.

2.2, AOF

2.2.1 What is AOF?

  • Each write operation is recorded in the form of a log (incremental save), and all write instructions performed by Redis are recorded (read operations are not recorded). Only files can be appended but files cannot be overwritten. Redis will read the file to rebuild data at the beginning of startup.
  • In other words, redis restarts and executes write instructions from front to back based on the contents of the log file to complete data recovery.

2.2.2 AOF Persistence process

  1. The client request write command is appended to the AOF buffer by append.
  2. The AOF buffer synchronizes the operation sync to the AOF file on disk according to the AOF persistence policy [always, Everysec,no].
  3. When the AOF file size exceeds the rewrite policy or manual rewriting, the AOF file will be rewritten to reduce the AOF file size.
  4. When the Redis service restarts, it loads the write operations in AOF files to restore data.

2.2.3 File name and path

  • AOF is disabled by default.

  • You can configure the file name in redis.conf, which defaults to appendone.aof

  • The path for saving AOF files is the same as that for saving RDB files.

If AOF and RDB are enabled at the same time, the system reads AOF data by default (data will not be lost).

2.2.4 AOF restoration

1. Normal recovery

  • Change the default appendonly no to yes

  • Make a copy of the aOF file with data and save it to the corresponding directory (see config get dir).

  • Recovery: Restart Redis and reload

2. Abnormal recovery

  • Change the default appendonly no to yes

  • If the AOF file is damaged, run /usr/local/bin/redis-check-aof –fix appendone. AOF to restore it

  • Back up the bad AOF file

  • Recovery: Restart Redis and then reload

2.2.5 AOF synchronization frequency setting

#Always synchronize, every Redis write is logged immediately; Poor performance but good data integrity
appendfsync always


#Data is synchronized every second and logged every second. If the system goes down, the data in this second may be lost.
appendfsync everysec


#Redis does not synchronize actively, leaving the timing to the operating system
appendfsync no
Copy the code

2.2.6, compressed

  • When the size of an AOF file exceeds a set threshold, Redis starts the content compression of the AOF file, reserving only the minimal instruction set that can recover the data.

For example,

set k1 v1 
set k2 v2

# It will be compressed to
set k1 v1 k2 v2
Copy the code
  • If the RedisAOF Current size >= base_size +base_size*100% (default) and current size >= 64MB (default)Redis will rewrite AOF.

2.2.7 advantages,

  • The backup mechanism is more robust and the probability of data loss is lower.

  • Readable log text that can handle misoperations by manipulating AOF robustness.

2.2.8 and disadvantages

  • Takes up more disk space than RDB.

  • The backup restoration speed is slow.

  • There is some performance pressure to synchronize every read and write.

  • Some bugs exist, causing the recovery failure.

2.3, select,

  • The official recommendation is to enable both.

  • If you are not sensitive to data, you can choose to use RDB alone.

  • AOF alone is not recommended because of the potential for bugs.

  • If you’re just doing pure memory caching, you don’t have to do either.