Sorted out some Java architecture, interview materials (micro services, clusters, distributed, middleware, etc.), partners in need can pay attention to the public number [programmer internal point], no routine to get

Redis high availability Overview

Before introducing Redis high availability, let me explain what high availability means in the context of Redis.

As we know, in a Web server, high availability refers to how long the server is accessible, measured by how long it is available for normal service (99.9%, 99.99%, 99.999%, etc.). However, in the context of Redis, the meaning of high availability seems to be broader. In addition to ensuring normal services (such as master/slave separation and fast disaster recovery technology), data capacity expansion and data security should also be considered.

In Redis, the technologies for high availability include persistence, replication, sentry, and clustering. Here’s what they do and what problems they solve.

1. Persistence: Persistence is the simplest method of high availability (sometimes it is not even classified as a high availability method). It is mainly used for data backup. That is, data is stored on hard disks to ensure that data is not lost due to process exit.
2. Replication: Replication is the foundation of highly available Redis, and both Sentinels and clusters are highly available on top of replication. Replication mainly realizes multi-machine data backup, load balancing for read operations and simple fault recovery. Defects: Failure recovery cannot be automated; Write operations cannot be load balanced. The storage capacity is limited by a single machine.
3. Sentinel: Based on replication, Sentinel implements automated failover. Defects: Write operations cannot load balance; The storage capacity is limited by a single machine.
4. Clustering: Through clustering, Redis solves the problem that write operations cannot be load balanced and storage capacity is limited by single machine, realizing a relatively complete high availability solution.

Redis persistence overview

Persistent function: Redis is an in-memory database, and data is stored in memory. In order to avoid permanent loss of data caused by process exit, Redis data in some form (data or commands) needs to be saved from memory to hard disk periodically. The next time Redis restarts, data is recovered using persistent files. In addition, you can copy persistent files to a remote location for disaster backup.

Redis persistence is divided into RDB persistence, which saves the current data to disk, and AOF persistence, which saves each write command to disk (similar to MySQL’s binlog). AOF is currently the dominant persistence method because it is more real-time, meaning less data is lost when the process unexpectedly exits, but RDB persistence still has its place.

RDB persistence and AOF persistence are described in turn; Due to differences between Redis versions, Redis3.0 shall prevail unless otherwise specified.

RDB persistence

RDB persistence saves a snapshot of the data in the current process to the hard disk (also known as snapshot persistence). The file suffix is RDB. When Redis restarts, the snapshot file can be read to recover data.

1. Trigger conditions

RDB persistence can be triggered manually or automatically.

1) Manual trigger

Both the save command and the BGsave command generate RDB files.

The save command blocks the Redis server process until the RDB file is created, and the server cannot process any command requests while the Redis server is blocked.

The BGSave command creates a child process that creates the RDB file, while the parent (the main Redis process) continues to process the request.

The server logs are as follows:

When executing bgSave, only the child process of the bgSave command blocks the server, but for save command, the whole process blocks the server, so save has been basically abandoned, and the online environment should not use save. Only the BGsave command will be covered later. In addition, when RDB persistence is automatically triggered, Redis also selects BGSave instead of Save for persistence; The following describes the conditions that automatically trigger RDB persistence.

2) Automatic trigger

save m n

The most common case for automatic triggering is to specify that BGSave is triggered when n changes occur within m seconds, via Save m n in the configuration file.

For example, if you view the default redis configuration file (redis. Conf in the root directory of redis on Linux), you can view the following configuration information:

Save 900 1 means: when the time reaches 900 seconds, if the REDis data has changed at least once, bgSave is executed. Save 300 10 and 60 10000. Bgsave is invoked when any of the three Save criteria are met.

The realization principle of Save M N

Redis’s save is implemented using the serverCron function, the dirty counter, and the LastSave timestamp.

ServerCron is a periodic operation function of the Redis server, which is executed every 100ms by default. This function maintains the state of the server by, among other things, checking that the criteria for the SAVE configuration are met and executing bgSave if so.

The dirty counter is a state maintained by the Redis server. It records how many times the server state has changed since the last bgSave/Save command was executed. When save/ BGSave completes, dirty is reset to 0.

For example, if Redis executed set MyKey helloWorld, the dirty value would be +1; If sadd myset v1 v2 v3 is executed, the dirty value will be +3; Note that dirty records how many changes the server makes, not how many commands the client executes to modify the data.

The LastSave timestamp is also a state maintained by the Redis server and records the last time a save/ BGSave was successfully executed.

The principle of save M n is as follows: Execute the serverCron function every 100ms; In the serverCron function, the save conditions configured are iterated and bgSave is performed as long as one of the conditions is met. For each save m n condition, only the following two conditions are met at the same time:

(1) Current time -lastsave > m

(2) Dirty >= n

Save M n Execution logs

Save m n triggers bgSave execution.

Other automatic trigger mechanisms

In addition to save, there are a few other situations that trigger BGSave:

• In the master/slave replication scenario, if the slave node performs full replication, the master node executes the BGsave command and sends the RDB file to the slave node
• RDB persistence is automatically executed when shutdown is executed, as shown in the following figure:

2. Execute the process

Introduced in front of the trigger bgsave conditions, the following will show bgsave command execution flow, as shown in the figure below (photo: https://blog.csdn.net/a1007720052/article/details/79126253) :

The five steps in the picture do the following:

1) The parent of Redis determines whether it is currently executing save, or a child of bgSave /bgrewriteaof (more on this later). If it is executing, the bgsave command returns directly. Bgsave/BgrewriteAof child processes cannot be executed at the same time, mainly for performance reasons: two concurrent child processes performing a large number of disk writes at the same time can cause serious performance problems.

2) The parent process forks to create a child process. This process is blocked and Redis cannot execute any commands from the client

3) After the parent fork, bgsave returns “Background saving Started” and no longer blocks the parent and can respond to other commands

4) The child process creates an RDB file, generates a temporary snapshot file based on the parent process memory snapshot, and replaces the original file atomic after completion

5) The child sends a signal to the parent to indicate completion, and the parent updates the statistics

3. The RDB file

RDB files are compressed binaries. Here are some details about RDB files.

The store path

The path for storing RDB files can be configured before startup or dynamically configured through commands.

Configuration: dir specifies a directory and dbfilename specifies a filename. The default is the dump. RDB file in the root directory of Redis.

Dynamic setting: The RDB storage path can also be dynamically modified after Redis is started, which is very useful in the case of disk damage or insufficient space. Run the config set dir {newdir} and config set dbfilename {newFileName} commands. As shown below (in Windows) :

RDB file format

The RDB file format is shown below (image source: Redis Design & Implementation) :

The meanings of each field are described as follows:

1) REDIS: constant, save “REDIS” 5 characters.

Db_version: specifies the RDB file version number, not the Redis version number.

3) SELECTDB 0 pairs: indicates a complete database (database 0). Similarly, SELECTDB 3 pairs indicates a complete database 3. The RDB file contains key-value pairs only if there are key-value pairs in the database (in Redis, there are only key-value pairs in database 0 and database 3). If all databases in Redis do not have key-value pairs, this section is omitted. Where: SELECTDB is a constant, representing the database number followed; 0 and 3 are database numbers. Pairs store specific key-value pair information, including key and value values, their data types, internal encoding, expiration time, compression information, and so on.

4) EOF: constant, indicating the end of the RDB file body.

5) check_sum: checksum of all previous contents; When loading an RBD file, Redis calculates the checksum and compares it with the check_sum value to determine whether the file is corrupt.

The compression

Redis uses the LZF algorithm to compress RDB files by default. Although the compression time, but can greatly reduce the size of the RDB file, so compression is enabled by default; You can disable it by running the following command:

It is important to note that RDB file compression is not done for the entire file, but for strings in the database, and only if the string reaches a certain length (20 bytes).

4. Load it upon startup

The loading of RDB files is done automatically when the server starts, and there is no special command. However, because AOF has a higher priority, when AOF is enabled, Redis will load AOF files first to recover data. Only when AOF is off will the RDB file be detected at Redis server startup and automatically loaded. The server blocks while loading the RDB file until the load is complete.

You can see the auto-loaded execution in the Redis startup log:

When Redis loads the RDB file, it verifies the RDB file. If the file is damaged, an error is displayed in the log, and Redis fails to start.

5. Summary of common RDB configurations

The following are common RDB configuration items and their default values. I won’t go into details here.

• Save m n: bgSave automatically triggers the condition; Without the save m n configuration, automatic RDB persistence is turned off, although it can still be triggered in other ways
• Stop-write-on-bgsave-error yes: Indicates whether Redis stops executing write commands when bgSave errors occur. If the parameter is set to yes, disk faults can be detected in a timely manner to avoid massive data loss. Set to no, Redis ignores bgSave errors and continues to execute write commands. Consider setting this option to No when monitoring Redis server systems (especially hard disks)
• Rdbcompression yes: indicates whether to enable RDB file compression
• Rdbchecksum yes: indicates whether to enable RDB file checksum, which takes effect when files are written and read. Turning off checksum provides approximately a 10% performance improvement when writing files and starting files, but data corruption is not detected
• Dbfilename dump. RDB: indicates the RDB filename
• Dir./ : Directory where RDB files and AOF files reside

4. AOF persistence

RDB persistence writes process data to a File, while AOF persistence (Append Only File persistence) records every write command executed by Redis to a separate log File (a bit like MySQL binlog). When Redis restarts, the command in the AOF file is executed again to recover the data.

Compared with RDB, AOF has better real-time performance, so it has become the mainstream persistence scheme.

1. Open AOF

Redis server by default RDB is enabled, AOF is disabled; To enable AOF, you need to configure it in the configuration file:

appendonly yes

2. Execute the process

Since every Redis write command needs to be recorded, AOF does not need to be triggered. The following describes the AOF execution process.

AOF implementation process includes:

• Command append: appends the Redis write command to the buffer aof_buf;
• File write and file sync: Synchronize contents in aOF_BUF to hard disks according to different synchronization policies.
• File rewrite: Periodically rewrite AOF files for compression purposes.

1) Command append (append)

Redis appends write commands to the buffer rather than directly writing them to files to avoid that every write command is directly written to the hard disk. As a result, the DISK I/O becomes a load bottleneck of Redis.

The format of command append is the protocol format of Redis command request. It is a pure text format, which has the advantages of good compatibility, readability, easy processing, simple operation and avoidance of secondary overhead. The specific format is omitted. In the AOF file, all write commands are sent by the client, except that the select command used to specify the database (e.g., select 0 for database 0) is added by Redis.

2) File write and file sync

Redis provides a variety of file synchronization policies for the AOF cache, which involve the write function and fsync function of the operating system, as described below:

To improve file writing efficiency, in modern operating systems, when a user calls the write function to write data to a file, the operating system usually stores the data temporarily in a memory buffer. When the buffer is filled up or exceeds the specified time limit, the data in the buffer is actually written to the disk. While this improves efficiency, it also raises security issues: if the computer is down, the data in the memory buffer can be lost; Therefore, the system also provides synchronization functions, such as fsync and fdatasync, to force the operating system to immediately write data in the buffer to disks, ensuring data security.

The file synchronization policy for the AOF cache is controlled by the appendfsync parameter. The values have the following meanings:

• Always: immediately after writing the aof_buf command, the fsync operation is invoked to synchronize the command to the AOF file. After fsync is complete, the thread returns. In this case, every write command needs to be synchronized to the AOF file, and the disk IO becomes a performance bottleneck. Redis can only support several hundred TPS writes, which seriously reduces the performance of Redis. Even with solid-state drives (SSDS), you can only process tens of thousands of commands per second, and the SSD life is greatly reduced.
• No: after the command is written into aof_buf, the system calls the write operation and does not perform fsync for the AOF file. The operating system is responsible for the synchronization, and the synchronization period is usually 30 seconds. In this case, the file synchronization time is not controllable, and the data in the buffer is too large. Therefore, data security cannot be guaranteed.
• Everysec: after writing aof_buf, the system calls write operation, and the thread returns after write. The fsync file synchronization operation is called once per second by a dedicated thread. Everysec is a compromise between the two strategies, a balance between performance and data security, and therefore the default and recommended configuration for Redis.

3) Rewrite

As the Redis server executes more and more write commands over time, AOF files get bigger and bigger; A large AOF file not only affects the normal running of the server, but also takes a long time to recover data.

File rewriting refers to periodically rewriting AOF files to reduce the size of AOF files. It should be noted that the AOF rewrite is to convert the data in the Redis process into write commands and synchronize them to the new AOF file. No reads or writes to old AOF files!

Another thing to note about file rewriting is that for AOF persistence, file rewriting, while highly recommended, is not required; Even without file rewriting, data can be persisted and imported when Redis is started; So in some implementations, automatic file rewriting is turned off and then timed at a certain time each day through a scheduled task.

File overwriting can compress AOF files because:

• Expired data is no longer written to the file
• Invalid commands are no longer written to files: some data is duplicated (set mykey v1, set mykey v2), some data is deleted (sadd myset v1, del myset), and so on
• Multiple commands can be combined into one, for example, sadd myset v1, sadd myset v2, and sadd myset v3 can be combined into sadd myset v1 v2 v3. However, to prevent client buffer overflow caused by a large command, you are not required to use only one command for keys of the list, set, hash, and zset types. Instead, it splits the command into multiple lines bounded by a constant. This constant is defined in redis. H /REDIS_AOF_REWRITE_ITEMS_PER_CMD and cannot be changed. In version 3.0, the value is 64.

It can be seen from the above content that file rewriting can not only reduce the space occupied by files, but also speed up the recovery speed because the command executed by AOF is reduced after rewriting.

File rewrite triggered

Triggering of file rewriting can be divided into manual triggering and automatic triggering:

Manually triggered: The bgrewriteaof command is invoked directly, which is somewhat similar to bgSave: both fork child processes do specific work, and both block only when forked.

The server logs are as follows:

Automatic trigger: The trigger time is determined according to the auto-aof-rewrite-min-size and auto-aof-rewrite-percentage parameters, as well as the aOF_current_size and aof_base_size states.

• Auto-aof -rewrite-min-size: specifies the minimum size of a file when aof rewriting is performed. The default value is 64MB.
• Auto-aof -rewrite-percentage: percentage of the current AOF size (that is, aOF_current_size) when aOF rewrite is performed to the aOF size (aof_base_size) when aOF rewrite was performed.

You can run the config get command to view the parameters:

The status can be viewed via Info Persistence:

Aof rewriting, that is, the bgrewriteAOF operation, is triggered only when the parameters auto-aof-rewrite-min-size and auto-aof- percentage are both met.

When bgreWriteAOF is triggered automatically, you can see the following server logs:

The process of document rewriting

File override process as shown in the figure below (photo: http://www.cnblogs.com/yangmingxianshen/p/8373205.html) :

There are two important points to note about the process of file rewriting :(1) rewriting is performed by the parent process and the child process; (2) Write commands executed by Redis during rewrite need to be appended to new AOF files, for which Redis introduces aof_rewrite_buf cache.

According to the figure above, the process of file rewriting is as follows:

1) The parent process of Redis checks whether there is a bgSave/bgrewriteAof child process. If there is a BGSave/bgrewriteAof child process, the bgrewriteAof command will be returned directly. If there is a BGSave command, the bgSave command will be executed after completion. As mentioned earlier, this is primarily for performance reasons.

2) The parent process forks the child process, and the parent process blocks.

2.1) After the parent process forks, the bgrewriteaof command returns “Background append only file rewrite started” and no longer blocks the parent process and can respond to other commands. All Redis write commands are still written to the AOF buffer and synchronized to disk according to the appendfsync policy to ensure that the original AOF mechanism is correct.

2.2) Since fork uses a copy-on-write technique, child processes can only share the memory data at fork. Because the parent process is still responding to commands, Redis uses the AOF rewrite buffer (aOF_rewrite_buf in the figure) to store this data, preventing it from being lost during the generation of a new AOF file. That is, during bgrewriteAof execution, Redis writes to both aof_buf and aof_rewirte_buf buffers.

4) The child process writes to the new AOF file according to the command merge rule according to the memory snapshot.

4.1) After the child process writes the new AOF file, it sends a signal to the parent process, and the parent process updates the statistics, which can be checked through Info Persistence.

4.2) The parent process writes the data in the AOF rewrite buffer to the new AOF file, thus ensuring that the database state stored in the new AOF file is consistent with the current state of the server.

4.3) Replace old files with new AOF files to complete AOF rewrite.

3. Load the file upon startup

As mentioned earlier, when AOF is enabled, Redis starts with AOF files loaded first to recover data. Only when AOF is closed will the RDB file be loaded to recover the data.

Redis startup log when AOF is enabled and the AOF file exists:

When AOF is enabled, but the AOF file does not exist, the RDB file will not be loaded even if it exists (some earlier versions may be loaded, but 3.0 will not), the Redis startup log is as follows:

File check

Similar to loading RDB files, when Redis loads AOF files, it verifies the AOF files. If the files are damaged, an error will be printed in the log, and Redis fails to start. However, if the end of AOF files is incomplete (such as sudden machine downtime) and the aof-load-truncated parameter is enabled, a warning is displayed in the log. Redis ignores the end of AOF files, and the file is successfully started. The aof-load-TRUNCated parameter is enabled by default:

The pseudo client

Because Redis commands can only be executed in the client context, and when AOF files are loaded, commands are read directly from the file, not sent by the client. Therefore, before loading the AOF file, the Redis server will create a client without network connection and then use it to execute the commands in the AOF file. The command execution effect is exactly the same as that of the client with network connection.

4. Summarize common AOF configurations

The following are common configuration items for AOF and their default values; I won’t go into details here.

• Appendonly No: Whether to enable AOF
• Appendfilename “appendonly.aof” : aof filename
• Dir./ : Directory where RDB files and AOF files reside
• Appendfsync Everysec: Fsync persistence policy
• No-appendfsync-on-rewrite no: Whether to disable fsync during AOF rewriting; If this option is enabled, the load on the CPU and hard disk (especially the hard disk) during file rewriting can be reduced, but the data during AOF rewriting may be lost. There is a balance between load and security
• Auto-aof-rewrite-percentage 100: specifies one of the triggering conditions for file rewriting
• Auto-aof -rewrite-min-size 64MB: one of the file rewrite trigger commits
• Aof-load-truncated yes: If the end of aOF file is damaged, whether the AOF file is still loaded when Redis starts

5. Program selection and common problems

Previously, RDB and AOF are introduced in detail. The following are the features of RDB and AOF, how to choose a persistence scheme, and common problems encountered in the process of persistence.

1. Advantages and disadvantages of RDB and AOF

Both RDB and AOF have their advantages and disadvantages:

RDB persistence

Advantages: RDB file compact, small size, fast network transmission, suitable for full copy; The recovery rate is much faster than AOF. Of course, one of the most important advantages of RDB over AOF is its relatively small impact on performance.

Disadvantages: The fatal disadvantage of RDB files is that the persistence mode of data snapshot makes it impossible to do real-time persistence, and in today’s increasingly important data, a large amount of data loss is often unacceptable, so AOF persistence becomes the mainstream. In addition, RDB files need to meet a specific format, poor compatibility (for example, the old version of Redis is not compatible with the new version of RDB files).

AOF persistence

In contrast to RDB persistence, AOF has the advantages of supporting second-level persistence and good compatibility, but has the disadvantages of large files, slow recovery, and large impact on performance.

2. Select a persistence policy

In the case of RDB persistence, the main Redis process blocks when the bgSave is forked. On the other hand, the child process writes data to the hard disk, causing IO pressure. For AOF persistence, data is written to hard disk much more frequently (in seconds under EverySec), IO pressure is higher, and it can even cause AOF appending blocking (more on this later). In addition, the rewriting of AOF files is similar to RDB’s BGSave. There are problems with blocking at forks and IO pressure on child processes. Because AOF writes data to hard disk more frequently, the impact on Redis main process performance is greater.

In actual production environments, there are various persistence strategies based on data volume, data security requirements of applications, and budget constraints. For example, do not use any persistence at all, use either RDB or AOF, or enable both RDB and AOF persistence. In addition, the choice of persistence must be considered in conjunction with the master-slave policy of Redis, because master-slave replication and persistence also have the function of data backup, and the master and slave can choose the persistence scheme independently.

The following discussion is for reference only. The actual solution may be more complex and diverse.

(1) It does not matter if the data in Redis is completely discarded (for example, Redis is completely used as the cache of DB layer data), then no matter it is stand-alone or master-slave architecture, any persistence can be carried out.

(2) In a standalone environment (which may be common for individual developers), if you can accept data loss of ten minutes or more, choosing RDB is better for Redis performance; If only second data loss is acceptable, select AOF.

(3) In most cases, we will configure the master/slave environment. The slave can not only realize hot backup of data, but also perform read/write separation and share Redis read requests, and continue to provide services after the master breakdown.

In this case, one possible course of action is:

Master: Turn off persistence completely (including RDB and AOF) to maximize master performance

Slave: Disables the RDB and AOF (you can also enable the RDB and AOF if data security requirements are not high), and periodically backs up persistent files (for example, back up to other folders and mark the backup time). Then turn off automatic rewriting of AOF, and then add a scheduled task that calls bgreWriteAof every day when Redis is idle, such as 12 a.m.

Here we need to explain why primary/secondary replication can be enabled to achieve hot backup of data, but also need to set persistence? In some special cases, master/slave replication is not enough to secure data, for example:

• The master and Slave processes stop at the same time: Consider a scenario where if the master and slave are in the same building or machine room, a power failure may cause the master and slave machines to shut down at the same time and the Redis process to stop. Without persistence, you face complete loss of data.
• The master restarts by mistake: Consider a scenario where the master service breaks down due to a fault. If the system has an automatic pull up mechanism (that is, the service is restarted when the service stops), the master automatically restarts. Since there is no persistent file, the data on the master is empty after the restart, and the data on the slave is also empty. If neither master nor slave is persisted, they also face complete data loss. Note that even with automatic master/slave switching using sentinels (more on sentinels later), it is possible for the automatic pull-up mechanism to restart before the sentinels poll to the master. Therefore, “automatic pull-up” and “do not persist” should be avoided at the same time.

(4) Remote DISASTER recovery: The persistence policies discussed above are aimed at common system faults, such as process exit, outage, and power failure. These faults do not damage disks. However, for disasters that may damage hard disks, such as fires and earthquakes, remote DISASTER recovery is required. For example, in the case of a single machine, RDB files or rewritten AOF files can be regularly copied to remote machines, such as AWS, through SCP. In the master/slave case, bgSave can be periodically executed on the master and RDB files copied to the remote machine, or bgrewriteAOF can be executed on the slave to rewrite AOF files and copy the AOF files to the remote machine. Generally speaking, RDB files are commonly used in disaster recovery due to their small size and fast recovery. The frequency of remote backup depends on data security requirements and other conditions, but should not be less than once a day.

3. Fork: CPU is blocked

In Redis practice, there are many factors that limit the memory size of a Redis single machine. For example:

• In the face of the sudden increase of requests, it is necessary to expand the capacity from the library, and the capacity expansion time is too long because the Redis memory is too large.
• When the host is down, the secondary library needs to be mounted after the host is switched, and the mounting speed is too slow due to the large memory of Redis.
• And fork operations during persistence, as described below.

Let’s start with the fork operation:

The parent process can create a child process by forking. After the child process is created, the parent and child processes share the code snippet, but do not share the process’s data space, but the child process gets a copy of the parent process’s data space. In the actual implementation of fork in operating system, the copy-on-write technique is basically adopted, that is, the parent and child processes actually share the data space before the parent and child processes attempt to modify the data space. But when either of the parent/child processes tries to modify the data space, the operating system makes a copy of the modified portion (a page of memory).

Although the child process does not copy the parent process’s data space, it does copy the memory page table (page table is the index of memory, directory). The larger the parent process’s data space, the larger the memory page table, and the more time it takes to fork.

In Redis, both RDB persisted BGSave and AOF overwritten BgrewriteAof require a child process to be forked. If the Redis memory is too large, it will take too long to copy the memory page table during fork operation. However, when the main Redis process forks, it is completely blocked, which means it cannot respond to client requests, resulting in a large request delay.

The fork operation takes different time depending on hardware and operating system. Generally speaking, if a Redis machine has up to 10GB of memory, the fork operation may take up to 100 milliseconds (or up to seconds if a Xen VM is used). Therefore, Redis is generally limited to 10GB of memory per machine; This figure is not absolute, however, and can be adjusted by looking at the time spent on fork in the online environment. Run the info stats command to view the value of latest_fork_usec in microseconds.

In order to reduce the blocking problem caused by fork operations, in addition to controlling the memory size of Redis single machine, the triggering conditions of AOF rewriting can be relaxed appropriately, and physical machines or virtualization technologies that support fork operations can be selected. For example, Vmware or KVM virtual machines are used instead of Xen virtual machines.

4. AOF appending block: the hard disk is blocked

As mentioned earlier, in AOF, if the file synchronization policy of the AOF buffer is everysec, then: in the main thread, after the command is written to aof_buf, the system call write operation is called. After the write operation, the main thread returns. The fsync file synchronization operation is invoked once per second by a dedicated file synchronization thread.

The problem with this approach is that if the hard disk is overloaded, the fsync operation may take more than 1s; If the Redis main thread continues to write commands to aOF_buf at a high speed, the hard disk may become increasingly overloaded and IO resources consumed faster. If the Redis process exits unexpectedly at this point, more and more data will be lost, possibly much longer than 1s.

To do this, Redis’s processing strategy looks like this: Each AOF of the main thread is compared to the last successful fsync; If it is less than 2s since last time, the main thread returns directly; If more than 2s, the main thread blocks until the fsync synchronization is complete. Therefore, if the fsync speed is too slow due to heavy system disk load, the main Redis thread will be blocked. Furthermore, with the Everysec configuration, AOF may lose up to 2s of data, not 1s.

AOF append blocking problem location method:

(1) Monitor aOF_delayed_fsync in info Persistence: this indicator accumulates when AOF append blocks occur (i.e. the main thread is blocked waiting for fsync).

(2) Redis log when AOF is blocked:

Asynchronous AOF fsync is taking too long (disk is busy?) . Writing the AOF buffer without waiting for fsync to complete, this may slow down Redis.

(3) If AOF appending blocking occurs frequently, the hard disk load of the system is too heavy. You can replace a hard disk with a faster I/O speed or use an I/O monitoring and analysis tool to analyze the I/O load, such as IOSTAT (system-level I/O), IOTOP (TOP I/O version), and pidstat.

5. Info command and persistence

Some of the ways to view persistence-related status through the INFO command were mentioned earlier, and are summarized below.

(1) Info Persistence

The result is as follows:

Some of the more important ones include:

• Rdb_last_bgsave_status: Indicates the result of the last BGSave execution, which can be used to find BGSave errors
• Rdb_last_bgsave_time_sec: Indicates the last BGSave execution time, in seconds. This parameter is used to check whether the BGSave time is too long
• Aof_enabled: Whether the AOF function is enabled
• Aof_last_rewrite_time_sec: indicates the last file rewrite execution time (unit: s). This parameter is used to check whether the file rewrite takes too long
• Aof_last_bgrewrite_status: Result of the last Bgrewrite execution, which can be used to find Bgrewrite errors
• Aof_buffer_length and AOF_rewrite_BUFFer_LENGTH: aOF cache size and AOF rewrite buffer size
• Aof_delayed_fsync :AOF append blocking statistics

(2) Info stats

Latest_fork_usec represents the last fork, as discussed earlier.

Six, summarized

The main contents of this paper can be summarized as follows:

1. The role of persistence in Redis high availability: data backup, compared with master-slave replication, emphasizes the backup from memory to hard disk.

2. RDB persistence: Data snapshot is backed up to hard disk. The trigger condition (including manual start and automatic trigger), execution process, RDB file and so on are introduced. It should be noted that the file saving operation is carried out by the sub-process of fork.

3. AOF persistence: back up write commands to hard disk (similar to MySQL binlog). It introduces how to start AOF persistence and how to execute AOF persistence.

4. Some practical issues: how to choose persistence strategy, fork block, AOF append block, etc.

Hats off to the big guy.
www.cnblogs.com/kismetv/p/9…

Small benefits:

Hundreds of technical e-books, SHH ~, free for your friends. Concern public number reply [666] to get by yourself