background
We all know that master/slave synchronization in Redis is divided into two phases: full replication and incremental replication, and the two phases are easy to understand. But the two stages in the concrete problems involved in buffer for use in these two stages, buffer can understand core role: temporary data, avoid to send web request blocking the main thread processing orders, and himself in the learning process is a bit confusing, there was a touch of the soul problem: why need to design two, keep only one?
Why do we need two? Can we just keep one of them?
First, let’s take a look at the replication buffer, repl Backlog Buffer basics:
- A Replication buffer occurs during the full replication phase, where the master allocates a replication buffer to each newly connected slave. The Repl Backlog Buffer occurs during the incremental replication phase, where only one repl Backlog buffer is allocated to a master library.
- Each Buffer has a size limit when the Buffer is full. The repl Backlog Buffer, because of its circular structure, overwrites the starting location data directly, while the replication buffer causes the connection to be disconnected, the cache removed, the library reconnected, and the full replication restarted
Secondly, I think the characteristics of these two stages are as follows:
| phase | The characteristics of |
|---|---|
| Full amount of copy | 1. Different secondary libraries connect to the master library at different times (some may have been replicated and entered the next stage; 2. At this stage, after receiving data from the library, the processing time will be quite long, and then the communication with the master database is completed |
| Incremental replication | 2. The slave database frequently communicates with the master database and synchronizes incremental data. The amount of data processed each time is small and fast |
Then, make a scheme comparison:
| plan | advantages | disadvantages | Applicable scenario |
|---|---|---|---|
| replication buffer | Isolated from each other, no data coverage | Each command executed needs to be copied to the corresponding Buffer, consuming memory and affecting performance | The processing schedule varies greatly from library to library |
| repl backlog buffer | Saves memory and does not affect performance | Data overwrite, which affects the data to be synchronized from the library, but does not exist in the buffer, resulting in either loss of data from the library or full replication | There is not much difference between the processing progress and the main processing speed |
In fact, it can be seen from the advantages and disadvantages that the two of them are just complementary, and then combined with the characteristics of the full copy and incremental copy stage of self-thinking, just corresponding to the two advantages, using a dual structure, just avoid its disadvantages. That’s why you need two instead of just one.
conclusion
- Replication buffer and Repl Backlog Buffer designs are closely combined with the characteristics of different processing processes. In fact, our daily design of technical solutions must be closely combined with the characteristics of our business scenarios. Each technical solution has its own advantages and disadvantages. Ability makes its maximum limit achieves foster strong points circumvent weak points.
- Replication buffers and repl backlog buffers are also treated differently when their size overflows, which should also be of concern
- The replication buffer and the repl Backlog Buffer are personally linked because when the replication buffer data is synchronized from the repository, it is necessary to move to the repl Backlog buffer to synchronize data from the repository. Where does the synchronization start from the repl Backlog buffer? We all know that the repl Backlog Buffer has an offset. I think this offset exists not only in the Repl Backlog buffer, but also in the Replication buffer, which is at least consistent, so that there is no trace switching. Know where to read from?
(PS: which involves characteristics, advantages and disadvantages summary and so on are personal thinking, if there is an understanding error or with the official design/official source processing discrepancy, welcome to point out, can be corrected to learn)