This is the fifth day of my participation in the August Wen Challenge.More challenges in August

background

Redis is a key-value database server. The server usually contains any non-empty database, and each non-empty database can contain any key value pair. We collectively refer to the server’s non-empty databases and their key-value pairs as database state.

Redis is an in-memory database that stores its own database state in memory. If you do not save the database state stored in memory to disk, the database state in the server will also disappear once the server process exits

RDB persistence and disk interaction

Redis provides RDB persistence mode, this function can save Redis database state in memory to disk, avoid data loss.

RDB persistence, which can be performed either manually or periodically depending on server configuration options, saves the database state at a point in time in an RDB file.

The RDB file is a compressed binary file that restores the database state when the RDB file was generated.

RDB file creation and loading

There are two Redis commands that can be used to generate RDB files, one is SAVE and the other is BGSAVE.

  • SAVECommands block the Redis server process until the RDB file is created, and the server cannot process any command requests while the server process is blocked
  • BGSAVEThe command generates a child process, which then creates the RDB file, and the server process (parent) continues to process the command request.

The actual work of creating an RDB file is done by the rdb.c/rdbSave function.

void saveCommand(client *c) {
    if(server.rdb_child_pid ! =- 1) {
        addReplyError(c,"Background save already in progress");
        return;
    }
    rdbSaveInfo rsi, *rsiptr;
    rsiptr = rdbPopulateSaveInfo(&rsi);
    
    // Create the RDB file
    if (rdbSave(server.rdb_filename,rsiptr) == C_OK) {
        addReply(c,shared.ok);
    } else{ addReply(c,shared.err); }}Copy the code
void bgsaveCommand(client *c) {
    // ...
    if(server.rdb_child_pid ! =- 1) {
        addReplyError(c,"Background save already in progress");
    } else if (hasActiveChildProcess()) {
        // ...
    } else if (rdbSaveBackground(server.rdb_filename,rsiptr) == C_OK) {
        addReplyStatus(c,"Background saving started");
    } else{ addReply(c,shared.err); }}int rdbSaveBackground(char *filename, rdbSaveInfo *rsi) {
    pid_t childpid;

    // ...
    
    // Create a child process
    if ((childpid = redisFork(CHILD_TYPE_RDB)) == 0) {
        // ...
        
        / / save
        retval = rdbSave(filename,rsi);
        
        // ...
    } else {
        // ...
    }
    return C_OK; /* unreached */
}
Copy the code
/* Save the DB on disk. Return C_ERR on error, C_OK on success. */
int rdbSave(char *filename, rdbSaveInfo *rsi) {
    char tmpfile[256];
    char cwd[MAXPATHLEN]; /* Current working dir path for error messages. */
    FILE *fp = NULL;
    rio rdb;
    int error = 0;

    snprintf(tmpfile,256."temp-%d.rdb", (int) getpid());
    fp = fopen(tmpfile,"w");
    if(! fp) {char *cwdp = getcwd(cwd,MAXPATHLEN);
        serverLog(LL_WARNING,
            "Failed opening the RDB file %s (in server root dir %s) "
            "for saving: %s",
            filename,
            cwdp ? cwdp : "unknown",
            strerror(errno));
        return C_ERR;
    }

    rioInitWithFile(&rdb,fp);
    startSaving(RDBFLAGS_NONE);

    if (server.rdb_save_incremental_fsync)
        rioSetAutoSync(&rdb,REDIS_AUTOSYNC_BYTES);

    if (rdbSaveRio(&rdb,&error,RDBFLAGS_NONE,rsi) == C_ERR) {
        errno = error;
        goto werr;
    }

    /* Make sure data will not remain on the OS's output buffers */
    if (fflush(fp)) goto werr;
    if (fsync(fileno(fp))) goto werr;
    if (fclose(fp)) { fp = NULL; goto werr; }
    fp = NULL;
    
    // ...
}
Copy the code

Automatic interval saving

The user can set multiple save criteria with the Save option, but the server executes the BGSAVE command whenever any of the criteria is met

For example, if we provide the following configuration to the server:

Save 900 1 # Server 900 seconds, at least 1 change to the database save 300 10 # server 300 seconds, at least 10 changes to the database save 60 10000 # server 60 seconds, Make at least 10,000 changes to the databaseCopy the code

The BGSAVE command is executed as long as any of the above three conditions are met

Set save conditions

The server program sets the saveParams property of the server state redisServer structure based on the save conditions set by the Save option

struct redisServer {
    // ...
    
    // An array of save conditions
    struct saveparam *saveparams;
    
    // ...
}

struct saveparam {
    / / the number of seconds
    time_t seconds;
    / / modify
    int change;
}
Copy the code

For example, the value of the save option above is stored in the saveParam array in the server, as shown below

The dirty counter and the Lastsave attribute

In addition to the SaveParams array, the server state maintains a dirty counter, as well as a LastSave attribute

  • The dirty counter records the distance to the last successful executionSAVEorBGSAVEHow many times does the server modify the database state after the command is executed
  • Lastsave is a timestamp that records the last successful execution of the serverSAVEorBGSAVECommand time
struct redisServer {
    // ...
    
    // Modify the counter
    long long dirty;
    // The last time the save was performed
    time_t lastsave;
    
    // ...
}
Copy the code

When the server successfully executes a database modification command, the program updates the dirty counter: The dirty counter increases as many times the command changes the database

Check whether the storage conditions are met

The Redis server periodic operation function serverCron is executed every 100ms by default. This function is used to maintain the running server. One of its jobs is to check whether there are any conditions that meet the saveParams array by traversing all the saveParams array. BGSAVE is executed.