It takes about 19 minutes to read this article

Hello, I’m Tiger!

How is a Redis command executed? To use Redis well, it is essential to understand its inner workings.

Only if we are familiar with the internal execution principle of Redis, we can take into account every execution step of Redis during development and be sure of everything.

Note: This article is based on Redis 6.2

01 Redis module and architecture

First, let’s take Apart Redis. When we are familiar with the Redis module, the problem of locating can get to the heart of the matter.

Let me first ask you a question. What is your understanding of the architecture of Redis?

Here I summarize the core modules of Redis, as shown below:

From a macro perspective, Redis can be divided into single node, master/slave copy, Sentinel (Sentinel) and Cluster (Cluster).

From the micro point of view, Redis is divided into event driven layer, command layer, memory allocation/reclamation, RDB/AOF persistence, monitoring and statistics.

Redis client: officially provides C language development client. In addition to sending commands, it also supports performance analysis and performance testing. You can check it out with redis-cli -h.
Event-driven layer: Redis is based on IO multiplexing and encapsulates the short and robust high-performance network framework AE. Internal integration of fileEvent (new connection, read, write events), timeEvent (time events).
Command layer: responsible for executing various commands. GET, SET, LPUSH, and so on.
Memory allocation/reclamation: Redis provides a fast, low-fragmentation memory allocation module based on Jemalloc.
RDB and AOF: Persistence policies provided by Redis to ensure data reliability.
Replaction: Redis uses replicas to implement a master-slave operation mode, which is the cornerstone of failover and improves system reliability. Also supports read and write separation to improve performance.
Sentinel: Sentinel is used to support automatic switchover between primary and secondary nodes in case of failure. Sentinel guarantees high availability for Redis.
Cluster: Redis is a high-performance model based on data sharding to support horizontal scaling.
Monitoring and Statistics: Redis provides a wealth of monitoring information and performance analysis tools, including memory usage, Big Key statistics, hot key statistics, benchmarks, and more.

02 Redis equal to single thread?

You may have heard the expression “Redis is single-threaded” on the web, or “Redis executes commands single-threaded”.

So, does Redis equal to single threads? I drew a picture for your reference.

The figure above can be divided into three modules

Main thread and IO thread: responsible for command reading, parsing, result return. Command execution is done by the main thread.
Bio thread: Responsible for executing time-consuming asynchronous tasks.
Background process: fork child process to execute time-consuming commands.

Prior to Redis 6, the main thread was responsible for receiving and executing commands. Redis 6 introduces IO multithreading. IO thread function is to receive commands, parse commands, send results.

In addition, Redis also has background threads for processing time-consuming tasks, called the BIO thread family. Bio Thread functionality currently has 3 points:

Close fd: Closes the file descriptor.
AOF fsync: Fsync flushs disks.
Lazy Free: Asynchronously releases the object’s memory.

IO threads and BIO threads are initialized when Redis Server starts. You can find it in the source code.

void InitServerLast(a) {
    // Initialize the bio thread
    bioInit();
    // Initializes the IO threadinitThreadedIO(); . }Copy the code

IO threads and BIO threads perform tasks through a producer-consumer model. As shown in the figure below

The main thread distributes the list of ready-to-write and ready-to-read clients to the IO thread queue IO_threads_list. IO threads are consumed by the IOThreadMain function.

The main thread submits the BIO task to the BIO_JOBS task queue, which is consumed by the BIO background thread through the bioProcessBackgroundJobs function.

In addition to these threads, Redis forks the bgsave, bgrewriteaof commands to avoid blocking the main thread. You can refer to the following functions:

// RDB background process task
int rdbSaveBackground(char *filename, rdbSaveInfo *rsi);
// AOF background process task
int rewriteAppendOnlyFileBackground(void);
Copy the code

03 Redis event-driven model

Before analyzing command execution, let’s take a look at the core Redis module, event-driven.

The “event-driven model” is common and can be considered the standard for high-performance network components. Generally, event-driven is divided into three steps: registering events, triggering events, and handling events.

Events registered by Redis can be divided into two categories:

FileEvent: indicates network events, including new connections, read events, and write events.
TimeEvent: a timeEvent. A task is executed at a specific time.

Where the new connection event is registered when Redis is started. When Redis receives a new connection request, it calls “acceptTcpHandler”.

void initServer(void) {
    // Register the new connection callback function acceptTcpHandler
    if(createSocketAcceptHandler(&server.ipfd, acceptTcpHandler) ! = C_OK) { serverPanic("Unrecoverable error creating TCP socket accept handler."); }}Copy the code

Read the event handler, readQueryFromClient, which is registered when a new connection is created.

Write the event handler “sendReplyToClient”, which is registered when the execution result is sent.

// Read the event handler. Register when creating a new connection
connSetReadHandler(conn, readQueryFromClient);

// Write the event handler. Single event loop, cannot send data when registration
connSetWriteHandler(c->conn, sendReplyToClient)
Copy the code

After Redis Server starts, the event loop “aeMain” is entered.

void aeMain(aeEventLoop *eventLoop) {
    eventLoop->stop = 0;
    while(! eventLoop->stop) {// event loop handler
        // Pay attention to reading, writing, and time eventsaeProcessEvents(eventLoop, AE_ALL_EVENTS| AE_CALL_BEFORE_SLEEP| AE_CALL_AFTER_SLEEP); }}Copy the code

After simplifying the aeProcessEvents function, the procedure is as follows.

int aeProcessEvents(aeEventLoop *eventLoop, int flags)
{
    int processed = 0, numevents;


    if(eventLoop->maxfd ! =- 1|| ((flags & AE_TIME_EVENTS) && ! (flags & AE_DONT_WAIT))) {// Execute the function beforeSleep before the event is triggered
        if(eventLoop->beforesleep ! =NULL && flags & AE_CALL_BEFORE_SLEEP)
            eventLoop->beforesleep(eventLoop);
 
        // Get the trigger event
        numevents = aeApiPoll(eventLoop, tvp);

        AfterSleep is executed after the event is triggered
        if(eventLoop->aftersleep ! =NULL && flags & AE_CALL_AFTER_SLEEP)
            eventLoop->aftersleep(eventLoop);
        
        // Loop through events
        for (j = 0; j < numevents; j++) {
            aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j].fd];

            // Executes the read event callback function rfileProc
            if (fe->mask & mask & AE_READABLE)
                fe->rfileProc(eventLoop,fd,fe->clientData,mask);

            // Executes the write event callback function wfileProc
            if(fe->mask & mask & AE_WRITABLE) fe->wfileProc(eventLoop,fd,fe->clientData,mask); }}// Time event
    if (flags & AE_TIME_EVENTS)
        processed += processTimeEvents(eventLoop);

    return processed;
}
Copy the code

Where “beforeSleep” function. Before each event is triggered, some specific function is performed.

04 How is a Redis command executed?

Having learned about the event module, let’s look at “how a Redis command is executed.” In the following figure, I have combed the implementation process.

First, the client initiates a request, which is received by the Redis event-driven module AE. Ae is a while infinite loop based on IO multiplexing (based on epoll under Linux).

After receiving a connection request, the AE module triggers a New connection event, which is executed by the acceptTcpHandler function. This function is responsible for receiving connections, creating new connections, and initializing client data structures.

You can refer to the following function call flow.

The last step, “createClient”, sets the read event callback function “readQueryFromClient” while initializing the client data structure. This callback function is the core entry point for Redis to execute commands.

client *createClient(connection *conn) {
    client *c = zmalloc(sizeof(client)); .if (conn) {
        // Set the readQueryFromClient callback function
        // Triggered when a command is receivedconnSetReadHandler(conn, readQueryFromClient); }... }Copy the code

After the “acceptTcpHandler” function completes, a Redis client connection is created!

Then we give orders

127.0.0.1:6379> SET foo bar
OK
Copy the code

After receiving the command, Redis triggers the AE module “read event” and enters the “readQueryFromClient” execution process. The process determines whether to enable IO multithreading by choosing one of the following two branches.

If enabled, the main thread adds the connected client to the clientS_pending_read read queue and flags the client as CLIENT_PENDING_READ, indicating that the client is readable. During the next loop, the clients_Pending_read queue is distributed to IO threads and the main thread to perform read requests, parse commands, and so on. Ultimately, the main thread executes the command.
If not enabled, the main thread “alone” performs all the processes of reading commands, parsing commands, executing commands, and sending results.

In the command flow parsing, the request string sent by the client is parsed. Specifically, the two steps are as follows.

Locate the corresponding execution function of the command and place it in client-> CMD ->proc.
Parse the parameters into client->argv and client->argc.

The Redis command execution functions are saved in redisCommandTable. The SET command corresponds to setCommand.

struct redisCommand redisCommandTable[] ={... {"set",setCommand,- 3."write use-memory @string".0.NULL.1.1.1.0.0.0},... }Copy the code

Next, we will focus on the IO multithreading scenario.

As mentioned earlier, for each event loop, Redis executes the preprocessor function “beforeSleep”, which distributes the clients_pending_read readready queue. The specific call function is as follows

int handleClientsWithPendingReadsUsingThreads(void) {
    // The IO thread is not started
    if(! server.io_threads_active || ! server.io_threads_do_reads)return 0; .// Otherwise, the read-ready queue is distributed to the thread private queue io_threads_list[target_id]
    while((ln = listNext(&li))) {
        client *c = listNodeValue(ln);
        inttarget_id = item_id % server.io_threads_num; listAddNodeTail(io_threads_list[target_id],c); item_id++; }...The main thread performs the io_threads_list[0] task
    listRewind(io_threads_list[0],&li);
    while((ln = listNext(&li))) {
        client *c = listNodeValue(ln);
        readQueryFromClient(c->conn);
    }
    listEmpty(io_threads_list[0]);

    // The main thread waits for other I/O threads to execute tasks
    while(1) {
        unsigned long pending = 0;
        for (int j = 1; j < server.io_threads_num; j++)
            pending += getIOPendingCount(j);
        if (pending == 0) break;
    }

    while(listLength(server.clients_pending_read)) {
        ...
        // Main thread, execute command (read finished, parsed command).
        if (processPendingCommandsAndResetClient(c) == C_ERR) {
            continue; }... }return processed;
}
Copy the code

This function traverses the clients_pending_read read ready queue, assigning read tasks to the IO thread and the main thread’s task queue io_threads_list. After receiving the task, the IO thread and main thread enter the readQueryFromClient execution process. Note that the client state has been set to “CLIENT_PENDING_READ” before readQueryFromClient is executed, so the client will not join the task queue again and will enter the actual execution process.

For your convenience, I have drawn a picture for your reference.

The last two steps of readQueryFromClient in the figure above read data from the socket via connRead and store it in the client QueryBuf. Next, parse and find the executable command setCommand. Finally, mark the client as CLIENT_PENDING_COMMAND, indicating that it is in an executable state.

Next, and most importantly, is the “main thread alone.” Execute the following function

int processPendingCommandsAndResetClient(client *c)
Copy the code

I have drawn the flow chart of this function for your reference

Where c-> CMD ->proc is used to execute the real command setCommand.

After executing the command, the main thread enters the last step “addReply”, calls prepareClientToWrite, and adds the execution result to the clients_pending_write write ready queue, waiting for the client to return.

void addReply(client *c, robj *obj) {
    // Join clients_pending_write write ready queue
    if(prepareClientToWrite(c) ! = C_OK)return; . }Copy the code

Upon entering the next event loop, the beforeSleep function dispatches the clients_pending_write write ready queue to the IO thread and main thread. Execute the following function:

int handleClientsWithPendingWritesUsingThreads(void) {
    // If the IO thread is enabled or the client connection is small
    // The main thread sends results directly
    if (server.io_threads_num == 1 || stopThreadedIOIfNeeded()) {
        returnhandleClientsWithPendingWrites(); }...// Otherwise, distribute clients_pending_write to IO thread and main thread for execution
    while((ln = listNext(&li))) {
        int target_id = item_id % server.io_threads_num;
        // Add to the thread task queuelistAddNodeTail(io_threads_list[target_id],c); item_id++; }...// The main thread processes tasks assigned to it
    listRewind(io_threads_list[0],&li);
    while((ln = listNext(&li))) {
        client *c = listNodeValue(ln);
        // Send it directly to the client
        writeToClient(c,0);
    }
 
    // Wait for the I/O thread to complete
    while(1) {
        unsigned long pending = 0;
        for (int j = 1; j < server.io_threads_num; j++)
            pending += getIOPendingCount(j);
        if (pending == 0) break;
    }

    // If the data is not written, the write event is registered
    // Emitted in the next event loop
    listRewind(server.clients_pending_write,&li);
    while((ln = listNext(&li))) {
        client *c = listNodeValue(ln);

        // Register write events
        if (clientHasPendingReplies(c) &&
                connSetWriteHandler(c->conn, sendReplyToClient) == AE_ERR)
        {
            freeClientAsync(c);
        }
    }
    listEmpty(server.clients_pending_write);
}
Copy the code

Finally, the IO thread and main thread send the command execution results to the client using the writeToClient function.

This is “a Redis command execution flow”.

05 summary

From the above, we can draw the conclusion that:

“Redis execution commands are single threaded, executed in the main thread”

Redis 6 IO multithreading helps the main thread read data, parse commands, and send results.

With this in mind, you can see why some Redis specifications say don’t use time-complex commands. It blocks the main thread and affects the execution of other commands.

And the IO multithreading provided by Redis 6 can effectively improve the performance of Redis single node. If you are using an earlier version of Redis, upgrade to Redis 6 and enable IO multithreading.

-End-

Finally, welcome to my public account “Tiger”.

I will continue to write better technical articles.

If my article is helpful to you, please give it a thumbs up

How is a Redis command executed?

01 Redis module and architecture

02 Redis equal to single thread?

03 Redis event-driven model

04 How is a Redis command executed?

05 summary

Related Posts

About my naked resignation and finding a job in a week

Merge sort, let me give you an example to get the idea

2020 Team OKR update