Redis server is responsible for receiving and processing user requests and providing services for users. The Redis server startup command format is as follows:
redis-server [ configfile ] [ options ]
Copy the codeThe configfile parameter specifies the configuration file. The options parameter specifies the startup configuration items, which can override configuration items in the configuration file, such as
redis-server /path/to/redis.conf --port 7777 --protected-mode no
Copy the codeThis command starts the Redis service, specifies the configuration file /path/to/redis.conf, and provides two startup configuration items: port and protected-mode.
This article analyzes the startup process of Redis by reading the source code of Redis. The content is extracted from the new book “Redis Core Principles and Practices”. This article covers many concepts of Redis, such as event circulator, ACL, Module, LUA, slow logging, etc. These features are analyzed in detail in the author’s new book “Redis Core Principles and Practices”, which can be referred to the book.
Server definition
Note: The codes in this chapter are in server.h and server.c unless otherwise specified.
H /redisServer structure is defined in Redis, which stores Redis server information, including server configuration items and runtime data (such as network connection information, database redisDb, command table, client information, slave server information, statistics and other data).
struct redisServer {
pid_t pid;
pthread_t main_thread_id;
char *configfile;
char *executable;
char**exec_argv; . }Copy the codeThere are many attributes in redisServer, and I will not list them here. I will explain the related server attributes when analyzing the specific functionality. H defines a redisServer global variable:
extern struct redisServer server;
Copy the codeAll server variables referred to in this book, unless otherwise specified, are the redisServer global variables. For example, the server.list_MAX_ziplist_size attributes mentioned in Part 1 refer to the attributes of this variable. You can run the INFO command to obtain information about the server. The following information is displayed:
- Server: General information about the Redis server.
- Clients: indicates the client connection information.
- Memory: Information about memory consumption.
- Persistence: RDB and AOF persistence information.
- Stats: General statistics.
- Replication: Master/copy replicates information.
- CPU: CPU consumption information.
- Commandstats: Redis command statistics.
- Cluster: Redis cluster Cluster information.
- Modules: Indicates the information about the modules.
- Keyspace: database-related statistics.
- Errorstats: Error statistics of Redis.
INFO command response data except memory and CPU statistics are mostly stored in redisServer.
The main function
The server.c/main function is responsible for starting the Redis service:
int main(int argc, char **argv) {.../ / [1]
server.sentinel_mode = checkForSentinelMode(argc,argv);
/ / [2]
initServerConfig();
ACLInit();
moduleInitModulesSystem();
tlsInit();
/ / [3]
server.executable = getAbsolutePath(argv[0]);
server.exec_argv = zmalloc(sizeof(char*)*(argc+1));
server.exec_argv[argc] = NULL;
for (j = 0; j < argc; j++) server.exec_argv[j] = zstrdup(argv[j]);
/ / [4]
if (server.sentinel_mode) {
initSentinelConfig();
initSentinel();
}
/ / [5]
if (strstr(argv[0]."redis-check-rdb") != NULL)
redis_check_rdb_main(argc,argv,NULL);
else if (strstr(argv[0]."redis-check-aof") != NULL)
redis_check_aof_main(argc,argv);
// more
}
Copy the code[1] Check whether the Redis server is started in Sentinel mode. [2] The initServerConfig function initializes the properties of recorded configuration items in redisServer to default values. The ACLInit function initializes the ACL mechanism and the moduleInitModulesSystem function initializes the Module mechanism. [3] Record the executable path and startup parameters of Redis program for subsequent server restart. [4] If the Sentinel mode is started, the Sentinel mechanism is initialized. [5] If the startup program is redis-check-rDB or redis-check-aof, then redis_check_rdb_main or redis_check_aof_main will be executed. They will try to check and repair RDB and AOF files and then exit the program. After Redis is compiled, five executable programs are generated:
- Redis-server: redis executor.
- Redis-sentinel: Redis Sentinel execution program.
- Redis – CLI: Redis client program.
- Redis-benchmark: Redis performance benchmarking tool.
- Redis-check-aof, redis-check-rdb: a tool for checking and repairing RDB and AOF persistence files.
Continuing with the main function:
int main(int argc, char **argv) {...if (argc >= 2) {
j = 1;
sds options = sdsempty();
char *configfile = NULL;
/ / [6]
if (strcmp(argv[1]."-v") = =0 ||
strcmp(argv[1]."--version") = =0) version(); ./ / [7]
if (argv[j][0] != The '-' || argv[j][1] != The '-') {
configfile = argv[j];
server.configfile = getAbsolutePath(configfile);
zfree(server.exec_argv[j]);
server.exec_argv[j] = zstrdup(server.configfile);
j++;
}
/ / [8]
while(j ! = argc) { ... }/ / [9]
if (server.sentinel_mode && configfile && *configfile == The '-') {...exit(1);
}
/ / [10]resetServerSaveParams(); loadServerConfig(configfile,options); sdsfree(options); }... }Copy the code[6] Give priority to -v, –version, –help, -h, –test-memory and other commands. The STRCMP function compares two strings str1 and str2, and returns zero if str1=str2. If str1
str2, a positive number is returned. [7] If the second parameter of the startup command does not start with “–“, it is the configuration file parameter. Convert the configuration file path to an absolute path and store it in server.configfile. [8] Read the startup configuration items in the startup command and concatenate them into a string. [9] To start in Sentinel mode, you must specify the configuration file, otherwise you will directly report an error and exit. [10] config. C/resetServerSaveParams function reset for server saveparams attribute (the attribute stored RDB SAVE configuration). Config. C/loadServerConfig function from the configuration file to load all the configuration items, and use the startup command configuration items cover configuration items in the configuration file.
Tip: The configs array in config.c defines the mapping of most configuration options to server properties:
standardConfig configs[] = {
createBoolConfig("rdbchecksum".NULL, IMMUTABLE_CONFIG, server.rdb_checksum, 1.NULL.NULL),
createBoolConfig("daemonize".NULL, IMMUTABLE_CONFIG, server.daemonize, 0.NULL.NULL),... }Copy the codeThe configuration item rdbchecksum corresponds to the server.rdb_checksum property. The default value is 1 (that is, bool yes). If the reader needs to find the server properties and default values corresponding to the configuration item, they can do so.
Let’s continue with the main function:
int main(int argc, char **argv) {.../ / [11]
server.supervised = redisIsSupervised(server.supervised_mode);
intbackground = server.daemonize && ! server.supervised;if (background) daemonize();
/ / [12]
serverLog(LL_WARNING, "oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo"); ./ / [13]
initServer();
if(background || server.pidfile) createPidFile(); .if(! server.sentinel_mode) { .../ / [14]
moduleLoadFromQueue();
ACLLoadUsersAtStartup();
InitServerLast();
loadDataFromDisk();
if (server.cluster_enabled) {
if (verifyClusterConfigWithData() == C_ERR) {
...
exit(1); }}... }else {
/ / [15]InitServerLast(); sentinelIsRunning(); . }.../ / [16]
redisSetCpuAffinity(server.server_cpulist);
setOOMScoreAdj(- 1);
/ / [17]
aeMain(server.el);
/ / [18]
aeDeleteEventLoop(server.el);
return 0;
}
Copy the code[11] The server. Supervised property specifies whether Redis is started with upstart or Systemd. If server.daemonize is configured and server.supervised is not configured, Redis starts as a daemon. [12] Print startup logs. [13] initServer function initializes Redis runtime data, and createPidFile function creates PID file. [14] If non-Sentinel mode is enabled, the following operations are completed: (1) moduleLoadFromQueue loads the Module specified in the configuration file; (2) ACLLoadUsersAtStartup loads the ACL user control list. (3) The InitServerLast function is responsible for creating background threads and I/O threads. This step should be performed after the Module is loaded. (4) The loadDataFromDisk function loads AOF or RDB files from disk. (5) If Cluster mode is used to start, it is necessary to verify whether the loaded data is correct. [15] If Sentinel mode is used, sentinelIsRunning function is called to start Sentinel mechanism. [16] Bind the main Redis thread as much as possible to the CPU list configured by server.server_cpulist. Redis 4 began to use multi-threading, which can reduce unnecessary thread switching and improve performance. [17] Start the event loop. The event circulator is an important component in Redis. While Redis is running, the service is provided by the event looper. [18] At this point, it indicates that the Redis service has stopped, and the aeDeleteEventLoop function clears the events in the event loop, and finally exits the program.
Redis initialization process
Let’s look at the initServer function, which initializes Redis runtime data:
void initServer(void) {
int j;
/ / [1]
signal(SIGHUP, SIG_IGN);
signal(SIGPIPE, SIG_IGN);
setupSignalHandlers();
/ / [2]
makeThreadKillable();
/ / [3]
if (server.syslog_enabled) {
openlog(server.syslog_ident, LOG_PID | LOG_NDELAY | LOG_NOWAIT,
server.syslog_facility);
}
/ / [4]server.aof_state = server.aof_enabled ? AOF_ON : AOF_OFF; server.hz = server.config_hz; server.pid = getpid(); ./ / [5]
createSharedObjects();
adjustOpenFilesLimit();
/ / [6]
server.el = aeCreateEventLoop(server.maxclients+CONFIG_FDSET_INCR);
if (server.el == NULL) {...exit(1);
}
// more
}
Copy the code[1] Set UNIX signal processing function, so that Redis server receives SIGINT signal and exits the program. [2] Set the thread to respond to the CANCEL signal at any time and terminate the thread so as to stop the program. [3] If the Unix system log is enabled, the openlog function is called to establish an output connection with the Unix system log so as to output the system log. [4] Initialize the properties of the server responsible for storing run-time data. [5] The createSharedObjects function creates a shared data set that can be shared across scenarios, such as small numbers 0 to 9999, common strings +OK\r\n (command processing success response string), +PONG\r\n (ping command response string). The adjustOpenFilesLimit function tries to modify the environment variable to increase the maximum number of file descriptors the system can open to avoid errors due to a large number of client connections (Socket file descriptors). [6] Create an event loop. UNIX programming: Signal, also known as soft interrupt, signal is a method provided by UNIX to handle asynchronous events. The program tells the system kernel what to do after the signal is generated by setting a callback function. Many scenarios in the system will generate signals, such as:
- The user presses certain terminal keys to make the terminal generate signals. For example, when a user presses the break key (usually Ctrl+C) on a terminal, a SIGINT signal is sent to tell the program to stop running.
- Certain events occur in the system, for example, when the timer set by the alarm function times out, the kernel sends SIGALRM, or when a process terminates, the kernel sends SIGCLD to its parent.
- Some hardware exceptions, such as an invalid memory reference with a divisor of 0.
- A program uses functions to send signals, for example, calling the kill function to send arbitrary signals to another process.
Interested readers can delve into UNIX programming on their own.
Next, analyze the initServer function:
void initServer(void) {
server.db = zmalloc(sizeof(redisDb)*server.dbnum);
/ / [7]
if(server.port ! =0 &&
listenToPort(server.port,server.ipfd,&server.ipfd_count) == C_ERR)
exit(1); ./ / [8]
for (j = 0; j < server.dbnum; j++) {
server.db[j].dict = dictCreate(&dbDictType,NULL);
server.db[j].expires = dictCreate(&keyptrDictType,NULL); . }/ / [9]
evictionPoolAlloc();
server.pubsub_channels = dictCreate(&keylistDictType,NULL); server.pubsub_patterns = listCreate(); . }Copy the code[7] If server.port is configured, TCP Socket service is enabled to receive user requests. If server.tls_ port is configured, TLS Socket service is enabled. Redis 6.0 supports TLS connections. If server.unixsocket is configured, the UNIX Socket service is enabled. If none of the preceding three options is configured, an error message is displayed. [8] Initialize database server.db for storing data. [9] The evictionPoolAlloc function initializes the LRU/LFU sample pool and is used to implement the LRU/LFU approximation algorithm. Continue initializing the properties in the server that store runtime data:
void initServer(void) {.../ / [10]
if (aeCreateTimeEvent(server.el, 1, serverCron, NULL.NULL) == AE_ERR) {
serverPanic("Can't create event loop timers.");
exit(1);
}
/ / [11]
for (j = 0; j < server.ipfd_count; j++) {
if (aeCreateFileEvent(server.el, server.ipfd[j], AE_READABLE,
acceptTcpHandler,NULL) == AE_ERR)
{
serverPanic(
"Unrecoverable error creating server.ipfd file event."); }}.../ / [12]
aeSetBeforeSleepProc(server.el,beforeSleep);
aeSetAfterSleepProc(server.el,afterSleep);
/ / [13]
if (server.aof_state == AOF_ON) {
server.aof_fd = open(server.aof_filename,
O_WRONLY|O_APPEND|O_CREAT,0644); . }/ / [14]
if (server.arch_bits == 32 && server.maxmemory == 0) {... server.maxmemory =3072LL* (1024*1024); /* 3 GB */
server.maxmemory_policy = MAXMEMORY_NO_EVICTION;
}
/ / [15]
if (server.cluster_enabled) clusterInit();
replicationScriptCacheInit();
scriptingInit(1);
slowlogInit();
latencyMonitorInit();
}
Copy the code[10] Create a time event and execute the function serverCron, which is responsible for dealing with scheduled tasks in Redis, such as clearing expired data and generating RDB files. [11] Register for TCP sockets, TSL Socks, and UNIX sockets to listen for AE_READABLE file events. The acceptTcpHandler, acceptTLSHandler, and acceptUnixHandler functions accept new connections in the Socket. The acceptTcpHandler functions will be analyzed in detail later in this book. [12] Registers the hook function of the event looper, which the event looper calls before and after each block. [13] If AOF is enabled, the AOF file will be opened in advance. [14] If Redis runs on a 32-bit operating system, the memory space of the 32-bit operating system is limited to 4GB, so the memory used by Redis is limited to 3GB to avoid the crash of the Redis server due to insufficient memory. [15] If Cluster mode is started, clusterInit function is called to initialize the Cluster mechanism.
- ReplicationScriptCacheInit function to initialize the server repl_scriptcache_dict properties.
- The scriptingInit function initializes LUA.
- The slowlogInit function initializes the slowlog mechanism.
- The latencyMonitorInit function initializes the delay monitoring mechanism.
Conclusion:
- The redisServer structure stores server configuration items and runtime data.
- Server. c/main is the Redis startup method that loads the configuration, initializes the database, starts the network service, and creates and starts the event loop.
Finally, introduce the new book “Redis core principles and practice”, this book through in-depth analysis of Redis 6.0 source code, summarized the design and implementation of the core functions of Redis. Through reading this book, readers can deeply understand the internal mechanism and the latest features of Redis, and learn Redis related data structure and algorithm, Unix programming, storage system design, distributed system architecture and a series of knowledge. With the consent of the editor of the book, I will continue to publish some chapters in the book on binecy, as a preview of the book, you are welcome to check it out, thank you.
Yuqu platform preview: Redis core Principles and Practices jd link