Redis for Nosql database

Non-relational database NoSql (Not only SQL)

  • 1. Solve high concurrent read and write (such as dynamic website, moments of friends, news content…)
  • 2. Massive data storage and read/write (SNS interactive site)
  • 3. High scalability and availability
  • 4. All operations of Redis are atomic, and multiple instructions as a whole are either executed or not executed.

Redis core functions: file caching system, remote memory database Additional functions: publish and subscribe, master-slave replication, persistence, scripting and other features: replication, persistence, client sharding

Sharding, a method of dividing data into multiple parts, provides linear performance gains:

  • A. Users can store data to multiple machines
  • B. Obtain data from multiple machines

Redis data type

  • 1.STRING A STRING, integer, or floating point number
  • Each node is a string
  • 3.SET unordered collector
  • 4.HASH An unordered HASH containing key/value pairs
  • 5.ZSET ordered set

For example: Using Hash to store objects

User object — “json(string) –” redis

Conf Configuration items of redis. Conf are described as follows:

  1. Redis does not run as a daemon by default. You can modify this configuration item by using yes to enable daemonize no
  2. Pid is written to the /var/run/redis.pid file by default when Redis is running as a daemon. You can specify pidfile /var/run/redis.pid with pidfile
  3. The default port is 6379. In my blog post, I explained why 6379 was chosen as the default port because 6379 is the same number as MERZ on the phone keys, which is named after Alessia MERZ. Port 6379
  4. Host ADDRESS bind 127.0.0.1 5. Close the connection after the client is idle. If the value is 0, timeout 300 is disabled
  5. Redis supports four log levels: debug, verbose, notice, and warning. The default value is verbose loglevel verbose
  6. The log mode is standard output by default. If Redis is configured to run in daemon mode and the log mode is configured as standard output, the log will be sent to /dev/null logfile stdout
  7. Set the number of databases. The default database is 0. You can use the SELECT command to specify the database ID on the connection
  8. The default Redis configuration file provides three conditions: Save 900 1 Save 300 10 Save 60 10000 indicates 1 change in 900 seconds (15 minutes), 10 changes in 300 seconds (5 minutes), and 10000 changes in 60 seconds respectively.
  9. Redis uses LZF compression. If you want to save CPU time, you can turn this option off, but it will cause database files to become large Rdbcompression yes
  10. Specifies the name of the local database file. The default value is dump. RDB dbfilename dump. RDB
  11. Specify the local database directory dir./
  12. If the Slav service is used, set the IP address and port number of the master service. When Redis starts, it will automatically synchronize data from the master to slaveof
  13. If password protection is enabled for the master service, the Slav service connects to the Master using the masterauth password
  14. Set the Redis connection password. If the password is configured, the client needs to provide the password by using the AUTH command when connecting to Redis. Requirepass foobared is disabled by default
  15. The maximum number of client connections that Redis can open at a time is the maximum number of file descriptors that the Redis process can open at the same time. If maxClients 0 is set, there is no limit. When the number of clients reached the limit, Redis will close new connections and return Max Number of clients reached error message maxClients 128
  16. Specifies the maximum memory limit for Redis. When Redis starts up, it loads data into memory. When the maximum memory is reached, Redis tries to clear expired or expiring keys first. The new VM mechanism of Redis will store the Key in memory and the Value in swap maxMemory
  17. Specifies whether to log after each update operation. Redis writes data to disk asynchronously by default. If not enabled, data may be lost for a period of time in the event of a power outage. Because redis itself synchronizes data files according to the above save conditions, some data will only exist in memory for a period of time. The default is no appendonly No
  18. Appendone.aof appendfilename appendone.aof specifies the update log filename
  19. Three values are available: no: data is cached by the operating system and synchronized to disk (fast) always: data is written to disk (slow and safe) manually by calling fsync() after each update operation everysec: Represents synchronization once per second (compromise, default) appendfSync everysec
  20. Specifies whether to enable the virtual memory mechanism. The default value is no. For a brief introduction, the VM mechanism stores data in paging mode, and Redis swaps cold data on less visited pages to disk. Frequently visited pages are automatically swapped out from disk to memory (I will analyze Redis VM mechanism in detail in a later article) VM-enabled No
  21. The default value is/TMP /redis.swap. Multiple Redis instances cannot share vm-swap-file/TMP /redis.swap
  22. All data larger than VM-max-memory is stored in virtual memory. No matter how small the VM-max-memory setting is, all index data is stored in memory (Redis index data) Keys), which means that when vM-max-memory is set to 0, all values actually exist on disk. The default value is 0 vM-max-memory 0
  23. The Redis swap file is divided into many pages. An object can be stored on multiple pages, but a page cannot be shared by multiple objects. Vm-page-size is set according to the size of the stored data. The page size is best set to 32 or 64bytes; If you store large objects, you can use a larger page, or if in doubt, use the default vM-page-size 32
  24. Set the number of pages in the swap file. Since the page table (a bitmap indicating that a page is free or used) is placed in memory, every 8 pages on disk consumes 1byte of memory. vm-pages 134217728
  25. Set the number of threads that can access the swap file. It is best not to exceed the number of cores on the machine. If set to 0, all operations on the swap file are serial and may cause a long delay. The default value is 4 VM-max-threads 4
  26. Glueoutputbuf Yes Specifies whether to combine smaller packets into one packet when replying to clients. By default, glueOutputBuf Yes is enabled
  27. Hash-max-zipmap-entries 64 hash-max-zipmap-value 512 Specifies that a special hash algorithm is used when the number of elements exceeds a certain threshold or the maximum number exceeds a certain threshold
  28. Activerehashing Yes specifies whether rehash hash is enabled. The default is enable
  29. Include additional configuration files. You can use the same configuration file among multiple instances of Redis on the same host, and each instance has its own specific configuration file include /path/to/local.conf

Supplement 1: How to prevent server shutdown loss: a. Write data to hard disk B at a specified time and quantity. Set the interval for synchronizing data writes to disks

Supplement 2: What is a daemon? Daemon processes, also known as Daemon processes (sprites), are background service processes in Linux. It is a long-lived process that is usually independent of the control terminal and periodically performs some task or waits for some event to occur. The daemon is a special orphan process that detours from the terminal. Why detach from the terminal? The reason why the process is separated from the terminal is to avoid being interrupted by any information generated by the terminal, and the information during its execution is not displayed on any terminal. In Linux, the interface for each system to communicate with users is called terminal, and every process running from this terminal is attached to this terminal, which is called the control terminal of these processes. When the control terminal is shut down, the corresponding process will be shut down automatically.

Redis publishes subscriptions

Redis publish subscription (PUB/SUB) is a message communication model: the sender (PUB) sends the message and the subscriber (sub) receives the message.

The Redis client can subscribe to any number of channels.

The following figure shows channel Channel1 and the relationship between the three clients that subscribe to this channel — Client2, Client5, and Client1:



When a new message is sent to channel Channel1 via PUBLISH, the message is sent to the three clients that subscribed to it:

Redis transactions Redis transactions can execute multiple commands at once with the following guarantees: 1. Batch operations are placed in the queue cache before the EXEC command is sent. 2. After receiving the EXEC command, the transaction is executed. If any command fails to be executed, other commands are still executed. 3. During the transaction execution, command requests submitted by other clients are not inserted into the transaction execution command sequence.

A transaction goes through three phases from start to execution: Start transaction. Order in. Execute transactions.

Redis pipeline technology: Redis is a TCP service based on the client-server model and request/response protocol. This means that typically a request will follow the following steps: the client sends a query request to the server and listens for the Socket return, usually in blocking mode, waiting for the server to respond. The server processes the command and returns the result to the client.

Redis pipeline technology allows the client to continue sending requests to the server when the server does not respond, and eventually read all of the server’s responses at once. (Realize concurrent operation)