This is the 26th day of my participation in the Gwen Challenge
1. What are the underlying data structures of Redis
(1) String
(2) Hash
(3) Linked lists
(4)
(5) Zset ordered set
2. What are the differences between SDS in Redis and strings in C? What are the advantages
SDS (Simple Dynamic String) is the String representation used at the bottom of Redis.
SDS has one more SDSHDR header than C strings and stores free, len and buF.
Advantages:
(1) Faster to get string length. C language to obtain the string requires traversing the entire string, the time complexity is O(N), SDS header Len has stored the used space, the length of the string only needs O(1).
(2) Prevent buffer overflow. The C language string itself does not record the string length and used space, which may cause buffer overflow. The free member in the SDS table header stores free space. Before joining strings, free will be used to determine whether the remaining space is enough. If not, expansion will be performed.
(3) Reduce the number of memory allocation. C reallocates memory to grow or shorten a string. SDS reduces the number of memory allocations by using space preallocation and lazy space release strategies.
(4) Binary security. C truncates the file if the string ends at 0. SDS determines whether a string ends based on len members of the header and does not change the binary.
SDS: redisbook. Readthedocs. IO/en/latest/I…
3. How to implement the dictionary in Redis, how to resolve conflicts and expand capacity
The hash and index values are computed based on the key of the key-value pair, and the hash node containing the key-value pair is placed on the specified index of the hash array based on the index value.
Resolving conflicts: Redis uses the chained address method to resolve conflicts. Each hash node has a next pointer, and multiple hash nodes form a one-way linked list through the next pointer, always adding the latest node to the table header.
Capacity expansion: The capacity expansion of Redis is implemented by rehash to allocate space for dictionary HT [1], which is the first 2^n greater than or equal to HT [0]. Used * 2.
4. What is the usage scenario of redis hop table? Can you implement it
Usage scenario: Ordered list. Such as real-time leaderboards.
Java implementation
Public class SkipList {private static final int MAX_LEVEL = 16; private static final int MAX_LEVEL = 16; Private int levelCount = 1; Private Node head = new Node(); Private Random Random = new Random(); Public Node find(int value){Node p = head; for(int i = levelCount - 1; i >= 0; --i){ while(p.next[i] ! = null && p.next[i].data < value){ p = p.next[i]; } } if(p.next[0] ! = null && p.next[0].data == value){ return p.next[0]; }else{return null; Public void insert(int value){int level = randomLevel(); Node newNode = new Node(); newNode.data = value; newNode.maxLevel = level; Node update[] = new Node[level]; for(int i = 0; i < level; ++i){ update[i] = head; } Node p = head; for(int i = level - 1; i >= 0; --i){ while(p.next[i] ! = null && p.next[i].data < value){ p = p.next[i]; } update[i] = p; } for(int i = 0; i < level; ++i){ newNode.next[i] = update[i].next[i]; update[i].next[i] = newNode; } if(levelCount < level){ levelCount = level; Public void delete(int value){Node[] update = new Node[levelCount]; Node p = head; for(int i = levelCount - 1; i >= 0; --i){ while(p.next[i] ! = null && p.next[i].data < value){ p = p.next[i]; } update[i] = p; } if(p.next[0] ! = null && p.next[0].data == value){ for(int i = levelCount - 1; i >= 0; --i){ if(update[i].next[i] ! = null && update[i].next[i].data == value){ update[i].next[i] = update[i].next[i].next[i]; }}}} // private int randomLevel(){int level = 1; for(int i = 1; i < MAX_LEVEL; ++i){ if(random.nextInt() % 2 == 1){ level++; } } return level; } // Node inner class public class Node{private int data = -1; private Node next[] = new Node[MAX_LEVEL]; private int maxLevel = 0; @override public String toString(){StringBuilder Builder = new StringBuilder(); builder.append("{data:"); builder.append(data); builder.append("; leves: "); builder.append(maxLevel); builder.append(" }"); return builder.toString(); Public void display(){Node p = head; while(p.next[0] ! = null){ System.out.println(p.next[0] + " "); p = p.next[0]; } System.out.println(); }}Copy the codeData structure and algorithm – jump table: www.jianshu.com/p/fef9817cc…
5. Redis cache penetration, cache breakdown, cache avalanche, hot data set failure (FAQ)
(1) Cache penetration: no data is found in the cache layer and the database layer. Solution 1: Store the empty data and set an expiration time. Solution 2: Use a Bloom filter.
(2) Cache breakdown: the data in the cache will expire in large quantities at some point, leading to most requests falling directly on the database. Solution 1: Set different expiration times for different types of data. Solution 2: Use distributed locks.
(3) Cache avalanche: the cache will fail in a centralized manner at some point, or the cache system will fail, and all the concurrent traffic will directly reach the database. Solution 1: Ensure high availability of Redis and use cluster deployment. Solution 2: Current reduction and stage limiting.
(4) Hotspot data set failure: similar to cache breakdown. Hotspot data fails at the same time, causing the database to be queried. Solution: Use cluster deployment to ensure high availability of Redis.
6, Redis elimination strategy, write about LRU
Noeviction: Errors will be returned when memory usage exceeds configuration, no keys will be expelled
(2) Allkeys-LRU: When adding keys, if the limit is exceeded, the lRU algorithm will first expel the keys that have not been used for the longest time
(3) volatile-lru: When adding a key, if the limit is exceeded, the oldest unused key is expelled from the set of expired keys
(4) allkeys-random: when adding keys, if the limit is exceeded, allkeys will be randomly deleted
(5) volatile-random: When adding a key, if the limit is exceeded, it is randomly expelled from the set of expired keys
(6) volatile- TTL: expels soon-to-expire keys from those whose expiration time is configured
(7) volatile- lFU: Expel the least frequently used key from all keys with expiration time configured
(8) allkeys-lfu: remove the least frequently used key from allkeys
implementation
Public class LRU<K,V> {private static final Float hashLoadFactory = 0.6f; private LinkedHashMap<K,V> map; private int cacheSize; public LRU(int cacheSize) { this.cacheSize = cacheSize; int capacity = (int)Math.ceil(cacheSize / hashLoadFactory) + 1; map = new LinkedHashMap<K,V>(capacity, hashLoadFactory, true){ private static final long serialVersionUID = 1; @Override protected boolean removeEldestEntry(Map.Entry eldest) { return size() > LRU.this.cacheSize; }}; } public synchronized V get(K key) { return map.get(key); } public synchronized void put(K key, V value) { map.put(key, value); } public synchronized void clear() { map.clear(); } public synchronized int usedSize() { return map.size(); } public void print() { for (Map.Entry<K, V> entry : map.entrySet()) { System.out.print(entry.getValue() + "--"); } System.out.println(); }}Copy the code7. Redis persistence mode, RDB and AOF usage scenarios respectively
Persistent mode
(1) RDB: The memory snapshot at a certain time is written to the disk in binary mode
(2) AOF: Write operation logs of Redis into files in the way of adding
Usage scenarios
(1) If the speed of backup is desired, some data loss can be ignored and RDB is recommended
(2) If the pursuit of data integrity, can accept the sacrifice of backup speed, recommended to use AOF
8. How does Redis handle transactions
Redis uses four transaction commands: MULTI, EXEC, DISCARD and WATCH
Using MULTI to start a transaction, a client can send any number of commands to the server. These commands are not executed immediately, but are placed in a queue, and when the EXEC command is invoked, all the commands in the queue are executed. If an exception occurs during the execution of a transaction and EXEC is not executed, none of the commands in the transaction will be executed. If an error occurs after the EXEC command is executed, even if an error occurs in one of the commands in the transaction, other transactions will execute normally without rollback
By calling DISCARD, the client can empty the transaction queue and abandon transaction execution
The WATCH command provides CAS behavior for Redis
9. Why is Redis so fast?
(1) Multiplexing IO blocking mechanism is adopted
(2) The data structure is simple and the operation saves time
(3) Run in memory
10. Why is Redis so fast?
(1) Non-blocking IO multiplexing mechanism is adopted
(2) Single thread operation, avoiding frequent context switching
(3) Run in memory
11. Why is the operation of Redis atomic and how to ensure atomicity
Because Redis is single-threaded
For Redis, the implementation of GET, SET, eval and other apis is one task after another. These tasks are performed by the Redis thread, and the task either succeeds or fails
12. What schemes have been used by Redis cluster and how to do them respectively? Let’s talk about consistent hashing
Cluster solution
(1) Master-slave mode. You only need to add Slaveof 192.169.x.x6379 to the configuration file to specify the IP address and port number of the master
(2) Sentinel mode. Add different ports to several sentinel services, configure the IP address and port of the primary server, and add weights. Run the Redis command to start the sentinel,./redis-sentinel sentinel-26380.conf
(3) Cluster Cluster mode. Run the following command to configure cluster address: cluster-config-file nodes-7000.conf cluster configuration file: cluster-config-file nodes-7000.conf Cluster-node-timeout 15000 run the following command to check whether the slot is fully covered: cluster-require-full-coverage no Run the following command to check whether the slot is fully covered: Logfile “./redis.log”, listen on port: port 7000, copy the same configuration file, modify the port
13, when redis performance problems will occur, what is the solution?
(1) If the master writes a snapshot to the memory, the save command will schedule the rdbSave function, which will block the work of the main thread. If the snapshot is large, the performance will be greatly affected, and the service will be interrupted intermittently. Therefore, the master should not write a snapshot to the memory.
(2) Master AOF persistence. If AOF files are not rewritten, the impact of this method on performance is minimal, but AOF files will continue to grow, too large AOF files will affect the recovery speed of Master restart. The Master is advised not to do any persistent work, including memory snapshot and AOF log files. In particular, do not enable memory snapshot for persistence. If data is critical, a Slave enables AOF backup and synchronizes data once per second.
(3) The Master calls BGREWRITEAOF to rewrite the AOF file. AOF will occupy a large amount of CPU and memory resources during the rewrite, resulting in high load of services and temporary service suspension.
(4) Performance problems of Master/Slave replication in Redis. For the speed of Master/Slave replication and connection stability, it is better for Slave and Master to be in the same LAN
14. Have you ever used redis distributed lock, what are its advantages and disadvantages
Advantages:
(1) Redis has good performance
(2) Redis has better support for commands and is more convenient to implement
Disadvantages:
(1) Redis distributed lock requires continuous attempts to acquire the lock, which consumes performance
(2) The client from which Redis acquired the lock is suspended and can only release the lock after the timeout period
15. Let’s talk about the redis memory model
(1) Data
(2) The memory occupied by the main redis process itself. Code, constant pools, etc
(3) Buffer memory.
(4) Memory fragmentation. Redis is generated when physical memory is allocated and reclaimed
16. Tell me the difference between Redis and memcache
(1) Memcache only supports simple key-value data structure, while Redis supports string, hash, linked list, set set and ordered set set
(2) Redis does not store all data in memory and will swap some values that have not been used for a long time to disk
(3) Redis uses single-core, while Memcache can use multi-core. Redis has better performance when storing small data, while Memcache has better performance when storing big data
(4) Redis cluster can adopt one master with many slaves, one master with one slave. A memcache cluster can use only one primary and multiple secondary nodes
(5) Redis supports data recovery. Memcache is not supported
What have you done with Redis? (Try not to only do caching here, say queue, leaderboard/counter, publish/subscribe)
(1) Cache
(2) Ranking of commodity sales
(3) Message queue
18, you have used what non-relational database, what are the characteristics, what are the use scenarios (reflect your technology wide – degree of the moment, as much as possible to say, but not to say, to prevent being asked to death)
(1) Redis: data can be added, queried or deleted by key. Since primary key access is used, good performance and scalability can be achieved. Suitable for storing user information such as sessions, configuration files, parameters, shopping cart, and so on.
(2) MongoDB: Store data in the form of documents. Applicable to log and analysis
(3) HBase: Data is stored in a column family. A column family stores related data that is frequently queried together. It is suitable for log and blog platform
19. What are the advantages of Mongodb over Mysql? What is the data structure used for the underlying index
(1) Mongod’s access speed is better than MySQL’s
(2) Mongodb obtains data faster than MySQL
(3) The expansion, deletion and maintenance of Mongodb tables are simple and do not rely on SQL
(4) Mongodb is more friendly to development, and the data is in JSON format
(5) Mongodb will use system memory as cache
(6) Mongodb has its own distributed file system, which can be conveniently deployed in a server cluster
The Mongodb index data structure is b-tree
Why do you want to use this data structure, refer to the article: blog.csdn.net/u012939368/…
20. What is sharding in Mongodb
MongoDB sharding clusters distribute data across one or more shards. Each shard is deployed as a MongoDB replica set, which holds a portion of the cluster’s overall data. Because each shard is a set of replicas, they have their own replication mechanism that can automatically fail over. You can join individual shards directly, just as you would separate sets of replicas. However, if the set of connected replicas is part of a sharded cluster, only part of the data is visible.
Refer to the article: blog.csdn.net/wanght89/ar…