SpringBoot integrate Redis
Redis data type
| The data type | Maximum amount of data to be stored |
|---|---|
| key | 512M |
| string | 512M |
| hash | 2 ^ 32-1 |
| list | 2 ^ 32-1 |
| set | 2 ^ 32-1 |
| sorted set | |
| bitmap | 512M |
| hyperloglog | 12k |
Redis basic syntax
Storage string string
The string type is the most basic data storage type in Redis. It is binary safe in Redis, which means it can accept data in any format, such as JPEG image data or Json object descriptions. In Redis, a string Value can hold up to 512 megabytes of data.
Set key Value: Sets the key to hold the specified string value. If the key exists, the operation is overwritten. Always return “OK”
Get key: obtains the value of the key. If the value associated with the key is not a String, redis will return an error because the get command can only be used to get a String value. If the key does not exist, null is returned.
Getset Key Value: obtains (outputs) the key value and then sets the key value.
Del key: deletes the key value
Incr key: increments the atomicity of the value of the specified key by 1. If the key does not exist, its initial value is 0 and its value is 1 after incr. If the value of value cannot be converted to an integer, such as “Hello”, the operation fails and an error message is returned.
Decr key: decays the atomicity of the value of the specified key by 1. If the key does not exist, its initial value is 0 and its value is -1 after incR. If the value of value cannot be converted to an integer, such as “Hello”, the operation fails and an error message is returned.
Incrby key increment: Increments the value of the specified key atomically. If the key does not exist, the initial value is 0 and the value is increment after incrby. If the value cannot be converted to an integer, such as Hello, it fails with an error message.
Decrby Key Decrement: Decreases the atomicity of the value of a specified key. If the key does not exist, the initial value is 0 and, after decrby, decrement. If the value cannot be converted to an integer, such as Hello, it fails with an error message.
Append Key Value: If the key exists, the value is appended to the original value (concatenating substrings). If the key does not exist, create a new key/value
To store the hash
The Hashes type in Redis can be regarded as a map container with String keys and String values. So this type is very good for storing information about value objects. For example, Username, Password, and Age. If the Hash contains very few fields, that type of data will also take up very little disk space. Each Hash can store 4,294,967,295 key-value pairs.
Hset Key field Value: Sets the field/value pair (key-value pair) for the specified key.
Hgetall key: all filed-vaules in the key are obtained.
Hget key field: Returns the value of the field in the specified key.
Hmset Key Fields: Sets multiple filed/ values in a key.
Hmget key fileds: Obtains the filed values of the key.
Hdel key filed: Deletes the specified filed of the key
Del key: deletes the hash value of the key
Hincrby Key field increment: Increments the filed value of the key
Hexists Key Field: checks whether filed of the specified key exists
Hlen key: Gets the number of fields that the key contains
Hkeys key: Gets all the fieldName values in the key
Hvals key: Retrieves all fieldValues in the key
Store list
In Redis, the List type is a linked List of strings sorted by insertion order. Like a normal linked list in a data structure, we can add new elements to its head (left) and tail (right). At insert time, if the key does not exist, Redis creates a new linked list for that key. Conversely, if all the elements in the linked list are removed, the key will also be removed from the database. The maximum number of elements that a List can contain is 4294967295. From an efficiency point of view, if we were to insert or delete elements at both ends of the list, it would be very efficient and could be done in constant time even if there were millions of records stored in the list. It should be noted, however, that inserting or deleting elements in the middle of the list can be very inefficient. For developers with a good foundation of data structures, this should not be hard to understand.
Lpush key value1 value2… : Inserts all values into the head of the list associated with the specified key. If the key does not exist, this command creates an empty list associated with the key before inserting it, and then inserts data into the head of the list. Insert successfully, return the number of elements. [Note: If ipush key value1 value2 is inserted one by one from the left, value2 value1 is inserted.]
Rpush key value1 value2… : Adds elements to the end of the list
Lrange key start end: Obtain the value of the elements in the list from start to end (starting from 0). Start and end can be negative, -1 indicates the element at the end of the list, -2 indicates the penultimate element, and so on.
Lpushx key Value: Inserts a value into the header of the list associated with the specified key only if the specified key exists in the argument (the key will not be created if there is no value in the list managed with the key)
Rpushx key value: Adds elements to the end of the list
Lpop Key: Returns and pops the first element in the linked list associated with the specified key, the header element
Rpop Key: Pops elements from the tail
Rpoplpush resource destination: Pops up the tail element of the resource collection and adds it to the header of the Destination collection
Llen key: Returns the number of elements in the linked list associated with the specified key
Lrem key count value: delete all elements whose value is value. If count is greater than 0, delete all elements whose value is value. If count is less than 0, it is traversed from tail to head and deleted. If count is equal to 0, all elements in the list equal to value are deleted.
Lset key Index value: Sets the value (equivalent to substitution) of the index in the list. 0 represents the first element of the list and -1 represents the last element of the list.
Linsert key before | after the pivot value: in the pivot element inserted before or after the value of this element.
Store the set
In Redis, we can think of a Set as an unsorted collection of characters. Like a List, we can add, remove, or determine the existence of an element to its data values. It should be noted that the time of these operations is constant time. The maximum number of elements that a Set can contain is 4294967295. Unlike the List type, no duplicate elements are allowed in a Set. One of the most important features of the Set type compared to the List type is that it performs aggregating operations, such as unions, Intersections, and differences, between Sets on the server side. Because these operations are performed on the server, they are highly efficient and save a lot of NETWORK IO overhead.
Sadd key value1 value2… : Adds data to the set. If the key already has a value, it is not added again
Smembers key: Gets all the members in a set
Srem key is member2… : Deletes the member specified in set
Sismember Key Member: Checks whether the specified member is in the set. 1 indicates that the member exists, 0 indicates that the member does not exist, or the key does not exist
Scard key: Gets the number of members in the set
Sdiff key1 key2: Returns the members of key1 that differ from key2 (key1-key2), depending on the order of keys. That is, return the difference set
Sdiffstore Destination key1 Key2: Stores the different members of key1 and key2 on the destination
Sinter key1 key2: Returns the intersection of key1 and key2
Sinterstore Destination key1 Key2: Stores the returned intersection on destination
Sunion key1 key2: returns the union
Sunionstore Destination key1 Key2: Stores the returned union at destination
Srandmember Key: Returns a random member of the set
Storage sortedset
Sorted-Sets are very similar to Sets in that they are collections of strings and do not allow duplicate members to appear in a Set. The main difference between them is that each member of a Sorted- set is associated with a score, which Redis uses to sort the members of the set from smallest to largest. However, it is important to note that although the members in the Sorted-Sets must be unique, the score is repeatable.
Adding, deleting, or updating a member in a Sorted-Set is a very fast operation, with a time complexity of the logarithm of the number of members in the Set. Because the members in the Sorted-Sets are ordered in the collection, it is still very efficient to access even members in the middle of the collection. In fact, this feature of Redis is difficult to implement in many other types of databases. In other words, it is very difficult to model in other databases to achieve the same efficiency as Redis at this point.
Zadd Key score Member score2 member2… : stores all members and their weight scores in a sorted-set
Zrange key start end [withscores] : retrieves a member of the set with start-end. The [withscores] parameter indicates that the returned member contains its scores.
Zrevrange key start end [withscores] : Retrieves the members of the set with start-end in reverse order. The [withscores] parameter indicates that the returned members contain their scores.
Zrangebyscore key min Max [withscores][limit offset count] : returns the members whose scores are in [min, Max] and sorts them byscore from lowest to highest. [withscores] : displays scores; [limit offset count] : offset, starting from the element with offset and returning count members
Zcount key min Max: Gets the number of members whose scores are between [min, Max]
Zcard key: Gets the number of members in the collection
Zscore key member: Returns the score of a specified member
Zrem key member/member… : Removes a specified member from the collection. You can specify multiple members
Zremrangebyrank key start end: Remove elements byrank range (from small to large)
Zremrangebyscore key min Max: Delete elements byscore (rank from smallest to largest)
Zincrby key Increment Member zincrby Key increment Member Zincrby Key increment Member Zincrby Key increment Member zincrby Key increment Member
Zrank key member: Returns the position of the member in the collection
Zrevrank key Member: Returns the position of the member in the collection in reverse order
A generic operation of keys
Keys pattern: Gets all keys that match the pattern, but all keys that match the key. * indicates any one or more characters. Represents any character
Del key1 key2… : Deletes the specified key
Exists key: checks whether the key exists. 1 indicates that the key exists and 0 indicates that the key does not exist
Rename Key newkey: Renames the current key
Expire key time: sets the expiration time of the key, in seconds
TTL key: obtains the remaining timeout period of the key. If the timeout is not set, -1 is returned; if -2 is returned, it indicates that the key has exceeded the TTL (it does not exist); if the key still exists, the remaining time is returned
Type key: Gets the type of the specified key. This command is returned as a string. The returned strings are string, list, set, hash, and zset. Return None if key does not exist
Maven coordinates
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
Copy the codeConfiguration file parameters
#8 library 16 pieces
spring.redis.database=8
spring.redis.host=localhost
spring.redis.port=6379
Copy the codeConfiguration class writing
@Configuration
public class RedisConfig {
@Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(factory);
// Set the serialization of the key
template.setKeySerializer(RedisSerializer.string());
// Set the serialization method for value
template.setValueSerializer(RedisSerializer.json());
// Set the hash key serialization method
template.setHashKeySerializer(RedisSerializer.string());
// Set the hash value serialization method
template.setHashValueSerializer(RedisSerializer.json());
template.afterPropertiesSet();
returntemplate; }}Copy the codeExample of the like function
RedisKeyUtil is the utility class that generates redisKey
@Service
public class LikeService {
@Autowired
private RedisTemplate redisTemplate;
/ / thumb up
public void like(int userId, int entityType, int entityId, int entityUserId) {
redisTemplate.execute(new SessionCallback() {
@Override
public Object execute(RedisOperations operations) throws DataAccessException {
String entityLikeKey = RedisKeyUtil.getEntityLikeKey(entityType, entityId);
String userLikeKey = RedisKeyUtil.getUserLikeKey(entityUserId);
boolean isMember = operations.opsForSet().isMember(entityLikeKey, userId);
operations.multi();
if (isMember) {
operations.opsForSet().remove(entityLikeKey, userId);
operations.opsForValue().decrement(userLikeKey);
} else {
operations.opsForSet().add(entityLikeKey, userId);
operations.opsForValue().increment(userLikeKey);
}
returnoperations.exec(); }}); }// Query the number of likes for an entity
public long findEntityLikeCount(int entityType, int entityId) {
String entityLikeKey = RedisKeyUtil.getEntityLikeKey(entityType, entityId);
return redisTemplate.opsForSet().size(entityLikeKey);
}
// Query the "like" status of an entity
public int findEntityLikeStatus(int userId, int entityType, int entityId) {
String entityLikeKey = RedisKeyUtil.getEntityLikeKey(entityType, entityId);
return redisTemplate.opsForSet().isMember(entityLikeKey, userId) ? 1 : 0;
}
// Query the likes of a user
public int findUserLikeCount(int userId) {
String userLikeKey = RedisKeyUtil.getUserLikeKey(userId);
Integer count = (Integer) redisTemplate.opsForValue().get(userLikeKey);
return count == null ? 0: count.intValue(); }}Copy the codeFocus on Feature Examples
The CommunityConstant interface defines the common constants required by the project.
@Service
public class FollowService implements CommunityConstant {
@Autowired
private RedisTemplate redisTemplate;
@Autowired
private UserService userService;
public void follow(int userId, int entityType, int entityId) {
redisTemplate.execute(new SessionCallback() {
@Override
public Object execute(RedisOperations operations) throws DataAccessException {
String followeeKey = RedisKeyUtil.getFolloweeKey(userId, entityType);
String followerKey = RedisKeyUtil.getFollowerKey(entityType, entityId);
operations.multi();
operations.opsForZSet().add(followeeKey, entityId, System.currentTimeMillis());
operations.opsForZSet().add(followerKey, userId, System.currentTimeMillis());
returnoperations.exec(); }}); }public void unfollow(int userId, int entityType, int entityId) {
redisTemplate.execute(new SessionCallback() {
@Override
public Object execute(RedisOperations operations) throws DataAccessException {
String followeeKey = RedisKeyUtil.getFolloweeKey(userId, entityType);
String followerKey = RedisKeyUtil.getFollowerKey(entityType, entityId);
operations.multi();
operations.opsForZSet().remove(followeeKey, entityId);
operations.opsForZSet().remove(followerKey, userId);
returnoperations.exec(); }}); }// Query the number of concerned entities
public long findFolloweeCount(int userId, int entityType) {
String followeeKey = RedisKeyUtil.getFolloweeKey(userId, entityType);
return redisTemplate.opsForZSet().zCard(followeeKey);
}
// Query the number of fans of the entity
public long findFollowerCount(int entityType, int entityId) {
String followerKey = RedisKeyUtil.getFollowerKey(entityType, entityId);
return redisTemplate.opsForZSet().zCard(followerKey);
}
// Queries whether the current user has followed the entity
public boolean hasFollowed(int userId, int entityType, int entityId) {
String followeeKey = RedisKeyUtil.getFolloweeKey(userId, entityType);
returnredisTemplate.opsForZSet().score(followeeKey, entityId) ! =null;
}
// Query the person that a user follows
public List<Map<String, Object>> findFollowees(int userId, int offset, int limit) {
String followeeKey = RedisKeyUtil.getFolloweeKey(userId, ENTITY_TYPE_USER);
Set<Integer> targetIds = redisTemplate.opsForZSet().reverseRange(followeeKey, offset, offset + limit - 1);
if (targetIds == null) {
return null;
}
List<Map<String, Object>> list = new ArrayList<>();
for (Integer targetId : targetIds) {
Map<String, Object> map = new HashMap<>();
User user = userService.findUserById(targetId);
map.put("user", user);
Double score = redisTemplate.opsForZSet().score(followeeKey, targetId);
map.put("followTime".new Date(score.longValue()));
list.add(map);
}
return list;
}
// Query a user's fans
public List<Map<String, Object>> findFollowers(int userId, int offset, int limit) {
String followerKey = RedisKeyUtil.getFollowerKey(ENTITY_TYPE_USER, userId);
Set<Integer> targetIds = redisTemplate.opsForZSet().reverseRange(followerKey, offset, offset + limit - 1);
if (targetIds == null) {
return null;
}
List<Map<String, Object>> list = new ArrayList<>();
for (Integer targetId : targetIds) {
Map<String, Object> map = new HashMap<>();
User user = userService.findUserById(targetId);
map.put("user", user);
Double score = redisTemplate.opsForZSet().score(followerKey, targetId);
map.put("followTime".new Date(score.longValue()));
list.add(map);
}
returnlist; }}Copy the codeAdvanced data type -HyperLogLog
// Count the independent total of 200,000 duplicate data.
@Test
public void testHyperLogLog(a) {
String redisKey = "test:hll:01";
for (int i = 1; i <= 100000; i++) {
redisTemplate.opsForHyperLogLog().add(redisKey, i);
}
for (int i = 1; i <= 100000; i++) {
int r = (int) (Math.random() * 100000 + 1);
redisTemplate.opsForHyperLogLog().add(redisKey, r);
}
long size = redisTemplate.opsForHyperLogLog().size(redisKey);
System.out.println(size);// Result: 99553
}
// Merge the three groups of data, then count the independent total number of duplicate data after merging.
@Test
public void testHyperLogLogUnion(a) {
String redisKey2 = "test:hll:02";
for (int i = 1; i <= 10000; i++) {
redisTemplate.opsForHyperLogLog().add(redisKey2, i);
}
String redisKey3 = "test:hll:03";
for (int i = 5001; i <= 15000; i++) {
redisTemplate.opsForHyperLogLog().add(redisKey3, i);
}
String redisKey4 = "test:hll:04";
for (int i = 10001; i <= 20000; i++) {
redisTemplate.opsForHyperLogLog().add(redisKey4, i);
}
String unionKey = "test:hll:union";
redisTemplate.opsForHyperLogLog().union(unionKey, redisKey2, redisKey3, redisKey4);
long size = redisTemplate.opsForHyperLogLog().size(unionKey);
System.out.println(size); // Result: 19833
}
Copy the codeAdvanced data type -bitmap
// Count the Boolean values of a set of data
@Test
public void testBitMap(a) {
String redisKey = "test:bm:01";
/ / record
redisTemplate.opsForValue().setBit(redisKey, 1.true);
redisTemplate.opsForValue().setBit(redisKey, 4.true);
redisTemplate.opsForValue().setBit(redisKey, 7.true);
/ / query
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 0)); //false
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 1)); //true
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 2)); //false
/ / statistics
Object obj = redisTemplate.execute(new RedisCallback() {
@Override
public Object doInRedis(RedisConnection connection) throws DataAccessException {
returnconnection.bitCount(redisKey.getBytes()); }}); System.out.println(obj);/ / 3
}
// Count the Boolean values of 3 groups of data, and perform the OR operation on these 3 groups of data.
@Test
public void testBitMapOperation(a) {
String redisKey2 = "test:bm:02";
redisTemplate.opsForValue().setBit(redisKey2, 0.true);
redisTemplate.opsForValue().setBit(redisKey2, 1.true);
redisTemplate.opsForValue().setBit(redisKey2, 2.true);
String redisKey3 = "test:bm:03";
redisTemplate.opsForValue().setBit(redisKey3, 2.true);
redisTemplate.opsForValue().setBit(redisKey3, 3.true);
redisTemplate.opsForValue().setBit(redisKey3, 4.true);
String redisKey4 = "test:bm:04";
redisTemplate.opsForValue().setBit(redisKey4, 4.true);
redisTemplate.opsForValue().setBit(redisKey4, 5.true);
redisTemplate.opsForValue().setBit(redisKey4, 6.true);
String redisKey = "test:bm:or";
Object obj = redisTemplate.execute(new RedisCallback() {
@Override
public Object doInRedis(RedisConnection connection) throws DataAccessException {
connection.bitOp(RedisStringCommands.BitOperation.OR,
redisKey.getBytes(), redisKey2.getBytes(), redisKey3.getBytes(), redisKey4.getBytes());
returnconnection.bitCount(redisKey.getBytes()); }}); System.out.println(obj);/ / 7
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 0)); / / is all true
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 1));
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 2));
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 3));
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 4));
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 5));
System.out.println(redisTemplate.opsForValue().getBit(redisKey, 6));
}
Copy the codeUV(Unique Visitors) DAU(Daily Active Users)
service
@Service
public class DataService {
@Autowired
private RedisTemplate redisTemplate;
private SimpleDateFormat df = new SimpleDateFormat("yyyyMMdd");
// Count the specified IP into the UV
public void recordUV(String ip) {
String redisKey = RedisKeyUtil.getUVKey(df.format(new Date()));
redisTemplate.opsForHyperLogLog().add(redisKey, ip);
}
// Count uVs within the specified date range
public long calculateUV(Date start, Date end) {
if (start == null || end == null) {
throw new IllegalArgumentException("Parameters cannot be empty!");
}
// Collate keys in the date range
List<String> keyList = new ArrayList<>();
Calendar calendar = Calendar.getInstance();
calendar.setTime(start);
while(! calendar.getTime().after(end)) { String key = RedisKeyUtil.getUVKey(df.format(calendar.getTime())); keyList.add(key); calendar.add(Calendar.DATE,1);
}
// Merge the data
String redisKey = RedisKeyUtil.getUVKey(df.format(start), df.format(end));
redisTemplate.opsForHyperLogLog().union(redisKey, keyList.toArray());
// Return the result of the statistics
return redisTemplate.opsForHyperLogLog().size(redisKey);
}
// Count specified users as DAU
public void recordDAU(int userId) {
String redisKey = RedisKeyUtil.getDAUKey(df.format(new Date()));
redisTemplate.opsForValue().setBit(redisKey, userId, true);
}
// Count the DAU within the specified date range
public long calculateDAU(Date start, Date end) {
if (start == null || end == null) {
throw new IllegalArgumentException("Parameters cannot be empty!");
}
// Collate keys in the date range
List<byte[]> keyList = new ArrayList<>();
Calendar calendar = Calendar.getInstance();
calendar.setTime(start);
while(! calendar.getTime().after(end)) { String key = RedisKeyUtil.getDAUKey(df.format(calendar.getTime())); keyList.add(key.getBytes()); calendar.add(Calendar.DATE,1);
}
// Perform the OR operation
return (long) redisTemplate.execute(new RedisCallback() {
@Override
public Object doInRedis(RedisConnection connection) throws DataAccessException {
String redisKey = RedisKeyUtil.getDAUKey(df.format(start), df.format(end));
connection.bitOp(RedisStringCommands.BitOperation.OR,
redisKey.getBytes(), keyList.toArray(new byte[0] [0]));
returnconnection.bitCount(redisKey.getBytes()); }}); }}Copy the codeThe interceptor
@Component
public class DataInterceptor implements HandlerInterceptor {
@Autowired
private DataService dataService;
@Autowired
private HostHolder hostHolder;
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
The UV / / statistics
String ip = request.getRemoteHost();
dataService.recordUV(ip);
/ / statistical DAU
User user = hostHolder.getUser();
if(user ! =null) {
dataService.recordDAU(user.getId());
}
return true; }}Copy the codecontroller
@Controller
public class DataController {
@Autowired
private DataService dataService;
// Statistics page
@RequestMapping(path = "/data", method = {RequestMethod.GET, RequestMethod.POST})
public String getDataPage(a) {
return "/site/admin/data";
}
// Statistics website UV
@RequestMapping(path = "/data/uv", method = RequestMethod.POST)
public String getUV(@DateTimeFormat(pattern = "yyyy-MM-dd") Date start,
@DateTimeFormat(pattern = "yyyy-MM-dd") Date end, Model model) {
long uv = dataService.calculateUV(start, end);
model.addAttribute("uvResult", uv);
model.addAttribute("uvStartDate", start);
model.addAttribute("uvEndDate", end);
return "forward:/data";
}
// Count active users
@RequestMapping(path = "/data/dau", method = RequestMethod.POST)
public String getDAU(@DateTimeFormat(pattern = "yyyy-MM-dd") Date start,
@DateTimeFormat(pattern = "yyyy-MM-dd") Date end, Model model) {
long dau = dataService.calculateDAU(start, end);
model.addAttribute("dauResult", dau);
model.addAttribute("dauStartDate", start);
model.addAttribute("dauEndDate", end);
return "forward:/data"; }}Copy the codeRedis- Expiration policy
Redis puts keys with expiration dates into a separate dictionary and does not delete them immediately when they expire. Redis removes expired keys using the following two strategies:
-
Lazy to delete
When a client accesses a key, Redis checks whether the key is expired and deletes it.
-
Regularly scan
By default, Redis performs 10 expired scans per second (with the Hz option configured). The scan policy is as follows:
1. Randomly select 20 keys from the expired dictionary;
2. Delete expired keys from the 20 keys.
3. If more than 25% of keys are out of date, repeat Step 1.
Redis- Elimination strategy
When Red exceeds the maxmemory limit, Redis can use the maxmemory-policy to flush out some data so that it can continue to provide read and write services:
Noeviction returns bugs for commands that may cause memory enlargement (most write commands except DEL)
Volatile – TTL: Selects the key with the shortest remaining TTL from the key whose expiration time is set
Volatile – LRU: Selects the least used key (LRU) among keys with expiration time
Volatile -random: Randomly selects some keys with expiration time to discard them
Allkeys-lru: out of allkeys, select the least used key (lru) and discard it
Allkeys-random: select some keys randomly from allkeys and eliminate them
LRU algorithm
Maintains a linked list for storing the keys that have been accessed sequentially. When data is accessed, the most recently accessed key is moved to the top of the table, that is, the most recently accessed key is in the top of the table, and the least accessed key is in the bottom of the table.
Approximate LRU algorithm (Redis)
Maintain a timestamp for each key, and randomly sample 5 keys during elimination to eliminate the oldest key. If the memory limit is still exceeded, random sampling is continued.
Advantages: save memory than LRu algorithm, but can achieve very similar effect.
Redis- Cache penetration
scenario
Queries for non-existent data drive requests straight to the storage tier, resulting in overloads and even downtime.
The solution
- 1. Cache empty objects
Null values are still stored in the cache tier after a storage tier miss. When the data is accessed again, the cache layer returns a null value.
- 2, Bloom filter
All existing keys are stored in the Bloom filter in advance, which is intercepted by the filter before accessing the cache layer. If a non-existing key is requested, the null value is directly returned.
Redis- Cache breakdown
scenario
A hot piece of data. It gets a lot of traffic. At the moment its cache fails, a large number of requests are directed to the storage tier, causing the service to crash.
The solution
- 1. Add mutex
Access to data is mutex, and while one thread accesses the data, the other threads have to wait. After this thread accesses the cache, the data in the cache is rebuilt so that other threads can take values directly from the cache.
- 2. Never expire
No expiration time is set, so the above problems do not occur, this is “physical” expiration. A logic expiration time is set for each value, and a separate thread is used to rebuild the cache when the value logic expires.
Redis- Cache avalanche
scenario
For some reason, the cache layer cannot provide services. As a result, all requests are directed to the storage layer, causing the storage layer to break down.
The solution
- 1. Avoid expiration at the same time
When setting the expiration time, attach a random number to prevent a large number of keys from expiring at the same time.
- 2. Build a highly available Redis cache
Multiple Redis instances can be deployed and individual nodes go down, but the overall availability of the service can still be maintained.
- 3. Build a multi-level cache
Increase local caching and put one more barrier in front of the storage tier to reduce the chance of requests going straight to the storage tier.
- 4. Enable traffic limiting and degradation measures
Add traffic limiting measures to the storage tier, and degrade the request when the request exceeds the limit.
Redis- Distributed lock
scenario
Data is often read into memory, modified in memory, and then saved back. In distributed applications, it is possible for multiple processes to perform the above operations simultaneously while reading and modifying non-atomic operations, resulting in conflicts. Adding distributed locks can solve this problem.
The basic principle of
Synchronization lock: A marker that indicates access to data in a place that can be accessed by multiple threads. Distributed lock: A marker that identifies access to data in a place that can be accessed by multiple processes.
implementation
1. Distributed lock based on database;
2. Distributed lock based on Redis;
3. Distributed lock based on ZooKeeper;
Redis implements the principle of distributed locking
1. Security attributes: Exclusive. Only one client holds the lock at any one time.
2. Activity A: No deadlocks. The lock can be acquired even if the client holding the lock crashes or the network is split.
3. Activity B: fault tolerant. As long as most of the Redis nodes are alive, the client can acquire and release locks.
A single Redis instance implements distributed locking
1. Obtain the lock using the following command:
SET resource_name my_random_value NX PX 3000oNX: Execute successfully only if the key does not exist. Px: Set the lock automatic expiration time.
2. Release lock via Lua script:
if redis.call ( "get",KEYS[1])== ARGV [1] then
return redis.call ( "del",KEYS [1])
else return o end
Copy the codeYou can avoid deleting the lock obtained successfully by other clients: LOCK A -> block A -> release lock due to timeout -> lock B -> Restore A -> release lock
Multiple Redis instances implement distributed locking
Redlock algorithm, the algorithm has a ready-made implementation, its Java version of the library for Redisson.
1. Gets the current Unix time in milliseconds.
2. Try, in turn, to get the lock from n instances with the same key and random value, and set the response timeout. If the server does not respond within the specified time, the client should try another Redis instance as soon as possible.
3. The client obtains the lock usage time by subtracting the current time from the time it started acquiring the lock. The lock is successful if and only if most of the Redis nodes have picked up the lock for a shorter time than the lock expires.
4. If a lock is obtained, the key’s true validity time is equal to the validity time minus the time used to acquire the lock.
5. If the lock fails to be obtained, the client should unlock it on all instances of Redis.