preface

As mentioned in the previous article, I did not write about Redis series recently, but because I was struggling with the choiceSpingorNetty. All of a sudden, I remembered that there was still something related to this article, which was also regarded as a transitional period. To be reasonable, this article is very necessary to understand and understand whether it is an interview or a trial of the actual work, but to understand this article needs to understand a lot of knowledge points I used to do, if good brothers in the process of reading some unfamiliar points can be readRedis advanced advanced knowledge points, basically all the Reids stuff. Give beefcake a like and follow something if you like.

Ordinary lock

Above all the lock is to be placed on the implement that can open close, open with key or secret code, lock like door lock, password. From our daily life, normal every door will have a lock (not the normal bar), if there is no configuration of the lock, what kind of people can go in (world datong is this case). This is obviously not safe, so the lock is introduced here.

So what does a lock look like in a program? In programs, locks typically occur in multithreaded environments. When multiple threads access the same shared resource, a mechanism is needed to ensure that only the thread that meets a certain condition (lock acquisition success) can access the resource, while the thread that does not meet the condition (lock acquisition failure) can only wait until the next round of contention to obtain the lock to access the resource. So locks are a tool to solve the problem of errors or data inconsistencies when multiple threads of execution access a shared resource.

implementation

It should be noted here that all of the implementation methods described below are implemented in Java, and of course there will be no principle analysis of the implementation method (later can be done).

  1. usesynchronizedThe keyword.synchronizedIs an implicit lock because it is unlocked and locked automatically by the JVM through the object’s Monitor lock, and we have no access to the entire lock and unlock process.
  2. usejava.util.concurrent.locks.LockFor examplejava.util.concurrent.locks.ReentrantLock. This way is to show the lock, need to manually lock and unlock.
  3. useCAS(Compare And Swap) No locking mechanism, this is actually optimistic locking mechanism, the core concept is to Compare And Swap. In the implementationCASThe value of the memory location is compared to the expected value. If it matches, the processor automatically updates the value to the new value. Otherwise, the processor does nothing.

A distributed lock

First, in a distributed, microservice architecture, our projects are typically deployed in the following pattern, of courseThe gateway,nginxThere are also clusters.

As you can see in the picture above,Variable AExists in three server memory (this oneVariable AA member variable in a class is a stateful object. Assuming we are using a normal lock, since variables are not visible between the three servers, when multiple requests are invoked, it is possible to manipulate data from three different memory regions and read data that is inconsistent with the set values.

Based on these problems, the concept of a distributed lock (the author of this name is unknown) emerged. To put it simply, we add an extra step to request our interface, but this step requires an intermediate medium, such as Redis, ZooKeeper or even Mysql. This intermediary acts as a marker to mark who currently has the lock. The whole process is the request, first call to determine whether the intermediary exists, does not exist to try to obtain, obtain success or failure, and ordinary lock processing logic.

implementation

The implementation also mentioned above, the main need for an intermediate medium.

  1. Based on theRedisFor those not familiar with RedisYou don’t know what Redis isAnd that’s the topic of today.
  2. Based on the database, the general data have to achieve pessimistic lock, of course, can also use the version number or other optimistic lock to achieve, pessimistic lock is mainlyfor updateThe keyword
  3. Based on theZookeeperThe principle is to useZookeeperTo implement the temporary node.

It is recommended to use Zookeeper for distributed locking, mainly because zooKeeper-related features are ideal for distributed locking, which will be covered later. Of course, if Zookeeper is used in the project, otherwise building a Cluster of Zookeeper to implement a distributed lock is not worth the cost. Redis is different. Basically all projects will use it. There will be some problems when using Redis, so Zookeeper is recommended.

Redis implementation

uniqueness

The use of Redis to achieve lock uniqueness is mainly through the SETNX(SET if Not eXists) command, which sets the value of the specified key if it does Not exist. If the setting succeeds, 1 is returned. If the setting fails, 0 is returned.

Alas, this solves the above problem by marking that only one of multiple requests can be SET successfully by SETNX(SET if Not eXists). But yes, if the request fails, or the project crashes, will all subsequent threads fail? This leads to the second problem, how to prevent deadlocks.

The lock timeout

Deadlocks can be prevented in Redis by setting the timeout period for the key. To be fair, this timeout must be set. Redis also provides the expire command to set the expiration time for keys. So is that the solution?

The answer is no. Why? If you have read my series of articles on Redis, you will know that Reids are single-threaded models. I will not go into details about this. There is a table of contents in this article, just look at the articles in it. Due to this model, the execution of the entire Redis command is put into the queue, and then queue execution. This means that when setnx and EXPIRE are executed, there will be two request and response steps, which will cause atomicity problems. For example, after setting setNx, the program directly crashes, which is back to the previous problem. So how do you solve the principle problem of the two commands? Good brothers look down.

Command atomicity

The easiest way to solve this problem is to combine two commands into one. Does Redis support this mode? The answer is yes, there are three ways to do this: Pipeline, Redis transactions, and Lua scripts. However, both Pipeline and Redis transactions are not atomic, so we have to use Lua scripts. If you are not familiar with how Redis uses Lua scripts, read the above article first.

Here’s a locked Lua, here’s actually a string.

"if (redis.call('exists', KEYS[1]) == 0) then " +
                        "redis.call('hincrby', KEYS[1], ARGV[2], 1); " +
                        "redis.call('pexpire', KEYS[1], ARGV[1]); " +
                        "return nil; " +
                        "end; " +
                        "if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then " +
                        "redis.call('hincrby', KEYS[1], ARGV[2], 1); " +
                        "redis.call('pexpire', KEYS[1], ARGV[1]); " +
                        "return nil; " +
                        "end; " +
                        "return redis.call('pttl', KEYS[1]);"
Copy the code

Basically, using Lua scripts to ensure atomicity of multiple commands. Would that be perfect if the deadlock was solved? Not as perfect as imagined, good brothers want such a problem, if the timeout time for the lock is 5 seconds, but we do logical operations, may be due to the need to call someone else’s interface or query database slow query more than 5 seconds, normal logic is the current thread still need to hold the lock. Then when the lock expires, other threads can compete for the lock. Is there another thread unsafe problem?

Analysis of Redisson’s principle

Redisson is needed to solve the problem of lock expiration before the above logic is completed. In fact, Redis does not provide support for this part because it cannot be done. This can only be done by writing the corresponding business logic to the application, which Redisson has already integrated into issues like lock timeouts, uniqueness, and atomicity. If we choose to use Redis to do distributed lock solution, we recommend to use this framework, otherwise we need to consider a lot of concurrent problems in distributed environment.

The principle of

The wholeRedissonThe locking process is as follows:

The whole process without words to express, basically look at the picture can understand. Here are the points that need to be paid attention to as follows:

  1. Locking mechanism bath towel mentioned aboveSETNXInstructions andLuaScript implementation
  2. watch dogThe automatic delay mechanism, which starts a background thread, timing for example10Seconds checks if the current thread still holds the lock, and suspends if it does. This is to solve the above logic is not complete and the lock has expired. However, it is not recommended to enable this function, which is very expensive in resources and performance.
  3. ReentrantRedissonThe data type of the storage lock isHashType, andHashData typedkeyValue contains information about the current thread.
  4. Mutual exclusion is guaranteed by Redis data structures to ensure the uniqueness of distributed locks.
  5. Deadlocks are avoided through Redis pairskeySet a timeout to ensure that.

Existing problems

Yeah, it’s not over here. Redisson is the implementation of distributed locks will still exist in the single point/multi-point, here is mainly about the deployment mode. When using a single point Redis is down, there is nothing to say about it, it can’t lock directly, no matter which medium is used to solve this problem.

The sentry and Cluster clusters are the most common ones. If client 1 writes a Redisson lock to a master node, the lock is asynchronously replicated to the slave node. However, once the master node breaks down and the master/slave switchover occurs, the slave node becomes the master node. When client 2 attempts to lock the distributed lock, the master node on client 1 is successfully locked, and client 2 can also lock the new master node. As a result, multiple clients complete locking the distributed lock.

The solution

  1. Do not solve, do not hit me (manual dog head), do not solve me to join him. This is really not easy to do, did not think of the right way to deal with.
  2. withZookeeperMake it happen, get rid of it (it’s used by men who cheat on women’s feelings). This is what I recommendZookeeperTo achieve distributed lock reason, good brothers are not also found that the use of Redis to achieve distributed lock is really complicated, all kinds of problems are too many.

Application scenarios

  1. Avoid repeated execution of a piece of logic by different nodes, such as our timed task, which may be executed multiple times at a certain point in time without locking.
  2. Prevent forms from being submitted repeatedly, which can also be problematic.
  3. Avoid damaging data correctness. When multiple threads are working on a piece of logic at the same time, data may be corrupted or inconsistent.

That’s it for this issue. Please leave your comments in the comments sectionAsk for attention, ask for likes