In Java, synchronized keyword and ReentrantLock are commonly used to control concurrent access to resources in a multithreaded environment. However, with the rapid development of distribution, Local locking often fails to meet our needs, and in our distributed environment the above locking method will not work. In order to realize the effect of local lock in distributed environment, people put forward the concept of distributed lock.

A distributed lock

Distributed lock scenario

Generally, distributed locks are required in the following scenarios:

  • Efficiency: Using distributed locks can avoid repeating the same work of different nodes, such as avoiding repeated execution of scheduled tasks.
  • Correctness: Using distributed locks also prevents data correctness from being corrupted. Concurrency problems may occur if two nodes operate on the same piece of data.

Distributed lock features

A well-developed distributed lock needs to meet the following characteristics:

  • Mutual exclusion: Mutual exclusion is a basic feature. Distributed locks require mutual exclusion at the thread or node level as required. ;
  • Reentrancy: the lock acquired by the same node or thread can be reentrant to acquire the lock.
  • Lock timeout: Supports lock timeout release, preventing lock cannot be released after a node becomes unavailable.
  • High efficiency: lock and unlock efficiency is high, can support high concurrency;
  • High availability: A high availability mechanism is required to prevent the lock service from being unavailable, such as increased degradation;
  • Block: support blocking lock acquisition and non-blocking lock acquisition two ways;
  • Fair: Supports fair locks and unfair locks. Fair locks ensure that locks are obtained in the sequence of lock requests, while non-fair locks do not.

Implementation of distributed lock

There are three common implementations of distributed locks, and we will introduce them one by one below:

  • Distributed lock based on database;
  • Distributed lock based on Redis;
  • Distributed lock based on Zookeeper.

Database based distributed locking

The distributed lock based on database can be implemented in different ways. This paper will introduce a database non-blocking distributed lock implemented by the author in the actual production.

Project overview

We listed above the characteristics of distributed lock needs to meet, using the database to achieve distributed lock also needs to meet these characteristics, let’s introduce the implementation method one by one:

  • Mutual exclusion: Mutual exclusion between two locks acquired through atomicity of database update;
  • Reentrancy: Retain a field in the database to store the holder of the current lock
  • Lock timeout: the lock acquisition time and timeout duration are stored in the database.
  • High efficiency: The database itself can support relatively high concurrency;
  • High availability: Add primary and secondary database logic to improve database availability.
  • Blocking: thread blocking can be realized by the way of watchdog polling;
  • Fairness: You can add lock queues, but it is not recommended. It is complicated to implement.

Table structure design

The name of the database table is LOCK, and the definitions of each field are as follows:

Field name Name The field type instructions
lock_key varchar Unique identifier of a lock
lock_time timestample Locking time
lock_duration integer Timeout period of the lock. The unit can be customized, usually in seconds
lock_owner varchar The owner of a lock can be the unique identity of a node or thread. Different reentrant granularity locks have different meanings
locked boolean Whether the current lock is occupied

SQL statement to get the lock

If the lock does not exist, then the lock needs to be created first, and the thread that created the lock can acquire the lock:

insert into lock(lock_key,lock_time,lock_duration,lock_owner,locked) values ('xxx',now(),1000.'ownerxxx'.true)
Copy the code

If the lock already exists, an attempt is made to update the information about the lock. If the update succeeds, the lock is successfully acquired, and if the update fails, the lock is not acquired.

update lock set 
    locked = true, 
    lock_owner = 'ownerxxxx', 
    lock_time = now(), 
    lock_duration = 1000
where
    lock_key='xxx' and(
    lock_owner = 'ownerxxxx' or
    locked = false or
    date_add(lock_time, interval lock_duration second) > now())
Copy the code

SQL statement to release the lock

When the user has finished using the lock and needs to release it, the locked identifier can be updated to false.

update lock set 
    locked = false.where
    lock_key='xxx' and
    lock_owner = 'ownerxxxx' and
    locked = true
Copy the code

Guard dog

With the above steps, we can acquire locks and release locks, but what does the watchdog do?

Imagine if the time between the user acquiring the lock and releasing it was longer than the lock timeout, wouldn’t that be a problem? Is it possible for multiple nodes to acquire locks at the same time? This is where a watchdog is needed, which can constantly refresh lock acquisition events through a scheduled task to keep the lock held from the time the user acquires it until it is released.

Distributed lock based on Redis

Redisson, the Java client of Redis, implements distributed locking, which can be implemented using lock-release logic similar to ReentrantLock.

RLock disLock = redissonClient.getLock("DISLOCK");
disLock.lock();
try {
    // Business logic
} finally {
    // Do you want to unlock it anyway
    disLock.unlock();
}
Copy the code

The underlying principle of Redisson distributed lock

The following diagram shows the logic of Redisson client locking and releasing locks:

Locking mechanism

As can be seen from the above figure, when the Redisson client needs to acquire the lock, it needs to send a Lua script to the Redis cluster for execution. Why use Lua script? Because a piece of complex business logic can be encapsulated in a Lua script and sent to Redis, the atomic execution of the complex business logic can be guaranteed.

Lua source code analysis: Following is the Lua source code for Redisson lock, we will analyze the source code.

The Lua script has three input parameters: KEYS[1], ARGV[1], and ARGV[2].

  • KEYS[1] represents the lock Key, for example RLock lock = “myLock” in redisson.getLock(“myLock”);
  • ARGV[1] represents the default lifetime of the lock Key, which is 30 seconds.
  • ARGV[2] represents the ID of the locked client, similar to the following: 8743C9c0-0795-4907-87FD-6C719a6b4586:1.

The Lua script and locking steps are shown in the following code block. The general principle is as follows:

  • If the lock does not exist, create the lock and set the expiration time.
  • Refresh the lock expiration event in a reentrant scenario when the lock exists.
  • Otherwise, the lock failure and lock expiration time are returned.
Check whether the lock exists
if (redis.call('exists', KEYS[1= =])0) then 
    Add locks and set client and initial lock re-entry times
    redis.call('hincrby', KEYS[1], ARGV[2].1); 
    -- Set the lock timeout event
    redis.call('pexpire', KEYS[1], ARGV[1]);  
    -- Returns lock success
    return nil;  
end;  
Determine whether the owner of the current lock is the requester of the lock
if (redis.call('hexists', KEYS[1], ARGV[2= =])1) then  
    -- The current lock is held by the requester, re-entrant the lock, increase the lock re-entrant times
    redis.call('hincrby', KEYS[1], ARGV[2].1);  
    -- Refresh the lock expiration time
    redis.call('pexpire', KEYS[1], ARGV[1]);  
    -- Returns lock success
    return nil;  
end;  
-- Returns the expiration time of the current lock
return redis.call('pttl', KEYS[1]);
Copy the code

Watchdog logic

The default lifetime of the lock Key on client 1 is 30 seconds. What if client 1 wants to keep the lock for longer than 30 seconds? As long as client 1 successfully locks, a watchdog watchdog will be started. This background thread will check every 10 seconds. If client 1 still holds the lock Key, it will continuously extend the lifetime of the lock Key.

Lock release mechanism

If lock.unlock() is executed to release a distributed lock, the business logic is very simple. That is, each time the lock in the myLock data structure is reduced by one.

If the lock count is 0, the client no longer holds the lock. In this case, the “del myLock” command is used to delete the Key from Redis.

The other client 2 can then attempt to complete the lock. This is the implementation mechanism of the open source Redisson framework for so-called distributed locks.

In general, we can use the class library provided by Redisson framework to add and release distributed locks based on Redis in the production system.

Defects in Redisson distributed locks

Redis distributed locks have a flaw in Redis sentinel mode:

  1. Client 1 writes a Redisson lock to a master node, which is asynchronously replicated to the slave node. However, once the master node breaks down and the master/slave switchover occurs, the slave node becomes the master node.
  2. When client 2 tries to lock, it can also lock on the new master node, causing multiple clients to complete the lock on the same distributed lock.
  3. The system is bound to have problems with business semantics, resulting in all kinds of dirty data.

This flaw can cause multiple clients to lock simultaneously in sentinel or master-slave mode if the master instance goes down.

Distributed lock based on Zookeeper

The distributed lock implemented by Zookeeper applies to the services introduced from Zookeeper. As shown in the following figure, two services are registered with Zookeeper and need to obtain the distributed lock from Zookeeper.

Step 1

Suppose client A gets ahead of ZK and initiates A distributed lock request using A special ZK concept called A “temporary order node.” In simple terms, it is directly under the lock” my_lock” node, create a sequence node, this sequence node has a node number internal maintenance ZK.

  • For example, the first client to fetch a sequential node, ZK internally generates the name XXX-000001.
  • The second client then retrieves a sequential node, which internally generates the name XXX-000002.

The last number is in ascending order, starting with 1. ZK will maintain this order. So client A initiates the request first, and A sequence node is generated as follows:

Client A initiates A lock request, and A temporary sequential node is generated under the node that locks first. Because client A initiates the request first, the last digit of the node name is “1”. After the sequence section is successfully created, client A queries all nodes under the lock and sorts them in ascending order to determine whether the current node is the first node. If the current node is the first node, the lock is successful.

Step 2

When client A is done locking and client B wants to lock, A temporary sequential node is created under the lock node. The last digit of the node name is “2”.

Client B will judge the locking logic, query all child nodes under the lock node, and arrange them in sequence. At this time, the first node is the sequence node created by client A, and the serial number is “01”. So the lock failed. When the lock fails, client B applies a listener to the previous sequence node of his sequence node through ZK’s API. ZK can naturally listen on a node.

Step 3

After client A locks, it may process some code logic and then release the lock. Zookeeper releases the lock by removing the sequence node Zk_random_000001 created by client A.

After the node on client A is deleted, Zookeeper is responsible for notifying the listener listening on this node, that is, before client B adds A listener. The listener of client B knows that the last sequential node was deleted, that is, one of the clients before him released the lock. At this point, client B will try again to obtain the lock, that is, to obtain the set of child nodes under the lock node and determine whether it is the first node to obtain the lock.

Advantages and disadvantages of the three locks

Database based distributed locks:

  • The database concurrency performance is poor;
  • The implementation of blocking lock is complicated.
  • Fair lock implementation is complicated.

Distributed lock based on Redis:

  • In the case of primary/secondary switchover, multiple clients may obtain the lock.
  • Lua scripts are atomic on a single machine, but not on master/slave synchronization.

Distributed lock based on Zookeeper:

  • The Zookeeper cluster needs to be introduced.
  • The reentrant granularity of distributed locks can only be node level.

Reference documentation

A distributed lock

Comparison of three distributed locks

Comparison of three implementations of distributed locking

I am the god of the royal Fox. Welcome to follow my wechat official account: Wzm2ZSD

This article was first published to wechat public account, all rights reserved, reprint prohibited!