This is the 17th day of my participation in the August Challenge

Today we’re going to talk about distributed locks.

I believe you are familiar with locking. In a multi-threaded environment, if you need to operate on the same resource, in order to avoid data inconsistency, you need to lock the shared resource before operating on it. In computer science, a lock or mutex is a synchronization mechanism used to enforce access restrictions on resources in an environment with many threads of execution.

For example, if you go on a blind date and find that you and your brother are dating a woman at the same time, how can you do that? I might even get beat up.

So what is distributed locking? When multiple clients need to compete for locks, we need distributed locks. This lock cannot be local to a client, otherwise other clients cannot access it. Therefore, distributed locks must be stored in shared storage systems, such as Redis and Zookeeper, and can be accessed and obtained by multiple clients. Today we will look at how to implement distributed locks using Redis.

One, foreword

SETNX key Value the name means that when a key is present, no assignment is performed. If the key does not exist, create the key and assign it to value, i.e. SET the key value [EX seconds | PX milliseconds] NX NX options, add after the SET is similar to SETNX command, also does not exist or SET of functions. In addition, when executing this command, you can set the expiration time of the key-value pair by EX or PX.

Second, the body

Before we begin, let’s introduce a scenario:

Suppose we want to hold a second kill for an item, we have stored the inventory data 100 in Redis beforehand, we now need to do the inventory deduction.

As shown in the figure, we assume that there are 1000 clients to do the inventory deduction operation. What can we do to ensure that the inventory deduction order is consistent and does not overdeduct?

* * * *

The first thing that comes to mind is locking. We take the lock, make the deduction, and release the lock before we make the inventory deduction. In Redis we create a key to represent a lock variable, and then the corresponding value to represent the value of the lock variable. Let’s take a look at locking.

Suppose 1000 clients make lock requests simultaneously. Because Redis uses a single thread to process requests, Redis executes their requests serially. Assume that Redis first processes the request from client 2, reads the lock_key value, and finds that the lock_key value is 0. Therefore, client 2 sets the value of lock_key to 1, indicating that the lock operation has been performed. If client 3 is processed at this point, the lock_key value is already 1, so a locking failure message is returned.

When client 2, which has obtained the lock, finishes processing the shared resource, it releases the lock. To release the lock, it simply sets the lock_key to 0.

The lock operation contains three operations (reading the lock variable, determining the value of the lock variable, and setting the value of the lock variable to 1), and these three operations need to ensure atomicity in the execution process. So how do you guarantee atomicity?

We can use the SETNX command to implement the lock operation, SETNX command to create the key if it does not exist, no assignment is done if the key exists. When the lock is being held, we execute SETNX lock_key 1. For the lock release operation, we can use the DEL command to delete the lock variable. For example, if client 2 locks, run SETNX lock_key 1. If the lock_key does not exist, the system creates a lock_key and returns that the lock_key is successfully locked. In this case, client 2 can access shared resources. If client 1 initiates a lock request and lock_key already exists, SETNX lock_key 1 does not perform any assignment operation and returns a lock failure, so client 1 fails to lock. After client 2 accesses the shared resource, run the DEL command to release the lock. When another client attempts to access the lock_key, the lock_key will no longer exist, and the lock can be performed normally. Therefore, we can use a combination of SETNX and DEL commands to lock and release locks.

But there are two problems:

1. An exception occurs after the SETNX command is executed on a client and the lock is locked. As a result, the DEL command is not executed to release the lock. Therefore, this client holds the lock all the time, and no other client can get the lock.

An effective way to solve this problem is to set an expiration time for the lock variable. This way, even if the client holding the lock fails to release the lock actively due to an exception, Redis will remove the lock variable based on its expiration date. Other clients can re-lock after the lock variable expires.

2. If client 1 runs the SETNX command to lock the client. If client 2 runs the DEL command to delete the lock, the lock on client A is mistakenly released. This is not acceptable to us.

To solve this problem, we need to be able to differentiate locking operations from different clients. How do we do that? We can generate a unique value for each client and assign the lock variable to this unique value during locking. In this way, when releasing the lock, the client needs to determine whether the value of the current lock variable is equal to its unique identifier. If the value is equal, the lock can be released.

Here’s how to do it in Redis. We can use SET plus EX/PX and NX options to lock. SET lock_key uuid NX PX 100 Lock_key indicates the lock variable, uuid indicates the unique identifier of the client, and PX 100 indicates that 100ms has expired. Since we need to compare the identity of the client and the value of the lock variable when releasing the lock, this involves multiple operations. To ensure atomicity, we need to use lua script. The following is the implementation of lua script.

if redis.call("get",KEYS[1]) == ARGV[1] then   
   return redis.call("del",KEYS[1])
else  
   return 0
end
Copy the code

KEY[1] represents lock_key and ARGV[1] represents the unique identity of the current client, which we pass in as arguments when executing the Lua script. Let’s take a look at the complete code implementation.

import redis import traceback import uuid import time class Inventory(object): def __init__(self): pool = redis.ConnectionPool(host='localhost', port=6379, decode_responses=True) client = redis.StrictRedis(connection_pool=pool, max_connections=20) self.client=client self.uuid=str(uuid.uuid1()) print(self.uuid) self.key="lock_key" self.inventory_key="inventory" def unlock(self): unlock_script="" \ "if redis.call("get",KEYS[1]) == ARGV[1] then" \ " return redis.call("del",KEYS[1])" \ "else" \ " return 0 " \ "end" try: unlock_cmd=self.client.register_script(unlock_script) result=unlock_cmd(keys=[self.key],args=[self.uuid]) if result==1: Except: print(traceback.format_exc()) def lock(self): try: while True: print(traceback.format_exc()) def lock(self): try: while True: result=self.client.set(self.key,self.uuid,px=100,nx=True) print(result) if result==1: Break print("sleep 1s") time.sleep(1) print("sleep ") return True except: print(traceback.format_exc()) def inventory(self): if self.lock(): Print (self.client.decr(self.inventory_key) print(self.unlock() inv=Inventory() inv.inventory()Copy the code

Three, afterword.

Here, we put Redis to achieve distributed lock on the end of the chat. Since all read here, might as well give a “three even” bar, your three even is my biggest motivation. I’ll see you next time.