Introduction to the

It is common for a piece of logic to be executed by a single instance at the same time. In a microservice architecture, there may be multiple instances of a program, requiring serial execution through distributed locks.

The simplest distributed lock is nothing more than to find a single existence for multiple program instances, such as MySQL data only one or Redis only one, this single existence can be used to build a lock, multiple program instances to execute a certain piece of logic must first obtain the lock, and then execute.

For some reason, at work, my colleagues and I studied the Python Redis library distributed lock implementation source code, here is a brief share.

This library can be installed through PIP.

pip install redis==2.106.
Copy the code

Here to the 2.10.6 version of this library as an example, it Redis distributed lock source for a simple analysis.

Code words are not easy to nonsense two sentences: need python learning materials or technical questions to exchange “click”

The code analysis

Once the StrictRedis object is instantiated, a distributed lock is obtained using the lock method.

First take a look at the source code corresponding to the Lock method.

def lock(self, name, timeout=None, sleep=0.1, blocking_timeout=None,
             lock_class=None, thread_local=True) :
        if lock_class is None:
            if self._use_lua_lock is None:
                # the first time .lock() is called, determine if we can use
                # Lua by attempting to register the necessary scripts
                try:
                    LuaLock.register_scripts(self)
                    self._use_lua_lock = True
                except ResponseError:
                    self._use_lua_lock = False
            lock_class = self._use_lua_lock and LuaLock or Lock
        return lock_class(self, name, timeout=timeout, sleep=sleep,
                          blocking_timeout=blocking_timeout,
                          thread_local=thread_local)
Copy the code

The method provides multiple parameters, of which:

Name specifies the lock name. Timeout Specifies the timeout period for the lock. Sleep specifies the sleep time for the thread. Lock_class indicates that the class object using the lock, thread_local, indicates whether it is thread safeCopy the code

Lock_class = self._use_lua_lock and LuaLock or Lock.

Lock_class can be LuaLock or Lock, after simple analysis, Lock class is the key, LuaLock class inherited from Lock, through Lua code to achieve some operations of Redis, here focus on Lock class.

First look at the class’s __init__ method.

class Lock(object) :
    def __init__(self, redis, name, timeout=None, sleep=0.1,
                 blocking=True, blocking_timeout=None, thread_local=True) :
        self.redis = redis
        self.name = name
        self.timeout = timeout
        self.sleep = sleep
        self.blocking = blocking
        self.blocking_timeout = blocking_timeout
        self.thread_local = bool(thread_local)
        self.local = threading.local() if self.thread_local else dummy()
        self.local.token = None
        if self.timeout and self.sleep > self.timeout:
            raise LockError("'sleep' must be less than 'timeout'")
Copy the code

The __init__ method initializes different attributes, where self.local is the thread’s local field and is used to store thread-specific data that is not shared with other threads.

In addition, the __init__ method checks timeout and sleep, and returns an error if the thread waits for a lock for more than the timeout period.

Next, focus on the acquire method in the Lock class, which is coded as follows.

import time as mod_time

class Lock(object) :

    def acquire(self, blocking=None, blocking_timeout=None) :
        sleep = self.sleep
        token = b(uuid.uuid1().hex)
        if blocking is None:
            blocking = self.blocking
        if blocking_timeout is None:
            blocking_timeout = self.blocking_timeout
        stop_trying_at = None
        if blocking_timeout is not None:
            stop_trying_at = mod_time.time() + blocking_timeout
        while 1:
            if self.do_acquire(token):
                self.local.token = token
                return True
            if not blocking:
                return False
            if stop_trying_at is not None and mod_time.time() > stop_trying_at:
                return False
            mod_time.sleep(sleep)
Copy the code

The main logic of acquire method is an infinite loop, in which do_acquire method is called to obtain Redis distributed lock, if the lock is successfully obtained, then the token is stored in the local object of the current thread, if not, then the blocking is considered, and if blocking is Flase, then the token is blocked. If the current time exceeds blocking_timeout, False is also returned. If the current time exceeds blocking_timeout, False is returned. If the current time exceeds blocking_timeout, False is returned.

Further analysis of do_acquire method, the code is as follows:

  def do_acquire(self, token) :
        if self.redis.setnx(self.name, token):
            if self.timeout:
                # convert to milliseconds
                timeout = int(self.timeout * 1000) # convert to milliseconds
                self.redis.pexpire(self.name, timeout)
            return True
        return False
Copy the code

In do_acquire method, name is used as key and token is used as value by redis setnx method at the beginning. Setnx method can normally store value into Redis only when there is no key. If the key is dependent, this method does not do any operation. At this point, the lock is not acquired.

After the token is inserted successfully, the timeout period is determined. If timeout is set, the pEXPIRE method is used to set the timeout of the redis key name. Because the pEXPIRE method is in milliseconds, you need to convert timeout to milliseconds first.

If timeout is not set, the name key can only be cleared by logic in the do_release method.

So far, we clearly know that the essence of Redis distributed lock is actually a Redis key-value, very simple…

After clarifying the lock acquisition logic, let’s look at the corresponding release logic, focusing on the release method, which is coded as follows.

def release(self) :
        "Releases the already acquired lock"
        expected_token = self.local.token
        if expected_token is None:
            raise LockError("Cannot release an unlocked lock")
        self.local.token = None
        self.do_release(expected_token)
Copy the code

In the release method, the token in the thread is first taken out and set to None. Then the do_release method is called to release the lock. The code of do_release method is as follows.

def do_release(self, expected_token) :
        name = self.name

        def execute_release(pipe) :
            lock_value = pipe.get(name)
            iflock_value ! = expected_token:raise LockError("Cannot release a lock that's no longer owned")
            pipe.delete(name)

        self.redis.transaction(execute_release, name)
Copy the code

The logic of the do_release method is very simple. The main logic of the do_release method is execute_release. The logic of the execute_release method is executed by starting a transaction using Redis’s transaction method.

In execute_release, the value corresponding to the key name is first obtained by the get method, and then deleted by the delete method to release the Redis distributed lock.

Blocking properties

Observe this code for the Acquire method.

 while 1:
            if self.do_acquire(token):
                self.local.token = token
                return True
            if not blocking:
                return False
            if stop_trying_at is not None and mod_time.time() > stop_trying_at:
                return False
            mod_time.sleep(sleep)
Copy the code

If blocking is True and the lock is not available, execute the following logic to let the thread sleep and block until the other thread releases the lock. If blocking is False, the lock cannot be obtained.

This leads to several situations where thread A and thread B both need to execute the same logic and need to acquire the lock before executing.

If thread A is blocking and thread B is blocking, as opposed to executing, then thread B is blocking and waiting to see if thread A is blocking. If blocking is False, thread B does not acquire the lock and does not perform the same logic.

If thread A finishes executing and thread B comes in, then blocking is True or False and thread B does not block and acquires the lock, same logic.

A simple conclusion is that blocking cannot guarantee that the logic is executed once. If you want to execute the logic only once through Redis distributed lock, you still need to control the business level, such as whether the business data in MySQL is modified or whether the business data is recorded in Redis.

As a Python developer, I spent three days to compile a set of Python learning tutorials, from the most basic Python scripts to Web development, crawlers, data analysis, data visualization, machine learning, etc. These materials can be “clicked” by the friends who want them