Who’s better, Redis or Zookeeper?

Why distributed locks?

Before we get to that, let’s look at a business scenario:

System A is an e-commerce system, which is currently deployed on A machine. There is an interface for users to place orders in the system, but before placing orders, users must check the inventory to ensure that the inventory is enough before placing orders to users.

Because the system has certain concurrency, the inventory of goods will be stored in Redis in advance, and the inventory of redis will be updated when users place orders.

The system architecture is as follows:

However, there is a problem with this: if at some point the inventory of an item in Redis is 1, two requests come in at the same time, one of which goes to step 3 in the figure above and updates the inventory of the database to 0, but step 4 has not yet been executed.

The other request reaches step 2, finds that the inventory is still 1, and continues with step 3.

As a result, 2 items are sold when there is only 1 item in stock.

Obviously not! This is a classic oversold inventory problem

At this point, it’s easy to think of a solution: lock steps 2, 3, and 4 so that when they’re done, another thread can come in and execute step 2.

As shown above, use the Java-provided synchronized or ReentrantLock to hold the lock during step 2 and release the lock after step 4.

Thus, steps 2, 3, and 4 are “locked” and can only be executed sequentially between threads. Follow the public account Internet architect, reply keyword 2T, get the latest architecture video

But the good times did not last long, the whole system concurrency soared, a machine can not bear. Now we need to add a machine as shown below:

After adding machines, the system looks like the one above. Oh my God!

Suppose that the requests from two users come at the same time, but fall on different machines, then can these two requests be executed at the same time, or there will be the problem of overselling inventory.

Why is that? Because the two A systems in the figure above are running on two different JVMS, the locks they add are valid only for threads in their own JVMS, not for threads in other JVMS.

Therefore, the problem here is that the native locking mechanism provided by Java fails in multi-machine deployment scenarios

This is because the locks on both machines are not the same (they are in different JVMS).

So, we just have to make sure that the locks on both machines are the same, and the problem is solved, right?

At this point, the distributed lock grand stage, distributed lock idea is:

Provide a global and unique lock acquisition “thing” in the whole system, and then each system needs to add a lock, to ask this “thing” to get a lock, so that different systems can be considered the same lock.

This “thing” could be Redis, Zookeeper, or a database.

Text description is not very intuitive, let’s look at the picture below:

Through the above analysis, we know that the thread safety cannot be guaranteed by using Java native locking mechanism in the case of distributed deployment system, so we need to use distributed locking scheme.

So how do you implement distributed locks? Keep reading!

Distributed lock based on Redis

SET anyLock unique_value NX PX 30000 // Release lock: By executing a lua script // releasing the lock involves two instructions that are not atomic // using Redis lua script support features, If redis. Call ("get",KEYS[1]) == ARGV[1] then return Redis. Call ("del",KEYS[1]) else return 0 endCopy the code

Another way: Redisson

In addition, Redis distributed lock, in addition to their own implementation based on Redis client native API, you can also use the open source framework: Redission

Redisson is an enterprise-level open source Redis Client that also provides distributed lock support. I highly recommend it. Why?

Recall from above that if you write code to set a value via Redis, you do so with the following command.

• SET anyLock unique_value NX PX 30000

The timeout is set to 30 seconds. If I do not complete the business logic after 30 seconds, the key will expire and other threads may acquire the lock.

In this way, the second thread will come in before the first thread has finished executing the business logic. Therefore, we still need to maintain the expiration time, which is too troublesome

How does Redisson do that? Feel the thrill of using Redission:

Config config = new Config();
config.useClusterServers()
.addNodeAddress("Redis: / / 192.168.31.101:7001")
.addNodeAddress("Redis: / / 192.168.31.101:7002")
.addNodeAddress("Redis: / / 192.168.31.101:7003")
.addNodeAddress("Redis: / / 192.168.31.102:7001")
.addNodeAddress("Redis: / / 192.168.31.102:7002")
.addNodeAddress("Redis: / / 192.168.31.102:7003");
 
RedissonClient redisson = Redisson.create(config);
 
 
RLock lock = redisson.getLock("anyLock");
lock.lock();
lock.unlock();
Copy the code

It’s as simple as that, we just need to use the API lock and unlock to complete the distributed lock, it helps us to consider a lot of details:

• All redisson instructions are executed through Lua scripts, and Redis supports atomic execution of Lua scripts

• Redisson sets the default expiration time of a key to 30s. What if a client holds a lock for more than 30s?

  Redisson has the concept of a watchdog, which sets the key timeout to 30 seconds every 10 seconds after you acquire the lock

  This way, if the lock is held all the time, the key will not expire and another thread will acquire the lock.

• Redisson’s “watchdog” logic ensures that no deadlocks occur.

  (If the machine goes down, so does the watchdog. The lock will expire automatically after 30 seconds, and other threads can acquire the lock.

Here is the implementation code:

// Lock logic
private <T> RFuture<Long> tryAcquireAsync(long leaseTime, TimeUnit unit, final long threadId) {
    if(leaseTime ! = -1) {
        return tryLockInnerAsync(leaseTime, unit, threadId, RedisCommands.EVAL_LONG);
    }
    // Call a Lua script to set some keys and expiration times
    RFuture<Long> ttlRemainingFuture = tryLockInnerAsync(commandExecutor.getConnectionManager().getCfg().getLockWatchdogTimeout(), TimeUnit.MILLISECONDS, threadId, RedisCommands.EVAL_LONG);
    ttlRemainingFuture.addListener(new FutureListener<Long>() {
        @Override
        public void operationComplete(Future<Long> future) throws Exception {
            if(! future.isSuccess()) {return;
            }
 
            Long ttlRemaining = future.getNow();
            // lock acquired
            if (ttlRemaining == null) {
                // Watchdog logicscheduleExpirationRenewal(threadId); }}});return ttlRemainingFuture;
}
 
 
<T> RFuture<T> tryLockInnerAsync(long leaseTime, TimeUnit unit, long threadId, RedisStrictCommand<T> command) {
    internalLockLeaseTime = unit.toMillis(leaseTime);
 
    return commandExecutor.evalWriteAsync(getName(), LongCodec.INSTANCE, command,
              "if (redis.call('exists', KEYS[1]) == 0) then " +
                  "redis.call('hset', KEYS[1], ARGV[2], 1); " +
                  "redis.call('pexpire', KEYS[1], ARGV[1]); " +
                  "return nil; " +
              "end; " +
              "if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then " +
                  "redis.call('hincrby', KEYS[1], ARGV[2], 1); " +
                  "redis.call('pexpire', KEYS[1], ARGV[1]); " +
                  "return nil; " +
              "end; " +
              "return redis.call('pttl', KEYS[1]);",
                Collections.<Object>singletonList(getName()), internalLockLeaseTime, getLockName(threadId));
}
 
 
 
// The watchdog will eventually call here
private void scheduleExpirationRenewal(final long threadId) {
    if (expirationRenewalMap.containsKey(getEntryName())) {
        return;
    }
 
    // This task will be delayed for 10 seconds
    Timeout task = commandExecutor.getConnectionManager().newTimeout(new TimerTask() {
        @Override
        public void run(Timeout timeout) throws Exception {
 
            // This operation will reset the expiration time of the key to 30s
            RFuture<Boolean> future = renewExpirationAsync(threadId);
 
            future.addListener(new FutureListener<Boolean>() {
                @Override
                public void operationComplete(Future<Boolean> future) throws Exception {
                    expirationRenewalMap.remove(getEntryName());
                    if(! future.isSuccess()) { log.error("Can't update lock " + getName() + " expiration", future.cause());
                        return;
                    }
 
                    if (future.getNow()) {
                        // reschedule itself
                        // Call this method recursively, extending the expiration time in an infinite loopscheduleExpirationRenewal(threadId); }}}); } }, internalLockLeaseTime /3, TimeUnit.MILLISECONDS);
 
    if (expirationRenewalMap.putIfAbsent(getEntryName(), newExpirationEntry(threadId, task)) ! =null) { task.cancel(); }}Copy the code

RedissonClient redisson = redisson.create (config); RedissonClient redisson.create (config); RLock lock1 = redisson.getFairLock("lock1"); RLock lock2 = redisson.getFairLock("lock2"); RLock lock3 = redisson.getFairLock("lock3"); RedissonRedLock multiLock = new RedissonRedLock(lock1, lock2, lock3); multiLock.lock(); multiLock.unlock(); Summary: This section analyzes the specific landing scheme of using Redis as a distributed lock and some of its limitations. Then it introduces a Redis client framework redisson, which I recommend everyone to use, which will be much less care than writing code.Copy the code

Distributed lock based on ZooKeeper

Curator introduce

Curator is an open source zooKeeper client that also provides an implementation of distributed locks.

The way he uses it is simple:

InterProcessMutex interProcessMutex = new InterProcessMutex(client,"/anyLock");
interProcessMutex.acquire();
interProcessMutex.release();
Copy the code

The core source code of the distributed lock is as follows:

private boolean internalLockLoop(long startMillis, Long millisToWait, String ourPath) throws Exception
{
    boolean  haveTheLock = false;
    boolean  doDelete = false;
    try {
        if( revocable.get() ! =null ) {
            client.getData().usingWatcher(revocableWatcher).forPath(ourPath);
        }
 
        while( (client.getState() == CuratorFrameworkState.STARTED) && ! haveTheLock ) {// Get the sorted set of all current nodes
            List<String>        children = getSortedChildren();
            // Get the name of the current node
            String              sequenceNodeName = ourPath.substring(basePath.length() + 1); // +1 to include the slash
            // Determine whether the current node is the smallest node
            PredicateResults    predicateResults = driver.getsTheLock(client, children, sequenceNodeName, maxLeases);
            if ( predicateResults.getsTheLock() ) {
                // Get the lock
                haveTheLock = true;
            } else {
                Register a listener on the previous node of the current node
                String  previousSequencePath = basePath + "/" + predicateResults.getPathToWatch();
                synchronized(this){
                    Stat stat = client.checkExists().usingWatcher(watcher).forPath(previousSequencePath);
                    if( stat ! =null) {if( millisToWait ! =null ){
                            millisToWait -= (System.currentTimeMillis() - startMillis);
                            startMillis = System.currentTimeMillis();
                            if ( millisToWait <= 0 ){
                                doDelete = true;    // timed out - delete our node
                                break;
                            }
                            wait(millisToWait);
                        }else{ wait(); }}}// else it may have been deleted (i.e. lock released). Try to acquire again}}}catch ( Exception e ) {
        doDelete = true;
        throw e;
    } finally{
        if( doDelete ){ deleteOurPath(ourPath); }}return haveTheLock;
}
Copy the code

In fact, the underlying principle of distributed locking implemented by curator is similar to that analyzed above. Here we use a diagram to illustrate the principle in detail:

Summary:

This section introduces the distributed lock scheme implemented by Zookeeperr and the basic use of zK open source client, and briefly introduces its implementation principle.

Compare the advantages and disadvantages of the two schemes

advice

From the previous analysis, there are two common solutions for implementing distributed locks: Redis and ZooKeeper, each of which has its own advantages. How to choose?

Personally, I prefer the lock implemented by ZK:

Because Redis may have hidden dangers, which may lead to incorrect data. However, how to choose depends on the specific scene in the company.

If there is zK cluster condition in the company, ZK is preferred. But if there is only Redis cluster in the company, there is no condition to build ZK cluster.

In fact, it is also possible to use Redis to achieve, in addition, the system designer may consider the system has Redis, but do not want to introduce some external dependencies again, can choose Redis.

This is an architecture-based consideration for the system designer