The problem

High concurrency scenarios (such as thousands of orders for the every second, and although at ordinary times to reach this level of concurrent probability is small, but it is possible, such as holidays to promote, or a single commodity seconds kill activity, etc.), if use distributed lock to prevent inventory oversold, how to deal with high concurrency optimization for distributed lock this scene? When I was doing technical research, I found an article that gave me some ideas. Well, you are all big shots, so I will Follow it.

Inventory oversold phenomenon is how to produce

Although is proprietary electric company, and mainly all kinds of home appliance products, but the mall (including internal mall) there are all kinds of goods such as various types of home appliances, digital products (with various mobile phone manufacturers, such as the VIVO), daily products, food and beverage, etc., on sale, with preferential marketing scenarios, such as reduction of concurrent volume is still relatively high. So how does inventory oversold come about? Use a flow chart on the net below to try to explain

This diagram, in fact, is very clear. Suppose the order system is deployed on two machines, and different users want to buy 10 iphones at the same time, and each sends a request to the order system. Each order system instance is then checked in the database, and the current inventory is 12 iphones. Both users can place orders without overselling inventory! As a result, each order system instance sent SQL to the database to place an order and then deducted 10 stocks, one from 12 to 2 and the other from 2 to -8. It’s over now, the inventory is negative! There aren’t 20 iphones for two users! . The above scenario is just like that, everyone is fighting for a 2-piece COLMO high-end air conditioner. Under the condition of overselling, the company suffers serious losses and the programmer is blamed for the loss.

How to solve the problem of overselling inventory with distributed lock?

In fact, inventory oversold problem is a normal problem, there are many technical solutions, such as pessimistic lock, distributed lock, optimistic lock, queue serialization, Redis atomic operation, and so on. But in the case of distributed locking (mainly for data accuracy), how to solve the oversold inventory problem?

The implementation principle of distributed lock:

For a lock key, only one client can obtain the lock at a time. Other clients will wait indefinitely to try to obtain the lock. Only the client that has obtained the lock can execute the following business logic.As you can see from the logic above, only one instance of the order system can successfully add a distributed lock, and then only one instance can check the inventory, determine whether the inventory is sufficient, place an order to reduce the inventory, and then release the lock. After the lock is released, another order system instance can be locked. Then we check the inventory and find that there are only 2 sets in stock. The inventory is insufficient and we cannot buy, so the order fails. It’s not going to subtract inventory to -8. Are there any other solutions to the oversold inventory problem?

Of course! For example, pessimistic locks, distributed locks, optimistic locks, queue serialization, asynchronous queue dispersion, Redis atomic operation, etc., many other solutions, we have our own set of optimization mechanisms for inventory oversold. However, in the original actual landing scenario, distributed locking was chosen and several versions were iterated. Therefore, other solutions are not on the table. Now, what are the problems with distributed locking schemes in high concurrency scenarios? That’s a big problem! Once the distributed lock is added, all clients must lock the inventory lock key of the same item in order to order the same item. For example, all orders for iPhone must be locked with the “iphone_stock” lock key. As a result, single requests for the same item must be serialized and processed one after another. Let’s assume that after the lock is added and before the lock is released, check the inventory -> create the order -> deduct the inventory. This process is very high performance, calculate the whole process 20 milliseconds, this should be good. So a second is 1000 milliseconds, and only 50 requests for this good can be processed sequentially.

So at least you can see that simply using distributed locks to deal with oversold inventory has its drawbacks. The defect is that when multiple users place an order for the same commodity at the same time, it will be serialized based on distributed lock, resulting in the inability to process a large number of orders for the same commodity at the same time. This solution may be acceptable for ordinary small e-commerce systems with low concurrency and no second kill scenarios. (In fact, this is the initial version of the company’s solution. Why did you choose this solution? Have to ask the departed bigwigs). Because if the concurrency is very low, there will be less than 10 requests per second. If there is no scene of instantaneous high concurrency killing a single commodity in a second, in fact, it is rare to place 1000 orders for the same commodity in a second, because the small e-commerce system does not have that scene.

How to optimize distributed locks for high concurrency?

If the product key is not locked, the accuracy of data cannot be guaranteed, and overselling of inventory will occur. If the key is locked, it will lead to serialization, and the concurrency can not increase, what to do? Is there a compromise? There are! Inspired by JDK segmented locks such as ConcurrentHashMap, consider splitting data into multiple segments, each of which is a separate lock, so that multiple threads can concurrently modify different segments of data. Not to mention that only one thread can exclusively modify the data in ConcurrentHashMap at a time. In addition, Java 8 has a new LongAdder class, which is also an optimization of Java 7’s AtomicLong, to solve the problem of CAS class operations in high concurrency scenarios, using optimistic locking ideas, which can cause a large number of threads to repeat the loop for a long time. LongAdder also adopts a similar segmentation CAS operation. If it fails, it will automatically migrate to the next segment for CAS. In fact, distributed lock optimization ideas are similar.In fact, this is segment locking. You know, if you have 1,000 inventory items on your iPhone right now, you can split it into 20 inventory items, and if you want, you can have 20 inventory items in a table in your database, like stock_01, stock_02, something like that, or 20 inventory keys in a place like Redis.

In short, it is to break down your 1000 pieces of inventory to him, and each stock segment is 50 pieces of inventory. For example, stock_01 corresponds to 50 pieces of inventory, and stock_02 corresponds to 50 pieces of inventory. And then 1,000 requests per second came in, and you could have written a simple random algorithm, where each request was randomly selected from 20 segments of inventory, and one of them was locked. At the same time, there can be up to 20 order requests executed together, each order request locks a section of inventory, and then in the business logic, the database or Redis that section of inventory can be operated, including checking inventory -> judging whether the inventory is sufficient -> deducting inventory. So what is this equivalent to? It is equivalent to processing 20 order requests simultaneously in 20 milliseconds, so in 1 second, it can process 20 * 50 = 1000 order requests to iPhone in turn.

  butOnce a certain data is segmented, there are several pits that you must pay attention to:

(1) What should I do if I click and lock an order request and find that the inventory in this section is insufficient? At this point, you automatically release the lock, then immediately change the next section inventory, try locking again and try processing. This process must be implemented in order to avoid an indefinite block by other users with an unlimited hold lock due to insufficient segmenting inventory.

(2) If the segmented inventory is insufficient for a single user to place an order at a certain moment, but the total inventory is greater than the sum of all requested order stocks, how to deal with such a scenario where an order cannot be placed even with inventory? (To solve the problem of this scenario, it is difficult to implement), but in terms of the second kill scenario, it is less likely to occur, because this kind of user order quantity is limited, such as each user can buy a maximum of one item.

Are there any disadvantages to distributed lock concurrency optimization?

Implementation is too complex, inconvenient and costly!

  • First of all, you have to store a single piece of data in segments. One good inventory field is now divided into 20 segmented inventory fields.
  • Secondly, every time you deal with inventory, you have to write your own selection algorithm and pick a segment to deal with;
  • Finally, if you run out of data in one segment, you have to automatically move to the next.

Of course, in this scenario, we also need to adopt a separate database and table scheme.