I’m participating in nuggets Creators Camp # 4, click here to learn more and learn together!

background

In the section optimization of the use of distributed lock, we mentioned the logic of ordering and deducting inventory, but did not introduce it. In this paper, I will introduce the evolution process of inventory deduction scheme in actual business.

To improve the InventoryService inventory deduction code, let’s look at the problem of inventory deduction.

@Service
public class InventoryService {

    @Autowired
    private InventoryRepository inventoryRepository;

    /** * deduct inventory *@param inventoryId
     * @param deductCount
     * @return* /
    public void deduct(Long inventoryId, Integer deductCount) {
        Inventory inventory = inventoryRepository.get(inventoryId);
        Integer newTotalNum = inventory.getTotalNum() + deductCount;
        if (newTotalNum > inventory.getTotalLimit()) {
            throw new RuntimeException("Out of stock"); } inventory.setTotalNum(newTotalNum); inventoryRepository.update(inventory); }}Copy the code

Where, the attribute of Inventory is as follows:

public class Inventory {

  	/ / inventory id
    private Long inventoryId;

  	// Total inventory limit
    private Integer totalLimit;

  	// Total inventory consumption
    private Integer totalNum;
  
    / /... I don't care about the rest of the information
    // Getter an setter is omitted
}
Copy the code

According to the code above, if two buyers order in order, there will be no problem, the inventory will be updated as shown in the picture below.

However, due to the large number of buyers in our mall, it is impossible for us to make an order before placing an order. Therefore, the situation of scrambling is inevitable, and inventory deduction may occur as shown in the following figure. Two threads simultaneously bypass the judgment of whether the inventory is sufficient, so that the inventory is updated to the previously calculated newTotalNum = 199, instead of 198.

In fact, the core of the above problem is to control the sequential execution of threads, so that multiple threads can not enter the inventory deduction logic at the same time. If you have used distributed locks, it is easy to think of the first solution, which is to add distributed locks to control high concurrency before deducting inventory.

Redis distributed lock

We use the previously mentioned section with annotations to optimize the code as follows:

@Service
public class InventoryService {

    @Autowired
    private InventoryRepository inventoryRepository;

    /** * deduct inventory *@param inventoryId
     * @param deductCount
     * @return* /
    @RedisLock(keySuffix = "#inventoryId", keyPrefix = "inventory:")
    public void deduct(Long inventoryId, Integer deductCount) {
        Inventory inventory = inventoryRepository.get(inventoryId);
        Integer newTotalNum = inventory.getTotalNum() + deductCount;
        if (newTotalNum > inventory.getTotalLimit()) {
            throw new RuntimeException("Out of stock"); } inventory.setTotalNum(newTotalNum); inventoryRepository.update(inventory); }}Copy the code

The addition of distributed lock control ensures that only one thread is entered at the nt value, addressing the oversold issue mentioned earlier.

However, this scheme introduces a new problem. Threads that do not grab the lock are intercepted and the system is reported to be busy. It doesn’t mean stock is low, so consumers can just click the button again and try again. In fact, this is very unfriendly, it will make consumers mistakenly think that the system crashed, and because the locking is in the inventory of skU of goods, with high concurrency, the error frequency is often very high, which should not order a WB hot search?

Maybe another solution is to add the lock waiting time, such as 2s, 2s still failed to grab the lock and then report an error. It may be better in theory, but the concurrency will still encounter the same problem, treating the symptoms rather than the root cause.

If (newTotalNum > Inventory.getTotalLimit ())) is an atomic operation. If (newTotalNum > Inventory.getTotalLimit ()) is an atomic operation. In this case, there is no need to lock this code, we use Redis with lua script to try to optimize.

Redis works with Lua scripts

Benefits of using Lua scripts in Redis:

  • Atomic operations: Lua scripts are executed as a unit, so no other commands can be inserted in between, which is the most important feature.
  • Reduce network overhead: Multiple requests can be sent in script form at a time, reducing network latency.
  • Reusability: Lua scripts can reside in Redis memory, so they can be reused when used.

Note: Lua script execution is supported only after Redis 2.6.0.

Let’s take a look at the inventory reduction process after Redis and Lua script, as shown below:

Since our data was stored in Mysql at the beginning, we need to determine whether the inventory is deducted for the first time. If so, we need to synchronize Mysql data to Redis before deducting. The data model stored in Redis should have a hash structure, assuming the Inventory id above is 1:

`key`:
	inventory:1
`value`:
	{
    "totalLimit":200."totalNum":0
  }
Copy the code

The hash storage structure is mainly used to facilitate expansion. For example, in subsequent business, there is not only the concept of total inventory, but also daily inventory and hourly inventory.

The lua script for initializing inventory and inventory deductions is as follows, saved to Resources/Redis,

inventoryInit.lua

-- Inventory initialization
local is_exists = redis.call('exists',KEYS[1])
if is_exists == 1 then
	return 0
else
redis.call('hmset',KEYS[1].'totalLimit',ARGV[1].'totalNum',ARGV[2])
return 1 end
Copy the code

inventoryDeduct.lua

-- Inventory deduction
local inventory = KEYS[1]
local deductCount = tonumber(ARGV[1])
local inventoryMap = redis.call('hmget',inventory,'totalLimit'.'totalNum')
local totalLimit = tonumber(inventoryMap[1])
local totalNum = tonumber(inventoryMap[2])
if totalNum + deductCount > totalLimit then
	return 0
else
    totalNum = totalNum + deductCount
    redis.call('hmset',inventory,'totalNum'.tostring(totalNum))
    return 1
end
Copy the code

Now optimize the inventory deduction code using the StringRedisTemplate integrated with SpringBoot to assist in executing the redis command, as shown below:

@Service
public class InventoryService {

    @Autowired
    private InventoryRepository inventoryRepository;

    @Autowired
    private StringRedisTemplate redisTemplate;

    /** ** deduct inventory **@param inventoryId
     * @param deductCount
     * @return* /
    public void deduct(Long inventoryId, Integer deductCount) {
        String redisKey = "inventory:" + inventoryId;
        DefaultRedisScript<Long> redisScript = new DefaultRedisScript<>();
        // Determine the first deduction
        if (Boolean.FALSE.equals(redisTemplate.hasKey(redisKey))) {
            // Initialize inventory
            Inventory inventory = inventoryRepository.get(inventoryId);
            // Specify the lua script
            redisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource("redis/inventoryInit.lua")));
            // Specify the return type
            redisScript.setResultType(Long.class);
            Long result = redisTemplate.execute(redisScript, Collections.singletonList(redisKey)
                    , inventory.getTotalLimit().toString(), inventory.getTotalNum().toString());
            System.out.println("Inventory initialization" + (result == 1 ? "Success" : "Failure"));
        }
        redisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource("redis/inventoryDeduct.lua")));
        redisScript.setResultType(Long.class);
        // Parameter 1: redisScript, parameter 2: key list, parameter 3: arg (multiple)
        Long result = redisTemplate.execute(redisScript, Collections.singletonList(redisKey), deductCount.toString());
        System.out.println("Inventory deduction" + (result == 1 ? "Success" : "Failure")); }}Copy the code

The execution result

Inventory initialization succeeded Inventory deduction succeededCopy the code

The corresponding redis data is changed to totalNum = 1, and one item is sold.

disadvantages

Although this solution can carry more traffic, it also brings corresponding maintenance costs, as follows:

  1. May come upLess to sellIn the case.The Lua scriptPacking multiple commands together, while atomicity is guaranteed, is notTransaction rollbackFeatures. For example, batch deduction inventory process becomes suddenlyRedis downtime, the application thinks this timeRedis callisfailureThe front desk gives the user feedback on the error, but the amount deducted will not be rolled back. When the Redis fault is rectified, the user tries again and the recovered data is inconsistent. Need to combineRedisandThe databaseMake data check, and make incremental repair of data combined with the log of deduction service.
  2. In order to avoidLess to sellSituations occur where the program needs to be combinedRedisandThe databaseDo data check, combined with the deduction service log, do incremental data repair, system complexity increased a lot.
  3. Need to write Redis synchronization to Mysql scheduled task, increase development volume.

Mysql with optimistic locking

In fact, when the project was just built, the traffic was not necessarily very large. In order to quickly support the launch, it was not necessary to abandon Mysql’s plan of reducing inventory. Mysql has some built-in properties that can greatly reduce maintenance complexity. In fact, the project I worked on initially did this before moving to Redis as traffic picked up.

Mysql database has the following two important features:

  1. Depending on the database row locking, with optimistic locking (version number or inventory number) to ensure strong consistency of data concurrent deductions
  2. With the help of the transaction feature, for batch deduction (such as shopping cart order), part of the deduction fails and the data is rolled back

The overall inventory deduction process is as follows:

At the top, it will query the current remaining inventory for pre-check. If there is no inventory, pre-check will take effect to reduce the write operation to the database (this is only the first step of rough check, there is no need to ensure that the check is accurate). After all, read operations do not involve locking and have high concurrency performance.

The inventory deduction code is omitted not to write, the more critical is the inventory deduction SQL statement,

update inventory set total_num = total_num + #{deductCount} 
	where inventory_id = #{inventory_id} and total_num + #{deductCount} < total_limit + 1
Copy the code

Mysql database in the Innodb engine, when a single row with a unique index is updated, a row lock is added to ensure that the operation is mutually exclusive. In the deduction code, determine the return value of this SQL. If the value is 1, the deduction is successful. Otherwise, 0 is returned, indicating that the inventory is insufficient and needs to be rolled back.

Mysql > select * from user;

When there is a certain increase in traffic compared to the initial stage of online, it is not necessary to migrate to Redis deduction plan immediately. If a lot of business scenarios are second kill scenarios, for example, the inventory is only 200 pieces, but the Qps of access is high, most of the traffic may be blocked by the inventory check of the first step, the Qps of actual database write operations is not high.

In this case, the read-write separation of the database can effectively relieve the pressure of the whole database instance. The introduction of a set of slave libraries, read requests all read data from the slave library, the number of slave libraries depends on the situation, but eventually if the write operation reaches the thousand TPS level claimed by Mysql, we need to consider migrating Redis solution. Finally, the flow chart of the scheme is shown as follows:

conclusion

The choice of the plan is best according to the current business and business development speed to choose, sometimes do not necessarily use the most perfect plan, because perfect also means high maintenance costs, make some problems have to pay a small amount of money, maybe the company went bankrupt on the way maybe ~

In addition, there are some other issues not discussed in this article, such as

  1. For the scenario where inventory deduction is actually successful, such as network timeout, but it is considered as a failure when the order side is called, another retry is initiated, resulting in an additional inventory deduction, idempotent control is needed to ensure that the second call does not have actual impact.
  2. The order can be cancelled after the order is placed, and the inventory needs to be returned after the cancellation.

Due to limited space, I will introduce ~ in the next article