In the last article, the red envelope snatching achieved by Redis was tested and found to have serious blocking problems. Users who grabbed the red envelope could get feedback quickly, while those who could not grabbed the red envelope could not get the result for a long time (more than 10 seconds). The main reasons are as follows:

1, the use of distributed lock, resulting in all operations can only be queued in sequence, and the behind did not grab the red envelope need to wait for the front of the students grab the red envelope after he can see whether he has grabbed the red envelope

2. It takes a lot of time to interact with Redis for several times (tens to hundreds of milliseconds for each interaction), and the distributed lock itself also needs to interact with Redis

So after careful polishing, I decided to use Lua expressions to reduce the number of redis interactions and ensure the atomicity of multiple interactions with Redis in the case of high concurrency

1. Optimize the red envelope snatching process

In addition to adding lua scripts to handle the actual snatching process, distributed locks are removed, and lua scripts are used to verify whether a user has grabbed a red envelope through a Bloom filter

// The process of grabbing red packets must be atomic, and the distributed lock is added here. // However, the distributed lock is used, and the block time is too long, so some threads need to be blocked for more than 10s, which is very poor performance. If (integer.parseint (redisUtil. Get (redPacketId + TAL_PACKET)) > 0) {if (integer.parseint (redisUtil. Get (redPacketId + TAL_PACKET)) > 0) { // Lua script logic contains the calculation of the amount of red envelope // any balance and other instantaneous information is taken from the snapshot here, otherwise not allowed // If we write logic separately here, If atomicity is not guaranteed, it may cause the red envelope to be different from the original amount when it is used later. String result = grubFromRedis(redPacketId + TAL_PACKET, redPacketId + TOTAL_AMOUNT, userId, redPacketId); // Prepare to return the resultCopy the code

Many of these operations are compressed into Lua scripts

Localpacket_amount_id = KEYS[2] localpacket_amount_id = KEYS[3] localpacket_count_id = KEYS[2] localpacket_amount_id = KEYS[3] localpacket_count_id = KEYS[3 Local red_packet_id = KEYS[4] -- GRUB local bloom_name = red_packet_id.. '_BLOOM_GRAB_REDPACKET'; -- Blon filter ID local rcount = redis. Call ('GET', packet_count_id) -- Ramount = redis. Call ('GET', packet_count_id) Packet_amount_id) -- local amount = ramount; -- The default red envelope amount is the balance, If tonumber(rcount) > 0 then local flag = redis. Call (' bf.exists ', bloom_name, Return "1" -- It is not completely certain that the user already exists elseIf (tonumber(rcount) ~= Local maxamount = ramount/rcount * 2; amount = math.random(1,maxamount); end local result_2 = redis.call('DECR', packet_count_id) local result_3 = redis.call('DECRBY', packet_amount_id, amount) redis.call('BF.ADD', bloom_name, user_id) return amount .. "SPLIT" .. rcount else return "0" endCopy the code

2. Optimize the write back logic (using MQ instead is more reliable and appropriate)

@Async @Transactional(propagation = Propagation.REQUIRES_NEW) public void callback(String userId,String redPacketId,int Throws Exception {log.info(" User: {}, grab the current red envelope: {}, amount: {}, write back successfully!" , userId, redPacketId, amount); // Add information about grabbing red packets RedPacketRecord = new RedPacketRecord().Builder () .user_id(userId).red_packet_id(redPacketId).amount(amount).build(); redPacketRecord.setId(UUID.randomUUID().toString()); redPacketRecordRepository.save(redPacketRecord); }Copy the code

JPA+mysql auto-increment ID deadlock problem

So we adjusted the primary key generation logic for both tables:

@mappedsuperclass @data @noargsconstructor @allargsconstructor public class BaseEntity {@id // public primary key // @generatedValue // Increment sequence private String ID; @column (updatable = false) // Not allowed to change @creationTIMESTAMP // Automatically assigned when created private Date createTime; @updatetimestamp // Automatically changes when changing private Date updateTime; }Copy the code

@entity // Identifies this as a JPA database Entity class @table @data // Lombok getter toString @toString (callSuper = true) // overwrite toString containing fields of the parent class @slf4j // Slf4j log@builder //biulder constructor @noargsconstructor // AllArgsConstructor // full parameter constructor @JsonIgnoreProperties({"hibernateLazyInitializer", "handler"}) public class RedPacketInfo extends BaseEntity implements Serializable { private String red_packet_id; private int total_amount; private int total_packet; private String user_id; }Copy the code

@entity // Identifies this as a JPA database Entity class @table @data // Lombok getter toString @toString (callSuper = true) // overwrite toString containing fields of the parent class @slf4j // Slf4j log@builder //biulder constructor @noargsconstructor // AllArgsConstructor // full parameter constructor @JsonIgnoreProperties({"hibernateLazyInitializer", "handler"}) public class RedPacketRecord extends BaseEntity implements Serializable { private int amount; private String red_packet_id; private String user_id; }Copy the code

@Transactional
public RedPacketInfo handOut(String userId, int total_amount, int tal_packet) {
    RedPacketInfo redPacketInfo = new RedPacketInfo();
    redPacketInfo.setRed_packet_id(genRedPacketId(userId));
    redPacketInfo.setId(redPacketInfo.getRed_packet_id());
    redPacketInfo.setTotal_amount(total_amount);
    redPacketInfo.setTotal_packet(tal_packet);
    redPacketInfo.setUser_id(userId);
    redPacketInfoRepository.save(redPacketInfo);

    redisUtil.set(redPacketInfo.getRed_packet_id() + TAL_PACKET, tal_packet + "");
    redisUtil.set(redPacketInfo.getRed_packet_id() + TOTAL_AMOUNT, total_amount + "");

    return redPacketInfo;
}
Copy the code

The test code

Test 1000 concurrent, grab 10 yuan 20 red envelope, the average time of each person grab red envelope within 1 second (average 600ms), much better than the previous version of the data grab red envelope

@GetMapping("/concurrent") public String concurrent(){ RedPacketInfo redPacketInfo = RedPacketService. HandOut (" ZXP ", 1000, 20); String redPacketId = redPacketInfo.getRed_packet_id(); for(int i = 0; i < 1000; i++) { Thread thread = new Thread(() -> { String userId = "user_" + randomValuePropertySource.getProperty("random.int(10000)").toString(); Date begin = new Date(); GrabResult grabResult = redPacketService.grab(userId, redPacketId); Date end = new Date(); Log.info (grabresult.getmsg ()+" +(end.gettime () -begin.gettime ()))); }); thread.start(); } return "ok"; }Copy the code

Fork From Git

redistest