What is idempotency?

Idempotent (idempotence) is a mathematical and computer concept commonly found in abstract algebra. In programming, an idempotent operation is characterized by any number of executions having the same effect as a single execution. An idempotent function, or idempotent method, is a function that can be executed repeatedly with the same parameters and achieve the same results. These functions do not affect system state, nor do they have to worry about system changes caused by repeated execution. For example, the “getUsername() and setTrue()” functions are idempotent. More complex operational idempotent guarantees are implemented using unique transaction numbers (serial numbers). Idempotent is an operation that produces the same effect and returns the same result no matter how many times it is performed.

Idempotence problem under high concurrency

Here are two examples of idempotent problems under high concurrency;

I. Examples of ticket purchase

The ticket purchase process is as follows:

Step1: Check whether there is a ticket, if there is a ticket, continue the next step, otherwise there is no ticket, end;

Step2: Deduct the ticket from the user account;

Step3: subtract one from the remaining ticket;

In this case, there is no problem under normal circumstances, but for example, if the user has ordered several times in a row, or the network problem is caused, or the concurrency when many people buy at the same time, two or more threads will enter step1 at the same time, at this time, it is judged that there are tickets, and then continue to step2, step3, at this time, There may be a negative number of surplus tickets, sell more;

Second, recharge instance

Recharge process is as follows:

Step1: The user enters the recharge amount and requests the back-end business system;

Step2: The backend generates an order, and the status of the order is unpaid, and then requests the third-party payment interface;

Step3: the client confirms payment;

Step4: the third-party payment notifies the payment result asynchronously through the callback interface provided by us;

The specific demo code of step4 is as follows:

System.out.println("Query order");
Order order = orderMapper.getByOrderId(orderId); // Get the order based on the order ID
if(order.getStatus()==0) {// If it is not paid
  System.out.println("Unpaid status");
  order.setStatus(1); // Set the status of successful payment
  System.out.println("Update payment status...");
  orderMapper.update(order); // Update payment status
  System.out.println("Account recharge...");
  userAccountMapper.addAmount(order.getAmount(),userAccount.getUserId()); // Top up your account
  System.out.println("Recharge completed...");
  return true;
}else{ // The payment has been successful and the order has been processed
  System.out.println("Found order processed");
  return true;
}
Copy the code

This fourth step is flawed, if there is a third-party payment system problem or network problem, there are multiple threads executing at the same time

Order order = orderMapper.getByOrderId(orderId);
Copy the code

Query the order information according to the order ID, and find that the status status is not paid, so they enter if, at this time, the situation of account recharging occurs;

Summary of idempotent problem

As long as update data is based on read data, idempotent problems may occur when high concurrency is encountered. For example, when updating data, there will be no problem if it does not depend on the data queried. For example, when modifying the name of a user, multiple people will modify it at the same time, and the result does not depend on the name of the previous user, there will be no concurrent update problem.

Idempotent problem solution

As for the solution of idemidematism problem, the industry provides many solutions, such as Java synchronous lock for single-machine system, optimistic lock, pessimistic lock, distributed lock, unique index, token mechanism to prevent page repeat submission, etc., each solution has advantages and disadvantages; But the mainstream, or optimistic lock and distributed lock these two programs;

Java synchronization lock scheme

We can use a synchronized synchronous locks, put the query status code and update the code within a synchronized lock, so the same time there is only one thread into execution, and execution of the other threads can enter, this can solve the problem of idempotence, but if the inside of the synchronized block business code execution time is longer, it will seriously affect the user experience, And throughput of the system. So it’s not optimal;

Pessimistic locking scheme

Pessimistic locks, as the name implies, are Pessimistic. Each time I fetch the data, I think someone else will change it, so I Lock the data each time I fetch it, so that someone else will try to fetch it and block it until it gets the Lock.

Pessimistic locking: Shielding all operations that might violate data integrity, assuming concurrency conflicts.

Java synchronized is an implementation of pessimistic locking. Each time a thread attempts to modify data, the lock is acquired first, ensuring that only one thread can manipulate data at a time, while the other threads are blocked.

Pessimistic database locking is implemented through for UPDATE;

select * from t_order where orderId=#{orderId} for update
Copy the code

Pessimistic locking is generally used together with transactions. The data locking time may be long, affecting user experience and system throughput. Therefore, pessimistic locking is generally not used.

Optimistic locking scheme

Optimistic Lock, as the name implies, is very Optimistic. Every time I go to get data, I think that others will not change it, so I will not Lock it, but when SUBMITTING updates, I will judge whether others have updated the data during this period. Optimistic locking is applicable to scenarios where read is excessive and write is insufficient to improve throughput.

Optimistic locking: Data integrity violations are checked only at commit time, assuming no concurrency conflicts will occur.

Optimistic locking generally has the following two ways:

  1. This is implemented using the Version logging mechanism, which is the most common implementation of optimistic locking. What is data version? This is to add a version identifier to the data, typically by adding a numeric version field to the database table. When the data is read, the value of the Version field is read together, incrementing the version value with each update of the data. When we submit the update, compare the current version information of the database table with the version value extracted for the first time. If the current version number of the database table is equal to the version value extracted for the first time, it will be updated; otherwise, it is considered as expired data.
  2. Use a timestamp. The second way to implement optimistic locking is similar to the first way. The same way is to add a field in the table that needs optimistic lock control. The name does not matter, and the field type is timestamp, similar to the above version. Also check the timestamp of the data in the current database and compare it with the timestamp obtained before the update when the update is submitted. If the timestamp is consistent, it is OK; otherwise, it is version conflict.

Optimistic locking scheme is widely used because it solves the problem of high concurrent idempotence without affecting system performance. The only drawback is that it is intrusive to the code.

A distributed lock

For distributed systems, multiple systems run independently, so synchronous locking is definitely not possible; For distributed system, optimistic lock or distributed lock can be used to solve the idempotent problem.

Specific plans are as follows:

  1. Distributed lock based on cache (Redis, etc.);
  2. Distributed lock based on Zookeeper;

(Note: We will provide a video tutorial of the specific implementation scheme in the next period, thank you for your attention)

Video tutorial on solving idempotency based on “Optimistic Lock”

Thank you brothers and sisters attention, feng elder brother in order to everyone can more profound grasp “optimistic lock” to solve the idempotent problem, specially recorded a video tutorial. The main account recharge as an example, using IDEA development tool, database Mysql5.7, Demo based on Springboot + Mybatis architecture, using JMeter test tool simulation, high concurrency, to test the idempotent problem, that is, the account is repeated recharge scenario. The idempotent problem is then solved with an optimistic locking solution based on the state machine version field, which also comes with complete code.

The paper come zhongjue shallow, and must know this to practice.

It takes a lot of practice and thinking.

B station video tutorial online address

About feng brother

[author] : Feng Ge [wechat] : Java9568 (add friends, please note CSDN) [public number] : Java1234. Welcome everyone to pay attention to ~ 【 author introduction 】 : Jiangsu Normal University computer department, Java senior old driver, has been the State Grid power, many small companies at the front line of the code; At present, I have started a studio in Nantong, my hometown. I have settled the house, car, wife and children. Hope to become friends with all readers; Discuss Java technology and Java entrepreneurship together;