1 the introduction

In real development, you often encounter situations where no matter how many times you do something, it should have the same effect or return the same result. Such as:

  • When a user initiates a payment request, the user’s balance should only be deducted once, whether due to network retransmission or a system bug
  • Send the notification message only once. If the user receives the notification repeatedly, the customer will resent it
  • Commission is calculated for promoters in the distribution system, and the commission can only be deposited once in each cycle, which will inevitably cause financial losses if it is deposited repeatedly

2 Concept of idempotence

Idempotence is a mathematical and computer concept commonly found in abstract algebra. In computer programming, an idempotent operation is characterized by any number of executions producing the same effect as a single execution. Idempotent operations in more complex scenarios are often implemented with unique transaction numbers (serial numbers).

3. Technical implementation scheme of idempotent operation

3.1 Query Operations

If the data is not changed, the query result is the same as if the data is not changed. SELECT is a natural idempotent operation.

3.2 Deleting operations

The DELETE operation is also idempotent. Deleting a number of items once has the same effect as deleting a number of items multiple times.

3.3 Unique Index

Prevent new dirty data. Pay treasure to user account, for example, when a new user for a phone number became new users need to create a paypal account, give a phone number set a unique index, thus preventing the same mobile phone number has been repeated registration (when using a unique index, in order to prevent the INSERT times wrong affect user experience, so you need to use the try {when INSERT… } the catch () {… } body and catch the exception DuplicateKeyException).

3.4 pessimistic locks

Lock when fetching data. SELECT * FROM table t WHERE t.id=#{} for update; Note that the id field must be the primary key or the unique index, otherwise the table will lock, high concurrency will cause service downtime. Pessimistic locks are typically used with transactions.

3.5 optimistic locking

Optimistic locking only locks the table at the moment of update and does not lock the table at other times, so it is more efficient than pessimistic locking. Optimistic locking can be implemented with version numbers and other field states:

1. UPDATE table SET username=#{newVal},version=2 WHERE username=#{originVal},version=1
2. UPDATE table SET username=#{newVal} WHERE username=#{originVal}
Copy the code

The second implementation is suitable for scenarios where no version number is used and only data security checks are performed when updating, and it is suitable for the inventory model: for example, share deduction and share rollback, performance is higher. Update operation of optimistic lock, it is better to use primary key or unique index update, which is row lock, otherwise update will lock table. The following statement is better:

1. UPDATE table SET id=#{newVal},version=2 WHERE id=#{originVal},version=1
2. UPDATE table SET id=#{newVal} WHERE id=#{originVal}
Copy the code

3.6 Distributed Lock

As long as distributed lock is to solve the problem that a process can only be executed by one system under distributed conditions, it is difficult to construct a global unique index at this time, for example, unique field is not easy to determine, so it is necessary to introduce distributed lock. If the only index in Section 3.3 is to intercept requests downstream, distributed locking is to intercept requests upstream. The implementation mechanism is as follows: the distributed lock is constructed by Redis or Zookeeper (such as setnx() instruction of Redis). When the service system inserts or updates data, the distributed lock needs to be obtained first. Since the lock can only be obtained by one system, other systems will reject the distributed lock.

3.7 token mechanism

The main purpose is to prevent page duplication.

Note: The token needs to be applied for each time and can be valid only once. You can also use SELECT+DELETE to verify tokens, but this method has concurrency problems and is inefficient.

3.8 SELECT + INSERT

The background system with low concurrency supports idempotent and repeated execution. The simple logic is: first check the key data, determine whether it has been executed, and then perform business processing. However, this approach does not support high concurrency scenarios.

3.9 Apis provided externally

For example, the payment interface provided by Alipay: In order to ensure idempotency, two parameters need to be attached to the request submitted by the customer: source (source) and SEQ (serial number). Use source+seq to combine unique indexes in the database to prevent duplicate support.

4 summary

Idempotent operations have nothing to do with whether they are distributed or highly concurrent. Idempotence is a gene of a qualified programmer, which is the primary consideration when designing a system, especially in systems involving money, such as Alipay, banks and Internet finance companies, which should be both efficient and ensure data accuracy.

A little attention, won’t get lost