“This is the 17th day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

A concept,

Idempotence was originally a concept in mathematics, but was later used in computers to indicate that any number of requests will have the same result as a single request. In other words, an interface will get the same result no matter how many times it is called. For example:

public class IdempotentExample {
    / / variable
    private static int count = 0;
    /** * non-idempotent method */
    public static void addCount(a) {
        count++;
    }
    /** * idempotent method */
    public static void printCount(a) { System.out.println(count); }}Copy the code

In the case of count, if you call addCount() repeatedly, the value of count will accumulate because addCount() is nonidempotent; The printCount() method is just used to print console information. So it gets the same result no matter how many times it’s called, so it’s idempotent.

Idempotence implementation schemes are usually divided into the following categories:

  • The front intercept
  • Use databases for idempotency
  • Idempotent using JVM locks
  • Idempotent using distributed locks

1. Front-end interception

Front-end interception refers to request interception through the page of the Web site. For example, after the user clicks the “submit” button, we can set the button as unavailable or hidden to avoid repeated clicking by the user. The core implementation code is as follows:

<script>
    function subCli(){
        // The button is set to unavailable
        document.getElementById("btn_sub").disabled="disabled";
        document.getElementById("dv1").innerText = "The button has been clicked.";
    }
</script>
<body style="margin-top: 100px; margin-left: 100px;">
    <input id="btn_sub" type="button"  value=" 提 交 "  onclick="subCli()">
    <div id="dv1" style="margin-top: 80px;"></div>
</body>
Copy the code

However, there is a fatal problem with front-end interception. If a knowledgeable programmer or hacker can directly bypass the JS execution of the page and simulate the interface of the request backend, then our front-end interception will not work. So in addition to front-end interception of some normal misoperations, back-end validation is essential.

2. Database implementation

There are three ways to realize idempotent database:

  • Idempotency is achieved through pessimistic locking
  • Idempotency is achieved through unique indexes
  • Idempotency is achieved through optimistic locking

3. JVM lock implementation

JVM Lock implementations are idempotent through jVM-provided built-in locks such as Lock or synchronized. The general process of using JVM Lock to achieve idempotency is as follows: first Lock the code segment through Lock, then determine whether the order has been processed, if not, start transaction execution order processing, commit the transaction after processing and release the Lock, the execution process is as follows:

The biggest problem with JVM locking is that it can only be used in a single-machine environment, because the Lock itself is a single-machine Lock, so it is not suitable for a distributed multi-machine environment.

4. Distributed lock implementation

The logic for realizing the idempotency of distributed lock is to determine whether the distributed lock can be obtained before each execution of the method. If yes, it indicates that the method is executed for the first time. Otherwise, the request can be directly discarded. We usually use Redis or ZooKeeper to implement distributed locks; If Redis is used, the set command is used to create and obtain distributed locks as shown in the following example:

127.0.0.1:6379> set lock true ex 30 nx OKCopy the code

Ex is used to set the timeout period. Nx stands for not exists, which determines whether a key exists. If OK is displayed, the lock is successfully created. Otherwise, the request is repeated and should be discarded.

Analysis of the

Idempotence problem seems to be on the “tall” in fact, namely how to avoid the problem of repeated request submission, for reasons of safety, we must be verified idempotence in front and back side, at the same time idempotence problems in daily work and is particularly common, solutions are also many, but considering the distributed system, we should give priority to use distributed lock.

extension

1. Idempotent considerations

The realization and judgment of idempotency need some resources. Therefore, it is not necessary to add idempotency judgment to every interface. It should be differentiated according to the actual business situation and operation type. For example, we do not need to make idempotent judgments when performing query operations and delete operations. The result of a query operation is the same as that of a query operation, so we do not need to make idempotent judgment. Delete operation is the same, delete once and delete many times is to delete the related data (delete here refers to the conditional deletion rather than delete all data), so there is no need to perform idempotent judgment.

2. Key steps of idempotence

The key steps for realizing idempotence are divided into the following three steps:

  • Each request operation must have a unique ID, and this ID is the Key evidence used to indicate whether the business has been executed. For example, the order ID is used as the Key of idempotency verification for the request of order payment business.
  • Check whether the service has been processed before each service execution.
  • After the first transaction is completed, the state of the transaction should be saved, such as in Redis or a database, to prevent the transaction from being processed repeatedly.

3. Database idempotency

There are three ways to achieve idempotency using a database:

  • Idempotency is achieved through pessimistic locking
  • Idempotency is achieved through unique indexes
  • Idempotency is achieved through optimistic locking

1. The pessimistic locking

Use pessimistic lock to achieve idempotent, generally with the transaction to achieve, when not using pessimistic lock, we usually execute the process is like this, first to judge the state of the data, execute SQL as follows:

select status from table_name where id='xxx';
Copy the code

Finally, modify the state:

update table_name set status='xxx';
Copy the code

However, this situation, because of non-atomic operations, can cause the problem of a business being executed twice in a high concurrency environment, when one program is executing and the other program also starts a state determination operation. Because the first program didn’t have time to change state, the second program was able to execute successfully, resulting in one business being executed twice.

In this case we can use pessimistic locks to avoid the problem, implementing SQL as follows:

begin;  # 1.Start the transactionselect * from table_name where id='xxx' for update; # 2.State of the queryinsert into table_name (id) values ('xxx'); # 3.Add operation update table_nameset status='xxx'; # 4.Change the operationcommit; # 5.Commit the transactionCopy the code

In the implementation process, the following two issues need to be paid attention to:

  • If you use a MySQL database, you must use the InnoDB storage engine because InnoDB supports transactions.
  • The ID field must be the primary key or unique index; otherwise, the table will be locked and other services will be affected.

2. Unique index

We can create a uniquely indexed table to achieve idempotency, and perform an insert before each transaction, because the unique field is the ID of the transaction, so repeated inserts will trigger a unique constraint and cause the insert to fail. In this case (insert failure) we can judge it as a duplicate request.

The following is an example of creating a unique index table:

CREATE TABLE `table_name` (
  `id` int NOT NULL AUTO_INCREMENT,
  `orderid` varchar(32) NOT NULL DEFAULT ' ' COMMENT 'the only id'.PRIMARY KEY (`id`),
  UNIQUE KEY `uq_orderid` (`orderid`) COMMENT 'Unique constraint'
) ENGINE=InnoDB;
Copy the code

3. Optimistic locking

Optimistic locking means that the data operation (change or add) is locked at the time of execution, but not at other times, so it is much more efficient than pessimistic locking, which is locked throughout execution.

Optimistic locking can be implemented with a version number, such as the following SQL:

update table_name set version=version+1 where version=0;
Copy the code

summary

Idempotence can not only ensure the normal execution of the program, but also prevent some garbage data and invalid requests from consuming system resources. The six implementation methods of idemidemality are introduced above, including front-end interception, database pessimistic lock implementation, data unique index implementation, database optimistic lock implementation, JVM lock implementation, and distributed lock implementation, among which front-end interception can not prevent people who know the business directly bypass the front-end to simulate the operation of the request. Therefore, idempotent processing must be implemented on the back end, and distributed locking is recommended as a more general solution.