What is idempotent?

Take a look at what Wikipedia says:

Idempotent: Calling a method or interface multiple times does not change the business state, ensuring that the result of repeated calls is the same as the result of a single call.

Use idempotent scenarios

1. Repeated front-end submissions

User registration, user creation of goods and other operations, the front end will submit some data to the background service, the background needs to create records in the database according to the data submitted by the user. If the user accidentally orders several times and the back end receives several commits, multiple records will be created repeatedly in the database. This is the bug of not having idempotent interfaces.

2. The interface times out and retry

An interface called to a third party may fail to be called due to network reasons. In this case, a retry mechanism will be added to the interface call. If a network exception occurs halfway through the first call. The next call will result in a call exception due to the presence of dirty data.

3. Repeated message consumption

When message-oriented middleware is used to process message queues and a manual ACK confirms that messages are properly consumed. If the consumer suddenly disconnects, the half-executed message is put back on the queue.

When the message is re-consumed by other consumers, if there is no idempotency, it will lead to abnormal results when the message is re-consumed, such as database duplicate data, database data conflict, resource duplication, etc.

Iii. Solutions

1. Token mechanism implementation

The idempotency of the interface is realized by token mechanism, which is a relatively universal implementation method.

The schematic diagram is as follows:

Specific process steps:

  • The client will first send a request to obtain the token, and the server will generate a globally unique ID stored in Redis as the token, and return this ID to the client
  • The client must carry this token the second time it invokes a business request
  • The server verifies the token. If the verification succeeds, services are executed and the token in redis is deleted
  • If the verification fails, there is no corresponding token in redis. The operation is repeated and the specified result is directly returned to the client

Note:

  • It is recommended to use Lua script to ensure the atomicity of the token in Redis and the deleted code logic
  • The global unique ID can be generated by baidu’s UID-Generator and Meituan’s Leaf

2. Implementation based on mysql

This approach takes advantage of mysql’s unique index feature.

The schematic diagram is as follows:

Specific process steps:

  • Create a de-duplicate table where a field requires a unique index
  • The client requests the server, and the server inserts some information about the request into the deduplication table
  • Because a field in the table has a unique index, if the insert is successful, the table has no information about this request, and the subsequent business logic is executed
  • If the insert fails, the current request has been executed and is returned

3. Based on Redis

This implementation is based on SETNX command implementation

SETNX key Value: Sets the key value to value if and only if the key does not exist. If the given key already exists, SETNX does nothing.

This command returns 1 on success and 0 on failure.

The schematic diagram is as follows:

Specific process steps:

  • The client requests the server first and gets a unique field that represents the requested business
  • This field is stored in Redis in SETNX mode, and the corresponding timeout time is set according to the business
  • If the setup is successful and proves that this is the first request, the subsequent business logic is executed
  • If the setting fails, it indicates that the current request has been executed

conclusion

In fact, these several idempotent ways are almost the same, similar to the use of state machines, pessimistic locks, optimistic locks to achieve, are relatively simple.

When designing an interface, idempotent is the most important thing to consider, especially when designing an interface that involves money, such as money transfer and payment.