One, foreword
By repeated requests, we mean that our server receives multiple repeated requests with the same content within a short period of time. Such repeat request if it is idempotent (every request results are the same, such as query requests), that in fact there is no effect for us, but if be idempotent (each request will affect key data, such as delete relationships, relationships, etc.), it will light produces dirty data, or cause the system error.
Therefore, in the current situation of universal distributed services, how to avoid and solve the data anomaly caused by repeated requests has become an urgent problem to be solved. The best way to avoid repeated requests is to do both ends together.
1. The front end or client directly prohibits the submission of repeated requests on non-idempotent buttons.
2. The backend locks the device when receiving the request and unlocks the device after the request is received.
This blog post is about coming up with a generic solution for repeated requests based on the concept of distributed locks at the back end.
Second, the body
Why use distributed locks? Because our current common architecture is distributed server, the front-end requests are forwarded to the back-end through the gateway layer, as shown in the figure below. Therefore, it is impossible to deal with the frequent repeated requests in distributed services if the restriction is made only on a single server.
The basic idea
Add a distributed lock to the interface that needs to prevent repeated requests, as follows:
- After receiving the request, the md5 value is obtained according to the method name + parameter to obtain the method and the unique identifier of the parameter.
- After obtaining the identity, set the distributed lock and set the expiration time;
- At the end of the request, the distributed lock is released.
This completes the disallow of repeated requests for the current request. If you want to make a general solution, you need to make a small function out of the above steps, because I am familiar with Java, Spring framework, take this to do an example.
Based on spring section, redis implementation
Some students familiar with Spring already know what I want to use, do general type, must use spring AOP features, annotation + section + MD5key + reflection + Redis implementation, specific as follows:
- Define a distributed lock annotation with expiration time Settings and ignore parameters.
- Define a section, the cutting point is distributed lock annotation, in the section to obtain the method name, parameters, expiration time of the need to use distributed lock, and the method name and not ignored parameters as md5 to get unique identification;
- Then set redSIS distributed lock according to the unique identifier above;
- Unlock after the method is complete.
The code is as follows:
annotations
Define an annotation named RepeatOperationLock with lock expiration time and ignore property (i.e. property not participating in MD5 calculation of distributed lock identifier).
@Documented @Inherited @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.METHOD}) @Component public @interface RepeatOperationLock {/** * lock duration, default 500ms * @return*/ long timeOut() default 500; /** * ignore the lock parameter position, starting from 0 * @return
*/
int[] ignoreIndex();
}
Copy the codesection
The cut point is the annotation above. The following things are done in the section: get the method name, get the annotation attribute (expiration time, ignore attribute), calculate the MD5 value of the method + attribute, and call the method of the external distributed lock.
@Aspect
@Slf4j
@Component
public class LockAspect {
@Autowired
RepeatLockService repeatLockService;
@Pointcut("@annotation(com.ls.javabase.aspect.annotation.RepeatOperationLock)")
public void serviceAspect() {
}
@Before("serviceAspect()")
public void setLock(JoinPoint point) {
log.info("Prevent method from repeatedly calling interface lock, lock,point:{}", point);
Method method = ((MethodSignature) point.getSignature()).getMethod();
RepeatOperationLock repeatOperationLock = method.getAnnotation(RepeatOperationLock.class);
if (Objects.isNull(repeatOperationLock)) {
log.warn("---repeatOperationLock is null---");
return;
}
long timeOut = repeatOperationLock.timeOut();
int [] ignoreIndex = repeatOperationLock.ignoreIndex();
log.info("LockTime - {}", timeOut);
if (Objects.isNull(timeOut)) {
log.warn("---timeOut is null");
return;
}
String methodName = method.getName();
Object[] args = point.getArgs();
repeatLockService.setRepeatLock(methodName, args, timeOut);
}
@After("serviceAspect()")
public void removeLock(JoinPoint point) {
log.info("Prevent method from repeatedly calling interface lock, unlock,point:{}",point);
Method method = ((MethodSignature) point.getSignature()).getMethod();
RepeatOperationLock repeatOperationLock = method.getAnnotation(RepeatOperationLock.class);
if (Objects.isNull(repeatOperationLock)) {
log.warn("---repeatOperationLock is null---");
return;
}
long timeOut = repeatOperationLock.timeOut();
log.info("LockTime - {}", timeOut);
if (Objects.isNull(timeOut)) {
log.warn("---timeOut is null");
return;
}
String methodName = method.getName();
Object[] args = point.getArgs();
repeatLockService.removeRepeatLock(methodName, args);
}
/**
*
* @param args
* @param ignoreIndex
* @return
*/
private Object [] getEffectiveArgs(Object[] args,int [] ignoreIndex) {
for (int i:ignoreIndex){
args[i] = null;
}
for (Object obj:args){
if (obj==null){
}
}
returnargs; }}Copy the codeMd5 method
public class Md5Encode {
/**
* constructors
*/
private Md5Encode() {} /** * @param s needshashThe string * @return hashPublic static final String md5(final String s) {char[] hexDigits = {'0'.'1'.'2'.'3'.'4'.'5'.'6'.'7'.'8'.'9'.'a'.'b'.'c'.'d'.'e'.'f'}; try { byte[] btInput = s.getBytes(Charset.defaultCharset()); / / get the MD5 digest algorithms MessageDigest object MessageDigest mdInst = MessageDigest. GetInstance ("MD5"); // Update the digest mdinst.update (btInput) with the specified byte; Byte [] md = mdinst.digest (); byte[] md = mdinst.digest (); // Convert the ciphertext to a hexadecimal string form int j = md.length; char[] str = new char[j * 2]; int k = 0;for (int i = 0; i < j; i++) {
byte byte0 = md[i];
str[k++] = hexDigits[byte0 >>> 4 & 0xf];
str[k++] = hexDigits[byte0 & 0xf];
}
return new String(str);
} catch (Exception e) {
e.printStackTrace();
returnnull; }}}Copy the codeA distributed lock
Here distributed lock using Redis, such as lock picture book misunderstanding, will make improvements later, to achieve a complete distributed lock scheme, written in the blog.
@Slf4j
@Service
public class RepeatLockService {
@Autowired
RepeatRedisUtil repeatRedisUtil;
public void setRepeatLock(String methodName, Object[] args, Long expireTime) {
for (Object obj : args) {
log.info("Method name :{}, object :{}, object hashCode :{}", methodName, obj, obj.hashCode());
}
Boolean lock = repeatRedisUtil.setRepeatLock(methodName, args, expireTime);
if(! lock) { log.info("Same request already made");
}
}
public void removeRepeatLock(String methodName, Object[] args) {
repeatRedisUtil.removeRepeatLock(methodName, args);
}
}
@Component
public class RepeatRedisUtil {
@Autowired
RedisTemplate redisTemplate;
private static final String repeatLockPrefix = "repeat_lock_"; Redis distributed lock * @param methodName * @param args * @param expireTime expiration time ms * @return
*/
public boolean setRepeatLock(String methodName, Object[] args,long expireTime) {
String key = getRepeatLockKey(methodName, args);
try {
boolean b = (boolean) redisTemplate.execute(new RedisCallback<Boolean>() {
@Override
public Boolean doInRedis(RedisConnection connection) throws DataAccessException {
Jedis jedis = null;
try {
jedis = (Jedis) connection.getNativeConnection();
String status = jedis.set(key, "1", NX, EX, expireTime);
if (setNXStatus.equals(status)) {
return Boolean.TRUE;
}
returnBoolean.FALSE; }finally { connection.close(); }}});return b;
} catch (Exception e) {
log.error(Redis operation exception :{},e);
returnBoolean.FALSE; @param args public void removeRepeatLock(String methodName, String methodName) Object[] args){ String key = getRepeatLockKey(methodName, args); redisTemplate.delete(key); } /** * get the repeat request lock Key ** @param methodName * @param args * @return
*/
public String getRepeatLockKey(String methodName, Object[] args) {
String repeatLockKey = repeatLockPrefix + methodName;
for (Object obj : args) {
repeatLockKey = repeatLockKey+"_"+ obj.hashCode();
}
returnrepeatLockKey; }}Copy the codeTesting the Service interface
Annotate the method to indicate an expiration time of 200000ms, ignoring the second parameter.
@Slf4j
@Service
public class TestLockService {
@RepeatOperationLock(timeOut = 200000, ignoreIndex = 1)
public void testLock(UserDto userDto,int i){
log.info("Attributes in service :{},{}",userDto,i);
log.info("Service hashcode, userDto: {}, I: {}",userDto.hashCode(),i); }}Copy the codeconclusion
Such a spring-based general distributed lock solution share finished, do still exist some defects, such as unlock when there is no judgment will be misunderstood, etc., the subsequent will specially make a summary and improvement of distributed lock, it is proposed based on a distributed lock solve the thought of repeated requests, also hope to communicate more.