How does Taobao shopping cart use Spring Boot+Redis to carry thousands of repeated submissions in an instant?

In a real development project, an exposed interface is often faced with a large number of repeated requests submitted in an instant. If you want to filter out the repeated requests and cause damage to the business, you need to implement idempotent!

Let's explain the concept of idempotent:

Any number of executions has the same impact as a single execution. According to this meaning, the final meaning is that the impact on the database can only be one-time and cannot be processed repeatedly.

How to ensure its idempotence, there are usually the following means:

1. Establishing a unique index of the database can ensure that only one piece of data is finally inserted into the database

2. Token mechanism: obtain a token before each interface request, and then add the token in the header body of the next request for background verification. If the verification passes deleting the token, the next request will judge the token again

3. Pessimistic lock or optimistic lock, pessimistic lock can ensure that other sql cannot update data every time for update (when the database engine is innodb, the condition of select must be unique index to prevent locking the whole table)

4. First, query and then judge. First, query whether there is data in the database. If the existence certificate has been requested, directly reject the request. If it does not exist, it is the first time to enter and directly release.

The schematic diagram of redis realizing automatic idempotent:

Build the service Api of Redis

1. First, build a redis server.

2. It is also possible to introduce the redis state or Spring packaged jedis from springboot. The main api used later is its set method and exists method. Here we use the Spring boot packaged redisTemplate

/**
 * redis Tool class
 */
@Componentpublic
class RedisService {
    @Autowired
    private RedisTemplate redisTemplate;
​
    /**
     * Write cache * @ param key * @ param value * @ return
     */
    public boolean set(final String key, Object value) {
        boolean result = false;
        try {
            ValueOperations<Serializable, Object> operations = redisTemplate.opsForValue();
            operations.set(key, value);
            result = true;
        } catch (Exception e) {
            e.printStackTrace();
        }
        return result;
    }
​
    /**
     * Write cache set time effective * @ param key * @ param value * @ return
     */
    public boolean setEx(final String key, Object value, Long expireTime) {
        boolean result = false;
        try {
            ValueOperations<Serializable, Object> operations = redisTemplate.opsForValue();
            operations.set(key, value);
            redisTemplate.expire(key, expireTime, TimeUnit.SECONDS);
            result = true;
        } catch (Exception e) {
            e.printStackTrace();
        }
        return result;
    }
​
    /**
     * Determine whether there is a corresponding value * @ param key * @ return in the cache
     */
    public boolean exists(final String key) {
        return redisTemplate.hasKey(key);
    }
​
    /**
     * Read cache * @ param key * @ return
     */
    public Object get(final String key) {
        Object result = null;
        ValueOperations<Serializable, Object> operations = redisTemplate.opsForValue();
        result = operations.get(key);
        return result;
    }
​
    /**
     * Delete the corresponding value * @ param key
     */
    public boolean remove(final String key) {
        if (exists(key)) {
            Boolean delete = redisTemplate.delete(key);
            return delete;
        }
        return false;
    }
}

Custom annotation AutoIdempotent

Customize an annotation. The main purpose of defining this annotation is to add it to the method that needs to implement idempotent. If a method annotates it, it will implement automatic idempotent.

If the annotation is scanned by reflection in the background, the method will be processed to realize automatic idempotence. The meta annotation ElementType.METHOD indicates that it can only be placed on the method, and etentionPolicy.RUNTIME indicates that it is running.

@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface AutoIdempotent {
}

token creation and verification

1. token service interface

We create a new interface to create a token service. There are two main methods, one is used to create a token and the other is used to verify the token.

Creating a token mainly produces a string. When checking a token, it mainly conveys the request object. Why do you want to pass the request object?

The main function is to obtain the token in the header, and then check it. The specific error reporting information will be obtained and returned to the front end through the Exception thrown.

public interface TokenService {
    /**
     * Create token * @ return
     */
    public String createToken();
​
    /**
     * Verify token * @ param request * @ return
     */
    public boolean checkToken(HttpServletRequest request) throws Exception;
}

2. Service implementation class of token

The token refers to the redis service, creates a token, generates a random uuid string using a random algorithm tool class, and then puts it into redis (in order to prevent redundant retention of data, the expiration time is set to 10000 seconds here, depending on the business). If the token is put in successfully, the value of the token will be returned at last.

The checkToken method is to get the token from the header to the value (if the header can't get it, get it from paramter). If it doesn't exist, throw an exception directly. This exception information can be captured by the interceptor and then returned to the front end.

@Servicepublic
class TokenServiceImpl implements TokenService {
    @Autowired
    private RedisService redisService;
​
    /**
     * Create token * * @ return
     */
    @Override
    public String createToken() {
        String str = RandomUtil.randomUUID();
        StrBuilder token = new StrBuilder();
        try {
            token.append(Constant.Redis.TOKEN_PREFIX).append(str);
            redisService.setEx(token.toString(), token.toString(), 10000L);
            boolean notEmpty = StrUtil.isNotEmpty(token.toString());
            if (notEmpty) {
                return token.toString();
            }
        } catch (Exception ex) {
            ex.printStackTrace();
        }
        return null;
    }
​
    /**
     * Verify token * * @ param request * @ return
     */
    @Override
    public boolean checkToken(HttpServletRequest request) throws Exception {
        String token = request.getHeader(Constant.TOKEN_NAME);
        if (StrUtil.isBlank(token)) {// token does not exist in header            
            token = request.getParameter(Constant.TOKEN_NAME);
            if (StrUtil.isBlank(token)) {// token does not exist in parameter 
                throw new ServiceException(Constant.ResponseCode.ILLEGAL_ARGUMENT, 100);
            }
        }
        if (!redisService.exists(token)) {
            throw new ServiceException(Constant.ResponseCode.REPETITIVE_OPERATION, 200);
        }
        boolean remove = redisService.remove(token);
        if (!remove) {
            throw new ServiceException(Constant.ResponseCode.REPETITIVE_OPERATION, 200);
        }
        return true;
    }
}

Interceptor configuration

1. web configuration class

To implement WebMvcConfigurerAdapter, the main function is to add autoIdempotentInterceptor to the configuration class, so that we can enter the interceptor to take effect.

Pay attention to the @ Configuration annotation, so that it can be added into the context when the container is started

@Configurationpublic
class WebConfiguration extends WebMvcConfigurerAdapter {
    @Resource
    private AutoIdempotentInterceptor autoIdempotentInterceptor;
​
    /**
     * Add interceptor * @ param registry
     */
    @Override
    public void addInterceptors(InterceptorRegistry registry) {
        registry.addInterceptor(autoIdempotentInterceptor);
        super.addInterceptors(registry);
    }
}

2. Intercept processor

The main function is to intercept scan to AutoIdempotent to annotation to the method, then call the checkToken() method of tokenService to check if token is correct. If the exception is caught, render the abnormal information to json and return it to the front end.

/**
 * Interceptor
 */
@Componentpublic
class AutoIdempotentInterceptor implements HandlerInterceptor {
    @Autowired
    private TokenService tokenService;
​
    /**
     * Preprocessing * @ param request * @ param response * @ param handler * @ return * @ throws exception
     */
    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
        if (!(handler instanceof HandlerMethod)) {
            return true;
        }
        HandlerMethod handlerMethod = (HandlerMethod) handler;
        Method method = handlerMethod.getMethod(); //Scan marked by ApiIdempotment 
        AutoIdempotent methodAnnotation = method.getAnnotation(AutoIdempotent.class);
        if (methodAnnotation != null) {
            try {
                return tokenService.checkToken(request);// Idempotent check: if the check is passed, it will be released; if the check fails, an exception will be thrown, and a friendly prompt will be returned through unified exception handling 
            } catch (Exception ex) {
                ResultVo failedResult = ResultVo.getFailedResult(101, ex.getMessage());
                writeReturnJson(response, JSONUtil.toJsonStr(failedResult));
                throw ex;
            }
        } //Must return true, otherwise all requests will be blocked 
        return true;
    }
​
    @Override
    public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {
    }
​
    @Override
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
    }
​
    /**
     * Returned json value * @ param response * @ param json * @ throws exception
     */
    private void writeReturnJson(HttpServletResponse response, String json) throws Exception {
        PrintWriter writer = null;
        response.setCharacterEncoding("UTF-8");
        response.setContentType("text/html; charset=utf-8");
        try {
            writer = response.getWriter();
            writer.print(json);
        } catch (IOException e) {
        } finally {
            if (writer != null) writer.close();
        }
    }
}

test case

1. Simulate business request class

First of all, we need to get the specific token through the / get/token path through the getToken() method, and then we call the testIdempotence method, which annotates @ AutoIdempotent. The interceptor will intercept all requests. When it is judged that there is such annotation on the processed method, it will call the checkToken() method in the TokenService. If an exception is caught, it will be different The caller is often thrown.

Let's simulate the request as follows:

@RestControllerpublic
class BusinessController {
    @Resource
    private TokenService tokenService;
    @Resource
    private TestService testService;
​
    @PostMapping("/get/token")
    public String getToken() {
        String token = tokenService.createToken();
        if (StrUtil.isNotEmpty(token)) {
            ResultVo resultVo = new ResultVo();
            resultVo.setCode(Constant.code_success);
            resultVo.setMessage(Constant.SUCCESS);
            resultVo.setData(token);
            return JSONUtil.toJsonStr(resultVo);
        }
        return StrUtil.EMPTY;
    }
​
    @AutoIdempotent
    @PostMapping("/test/Idempotence")
    public String testIdempotence() {
        String businessResult = testService.testIdempotence();
        if (StrUtil.isNotEmpty(businessResult)) {
            ResultVo successResult = ResultVo.getSuccessResult(businessResult);
            return JSONUtil.toJsonStr(successResult);
        }
        return StrUtil.EMPTY;
    }
}

2. Using postman requests

First, access the get/token path to get the specific token:

Get the token and put it in the specific request to the header. You can see that the first request is successful. Then we request the second time:

The second request is a repetitive operation. It can be seen that the repeatability verification is passed. When the second request is repeated, we will only let it succeed for the first time. The second request is a failure:

summary

This paper introduces how to use springboot, interceptor and redis to implement interface idempotent gracefully.

For idempotent is very important in the actual development process, because an interface may be called by countless clients, how to ensure that it does not affect the background business processing, how to ensure that it only affects the data once is very important, it can prevent the generation of dirty data or disorderly data, but also reduce the amount of concurrency, which is a very useful thing.

The traditional method is to judge the data every time. This method is not intelligent and automatic, which is troublesome. Today's automation can also improve program scalability.

Source network, only for learning, if there is infringement, contact delete.

To facilitate your study, I also arranged a set of learning materials, including Java virtual machine, spring framework, java thread, data structure, design mode, etc., which are provided free of charge to students who love Java~

Pay attention to the official account [java circle].

file

Tags: Programming Redis Database JSON Java

Posted on Sat, 18 Apr 2020 01:06:43 -0700 by aaronxbond