Record the solution that the response time is too long

The analysis reason

First, paste the pseudo code that takes a long time

// Iterate over the data
for() {/ / to get idInteger id = .... ;// Determine whether the data exists in the database based on the ID
    if (true) {
    	// Update data if it exists
    } else {
        // If no, add a new data}}Copy the code

This is a very simple code, traversing the table data, data exists update, data does not exist in the insert new data. The reason is that in the process of connecting to the database, this piece of code will connect to the database twice as much as the amount of data. When there is too much data in the loop, the response time is too long.

Add the cache

In the part of judging whether the data exists, we need to visit the database to determine whether the data exists once every cycle. We can store the results of each visit with id as the key value, and directly take out the data from the cache when judging, so as to reduce the response time. The modified pseudocode is as follows

// Iterate over the data
for() {/ / to get idInteger id = .... ;if (cache(id)) {
    	// Update data if it exists
    } else {
        // If no, add a new data}}// Cache methods
boolean cache(Integer id){
	// Determine whether the data exists in the cache based on the id
	if (true) {returnCached data; }else {
		// Determine whether the data exists in the database according to the id
		// Store data in the cache
		returnData returned by the database; }}Copy the code

Of course, when using the cache, it is important to be aware of when the cache data needs to be updated. In this case, the cache data does not need to be updated. If data deletion is involved, the cache data needs to be updated.

Batch update and batch insert

We can not update or insert the data immediately after determining whether the data exists, but store the ids of the data to be updated and inserted in a list respectively. When all the ids are traversed, the <foreach> tag of Mybatis will be used for batch processing. The modified pseudocode is as follows

// Iterate over the data
// The id of the data to update
List<Integer> updateIdList = new ArrayList<>();
// Id of the data to be inserted
List<Integer> insertIdList = new ArrayList<>();
for() {/ / to get idInteger id = .... ;if (cache(id)) {
    	updateIdList.add(id);
    } else{ insertIdList.add(id); }}// Batch update
// Batch delete
// Cache methods
boolean cache(Integer id){
	// Determine whether the data exists in the cache based on the id
	if (true) {returnCached data; }else {
		// Determine whether the data exists in the database according to the id
		// Store data in the cache
		returnData returned by the database; }}Copy the code

conclusion

Successfully reduced the interface response time from 1 to 2s to 100ms through caching and batch processing. This solution is just a record of personal growth experience, not necessarily suitable for most scenarios, if you have a better solution or code can be optimized, please post the solution in the comments section.