concept

First, explain the concepts of memory overflow and memory leak. Out of memory (OOM) occurs in the heap, method area, and method stack. A memory leak refers to the failure of a program to release the allocated memory after it applies for memory space. As a result, the memory address is unreachable and the memory space is occupied forever in subsequent programs. For example, the goods cabinet in the mall is designed with 10 drawers. After each person uses them, they will be returned to the next user. If someone keeps using them and does not return them, other users can only use the remaining 9 drawers. Therefore, memory leakage is related to memory overflow. A memory leak does not have a big impact, and memory leakage will lead to memory overflow after accumulation.

There should be a picture here, so imagine the picture.

Demand background

A personal account system to C has about 2000 W of data; One day, the product suddenly said that it would build a to B merchant account system, which would be used in the promotion activity about a week later. The product has been blown out to the big boss, saying that our system has this capability and can support seamless immediately. Now they come to us in a hurry, asking whether it is ok to directly provide the ability of our to C account system.

Demand analysis

First of all, it is unrealistic to develop, test and launch a new ACCOUNT system to B in such a short time. No matter how strong our programmers are, they cannot produce the system by assembly line. We can only rely on the current account system to support this business first. The core domain model at the bottom of the two sets of account systems can be abstract and consistent, including accounting documents, accounting subjects, accounting flow. The difference lies in that the rules of entry and exit account are different in different business usage scenarios, so the domain model of rule layer needs customized configuration development. At the same time, there are more than 2000 W accounts in the system, and the daily increase flow is 10W level, so the amount of data is already large. In addition, the two sets of data in business will affect each other, so it is necessary to make a vertical table in business to isolate the two sets of data.

Implementation process

After the domain model and data storage scheme were finalized, we started to implement it immediately. At that time, in order to implement it more elegantly and have the least impact on the current BUSINESS of TOC, we deliberately added a judgment mark to the threadLocal of the current worker thread after uniformly intercepting the interface entry. It is used to identify whether it is a business request to C or a business request to B, and then the business rule layer carries out filtering through different filters, and finally in the database JDBC layer carries out corresponding DML processing for different tables through different tags. The implementation of the scheme was very fast, and the test was completed in two or three days. However, two problems were found in the test process. The first problem was that the data would sometimes be confused for no reason, and the data originally written in the structure of the TO B table would be written in the structure of the to C table. In addition, it was found that the memory monitoring curve was growing slowly during the pressure measurement, and a memory leak occurred.

Problem analysis

A close analysis of the source code revealed that the problem was in ThreadLocal. Each thread has a ThreadLocalMap data structure whose value is stored in the threadLocals variable of the current thread when the set method is used, and is retrieved from the threadLocals variable when the get method is used. So the value of a set in thread 1 is invisible to thread 2, and if you set it again in thread 2, it doesn’t affect the value in thread 1, so that the threads don’t interfere with each other

public void set(T value) { Thread t = Thread.currentThread(); ThreadLocalMap map = getMap(t); if (map ! = null) map.set(this, value); else createMap(t, value); } public T get() { Thread t = Thread.currentThread(); ThreadLocalMap map = getMap(t); if (map ! = null) { ThreadLocalMap.Entry e = map.getEntry(this); if (e ! = null) { @SuppressWarnings("unchecked") T result = (T)e.value; return result; } } return setInitialValue(); } ThreadLocalMap getMap(Thread t) { return t.threadLocals; } //Entry is a static inner class of ThreadLocalMap, Static class Entry extends WeakReference<ThreadLocal<? >> { /** The value associated with this ThreadLocal. */ Object value; Entry(ThreadLocal<? > k, Object v) { super(k); value = v; }}Copy the code

Memory leak analysis

When a thread calls ThreadLocal’s set method to set variables, the current thread’s ThreadLocalMap contains an Entry object. The key of this Entry is the reference to ThreadLocal, and the value is the set value. If the current thread exists without calling the remove method of ThreadLocal, and there are references to ThreadLocal elsewhere, The ThreadLocalMap variable of the current thread contains references to ThreadLocal variables and references to value objects that will not be released, causing a memory leak. But consider that if the ThreadLocal variable has no other strong dependencies and the current thread still exists, because the key in the thread’s ThreadLocalMap is weakly dependent, Weak references to ThreadLocal variables in the current thread’s ThreadLocalMap will be reclaimed during GC, but the corresponding values will still leak memory.

Data write error analysis

Java processes are started by an external Web container such as Tomcat. When the Web container is started, a thread pool is started, creating a core of initial threads that process one task and then move on to another. If the core thread has just finished processing a to B task and then continues processing a to C task, the variable marker in threadLocal is still a direct marker, which will cause the JDBC layer to continue routing to the to B table structure, resulting in data errors.

Repair processing

Once you understand the storage structure of threadLoca, create an AOP slice in the interface entry, and set a judgment flag in the current worker thread’s threadLocal by surrounding the notification before the method is processed. After the method is processed, manually remove the flag. Ensure that memory is freed correctly after each processing.

Thank you for attention

You can pay attention to the wechat public account “Roll back the code”, read the first time, the article continues to update; Focus on Java source code, architecture, algorithms, and interviews.