The background,

The periodic FULLGC problem in an application production environment was analyzed last time, and the specific reference was to the periodic FULLGC problem (1) of the application true line. However, the cause of locating this problem was also quite coincidental. I knew the result, and then I went to prove that the app’s periodic fullGC was caused by the problem; At the end of the preceding paragraph, several methods of forward screening are also given. The stack information at fullGC is printed to verify that CXF is indeed the cause. This analysis uses the open source tool BTrace to pinpoint the method calling System.gc;

Ii. Introduction to BTrace

When a fault occurs in the production environment, obtain system running data, such as method parameters, return values, global variables, and stack information. To capture this data, you need to modify the code, output the data to a log file, and publish it to the production environment. In this way, on the one hand, the cost and cycle of locating problems will increase, and urgent problems cannot be located and solved in time. On the other hand, the redeployment environment is largely destroyed, making it difficult to reproduce the problem. BTrace came into being in this context. Byte Trace (Btrace) is a Java dynamic and secure tracing (monitoring) tool developed by Sun. It can monitor the running status of the system without shutting down, with minimal intrusion and occupation of system resources. BTrace works by dynamically (bytecode) detecting classes of running Java programs. BTrace inserts trace operations into the classes of the running Java program and hot-swaps the traced program classes. Github:github.com/btraceio/bt… . You cannot create objects, use arrays, throw or catch exceptions, use loops, use the synchronized keyword, and use static attributes and methods in scripts. For details about these restrictions, refer to the user manual. If you want to know more, you can find yourself online, more relevant information;

3. Use BTrace locally

3.1 First of all, you need to download the jar package related to Btrace. Currently, the latest one on Github is V2.0.0. You can also download the source code and compile it by yourself (but the source code is not maven project but Gradle project).Btrace (org.openJdk. Btrace); openJdk (org.openJdk. Btrace); See github for more details on the differences; 3.2 Configuring Environment Variables

sudo vi ~/.bash_profile Source ~/.bash_profile Makes the configuration take effect Echo $PATH To check whether the configuration takes effect3.3 Writing a Listening Script BtraceTest is a listener class. The monitor method is used to monitor all methods that call System.gc and print out the call stack. Of course, there are many other uses, please check the related information, this article will not be extended; It is better to use only the bTrace and JDK classes in the script. If you use the other classes, you may fail to find the class. ; Note: Scripts do not need to be placed in the projects that need to be checked, but can be created in any directory. In this way, using BTrace can achieve zero intrusion to business codes and no perception to business codes. In this way, the system operation can be monitored without downtime as mentioned above. 3.4 Running the Script Run the application to be observed first. JPS obtains the corresponding process number as shown in Figure 5, and the running script is shown in Figure 6: /Users/xxx/Documents/btrace-bin/bin/btrace -v 12595 /Users/xxx/IdeaProjects/myProjects/easyexcel/easyexcel-boot/src/main/java/cn/com/lyd/easyexcel/boot/scripts/BtraceTest.j Ava (process number inconsistent, because the screenshot time is different, ignore ha…) Figure 6 shows that it is an internal static Daemon of sun.misc.GC, which is a Daemon thread:Figure 7 is consistent with the last analysis. For specific source code analysis, please refer to itApplication of Periodic FULLGC problem (1); Then, where is the thread created? Next, two methods are used to analyze and locate the thread: Method 1:According to the call link, the Gc class uses the requestLatebcy method in Figure 8 to create Daemon threads.Figure 9 will see requestLatebcy method has two calls, respectively is JreMemoryLeakPreventionListener JDKBugHacks figure 10 and figure 11 The last timeApplication of Periodic FULLGC problem (1)In the requestLatebcy class, doHacks are used to create threads in the requestLatebcy class. In the JDKBugHacks class, doHacks are used to create threads in the requestLatebcy class. In addition to the normal method described above, you can also use the btrace method described in this article to add a listening script According to the call link printed by BTrace in Figure 13, it is clear that the jdkbugs. doHacks of the CXF package under Apache created the thread. Figure 14. The Spring framework calls the classes under the CXF package during initial loading, which is why the package was introduced. The reason why the jdkbugs.dohacks method is called is not used in the Inquiry project;Execute the script, the project was first started without fullgc, this is confusing, actually really line every time to start is to have fullgc once, but it does not appear, the local view source code analysis, is due to GC. MaxObjectInspectionAge (), As shown in Figure 15, the value will determine whether system. gc will be executed for the first time. The value is different every time it is started.