These days, when I was investigating an out-of-heap memory leak problem, I saw many people mentioned the magic tool GperfTools. They wanted to try it and found it was not friendly to macOS support. And most of the tutorials are for C++, there is a compilation link operation to see my Java boy dazzling. So I put together a tutorial for the MAC and Java versions so that you don’t have to step on it again.

A list,

Gperftools is a set of analysis tools provided by Google, including heap memory detection hep-profiler, memory leak analysis tool hep-Checker, and CPU performance monitoring tool CPU-Profiler. It is well known that out-of-heap memory leaks are difficult to trace, and dump analysis tools such as MAT can only analyze leaks from the largest or most objects in the heap. Gperftools replaces malloc calls with its own TCMALloc, which counts all memory allocation behavior and helps us locate leaks more quickly.

Second, the installation

Just use Homebrew to install it.

brew install gperftools
Copy the code

3. Use GperfTools to locate memory leaks

1. Sample program

We simulate a Native Memory leak scenario using the following code, which allocates Memory using the Native method and holds its reference by default using SoftReference. Therefore, if a large number of objects live in the heap and do not trigger the Full GC, their Native Memory will not be released and eventually run out of Memory of the physical machine.

The code address

public class NativeMemoryLeakDemo {

    public static void main(String[] args) throws IOException, FontFormatException {
        while (true) {
            test(a); } } private static voidtest() throws IOException, FontFormatException {
        Resource resource = new ClassPathResource("font/font.ttf");
        Font rawFont = Font.createFont(Font.TRUETYPE_FONT, resource.getFile());
        Font usedFont = rawFont.deriveFont(Font.PLAIN, 30);

        BufferedImage bufferedImage = new BufferedImage(100, 100, BufferedImage.TYPE_INT_ARGB);
        Graphics2D g2 = bufferedImage.createGraphics();
        g2.setFont(usedFont);
        g2.drawString("hello world", 16, 35); }}Copy the code

Let’s run it for a while with the following VM parameters (Java8)

-XX:CMSInitiatingOccupancyFraction=80
-XX:CompressedClassSpaceSize=528482304
-XX:InitialHeapSize=3221225472
-XX:MaxDirectMemorySize=536870912
-XX:MaxHeapSize=3221225472
-XX:MaxMetaspaceSize=536870912
-XX:MaxNewSize=1157627904
-XX:MetaspaceSize=536870912
-XX:NewSize=1157627904
-XX:SurvivorRatio=8
Copy the code

As you can see from the graph, the process is taking up much more memory than we can allocate. It is clear that there is a memory leak. So let’s see how we can use the heap-profiler tool provided by GperfTools to locate where a memory leak is occurring.

2. Use heap_profiler to locate memory leaks

1) Replace malloc with TCmalloc

Open the following

vi ~/.bash_profile
Copy the code

Specify the PATH to the TCMALloc library and add it to the PATH

export DYLD_INSERT_LIBRARIES=<gperftools_lib_path>/lib/libtcmalloc_and_profiler.dylib
Copy the code

Where < gperftools_lib_path > gperftools is installed on the machine, such as I’m with homebrew is installed in/usr/local/Cellar/gperftools / 2.7 / down, that is my path

export DYLD_INSERT_LIBRARIES=/usr/local/ Cellar/gperftools / 2.7 / lib/libtcmalloc_and_profiler dylibCopy the code

Save and take effect configuration (requires IDE restart)

source ~/.bash_profile
Copy the code

Note: Replacing malloc here does not run heap-profiler, however since anyone can start heap-profiler after adding environment variables, Google does not recommend it in a production environment.

2) Monitor memory allocation

Import or create our sample program in Idea and add environment variables for the heap-profiler run in the run Settings

HEAPPROFILE=<heap_output_path>
Copy the code

<heap_output_path> is the output address of the heap file. For example, to output the result to the memTrack file in the TMP folder, is

HEAPPROFILE=/tmp/memTrack
Copy the code

Run the program, and you can see in the log that heap-profiler starts tracking memory allocations, with a default sampling rate of 100M per allocation.

Heap-profiler logs can also be seen in the/TMP directory.

3) Analyze the output

Heap-profiler uses pprof to convert the results into a variety of formats. Here are the TXT and PDF outputs respectively

Output TXT

Select the last sampling record memtrack.0026. heap, convert it to TXT file and output it to ~/HeapFile folder

pprof $JAVA_HOME/bin/java --text /tmp/memTrack.0026.heap > ~/HeapFile/memTrack.txt
Copy the code

The result is quite large, so here is the output of the Java section

Total: 2544.9 MB 2541.9 99.9% 99.9% 2541.9 99.9% 0x00007ffF6F5BB1BD 0.0 0.0% 100.0% 298.4 11.7% _JavaMain 0.0 0.0% 100.0% 0.0 0.0 0.0% _Java_com_apple_eawt_Application_nativeInitializeApplicationDelegate 0.0 0.0% 100.0% to 0.0% _Java_java_awt_image_BufferedImage_initIDs 0.0 0.0% 100.0% 0.0 0.0% _Java_java_awt_image_ColorModel_initIDs 0.0 0.0% 100.0% 0.0 0.0% _Java_java_awt_image_Raster_initIDs 0.0 0.0% 100.0% 0.0 0.0% _Java_java_awt_image_SampleModel_initIDs Java_java_io_unixfilesystem_checkaccess 0.0 0.0% 100.0% 0.1 0.0% _Java_java_io_UnixFileSystem_getBooleanAttributes0 0.0 0.0% 100.0% 0.3 0.0% _Java_java_lang_ClassLoader_00024NativeLibrary_load 0.0 0.0% 100.0% 0.1 0.0% _Java_java_lang_ClassLoader_defineClass1 0.0 0.0% 100.0% 0.1 0.0% _Java_java_lang_ClassLoader_findBootstrapClass 0.0 0.0% 100.0% 0.0 0.0% _Java_java_lang_Class_forName0 0.0 0.0% 100.0% 0.2 0.0% _Java_java_lang_System_initProperties 0.0 0.0% 100.0% 0.0 0.0% _Java_java_net_Inet6Address_init 0.0 0.0% 100.0% 0.0 0.0% _Java_java_net_NetworkInterface_init 0.0 0.0% 100.0% 0.0 0.0% _Java_java_net_PlainSocketImpl_initProto 0.0 0.0% 100.0% 0.0 0.0% _Java_java_net_PlainSocketImpl_socketConnect 0.0 0.0% 100.0% 0.9 0.0% _Java_java_util_zip_Inflater_inflateBytes 0.0 0.0% 100.0% 0.2 0.0% _Java_java_util_zip_Inflater_init 0.0 0.0% _Java_java_util_zip_ZipFile_getEntry 0.0 0.0% 100.0% 0.4 0.0% _Java_java_util_zip_ZipFile_open 0.0 0.0 0.0% 100.0% 0.0% 100.0% 0.0% _Java_sun_awt_CGraphicsEnvironment_registerDisplayReconfiguration 0.0 0.5 0.0% _Java_sun_awt_image_BufImgSurfaceData_initRaster 0.0 0.0% 100.0% 0.1 0.0% _Java_sun_font_CFontManager_loadNativeDirFonts Java_sun_font_strikecache_freeintmemory 0.0 0.0% 100.0% 0.4 0.0% _Java_sun_font_T2KFontScaler_createScalerContextNative 0.0 764.7 0.0% 100.0% to 30.0% _Java_sun_font_T2KFontScaler_getGlyphImageNative 0.0 0.0% 100.0% 0.0 0.0% _Java_sun_font_T2KFontScaler_initIDs 0.0 0.0% Java_sun_font_t2kfontscaler_initnativescaler 0.0 0.0% 100.0% 0.0 0.0% _Java_sun_java2d_SurfaceData_initIDs 0.0 0.0% 100.0% 0.0 0.0% _Java_sun_java2d_loops_GraphicsPrimitiveMgr_initIDs 0.0 0.0% 100.0% 0.4 0.0% _Java_sun_java2d_opengl_CGLGraphicsConfig_getOGLCapabilities 0.0 0.0% 100.0% 0.0 0.0% _Java_sun_java2d_opengl_OGLRenderQueue_flushBufferCopy the code

You can see that the first line is the total memory occupied by the entire program, followed by the memory usage of each method in order of call stack (in MB).

  • The first column is the Direct Memory used
  • The fourth column is the total memory used by the process and all the methods it calls
  • Columns 2 and 5 show the percentage of the memory in columns 1 and 4 to the total memory of the process, respectively
  • The third column is a summation of the second column

Since gperftools is a tool in C++, you can see that you don’t get complete monitoring information in Java. But we can still see from the fourth column that _Java_sun_font_T2KFontScaler_initNativeScaler is the method that takes up the most memory. If you look at the code, you can see that this method is modified by the native keyword. It is likely that the memory allocated here has not been reclaimed by the JVM. If you do a search, you can see that the memory allocated here is actually held by the Font2D object that ultimately caused the leak.

The output PDF

Pprof also supports graphic output of statistical results to PDF, making it easier to find the most memory footprint. Here also use memtrack.0026. heap, convert it to PDF format and export it to ~/HeapFile folder

pprof $JAVA_HOME/bin/java --pdf /tmp/memTrack.0026.heap > ~/HeapFile/memTrack.pdf
Copy the code

You can then see the generated PDF file under ~/HeapFile. It’s a big picture, so I’ll just take a part of it here.

It can be seen from the figure that the call stack of memory allocation is transformed into multiple call links, which eventually point to AllocMem for memory allocation, and the links with high memory ratio are carefully bolded.

Note: If you encounter any of the following errors while exporting the PDF, you will need to install the corresponding dependencies

Graphviz brew install Graphviz ps2pdfcommandNot found Brew install ghostscript is requiredCopy the code

Four,

Gperftools is a great tool for detecting memory leaks, providing out-of-heap memory monitoring capabilities. Unfortunately, it is a tool designed for C++, and many other features don’t seem to work in Java. I’ll stop here and see what other features you can use in Java.

The official documentation