Actual combat: OutOfMemoryError

In the Description of the Java Virtual Machine specification, OutOfMemoryError(OOM) exceptions are possible in several run-time areas of the virtual machine memory in addition to the program counter, which will be verified by several instances (Listing 2-3 to listing 2-9). It also introduces some of the most basic vm parameters related to memory.

The purpose of the content is twofold: first, to validate through code what is stored in the various runtime areas described in the Java Virtual Machine specification; Secondly, it is hoped that when readers encounter actual memory overflow exceptions in their work, they can quickly judge which area of memory overflow according to the information of exceptions, know what kind of code may cause memory overflow in these areas, and how to deal with these exceptions.

Here are some commonly used GC debug parameter, each in the official document has explanation docs.oracle.com/javase/7/do…

* -verbose: GC reports each garbage collection event, stable version, * information collection for garbage collection * -xx :+PrintGC unstable version, which may be deleted without notice, * -xx: -printgc in the official documentation below. * Because they are marked manageable, they can be modified in one of three ways: * 1, com. Sun. Management. * 2, JConsole * 3, jinfo HotSpotDiagnosticMXBean API - flag * - heap Xms20m setting minimum 20 m * - Xmx20m setting pile up to 20 m Will heap of maximum and minimum value is set to the same, can avoid the heap automatically expand * - XX: + HeapDumpOnOutOfMemoryError parameter when the JVM OOM occurs, * to automatically generate the DUMP file. * -xx :HeapDumpPath=${directory} * You can also specify the file name * -xx :+PrintGCDetails Console prints GC log details including -xx :+PrintGC * -xx :SurvivorRatio=8 Set the ratio of two Survivor zones and Eden, * 8 means two Survivor: Eden =2:8 that is, one Survivor accounts for 1/10 of the young generation * -xx: the ratio of NewRatio in the new generation (Eden +2*s) to the old (excluding permanent zones). * 4 indicates the new generation: old age =1:4, that is, the young generation occupies 1/5 of the heap * -xSS128K stack capacity is only set by -xSS parameter * -xx: PermSize and -xx: MaxPermSize to limit the size of the method area, thereby indirectly limiting the capacity of the constant poolCopy the code
Scenario 1 :Java heap overflow

The Java heap is used to store object instances, and as long as objects are constantly created and there is a reachable path between the GC Roots and the objects to avoid garbage collection, overflow exceptions can occur after the maximum heap capacity is reached.

/ * * *@version 2019/5/20
 * @description: Below is set to start in the IDEA of the class in the inside of the VM properties - verbose: gc - Xms20m - Xmx20m / / set the heap size stability in 20 m can avoid of automatic extension - XX: + HeapDumpOnOutOfMemoryError -XX:+PrintGCDetails -XX:SurvivorRatio=8 *@since2019/5/20 * /
public class HeapOOM {
    static class OOMObject {
        public static void main(String[] args) {
            List<OOMObject> list = new ArrayList<OOMObject>();
            while (true) {
                list.add(newOOMObject()); }}}}Copy the code

You can see the printed information

HeapOOM. Java: // The stack overflow is caused by the stack overflow. We set the maximum heap memory size to 20M. java.lang.OutOfMemoryError: Java heap space Dumping heap to java_pid402356.hprof ... Exceptionin thread "main" Heap dump file created [28351433 bytes in0.286 secs] Java. Lang. OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:3210) at java.util.Arrays.copyOf(Arrays.java:3181) at java.util.ArrayList.grow(ArrayList.java:261) at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:235) at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:227) at java.util.ArrayList.add(ArrayList.java:458) at com.zxy.test.jvmtest.HeapOOM$OOMObject.main(HeapOOM.java:17)
Copy the code

How to solve:

In real engineering projects, it is often more complicated than this, and we can analyze the data dumped by the Memory image analysis tool heap. The focus is to determine whether the object is necessary, that is, to distinguish between Memory leak and Memory OverFlow.

Summary: Stack overflow is a memory overflow exception that occurs when the number of object instances reaches the maximum heap capacity limit.

Scenario 2: The VM stack and local method stack overflow

Since there is no distinction between the virtual machine stack and the local method stack in the HotSpot VIRTUAL machine, for HotSpot the -xoss parameter (setting the local method stack size) exists but is actually invalid and the stack size is only set by the -xss parameter. With respect to the virtual machine stack and the local method stack, two exceptions are described in the Java Virtual Machine specification:

StackOverflowError is thrown if the stack depth requested by the thread is greater than the maximum depth allowed by the virtual machine. An OutOfMemoryError is thrown if the virtual machine cannot allocate enough memory while extending the stack.

When the stack space can no longer be allocated, the choice between too little memory and too much used stack space is essentially just two ways of describing the same thing.

In the author’s experiment, the scope of the experiment was limited to operations in a single thread, and the following two methods were tried to make the virtual machine generate OutOfMemoryError exceptions. StackOverflowError exceptions were obtained as a result of both attempts. The test code is shown in Listing 2-4.

Use the -xss argument to reduce stack memory capacity. Result: A StackOverflowError exception is raised, and the output stack depth shrinks when the exception occurs. A number of local variables are defined to increase the length of the local variable table in this method frame. Result: The output stack depth shrinks when a StackOverflowError is thrown.

@author ZZM */ public class JavaVMStackSOF {private int stackLength=1; public void stackLeak(){ stackLength++; stackLeak(); } public static void main(String[] args) { JavaVMStackSOF oom = new JavaVMStackSOF(); Try {system.out. println("stack length: "+oom. StackLength); }catch (Exception e){ e.getMessage(); }}} stack length: 2402 the Exception in the thread "main" Java. Lang, StackOverflowError at org. Fenixsoft. Oom. VMStackSOF. Leak (VMStackSOF. Java: 20) at org. Fenixsoft. Oom. VMStackSOF. Leak (VMStackSOF. Java: 21)Copy the code

Experimental results show that in a single thread, the virtual machine always throws StackOverflowError when memory cannot be allocated, either because the stack frame is too large or the virtual machine stack capacity is too small.

If the test is not limited to a single thread, it is possible to generate out-of-memory exceptions by continually creating threads, as shown in Listing 2-5. However, there is no correlation between overflow exceptions and whether the stack space is large enough, or rather, in this case, the more memory allocated for each thread’s stack, the more likely it is to overflow exceptions.

The reason is understandable: the operating system has a limit on the amount of memory that can be allocated to each process, such as 2GB on 32-bit Windows. The virtual machine provides parameters to control the maximum amount of memory in the Java heap and method area. The remaining memory is 2GB (the operating system limit) minus Xmx (the maximum heap size), minus MaxPermSize (the maximum method area size), and the program counter is so small that it can be ignored. If the memory consumed by the virtual machine process itself is not counted, the remaining memory is “split” between the virtual machine stack and the local method stack. The larger the stack size allocated to each thread, the smaller the number of threads that can be created, and the easier it is to run out of memory when creating threads.

This is something to be aware of when developing multithreaded applications. StackOverflowError can be read when an error occurs, making it relatively easy to find the problem.

Key points:

If you use the virtual machine default, the stack depth should be in the range of 1000 to 2000 in most cases (since each method does not have the same frame size) and should be sufficient for normal method calls (including recursion).

If the memory overflow is caused by the creation of multiple threads, without reducing the number of threads or replacing the 64-bit VIRTUAL machine, you can only exchange for more threads by reducing the maximum heap and stack size.

Summary: OutOfMemoryError is thrown if the virtual machine cannot allocate enough memory space while extending the stack.

2.4.3 Method area and runtime constant pool overflow

Because the runtime constant pool is part of the method area, overflow tests for the two areas are done together.

String. Intern () is a Native method that returns a String representing the String in the String constant pool if it already contains a String equal to the String. Otherwise, the String contained in this String is added to the constant pool and a reference to this String is returned.

-xx: PermSize = -xx: MaxPermSize = -xx: MaxPermSize = -xx: MaxPermSize = -xx: MaxPermSize = -xx: MaxPermSize

Memory overflow exception caused by run-time constant pool/** *VM Args: -xx: PermSize=10M-XX: MaxPermSize=10M Limit method size *@author zzm
*/
public class RuntimeConstantPoolOOM{
public static void main(String[]args){
// Use List to keep constant pool references to avoid Full GC recycling constant pool behavior
List<String>list=newArrayList < String > ();//10MB PermSize is sufficient to generate OOM in the integer range
int i=0;while(true) {list. Add (String. The valueOf (i++). Intern ());// Result:
Exception in thread"main"Java. Lang. OutOfMemoryError: PermGen space at java.lang.String.intern(Native Method) at Org. Fenixsoft. Oom. RuntimeConstantPoolOOM. Main (RuntimeConstantPoolOOM. Java:18)// As you can see from the run result, the runtime constant pool overflows, followed by an OutOfMemoryError
// is the "PermGen space", indicating that the runtime constant pool is part of the method area (the persistent generation in the HotSpot VIRTUAL machine).
Copy the code

Running this program with JDK 1.7 will not yield the same result, and the while loop will continue.

A more interesting implication can be derived from the implementation of this string constant pool. The following code looks like this:

String.intern() returns the referenced testpublic class RuntimeConstantPoolOOM{
	public static voidMain (String[]args) {String str1=newThe StringBuilder ("Computer"), append ("Software"). The toString (); System.out.println (str1.intern () ==str1); String str2=newThe StringBuilder ("ja"), append ("va").toString (); System.out.println (str2.intern () ==str2); }}Copy the code

This code, when run in JDK 1.6, gets two false, and when run in JDK 1.7, gets one true and one false. The reasons for the difference are: In JDK 1.6, the intern () method copies the first encountered string instance into the persistent generation and returns a reference to the same string instance in the persistent generation. String instances created by StringBuilder are on the Java heap, so they must not be the same reference and will return false. Whereas the Intern () implementation of JDK 1.7 (and some other virtual machines, such as JRockit) no longer copies instances, only records the first instance reference in the constant pool, so that intern () returns the same reference as the string instance created by The StringBuilder.

The str2 comparison returns false because the string “Java” is already present before stringBuilder.toString () is executed and has a reference to it in the string constant pool, which does not comply with the “first occurrence” rule, whereas the string “computer software” is present for the first time. So return true.

Summary: When the memory of the method area is too small, the run-time constant pool is also indirectly small, and OutOfMemoryError is thrown if there is not enough memory.

Method area overflow

The method area is used to store information about the Class, such as the Class name, access modifiers, constant pools, field descriptions, method descriptions, and so on. For testing these areas, the basic idea is to generate a large number of classes at runtime to fill the method area until it overflows. In Listing 2-8, I generated a large number of dynamic classes using the CGLib[1] direct bytecode runtime.

It is important to note that the scenario we simulate in this example is not purely an experiment, and such applications are often found in practical applications: Many of the current mainstream frameworks, such as Spring and Hibernate, use bytecode technologies such as CGLib when enhancing classes. The more classes are enhanced, the larger the method area is required to ensure that dynamically generated classes can be loaded into memory. In addition, dynamic languages on the JVM (such as Groovy, etc.) typically continue to create classes to make the language dynamic, and as such languages become more popular, it becomes easier to encounter overflow scenarios similar to those in Listing 2-8.

/** *VM Args: -xx :PermSize=10M-XX:MaxPermSize=10M *@author zzm
*/
public class JavaMethodAreaOOM{
public static voidMain (String[]args) {while(true) {Enhancer Enhancer =newEnhancer (); Enhancer.setsuperclass (oomObject.class); Enhancer. SetUseCache (false); Enhancer. SetCallback (newMethodInterceptor () {publicObject Intercept (Object obj,Method Method,Object[]args,MethodProxy proxy)throws Throwable{ 
    returnProxy. InvokeSuper (obJ, ARGS); }}); enhancer.create(); }}static class OOMObject{}} running results: under Caused by: Java. Lang. OutOfMemoryError: PermGen space at Java. Lang. This. DefineClass1 (Native Method) at Java. Lang. This defineClassCond (this. Java:632) at the Java. Lang. This defineClass (this. Java:616)Copy the code

Method area overflow is also a common memory overflow exception, and the criteria for a class to be collected by the garbage collector are more stringent. In applications where a large number of classes are frequently generated dynamically, special attention needs to be paid to Class reclamation. Such scenes in addition to the above mentioned procedure using additional bytecode enhancement and dynamic language, common: a large number of JSP or dynamically generated JSP file application (JSP to run for the first time need to compile Java classes), based on the OSGi application (even if the same class files, be different loader loads will be seen as different classes), etc.

2.4.4 Local direct Memory Overflow

// -xx :MaxDirectMemorySize Sets the default direct memory value to the maximum heap size, i.e. -xmx

Garbage collection will be triggered when the direct memory reaches the upper limit. If it overflows, an OOM exception will be raised

The application in NIO skips the Java heap so that programs can access it directly, and the native heap space thus speeds up access to the memory space to some extent

Note that after JDK 1.7, you don’t need to worry about whether to configure it or not

DirectMemory capacity can be accessed through -xx: MaxDirectMemorySize specifies, if not, the default is the same as the Java heap maximum (specified by -xmx). Listing 2-9 skips the DirectByteBuffer class, Retrieving Unsafe instances directly for memory allocation via reflection. (The getUnsafe () method of the Unsafe class restricts returning instances only to the bootstrap class loader, meaning that the designers wanted only classes in rt.jar to use Unsafe’s functionality.) DirectByteBuffer does not actually ask the operating system to allocate memory. Instead, it calculates that the memory cannot be allocated, so it manually throws an exception. The real method to allocateMemory is unsafe.allocatememory ().

/** *VM Args: -xmx20m-xx: MaxDirectMemorySize=10M *@author zzm
*/
public class DirectMemoryOOM{
private static final int_1MB=1024*1024;public static voidThe main (String [] args)throwsThe Exception {Field unsafeField = Unsafe. Class. GetDeclaredFields () [0]; UnsafeField. SetAccessible (true); Unsafe Unsafe = (Unsafe) unsafeField. Get (Unsafe)null);while(true) {unsafe. AllocateMemory (_1MB); Exception in thread"main"Java. Lang. OutOfMemoryError at sun. Misc. Unsafe. AllocateMemory (Native Method) at org. Fenixsoft. Oom. DMOOM. Main (DMOOM. Java:20)Copy the code

An obvious feature of a directMemory-induced memory overflow is that there are no obvious anomalies in Heap Dump files. If the Dump file is small after OOM and NIO is used directly or indirectly, you may want to check for this.

Summary: Garbage collection will be triggered when direct memory reaches its limit. If it overflows, an OOM exception will be raised.

These are some of the situations when OutOfMemoryError is raised and some of the debugging handling.