This is the 10th day of my participation in the August Text Challenge.More challenges in August

I believe that people with certain Java development experience will more or less encounter OutOfMemoryError, which has bothered me for a long time. With the accumulation of experience in solving various problems and the exploration of the root cause of the problem, I finally have a relatively in-depth understanding.

Before you can solve the Java memory overflow problem, you need to have some understanding of memory management for the JVM (Java Virtual Machine). The memory managed by the JVM generally consists of three different types of memory areas: Permanent Generation space, Heap space, and Java Stacks. When a Class is loaded for the first time, it is put into the PermGen space. The contents that a Class needs to store mainly include methods and static properties. The heap area is used to hold instances of Class (that is, objects), and objects need to store primarily non-static properties. Each time an object instance is created with new, the object instance is stored in the heap area, which is also managed by the JVM’s garbage collection mechanism. The Java stack functions like most programming languages, including assembly languages, in terms of primary type variables and input and output parameters for methods. Each thread of a Java program has a separate stack. Memory space that is prone to memory overflow includes the Permanent Generation space and the Heap space.

The first kind of OutOfMemoryError: PermGen space

The Java virtual machine (JVM) does not have enough space to load classes into the Permanent Generation space. There are two ways to solve this problem:

  1. Increase the values of XX:PermSize and XX:MaxPermSize in the Java VM, where XX:PermSize indicates the initial permanent storage area size and XX:MaxPermSize indicates the maximum permanent storage area size. For tomcat6.0, add the following line at the end of a series of environment variable names in catalina.sh or catalina.bat:JAVA_OPTS=" -XX:PermSize=64M -XX:MaxPermSize=128m"If the server is Windows, you can also set this parameter in system environment variables. This memory overflow error can easily occur when tomcat is used to distribute sprint+ Struts + Hibernate architecture applications. Using the above approach, I successfully resolved the frequent downtime of the Tomcat server where THE SSH project was deployed.
  2. Clear the JARS in the WEB-INF /lib directory of the application. If Tomcat has multiple applications deployed and many of them use the same JAR, you can move the common JAR to the common Lib directory of Tomcat to reduce the repeated loading of classes. This method is recommended by some people on the Internet, I have not tried, but the feeling can not reduce too much space, the most reliable or the first method.

The second kind of OutOfMemoryError: Java Heap space

This problem occurs because the Java virtual machine creates too many objects, and the virtual machine has used up the Heap space allocated to it between garbage collection, related to the Heap space. There are two ways to solve this problem:

  1. Check your program to see if it has an infinite loop or creates a large number of objects repeatedly unnecessarily. After finding the cause, modify the program and algorithm. When I used to write a text clustering algorithm using K-means for tens of thousands of text records (each record has about 10 feature vectors), there was a problem in the details of the program, which resulted in the memory overflow of the Java heap space, which was solved later by modifying the program.
  2. Increase the size of the Xms (initial heap size) and Xmx (maximum heap size) parameters in the Java virtual machine. Such as:set JAVA_OPTS= -Xms256m -Xmx1024m

Third type OutOfMemoryError: Unable to create new native thread

In Java applications, OutOfMemoryError: Unable to create new native thread. This is odd because the JVM already has a lot of memory allocated by the system (say 1.5 GIGABytes), and it takes up at least half of the available memory. It has been found that with many threads, the more memory you allocate to the JVM, the more likely these errors are to occur.

So what causes this problem?

Each 32-bit process can use up to 2 gigabytes of available memory, because the other 2 gigabytes are reserved by the operating system. Assuming 1.5 GIGABytes for the JVM, you still have 500 MB of free memory left. A portion of this 500 MB memory must be used to load system DLLS, so maybe only 400 MB is really left. Now here’s the key: When you create a Thread in Java, a Thread object is also created in the JVM’s memory, but an actual physical Thread is also created in the operating system (see the JVM specification). The operating system creates the physical Thread in the remaining 400 megabytes of memory, not in the JVM’s 1500 megabytes of memory heap. In JDK1.4, the default stack size was 256KB, but in JDK1.5, the default stack size was 1M per thread, so we could only create a maximum of 400 available threads out of the remaining 400M of free memory.

This leads to the conclusion that to create more threads, you must reduce the maximum memory allocated to the JVM. Another option is to have the JVM hosted in your JNI code.

Give a formula for estimating the maximum number of threads that can be created:

(MaxProcessMemory - JVMMemory - ReservedOsMemory) / (ThreadStackSize) = Number of threads
Copy the code

For JDK1.5, assume that the operating system reserves 120M of memory:

1.0GB JVM: (2GB-1.0GB-120MB)/(1MB) = ~880 ThreadsCopy the code

For JDK1.4 with 256KB stack size,

Allocated TO JVM: ~3520 Threads 1.0GB Allocated to JVM: ~3520 threadsCopy the code

For this exception, we first need to determine what kind of threads exist in the process when the memory overflow occurs, whether these threads should exist, whether the number of threads can be reduced through optimization; On the other hand, by default Java for each thread’s stack memory size is 1 m, usually, the 1 m stack memory space is adequate, because the store is on the stack usually basic types of data or object reference, and all these things will not occupy too much memory, we can adjust the JVM parameters, Reduce the stack memory allocated for each thread to solve the problem, for example by adding -xss128K to the JVM parameter to set the thread stack memory size to 128K.