Key TakeAways

  1. StackOverflowError: The call stack is too deep, causing the thread stack to oversize-Xss(or-XX:ThreadStackSizeRestrictions)
  2. OutOfMemoryError: Java Heap Space: This exception is thrown when there is not enough heap memory to allocate more memory.
  3. OutOfMemoryError: Unable to create native Thread: This is creating too many threads, exceeding the system configuration limit. For example, Linux allows a single process to create a maximum of 1024 threads by default.
  4. OutOfMemoryError: GC Overhead limit exceeded: By default, an OutOfMemoryError will not be reported until the heap is exhausted, but will be reported if the JVM feels that GC is not efficient.
  5. OutOfMemoryError: Direct Memory: This exception is thrown when a direct memory request is made to the system
  6. OutOfMemoryError: Map failed: This exception is thrown if the system runs out of memory when this is File MMAP
  7. OutOfMemoryError: Requested array size exceeds VM Limit: This exception is thrown when the Requested array size exceeds the heap memory limit.
  8. OutOfMemoryError: Metaspace: The space occupied by Metadata exceeds the upper limit
  9. OutOfMemoryError: Compressed class space: The memory is overloaded
  10. OutOfMemoryError: Reason stack_trace_WITH_native_method: This occurs in a JNI call and is out of memory

This article is based on Java 15

StackOverflowError and OutOfMemoryError are two well-worn Java errors. Virtualmachine Errors in Java include the following four types:

We are concerned about StackOverflowError and OutOfMemoryError. The remaining InternalError is an InternalError, and UnknownError is an UnknownError on a vm. We will not discuss these two errors here.

StackOverflowError and OutOfMemoryError in the virtual machine specification

As you can see from the official Java Virtual Machine specification documentation: Run-time Data Areas, these two types of errors are thrown in the following situations:

  • When the size of the Java Virtual Machine Stack exceeds the specified size, a StackOverflowError is raised
  • OutOfMemoryError is raised if the Java VM stack can be dynamically expanded and the memory is insufficient during capacity expansion or when the Java VM stack is created
  • OutOfMemoryError is thrown when the required heap memory size is insufficient
  • An OutOfMemoryError is raised if the Method Area is not large enough to allocate
  • An OutOfMemoryError is raised when a class or interface is created and the remaining size of the runtime constant area is insufficient
  • A StackOverflowError is raised when the Native Method Stack is too small
  • An OutOfMemoryError is raised when the memory of the Native Method Stack is insufficient during capacity expansion or when the memory of a new local Method Stack is insufficient

Implementation of Hotspot JVM

To further clarify StackOverflowError versus OutOfMemoryError, let’s look at the implementation. The general JVM is the official website HotSpot JVM, we here with HotSpot JVM implementation to illustrate.

What does JVM memory include

Since pmap is a bit complex and a lot of memory mapping is anon, Here we use the vm. native_memory command in JCMD to look at each portion of JVM memory. One thing to point out is,

For more details on Native Memory Tracking, please refer to my article on JVM Memory Footprint and Analysis

Native Memory Tracking: Total: reserved=6308603KB, committed=4822083KB - Java Heap (reserved=4194304KB, committed=4194304KB) (mmap: reserved=4194304KB, committed=4194304KB) - Class (reserved=1161041KB, committed=126673KB) (classes #21662) ( instance classes #20542, array classes #1120) (malloc=3921KB #64030) (mmap: reserved=1157120KB, committed=122752KB) ( Metadata: ) ( reserved=108544KB, Committed =107520KB) (used=105411KB) (free=2109KB) (waste=0KB =0.00%) (Class space:) (reserved=1048576KB, Committed =15232KB) (used=13918KB) (Free =1314KB) (waste=0KB =0.00%) - Thread (reserved=355251KB, committed=86023KB) (thread #673) (stack: reserved=353372KB, committed=84144KB) (malloc=1090KB #4039) (arena=789KB #1344) - Code (reserved=252395KB, committed=69471KB) (malloc=4707KB #17917) (mmap: reserved=247688KB, committed=64764KB) - GC (reserved=199635KB, committed=199635KB) (malloc=11079KB #29639) (mmap: reserved=188556KB, committed=188556KB) - Compiler (reserved=2605KB, committed=2605KB) (malloc=2474KB #2357) (arena=131KB #5) - Internal (reserved=3643KB, committed=3643KB) (malloc=3611KB #8683) (mmap: reserved=32KB, committed=32KB) - Other (reserved=67891KB, committed=67891KB) (malloc=67891KB #2859) - Symbol (reserved=26220KB, committed=26220KB) (malloc=22664KB #292684) (arena=3556KB #1) - Native Memory Tracking (reserved=7616KB, committed=7616KB) (malloc=585KB #8238) (tracking overhead=7031KB) - Arena Chunk (reserved=10911KB, committed=10911KB) (malloc=10911KB) - Tracing (reserved=25937KB, committed=25937KB) (malloc=25937KB #8666) - Logging (reserved=5KB, committed=5KB) (malloc=5KB #196) - Arguments (reserved=18KB, committed=18KB) (malloc=18KB #486) - Module (reserved=532KB, committed=532KB) (malloc=532KB #3579) - Synchronizer (reserved=591KB, committed=591KB) (malloc=591KB #4777) - Safepoint (reserved=8KB, committed=8KB) (mmap: reserved=8KB, committed=8KB)Copy the code

Here mmap, malloc are two different ways of allocating memory, for example:

Internal (reserved=3643KB, committed=3643KB)
                            (malloc=3611KB #8683) 
                            (mmap: reserved=32KB, committed=32KB) 
Copy the code

Indicates that a total of 3643KB is occupied by Internal resources. 3611KB is used in malloc mode and 32KB is used in mmap mode. Arenas are malloc allocated memory that is not released after code execution and will continue to be used after being placed in Arena Chunk, see MallocInternals

As you can see, Java process memory includes:

  • Java Heap: Heap memory, i.e-XmxLimit the maximum heap size of memory.
  • Metaspace: metadata that contains information about classes and methods to be loaded-XX:MaxMetaspaceSizeLimit the maximum size, the other is class space, by-XX:CompressedClassSpaceSizeLimit maximum size
  • Thread: Thread and Thread stack occupy memory, the size of each Thread stack-XssLimit, but there is no limit to the total size.
  • Code: JIT just-in-time compiled (C1 C2 compiler optimized) Code takes up memory, subject to-XX:ReservedCodeCacheSizelimit
  • GC: Garbage collection takes up memory, such as CardTable, tag count, partition records, tag GC Root, etc. It’s not restricted, it’s usually not very big.
  • The memory used by the C2 Compiler’s own code and markup is not limited and is generally not very large
  • Internal: The amount of memory used by JVMTI for command line parsing. This is unlimited and generally not very large
  • Symbol: Size occupied by the string constant pool-XX:StringTableSizeNumber limit, total memory size is not limited
  • Native Memory Tracking: The size of the Memory used by the acquisition itself, if not enabled, will not be used, this is not limited, generally not very large
  • Arena Chunk: All memory allocated via Arena. This is unlimited and usually not very large
  • Tracing: Memory occupied by all collections. If JFR is enabled, it is mainly the memory occupied by JFR. It’s not restricted, it’s usually not very big
  • Logging, Arguments, Module, Synchronizer, Safepoint, and Other are things we don’t care about.

There are two types of Memory Native Memory Tracking that are not recorded:

  • JAVA out-of-heap memory, zero copy, Direct memory, and thoughts on FileChannel in NIO
  • MMap Buffer: File mapping memory, please refer to: JDK core JAVA source code parsing (5) – JAVA File MMap principle parsing

Various StackOverflowError and OutOfMemoryError scenarios and positioning methods

1. StackOverflowError

The call stack is too deep, causing the ThreadStackSize to exceed the limit of -xss (or -xx :ThreadStackSize). If -xss is not specified, the default maximum size is determined depending on the system.

See the code for determining the default size:

  • Windows: os_windows. CPP
  • Linux: os_linux. CPP

To sum up, 32-bit systems are typically 512K and 64-bit systems are 1024K

This error is usually reported because of recursive loops, or because the call stack is really deep and the thread stack is not large enough, such as in a framework like Netty + Reactor, the thread stack needs to be larger.

2. OutOfMemoryError: Java heap space

This exception is thrown when the heap is out of memory to allocate more memory. Usually this happens after, need to look at heap dump, online application general plus – XX: + HeapDumpOnOutOfMemoryError OutOfMemoryError occurs in carries on the heap dump, after analysis.

Heap dump viewing tools typically use Memory Analyzer (MAT)

3. OutOfMemoryError: unable to create native thread

This is creating too many threads, exceeding the system configuration limit. For example, Linux allows a single process to create a maximum of 1024 threads by default.

The first thing to consider when reporting this error is not to create so many threads, thread pooling and pooling as much as possible with business reuse. If you really want to create multiple threads, consider modifying the server configuration:

/ / check the limit the number of the ulimit -u / / edit vim/etc/security/limits. D / 90 - nproc. ConfCopy the code

4. OutOfMemoryError: GC Overhead limit exceeded

By default, OutOfMemoryError is not reported when the heap runs out of memory, but if the JVM feels that GC is inefficient.

So how to evaluate GC efficiency is not high? Take a look at the source code:? Gcoverheadchecker.cpp:

void GCOverheadChecker::check_gc_overhead_limit(GCOverheadTester* time_overhead, GCOverheadTester* space_overhead, Bool is_full_gc, GCCause::Cause gc_cause, SoftRefPolicy* soft_ref_policy) {// Ignore explicit gc commands, such as system.gc (), or gc via JVMTI, Or by JCMD command gc if (GCCause: : is_user_requested_gc (gc_cause) | | GCCause: : is_serviceability_requested_gc (gc_cause)) { return; } bool print_gc_overhead_limit_would_be_exceeded = false; If (is_full_gc) {// If (is_full_gc) {// If (is_full_gc) {// If (is_full_gc) {// If (is_full_gc) {// If (is_full_gc) {// If (is_full_gc) {// If (is_full_gc) {// If (is_full_gc) { GCTimeLimit // If the gc space is less than 2%, the gc space is not enough. You can configure GCTimeLimit with -xx :GCHeapFreeLimit=. GCHeapFreeLimit if (time_overhead->is_exceeded() && space_overhead->is_exceeded()) { _gc_overhead_limit_count++; // If the status bit of UseGCOverheadLimit is enabled // By default, it is enabled and can be disabled with the startup parameter -xx: -usegcoverheadLimit. See gc_globals.hpp: UseGCOverheadLimit if (UseGCOverheadLimit) {if (UseGCOverheadLimit) {if (UseGCOverheadLimit) {if (UseGCOverheadLimit) {if (UseGCOverheadLimit) {if (UseGCOverheadLimit) {if (UseGCOverheadLimit) { GCOverheadLimitThreshold if (_gc_overhead_limit_count >= GCOverheadLimitThreshold){// Set the status bit, Set_gc_overhead_limit_exceeded (true); // Clear count reset_gc_overhead_limit_count(); } else {// Empty all soft references bool NEAR_LIMIT = gc_overhead_limit_NEAR (); if (near_limit) { soft_ref_policy->set_should_clear_all_soft_refs(true); log_trace(gc, ergo)("Nearing GC overhead limit, will be clearing all SoftReference"); Print_gc_overhead_limit_would_be_exceeded = true; } else { // Did not exceed overhead limits reset_gc_overhead_limit_count(); } } if (UseGCOverheadLimit) { if (gc_overhead_limit_exceeded()) { log_trace(gc, ergo)("GC is exceeding overhead limit of " UINTX_FORMAT "%%", GCTimeLimit); reset_gc_overhead_limit_count(); } else if (print_gc_overhead_limit_would_be_exceeded) { assert(_gc_overhead_limit_count > 0, "Should not be printing"); log_trace(gc, ergo)("GC would exceed overhead limit of " UINTX_FORMAT "%% %d consecutive time(s)", GCTimeLimit, _gc_overhead_limit_count); }}}Copy the code

The default configuration is gc_globals.hpp

product(bool, UseGCOverheadLimit, true, \ "Use policy to limit of proportion of time spent in GC " \ "before an OutOfMemory error is thrown") \ \ product(uintx,  GCTimeLimit, 98, \ "Limit of the proportion of time spent in GC before " \ "an OutOfMemoryError is thrown (used with GCHeapFreeLimit)") \  range(0, 100) \ \ product(uintx, GCHeapFreeLimit, 2, \ "Minimum percentage of free space after a full GC before an " \ "OutOfMemoryError is thrown (used with GCTimeLimit)") \ range(0, 100) \ \ develop(uintx, GCOverheadLimitThreshold, 5, \ "Number of consecutive collections before gc time limit fires") \ range(1, max_uintx)Copy the code

To summarize: By default, UseGCOverheadLimit is enabled, and this exception is thrown when the GC time exceeds 98% and the GC has recovered less than 2% of memory for five consecutive times.

5. OutOfMemoryError: direct memory

This exception is thrown when the system is running out of memory, and the source code is bits.java:

static void reserveMemory(long size, int cap) {
    synchronized (Bits.class) {
        if (!memoryLimitSet && VM.isBooted()) {
            maxMemory = VM.maxDirectMemory();
            memoryLimitSet = true;
        }
        // -XX:MaxDirectMemorySize limits the total capacity rather than the
        // actual memory usage, which will differ when buffers are page
        // aligned.
        if (cap <= maxMemory - totalCapacity) {
            reservedMemory += size;
            totalCapacity += cap;
            count++;
            return;
        }
    }
    System.gc();
    try {
        Thread.sleep(100);
    } catch (InterruptedException x) {
        // Restore interrupt status
        Thread.currentThread().interrupt();
    }
    synchronized (Bits.class) {
        if (totalCapacity + cap > maxMemory)
            throw new OutOfMemoryError("Direct buffer memory");
        reservedMemory += size;
        totalCapacity += cap;
        count++;
    }
}
Copy the code

The Bits class has a global totalCapacity variable that records the total size of all DirectBytebuffers. By default, the out-of-heap memory limit is similar to the in-heap memory limit (set by -xmx), which can be reset by -xx :MaxDirectMemorySize.

If not specified, the default value of this parameter is the value of Xmx minus the value of 1 Survior. If the boot parameter -xmx20m-xmn10M-XX: SurvivorRatio=8, then sytem.gc () will be executed if DirectMemory (20M-1M=19M) exceeds the limit. System.gc() will trigger a full gc, of course, if you don’t have the displayed setting -xx :+DisableExplicitGC to DisableExplicitGC. Also, you need to know that calling system.gc () does not guarantee that full GC will be executed immediately. Then go to sleep for 100 milliseconds to see if totalCapacity is down. If it is still out of memory, raise OOM exception. If the limit is approved, the famous sun.misc.Unsafe is called to allocate memory, returning the address of the memory base

In this exception occurs, usually by JMX Java nio. BufferPool. Direct the attribute to monitor the change of direct memory inside and use (is actually BufferPoolMXBean), to locate the problem.

6. OutOfMemoryError: map failed

If the system runs out of memory, this exception will be raised. The source code for this exception is:

  • Windows: FileDispatcherImpl. C
  • Linux: FileDispatcherImpl. C

Take Linux as an example:

JNIEXPORT jlong JNICALL Java_sun_nio_ch_FileChannelImpl_map0(JNIEnv *env, jobject this, jint prot, jlong off, jlong len) { void *mapAddress = 0; jobject fdo = (*env)->GetObjectField(env, this, chan_fd); jint fd = fdval(env, fdo); int protections = 0; int flags = 0; if (prot == sun_nio_ch_FileChannelImpl_MAP_RO) { protections = PROT_READ; flags = MAP_SHARED; } else if (prot == sun_nio_ch_FileChannelImpl_MAP_RW) { protections = PROT_WRITE | PROT_READ; flags = MAP_SHARED; } else if (prot == sun_nio_ch_FileChannelImpl_MAP_PV) { protections = PROT_WRITE | PROT_READ; flags = MAP_PRIVATE; } // Mmap mapAddress = mmap64(0, /* Let OS decide location */ len, /* Number of bytes to map */ protections, /* File permissions */ flags, /* Changes are shared */ fd, /* File descriptor of mapped file */ off); /* Offset into file */ / Throws OutOfMemoryError if (mapAddress == MAP_FAILED) {if (errno == ENOMEM) {JNU_ThrowOutOfMemoryError(env, "Map failed"); return IOS_THROWN; } return handle(env, -1, "Map failed"); } return ((jlong) (unsigned long) mapAddress); }Copy the code

In this case, consider:

  1. Increasing system memory
  2. Use file fragmentation, do not mmap a large file, that is, reduce the size of each Mmap file

7. OutOfMemoryError: Requested array size exceeds VM limit

This exception is thrown when the requested array size exceeds the heap memory limit.

8. OutOfMemoryError: Metaspace

This exception is raised when the Metadata uses up more than the specified amount of space. If the Metadata uses up more than the specified amount of space, the Metadata uses up more than the specified amount of space.

You can use two commands with the same output:

  • jmap -clstats

  • JCMD GC. Class_stats (the need to launch parameters: – XX: + UnlockDiagnosticVMOptions)

    Index Super InstBytes KlassBytes annotations CpAll MethodCount Bytecodes MethodAll ROAll RWAll Total ClassName 1 -1 214348176 504 0 0 0 0 0 24 616 640 [C 2 -1 71683872 504 0 0 0 0 0 24 616 640 [B 3 -1 53085688 504 0 0 0 0 0 24 616 640 [Ljava.lang.Object; 4 -1 28135528 504 0 0 0 0 0 32 616 648 [Ljava.util.HashMap Node;5174781258221614400700864268139040112323724848480java.util.ArrayList………25255250528059234256844814481896zipkin 2.reporter.metrics.micrometer.MicrometerReporterMetricsNode; 5 17478 12582216 1440 0 7008 64 2681 39040 11232 37248 48480 java.util.ArrayList ……… 25255 25 0 528 0 592 3 42 568 448 1448 1896 zipkin2.reporter.metrics.micrometer.MicrometerReporterMetricsNode;5174781258221614400700864268139040112323724848480java. util.ArrayList………25255250528059234256844814481896zipkin2.reporter.metrics.micrometer.MicrometerReporterMetricsBuil Der 472572680 16436464 283592 41813040 225990 8361510 75069552 39924272 101013144 140937416 Total 335.3% 11.7% 0.2% 29.7%-5.9% 53.3% 28.3% 71.7% 100.0% Index Super InstBytes KlassBytes Annotations CpAll MethodCount Bytecodes MethodAll ROAll RWAll Total ClassName

The meanings of each indicator are as follows:

  • InstBytes: indicates the size of an instance
  • KlassBytes: Size occupied by a class
  • Annotations: Size of annotations
  • CpAll: indicates the usage of the constant pool
  • MethodCount: Number of methods
  • Bytecodes: byte code size
  • MethodAll: Method footprint
  • ROAll: Memory usage in read-only memory
  • RWAll: memory occupied by read and write memory

9. OutOfMemoryError: Compressed class space

The class space memory overflow is similar to the previous exception. You need to view the class information to locate the fault.

10. OutOfMemoryError: reason stack_trace_with_native_method

This happens in a JNI call and is out of memory

Daily brush, easy to improve skills, get a variety of offers: