This article is based on Java 17-EA, but the design is much the same after Java 11

We often ask in interviews whether system.gc () will trigger Full GC immediately, and there are a lot of answers on the Internet, but these answers are a little out of date. This article takes an in-depth look at the story behind System.gc(), based on the source code for Java 17 (EA), the latest LTS release of Java.

Why system.gc () is needed

1. The framework for using and managing out-of-heap memory requires the Full GC mechanism to trigger the out-of-heap memory reclamation

The Memory of the JVM, not only the heap Memory, but also many other blocks, can be seen by Native Memory Tracking.

Native Memory Tracking: Total: reserved=6308603KB, committed=4822083KB - Java Heap (reserved=4194304KB, committed=4194304KB) (mmap: reserved=4194304KB, committed=4194304KB) - Class (reserved=1161041KB, committed=126673KB) (classes #21662) ( instance classes #20542, array classes #1120) (malloc=3921KB #64030) (mmap: reserved=1157120KB, committed=122752KB) ( Metadata: ) ( reserved=108544KB, Committed =107520KB) (used=105411KB) (free=2109KB) (waste=0KB =0.00%) (Class space:) (reserved=1048576KB, Committed =15232KB) (Used =13918KB) (Free =1314KB) (waste=0KB =0.00%) - Thread (reserved=355251KB, committed=86023KB) (thread #673) (stack: reserved=353372KB, committed=84144KB) (malloc=1090KB #4039) (arena=789KB #1344) - Code (reserved=252395KB, committed=69471KB) (malloc=4707KB #17917) (mmap: reserved=247688KB, committed=64764KB) - GC (reserved=199635KB, committed=199635KB) (malloc=11079KB #29639) (mmap: reserved=188556KB, committed=188556KB) - Compiler (reserved=2605KB, committed=2605KB) (malloc=2474KB #2357) (arena=131KB #5) - Internal (reserved=3643KB, committed=3643KB) (malloc=3611KB #8683) (mmap: reserved=32KB, committed=32KB) - Other (reserved=67891KB, committed=67891KB) (malloc=67891KB #2859) - Symbol (reserved=26220KB, committed=26220KB) (malloc=22664KB #292684) (arena=3556KB #1) - Native Memory Tracking (reserved=7616KB, committed=7616KB) (malloc=585KB #8238) (tracking overhead=7031KB) - Arena Chunk (reserved=10911KB, committed=10911KB) (malloc=10911KB) - Tracing (reserved=25937KB, committed=25937KB) (malloc=25937KB #8666) - Logging (reserved=5KB, committed=5KB) (malloc=5KB #196) - Arguments (reserved=18KB, committed=18KB) (malloc=18KB #486) - Module (reserved=532KB, committed=532KB) (malloc=532KB #3579) - Synchronizer (reserved=591KB, committed=591KB) (malloc=591KB #4777) - Safepoint (reserved=8KB, committed=8KB) (mmap: reserved=8KB, committed=8KB)Copy the code
  • Java Heap: Heap memory-XmxLimit the maximum heap size of memory.
  • Class: The loaded Class and method information, which is actually metaspace, consists of two parts: Metadata-XX:MaxMetaspaceSizeLimit the maximum size, plus class space, by-XX:CompressedClassSpaceSizeLimit maximum size
  • Thread: Memory occupied by threads and Thread stacks. The size of each Thread stack is determined by-XssLimit, but there is no limit to the total size.
  • Code: Memory occupied by JIT just-in-time (C1, C2, compiler optimized) Code, by-XX:ReservedCodeCacheSizelimit
  • GC: Garbage collection takes up memory, such as the CardTable, the number of tags, the partition record, the GC Root tag, and so on. This is not limited, usually not very big.
  • The amount of memory occupied by the code and markup of the C2 Compiler itself. This is not limited and is generally not large
  • Internal: Command line parsing, JVMTI use memory, this is not limited, generally not large
  • Symbol: Size of the constant pool. String constant pool-XX:StringTableSizeThe total memory size is not limited
  • Native Memory Tracking: the size of Memory used by the collection itself. If collection is not enabled, it will not be used, it is not limited, it is usually not very large
  • Arena Chunk: All the memory allocated through Arena. This is not limited and is usually not large
  • Tracing: All memory occupied by collection, mainly JFR if JFR is enabled. This is not limited, usually not very big
  • Logging, Arguments, Module, Synchronizer, Safepoint, Other, these are things we don’t normally care about.

In addition to the Memory usage that Native Memory Tracking records, there are two types of Memory that Native Memory Tracking does not record:

  • Direct Buffer: indicates Direct memory
  • MMap Buffer: file mapped memory

For memory other than heap memory, some also require GC. Examples: MetaSpace, CodeCache, Direct Buffer, MMap Buffer, etc. Early JVMS, prior to Java 8, had imperfect mechanisms for these memory reclaims, and in many cases required the FullGC to scan the entire heap to determine which memory in these areas could be reclaimed.

There are frameworks that use and manage a lot of this out-of-heap space. For example, Netty uses Direct buffers, Kafka and RocketMQ use Direct buffers and MMap buffers. They all apply for a piece of memory from the system in advance, and then manage and use it. When the space is insufficient, continue to apply to the system, and there will be a reduction. For example, netty will try to add the Reference object to the Reference list after the Direct Buffer used reaches the limit of -xx :MaxDirectMemorySize. The reference-dependent internal daemon triggers the Cleaner run() method that can be reclaimed from the DirectByteBuffer association. If the memory is insufficient, the execution System., expected to trigger a full gc, gc () to recycle DirectByteBuffer objects in the heap memory to trigger the heap memory recovery, if still more than limit, it throws Java. Lang. OutOfMemoryError.

2. The program of WeakReference, SoftReference is used, and corresponding GC collection is required.

For WeakReference, as long as GC occurs, whether Young GC or FullGC will be recycled. SoftReference is reclaimed only in FullGC. When our program wants to proactively collect these references, we need methods that can trigger GC, and we use system.gc ().

3. Testing, learning about JVM mechanisms

Sometimes, in order to test and learn some of the JVM’s mechanisms, we need to start by having the JVM do a GC, which also uses system.gc (). But there’s a better way, as you’ll see.

The principle behind system.gc ()

System.gc() actually calls runtime.geTruntime ().gc():

public static void gc() {
    Runtime.getRuntime().gc();
}
Copy the code

This method is a native method:

public native void gc();
Copy the code

JVM source code:

JVM_ENTRY_NO_ENV(void, JVM_GC(void)) JVMWrapper("JVM_GC"); // If the JVM startup parameter DisableExplicitGC is not set to false, then GC is performed. The GC cause is triggered by system. GC, corresponding to GCCause:: _JAVA_lang_system_GC if (! DisableExplicitGC) { Universe::heap()->collect(GCCause::_java_lang_system_gc); } JVM_ENDCopy the code

First, according to the status of DisableExplicitGC, the JVM startup parameter, determine whether GC will occur. If GC is required, different GC will have different processing.

1. G1 GC processing

If it is a System. The gc () trigger gc, G1 gc could decide according to ExplicitGCInvokesConcurrent the JVM parameter is the default gc (lightweight gc, YoungGC) or FullGC.

Reference code g1CollectedHeap. CPP:

GCCause:: _javA_lang_system_GC; // If GCCause:: _javA_lang_system_GC, Here is whether the JVM ExplicitGCInvokesConcurrent to true if (should_do_concurrent_full_gc (cause)) {return try_collect_concurrently(cause, gc_count_before, old_marking_started_before); } else {// Otherwise enter full GC VM_G1CollectFull op(gc_count_before, full_gc_count_before, cause); VMThread::execute(&op); return op.gc_succeeded(); }Copy the code

2. ZGC processing

No direct processing, no support for triggering GC through system.gc ().

Reference source: zdriver.cpp

Void ZDriver::collect(GCCause::Cause Cause) {switch (Cause) { Case GCCause:: _wb_young_GC: case GCCause::_wb_conc_mark: case GCCause:: _wb_full_GC: case GCCause::_dcmd_gc_run: case GCCause::_java_lang_system_gc: case GCCause::_full_gc_alot: case GCCause::_scavenge_alot: case GCCause::_jvmti_force_gc: case GCCause::_metadata_GC_clear_soft_refs: // Start synchronous GC _gc_cycle_port.send_sync(cause); break; case GCCause::_z_timer: case GCCause::_z_warmup: case GCCause::_z_allocation_rate: case GCCause::_z_allocation_stall: case GCCause::_z_proactive: case GCCause::_z_high_usage: case GCCause::_metadata_GC_threshold: // Start asynchronous GC _gc_cycle_port.send_async(cause); break; case GCCause::_gc_locker: // Restart VM operation previously blocked by the GC locker _gc_locker_port.signal(); break; case GCCause::_wb_breakpoint: ZBreakpoint::start_gc(); _gc_cycle_port.send_async(cause); break; GCCause:: _javA_lang_system_GC will go here default: GCCause:: _javA_lang_SYSTEM_GC will go here // Other causes not supported fatal("Unsupported GC cause (%s)", GCCause::to_string(cause)); break; }}Copy the code

3. Shenandoah GC processing

Shenandoah’s processing is similar to that of G1 GC. It first determines whether GC is triggered explicitly by the user, and then determines whether GC is allowed by the JVM parameter DisableExplicitGC. Because the outer JVM_ENTRY_NO_ENV(void, JVM_GC(void)) already handles this status bit). If so, request GC and block until the GC request is processed. Then according to ExplicitGCInvokesConcurrent the JVM parameter is the default GC (parallel GC, lightweight YoungGC) or FullGC.

Reference source shenandoahControlThread. CPP

void ShenandoahControlThread::request_gc(GCCause::Cause cause) { assert(GCCause::is_user_requested_gc(cause) || GCCause::is_serviceability_requested_gc(cause) || cause == GCCause::_metadata_GC_clear_soft_refs || cause == GCCause::_full_gc_alot || cause == GCCause::_wb_full_gc || cause == GCCause::_scavenge_alot, "only requested GCs here"); GCCause:: _java_lang_system_GC, GCCause::_dcmd_gc_run, GCCause:: _jvmti_force_GC, GCCause::_heap_inspection, GCCause::_heap_dump) if (is_explicit_GC (cause)) {// DisableExplicitGC is false if (! DisableExplicitGC) {// Request GC handle_requested_GC (cause); } } else { handle_requested_gc(cause); }}Copy the code

The code flow for requesting GC is:

void ShenandoahControlThread::handle_requested_gc(GCCause::Cause cause) { MonitorLocker ml(&_gc_waiters_lock); // Get the current global GC id size_t current_gc_id = get_gc_id(); Id + 1 size_t required_gc_id = current_gc_id + 1; id + 1 size_t required_gc_id = current_gc_id + 1; While (current_gc_id < required_gc_id) {// Set the GC status bit, and another thread will scan to perform GC _gc_requester. GCCause::_java_lang_system_gc _requested_gc_cause = cause; GCCause:: _java_lang_system_GC _requested_gc_cause = cause; // Wait for gc lock object notify, indicating that GC is executed and completed ml. Wait (); current_gc_id = get_gc_id(); }}Copy the code

For GCCause:: _JAVA_lang_system_GC, the GC execution process looks like this:

bool explicit_gc_requested = _gc_requested.is_set() && is_explicit_gc(_requested_gc_cause); Else if (explicit_gc_requested) {cause = _requested_gc_cause; log_info(gc)("Trigger: Explicit GC request (%s)", GCCause::to_string(cause)); heuristics->record_requested_gc(); / / if the JVM parameter ExplicitGCInvokesConcurrent is true, Is the default light GC if (ExplicitGCInvokesConcurrent) {policy - > record_explicit_to_concurrent (); mode = default_mode; // Unload and clean up everything heap->set_unload_classes(heuristics->can_unload_classes()); } else {// Otherwise, execute FullGC policy->record_explicit_to_full(); mode = stw_full; }}Copy the code

JVM parameters associated with system.gc ()

1. DisableExplicitGC

Note: Whether to disable explicit GC, it is not allowed by default. For Shenandoah GC, explicit GC includes: GCCause::_java_lang_system_gc, GCCause::_dcmd_gc_run, GCCause:: _jvmti_force_GC, GCCause::_heap_inspection, GCCause:: _jvmti_force_GC GCCause::_heap_dump, and for other GCS, only GCCause:: _java_lang_system_GC is restricted

Default: false

For example, to DisableExplicitGC: -xx :+DisableExplicitGC

2. ExplicitGCInvokesConcurrent

Note: For explicit GC, whether to execute the light parallel GC (YoungGC) or FullGC (YoungGC). If true, the light parallel GC (YoungGC) is executed. If false, the FullGC is executed

Default: false

Example: to enable the specified: – XX: + ExplicitGCInvokesConcurrent

In fact, someone put forward on the design (see links) want to change the ExplicitGCInvokesConcurrent to true. However, not all GCS can reclaim all Java memory areas in a lightweight parallel GC, and sometimes it has to be done through FullGC. So, for now, the default is false

3. Expired ExplicitGCInvokesConcurrentAndUnloads and ClassUnloadingWithConcurrentMark alternative use

If explicit GC uses lightweight parallel GC, Class Unloading cannot be performed, and exceptions may occur if Class Unloading is enabled. So this status bit is used to mark that in explicit GC, even with lightweight parallel GC, scanning for class unload is required. ExplicitGCInvokesConcurrentAndUnloads has expired, use ClassUnloadingWithConcurrentMark instead

Refer to the BUG – JDK – 8170388

How to trigger various GCS flexibly and controllably?

The answer is through the WhiteBox API. But this is not to be done in production, just to test the JVM and learn how to use the JVM. The WhiteBox API is a white-box testing tool that comes with HotSpot VM and exposes many of the core mechanisms of the INTERNAL API for white-box testing of the JVM, for testing JVM features, and for learning to understand the JVM and tuning parameters. The WhiteBox API was introduced in Java 7 and is currently available in Java 8 LTS and Java 11 LTS. Java 9 introduced modularity, so the WhiteBox API changed). By default, this API is not compiled in the JDK, but its implementation is. So if you want to use this API, you need to compile the required API, add Java BootClassPath, and enable the WhiteBox API. Let’s use the WhiteBox API to actively trigger the various GCS.

1. Compile the WhiteBox API

Remove https://github.com/openjdk/jdk/tree/master/test/lib path under sun directory, compiled into a jar package, name is whitebox hypothesis. The jar

2. Write test programs

Add the Whitebox.jar to your project dependencies and then write the code

public static void main(String[] args) throws Exception { WhiteBox whiteBox = WhiteBox.getWhiteBox(); // Execute youngGC whitebox.younggc (); System.out.println("---------------------------------"); whiteBox.fullGC(); // Run fullGC whitebox.fullgc (); Thread currentThread().join(); }Copy the code

3. Start the program to see the effect

Use the launch parameters – Xbootclasspath/a: / home/project/whitebox jar – XX: XX: + UnlockDiagnosticVMOptions – + WhiteBoxAPI Xlog: gc start the program. The first three Flag indicates enabling the WhiteBox API, and the last Flag indicates printing GC Info-level logs to the console.

My output:

[info][gc] Using G1 [0.048s][info][gc,init] Version: 17 - internal + 0 - adhoc. Administrator. JDK (fastdebug) [s] 0.048 [info] [gc, init] CPUs: 16 total, 16 available [0.048s][info][GC,init] Memory: 16304M [0.048s][info][GC,init] Large Page Support: Disabled [0.048s][info][GC,init] NUMA Support: Disabled [0.048s][info][GC,init] Compressed Oops: Enabled (32-bit) [0.048s][info][GC,init] Heap Region Size: 1M [0.048s][info][GC,init] Heap Min Capacity: 512M [0.048s][info][GC,init] Heap Initial Capacity: 512M [0.048s][info][GC,init] Heap Max Capacity: 512M [0.048s][info][GC,init] Pre-touch: Disabled [0.048s][info][GC,init] Parallel Workers: 13 [0.048s][info][GC,init] Concurrent Workers: 3 [0.048s][info][GC,init] Concurrent Workers: 13 [s] 0.048 [info] [gc, init] Periodic gc: Disabled [0.049s][info][GC,metaspace] CDS Disabled. [0.049s][info][GC,metaspace] Compressed class space mapped at: 0x0000000100000000-0x0000000140000000, reserved size: 1073741824 [0.049s][info][GC, Metaspace] Narrow KLass base: 0x0000000000000000, Narrow klass shift: 3, Narrow klass range: 0x140000000 [1.081s][info][GC,start] GC(0) Pause Young (Normal) (WhiteBox Initiated Young GC) [1.082s][info][GC, Task] GC(0) Using 12 workers of 13 for evacuation [1.089s][INFO][GC, Phases] GC(0) Pre Evacuate Collection Set: 1.5ms [1.089s][INFO][GC, GC(0) Merge Heap Roots 1.2ms [1.22s][info][GC, PHASES] GC(0) Evacuate Collection Set: 1.6ms [1.22s][info][GC, Phases] GC(0) Post Evacuate Collection Set: 1.6ms [1.22s][INFO][GC, Phases] GC(0) Other: 1.3ms [1.089s][info][GC,heap] GC(0) Eden Regions: 8->0(23) [1.089s][info][GC,heap] GC(0) Survivor Regions: 0->2(4) [1.089s][info][GC,heap] GC(0) Old regions: 0->0 [1.089s][info][GC,heap] GC(0) Archive Regions: 0->0 [1.089s][info][gc,heap] GC(0) Humongous regions: 0->0 [1.089s][info][gc,heap] GC(0) metaspace: 6891K(7104K)->6891K(7104K) NonClass: 6320K(6400K)->6320K(6400K) Class: [INFO][GC] GC(0) Pause Young (Normal) (WhiteBox Initiated Young GC) 7M->1M(512M) 7.864ms [s] 1.089 [info] [gc, CPU] gc (0) the User s Sys = = 0.00 0.00 s Real = 0.01 s -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 1.091 [s] [info] [gc, task GC(1) Using 12 workers of 13 for full compaction [1.102s][info][GC,start] GC(1) Pause full (WhiteBox Initiated full) (1) Phase 1: Mark live Objects [1.117s][INFO][GC, Phase 1] Mark Live Objects 8.409ms [1.117s][INFO][GC, Phase 2] Gc (1) Phase 2: Phase 2: [Phases][PHASES] Phase 1: Phase 1 [Phases][PHASES] Phase 1: Phase 1 Gc (1) Phase 4: Compact heap 63.812ms [1.193s][info][GC,heap] GC(1) Eden Regions: 1->0(25) [1.193s][info][gc,heap] GC(1) Survivor regions: 2->0(4) [1.193s][info][GC,heap] GC(1) Old regions: 0->3 [1.193s][info][GC,heap] GC(1) Archive regions: 0->0 [1.193s][info][GC,heap] GC(1) Humongous regions: [info][gc,metaspace] GC(1) metaspace: 6895K(7104K)->6895K(7104K) 6323K(6400K)->6323K(6400K) Class: [info][GC] GC(1) Pause Full (WhiteBox Initiated Full GC) 1M->0M(512M) 84.846ms [info][gc, CPU] GC(1) User=0.19s Sys=0.63s Real=0.11sCopy the code

Wechat search “my programming cat” to follow the public account, every day, easy to improve technology, win a variety of offers