preface
I have developed the data center system for more than half a year. Recently, I went online to pressure test in the test environment and found frequent GC when I monitored THE JVM. After I started the GC log, I found that Minor GC would be performed every few seconds (to reclaim the memory of the new generation). And by GC with less than a tenth of the heap (test -Xmx was set to 10G and GC with less than 1g), the system was optimized from two aspects (code level and JVM startup parameters).
1. Code level
Observe the JVM memory usage, found in pile rapidly rising 1 g, 5 seconds when the number of HTTP access to 1, then the number of concurrent access to a few cases, the cause of the memory rises fast should is the background of monitoring thread (thread pool to do service in the background process monitoring, data warehouse and middleware kafka), In these monitoring threads, there are a lot of RESTful API requests and detection (remote) of server ports. After analyzing the code, we found that because the monitor thread runs periodically, most of the remote connections and API requests can be continuously opened, saved in memory to reuse these objects (locking mechanism). Through this part of the transformation and then observe the use of JVM, found that the effect is significant.
2. JVM parameter level
Through the analysis of the code, it is concluded that the new generation of system has a large memory footprint, and a large amount of memory will be reclaimed during GC. As mentioned above, there is a total of 10G heap memory, but GC is performed with less than 1G. So the system is tuned in terms of the PARAMETERS of the JVM. The following sections will cover the basics of the JVM and help you understand parameter tuning.
2.1 JVM Structure
The class loader is responsible for loading the class bytecode into the JVM memory region when the JVM starts and the program starts executing
The execution engine is responsible for executing the bytecode instructions contained in the class file
Local method library
Basically, it calls native methods implemented in C or C++ and returns results
Runtime data area
Method areas are used to store class structure information, including constant pools, static variables, constructors, and so on.
The Java Heap (Heap)
A place to store Java instances or objects. This is the main area of GC. The method area and heap are shared by all Java threads.
The Java Stack has always been associated with threads, and each time a thread is created, the JVM creates a Java Stack for that thread. This Java stack contains multiple stack frames, one created for each method run, to store local variables, operation stacks, method return values, and so on. The process of each method from invocation to completion corresponds to the process of a stack frame being pushed into and out of the Java stack. So the Java stack is off-the-shelf and private.
A program counter (PC Register) is a small memory space that can be thought of as a line number indicator of the bytecode being executed by the current thread. Basic functions such as branching, looping, jumping, exception handling, thread recovery, and so on rely on this counter.
The Native Method Stack is similar to the Java Stack, except that it serves Native methods used by the JVM.
2.2. JVM memory allocation and reclamation strategy
VMS are divided into three generations: Young Generation, Old Generation, and Permanent Generation. The persistent generation mainly stores the class information of Java classes and has little to do with the Java objects to be collected by garbage collection. The division of the young generation and the old generation has a greater impact on garbage collection.
Young generation:
All newly generated objects are first placed in the young generation. The goal of the younger generation is to collect objects with short life spans as quickly as possible. The younger generation is divided into three sections. One Eden zone and two Survivor zones (generally). Most objects are generated in the Eden zone. When Eden is full, the surviving objects will be copied to the Survivor zone (one of the two). When this Survivor zone is full, the surviving objects in this Survivor zone will be copied to another Survivor zone. When this Survivor zone is full, Objects copied from the first Survivor zone that are still alive will be copied to “Tenured”. It should be noted that the two Survivor zones are symmetric and have no sequence relationship, so there may be objects copied from Eden and the object copied from the previous Survivor in the same zone, while only the object copied from the first Survivor to the old zone. Also, one of the Survivor zones is always empty. At the same time, Survivor zones can be configured to be multiple (more than two) depending on program requirements, which increases the duration of an object in the young generation and reduces the likelihood of it being placed in the old generation.
Old generation:
Objects that survive N garbage collections in the young generation are put into the old generation. Therefore, you can think of the tenured generation as holding objects with long life cycles.
Lasting generation:
Used to store static files, nowadays Java classes, methods, etc. Persistent generation has no significant impact on garbage collection, but some applications may generate or call classes dynamically, such as Hibernate, etc. In such cases, a large persistent generation space needs to be set up to store the classes added during the run. The persistent generation size can be set by -xx :MaxPermSize=.
JVM memory allocation is the same as GC memory allocation.
Objects are allocated in Eden with priority to large objects. Objects that survive for a long time will enter dynamic objects of the old age for age determination and then generation GC type
The Scavenge Avenge is normally triggered when a new object is created and the Eden Exploiture fails to claim space. The Scavenge avenge avenge the Eden exploiture, and move surviving objects to a Survivor zone. Then the two zones of Survivor are collated. In this way, GC is performed on the Eden area of the young generation without affecting the old generation. Since most objects start from Eden, and Eden is not allocated very large, GC in Eden will occur frequently. Therefore, a fast and efficient algorithm is generally needed to make Eden free as soon as possible.
Full GC collates the entire heap, including Young, Tenured, and Perm. Full GC is slower than Scavenge due to the need to recycle the entire pair, so minimize the amount of Full GC you do. A large part of the process of tuning the JVM is tuning FullGC. A Full GC can be caused by: · Tenured generations are Full · Perm is Full · System.gc() is displayed as a call · The allocation policy for each field of the Heap has changed dynamically since the last GC
2.3 JVM Memory Leaks and Overflows
Definition 2.3.1.
Memory leaks
A program that dynamically allocates memory to temporary objects that are not collected by GC. That is, the allocated object is reachable but no longer useful.
Memory overflow is an error that occurs when a program fails to allocate enough memory during execution. A memory overflow usually occurs when there is no memory space for new Java objects after the OLD or Perm segment garbage collection.
By definition, memory leaks are a cause of memory overflow, not the only one. OutOfMemoryError of Java heap memory is the most common out-of-memory exception in practical applications. Java heap memory leak, “Java. Lang. OutOfMemoryError exception stack information will follow further prompt” Java heapspace “.
2.3.2. Common memory leak scenarios
Long-life objects hold references to short-life objects
This is the most common scenario for memory leaks and is a common problem in code design. For example, if local variables are cached in a global static map and no clearing operation is performed, the map will grow larger over time, resulting in memory leaks.
2. Modify the parameter value of the object in the hashSet, and the parameter value is the field to calculate the hash value
When an object is stored in a HashSet, it cannot modify the fields in the object that contribute to the hash, otherwise the modified hash of the object will be different from the hash originally stored in the HashSet. In this case, Even if the Contains method retrieves an object from the HashSet collection using the current reference to the object as an argument, it returns a result that the object cannot be found, which would result in the inability to remove the current object from the HashSet collection, causing a memory leak.
3. Set the connection number and shutdown time of the machine
Memory leaks can also result from having expensive connections open for long periods of time.
2.3.3. Several cases of memory overflow
1. Overflow of heap memory (outOfMemoryError: Java Heap space)
In the JVM specification, memory in the heap is used to generate object instances and arrays. If subdivided, the heap memory can also be divided into young and old generations, with the young generation including one Eden region and two survivor regions. When a new object is generated, the memory request process is as follows:
A. JVM tries to allocate the memory needed to create new objects in Eden; B. If the memory size is sufficient, the application ends; otherwise, the next step; C. JVM starts youngGC and tries to release inactive objects in Eden area. If Eden space is still insufficient for new objects after release, it tries to put some active objects in Eden into Survivor area; D. The Survivor area is used as the intermediate exchange area between Eden and OLD. When the old area has enough space, objects in the Survivor area will be moved to the old area; otherwise, they will be kept in the Survivor area. E. The JVM will perform full GC in the OLD region when there is not enough space in the OLD region. F. After full GC, if Survivor and OLD zones still cannot store some objects copied from Eden, so that JVM cannot create a memory region for the new objects in Eden zone, an “out of memory error” will occur: outOfMemoryError: Java Heap space2, method area memory overflow (outOfMemoryError: Permgem space)
In the JVM specification, the method area mainly holds class information, constants, static variables, and so on. Therefore, if the program loads too many classes, or uses such dynamic proxy generation techniques as reflection and GCLIb, it may cause memory overflow in this area. Generally, the error message for memory overflow in this area is outOfMemoryError: permgem space
3, thread stack overflow (Java. Lang. StackOverflowError)
A thread stack is a piece of memory structure that is unique to a thread, so any problem with the thread stack must be a fault of a thread running. Thread stack overflows are usually caused by too much recursion or too many levels of method calls. The stack overflow error information is: Java. Lang. StackOverflowError
4. What if there is a memory leak or overflow?
To resolve exceptions in this area, a common approach is to first analyze heap dump snapshots through a Memory image analysis tool (such as Eclipse Memory Analyzer). The focus is to determine whether the objects in Memory are necessary. In this case, we need to know whether there is a Memory Leak or Memory Overflow.
If it is a memory leak, you can further look at the leak object’s chain of references to GC Roots through the tool. You can then find out how the leak objects are associated with GC Roots and cause the garbage collector to fail to reclaim them automatically. With the information of the type of the leaking object and the GC Roots reference chain, the location of the leaking code can be more accurately located.
If there are no leaks, in other words, objects in memory that really must be alive, you should check the heap parameters of the virtual machine (-XMX and -xMS) against the physical memory of the machine to see if they can be increased, and code to see if some objects are too long in life and hold state. Try to reduce the memory consumption of the program’s runtime.
5. How to avoid memory leakage and overflow
(a) Release references to useless objects as soon as possible (b) use stringBuffers instead of strings, where each String occupies an area of memory independently (c) use static variables as little as possible, since static variables are stored in the permanent generation (method area). The permanent generation basically does not participate in garbage collection d, avoids creating objects e in the loop, opens large files, or takes too much data from the database at once, so it is easy to run out of memory, so calculate the maximum amount of data in these places, and set the minimum and maximum memory space values required. 2.4 Details about JVM parameters
2.4.1 Description of heap memory Parameters
-XMX10G: Sets the maximum available memory of the JVM to 10 GB. -xMS10g: Sets the JVM to force 10G memory. This value can be set to the same as -xmx to avoid the JVM reallocating memory after each garbage collection. -Xmn8g: Sets the size of the young generation to 8 GB. Total heap size = young generation size + Old generation size + persistent generation size. The permanent generation has a fixed size of 64m, so increasing the young generation will reduce the size of the old generation. This value has a significant impact on system performance. Sun officially recommends setting it to 3/8 of the entire heap. -xSS128K: Sets the stack size for each thread. After JDK5.0, the stack size of each thread is 1M. Before JDK5.0, the stack size of each thread is 256K. The size of memory required by more applied threads is adjusted. Reducing this value generates more threads for the same physical memory. However, the operating system has a limit on the number of threads in a process, which cannot be generated indefinitely. The experience value is about 3000~5000. -xx :NewRatio=4: Sets the ratio of the young generation (including Eden and two Survivor zones) to the old generation (excluding persistent generation). If it is set to 4, the ratio of young generation to old generation is 1:4, and young generation accounts for 1/5 of the whole stack. -xx :SurvivorRatio=4: sets the size ratio of Eden zone to Survivor zone in young generation. If it is set to 4, the ratio of two Survivor zones to one Eden zone is 2:4, and one Survivor zone accounts for 1/6 of the entire young generation. -xx :MaxPermSize=16m: Set the size of the persistent generation to 16m
.
XX:
MaxTenuringThreshold
0: Sets the maximum garbage age. If set to 0, the young generation object passes through the Survivor zone and goes directly to the old generation. For the older generation of more applications, can improve efficiency. If you set this value to a large value, young generation objects will be copied multiple times in Survivor zones, increasing the lifetime of the object in the young generation and increasing the probability that it will be reclaimed in the young generation.
-XX:+CMSClassUnloadingEnabled
This parameter indicates whether class unload is enabled when using the CMS garbage collection mechanism. This is set to disabled by default so if you want to enable this feature you need to do it in Java
Parameters explicitly set the following parameters:
XX:+CMSClassUnloadingEnabled If you enable CMSClassUnloadingEnabled
Garbage collection cleans up persistent generations and removes classes that are no longer used. This parameter is available only in
UseConcMarkSweepGC
It is also useful if enabled. The parameters are as follows:
XX:+UseConcMarkSweepGC
-XX:+CMSPermGenSweepingEnabled
This parameter indicates whether persistent generations are cleaned. The default is not clean, so we need to explicitly set this parameter to debug persistent generation memory overflow problems. This parameter was removed in Java6, so you need to use it
XX:+CMSClassUnloadingEnabled
If you are using Java6 or later. The argument to resolve the persistent memory size problem would look something like this:
XX:
MaxPermSize
128m
XX:+UseConcMarkSweepGCXX:+CMSClassUnloadingEnabled
XX:+CMSParallelRemarkEnabled
Reduce the marked pause 2.4.2 Detailed description of collector parameters
The JVM gives you three options: a serial collector, a parallel collector, and a concurrent collector, but serial collectors are only suitable for small amounts of data, so the choice here focuses on parallel and concurrent collectors. By default, JDK5.0 used a serial collector until now, and if you want to use another collector you need to add parameters at startup. After JDK5.0, the JVM makes a judgment based on the current system configuration.
XX:+UseSerialGC
: Sets the serial collector
XX:+UseParallelGC
: Sets the parallel collector
XX:+UseParalledlOldGC
: Sets the parallel generation collector
XX:+UseConcMarkSweepGC: Sets the concurrent collector
Parallel collector setup
-XX:
ParallelGCThreads
N: Sets the CPU used by the parallel collector for collection
The number. Collect the number of threads in parallel.
XX:
MaxGCPauseMillis
n
: Sets the maximum pause time for parallel collection
XX:
GCTimeRatio
N: Sets the garbage collection time as a percentage of the program running time. Formula for 1 / (1 + n)
Concurrent collector setup
-xx :+CMSIncrementalMode: set it to incremental mode. Applies to a single CPU
Situation.
XX:
ParallelGCThreads
N: Number of cpus used when the young generation collection mode of the concurrent collector is set to parallel collection. Collect the number of threads in parallel. Throughput first parallel collector As described above, the parallel collector is mainly to achieve a certain throughput as the goal, suitable for science and technology and background processing, etc. Typical configuration:
java
Xmx3800m
Xms3800m
Xmn2g
Xss128k
XX:+UseParallelGC
XX:
ParallelGCThreads
20
Parameter interpretation
-XX:+UseParallelGC
Select the garbage collector as a parallel collector. This configuration is valid only for the young generation. In this configuration, the young generation uses concurrent collection, while the old generation still uses serial collection.
XX:
ParallelGCThreads
20: Configure the number of threads for the parallel collector, i.e., how many threads do garbage collection together at the same time. This value is best configured to be equal to the number of processors.
java
Xmx3550m
Xms3550m
Xmn2g
Xss128k
XX:+UseParallelGC
XX:
ParallelGCThreads
20
XX:+UseParallelOldGC
Parameter interpretation
-xx :+UseParallelOldGC: Sets the garbage collection mode for the aged generation to parallel collection. JDK6.0 supports parallel collection for older generations.
java
Xmx3550m
Xms3550m
Xmn2g
Xss128k
XX:+UseParallelGC
XX:
MaxGCPauseMillis
100
Parameter interpretation
-XX:
MaxGCPauseMillis
100: Sets the maximum time for each young generation garbage collection. If this time cannot be met, the JVM automatically resizes the young generation to meet this value.
java
Xmx3550m
Xms3550m
Xmn2g
Xss128k
XX:+UseParallelGC
XX:
MaxGCPauseMillis
100
XX:+UseAdaptiveSizePolicy
Parameter interpretation
-xx :+UseAdaptiveSizePolicy: After this option is set, the parallel collector automatically selects the size of the young generation area and the proportion of Survivor areas to achieve the minimum corresponding time or collection frequency specified by the target system. It is recommended that the parallel collector be enabled all the time when using the parallel collector. As mentioned above, the concurrent collector is designed to ensure system response time and reduce pauses during garbage collection. Applicable to application servers and telecommunications. Typical configuration:
java
Xmx3550m
Xms3550m
Xmn2g
Xss128k
XX:
ParallelGCThreads
20
XX:+UseConcMarkSweepGC
XX:+UseParNewGC
Parameter interpretation
-xx :+UseConcMarkSweepGC: Enables concurrent collection for the elderly user. After this is configured in the test, -xx:
NewRatio
The configuration of 4 is invalid for unknown reasons. So, in this case, the best way to use the size of the young generation is minus Xmn
Settings.
XX:+UseParNewGC: sets concurrent collection for the young user. Can be used in conjunction with CMS collection. Above JDK5.0, the JVM will set itself based on the system configuration, so this value is no longer required.
java
Xmx3550m
Xms3550m
Xmn2g
Xss128k
XX:+UseConcMarkSweepGC
XX:
CMSFullGCsBeforeCompaction
5
XX:+UseCMSCompactAtFullCollection
Parameter interpretation
– XX: CMSFullGCsBeforeCompaction: as the concurrent collector wrong memory space is compressed, sorting, so run after a period of time will produce “fragments”, results in lower operation efficiency. This value sets how many GC runs
After the memory space compression, collation.
XX: + UseCMSCompactAtFullCollection: open to the compression of old generation. Performance may be affected, but fragmentation can be eliminated. 2.4.3 Auxiliary Information Parameter description
The JVM provides a number of command-line arguments that print information for debugging. The main ones are as follows:
PrintGC: [GC118250K->113543K(130112K),
0.0094143 secs]
[FullGC121376K->10414K(130112K),
0.0650971 secs]
XX:+PrintGCDetails: [GC[DefNew:
8614K->781K(9088K),
0.0123035 secs]
118250K->113543K(130112K),
0.0124633 secs]
[GC[DefNew:
8614K->8614K(9088K),
0.0000665 secs] [Tenured:
112761K->10414K(121024K),
0.0433488 secs]
121376K->10414K(130112K),
0.0436268 secs]
XX:+PrintGCTimeStamps
XX:+PrintGC: PrintGCTimeStamps can be mixed with the above two output forms: 11.851:
[GC98328K->93620K(130112K),
0.0082960 secs
]
XX: + PrintGCApplicationConcurrentTime: print before each garbage collection, the execution time of the procedure not interrupt. The output form can be mixed with the above: Applicationtime:
0.5291524
seconds
XX: + PrintGCApplicationStoppedTime: print recycling program suspended during the period of time. Totaltimeforwhich Application Threads were Stopped:
0.0468229
seconds
XX:PrintHeapAtGC: prints detailed stack information before and after GC. Output format: 34.702
[GC{Heap
before gc invocations
7:def
newgeneration total55296K,used52568K
[0x1ebd0000,
0x227d0000,
0x227d0000)eden space49152K,
99%used[0x1ebd0000,
0x21bce430,
0x21bd0000)fromspace6144K,
55%used[0x221d0000,
0x22527e10,
0x227d0000)to space6144K,
0%used[0x21bd0000,
0x21bd0000,
0x221d0000)tenured generation total69632K,used2696K
[0x227d0000,
0x26bd0000,
0x26bd0000)the space69632K,
3%used[0x227d0000,
0x22a720f8,
0x22a72200,
0x26bd0000)compacting perm gen total8192K,used2898K
[0x26bd0000,
0x273d0000,
0x2abd0000)the space8192K,
35%used[0x26bd0000,
0x26ea4ba8,
0x26ea4c00,
0x273d0000)ro space8192K,
66%used[0x2abd0000,
0x2b12bcc0,
0x2b12be00,
0x2b3d0000)rw space12288K,
46%used[0x2b3d0000,
0x2b972060,
0x2b972200,
0 x2bfd0000) 34.735:
[DefNew:
52568K->3433K(55296K),
0.0072126 secs]
55264K->6615K(124928K)Heap
after gc invocations
8:def
newgeneration total55296K,used3433K
[0x1ebd0000,
0x227d0000,
0x227d0000)eden space49152K,
0%used[0x1ebd0000,
0x1ebd0000,
0x21bd0000)fromspace6144K,
55%used[0x21bd0000,
0x21f2a5e8,
0x221d0000)to space6144K,
0%used[0x221d0000,
0x221d0000,
0x227d0000)tenured generation total69632K,used3182K
[0x227d0000,
0x26bd0000,
0x26bd0000)the space69632K,
4%used[0x227d0000,
0x22aeb958,
0x22aeba00,
0x26bd0000)compacting perm gen total8192K,used2898K
[0x26bd0000,
0x273d0000,
0x2abd0000)the space8192K,
35%used[0x26bd0000,
0x26ea4ba8,
0x26ea4c00,
0x273d0000)ro space8192K,
66%used[0x2abd0000,
0x2b12bcc0,
0x2b12be00,
0x2b3d0000)rw space12288K,
46%used[0x2b3d0000,
0x2b972060,
0x2b972200,
0x2bfd0000)},
0.0757599 secs
]
Xloggc:filename: Records related log information to a file for analysis. Tuning summary
1, young generation size selection
Response time first applications: Set as large as possible until approaching the minimum response time limit of the system (selected based on the actual situation). In this case, the frequency of young generation collection is the smallest. At the same time, reduce the number of objects that reach the aged generation. Throughput first applications: as large as possible, possibly up to Gbit. Since there is no response time requirement, garbage collection can be done in parallel and is generally suitable for applications with more than 8 cpus.
2, the size of the elderly generation selection
Response time first applications: Older generations use concurrent collectors, so their size needs to be carefully set, generally taking into account parameters such as concurrent session rate and session duration. If the heap is set too small, memory fragmentation, high recycle rates, and application pauses can result in traditional token cleanup. If the heap is large, it takes longer to collect. The optimal scheme generally needs to refer to the following data:
Concurrent garbage collection persistent generation Number of concurrent collections The ratio of the time spent on the collection of traditional GC information by the young generation and the old generation reduces the time spent by the young generation and the old generation, which generally improves the efficiency of the application
Typically throughput-first applications have a large young generation and a small old generation. The reason for this is that it is possible to recycle most of the short-term objects and reduce the medium-term objects, while the old generation is used to store the long-term living objects.
4. Fragmentation problems caused by smaller heaps
Because the concurrent collector of the older generation uses a mark-and-sweep algorithm, the heap is not compressed. When the collector collects, it merges adjacent Spaces so that they can be allocated to larger objects. However, when heap space is low, “fragmentation” can occur after running for a while, and if the concurrent collector cannot find enough space, the concurrent collector will stop and recycle using the traditional mark-and-sweep method. If there is a “fragments”, may need to have the following configuration: – XX: + UseCMSCompactAtFullCollection: when using the concurrent collector, open to the compression of old generation. – XX: CMSFullGCsBeforeCompaction = 0: the above configuration under the condition of open, set how much time after Full GC, here to compress old generation
Reference links: [https://blog.csdn.net/chizizhixin/article/details/87877265] [https://blog.csdn.net/shengmingqijiquan/article/details/77508471]
The system tuning has greatly improved the speed of the system, while learning the JVM memory management and garbage collection mechanism. Overall, the harvest was great. So sorted out to share with you, I hope to help the future contact of children’s shoes. If there is any mistake, please kindly instruct. Creation is not easy, give attention and praise yo ~
This article is published by OpenWrite!