1. A brief introduction

The test environment pressure found a CPU spike, which is usually a sign of a lot of computational logic in the code, dead loops, and frequent GC due to memory leaks. Therefore, this kind of problem can be investigated from the above aspects.

2. Dump concept

Dump files are helpful for fault location and performance analysis. Dump files record memory status and thread execution during the running of the JVM. Common files include Thread dump and heap dump.

In simple terms, Thread dump records thread execution information and heap dump records JVM memory information.

  • Thread dump: a text file that records the execution of various threads in the JVM at a certain point in time, in the form of stacks. Thread dump files can be used to analyze where the program’s problems are, so specific code can be located and fixed. Thread dump is meaningful only after analyzing the id of the thread that occupies system resources.

  • Heap dump: a snapshot of the JVM heap at a certain point in time. Heap dump is a binary file that analyzes which objects occupy too much heap space to find the objects that are causing memory leaks.

Dump files are real-time and need to be generated when service problems occur. And generate several more files for comparison analysis ~ (there is thread dump generation script behind)

3. Rectify the CPU surge

Pid top ## 2: pid top ## 2: pid top ## 2 Top Hp [PID] ## Check the CPU high dar ps - mp pid - o THREAD, dar, time | sort - nr - k 2 # # 3. Pid printf '%x\n' pid ## 4 Direct use jstack positioning threads in A process execution, with color display colors highlighted jstack pid | grep 'nid' - A 30 - color # # we can also be exported to A file, the process information Check the thread execution jstack -l/process ID > jstack. Log # # from the file again for screening the cat jstack. Log | grep 'nid' - C5 - colorCopy the code

Problems can be found by viewing thread execution information:

  • If the threads that are consuming too much CPU are GC threads, then the CPU increase caused by memory leaks needs to be considered (GC status needs to be observed, and jMAP generates heap dump files to analyze large objects using MAT/ JVisualVM).
    • Ref-1: How to use JPS, jMAP, jstack, and jstat to monitor JVM performance
    • The ref-2: jstat command checks GC status for the JVM
  • If the cpu-hogging thread is not a GC thread, viewing thread information requires attention to whether the thread is executing, and non-executing threads usually do not cause CPU increases. Because thread dump information is real-time, the execution of the id of the thread that occupied the most CPU at the last moment may have been completed when you view the information about the thread. Therefore, you can use scripts to monitor the thread status in real time. (The following script can print the execution information of high CPU usage thread in real time.
    • Ref-3: uses jStack to locate thread stack information

Related articles

  1. Java Virtual Machine — Directives

Thread dump generates scripts

#! Pid mandatory # 2. Linenum Number of threads to display information Mandatory (default 10) if [$# -le 0]; then echo "usage: $0 <pid> [line-number]" exit 1 fi # java home if test -z $JAVA_HOME; then JAVA_HOME="/usr/local/jdk" fi # pid pid=$1 if test -z $pid; then echo "pid can not be null" exit else echo "checking pid($pid)" fi #check pid if test -z "$($JAVA_HOME/bin/jps -l | cut -d ' ' -f 1 | grep $pid)"; then echo "process $pid not exist" exit fi #line num line_num=$2 if test -z $line_num; then line_num=10 fi # generate stack & thread file stackfile=stack$pid.dump threadsfile=thread$pid.dump $JAVA_HOME/bin/jstack -l $pid >> $stackfile ps -mp $pid -o THREAD,tid,time | sort -nr -k 2 | awk '{if ($1! ="USER" && $2! = "0.0" && $8! ="-") print $8; }' | xargs printf "%x\n" >> $threadsfile tids="$(cat $threadsfile)" for tid in $tids; do echo "-------------thread($pid)----------------" cat $stackfile | grep $tid -A $line_num done rm -rf $stackfile rm -rf $threadsfileCopy the code