When locating problems, knowledge and experience are the key basis, data is the basis, and tools are the means of using knowledge to process data.

In actual troubleshooting and performance monitoring, operating system tools and Java VIRTUAL machine tools are often used together.

1. Operating system tools

1.1. Top: Displays the overall resource usage of the system

The top command is a commonly used performance analysis tool in Linux. It can display the resource usage of each process in the system in real time.

Some of the output from the top command on Linux is as follows:

The output of the top command can be divided into two parts: the first part is system statistics, and the second part is process information.

In the statistics:

  • The first line shows the task queue information from left to right: current system time, system running time, and current login user. The last load Average indicates the average load of the system.
  • The second line shows the process statistics, including the number of running processes, the number of sleeping processes, the number of stopped processes, and the number of zombie processes.
  • The third line is the CPU statistics, where US represents the CPU usage of user space, Sy indicates the CPU usage of the kernel space, ni indicates the CPU usage of the user process whose priority has changed, ID indicates the idle CPU usage, WA indicates the percentage of CPU time waiting for input and output, HI indicates the hardware interrupt request, and SI indicates the software interrupt request.

In the process information area, the resource usage of each process in the system is displayed. Meaning of main fields:

  • PID: indicates the ID of a process
  • USER: indicates the USER name of the process owner
  • PR: priority
  • NI: nice value. A negative value indicates a high priority, and a positive value indicates a low priority
  • TIME+ : indicates the total CPU TIME used by a process, expressed in 1/100 of a second
  • COMMAND: indicates the COMMAND name/COMMAND line

1.2. Vmstat: Monitors memory and CPU

Vmstat is also a full-featured performance monitoring tool. It collects statistics on CPU usage, memory usage, and swap usage.

The vmstat tool is usually used with two numeric parameters. The first parameter is the number of sampling intervals, in seconds, and the second parameter is the number of sampling times, such as:

The following command samples samples once per second, three times in total.

The meaning of the individual columns in the output:

classification instructions
Procs R: number of processes in the runqueue

B: Number of processes waiting for I/OS
Memory SWPD: uses the virtual memory size

Free: free memory size

Buff: Size of memory used for buffering

Cache: The size of memory used for caching
Swap: Si: Write size from swap area to memory per second

So: the memory size written to the swap area per second

IO :(current Linux version block size is 1024bytes)

Bi: Number of blocks read per second

Bo: Number of blocks written per second
system In: Number of interrupts per second, including clock interrupts

Cs: number of context switches per second
CPU (in percentage) Us: User process execution time

Sy: System process execution time

Id: idle time (including IO wait time), the idle time of the CPU, expressed as a percentage.

Wa: Wait for I/O time

1.3 iostat: Monitors I/O usage

Iostat provides disk I/O monitoring data:

Avg-cpu: %user % Nice % System % IOwait % Steal % Idle 1.44 0.00 0.39 0.00 0.00 98.17 Device: TPS kB_read/s kB_wrtn/s kB_read kB_wrtn sda 0.37 0.47 30.30 3561197 229837730 DM-0 0.44 0.33 29.97 2518272 227313194 Dm-1 0.12 0.13 0.33 1013276 2520308 DM-2 0.00 0.00 0.00 502 2068Copy the code

The above command shows an overview of CPU usage and disk I/O information.

The meaning of each column of the output result:

Iostat results panel AVG-CPU describes system CPU usage:

  • %user: indicates the percentage of time the CPU is in user mode.
  • %nice: indicates the percentage of time the CPU is in user mode with the nice value.
  • %system: indicates the percentage of time the CPU is in system mode.
  • % ioWAIT: the percentage of time the CPU waits for input and output to complete.
  • % STEAL: The percentage of the virtual CPU’s unconscious wait time while the hypervisor maintains another virtual processor.
  • %idle: indicates the percentage of idle CPU time.

1.4. Netstat: Monitors network usage

In a Web application that may need a network to run, you can monitor network traffic using the netstat command.

netstat -a
 Active Internet connections (servers and established)
 Proto Recv-Q Send-Q Local Address           Foreign Address         State
 tcp        0      0 localhost:30037         *:*                     LISTEN
 udp        0      0 *:bootpc                *:*
 
Active UNIX domain sockets (servers and established)
 Proto RefCnt Flags       Type       State         I-Node   Path
 unix  2      [ ACC ]     STREAM     LISTENING     6135     /tmp/.X11-unix/X0
 unix  2      [ ACC ]     STREAM     LISTENING     5140     /var/run/acpid.socket
 ...

Copy the code

The preceding command lists the information about all ports.

2. JDK performance monitoring tool

In addition to the familiar java.exe, javac.exe command line tools, in the JDK bin directory, there are some other tools. In addition to compiling and running Java programs, they can be used for a variety of scenarios, including packaging, deployment, signing, debugging, monitoring, operations and maintenance.

2.1. JPS: Viewing VM processes

JPS is similar to PS under Linux in that it lists the processes of Java programs.

JPS command format:

jps [ options ] [ hostid ] 
Copy the code

JPS command example:

Common options for JPS are shown in the table:

A list of options describe
-q Output only the process ID, ignoring the main class information
-l Print the full name of the main class, or print the path by executing the JAR package
-m Prints the arguments passed to the main() function of the main class when the virtual machine process starts
-v Output THE JVM parameters for starting the virtual machine process

2.2 jstat: View vm running information

Jsta is a powerful tool. It can display runtime data such as class loading, memory, garbage collection, just-in-time compilation, and so on in local or remote virtual machine processes.

The format of the jstat command is as follows:

jstat [ option vmid [interval[s|ms] [count]] ]
Copy the code

The option option represents the vm information that the user wants to query. The options are classified into three categories: class loading, garbage collection, and runtime compilation.

The following command output is about the ClassLoader of the Java process 5728. The output is twice.

The following example shows the output of heap information related to GC:

The main options of the jstat tool are:

A list of options describe
-class Monitor class loading, unload number, total space, and time taken for class loading
-gc Monitor the Java heap, including Eden area, 2 Survivor areas, old age, permanent generation or JDK1.8 meta space, capacity, used space, total garbage collection time, etc
-gccapacity The monitoring is basically the same as -GC, but the output focuses on the maximum and minimum space used by each region of the Java heap
-gcutil The monitoring is basically the same as -GC, but the output focuses on the percentage of total space used
-gccause The same function as -gcutil, but with an additional output of the cause of the last garbage collection
-gcnew Monitor new generation garbage collection
-gcnewcapacity The monitoring content is basically the same as -gcnew, with the output focusing on the maximum and minimum space used
-gcold Monitor old age garbage collection
-gcoldcapacity The monitoring content is basically the same as -gcold, with the output focusing on the maximum and minimum space used
-compiler Outputs information about methods compiled by the just-in-time compiler, time taken, and so on
-printcompilation Outputs methods that have been compiled on the fly

2.3 jINFO: View vm configurations

Jinfo is used to view and adjust VM parameters in real time.

Jinfo command format:

jinfo [ option ] pid
Copy the code

The following example shows the maximum age at which a new love interest is promoted to an older age. The current value of this parameter is not specified at application startup, but is viewed through jinfo.

Check to see if GC details are printed:

2.4 jMAP: Memory Image (export)

The jmap command is used to generate heapdump snapshots (commonly known as heapdump or dump files)

Jmap is not just for taking heap dump snapshots. It can also query Finalize execution queues, Java heap, and method areas for details such as space utilization, which collector is currently in use, etc.

Jmap command format:

jmap [ option ] vmid
Copy the code

As shown below, jmap is used to generate object statistics for the Java program with PID 5728 and output to dump.txt.

The structure of dump.txt is as follows:

A more important feature of JMap is to get a snapshot of the current heap of a Java application:

As shown, the heap snapshot of the application is exported to the heap.hprof file on drive D. After that, the heap file can be analyzed using a variety of tools.

Jmap tool main options:

options describe
-dump Generate a Java heap dump snapshot.
-finalizerinfo Displays objects in the F-queue waiting for the Finalizer thread to execute the Finalize method. Linux platform
-heap Displays Java heap details, such as which collector is used, parameter configuration, and generation. Linux platform
-histo Displays heap object statistics, including classes, number of instances, and total capacity
-permstat Display permanent generation memory status, JDk1.7, permanent generation
-F If the VM process does not respond to the -dump option, you can force a snapshot to be generated. Linux platform

2.5. Jhat: Heap dump snapshot analysis

The JDK provides the jhat command to be used in conjunction with Jmap to analyze heap dump snapshots generated by JMap. Jhat has a mini-HTTP /Web server built in that generates the analysis results of the heap dump snapshots and allows you to view them in a browser.

Take the previously generated heap.hprof as an example:

If Server is ready is displayed, enter http://localhost:7000/ in the browser to view the analysis result

2.6 jStack: Java stack trace

The jstack command is used to generate a snapshot (threaddump or Javacore file) of a VM thread at the current time.

Jstack command format:

jstack [ option ] vmid
Copy the code

Here is a partial result of the thread stack using stack:

Main options of the JStack tool:

options describe
-F Forces the output thread stack when a normally output request is not responded to
-l Displays additional information about locks in addition to the stack
-m A C/C ++ stack can be displayed if a native method is called

2.7. JCMD: multi-function command

Since jdk1.7, a new command-line tool, JCMD, has been added to implement all of the above commands except jstat.

For example, using JCMD to list all running Java virtual machines on the current system:

JMCD command format:

 jcmd <pid | main class> <command. | PerfCounter.print | -f file>Copy the code

JMCD tool main options:

options describe
help Print help information, for example: JCMD help []
ManagementAgent.stop Stop the JMX Agent
ManagementAgent.start_local Enable the local JMX Agent
ManagementAgent.start Open the JMX Agent
Thread.print The -l argument prints java.util.concurrent lock information, equivalent to: jstack
PerfCounter.print Equivalent to: jstat -j-djstat. showUnsupported=true -snap
GC.class_histogram Equivalent to jmap-histo
GC.heap_dump Equivalent to: jmap -dump:format=b,file=xxx.bin
GC.run_finalization Equivalent: system.runfinalization ()
GC.run Equivalent to: system.gc ()
VM.uptime The -date parameter prints the current time, in seconds, since the VM started
VM.flags The -all parameter displays all parameters, equivalent to jinfo-flags or jinfo-flag
VM.system_properties Equivalent to: jinfo-sysprops
VM.command_line Equivalent to: jinfo-sysprops grep command
VM.version Equivalent to: jinfo-sysprops grep version



Reference:

[1] : Zhou Zhipeng, In Depth Understanding the Java Virtual Machine: Advanced JVM Features and Best Practices

[2] : The Definitive Guide to Java Performance

[3] : Practical JAVA Virtual Machine JVM Fault Diagnosis and Performance Optimization

[4] : JCMD command details