preface
Is CPU time, the operating system there are many threads, each thread of the running time is decided by the CPU, the CPU will be distributed to each thread a time slice, time slice is a short length of time, if within the time slice, has held the thread, is 100%, we should realize that the CPU speed soon (very high frequency), Except for cpu-intensive operations, all types of tasks end in less than the time slice.
There are two types of memory overflow and memory leak
- Memory overflow: The memory allocated by the program exceeds the memory size of the physical machine. As a result, the memory cannot be allocated and an ERROR occurs in OOM
- Memory leak: The unused objects occupy the memory and do not release the memory. As a result, the memory is wasted. As a result, the memory of the physical machine is used up and an OOM error occurs
Specific operation
How to monitor the JVM, we can through a case study in understanding some of the actual operation, you can see the following code, the code below is just a scene of the simulation when, a risk control of the scene, the general bank or third party company at the time of grant loans to a person, will call this person’s credit have repayment ability, give a response.
import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public class FullGCTest {
// A class that simulates a bank card
private static class CardInfo {
// Smallholder's bank card information records
BigDecimal price = new BigDecimal(10000000.0);
String name = "Small farmer";
int age = 18;
Date birthdate = new Date();
public void m(a) {}}// Thread pool Timing thread pool
// Set the rejection policy
private static ScheduledThreadPoolExecutor executor = new ScheduledThreadPoolExecutor(50.new ThreadPoolExecutor.DiscardOldestPolicy());
public static void main(String[] args) throws Exception {
executor.setMaximumPoolSize(50);
for(;;) { modelFit(); Thread.sleep(100); }}/** * Risk assessment for bank cards */
private static void modelFit(a){
List<CardInfo> taskList = getAllCardInfo();
// Take out every piece of information
taskList.forEach(info -> {
// do something
executor.scheduleWithFixedDelay(() -> {
// Call M
info.m();
}, 2.3, TimeUnit.SECONDS);
});
}
private static List<CardInfo> getAllCardInfo(a){
List<CardInfo> taskList = new ArrayList<>();
// query 100 cards at a time
for (int i = 0; i < 100; i++) {
CardInfo ci = new CardInfo();
taskList.add(ci);
}
returntaskList; }}Copy the codeProgram design is relatively simple, is our credit card case to, such as credit card CardInfo class, we put the corresponding credit card call all the records from this person, after doing some business processing method of their response to it for processing and calculation, to see if our model conforms to the modelFit, Specific how to do, in the application has a class is a CardInfo, a method called getAllCardInfo, get 100 every time, after 100 the use of thread pool in calculation, the thread pool with ScheduledThreadPoolExecutor (time), After new comes out of the thread pool, 50 thread pools, and then does the corresponding business logic processing, which calls modelFit() and simulates a business pause in 100 milliseconds.
First we need to compile the Java file using javac command javac fullgctest. Java to compile, and then print GC logs for risk monitoring
To PrintGC logs: Java -xms200m -xmx200m -xx :+PrintGC FullGCTest
How do I know the JVM memory is too high?
In a company, if the JVM memory is too high, it is common for the operation and maintenance team to receive an alarm message, and then notify the corresponding developer to check, so how should the developer check, or how to troubleshoot?
1. Top View processes
After receiving the alarm information, take the top command to query
[root@root ~]# top
View the increasing memory and high CPU usage. After top you will see its PID (31061). It has a higher proportion.
2. Top-hp Query threads
Top -Hp 31061: select PID from Java process where CPU usage is high. Top -Hp 31061: select PID from Java process where CPU usage is high.
We will see each thread’s accounted for about the same, occasionally there will be one thread is higher, is higher in some of the threads, this small example will eventually become garbage collection threads take is higher, because of garbage collection is not coming, so you need to keep going back and forth to recycle, recycle every time a little, actually this kind of case is very likely your business logic thread, The proportion of business logic threads in that area is very high, which is when another command, JStack, is needed
3, jstack
After we know which thread is using top-HP, we can next use the jstack command, for example, we want to check the thread number 31083, 31061 is our process PID, we want to locate the CPU ratio of a certain thread is much higher than other cpus. So when we need to locate the problem in the thread, we need to write down the thread number (31083).
Since jStack uses a hexadecimal thread number, we need to convert 31083 to hexadecimal
Features:
- Each thread has its own thread number, which contains the status of the thread and can be observed to see if the thread is blocked. If a long wait or block occurs, the thread is faulty
4. Convert to hexadecimal
Since the thread ID in the Java thread file is hexadecimal, we need to convert the thread ID from decimal to hexadecimal commands:echo "obase=16; 31083" | bc
5, jStack usage analysis
[root@root ~]# jstack
Usage:
jstack [-l] <pid>
(to connect to running process)
jstack -F [-m] [-l] <pid>
(to connect to a hung process)
jstack [-m] [-l] <executable> <core>
(to connect to a core file)
jstack [-m] [-l] [server_id@]<remote server IP or hostname>
(to connect to a remote debug server)
Options:
-F to force a thread dump. Use when jstack <pid> does not respond (process is hung)
-m to print both java and native frames (mixed mode)
-l long listing. Prints additional information about locks
-h or -help to print this help message
Copy the code6. Jstack views the output
We can also use the JPS or Java ps – ef | Java to view your Java process, here we use the JPS to look at it
[root@root ~]# jps
[root@root ~]# jstack 31061
"pool-1-thread-3" #10 prio=5 os_prio=0 tid=0x00007f3568105800 nid=0x7961 waiting on condition [0x00007f35455cf000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f8a81148> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"pool-1-thread-2" #9 prio=5 os_prio=0 tid=0x00007f3568103800 nid=0x7960 waiting on condition [0x00007f35456d0000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f8a81148> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"pool-1-thread-1" #8 prio=5 os_prio=0 tid=0x00007f3568102000 nid=0x795f waiting on condition [0x00007f35457d1000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f8a81148> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007f35680b4000 nid=0x795d runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f35680b1000 nid=0x795c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f35680af000 nid=0x795b waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f35680ad800 nid=0x795a runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f356807c800 nid=0x7959 in Object.wait() [0x00007f3558301000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000f8a86b38> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
- locked <0x00000000f8a86b38> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f3568077800 nid=0x7958 in Object.wait() [0x00007f3558402000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000f8a86cf0> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0x00000000f8a86cf0> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
"main" #1 prio=5 os_prio=0 tid=0x00007f3568009800 nid=0x7956 waiting on condition [0x00007f356ed59000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at FullGCTest.main(FullGCTest.java:35)
"VM Thread" os_prio=0 tid=0x00007f356806e000 nid=0x7957 runnable
"VM Periodic Task Thread" os_prio=0 tid=0x00007f35680b7000 nid=0x795e waiting on condition
JNI global references: 205
Copy the code
In most cases, threads are analyzed based on thead dump, such as whether there is a deadlock or whether a thread has held a lock for a long time
In dump, threads generally have the following states: 1) RUNNABLE; 2) BLOCKED; 3) WAITING
Locked < 0x000000076BF62208 > Indicates that the thread locks the object 0x000000076BF62208. Waiting to Lock < 0x000000076BF62208 > The thread is waiting for the lock on the object 0x000000076BF62208. Waiting for Monitor Entry [0x000000001e21F000] Indicates that the thread enters the critical area of the monitor using the synchronized keyword and is in the “Entry Set” queue waiting for monitor. Waiting on < 0x0000000088CA3310 > (a java.lang.object) Waiting for the lock to be released
If you have a process with 100 threads waiting on a lock, and the thread that should not be blocked is blocked, and the thread that should be terminated is not terminated, you must find which thread holds the lock, you can search jstack dump information, find <0X… The thread status is RUNNABLE, which means that the thread is running but holds the lock for a long time, thus causing the entire thread to deadlock
Jstack analyzes deadlocks
public class TestDeadLock {
private static Object obj1 = new Object();
private static Object obj2 = new Object();
public static void main(String[] args) {
new Thread(new Thread1()).start();
new Thread(new Thread2()).start();
}
private static class Thread1 implements Runnable {
@Override
public void run(a) {
synchronized (obj1) {
System.out.println("Thread1 has acquired the lock of obj1!");
try {// The point of pausing for 2 seconds is to let Thread2 take the lock on obj2
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
synchronized (obj2) {
System.out.println("Thread1 has acquired the lock of obj2!"); }}}}private static class Thread2 implements Runnable {
@Override
public void run(a) {
synchronized (obj2) {
System.out.println("Thread2 has acquired the lock of obj2!");
try {
// The point of pausing for 2 seconds is to let Thread1 take the lock of obj1
Thread.sleep(2000);
} catch (Exception e) {
e.printStackTrace();
}
synchronized (obj1) {
System.out.println("Thread2 has acquired the lock of obj1!");
}
}
}
}
}
Copy the code
[root@root fuccGC]# jps
485 Bootstrap
9877 Jps
10629 QuorumPeerMain
9846 TestDeadLock
[root@root fuccGC]# jstack 9846
Copy the code
Use of memory monitoring tools
We can use the JVM’s own commands to monitor GC information:Jinfo pid:This command lists some details about the process[root@root ~]# jinfo 9846This is helpful, but it’s not very helpful, so you just have to remember to have this command, so you don’t have to go into it
Most of the time we use tools like JConsole/JVisualVM
1, the jconsole
These two tools can be used to track a process on a remote server from the local machine. As shown in the following figure, few people will install a graphical interface on a Linux server:
When we start our program, we add parameters:
java -Djava.rmi.server.hostname=101.XX.XXX.XX -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8080 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.rmi.port=8080 FullGCTest
Copy the code
The basic situation is that we can monitor memory information on our remote server
2、 jvisualvm
Double-click the jvisualvm
Select Remote – click the File button
This allows you to view the memory information for our remote service
Locating Memory Overflow (OOM)
If the interviewer asks us how to position OOM, but we cannot answer with graphical interface, because as a service, it is running constantly. When we open a JMX service, it will image the original operation efficiency of the service. Then what can we use without graphical system? There is also a graphical interface called Jprofiler which is the best to use, but this is paid, so it is not generally used.
If not the graphical interface what do we use, what do we use cmdline and arthas for? Used for testing, monitoring during testing ~
If a project is already online, what can we use instead of a graphical interface? We can use Jmap
jmap
[root@root ~]# jmap -histo 19086 | head -20 
Online system, especially large memory, jMAP execution will have a great impact on the process, even lag (not suitable for e-commerce)
- If HeapDump is set to OOM, the HeapDump file will be generated automatically.
java -Xms20M -Xmx20M -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError FullGCTest) - Many servers are backed up (high availability), stopping this server does not affect other servers
- Next time, hahaha
Today’s JVM course here, the original is not easy, you remember a key three even ~, like/collect more than 100, I was pregnant with twins also out of the next article.
I am a small farmer, afraid of what truth is infinite, further have further joy, everyone refueling!!