The phenomenon of

One night, I suddenly received an online alarm email, and a certain machine broke down. I immediately checked the log on the server, and OOM appeared.

OOM analysis

All servers online are configured-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/apps/srv/instance/teacher.xiaozao100.com/logs/heapdump.hprof This is very helpful for troubleshooting problems, and generates a heap of memory snapshots (hprof files) when an OOM occurs.Download the heapdump.hprof file from the server

Dump files are analyzed using Jprofiler

Jprofiler dump fileJust focus on itHeap WalkerUnder theCurrent Object SetandThread Dump

  • Current Object Set: Identify related objects (usually find large objects), find the corresponding thread
  • Thread Dump: View Thread information to identify method callers and locate specific service operations

As it is OOM caused by memory overflow, we need to pay attention to the memory usage, so according tosizeByte [] specifies the object that occupies the most memory. PS:

  • Incoming References: Shows by whom this object is referenced
  • Outgoing References: Displays other objects referenced by this object

View the reference object information for these objects:Click on the topBiggers ObjectsViewing large object information:And then clickShow In GraphView the call chain:Finally, we can see the call chain information for this object: From the above information, we can know which thread the call stack belongs toXNIO-1 task-292 So, we can go nowThread DumpTo view the details of the thread and locate theXNIO-1 task-292On the thread:Finally locate toteacherThe modulepageUnReadCommentsOn the interface, there are many logical objects queried by this interface. The main reason is that the objects of all classes led by the tutor are all queried into the memory at one time. As a result, the loaded objects into the memory are too large for GC to deal with in time.

The enclosedteacherThe running status of the module server at a certain time:

Now that you know the cause of the problem, interface optimization can solve the memory overflow problem.