At the end of 19 years, I summarized a “LeakCanary Principle from 0 to 1”. At that time, I was relatively satisfied, thinking that I knew more about this framework, Too young, Too Simple.
On Friday, a friend in the group asked, “Do you have any ideas for online memory leak detection?” .
Memory leak detection first thought is LeakCanary, can you see from LeakCanary to find some ideas?
This article does not begin at 0 to explain how LeakCanary works, so for a better reading experience, readers who are not familiar with how LeakCanary determines object memory leaks can start with LeakCanary Principles from 0 to 1.
This article will start from the follow-up work of LeakCanary after memory leak, analyze how LeakCanary found the strong reference chain of the leaked object, analyze the reason why LeakCanary cannot be directly used for online memory detection, and try to find some ideas of online memory leak detection.
Generate Dump files
After a memory leak is determined, “LeakCanary” calls the debug.dumphProfData (File File) function provided by the system, which will generate a memory snapshot of the virtual machine. The File format is.hprof. The dump File is usually 10+M in size. (Dump files in this article refer to memory snapshot. Hprof files)
//:RefWatcher.java
Retryable.Result ensureGone(final KeyedWeakReference reference, final long watchStartNanoTime).// Call debug.dumphprofdata (File File) internally
File heapDumpFile = heapDumper.dumpHeap(file); HeapDump heapDump = heapDumpBuilder.heapDumpFile(heapDumpFile) .referenceKey(reference.key) .build(); heapdumpListener.analyze(heapDump); .return DONE;
}
Copy the codeAfter the dump file is generated, LeakCanary will enclose the referenceKey of the leaked object and the dump file object in the HeapDump object, and then give it to ServiceHeapDumpListener for processing. Create the Leakcanary process in ServiceHeapDumpListener and start the service HeapAnalyzerService.
Parsing a Dump file
The dump file is parsed in the HeapAnalyzerService. The main logic is as follows:
//HeapAnalyzerService.java
// Create a parser
HeapAnalyzer heapAnalyzer =
new HeapAnalyzer(heapDump.excludedRefs, this, heapDump.reachabilityInspectorClasses);
// Use the profiler to analyze the dump file and get the analysis result
AnalysisResult result = heapAnalyzer.checkForLeak(heapDump.heapDumpFile, heapDump.referenceKey,
heapDump.computeRetainedHeapSize);
// Submit the analysis results to the Listener component
AbstractAnalysisResultService.sendResultToListener(this, listenerClassName, heapDump, result);
Copy the codeTo continue tracking analyzer logic, look at the checkForLeak method of HeapAnalyzer:
//: HeapAnalyer.java
public AnalysisResult checkForLeak(File heapDumpFile, String referenceKey,
boolean computeRetainedSize) {
// Read the dump file, parse the file contents and generate a Snapshot object
HprofBuffer buffer = new MemoryMappedFileBuffer(heapDumpFile);
HprofParser parser = new HprofParser(buffer);
Snapshot snapshot = parser.parse();
// Eliminate duplicate GcRoots objects
deduplicateGcRoots(snapshot);
// Find the leaked object in the Snapshot object using the referenceKey
Instance leakingRef = findLeakingReference(referenceKey, snapshot);
// Find the leak path
return findLeakTrace(analysisStartNanoTime, snapshot, leakingRef, computeRetainedSize);
}
Copy the codeIn the checkForLeak method:
- The binary data of the dump file is parsed first, and then the contents of the file are stored
SnapshotObject, this can be taken fromSnapshotTo obtain memory information about the JVM. (Dump file format, interested canClick here toYou can also go to Squarecom.squareup.haha:haha+, LeakCanary uses this dump parsing library). - Then, in
SnapshotIn class,KeyedWeakReference且referenceKeyThe corresponding leak objectInstence. - Finally, in
SnapshotTo find the leak objectInstenceLeak strong reference chain
Find reference chains
How is the reference chain of the leaking object found? FindLeakTrace ();
//: HeapAnalyer.java
private AnalysisResult findLeakTrace(long analysisStartNanoTime, Snapshot snapshot,
Instance leakingRef, boolean computeRetainedSize) {
// Create the shortest path finder
ShortestPathFinder pathFinder = new ShortestPathFinder(excludedRefs);
// Use the finder to find the leaked instance node in snapshot
ShortestPathFinder.Result result = pathFinder.findPath(snapshot, leakingRef);
// Construct the shortest reference chain using node information
LeakTrace leakTrace = buildLeakTrace(result.leakingNode);
String className = leakingRef.getClassObj().getClassName();
long retainedSize = AnalysisResult.RETAINED_HEAP_SKIPPED;
// Encapsulate the node information of the leaking instance in an AnalysisResult object and return it
return leakDetected(result.excludingKnownLeaks, className, leakTrace, retainedSize,
since(analysisStartNanoTime));
}
Copy the codeTurning on the findPath function of ShortestPathFinder, it’s easy to see that it does a BFS walk through the heap structure of each GcRoot’s reference chain, then wraps the node where the leaking instance is in a Result and returns it.
//: ShortestPathFinder.java
Result findPath(Snapshot snapshot, Instance leakingRef) {
// Add all GcRoot nodes to the queue
enqueueGcRoots(snapshot);
LeakNode leakingNode = null;
while(! toVisitQueue.isEmpty() || ! toVisitIfNoPathQueue.isEmpty()) { LeakNode node;if(! toVisitQueue.isEmpty()) { node = toVisitQueue.poll(); }// Find the instance and end the traversal
if (node.instance == leakingRef) {
leakingNode = node;
break;
}
// Double check
if (checkSeen(node)) {
continue;
}
// Bind the node to its parent in visit
if (node.instance instanceof RootObj) {
visitRootObj(node);
} else if (node.instance instanceof ClassObj) {
visitClassObj(node);
} else if (node.instance instanceof ClassInstance) {
visitClassInstance(node);
} else if (node.instance instanceof ArrayInstance) {
visitArrayInstance(node);
} else {
throw new IllegalStateException("Unexpected type for "+ node.instance); }}return new Result(leakingNode, excludingKnownLeaks);
}
Copy the codeNow, how do you create a shortest path reference chain once you have the leak object node?
private LeakTrace buildLeakTrace(LeakNode leakingNode) {
List<LeakTraceElement> elements = new ArrayList<>();
// We iterate from the leak to the GC root
LeakNode node = new LeakNode(null.null, leakingNode, null);
// Add information from the leaking node to the list in reverse order
while(node ! =null) {
LeakTraceElement element = buildLeakElement(node);
if(element ! =null) {
elements.add(0, element);
}
node = node.parent;
}
List<Reachability> expectedReachability =
computeExpectedReachability(elements);
return new LeakTrace(elements, expectedReachability);
}
Copy the codeAt this point, the shortest reference chain for the leaking object has been found. Finally, the program uses the AnalysisResult object to save and return the information of the shortest reference chain.
The Listener component
In the first step of the analysis, we have seen that the AnalysisResult will be processed by a Listener component, which is DisplayLeakService. In DisplayLeakService there is a key piece of code:
@Override protected final void onHeapAnalyzed(HeapDump heapDump, AnalysisResult result) {
// Rename the dump file
heapDump = renameHeapdump(heapDump);
// Save the AnalysisResult object in the xxx.hprof.result fileresultSaved = saveResult(heapDump, result); . PendingIntent = DisplayLeakActivity.createPendingIntent(this, heapDump.referenceKey);
// New notification id every second.
int notificationId = (int) (SystemClock.uptimeMillis() / 1000);
showNotification(this, contentTitle, contentText, pendingIntent, notificationId);
}
private boolean saveResult(HeapDump heapDump, AnalysisResult result) {
File resultFile = new File(heapDump.heapDumpFile.getParentFile(),
heapDump.heapDumpFile.getName() + ".result");
FileOutputStream fos = null;
try {
fos = new FileOutputStream(resultFile);
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(heapDump);
oos.writeObject(result);
return true;
} catch (IOException e) {
CanaryLog.d(e, "Could not save leak analysis result to disk.");
} finally {
if(fos ! =null) {
try {
fos.close();
} catch (IOException ignored) {
}
}
}
return false;
}
Copy the codeIn the service component, the AnalysisResult object is written into the xxx.hprof.result file. At the same time, the service will display a Notification and after the Notification is clicked, the leak information will be displayed via DisplayLeakActivity.
The display of the leak reference chain
Finally, take a look at how our usual display of DisplayLeakActivity shows the reference chain of the leaked object. (Maybe see here you can also come out)
//:DisplayLeakActivity.java
@Override protected void onResume(a) {
super.onResume();
LoadLeaks.load(this, getLeakDirectoryProvider(this));
}
Copy the codeLook again at LoadLeaks# load ();
//: LoadLeaks.java
LoadLeaks is a subclass of Runnable
static final List<LoadLeaks> inFlight = new ArrayList<>();
static final Executor backgroundExecutor = newSingleThreadExecutor("LoadLeaks");
static void load(DisplayLeakActivity activity, LeakDirectoryProvider leakDirectoryProvider) {
LoadLeaks loadLeaks = new LoadLeaks(activity, leakDirectoryProvider);
inFlight.add(loadLeaks);
backgroundExecutor.execute(loadLeaks);
}
@Override public void run(a) {
final List<Leak> leaks = new ArrayList<>();
List<File> files = leakDirectoryProvider.listFiles(new FilenameFilter() {
@Override public boolean accept(File dir, String filename) {
return filename.endsWith(".result"); }});for (File resultFile : files) {
FileInputStream fis = new FileInputStream(resultFile);
ObjectInputStream ois = new ObjectInputStream(fis);
HeapDump heapDump = (HeapDump) ois.readObject();
AnalysisResult result = (AnalysisResult) ois.readObject();
leaks.add(new Leak(heapDump, result, resultFile));
mainHandler.post(new Runnable() {
@Override public void run(a) {
inFlight.remove(LoadLeaks.this);
if(activityOrNull ! =null) { activityOrNull.leaks = leaks; activityOrNull.updateUi(); }}});Copy the codeDisplayLeakActivity in the onResume method, uses the thread pool to read all the AnalysisResult objects in the XXX.prof result file, Use handler#post() to add them to the Activity member variable Leaks on the main thread and refresh the Activity interface at the same time.
The. Hprof file and the. Hprof result file will be deleted when the delete button is clicked;
void deleteVisibleLeak(a) {
final Leak visibleLeak = getVisibleLeak();
AsyncTask.SERIAL_EXECUTOR.execute(new Runnable() {
@Override public void run(a) {
File heapDumpFile = visibleLeak.heapDump.heapDumpFile;
File resultFile = visibleLeak.resultFile;
boolean resultDeleted = resultFile.delete();
if(! resultDeleted) { CanaryLog.d("Could not delete result file %s", resultFile.getPath());
}
boolean heapDumpDeleted = heapDumpFile.delete();
if(! heapDumpDeleted) { CanaryLog.d("Could not delete heap dump file %s", heapDumpFile.getPath()); }}}); visibleLeakRefKey =null;
leaks.remove(visibleLeak);
updateUi();
}
void deleteAllLeaks(a) {
final LeakDirectoryProvider leakDirectoryProvider = getLeakDirectoryProvider(this);
AsyncTask.SERIAL_EXECUTOR.execute(new Runnable() {
@Override public void run(a) { leakDirectoryProvider.clearLeakDirectory(); }}); leaks = Collections.emptyList(); updateUi(); }Copy the codeIn order to achieve a better display effect, the display will beautify the format of the node information in the reference chain, and the string will be displayed in HTML format. The specific logic can be viewed in the DisplayLeakAdapter class.
conclusion
LeakCanary determines that there is a memory leak by first generating a memory snapshot file (.hprof file), which is usually 10+M in size, then using the referenceKey to find the leaking instance, then using BFS in the snapshot heap to find the node where the instance is located. The minimum reference chain is generated by the node information. After the reference chain is generated, it is saved in the AnalysisResult object, and then the AnalysisResult object is written to the.hporf.result file, which is only tens of KB in size at this time. Finally, in DisplayLeakActivity’s onResume, all the.hprof.result files are read and displayed on the interface.
expand
LeakCanary directly online use will have what kind of problems, how to improve?
After understanding the work done by LeakCanary to determine the leak of the object, it is not difficult to know that there are some problems with applying LeakCanary directly online:
- After each memory leak, one is generated
.hprofThe file is then parsed and the results are written.hprof.result. Increase the burden of mobile phones, causing problems such as mobile phone card. - The same leakage problem will be repeated
.hprofFile, repeat analysis and write to disk. .hprofLarge file, information retrieval problem.
So how do you solve these problems?
- A memory threshold M can be set according to mobile phone information. When the used memory is less than M and there is a memory leak at this time, only the information of the leaking object is stored in the memory and not generated
.hprofFile. Generated when the use is greater than M.hprofFile; Of course, it can also determine that the number of leakage objects is greater than a specified value, generated and analyzed.hprofIn this case, the analysis results should contain information about one or more chains of leaked object references. - When the reference links are the same, the weight is weighted according to the actual situation.
- Do not directly retrieve
.hprofFile, you can choose to retrieve.hprof.resultFile, or local pair.hprof.resultFurther integration, re-weighting and optimization are carried out before retrieving. - Information on the reference chain is obfuscated, so it needs to be resolved further by versioning and maintaining the obfuscated mapping file.
Kangaroo level is limited, if the officials have a better idea can be discussed in the comments. The inadequacy of the article contains much, hope everyone not grudgingly gives instruction. In addition to memory optimization, advanced JVMTI monitoring object allocation, we have the strength to learn.