Fixing memory leaks

Before starting memory leak monitoring Activity, Resource Canary will first try to fix possible memory leaks, it is achieved by monitoring ActivityLifeCycleCallbacks, when the Activity callback onDestroy, It attempts to unreference the Activity with InputMethodManager and View:

public static void activityLeakFixer(Application application) {
    application.registerActivityLifecycleCallbacks(new ActivityLifeCycleCallbacksAdapter() {
        @Override
        public void onActivityDestroyed(Activity activity) { ActivityLeakFixer.fixInputMethodManagerLeak(activity); ActivityLeakFixer.unbindDrawables(activity); }}); }Copy the code

In the case of InputMethodManager, it may refer to several views in the Activity, so we can de-reference them:

public static void fixInputMethodManagerLeak(Context destContext) {
    final InputMethodManager imm = (InputMethodManager) destContext.getSystemService(Context.INPUT_METHOD_SERVICE);
    final String[] viewFieldNames = new String[]{"mCurRootView"."mServedView"."mNextServedView"};
    for (String viewFieldName : viewFieldNames) {
        finalField paramField = imm.getClass().getDeclaredField(viewFieldName); .// If the View referenced by the IMM references the Activity, the reference relationship is broken
        if (view.getContext() == destContext) {
            paramField.set(imm, null); }}}Copy the code

For a View, it may be associated with an Activity through a listener or Drawable, so we need to remove every possible reference:

public static void unbindDrawables(Activity ui) {
    final View viewRoot = ui.getWindow().peekDecorView().getRootView();
    unbindDrawablesAndRecycle(viewRoot);
}

private static void unbindDrawablesAndRecycle(View view) {
    // Remove the generic View reference
    recycleView(view);

    // Different types of views may have different reference relationships
    if (view instanceof ImageView) {
        recycleImageView((ImageView) view);
    }

    if (view instanceofTextView) { recycleTextView((TextView) view); }... }// Disconnect the Listener, Drawable, and other references
private static void recycleView(View view) {
    view.setOnClickListener(null);
    view.setOnFocusChangeListener(null);
    view.getBackground().setCallback(null);
    view.setBackgroundDrawable(null); . }Copy the code

Monitoring memory leaks

The specific monitoring work is left to ActivityRefWatcher by the ResourcePlugin.

The ActivityRefWatcher methods start, stop, and destroy are used to start, stop, and destroy the listener thread, respectively. Take start as an example:

public class ActivityRefWatcher extends FilePublisher implements Watcher.IAppForeground {

    @Override
    public void start(a) {
        stopDetect();
        final Application app = mResourcePlugin.getApplication();
        if(app ! =null) {
            // Listen for the Activity's onDestroy callback to record information about the Activity
            app.registerActivityLifecycleCallbacks(mRemovedActivityMonitor);
            // Listen for onForeground callback to modify the polling interval according to the application visible state
            AppActiveMatrixDelegate.INSTANCE.addListener(this);
            // Start the listener threadscheduleDetectProcedure(); }}}Copy the code

Recording Activity Information

The mRemovedActivityMonitor is used to record Activity information when the Activity is called back onDestroy, including the Activity class name and a key generated based on the UUID:

// Used to record Activity information
private final ConcurrentLinkedQueue<DestroyedActivityInfo> mDestroyedActivityInfos;

private final Application.ActivityLifecycleCallbacks mRemovedActivityMonitor = new ActivityLifeCycleCallbacksAdapter() {

    @Override
    public void onActivityDestroyed(Activity activity) { pushDestroyedActivityInfo(activity); }};// Record the Activity information when the Activity is destroyed
private void pushDestroyedActivityInfo(Activity activity) {
    final String activityName = activity.getClass().getName();
    final UUID uuid = UUID.randomUUID();
    final String key = keyBuilder.toString(); // Based on the UUID
    final DestroyedActivityInfo destroyedActivityInfo = new DestroyedActivityInfo(key, activity, activityName);
    mDestroyedActivityInfos.add(destroyedActivityInfo);
}
Copy the code

DestroyedActivityInfo contains the following information:

public class DestroyedActivityInfo {
    public final String mKey; // Based on the UUID
    public final String mActivityName; / / the name of the class
    public final WeakReference<Activity> mActivityRef; / / weak references
    public int mDetectedCount = 0; // The number of times a memory leak is detected. By default, the number of times a memory leak is detected is 10 times
}
Copy the code

Start the listener thread

After the thread is started and the application is visible, the default polling task is sent to the default background thread (MatrixHandlerThread) for execution every 1min (specified by IDynamicConfig) :

// A custom thread switching mechanism for sending a specified task delay to the main thread/background thread for execution
private final RetryableTaskExecutor             mDetectExecutor;

private ActivityRefWatcher(...). {
    HandlerThread handlerThread = MatrixHandlerThread.getDefaultHandlerThread();
    mDetectExecutor = new RetryableTaskExecutor(config.getScanIntervalMillis(), handlerThread);
}

private void scheduleDetectProcedure(a) {
    // Send the task to the MatrixHandlerThread for execution
    mDetectExecutor.executeInBackground(mScanDestroyedActivitiesTask);
}
Copy the code

See below mScanDestroyedActivitiesTask polling tasks, it is an inner class, the code is very long, we bit by bit.

Set the sentry to detect GC execution

First, as discussed in the principles section of the previous article, ResourceCanary sets up a sentinel element to check if GC has actually been performed, and if not, it does not proceed:

private final RetryableTask mScanDestroyedActivitiesTask = new RetryableTask() {

    @Override
    public Status execute(a) {
        Create a weak reference to a temporary object
        final WeakReference<Object> sentinelRef = new WeakReference<>(new Object());
        // Try to trigger GC
        triggerGc();
        // Check whether the object referred to by the weak reference is alive to determine whether the virtual machine has actually performed GC
        if(sentinelRef.get() ! =null) {
            // System ignored our gc request, we will retry later.
            returnStatus.RETRY; }...return Status.RETRY; // Retry, the task will continue}};private void triggerGc(a) {
    Runtime.getRuntime().gc();
    Runtime.getRuntime().runFinalization();
}
Copy the code

Filter reported activities

Next, go through all DestroyedActivityInfo and mark the Activity to avoid duplicate reporting:

final Iterator<DestroyedActivityInfo> infoIt = mDestroyedActivityInfos.iterator();

while (infoIt.hasNext()) {
    if(! mResourcePlugin.getConfig().getDetectDebugger() && isPublished(destroyedActivityInfo.mActivityName)// Skip if already marked&& mDumpHprofMode ! = ResourceConfig.DumpMode.SILENCE_DUMP) { infoIt.remove();continue;
    }

    if (mDumpHprofMode == ResourceConfig.DumpMode.SILENCE_DUMP) {
        if(mResourcePlugin ! =null && !isPublished(destroyedActivityInfo.mActivityName)) { // Skip if already marked. }if (null! = activityLeakCallback) {// But ActivityLeakCallback will also be calledactivityLeakCallback.onLeak(destroyedActivityInfo.mActivityName, destroyedActivityInfo.mKey); }}else if (mDumpHprofMode == ResourceConfig.DumpMode.AUTO_DUMP) {
        ...
        markPublished(destroyedActivityInfo.mActivityName); / / tag
    } else if (mDumpHprofMode == ResourceConfig.DumpMode.MANUAL_DUMP) {
        ...
        markPublished(destroyedActivityInfo.mActivityName); / / tag
    } else { // NO_DUMP. markPublished(destroyedActivityInfo.mActivityName);/ / tag}}Copy the code

Multiple detection to avoid misjudgment

At the same time, a memory leak is considered only when repeated detections are greater than or equal to mMaxRedetectTimes (specified by IDynamicConfig, default is 10) and references to the Activity are still available:

while (infoIt.hasNext()) {
    ...

    // The Activity was collected
    if (destroyedActivityInfo.mActivityRef.get() == null) {
        continue;
    }

    // If the Activity is not collected, there may be a memory leak, but to avoid misjudgment, you need to repeat the detection several times, if the Activity can be retrieved, then the memory leak is considered
    // Problems are reported only in debug mode. Otherwise, only one log is printed
    ++destroyedActivityInfo.mDetectedCount;
    if(destroyedActivityInfo.mDetectedCount < mMaxRedetectTimes || ! mResourcePlugin.getConfig().getDetectDebugger()) { MatrixLog.i(TAG,"activity with key [%s] should be recycled but actually still \n"
                + "exists in %s times, wait for next detection to confirm.",
            destroyedActivityInfo.mKey, destroyedActivityInfo.mDetectedCount);
        continue; }}Copy the code

Note that problems are reported only in debug mode. Otherwise, only one log is printed.

Report the problem

For silence_dump and no_dump modes, it just logs the Activity name and calls back onDetectIssue:

final JSONObject resultJson = new JSONObject();
resultJson.put(SharePluginInfo.ISSUE_ACTIVITY_NAME, destroyedActivityInfo.mActivityName);
mResourcePlugin.onDetectIssue(new Issue(resultJson));
Copy the code

For manual_dump, it generates a notification using the Intent specified by ResourceConfig:

. Notification notification = buildNotification(context, builder); notificationManager.notify(NOTIFICATION_ID, notification);Copy the code

For auto_dump, it automatically generates a hprof file and analyzes it:

final File hprofFile = mHeapDumper.dumpHeap();
final HeapDump heapDump = new HeapDump(hprofFile, destroyedActivityInfo.mKey, destroyedActivityInfo.mActivityName);
mHeapDumpHandler.process(heapDump);
Copy the code

Generate the hprof file

The dumpHeap method does two things: generates a file and writes Hprof data to the file:

public File dumpHeap(a) {
    final File hprofFile = mDumpStorageManager.newHprofFile();
    Debug.dumpHprofData(hprofFile.getAbsolutePath());
}
Copy the code

The HeapDumpHandler then processes the file:

protected AndroidHeapDumper.HeapDumpHandler createHeapDumpHandler(...). {
    return new AndroidHeapDumper.HeapDumpHandler() {

        @Override
        public void process(HeapDump result) { CanaryWorkerService.shrinkHprofAndReport(context, result); }}; }Copy the code

The processing process is as follows:

private void doShrinkHprofAndReport(HeapDump heapDump) {
    // Crop the hprof file
    new HprofBufferShrinker().shrink(hprofFile, shrinkedHProfFile);
    // Compress the clipped hprof file
    zos = new ZipOutputStream(new BufferedOutputStream(new FileOutputStream(zipResFile)));
    copyFileToStream(shrinkedHProfFile, zos);
    // Delete old files
    shrinkedHProfFile.delete();
    hprofFile.delete();
    // Report the result
    CanaryResultService.reportHprofResult(this, zipResFile.getAbsolutePath(), heapDump.getActivityName());
}

private void doReportHprofResult(String resultPath, String activityName) {
    final JSONObject resultJson = new JSONObject();
    resultJson.put(SharePluginInfo.ISSUE_RESULT_PATH, resultPath);
    resultJson.put(SharePluginInfo.ISSUE_ACTIVITY_NAME, activityName);
    Plugin plugin =  Matrix.with().getPluginByClass(ResourcePlugin.class);
    plugin.onDetectIssue(new Issue(resultJson));
}
Copy the code

It can be seen that the original Hprof file is very large, so Matrix first makes a clipping optimization for it, then compreses the clipped file and deletes the old file, and finally calls back onDetectIssue to report the file location, Activity name and other information.

The results of the analysis

The sample

When a memory leak is detected, ActivityRefWatcher prints the following log:

activity with key [MATRIX_RESCANARY_REFKEY_sample.tencent.matrix.resource.TestLeakActivity_...]  was suspected to be a leaked instance. mode[AUTO_DUMP]Copy the code

If mode is AUTO_DUMP and mDetectDebugger is set to true, then a hprof file is also generated:

hprof: heap dump "/storage/emulated/0/Android/data/sample.tencent.matrix/cache/matrix_resource/dump_*.hprof" starting...
Copy the code

/sdcard/data/[package name]/ matrix_Resource folder will generate a zip file, such as:

/storage/emulated/0/Android/data/sample.tencent.matrix/cache/matrix_resource/dump_result_*.zip
Copy the code

The zip file contains a dump_*_shinked. Hprof file and a result.info file, which contains device information and key Activity information, such as:

# Resource Canary Result Infomation. THIS FILE IS IMPORTANT FOR THE ANALYZER !!sdkVersion=23 manufacturer=vivo hprofEntry=dump_323ff84d95424d35b0f62ef6a3f95838_shrink.hprof leakedActivityKey=MATRIX_RESCANARY_REFKEY_sample.tencent.matrix.resource.TestLeakActivity_8c5f3e9db8b54a199da6cb2abf68bd 12Copy the code

Take the zip file, enter the path parameters, and run CLIMain in matrix-resource-cancanary – Analyzer to get a result.json file:

{
    "activityLeakResult": {
        "failure": "null"."referenceChain": ["static sample.tencent.matrix.resource.TestLeakActivity testLeaks". ."sample.tencent.matrix.resource.TestLeakActivity instance"]."leakFound": true."className": "sample.tencent.matrix.resource.TestLeakActivity"."analysisDurationMs": 185."excludedLeak": false
    },
    "duplicatedBitmapResult": {
        "duplicatedBitmapEntries": []."mFailure": "null"."targetFound": false."analyzeDurationMs": 387}}Copy the code

Note that CLIMain’s analysis of duplicate bitmaps requires the reflection of the “mBuffer” field in the Bitmap, which was removed in API 26. Therefore, for devices with API 26 or greater, CLIMain can only analyze Activity memory leaks, not duplicate bitmaps.

The analysis process

The following is a brief analysis of the execution process of CLIMain, which is developed based on Square Haha. The execution process is divided into five steps:

Get the hprof file, sdkVersion and other information from result.info
Analyzing Activity leaks
Analyze duplicate bitmaps
Generates a result.json file and writes the results
Prints duplicate Bitmap images locally

public final class CLIMain {
    public static void main(String[] args) {
        doAnalyze();
    }

    private static void doAnalyze(a) throws IOException {
        // Get the hprof file, sdkVersion, etc. from result.info and start analyzing
        analyzeAndStoreResult(tempHprofFile, sdkVersion, manufacturer, leakedActivityKey, extraInfo);
    }

    private static void analyzeAndStoreResult(...). {
        // Analyze Activity memory leaks
        ActivityLeakResult activityLeakResult
                = new ActivityLeakAnalyzer(leakedActivityKey, ).analyze(heapSnapshot);

        // Analyze duplicate bitmaps
        DuplicatedBitmapResult duplicatedBmpResult
                = new DuplicatedBitmapAnalyzer(mMinBmpLeakSize, excludedBmps).analyze(heapSnapshot);

        // Generate result.json file and write the result
        final File resultJsonFile = new File(outputDir, resultJsonName);
        resultJsonPW.println(resultJson.toString());

        // Prints duplicate Bitmap images
        for (int i = 0; i < duplicatedBmpEntryCount; ++i) {
            finalBufferedImage img = BitmapDecoder.getBitmap(...) ; ImageIO.write(img,"png", os); }}}Copy the code

The key to Activity memory leak detection is to find the shortest reference path, which works as follows:

Get the Activity node from the leakedActivityKey field in result.info
Use a collection to store all nodes that have strong references to the Activity
From these nodes, the width-first search algorithm is used to find the nearest GC Root, which could be a static variable, a local variable in a stack frame, a JNI variable, etc

The principle of repeated Bitmap detection was introduced in the previous article and is skipped here.

conclusion

Implementation of Resource Canary

Registered ActivityLifeCycleCallbacks, monitor onActivityDestroyed method, through a weak reference to judge whether the memory leaks, use a background thread (MatrixHandlerThread) periodically test
A sentinel object is used to verify that the system has GC
If it is found that an Activity cannot be reclaimed, repeat the judgment three times (the default value of 0.6.5 code is 10 times), and require that more than two activities have been created since the Activity was recorded before it is considered as leaking (no corresponding code is found). In case the Activity is held by a local variable at the time of judgment, resulting in misjudgment
The same Activity is never reported twice

Resource Canary restrictions

Can only be run in more than Android 4.0 devices, because ActivityLifeCycleCallbacks is 14 to join in the API
Duplicate bitmaps on Android 8.0 and above cannot be analyzed because the mBuffer field of the Bitmap was removed in API 26

Configurable options

DumpMode. There are four types: no_dump (reporting the name of the Activity class), silence_dump (reporting the name of the Activity class and calling back ActivityLeakCallback), auto_dump (generating a heap dump file), and manual_dump (sending a notification)
Debug mode: DumpMode takes effect only in debug mode. Otherwise, logs are continuously generated
ContentIntent, which generates a notification when DumpMode is manual_dump, specifies the Activity to jump to
The polling interval for monitoring threads when visible/invisible is applied. The default is 1min and 20min respectively
MaxRedetectTimes: A memory leak is considered only if the Activity is still detected after repeated detections are greater than or equal to MaxRedetectTimes