Buried point general idea
Ordinary interface dot, do not package through dialog and other windowManager directly add view.
By listening for acitivty’s lifecycle in your Application, in the Resumed method, traverse all of the views in your Activity view and set the View to an AccessibilityDelegate, When a View generates a click or long_click event, it sends a message to the AccessibilityDelegate in response to the original Listener method, and then performs a click operation in sendAccessibilityEvent.
How does the client collect APP interface data?
Source: ViewSnapshot
1. Data structure of control tree: JSON
Recursive parsing starts from rootView, and each view corresponds to a JSON string in the following format:
{ "hashCode": "view.hashCode()", "id": "view.getId()", "index": "getChildIndex(view.getParent(), view)", "sa_id_name": "getResName(view)", "top": "view.getTop()", // Top position of this view relative to its parent "left": "view.getLeft()", // Left position of this view relative to its parent "width": "view.getWidth()", // Return the width of the your view "height": "View.getheight ()", // Return the height of your view "scrollX":" view.getscrollx ()", // Return the left scrolling position of this view "scrollY": "View.getscrolly ()",// returns the scroll top position of this view. "visibility": "view.getVisibility()", "translationX": "view.getTranslationX()", // The horizontal location of this view relative to its left position. view offset "translationY": "view.getTranslationY()", // The vertical location of this view relative to its top position "classes": ["view.getClass()", "view.getSuperclass()", "view.getSupersuperclass ()", "... up to the next level of object.class"], "importantForAccessibility": true, "clickable": false, "alpha": 1, "hidden": 0, "background": { "classes": [ "android.graphics.drawable.ColorDrawable", "android.graphics.drawable.Drawable" ], "dimensions": { "left": 0, "right": 1200, "top": 0, "bottom": 1920 }, "color": -328966 } "layoutRules": [], // RelativeLayout subview: ["child1.hashCode()", "child2.hashCode()", "..."] }Copy the codeDifferent types of views need to collect different attributes. Mixpanel’s solution is to use a configuration file to define which attributes of which objects to collect: config example
Detailed explanation of layoutRules
View has several important properties:
Android :background association method: getBackground(), setBackground(ColorDrawable), setBackgroundResource(int) View background Android :alpha Association methods: getAlpha(), setAlpha(float) Attribute description: view transparency, value between 0 and 1. 0 is completely transparent and 1 is completely opaque. Android: clickable associated methods: isClickable (), setClickable (Boolean) attribute description: view can click on the android: importantForAccessibility associated methods: IsImportantForAccessibility (), setImportantForAccessibility (int) attribute description: Describes whether or not this view is important for accessibility. If it is important, the view fires accessibility events and is reported to accessibility services that query the screen. Note: While not recommended, an accessibility service may decide to ignore this attribute and operate on all views in the view tree. Android :visibility association methods: getVisibility(), setVisibility(int) attribute description: "View visibility. There are three possible values: gone -- invisible and does not occupy view space; Invisible -- Invisible, but taking up space in view; Visible - visible"Copy the codeClick events and text editing events can be buried in Android:
Click on the event
A control that inherits android.view.view and whose.clickable() property is true triggers an event when clicked
Text editing event
A control inherited from Android.Widget. EditText that fires an event after editing
When you select a control to bury on the management page, the system automatically determines the event type to bury.
2. Screenshot data structure: JSON
A liveActivitie corresponds to an instance of RootViewInfo:
private static class RootViewInfo { public RootViewInfo(String activityName, View rootView) { this.activityName = activityName; this.rootView = rootView; this.screenshot = null; This. Scale = 1.0 f; } public final String activityName; public final View rootView; public CachedBitmap screenshot; public float scale; }Copy the codeConstruct json data after processing the RootViewInfo instance:
{ "activity": "info.activityName", "scale": "info.scale", "serialized_objects": { "rootObject": "info.rootView.hashCode()", "objects": [ "view json 1", "view json 2", "..." ] }, "image_hash": "info.screenshot.getImageHash", "screenshot": "info.screenshot.writeBitmapJSON" }Copy the codeSample screenshot data structure
How do I identify unique controls? How to solve the problem that some controls cannot be monitored?
(1) How to express the View path
Each view node is represented by a PathElement, and the absolute path of each view node is represented by List< pathFinder.pathElement >
The path example:
"path": [
{
"prefix": null,
"view_class": "com.android.internal.policy.PhoneWindow.DecorView",
"index": "-1",
"id": "-1",
"sa_id_name": null
},
{
"prefix": "shortest",
"view_class": "com.android.internal.widget.ActionBarOverlayLayout",
"index": "0",
"id": "16909220",
"sa_id_name": null
},
{
"prefix": "shortest",
"view_class": "android.widget.FrameLayout",
"index": "0",
"id": "16908290",
"sa_id_name": "android: content"
},
{
"prefix": "shortest",
"view_class": "android.widget.LinearLayout",
"index": "0",
"id": "-1",
"sa_id_name": null
},
{
"prefix": "shortest",
"view_class": "android.widget.Button",
"index": "0",
"id": "2131558506",
"sa_id_name": "btn"
}
]
Copy the code(2) Reflect the R file to get the View ID
Files and mResourcePackageName ResourceReader for android. R.i, dc lass. R.c lass of inner class id of all the static int variables, like:
public static final class id { ... public static final int btnAddAlarm=0x7f0d0055; public static final int btnPause=0x7f0d005c; public static final int btnReset=0x7f0d005e; public static final int btnResume=0x7f0d005d; . }Copy the codeOrganize the names and values of static int variables into Map< String, Integer > mIdNameToId and SparseArray< String > mIdToIdName for future use.
(3) What is the meaning of index?
Index assignment rule: All views in each ViewGroup are classified by Class. Then check whether there is a Resource Id. If there is a Resource Id, index = 0; otherwise, index = the sequence number (starting from 0) of sub-views of the Class type.
ViewSnapshot
private int getChildIndex(ViewParent parent, View child) { if (parent == null || ! (parent instanceof ViewGroup)) { return -1; } ViewGroup _parent = (ViewGroup) parent; final String childIdName = getResName(child); String childClassName = mClassnameCache.get(child.getClass()); int index = 0; for (int i = 0; i < _parent.getChildCount(); i++) { View brother = _parent.getChildAt(i); if (! Pathfinder.hasClassName(brother, childClassName)) { continue; } String brotherIdName = getResName(brother); if (null ! = childIdName && ! childIdName.equals(brotherIdName)) { continue; } if (brother == child) { return index; } index++; } return -1; }Copy the codeIndex initial value 0
(1) Compare the current child node with the brother node one by one;
(2) If brother is a child or the same type, proceed (3); Otherwise, return to (1);
(3) If the current child has childIdName (id) and is not the same as brother’s brotherIdName, return to (1); Otherwise, proceed (4);
Child == brother; index++;
By analyzing the above algorithms, the value of index has the following rules:
(1) if child has an id, the match between child and brother on the left side of the tree will fail. Execute continue, index++ will not be executed until index = 0 is matched successfully.
(2) if the child id does not exist, the left side of the tree structure is traversed. If brother is a child or the same type, index++ is used until the match succeeds. You can think of index as the number of subclasses or equivalents of Child in brothers on the left. Therefore, the order of sibling nodes also affects the value of index.
Note: The order of sibling nodes also affects the index value
What configuration information is returned on the Web configuration page?
Deliver the configuration example
How do I transfer APP interface data to the Web configuration page?
websocket
The WebSocketClient used by the Android is Java-WebSocket
Implement WebSocketClient
/** * EditorClient should handle all communication to and from the socket. It should be fairly naive and * only know how to delegate messages to the ViewCrawlerHandler class. */ private class EditorClient extends WebSocketClient { public EditorClient(URI uri, int connectTimeout) throws InterruptedException { super(uri, new Draft_17(), null, connectTimeout); } @Override public void onOpen(ServerHandshake handshakedata) { if (SensorsDataAPI.ENABLE_LOG) { Log.d(LOGTAG, "Websocket connected: " + handshakedata.getHttpStatus() + " " + handshakedata .getHttpStatusMessage()); } mService.onWebSocketOpen(); } @Override public void onMessage(String message) { // Log.d(LOGTAG, "Received message from editor:\n" + message); try { final JSONObject messageJson = new JSONObject(message); final String type = messageJson.getString("type"); if (type.equals("device_info_request")) { mService.sendDeviceInfo(messageJson); } else if (type.equals("snapshot_request")) { mService.sendSnapshot(messageJson); } else if (type.equals("event_binding_request")) { mService.bindEvents(messageJson); } else if (type.equals("disconnect")) { mService.disconnect(); } } catch (final JSONException e) { Log.e(LOGTAG, "Bad JSON received:" + message, e); } } @Override public void onClose(int code, String reason, boolean remote) { if (SensorsDataAPI.ENABLE_LOG) { Log.d(LOGTAG, "WebSocket closed. Code: " + code + ", reason: " + reason + "\nURI: " + mURI); } mService.cleanup(); mService.onWebSocketClose(code); } @Override public void onError(Exception ex) { if (ex ! = null && ex.getMessage() ! = null) { Log.e(LOGTAG, "Websocket Error: " + ex.getMessage()); } else { Log.e(LOGTAG, "Unknown websocket error occurred"); }}}Copy the codeHow to parse APP data on the Web configuration page?
An idea summed up by myself:
Depth-first traverses the control tree, printing the absolute position and path, clickable, and the following properties for each View:
{
"prefix": null,
"view_class": "com.android.internal.policy.PhoneWindow.DecorView",
"index": "-1",
"id": "-1",
"sa_id_name": null
}
Copy the codeWhen the click event occurs, obtain the click position coordinates, and then traverse all views in the Activity interface (controls are also views) to determine which View area contains the click position, thus determining which View is clicked.
To narrow the search, you can search only clickable views.
Application. ActivityLifecycleCallbacks concrete realization in where?
Source: $LifecycleCallbacks ViewCrawler
In the constructor of ViewCrawler registerActivityLifecycleCallbacks:
public ViewCrawler(Context context, String resourcePackageName) { ... mLifecycleCallbacks = new LifecycleCallbacks(); final Application app = (Application) context.getApplicationContext(); app.registerActivityLifecycleCallbacks(mLifecycleCallbacks); . }Copy the codeSee LifecycleCallbacks for the actual implementation:
private class LifecycleCallbacks implements Application.ActivityLifecycleCallbacks { public LifecycleCallbacks() { } void enableConnector() { mEnableConnector = true; mEmulatorConnector.start(); } void disableConnector() { mEnableConnector = false; mEmulatorConnector.stop(); } @Override public void onActivityCreated(Activity activity, Bundle bundle) { } @Override public void onActivityStarted(Activity activity) { } @Override public void onActivityResumed(Activity activity) { if (mEnableConnector) { mEmulatorConnector.start(); } mStartedActivities.add(activity); If (mStartedActivities. The size () = = 1) {/ / app to recover from a background SensorsDataAPI. SharedInstance (mContext). AppBecomeActive (); } for (String className : MDisabledActivity) {// Retrieve the current activity if (className.equals(activity.getClass().getCanonicalName())) {// Retrieve the current activity in the list of Activities that ignore monitoring. return; } } mEditState.add(activity); } @Override public void onActivityPaused(Activity activity) { mStartedActivities.remove(activity); mEditState.remove(activity); if (mEditState.isEmpty()) { mEmulatorConnector.stop(); } } @Override public void onActivityStopped(Activity activity) { if (mStartedActivities.size() == 0) { SensorsDataAPI.sharedInstance(mContext).appEnterBackground(); } } @Override public void onActivitySaveInstanceState(Activity activity, Bundle bundle) { } @Override public void onActivityDestroyed(Activity activity) { } private final EmulatorConnector mEmulatorConnector = new EmulatorConnector(); private boolean mEnableConnector = false; }Copy the codeThe realization of a ViewTreeObserver. OnGlobalLayoutListener in where?
Source: $EditBinding EditState
In order to deal with the dynamic layout of the page, we need to implement event monitoring in a single thread, through a loop operation, so that each event matches all the views of the current page. After measurement, there is no discernible impact on application interaction.
ViewTreeObserver. OnGlobalLayoutListener: when global layout in a tree view is changed or the visual state of a view in the view tree is changed, to invoke the callback function interface classes
private static class EditBinding implements ViewTreeObserver.OnGlobalLayoutListener, Runnable {
public EditBinding(View viewRoot, ViewVisitor edit, Handler uiThreadHandler) {
mEdit = edit;
mViewRoot = new WeakReference<View>(viewRoot);
mHandler = uiThreadHandler;
mAlive = true;
mDying = false;
final ViewTreeObserver observer = viewRoot.getViewTreeObserver();
if (observer.isAlive()) {
observer.addOnGlobalLayoutListener(this);
}
run();
}
@Override
public void onGlobalLayout() {
run();
}
@Override
public void run() {
if (!mAlive) {
return;
}
final View viewRoot = mViewRoot.get();
if (null == viewRoot || mDying) {
cleanUp();
return;
}
// ELSE View is alive and we are alive
mEdit.visit(viewRoot);
mHandler.removeCallbacks(this);
mHandler.postDelayed(this, 5000);
}
...
}
Copy the codeHow is the configuration of the Web configuration page performed? That is, how to automatically bury the point?
Principle: By listening for acitivty’s lifecycle in your Application, in the Resumed method, traverse all of the views in your Activity view and set the View to an AccessibilityDelegate, When a View generates a click or long_click event, it sends a message to the AccessibilityDelegate in response to the original Listener method, and then performs a click operation in sendAccessibilityEvent.
ViewCrawler$LifecycleCallbacks.onActivityResumed(activity) ->
mEditState.add(activity) ->
EditState.applyEditsOnActivity(activity) ->
EditState.applyChangesFromList(activity,rootView,List<ViewVisitor> changes)
Copy the codeFinal EditBinding binding = New EditBinding(rootView, Visitor, mUiThreadHandler)
// Must be called on UI Thread private void applyChangesFromList(final Activity activity, final View rootView, final List<ViewVisitor> changes) { synchronized (mCurrentEdits) { if (! mCurrentEdits.containsKey(activity)) { mCurrentEdits.put(activity, new HashSet<EditBinding>()); } final int size = changes.size(); for (int i = 0; i < size; i++) { final ViewVisitor visitor = changes.get(i); final EditBinding binding = new EditBinding(rootView, visitor, mUiThreadHandler); mCurrentEdits.get(activity).add(binding); }}}Copy the codeIt is worth noting that the EditBinding is a Runnable object
/* The binding between a bunch of edits and a view. Should be instantiated and live on the UI thread */
private static class EditBinding implements ViewTreeObserver.OnGlobalLayoutListener, Runnable {
public EditBinding(View viewRoot, ViewVisitor edit, Handler uiThreadHandler) {
mEdit = edit;
mViewRoot = new WeakReference<View>(viewRoot);
mHandler = uiThreadHandler;
mAlive = true;
mDying = false;
final ViewTreeObserver observer = viewRoot.getViewTreeObserver();
if (observer.isAlive()) {
observer.addOnGlobalLayoutListener(this);
}
run();
}
@Override
public void onGlobalLayout() {
run();
}
@Override
public void run() {
if (!mAlive) {
return;
}
final View viewRoot = mViewRoot.get();
if (null == viewRoot || mDying) {
cleanUp();
return;
}
// ELSE View is alive and we are alive
mEdit.visit(viewRoot);
mHandler.removeCallbacks(this);
mHandler.postDelayed(this, 5000);
}
...
}
Copy the codeVisit (viewRoot) in the core statement run();
public void visit(View rootView) {
mPathfinder.findTargetsInRoot(rootView, mPath, this);
}
Copy the codeCall viewvisitor.accumulate (viewfound).
@Override
public void accumulate(View found) {
...
// We aren't already in the tracking call chain of the view
final TrackingAccessibilityDelegate newDelegate =
new TrackingAccessibilityDelegate(realDelegate);
found.setAccessibilityDelegate(newDelegate);
mWatching.put(found, newDelegate);
}
Copy the codeSet the View AccessibilityDelegate to TrackingAccessibilityDelegate later, when click the View, long_click while waiting for the event, In response to the original after sending a message to the Listener method AccessibilityDelegate, then in AccessibilityDelegate. SendAccessibilityEvent () method under the dot to do operation
/ * * * click event listener * / public static class AddAccessibilityEventVisitor extends EventTriggeringVisitor {private class TrackingAccessibilityDelegate extends View.AccessibilityDelegate { public TrackingAccessibilityDelegate(View.AccessibilityDelegate realDelegate) { mRealDelegate = realDelegate; }... @Override public void sendAccessibilityEvent(View host, int eventType) { if (eventType == mEventType) { fireEvent(host); } if (null! = mRealDelegate) { mRealDelegate.sendAccessibilityEvent(host, eventType); } } private View.AccessibilityDelegate mRealDelegate; }... }Copy the codeThe actual call fireEvent DynamicEventTracker OnEvent
public void OnEvent(View v, EventInfo eventInfo, boolean debounce) { final long moment = System.currentTimeMillis(); final JSONObject properties = new JSONObject(); try { properties.put("$from_vtrack", String.valueOf(eventInfo.mTriggerId)); properties.put("$binding_trigger_id", eventInfo.mTriggerId); properties.put("$binding_path", eventInfo.mPath); properties.put("$binding_depolyed", eventInfo.mIsDeployed); } catch (JSONException e) { Log.e(LOGTAG, "Can't format properties from view due to JSON issue", e); } // For Clicked events, track is called when the event occurs; Edited: for Edited events, Edit triggers multiple Edited events, // so we add a timer, Delayed sending Edited event if (debounce) {final Signature eventSignature = new Signature(v, eventInfo); final UnsentEvent event = new UnsentEvent(eventInfo, properties, moment); // No scheduling mTask without holding a lock on mDebouncedEvents, // so that we don't have a rogue thread spinning away when no events // are coming in. synchronized (mDebouncedEvents) { final boolean needsRestart = mDebouncedEvents.isEmpty(); mDebouncedEvents.put(eventSignature, event); if (needsRestart) { mHandler.postDelayed(mTask, DEBOUNCE_TIME_MILLIS); } } } else { try { SensorsDataAPI.sharedInstance(mContext).track(eventInfo.mEventName, properties); } catch (InvalidDataException e) { Log.w("Unexpected exception", e); }}}Copy the codeThe final implementation of track, and the code to achieve the same goal
SensorsDataAPI.sharedInstance(mContext).track(eventInfo.mEventName, properties);
Copy the codeThe Activity lifecycle calls have version requirements
Required API 14+ (Android 4.0+)