background

For wireless development students, no matter it is to support business data collection, or to build an automated test system, traceless burial point is one of the key technology paths.

One of the things I’ve come across is the need to do an end-to-end UI automated regression link, and the idea is to report these user actions as they interact with the UI, which is recording. When the user needs to play back the user behavior, the user can play back the user behavior based on the reported information. The requirement decomposition of this piece to the end is to do two things

  1. Unique label for all controls of the whole link;
  2. Intercept user interactions and report the preset labels

Basic ability

Looking at the design process above, several questions must come to mind

  1. How to get the full link of all controls
  2. How to determine the unique label of the control
  3. How to intercept user interactions

Page control traversal

GetChildAt () : getChildAt() : getChildAt() : getChildAt() : getChildAt() : getChildAt() : getChildAt() : getChildAt() : getChildAt() : getChildAt();

This diagram shows the relationship between Activity, PhoneWindow, and DecorView. You can see that the top-level View of an Activity is a DecorView, which can be obtained by getWindow().getDecorView ().

In order to change the page, for example, click add a new TextView, change we can real-time understanding of HTML controls, we can add setOnHierarchyChangeListener listening in to ViewGroup, This listener notifies the listener when the ViewGroup changes.

There are also special page changes, such as opening Dialog or PopupWindow. Instead of adding a View to the existing DecorView, they create a new DecorView on the PhoneWindow to override the existing DecorView. In this case, we need to fetch the DecorView generated by Dialog and Popupwindow and repeat the process to iterate over the controls above.

Control unique label

Once you have all the controls, the next step is to add a unique identity to the controls. When the concept of unique identity for controls came up, id was definitely the first thing that came to mind.

1. ViewId generates a unique identifier

ViewId is the ID set when the charge file is writing the XML, obtained from view.getResources().getResourceEntryName(view.getid (). This ID is guaranteed to be unique in the same layout file.

  • Advantages:

The setup is simple, and the generated buried point information is highly readable, which is convenient for subsequent manual reading to locate problems

  • Disadvantages:

  • You need to manually set the id of each control and get null

  • Ids are only guaranteed to be unique in XML, not unique throughout the page

Logos are not unique and have preconditions for business use, which is obviously not in line with our requirements. At this point we come up with the idea that each control must have a unique page layout hierarchy.

2. ViewPath generates a unique IDENTIFIER

ViewPath is defined as a coordinate for each view based on the ViewTree, and the construction path of a view is extracted.

For example, the Layout Inspector of Android Studio shows the following page-level information for all buttons

The layout structure can be described as Viewrotimpl /DecorView[0]/LinearLayout[0]/FrameLayout[0]/ ActionBarOverLayOut [0]/ContentFrameLayout[0]/Fra meLayout[0]/FloorContainerView[0]/CoordinatorLayout[0]/LinearLayout[1]/ConstraintLayout[1]/AppCompatCheckBox[0]

  • FrameLayout, ' 'FloorContainerView,' 'AppCompatCheckBoxThese define the types of controls
  • The slash is used to indicate the hierarchy,DecorViewDeep of is 1,FrameLayoutWhere deep is 4,AppCompatCheckBoxDeep to 12
  • [1] The parentheses represent the position of the View in the parent hierarchy. In order to reduce the ViewPath change caused by the insertion of new views, the star ring has been optimized to the number of views of the same type in the same hierarchy.

Advantages:

All controls that are in the ViewTree can be defined through the ViewPath

Disadvantages:

  • The readability of buried information is not high, and manual positioning is troublesome
  • The ViewPath can only be guaranteed to be unique at the moment of user behavior. The ViewPath may change when the page control changes
  • So when you have reusable views, the viewPath will be repeated, so for example, RecycleView has 20 items, and you can only display 5 items per page, and you’ll see that the 20th item is not at 20, it’s at your current location on the page.

The first two problems are not a problem in our current scenario. When doing page playback, we only need to keep the ViewPath of each control during the current recording or playback unchanged to ensure reliable playback.

The third problem is that we need to do special processing for the reuse of this control, such as the following View, we will give him a label position in the back, indicating that he is a reuse View, and is the reuse of the number

Its layout structure can be described as ViewRootImpl/DecorView[0]/LinearLayout[0]/FrameLayout[0]/ActionBarOverlayLayout[0]/ContentFrameLayout[0]/LinearLayout[0] /RecyclerView[0]/LinearLayout[position5]/AppCompatTextView[0]

Android pile insertion technology

The interception of user interaction is to add buried logic on the invocation method of user behavior operation. This requires the introduction of Android peg technology. There are roughly two ways of thinking about Android staking

1. hook

Hook simply means to add buried logic to the target variable of the target object through dynamic or static proxy at runtime.

For example, the VIew click event to add log requirements, according to the source we can know, he is actually in the process of setListener, add an mListener object to the VIew, as long as the subsequent onClick trigger, will call the listener onClick method. So we simply delegate the listener and add a bit of buried logic when the Listener’s onClick is triggered

Public class HookSetOnClickListenerHelper {public static void hook (Context Context, final View v) {/ / try {/ / step 1: View class getListenerInfo(), get the mListenerInfo object of v, This object is the holder of a click event Method Method = the class. The getDeclaredMethod (" getListenerInfo "); // Since the getListenerInfo() method is not public, we add this code to ensure access to method.setaccessible (true); Object mListenerInfo = method.invoke(v); // Get the current click event object Class<? > listenerInfoClz = Class.forName("android.view.View$ListenerInfo"); Field field = listenerInfoClz.getDeclaredField("mOnClickListener"); final View.OnClickListener onClickListenerInstance = (View.OnClickListener) field.get(mListenerInfo); // Step 2: Create a new proxy class that executes our logic while processing the proxied method onClick. Set (mListenerInfo, new ProxyOnClickListener(onClickListenerInstance))); } catch (Exception e) { e.printStackTrace(); }} // Can be dynamic or static proxy, Static class ProxyOnClickListener implements view.onClickListener {view.onClickListener oriLis; public ProxyOnClickListener(View.OnClickListener oriLis) { this.oriLis = oriLis; } @override public void onClick(View v) {log. d("HookSetOnClickListener", "HookSetOnClickListener"); if (oriLis ! = null) { oriLis.onClick(v); }}}}Copy the code

  • Advantages:

As a function module, the access cost is low and there is no need to introduce tripartite library

  • Disadvantages:

  • Hook requires reflection and is inefficient

  • If the listener is set externally after the hook is complete, the hookListener will be overwritten and the logic will not be executed

  • Hooks can only handle such variables and add method logic to them, not directly to methods such as the Activity’s onCreate

2. Aop

AOP represents section-oriented programming, and its main forms in Android are Aspectj and ASM.

Gradle_plugin_android_aspectjx plugin is the main plugin for using Aspectj. For details, see Android Aspectj. Its principle is to compile the logic into the target method.

For example, add the need to log a VIew’s click-listening event. The essence of this has been described above. The next step is to find a way through AspectJ to add this log to the back of it.

@After("execution(* android.view.View.OnClickListener+.onClick(..) )") public void afterOnClick(JoinPoint JoinPoint) {log. e(TAG, "click event was hooked "); }Copy the code

  • Advantages:

  • Very targeted, very little code

  • You can add logic to method

  • Disadvantages:

  • You need to import aspectJ’s libraries, and some compilation problems may occur at compile time

  • Because the character code is inserted at compile time, the compile time will be longer

Implementation scheme

Aspectj scheme implementation

1. Page control marking

Use Aspectj to intercept the Activity’s onCreate, Dialog’s show, and PopupWindow’s showAsDropDown, get the corresponding DecorView through the window, and iterate through all the child controls and mark them

@Aspect public class TrackerAspect { @After("execution(* android.app.Activity+.onCreate(..) )") public void onActivityResume(JoinPoint joinPoint) { Tracker.startViewTracker((Activity) joinPoint.getThis()); } @After("call(* android.app.Dialog+.show(..) )") public void onDialogShow(JoinPoint joinPoint) { Dialog dialog = (Dialog) joinPoint.getTarget(); Tracker.startViewTracker(dialog.getWindow().getDecorView()); } @After("call(* android.widget.PopupWindow+.showAsDropDown(..) )") public void onPopupWindowShow(JoinPoint joinPoint) { PopupWindow popupWindow = (PopupWindow) joinPoint.getTarget(); try { Field field = SuperClassReflectionUtils.getDeclaredField(popupWindow, "mDecorView"); field.setAccessible(true); FrameLayout popupDecorView = (FrameLayout) field.get(popupWindow); Tracker.startViewTracker(popupDecorView); } catch (IllegalAccessException e) { e.printStackTrace(); }}}Copy the code

RecycleView can listen to its onBindViewHolder and then build its ViewPath according to the above theory by RecycleView and the following itemView

@After("execution(* androidx.recyclerview.widget.RecyclerView.Adapter.onBindViewHolder(..) )") public void afterRecycleAdapterOnBind(JoinPoint joinPoint) { Object[] args = joinPoint.getArgs(); RecyclerView.ViewHolder viewHolder = (RecyclerView.ViewHolder) args[0]; int position = (int) args[1]; Try {/ / for listView object Field recyclerViewField = SuperClassReflectionUtils. GetDeclaredField (viewHolder, "mOwnerRecyclerView"); recyclerViewField.setAccessible(true); RecyclerView recyclerView = (RecyclerView) recyclerViewField.get(viewHolder); // Obtain the listView object Tracker. SetViewTracker (Viewholder. itemView, viewPath. getPath(Viewholder. itemView, recyclerView, position)); } catch (Exception e) { e.printStackTrace(); }}Copy the code

2. Interactive behavior interception

Interception of interactive behavior is actually the same idea as above, find the method of listening to trigger, insert the code of buried point.

/ * * * click OnClick event monitoring * / @ After (" execution (* android. View. The view. An OnClickListener +. The OnClick (..) ") public void afterOnClickMethodCall(JoinPoint JoinPoint) {} /** * Check OnLongClick event */ @after (" Execution (*) android.view.View.OnLongClickListener+.onLongClick(..) ) "public void afterOnLongClickMethodCall (JoinPoint JoinPoint) {} / * * * * / @ After monitoring OnCheckChange click events (" execution (* android.widget.CompoundButton.OnCheckedChangeListener+.onCheckedChanged(..) ) "public void afterOnCheckChangeMethodCall (JoinPoint JoinPoint) {} / * * * * / @ After monitoring text input events (" execution (* android.text.TextWatcher.onTextChanged(..) )") public void afterOnTextChangedMethodCall(JoinPoint joinPoint) { } int scrollerState = 0; int scrollerX = 0; int scrollerY = 0; /** * Monitor RecyclerView sliding events ** @param joinPoint */ @pointcut @after ("call(*) android.support.v7.widget.RecyclerView.onScrollStateChanged(..) ) "public void afterRecycleViewOnScrollStateChangedMethodCall1 (JoinPoint JoinPoint) {/ / method several parameters Object [] args = joinPoint.getArgs(); scrollerState = (int) args[0]; switch (scrollerState) { case RecyclerView.SCROLL_STATE_DRAGGING: break; case RecyclerView.SCROLL_STATE_IDLE: / / when the page scrolling stop data reporting, and zero parameter 1 Log. D (TAG, "afterRecycleViewOnScrollStateChangedMethodCall1:" + scrollerX); Log.d(TAG, "afterRecycleViewOnScrollStateChangedMethodCall1: " + scrollerY); Log.d(TAG, "afterRecycleViewOnScrollStateChangedMethodCall getThis->" + ((RecyclerView) joinPoint.getThis()).getContentDescription()); // scrollerX = 0; scrollerY = 0; break; Default:}} / monitor RecycleView * * * * / the @pointcut @ After sliding events (" call (* android. Support. V7. Widget. RecyclerView. OnScrolled (..) ) "public void afterRecycleViewOnScrolledMethodCall1 (JoinPoint JoinPoint) {/ / method several parameters Object [] args = joinPoint.getArgs(); // When the scroll state is 1 or 2, the move distance is stacked. if (scrollerState == RecyclerView.SCROLL_STATE_DRAGGING || scrollerState == RecyclerView.SCROLL_STATE_SETTLING) { scrollerX += (int) args[0]; scrollerY += (int) args[1]; }}Copy the code

Implementation of Native scheme

The Aspectj scenario above is remarkably simple, but requires additional import of tripartite libraries, which introduces some possible compatibility issues. Hook solution may be covered later. Does Android provide any native solution to this problem?

1. Page control marking

The page initialization display cannot be intercepted by Aspectj, but must be manually checked by tracker.startViewTracker (getWindow().getDecorView()) during the Activity onCreate.

2. Interactive behavior interception

1) Click input event interception

When the View click event is executed, sendAccessibilityEvent will be executed. As long as the View is set to mAccessibilityDelegate, Its click event goes to the sendAccessibilityEvent callback.

public boolean performClick() { /... // Send the click event sendAccessibilityEvent(Accessibilityevent.type_view_clicked); notifyEnterOrExitForAutoFillIfNeeded(true); return result; } public void sendAccessibilityEvent(int eventType) { if (mAccessibilityDelegate ! = null) { mAccessibilityDelegate.sendAccessibilityEvent(this, eventType); } else { sendAccessibilityEventInternal(eventType); }}Copy the code

By overriding this listener, we can listen for the View’s click and input events. This listener may also be replaced, but it is much less likely to be replaced than onClickListener.

HookOnClickListener.hook(view); public class HookOnClickListener { private static final String TAG = "HookOnClickListener"; public static void hook(View view) { view.setAccessibilityDelegate(new View.AccessibilityDelegate() { @Override public void sendAccessibilityEvent(View host, int eventType) { super.sendAccessibilityEvent(host, eventType); TYPE_VIEW_CLICKED: log. d(TAG, "sendAccessibilityEvent: " + host.getContentDescription()); break; // case accessibilityevent. TYPE_VIEW_TEXT_SELECTION_CHANGED: // TrackEvent.postUTSendKey(host.getContentDescription().toString(), ((TextView) host).getText().toString()); // Log.d(TAG, "postUTSendKey: " + host.getContentDescription() + "|value:" + ((TextView) host).getText().toString()); // break; // other default: break; }}}); }}Copy the code

2) List sliding event interception

RecyclerView can add multiple sliding listeners via addOnScrollListener, so we can customize a sliding listener without worrying about being overwritten by other listeners

public class HookAddOnScrollListenerHelper { private static final String TAG = "HookAddOnScrollListener"; public static void hook(final RecyclerView recyclerView) {// List<RecyclerView.OnScrollListener> mScrollListeners = (List<RecyclerView.OnScrollListener>) SuperClassReflectionUtils.getFieldValue(recyclerView, "mScrollListeners"); if (mScrollListeners ! = null && ! mScrollListeners.isEmpty()) { for (RecyclerView.OnScrollListener listener : Listeners instanceof HookOnScrollListener {return; } } } recyclerView.addOnScrollListener(new HookOnScrollListener()); } static class HookOnScrollListener extends RecyclerView.OnScrollListener { private static final String TAG = "OnScrollListener"; int scrollerState = 0; int scrollerX = 0; int scrollerY = 0; @Override public void onScrollStateChanged(@NonNull RecyclerView recyclerView, int newState) { super.onScrollStateChanged(recyclerView, newState); scrollerState = newState; switch (scrollerState) { case RecyclerView.SCROLL_STATE_DRAGGING: break; case RecyclerView.SCROLL_STATE_IDLE: / / when the page scrolling stop data reporting, and zero parameter 1 Log. D (TAG, "afterRecycleViewOnScrollStateChangedMethodCall1:" + scrollerX); Log.d(TAG, "afterRecycleViewOnScrollStateChangedMethodCall1: " + scrollerY); Log.d(TAG, "afterRecycleViewOnScrollStateChangedMethodCall getThis->" + recyclerView.getContentDescription()); // scrollerX = 0; scrollerY = 0; break; default: } } @Override public void onScrolled(@NonNull RecyclerView recyclerView, int dx, int dy) { super.onScrolled(recyclerView, dx, dy); if (scrollerState == RecyclerView.SCROLL_STATE_DRAGGING || scrollerState == RecyclerView.SCROLL_STATE_SETTLING) { scrollerX += dx; scrollerY += dy; }}}}Copy the code

The final scheme can be selected according to the actual situation of the project, and both schemes have their own advantages.

Code warehouse

Github.com/PettyWing/a…