The author | hangzhou xiao liu, after 95, hangzhou lightning and purchasing, main work contents for componentization, dynamic, multiterminal fusion research. Pseudo full stack (iOS, Web front and back end, crawler, anti-crawler), like table tennis, food, movies.

In the era of mobile Internet, user behavior data is very important for every company and enterprise. How important it is, how long the user stays on the page, what buttons they click, what content they view, what phone they are on, what network they are on, what version of the App they are using, etc. A lot of business achievements of some big factories are based on user operation behavior after recommended secondary conversion.

So with the above appeals, then how do technical personnel meet these needs? Which brings us to a technical point — “Bury.

Buried point means

There are three main schemes for code burying point in the industry: manual burying point, visual burying point and traceless burying point. Briefly talk about these several burying point schemes.

  • Manual burying point of code: according to business requirements (operation, product and development), manually call burying point interface where burying point is needed and upload burying point data.

  • Visual buried point: collect nodes through visual configuration tool, automatically analyze configuration and report buried point data at the front end, so as to realize visual “traceless buried point”.

  • Non-trace burial point: through technical means, the completion of user behavior data statistical upload work. In the later stage of data analysis and processing, appropriate data should be screened out for statistical analysis by technical means.

Technology selection

1. Manual embedding of code

In the case of this scheme, if buried points are needed, codes related to buried points need to be written in the engineering code. Because the business code is invaded and polluted, the obvious disadvantages are the high cost of embedding and the violation of the single principle.

Example 1: If you need to know the relevant information (phone model, App version, page path, dwell time, action, etc.) when the user clicks the “buy button”, you need to write the code of buried statistics in the button click event. The obvious drawback is that there is more code buried on top of the previous business logic code. The buried code is scattered, the workload of buried code is very large, the cost of code maintenance is high, and the late reconstruction is a headache.

Example 2: If the App adopts Hybrid architecture, when the first version of the App is released, the key business logic statistics of H5 are bridged by the key logic defined by Native (for example, H5 activates the sharing function of Native, so there is a buried event of sharing). If a scan function is added one day and the buried point bridge is not defined, then when the H5 page changes, the Native buried point code is not updated, and the changed H5 business is not accurately counted.

Advantages: less product and operation workload, the related business scenarios can be restored by referring to the business mapping table, and the data is fine without a lot of processing and processing

Disadvantages: heavy development workload, early-stage needs and operations, good business identification designated by products, so as to facilitate statistical analysis of product and operation data

2. Visualize buried sites

The appearance of visual burying point is to solve the problem that the process of code burying point is complex, the cost is high, and the newly developed page (H5, or the JSON delivered by the server to generate the corresponding page) cannot have the burying point ability in time

The front-end configures and binds the path of critical business modules in a “visual” way to uniquely identify the xpath process to the view in “buried edit mode”.

Each time the user manipulates a control, an xpath string is generated, and the xpath string is positioned uniquely in the front-end system through the interface. Take iOS as an example, App name, controller name, layer view, sequence number of view of the same type: “GoodCell. 21. RetailTableView. GoodsViewController. * baoApp”) to the real business module (” treasure App – mall controller – distribute goods list – a commodities by clicking the “21) the mapping relationship uploaded to the server. What xpath is is explained below.

The App is then manipulated to generate the corresponding xPaths and buried data (the developer uses technology to plug key data from the server into the front-end UI controls). For example, on iOS, the accessibilityIdentifier property of UIView can set the buried point data that we get from the server to upload to the server.

Advantages: relatively accurate amount of data, low cost of late data analysis

Disadvantages: early control of the unique identification, positioning need additional development; The development cost of visual platform is high; Analysis of additional requirements can be difficult

3. No trace burying point

The user’s behavior on the front page is recorded indiscriminately by technical means. Can correctly obtain PV, UV, IP, Action, Time and other information.

Disadvantages: high cost of technical products for developing basic statistical information in the early stage, large amount of data for data analysis in the later stage, and high cost of analysis (large amount of data in traditional relational database is under great pressure)

Advantages: small workload of developers, comprehensive data, no omission, on-demand analysis of products and operations, support statistical analysis of dynamic pages

4. How to choose

Combined with the advantages and disadvantages mentioned above, we choose the technical scheme of traceless burial point + visual burial point combination.

How can I put it? After critical business development goes live, bind to the server with a single click through a visual solution (similar to an interface, think Dreamweaver, where you can drag and drop controls on the interface and simply edit them to generate the corresponding HTML code).

So what is this correspondence? We need to uniquely locate a front-end element, so the way we think about it is whether Native or Web front-end, control or element is a tree hierarchy, DOM Tree or UI tree, so we use technology to locate this element, With Native iOS for example if I click on the product details page of the add to cart button according to the UI hierarchy generates a unique identifier “addCartButton. GoodsViewController. GoodsView. * BaoApp”. But when the user uses the App, he uploads the MD5 of this string of things to the server.

There are two reasons for doing this: server-side databases are not very good at storing this long string of things; It’s not good to see clear text when buried data is hijacked. So MD5 uploads again.

A knife is dry

Data collection

The implementation scheme consists of the following key indicators:

  • Existing code changes less, try not to invade business code to achieve interception system events

  • Full amount collected

  • How do I uniquely identify a control element

Do not invade business code to intercept system events

Take iOS for example. Aspect Oriented Programming comes to mind. To dynamically insert code before and after a function call, in Objective-C we can hook the corresponding function with Method Swizzling using the Runtime feature

To hook all classes easily, we can add a Category to NSObject called NSObject+MethodSwizzling

+ (void)swizzleMethod:(SEL)originalSelector swizzledSelector:(SEL)swizzledSelector{ Class class = [self class]; OriginalMethod = class_getInstanceMethod(class, originalSelector); SwizzledMethod = class_getInstanceMethod(class, swizzledSelector); // Add IMP to SEL BOOL didAddMethod = class_addMethod(class,originalSelector, method_getImplementation(swizzledMethod), method_getTypeEncoding(swizzledMethod)); If (didAddMethod) {// Add successfully: The source SEL does not implement IMP. Replace the IMP of the source SEL with the IMP class_replaceMethod(class,swizzledSelector, method_getImplementation(originalMethod) of the exchange SEL, method_getTypeEncoding(originalMethod)); Method_exchangeImplementations (originalMethod, swizzledMethod);} else {// Implementations implementations IMPs: the source SEL has AN IMP. }}+ (void)swizzleClassMethod:(SEL)originalSelector swizzledSelector:(SEL)swizzledSelector{ Class class = [self class]; OriginalMethod = class_getClassMethod(class, originalSelector); SwizzledMethod = class_getClassMethod(class, swizzledSelector); // Add IMP to SEL BOOL didAddMethod = class_addMethod(class,originalSelector, method_getImplementation(swizzledMethod), method_getTypeEncoding(swizzledMethod)); If (didAddMethod) {// Add successfully: Method_exchangeImplementations IMP (originalMethod, swizzledMethod); Method_exchangeImplementations (originalMethod, swizzledMethod);} else {// Implementations implementations IMPs. }}
Copy the code

Full amount collected

We think of hook AppDelegate delegate delegate methods, UIViewController lifecycle methods, button click events, gesture events, click callbacks for various system controls, application state switches, and so on.

action

The event

App state switch

Add classes to the Appdelegate, hook life cycles

UIViewController life cycle function

Add classes to UIViewController, hook lifecycle

UIButton and so on

UIButton adds categories, hook click events

UICollectionView, UITableView, etc

Add categories and hook click events in the corresponding Cell

Gesture events UITapGestureRecognizer, UIControl, UIResponder

Corresponding System Events

Take the opening time of the statistics page and the opening and closing requirements of the statistics page as an example, we hook UIViewController

static char *viewController_open_time = "viewController_open_time"; static char *viewController_close_time = "viewController_close_time"; // Add dispatch_once to load to prevent manual call. + (void)load{ static dispatch_once_t onceToken; dispatch_once(&onceToken, ^{ @autoreleasepool { [[self class] swizzleMethod:@selector(viewWillAppear:) swizzledSelector:@selector(viewWillAppear:)]; [[self class] swizzleMethod:@selector(viewWillDisappear:) swizzledSelector:@selector(viewWillDisappear:)]; }}); }#pragma mark - add prop- (void)setOpenTime:(NSDate *)openTime{ objc_setAssociatedObject(self,&viewController_open_time,  openTime, OBJC_ASSOCIATION_RETAIN_NONATOMIC); }- (NSDate *)getOpenTime{ return objc_getAssociatedObject(self, &viewController_open_time); }- (void)setCloseTime:(NSDate *)closeTime{ objc_setAssociatedObject(self,&viewController_close_time, closeTime, OBJC_ASSOCIATION_RETAIN_NONATOMIC); }- (NSDate *)getCloseTime{ return objc_getAssociatedObject(self, &viewController_close_time); }- (void)viewWillAppear:(BOOL)animated{ NSString *className = NSStringFromClass([self class]); NSString *refer = [NSString string]; If ([self getPageUrl: the className]) {/ / set the open time [self setOpenTime: [NSDate dateWithTimeIntervalSinceNow: 0]]. If (self. NavigationController) {if (self. NavigationController. ViewControllers. Count > = 2) {/ / get the current vc a vc on the stack UIViewController *referVC = self.navigationController.viewControllers[self.navigationController.viewControllers.count-2]; refer = [self getPageUrl:NSStringFromClass([referVC class])]; } } if (! refer || refer.length == 0) { refer = @"unknown"; } [SDGDataCenter openPage:[self getPageUrl:className] fromPage:refer]; } [self viewWillAppear:animated]; }- (void)viewWillDisappear:(BOOL)animated{ NSString *className = NSStringFromClass([self class]); if ([self getPageUrl:className]) { [self setCloseTime:[NSDate dateWithTimeIntervalSinceNow:0]]; [SDGDataCenter leavePage:[self getPageUrl:className] spendTime:[self p_calculationTimeSpend]]; } [self viewWillDisappear:animated]; }#pragma mark - private method- (NSString *)p_calculationTimeSpend{ if (! [self getOpenTime] || ! [self getCloseTime]) { return @"unknown"; } NSTimeInterval aTimer = [[self getCloseTime] timeIntervalSinceDate:[self getOpenTime]]; int hour = (int)(aTimer/3600); int minute = (int)(aTimer - hour*3600)/60; int second = aTimer - hour*3600 - minute*60; return [NSString stringWithFormat:@"%d",second]; }@end
Copy the code

How do I uniquely identify a control element

Xpath is a unique identifier that defines the manipulable area on the mobile side. Since you want to identify actionable controls in a front-end system as a string, xpath requires two metrics:

  • Uniqueness: No different controls have the same xPaths in the same system

  • Stability: In different versions of the system, the xpath of the same page and the same control must be consistent without changing the page structure.

When we think about rendering systems like Naive and H5 pages, we draw and render them as trees. So we take all the key points between the current View and the root element of the system (UIViewController, UIView, UIView container (UITableView, UICollectionView, etc.), UIButton…) Concatenation uniquely locates the control element.

To locate the element node exactly, see the figure below

Suppose a UIView has three child views in the order of label, Button1, button2, then the depth is 0, 1, 2. Suppose the user does something to remove Label1 from the parent view. UIView now has only two child views: button1 and button2, and the depth changes to 0 and 1.

The view hierarchy

You can see that just because one of the sub-views changes, the depth of the other sub-views changes. Therefore, in the design of the need to pay attention to, when adding/removing a view, to minimize the impact on the depth of the existing view, adjust the calculation of node depth: the current view in the parent view of all the current view in the same type of sub-view index value.

If we look at the example above, the initial depth of label, button1, button2 is 0, 0, 1. After the label is removed, the depths of button1 and button2 are 0 and 1 respectively. It can be seen that in this example, the removal of label does not affect the depth of button1 and button2. This adjusted calculation method enhances the anti-interference of xpath to a certain extent.

In addition, the calculation of the adjusted depth depends on the type of each node, so the names of each node must be placed in the viewPath instead of just for readability.

When identifying the hierarchy of control elements, you need to know the “index values of all views of the same type as the current view in its parent view”. See the figure above. Uniqueness is not guaranteed if it is not of the same type.

There is a problem, for example, we click on the element is a UITableViewCell, although it can locate to similar to the labeled xxApp. GoodsViewController. GoodsTableView. GoodsCell, Cell has many of the same type, So there’s no way to figure out which Cell was clicked just by looking at this string.

There are two solutions

  • Using the system-provided accessibilityIdentifier, the official definition is a string that identifies a user interface element

  • Find the index of the current element in the parent layer of the element of the same type. Traverses the child elements of the parent element of the current element according to the current element. If the same element appears, the current element needs to determine the level of the element in the hierarchy

/** *A string that identifies the user interface element.* */@property(nullable, nonatomic, copy) NSString *accessibilityIdentifier NS_AVAILABLE_IOS(5_0);
Copy the code

1. The server delivers a unique IDENTIFIER

Interface data that has a unique identity for the current element. For example, when requesting data from the UITableView interface, there is a field in the data source that stores dynamic, frequently changing data.

cell.accessibilityIdentifier = [[[SDGGoodsCategoryServices sharedInstance].categories[indexPath.section] children][indexPath.row].spmContent yy_modelToJSONString];
Copy the code

2, judge in the same level, the same type of control elements inside the serial number

All child views of the parent view of the current control element are traversed. If there are controls of the same type as the current control element, it is necessary to determine the position of the current control element in the same type of control element, so it can be uniquely positioned. For example: GoodsCell – 3. GoodsTableView. GoodsViewController. XxApp

(NSString *)xq_identifierKa{// if (self.xq_identifier_ka == nil) {if ([self isKindOfClass:[UIView) class]]) { UIView *view = (id)self; NSString *sameViewTreeNode = [view obtainSameSuperViewSameClassViewTreeIndexPath]; NSMutableString *str = [NSMutableString string]; / / special add and subtract Because with SPM but want to add and subtract need to bring TreeNode nsstrings * className = [nsstrings stringWithUTF8String: object_getClassName (view)]; if (! view.accessibilityIdentifier || [className isEqualToString:@"XQButton"]) { [str appendString:sameViewTreeNode]; [str appendString:@","]; } while (view.nextResponder) { [str appendFormat:@"%@,", NSStringFromClass(view.class)]; if ([view.class isSubclassOfClass:[UIViewController class]]) { break; } view = (id)view.nextResponder; } self.xq_identifier_ka = [self md5String:[NSString stringWithFormat:@"%@",str]]; // self.xq_identifier_ka = [NSString stringWithFormat:@"%@",str]; }// } return self.xq_identifier_ka; } / / UIView classification - (nsstrings *) obtainSameSuperViewSameClassViewTreeIndexPat classStr = {nsstrings * NSStringFromClass ([the self class]); //UITableView special superView (UITableViewContentView) //UICollectionViewCell BOOL shouldUseSuperView = ([classStr isEqualToString:@"UITableViewCellContentView"]) || ([[self.superview class] isKindOfClass:[UITableViewCell class]])|| ([[self.superview class] isKindOfClass:[UICollectionViewCell class]]); if (shouldUseSuperView) { return [self obtainIndexPathByView:self.superview]; }else { return [self obtainIndexPathByView:self]; }}- (NSString *)obtainIndexPathByView:(UIView *)view{ NSInteger viewTreeNodeDepth = NSIntegerMin; NSInteger sameViewTreeNodeDepth = NSIntegerMin; NSString *classStr = NSStringFromClass([view class]); NSMutableArray *sameClassArr = [[NSMutableArray alloc]init]; For (NSInteger index =0; index < view.superview.subviews.count; Index + +) {/ / if same type ([classStr isEqualToString: NSStringFromClass ([view. Superview. Subviews class [index]])]) { [sameClassArr addObject:view.superview.subviews[index]]; } if (view == view.superview.subviews[index]) { viewTreeNodeDepth = index; break; For (NSInteger index =0; index < sameClassArr.count; index ++) { if (view == sameClassArr[index]) { sameViewTreeNodeDepth = index; break; } } return [NSString stringWithFormat:@"%ld",sameViewTreeNodeDepth]; }
Copy the code

Upload of data

Data is collected through the above method, so how to timely and efficient upload to the back end for operation analysis and processing?

During App operation, users will click a lot of data, and real-time uploading will have a low utilization rate for the network. Therefore, a mechanism should be considered to control the uploading of buried data generated by users.

Here’s the idea. An interface is exposed externally for storing the generated data to the data center. The data generated by the user will be stored in the memory of AppMonitor, and a critical value (memoryEventMax = 50) is set. If the stored value reaches the threshold memoryEventMax, the data in the memory will be written to the file system and saved in the form of ZIP. And then upload it to the buried-point system. If the threshold is not reached but there are some App state transitions, it is necessary to save the data in time for persistence. The next time I open the App, I will read whether there is any unuploaded data from the local persistent place. If so, I will upload the log information and delete the local log compression package after success.

The App status switchover strategy is as follows:

  • DidFinishLaunchWithOptions: memory log information written to the hard disk

  • Upload didBecomeActive:

  • WillTerimate: Memory logs are written to hard disks

  • DidEnterBackground: Memory log information is written to disk

The following code is the App buried point data save and upload

// Write App log information to memory. - (void)joinEvent:(NSDictionary *)dictionary{if (dictionary) {if (dictionary) { NSDictionary *tmp = [self createDicWithEvent:dictionary]; if (! s_memoryArray) { s_memoryArray = [NSMutableArray array]; } [s_memoryArray addObject:tmp]; if ([s_memoryArray count] >= s_flushNum) { [self writeEventLogsInFilesCompletion:^{ [self startUploadLogFile]; }]; - (void)traceEvent:(AMStatisticEvent *)event{// synchronized (self) {if (event && event.userInfo) { [self joinEvent:event.userInfo]; }}} / / data from the memory written to the file, persistent storage - (void) writeEventLogsInFilesCompletion: (void) (^) (void) completionBlock {NSArray * TMP = nil; @synchronized (self) { tmp = s_memoryArray; s_memoryArray = nil; } if (tmp) { __weak typeof(self) weakSelf = self; dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{ NSString *jsonFilePath = [weakSelf createTraceJsonFile]; if ([weakSelf writeArr:tmp toFilePath:jsonFilePath]) { NSString *zipedFilePath = [weakSelf zipJsonFile:jsonFilePath]; if (zipedFilePath) { [AppMonotior clearCacheFile:jsonFilePath]; if (completionBlock) { completionBlock(); }}}}); }}// Upload each compressed package file from App buried folder to server. - (void)startUploadLogFile{NSArray *fList = [self listFilesAtPath:[self eventJsonPath]]; if (! fList || [fList count] == 0) { return; } [fList enumerateObjectsUsingBlock:^(id obj, NSUInteger idx, BOOL *stop) { if (! [obj hasSuffix:@".zip"]) { return; } NSString *zipedPath = obj;  unsigned long long fileSize = [[[NSFileManager defaultManager] attributesOfItemAtPath:zipedPath error:nil] fileSize];  if (! fileSize || fileSize < 1) { return; } / / buried call interface to upload some data [self uploadZipFileWithPath: zipedPath completion: ^ (nsstrings * completionResult) {if ([completionResult isEqual:@"OK"]) { [AppMonotior clearCacheFile:zipedPath]; } }]; }]; }
Copy the code

When used is to hook the system event, to call the statistics page to upload data

//UIViewController[SDGDataCenter openPage:[self getPageUrl:className] fromPage:refer]; [SDGDataCenter leavePage:[self getPageUrl:className] spendTime:[self p_calculationTimeSpend]]; // The page disappears
Copy the code

The binding page uniquely identifies the mapping between the binding page and the function description

Summarize the key steps:

  1. Hook system events (UIResponder, UITableView, UICollectionView agent events, UIControl events, UITapGestureRecognizers), Hook applications, controller life cycles. Add extra monitoring code before doing the original logic

  2. To click on the element according to generate the corresponding view tree unique identifier (addCartButton. GoodsView. GoodsViewController) md5 value

  3. After business development is completed, enter the embedded editing mode and bind MD5 to key events of key pages (key modules of operation and product statistics: App level, business module, key page and key operation). Such as addCartButton. GoodsView. GoodsViewController. TbApp corresponding tbApp – mall module – commodity details page – add to cart function.

  4. Store as much data as you need

  5. Design the mechanism to wait for the right moment to upload data

Recommended reading

Mechanism and application of interaction between TapGesture, UIResponder chain, and Target-Action events

Swift 5

Xcode 10.2

Core Graphic Guide: Patterns

From WebView to Flutter: An inside look at Web development in iOS

Just count while you’re watching