Step on the pit, choose practice, the open source project YourView was born

YourView is a desktop App developed in Objective-C language and based on Apple SceneKit technology framework. It supports remote rendering of the View structure of iOS App and 3D display mode, as well as dynamic display of View tree structure. It is convenient for developers to analyze and debug App UI.

In the previous article, YourView Open Source UI Analysis Tool — a rare tool for App Developers! , we have listed some YourView features, as well as the project making address: https://github.com/TalkingData/YourView, welcome to view Star&Fork.

Before developing YourView, we experimented with other UI analysis and debugging tools, but most of them were paid for and the existing open source tools were not functional enough. So we developed and open-source YourView ourselves.

At the beginning of development, we have been investigating the relevant technical implementation. I used to think that there was some kind of hack technology that could dump UI data in iOS memory directly into macOS, and then macOS can render directly, so I have been studying XPC and process communication.

First, iOS and macOS belong to ARM architecture and X86 architecture, and their instruction sets are different. It is incompatible to dump memory directly to devices with different architectures. Second, macOS has a different development framework than iOS. Even if you can dump memory, you still need to do a lot of framework bridging code. Therefore, another more direct way is finally chosen, which serializes UIView into JSON string structure and transmits it through network protocol. After receiving JSON data, the receiver deserializes it into objects in memory and then draws and displays it.

The network protocol can be WebSocket or HTTP. Finally, the HTTP protocol is used. Here’s why:

First, WebSocket needs the support of the server. Currently OC language does not have a good implementation of WebSocket Server.

Second, this development does not need particularly high real-time, so HTTP protocol is a better choice, and OC language already has better webServer implementation — GCDWebServer and CocoaHttpServer. Since CocoaHttpServer was last maintained a few years ago, I chose the more frequently maintained GCDWebServer as the Server side of the communication.

In consideration, we chose to provide the user with an IP entry box when YourView is launched on the desktop. After entering the IP address, users click Connect. If there is a problem with the network, a pop-up box will be displayed. Although it was technically more automatic than manual, the complexity and uncertainty of automatic, as well as the actual performance in practice, led to the choice of manual connection. In the same way that older drivers like to drive a stick shift, we all love the feeling of control (and laziness, in fact).

If you are not familiar with tree operations, you can go to LeetCode and type the question about tree operations. UIView is just a multi-fork tree. Each node has a data field and a pointer field. The data field is its own property, and the pointer field is a relation, which is subViews in UIView. So understanding this makes it easy to write serialization code.

With the help of a stack or queue, traverse the View tree to turn each node into a JSONObject, and then place these objects in an array. Eventually the whole tree was flattened and the tree became a list. The flattened node array is used as the data source to drive the TableView display, and then the corresponding indentation is drawn on the corresponding cell according to the depth of each node itself, which is used to represent the hierarchy of the tree.

This would represent a tree hierarchy in the UI, but it would actually miss two important features of a tree: sibling and parent-child relationships. When the precursor and successor relationships are lost, it becomes inconvenient to shrink and expand nodes.

Post a simplified version of the recursive code:

After this code is executed, the parent and brother relationships of UIView are stored in the JSON structure.

The iOS side needs to save UIView objects in preparation for subsequent operations (such as editing) on the macOS side. But as the interface scrolls, UIView can be released. So we chose NSMapTable as the storage container.

The stored Key is the memory address of the UIView:

-(NSString*)_address{    return [NSString stringWithFormat:@"%p",self]; }Copy the code

The UIView object itself is stored. When the UIView is released due to leaving the screen, the memory address value is null and no wild pointer is generated.

Modify our recursive function slightly by adding a map for recording to the recursive parameters. Note that this map is passed by reference, and each call in the recursion refers to the same map.

To get a ViewController for a UIView, use the nextResponser property until you find the UIViewController and stop. If the average depth of UIView is 10, and there are N views, then the number of iterations is N times 10. This increased the time complexity, so we optimized it. For UIView, just find the nearest level of nextResponder, and if that Responder is a UIViewController then select record in your own data field and pass that UIViewController as a recursive parameter to the next level, Otherwise, treat the parent controller of the recursion as your own ViewController. Transform the recursive function again:

-(NSDictionary*)traversal:(UIViewController*)vc{    UIViewController * vcToNext = vc;    if ([[self nextResponder]isKindOfClass:[UIViewController class]]) {        vcToNext = [self nextResponder];    }    NSMutableArray * subArr = [NSMutableArray array];    for (UIView * v in self.subviews) {        [subArr addObject:[v traversal:vcToNext]];    }    return@ {@"sub":subArr}; }Copy the code

The same goes for UITableViewCell and UICollectionViewCell. When you get an IndexPath, you only go up one level, otherwise you get it directly from the recursive argument, which starts with the default section=-1, row=-1.

Depth is the Depth of the tree in which the View is located. Depth can find the SuperView from the current View straight up until the SuperView is empty. But this kind of processing can cause the same problems as UIViewController fetching. So this is the same strategy that UIViewController uses, every time you recurse, you add the current depth by one, pass it down. Serialized view properties:

Depth and level, ViewController and IndexPath, they all come from a higher level recursion and concatenate their own parameters in a lower level recursion, so we chose to encapsulate these attributes in a StepIn object and abstract out a StepIn method. At the beginning of each recursion, the StepIn object performs a corresponding StepIn operation based on the current state of the view. Specific code can refer to libyourview serializer/UIView + YVTraversel

Since the screenshot needs to be transmitted in JSON, the imgData of the screenshot needs to be converted into a Base64-encoded string. Screenshots are for Layer operations, and must not be taken with sublayer. Therefore, when taking screenshots for each View, it is necessary to change the layer without hidden into hidden state and save it in the array. After the screenshot method is called, it is necessary to restore the hidden property of array layer.

There are three viewControllers on the desktop: Left, Middle, and Right. Left displays the tree structure, Middle displays the 3D structure, and Right displays the View property.

Since the serialization operations above have turned UIView into a tree-like JSONString, it is OK to simply convert the serialized string into an NSDictionary and display it as the data source driving the NSOutlineView.

Use a flat SCNPlane to show a screenshot of UIView. And when you show it, you need to convert the UIView coordinates from UIKit coordinates to SceneKit coordinates. The conversion formula is as follows:

When the mouse moves, it is necessary to highlight the frame of the View that the mouse points to. The frame is a subNode of the current Node. When it is pointed to, it will highlight the previous unhover, which is the current point. In the same way, when the mouse clicks, set the hidden property of the child Node where the ray hit the Node to No. SceneKit provides an API similar to ray detection, which calls the hitTest method directly with point and plane, and returns the 0th element of the result.

YourView currently supports three display modes. Now most open source software is to flatten all the views and arrange them in depth-first order. If there are too many views, all the Z-axes will be too large and the visual effect will be poor when rotating. YourView is an intelligent backtracking algorithm that recursively records the currently occupied levels and frames on a depth-first basis. Each new view comes in and backtracks from the depth-first basis until it finds the first unblocked location.

You can see this visually in Xcode’s scene editor. On the left is the perspective camera, on the right is the orthogonal camera.

• OrthographicCamera: what you see is what you get, the scale of all views does not vary with depth;

• Perspective camera: View near large far small.

For better visual effects, YourView chooses to use an orthogonal camera to present the View.

• Currently YourView only achieves 3D rendering, and the dynamic editing ability of UIView is still relatively weak. We will continue to improve the editing function in the future.

• Add UIViewController gestures and layout elements to the View tree;

• UI beautification and user experience enhancement.

• Apple SceneKit：https://developer.apple.com/scenekit/

• Bonjour：https://developer.apple.com/bonjour/

In the future, we will further improve and optimize YourView to provide a better user experience, and welcome you to use YourView (project address: https://github.com/TalkingData/YourView), also welcome to provide us with valuable Suggestions and comments, let us together to maintain this project