At present, there are a lot of App on the market support iOS9, iOS10 compatibility problems once in the low version of the machine, want to find a phone to the debug is a very difficult thing, and the resolution of the iOS also more and more, whether automated or daily compatibility, all need to have a more comprehensive model to do the compatibility test.

For mobile developers, no matter development, debugging, testing, or before and after the launch of the test machine, it is inseparable from the test machine, and even the product, operation and customer service, also need to use the real machine from time to time for product experience, product operation. Traditionally, developers had to purchase equipment to meet the testing needs of various departments. However, with the development of the business, the defects of this approach become more and more prominent: scarce models are in demand, the number of equipment is insufficient, the cost is high, and the resource utilization is not high.

After investigating many cloud real machine solutions in the industry, we found that people especially on the iOS side of the support is not good, and the charge is relatively high. After several versions of iteration, friends au + launched carrying on U – APM application performance monitoring platform of cloud true machine service, provides the flexibility to test for mobile developers operating interface, debugging, support the ADB WEB remote debugging, sweep the yard, caught, virtual testing functions such as positioning and follow-up check and provide the test report for developers. Especially with iOS support.

​

In terms of screen picture, umeng + cloud real high-end function has about 15 frames of relatively smooth high-definition screen transmission, and is second open, super flow saving.

In terms of user experience, a series of quick Settings such as ipA installation, keyboard input, directly paste content from computer to mobile phone, one-click Web debugging and other rapid use

On the Mac console side, an i5 Mac Mini can plug in 20 phones at a time

The technical architecture behind cloud real machine revealed

Basically similar to openSTF, Mini acts as a provider to manage mobile phones and process messages between them and the cloud server. When connected to the cloud server, the cloud server can support N provider access and easily expand itself, while the cloud server distributes messages. And do some proxy forwarding websocket services, the front end is directly connected to the cloud server.

Cloud real machine core driver

This part based on alibaba powerful technology ability itself completely, not based on WDA, also not based on open source products, now on the market a lot of iOS cloud of real machine based on WDA or other open-source automation test framework to do, but these frameworks designed not do cloud really machine, will introduce a lot of development do not need to function module. We hope that the core driver of cloud real machine can be as lightweight and stable as possible. The whole core driver is mainly divided into screen capture, analog control and auxiliary function interface.

Screen capture

Because of Apple’s limitations, this was the hardest to break through, and we tried a lot of solutions. For example, idevicescreenshot, the frame rate is too low, and the ScreenShorter in iOS system does not accept any parameters of width and height, picture quality and picture format through reverse discovery. This method will be slower in high resolution machines like iPhoneX, with only 1-2 frames.

The popular ios-minicap scheme can obtain the screen content in real time very efficiently, but the biggest disadvantage of this scheme is that one MAC can only provide the real-time screen stream of one iPhone, so the cost is too high. We have also tried to crack the CoreMediaIO in MAC.

The framework iOSScreenCapture protocol (ios-Minicap calls the iPhone screen recording interface provided by the system, provided by AVFoundation and CoreMediaIO). We also tried to isolate the entire Mac by virtualizing it. Multiple ios-Minicaps on the same physical machine failed.

The final solution is to use the private API to take screenshots in XCUITest. The private API can reach up to 15 frames, which basically meets our needs. Then, each frame is encoded into video stream, so as to save traffic. To get a sense of how much traffic a JPG screenshot compares to a video stream:

Taking XS Max as an example, each frame is about 90K on average. Screenshot of data transmission of each frame:

Compared with pictures, video streams with the same image quality visible to the naked eye require very little bandwidth when encoded into video streams with the same model and the same operation and the same picture.

Most importantly, streaming video saves not only traffic at the server exit, but also the CPU footprint of usBMUXD. Usbmuxd is currently single-process and can only use a single CPU core, which is easy to reach the bottleneck. One of the changes we have made to usBMUxD is to make full use of every CPU core and improve the overall MAC hardware utilization. We will write a separate article about this.

Analog control

Compared to the screen fetch, the click event is much simpler and can be triggered directly using XCUITest’s private API, referring to WDA. The message format is the same as openSTF, and the provider receives it and forwards it directly to XCUITest. The focus here is on using long connections to interact with the provider, rather than HTTP, to improve the overall link response speed.

Developer Application Scenarios

Error detection: Repeat the user error to rectify the fault. The test report contains error information

Network packet capture: Monitors and tests mobile phone network behaviors to troubleshoot network and security-related problems

Page debugging: A one-click debugging experience comparable to Chrome Devtools

Virtual positioning: support global high-precision virtual positioning, real-time simulation of device location verification problems

ADB Remote: Connect to ADB and debug Android remotely using the IDE or command line

For more information, please click on the link, click on the free use of UmENG + cloud real machine service

Scan to join the Amu + Tech community