In the era of mobile Internet, how to quickly catch users’ eyes, improve visual comfort, and emphasize brand recognition have become the core demands of UI design. In the face of increasingly refined, complex and diversified UI design, how to effectively control the degree of reduction, so that the design draft is delivered to the user’s mobile phone without loss, has become the focus of the design and development team.
Let the machine learn to design language, hands free with machines, iQIYI mobile development team adhering to the innovation effect of concept, grinding a set of UI design, UI annotation, UI development, UI and debugging, UI acceptance of the whole process of product, make the whole process of standardization, automation, improve the overall UI reduction degree and the UI design and development efficiency.
In view of the mature solutions in UI design, UI annotation, UI development and other dimensions in the industry, this paper will focus on the dimension of UI automation acceptance, and introduce the exploration and practice of iQiyi mobile development team in this direction.
When the construction of our UI acceptance platform was completed, the OVERALL UI acceptance went on the road of standardization and automation. For designers, the EFFICIENCY of UI acceptance is increased by 50%, and UI measurement and Bug labeling are eliminated. Designers can quickly and accurately complete pixel-level UI acceptance. In addition, reduce the cost of collaboration between designers and engineers, so that designers and engineers no longer repeated acceptance modification; Finally, the UI quality of mobile App is improved.
Next, we will share with you the experience of iQiyi mobile terminal in UI acceptance automation and standardization in the order of page measurement, page acceptance and acceptance report for your reference. If you have ideas and suggestions, you can leave a message in the comment area.
Page measurement
How to accurately find the control and identify the size of the control, width and height, background color and other properties? With this problem in mind, iQiyi mobile client team went deep into the design team to understand the existing workflow and core concerns, and combined with the existing technology accumulation to explore the best solution to the problem.
For the design team, control measurements need to be 100% accurate, and if they are less than 100% accurate, another form of repeated acceptance and modification can result.
Therefore, we comprehensively evaluated two control measurement schemes:
Sketch plug-in + Terminal acceptance SDK:
● Development page: Embed acceptance SDK in App, call control API to measure.
● Design page: Capture data for each control using the Sketch plug-in.
AI image control recognition:
Use image recognition algorithms to identify and measure controls from page screenshots.
| SketchThe plug-in +The acceptance of the SDK | AIImage control recognition | |
| advantage | Control 100% identification, high accuracy | No multi-terminal adaptation is required |
| disadvantage | Each end must be independently adapted | Under the continuous componentized model training, there is still about 30% control recognition error |
In conclusion, although the RECOGNITION scheme of AI image control does not require multiple independent adaptation, it fails to meet the strict interface acceptance criteria in terms of control accurate identification and control property identification for the time being. Therefore, we finally adopted the Sketch plug-in + acceptance SDK scheme in the production environment.
Acceptance of a page
The page acceptance section is responsible for the page acceptance. During the page acceptance, we have two modes: personalized acceptance mode and automatic acceptance mode.
Personalized acceptance mode:
Click the matching control in the development page and design page, compare it in the right column, check the wrong place, and upload and save.
Automatic acceptance mode:
Automatic acceptance mode is the user box selected development page and design page matching area, through a series of algorithms to automatically accept all UI controls in the matching area.
There are two problems with page acceptance:
-
How to establish a one-to-one relationship between the design page and the controls in the development page. As shown in the figure above, which control in Figure 2 (design page) should be compared to in Figure 1 (development page)?
-
How to determine spacing relationships between controls. As shown in the figure above, when accepting spacing for control B, should spacing B or spacing A be used?
If these two problems cannot be solved, then our acceptance automation cannot proceed. Therefore, we summarized the rules of mobile terminal UI interface, combined with the data characteristics of the development page and design page, and divided the page acceptance into three stages: pre-processing, relationship establishment and control comparison.
Clean up, unify, and merge data from design and development pages, and smooth out a series of differences caused by different screens, fonts, view structures, and so on. For example, uniform screen size, redundant view cleaning, and so on.
Through the specially designed control matching algorithm and spacing selection algorithm, the establishment of the development page and design page of the control of the one-to-one relationship and spacing relationship.
Core principle of control matching algorithm: integrated control position, type, adjacent control and other information, calculate the matching degree between the design page and the development page of the control, through the matching degree to determine the one-to-one relationship.
The core principle of the spacing selection algorithm is as follows: Sort according to the minimum spacing principle, calculate the spacing between the current control and all other controls based on the sorting result, and select the spacing with the smallest absolute value as the spacing of the current control.
The above are the core principles of the two algorithms. In the actual application process, we also made some adjustments according to the actual situation of the business, which will not be listed in detail here.
The style and spacing of each control are compared according to the one-to-one relationship and spacing data obtained in the relationship establishment stage.
Comparison of the two modes:
| Personalized acceptance mode | Automatic acceptance mode | |
| Acceptance of efficiency | general | good |
| Fine degree of | good | general |
| The accuracy of | good | general |
| Applicable scenario | After the requirements are tested, the designers use them to guard the final level | Before requirement testing, the engineer conducts UI self-test |
The acceptance report
After the completion of acceptance, we will automatically generate a multi-dimensional traceable acceptance report for engineers to view according to the acceptance results.
The acceptance report contains the following information:
● Problem priority: Determine the order of repair based on the priority and repair according to the prompts
● Problem description: UI acceptance of the actual style and expected style comparison.
● Repair state: engineers can view the repair state, and the control has been fixed to the repair state, easy to cooperate with many people.
Summary and Planning
After the launch of the platform, UI acceptance has achieved overall standardization and automation. With the support of the platform, THE effectiveness of UI acceptance has been increased by 50%.
In the future, we plan to combine the working methods of designers and engineers to continue to improve the entire UI acceptance platform and improve efficiency. On the other hand, it is not limited to THE UI acceptance link. Starting from the whole link from UI design to online, we should dig and improve the process, so as to achieve more efficient design and development and higher quality delivery, so as to improve the quality and efficiency of the whole link, and let technology change our way of working.
Maybe you want to see it again
Common AI elements identify best practices in UI automation testing
Iqiyi self-developed UI engine rendering technology analysis – Lyra