In the construction of UI standardization, standardization teams precipitate a large number of standard components based on standard specifications. Because many components in the process of using standard components be born, a line of research students generally encountered difficult to determine the accurate layout of the UI design draft “for standard components”, which is “standard component” problem, need to research and development according to the experience of speculation, manual search need to use components, time-consuming, affecting the effects of the ground. This paper will explain in detail the realization process of the automatic recognition and labeling ability of UI design draft components mentioned in “Sketch Design Guide” in Bruno scheme, and realize the recognition and labeling ability of standard components at a low cost to solve the problems encountered in front-line research and development.

background

With the rapid development of the business, different business modules are generally divided within the production and research team and assigned to different business teams for collaborative development. As the business iterates, differences in UI interactions between different modules become more apparent, eventually leading to UI consistency issues. To solve this problem, we often promote and implement a set of standard specifications to the team’s internal lines of business (see figure).

However, in this process, business research and development teams (such as RESEARCH and development teams A, B and C) often fail to accurately identify whether the layout in the design draft is A standard UI component or which component it is, which affects the landing effect of components. The ability to automatically identify and annotate a library of UI standard components is therefore very important to front-line developers.

Next, how to identify and automatically annotate components in design drafts will be introduced in detail from three aspects: “How to automatically identify components in design drafts”, “how to automatically annotate components in design drafts” and “how to improve Sketch plug-in to realize automatic identification and annotation of components”.

How to automatically identify components?

When we think of automatic identification components, we might think of “manual labeling” with the naked eye. Using transportation as an analogy, it is no different from “Route 11” walking. It takes time and effort, and it is easy to “go the wrong way”.

As we know, the essence of component identification is to identify the characteristics of UI page elements. Currently, there are some methods in the industry to recognize components through deep learning. However, the implementation threshold of such schemes is high. Without relevant professional foundation and appropriate data sets, the recognition accuracy cannot be guaranteed.

But deep learning solutions are already pointing the way. Now that the core of the component recognition is through the detection of target feature extracting, and comparative identification, so we can in some way, the target UI components on the additional page elements do component characteristic information directly related to the target UI elements on the page, so as to achieve recognition ability, this call this additional features.

The characteristics of the two types of schemes are summarized and compared from the perspectives of identification mode, accuracy, cost and scope of application respectively, as follows:

The core of both schemes is identified in the form of component features.

Deep learning schemes depend on the selection of recognition algorithms and training of data sets, so the accuracy of recognition is determined by the appropriateness of the algorithm and the quality of the data set. At the same time, due to the need to do a lot of preliminary work, such as data collection, it has a high cost. However, the scheme has a wide range of application and can be identified according to different design styles.

However, the additional feature scheme skips the step of feature recognition, and directly associates and annotates UI page elements with components. Because component characteristic information is directly associated with the component, there is almost no misidentification. At the same time, because only a specific set of components can be identified, no additional work is required, making implementation difficult and low cost.

Ii. How to automatically mark component information in the design draft?

Before discussing how to implement “automatic annotation of component information in design draft”, let’s take a look at the general workflow of designers (offline) :

  1. Designers use Sketch to design requirements, and select appropriate components from the UI Kit component library on the designer side to assemble UI pages and fine-tune them. Output demand design draft;
  2. The designer completes the requirement design and exports the design draft file as a draft file with additional UI annotation information through the Sketch Measure annotation plug-in.
  3. Development to get the annotation document for page development.

When designers use Sketch software to design, the design Sketch shows the design effect in the form of layers. Therefore, we took this as the starting point and proposed the “Additional feature scheme” for Sketch files. By attaching transparent layers to the bottom of standard library components and naming the layers as “component code”, standard components are bound with “component features” to realize component identification.

The Sketch Measure mentioned in the workflow is an annotation plug-in of Sketch software, which can mark the size, spacing, distance, text attributes, layers and other information of UI elements in Sketch files. UI designers do not need to write annotations by themselves, which greatly improves the communication efficiency of both parties. Therefore, when we add additional layers at the bottom of the component, we will export the additional layers to the annotation draft for development through Sketch Measure, and at the same time we will export the additional layers to the annotation draft. At this point, annotations have the ability to identify components, as shown below:

Meanwhile, we can see that some basic information of UI, such as width, height and color, has been carried in the annotations exported by the Sketch Measure plug-in. So is there any way to remodel the Sketch Measure plug-in to expand its ability to present richer component information?

Of course, it is possible. First, take a look at the exported annotation after the modification of the plug-in. Later, I will introduce the modification steps in detail.

Demo, we can see through effect, using the modified label draft Sketch plug-in export can be automatically tag design draft of standard components, standard components can be selected to highlight display component code, and selected component in the design the label draft immediately display component “sample code” and “document links,” without time-consuming search, to “see” the development.

Therefore, the modified Sketch Measure plug-in can improve the speed and accuracy of component identification, so as to greatly improve the convenience and accuracy of using standard component library, as well as the utilization rate of standard library, and promote the implementation of standard specifications.

The next steps of the Sketch plugin will give you a detailed introduction to the above capabilities.

3. How to improve Sketch plug-in to realize automatic component identification and annotation?

Take a look at the plugin Sketch Measure, which simply works by parsing the layer information in the. Sketch design file and exporting it to a static HTML file for presentation. So we can change the HTML related files to generate content, to enhance the purpose of the prompt.

3.0 Preparations

First of all, we need to download the Sketch Measure plugin file on GitHub (thanks to UTom, the creator of The Sketch Measure plugin, for creating a great tool that makes development much easier). This section uses Sketch Measure 2.7.7 as an example to open the package contents.

By comparing the internal code snippet of the annotated HTML file (as shown below) :

 

Find it in line 1570 of template.html, which is the template file for the annotations,

 

This is the file we will modify next.

The modification of the plug-in mainly includes four parts:

  1. Create a standard component information library;
  2. Identify standard library components in annotations;
  3. Add component name display;
  4. Add document links and template code.

So let’s do it step by step.

3.1 Creating a standard component information library

Before we start to implement the component annotation capability of the design draft, we need to standardize the standard component information, establish the component information base, and prepare for the subsequent component annotation. You can configure this either locally or remotely dynamically.

The basic component information format is defined as follows:

[
    {
        type: "component",
        group: "tab",
        componetName: "TabBarBadge",
        platform: "flutter",
        code: "Sample Code"
    },
    {
        type: "component",
        group: "tag",
        componetName: "SelectTagWidget",
        platform: "flutter",
        code: "Sample Code"},... ]Copy the code

ComponetName Component name Platform Supported platform code Displays component example code

3.2 Identify standard library components in annotations

Using Chrome debugging, you can see that when a UI element is selected in HTML, the class field is changed to Selected. As shown below:

At this point you can find the corresponding implementation in template.html:

We will key code:

if(self.selectedIndex == index) {
    classNames.push('selected');
}
Copy the code

Modify as follows to achieve the effect of “standard components are selected by default when opening annotations” :

if(self.selectedIndex == index) {
    classNames.push('selected');
} else {
    components.map(function(component){  
        if(layer.name ! =null && component == layer.name){
            classNames.push('selected'); }}); }Copy the code

Note here that components are a standard set of component names derived from component information in 3.1. This is checked by comparing the component name with layer.name, which is the layer name in the “Add-on features” scheme and the component feature information used to identify components used in the component annotations.

3.3 Adding component Name display

Once you’ve implemented the capability, the next step is to display the component name. How?

As shown in the following code, we added the title field to the layerhtml. push method to temporarily hold the layer’s component information (isComponent? ‘title=” + layer.name + ‘” : “”) :

layersHTML.push([
      '<div ' + (isComponent ? 'title=" ' + layer.name + '"' : "") 
      + '" id="layer-' + index + '" class="' + classNames.join(' ') 
      + '" data-index="' + index + '" percentage-width="' 
      + self.percentageSize(layer.rect.width, self.current.width) 
      + '" percentage-height="' + self.percentageSize(layer.rect.height, self.current.height) 
      + '" data-width="' + self.unitSize(layer.rect.width) 
      + '" data-height="' + self.unitSize(layer.rect.height) 
      + '" style="left: ' + x + 'px; top: ' + y + 'px; width: ' + width + 'px; height: ' + height + 'px;" > '.'<i class="tl"></i><i class="tr"></i><i class="bl"></i><i class="br"></i>'.'<b class="et h"></b><b class="er v"></b><b class="eb h"></b><b class="el v"></b>'.'</div>'
].join(' '));
Copy the code

Next, we take the title field out of the attr(title) element shown in the selection, as shown in the following code, which makes the component name display:

.distance.v div[data-height]:before,.selected:after{content:attr(data-height) attr(title); left:auto; right:0; top:50%; transform:translateX(calc(100%+3px)) translateY(-50%); width:max-content}Copy the code

This is the end result, but there are other ways to store component information and display it:

3.4 Show document links and template code

We now have the ability to identify, annotate, and display component names, but we still lack the ability to document and template code. Similarly, in the template.html template file, you can find the entry to add the corresponding platform code, just as you already do, without expanding the description here.

The modified code is as follows:

The final result looks like this:

The exported annotations can automatically mark standard components through the modifications made to the Sketch Measure plug-in in the above four parts, and the component names and sample codes are displayed when the standard components are selected.

At the same time, the above steps are the core transformation ideas, and there is still room for further optimization, such as: display optimization of component annotation information; Component data (document address + code sample) updates can be dynamically updated over the network; Build UI material platform. In addition, the TypeScript re-implementation of Sketch – Meaxure can be adapted in the same way to support higher versions of Sketch.

conclusion

This paper introduces the pain point of the r&d in the implementation of the UI consistency specification, that is, the failure to accurately judge whether the layout of the design draft is the UI component after the specification and which component is the problem, which affects the implementation of the standard specification. In order to enable the requirement design draft to automatically mark and display component information, we modified the Sketch Measure plug-in widely used by designers, and explained the modification steps of the plug-in in detail.

Of course, different technical teams have different solutions to this problem at different stages. The main reasons for not redeveloping a Sketch plug-in tool, but redeveloping existing plug-ins are as follows:

  • Considering the input-output ratio, it is time-consuming to develop a new tool from 0, and the second development of the plug-in has met the existing needs;
  • Plug-in transformation is transparent to design students, does not affect the original use habits, there is a learning cost of new plug-in development;
  • Plug-in transformation cost is low (only 1 person day can be completed), problem solving effect is obvious, suitable for fast landing, verification.

Finally, I hope this article can provide ideas for solving UI consistency and specification implementation problems.

Reference:

  1. Bruno’s official website: Bruindeke.com
  2. Bruno Github github.com/LianjiaTech…
  3. Website of Sketch: www.sketch.com/
  4. The Sketch Measure: github.com/utom/sketch…
  5. The Sketch Meaxure: github.com/qjebbs/sket…