directory

  • preface
  • scenario
  • pix2code
  • imgcook
  • start
  • Customized processing
  • Applicable scenario
  • End

preface

Design draft generation code has always been one of the directions pursued by the front end. Recently, there are a lot of demands for the team project activity type H5, and many App activities need to be maintained, especially during holidays.

The following is a summary of what problems I found, how to solve them and how to improve team productivity.

scenario

Some problems encountered during development are as follows

  • Page restore and design draft gap is very large, restore quality is not ideal.
  • The design draft is obviously very simple, but it takes some time to adjust the layout, compatibility and picture CND.
  • It’s easy to miss UI details.

These problems are common in project development for a variety of reasons. Tuning styles, layout, and graphics takes a lot of time in the development cycle. Based on this phenomenon, some design code generation schemes are investigated.

Originally, it wanted to generate code from the visual (images, etc.), this line, found pix2Code this solution. However, the cost is high and the end result is uncertain, as explained in the pix2code section below.

Imgcook, a product I’ve been following for a while, came up with the idea of giving design code generation a try. Add in some team customization and it becomes a great Arsenal of team productivity. See imgCook’s detailed section below for details.

pix2code

This project is a research project demonstrating the application of deep neural networks to generate code from visual inputs. With the pix2Code prototype requiring just one input image and showing 77% accuracy, the founders say Pix2Code has the potential to end the need for animations to become GUIs.

Portal: github.com/tonybeltram…

It’s basically a half-finished neural network algorithm, and you have to go through a lot of directional training to generate the final algorithm. According to the official website, the basic training needs 1500 times, which requires him to constantly identify the design draft and correct the correctness of identification.

Given that so many websites are already available online, and new ones are created every day, the web could theoretically provide an unlimited amount of training dataCopy the code

Of course, I personally think that 1500 times is not enough, and there will probably be some customization later, such as some framework, code style, design style, etc. We need to solve problems at the team level, not just individuals.

Learning costs

  • python

The official website has always emphasized “experimental”, “toy example” and so on. The final effect is uncertain and the risk is very high. Although I believe there will be a very well-trained algorithm, it is too risky for our team and is not recommended at the current team level. I hope I can worship someone after training. Maybe I’ll sneak off to “have a good time” in the future. I’ll share it again if I get something.

imgcook

Imgcook product is intelligent alibaba front committee team lands of inside and outside of service products, focus on the Sketch, PSD, such as static image visual draft as input, in the form of a key generation through intelligent technology can maintain the front end of the code, containing the view code, data field is bound, component code, part of the business logic code, etc. (You can talk about the advertising fee privately.)

Portal: www.imgcook.com/

I use the line of sechema generated by PSD in design Sketch. Team customization features such as DSL and Plugn are also provided.

So without further ado, what did I do?

start

First we need to download Sketch (I’m using Sketch for demo here)

Then we’ll download Sketch from imgCook and install it into Sketch.

Request the project UI Sketch source file from UI. Then we’ll open it locally.

Let’s open it up and see what it looks like.

In the Sketch interface, the first thing you can see is a sidebar of plugins on the right.

Let’s start with the three largest buttons: Export data, merge, and group.

  • Export data – Generate structured JSON according to the structure of the design draft. Structured JSON is used to generate sechema source code. Click this button to export data after page adjustment.
  • Merge – Used to merge layers. Why merge layers? If you take a closer look at the UI source file, many of the effects in the design draft are made up of multiple images, and the cut images we use in the project are usually the whole image exported from the UI merge, so this feature is still necessary. You can just remember this function, or you can just don’t do it.
  • Group – Identifies layers as a group, represented in code as a common parent box model.

Next, there’s a drop down box for adding protocols, and if you click on it, you can see some protocol rules in there.

  • Add type name prefix – anti contamination.
  • Add class name – Custom class
  • The data model
  • Add field name
  • Add a loop
  • Add logical name
  • Setting the component name
  • Setting the module name
  • Custom protocol
  • Setting an Export Image

These rules are used when doing DSL parsing, when parsing the schema source code, to adjust the parsing structure according to the configured protocol. Custom protocols give developers a lot of latitude.

Next, in the left menu, select an artboard and click Export Data. Then wait for the parsing to complete.

After parsing, the above popup will pop up, so we click Paste.

CTRL + V directly on the page to see the following.

Imgcook parses the Sechema source code into the Sechema tree, and makes changes to the Sechema source code on the right.

Click on the code button in the top navigation to see some examples of official DSLS for this sechema source code parsing function. You can directly select a certain type of DSL for code export function.

The exported code is a set of files, and let’s use the H5 development specification (static) export to see what it looks like.

The generated directory structure and files are as follows.

The effect of operation is as follows: the text recognition becomes the text, and the picture generates the picture, which is uploaded to the CDN of Ali Cloud.

The results are good, and the file generates three styles for you to use.

React, for example, what’s the use of three styles

  • Vw layouts stretch well and can be used directly in static H5
  • 750 px layout, this style, if you build the environment in the project configured with PX auto to REM. The PX layout is especially suitable for you.
  • Rem layout goes without saying, if you don’t have one, use this one.

Customized processing

The above are some processes given by the official fang. If the team wants to use them, we have encountered some problems.

Here it is important to say that there is no problem with the official access process. You just need to follow the steps I mentioned above to access. If your team or individual wants to access, it can be perfectly accessed. The following problems are mainly the problems encountered by our team and the solutions.

1. DSL issues

Although some samples are provided on the official website, they are not in line with our team’s code style and specification. The official has also thought about this issue, giving you the right to create and write your own DSL. Let’s take a look.

Of course, it also gives the corresponding DSL source code, very intimate.

  • React development specification: The React DSL supports generating views and logical code
  • Rax development specification: Rax 0.x DSL that supports generating views and logical code
  • H5 development specification: HTML DSL support for generating views and logical code (no lifecycle support)
  • Vue development specification: A Vue DSL that supports generating views and logical code

The principle is as follows:

There is no problem according to the normal process. After creating the Github repO, you will be automatically initiated to add your project partner privileges (you need to confirm joining in the Github message notification), and then you can directly Commit.

Next, there is the tedious approval process. Yes, it is tedious. You need to pass their review for your own DSL, and then you can see your own parsing module under the code DSL module of your own account. This link is not in line with our expectations, we hope is convenient and fast maintenance, debugging mode,

So I put the process of DSL parsing in our own hands, not official.

The following flow chart shows the difference between the two processes.

After customizing the private DSL, publish it to NPM, compile the file using nodeJS + Commander global command, and generate the compiled file. (Here if I do not understand the words can leave a message, I open source a demo for you to look good)

With the DSL compilation mode problem solved, we found another problem as follows.

2. Picture CDN

All the cut diagrams in the schema source code are uploaded to Ali Cloud CND by default, which is also explained in the official document.

Portal: imgcook.taobao.org/docs?slug=c…

Specific operation will not say, a word summary: “need customization, very troublesome”.

Another way to think about it, now that we have done the steps of DSL compilation with Node, why not write an automatic script to batch replace the CDN outer chain with the inner link? Here’s the idea:

  • Read the address of all ali CDN images in DSL compiled files (FS, VM)
  • Download it (fs)
  • Upload to our own CDN (Needle)
  • Then replace the uploaded link back. (fs)

So let’s look at the end result.

Our react component is based on react, and the resulting component looks like this. The DOM structure is a little different from the “H5 Static rules” above, but the quality of the generated page layout is good.

Here is the effect of my DSL.

Applicable scenario

Not all businesses are suitable for this process.

  • Mobile fine-grained module development scenarios – especially recommended
  • Mobile active page – especially recommended
  • Mobile full page development – recommended
  • ToC application on PC – Recommended
  • PC toB application
  • PC rich interactive applications – Not recommended
  • Game scene – Not recommended

Complex interactions and unfinalized UI designs are not suitable for this process. If it’s plugged in, it could have side effects. This point, I believe, will be solved with the improvement of algorithm and process.

End

About this process: it can’t replace all the front-end work for the time being. I think this tool only improves my productivity by 30%, which is actually quite high and still very good. Later, I will make an accurate statistics of our team’s improved team efficiency.

This process allows the front end to focus more on business logic rather than on layout styles.

Finally: a while ago or very busy, intermediate knowledge system is also sorted out, I have several times and then open out. Stay tuned. DIY a monitoring platform under this article will not rot tail, recently decided to restart. 🙏 long live understanding…