F(X) Team of Ali Tao Department – Zhen Zi
sequence
After more than three years, it was initially misunderstood as “another wheel for promotion”, “packaging our PPT products with AI”… , go to imgcook.com and post online. From tao Technology Double eleven zero research and development campaign, to empower more than ten BU. From the 10th anniversary of QCon sharing “a front-end intelligent practice” after all kinds of questions, to 23,000 + users and jingdong, Tencent, Bytedance…… And other companies imitate, ping an bank and other large enterprises request privatization deployment…… .
Along the way, THERE have been too many sleepless nights and bitter but still sunny and happy I want to make a brief summary of the past, and make a plan and outlook for the direction of long-term development in the future, hoping to make the front end intelligent what is? Speak clearly.
The origin of front-end intelligence
In 2017, I began to propose the concept of “front-end intelligence”. At that time, I was in charge of UC International Browser. Starting from the front-end scenarios, I made some attempts: user experience measurement, UI automatic testing, and code generation based on design draft. The term front-end intelligence wasn’t officially established until 2019, when the D2 Front-end forum released “www.imgcook.com.”
We use intelligent means to improve the front-end research and development efficiency as the first step, borrow imgCook’s design draft generated code as a breakthrough point, at one fell swoop completely solve the front-end cut diagram writing UI code part of the front end engineers r & D efficiency problem.
If design code generation is fundamental, then accuracy, availability, and maintainability of generated code are key to success from the point of view of the users we serve: the “front end engineers”. By machine vision recognition and deep understanding of the neural network, the code rule engine power of expression, the front end of the UI code using the machine to “see” instead of the front-end engineers design draft, “slice map”, “to generate the UI code”, thus, with the recognition, understanding and expression ability of ascension brings about the accuracy, availability, and maintainability.
Next, expand and enrich usage scenarios to improve delivery efficiency. We apply intelligent capabilities to intelligent UI to solve the massive module and component development work brought by personalized UI. Glue layer code generation is applied to the server to write specific business logic on top of business capabilities instead of the server and Node FaaS. Application to automated UI testing can significantly reduce testing time and labor costs.
Finally, forward intelligence requires business value. We apply intelligence to marketing activities and channel shopping guide business, with the help of intelligent UI recommendation algorithm to automatically generate user acceptance scenarios, and improve the core indicators of the business to varying degrees. It was applied to BizCook, a requirement generation code platform, and built a multi-role collaborative business r&d platform driven by business indicators, making collaboration online and coding automatic, and significantly improving the efficiency of business trial and error and iteration. The application to material compliance audit solves the compliance audit of merchant material of tmall U’s first trial and commercial advertising material of TRW’s Traffic treasure, and settles into a general configurable material compliance audit platform of the group.
This series of processes is the ultimate productivity road of R&D efficiency, delivery efficiency, business trial and error and iteration efficiency, which is built by more than 30 front end and supported by 52 businesses, using spare time and overtime.
The key link of intelligent front end
The 2-3-1-5:
There are two types of service scenarios: reach users and undertake users
Three areas: R&D, delivery, business
1 core technology: AI-driven code generation capability
Five key links: RunCook (put business scenarios into the second and third ring media to reach users), BizCook (trial-and-error, iterative, agile and innovative user undertaking capacity), UICook (refine user undertaking capacity with personalized UI), imgCook (design and production integrated UI Automatic code generation), LogicCook (automatic logical code generation based on PaaS business capability registration discovery), ReviewCook (automated Codereview guarantees the quality of generated code).
The words “business efficiency, delivery efficiency, r&d efficiency” seem to be easy to understand, but under the current size and scene of Alibaba, there are a lot of problems and challenges behind these words.
Taking the intelligent scene and intelligent design of UICook as an example, a page may contain 10+ modules, a module may contain about 5 components, and a module will design 20+ styles. A module will have 100 times of coding, debugging, testing and release. One page has at least 10 x 100 = 1000 times of page building, configuration, data binding, release scheduling work to do, and one promotion at least 1000+ pages, and the above work will do 1000 x 1000 = 100W times…… For Alibaba double 11, Double 12, New Year shopping festival…… It is difficult to deal with a series of marketing activity scenarios only relying on tools to optimize some local areas. Therefore, in the face of the conventional magnification of thousands of times, many problems become the problem of a new field: intelligent to create a new front-end productivity. This is an opportunity for Ali’s front end and for us as front end engineers.
The top level design and core task of front-end intelligence
The previous diagram split up the process of intelligent UI generation into application, and eventually we will have to aggregate it. Crumpled and digested, the result is called “intelligent top-level design for the front end”, which has been laid out and steadily advanced for three years.
We are committed to building intelligent UI assets that can support ali’s entire business system, which is the core of the whole design. Behind the processing and production of UI, it is necessary to use intelligent generation and intelligent UI recommendation algorithm, intelligent design and production technology upgrade, BizCook and UICook as the iterative evolution of technical products directly serving the business as the support. At the same time, we put the smart UI generation, UI recommendation algorithm, the intelligent design system into general ability and the basic technology to serve more business, make intelligent UI assets in the applications of various business scenarios at the same time, by using index data as the carrier of reverse transmission and form a closed loop, drive intelligent front-end system of mature and perfect.
There are two core tasks of front-end intelligence:
I. Continue to promote the construction of UI intelligent assets, and build a UI intelligent asset system of “reliable quality, safety and stability, economic production and convenient consumption”.
Among them, production economy and convenient consumption are more concerned about the problem. With the aid of intelligent generation of D2C capability and S2C capability, the ability to build machine production UI, interactive logic and business logic, reduce the production cost of UI, is called “production economy”. Today, ali’s different businesses develop at different pace, at different stages and encounter different problems, so convenience is relative. Just like a gear, you have to bite better to move forward together. For X business explored by the market, the key of “convenient consumption” should be put on out-of-the-box, and access and use costs should be reduced to the greatest extent, and access and use effects should be improved. For the conventional business of market expansion, the key of “convenient consumption” should be on the customization ability and openness, so as to support the business to meet the complex needs of users and business, and to provide technical personnel with the opportunity to customize their own business platform and technology engineering system.
Smart UI Assets
Intelligent design generation
Intelligent UI design template
2. Precipitate intelligent technology and product capabilities, enable intelligent business to intelligent business, release technology and organizational dividends, and drive business growth.
Customers and business parties often ask a very philosophical question about what “business intelligence” and “smart business” mean. Actions speak louder than words for this problem, and it is necessary to explain and get a sense of motion based on the scene. In short, the purpose of this task is to integrate these products and technologies into the business system through a series of product matrix and technology matrix, rather than simply defining some technical products from a technical perspective.
Intelligent UI business customization
Intelligent UI application
Methodology + Tools + organization – Building basic ideas and landing formulas
Methodology: A business development model driven by business metrics for product definition, technology generation, and operational use
Just like doing development, there is a set of basic ideas behind any set of framework, and a complete theoretical system behind the ideas. Without the guidance of the theoretical system, we cannot abstract and cut in the future complex scenes or in some changeable scenes. Front-end intelligent technology & business system must have a set of theories to support it: business index drive (BizCook), product definition (BizCook & The power of the Fish), technology generation (P2C, D2C, S2C), operation (small two workbench) four parts.
First, what is BizCook? Ali have taobao today, Tmall and getting a lot of business, business metrics definition is also very diverse, even if the indicator is unified, but if everyone is not the same business scenario, will be different understanding on the definition of active users, such as Tmall U active requires the user to get samples first, and visiting business taobao active requires the user to browse content and attention, etc. Therefore, business metrics need to be defined in their respective business scenarios. Drivers, on the other hand, can be unified with focused (definition), transparent (delivery), iterative (execution) patterns. Only when the target scenario is unified and the drive is unified can the effective business r&d mode be truly upgraded, the chemical reaction fission or the derivation of more business possibilities be achieved.
Second, product definition (BizCook & Whale power). Product definition is aimed at the core product element: UI intelligent asset, which abstracts everyone’s definition and understanding of this element into a common divisor in each scene: it can make each scene both standard and flexible, and provide personalized experience consistency. UI intelligent asset is a very important product infrastructure of digital business. In terms of consumption, a product may have more than twenty scene, a scene there will be dozens of living space and time dimension, and a dimension will correspond to the hundreds of layers of the user, only with the aid of intelligent UI assets, to define the product is good, let users in their own interest preferences and habits, to experience the value of digital business. As a result, technologists will be able to work with designers to continuously merge algorithms and data assets to restore our understanding of the consumer. Of course, this is carried out under the premise of security and privacy compliance, and in the field of front-end intelligent end intelligence, it is standardized to the unified product design and definition field, and the vector data of abstract user characteristics and scene characteristics is transmitted under the protection of user privacy.
The third is technology generation (P2C, D2C, S2C). Algorithms and data work together to build UI intelligence assets for services. Technology that does not provide services has only cost and no sense of value. The supply of UI intelligent assets is driven by services, and then the technical team of each business is scenario-based and customized, so as to produce the occlusion between the upper consumption and the lower supply of UI intelligent assets. At the same time, collect and recycle data from the consumer side, and use indicators to drive technology to generate targeted and efficient iteration and maturity of the system.
Finally, it is operational use (small two workbench). Driven by business metrics, product (product power), design (design power) and technology (business power) work together to deliver only the definition of the product. It’s like object-oriented programming, where we define a cup:
Product delivery definition: Class cup {init(material, height, diameter… “Operational use path here”){material handling (” Operational use result here!” )}
Func material treatment (material){if(material == crystal){1, check the crystal inventory 2, calculate the crystal cost 3,…… … }}}
Operation use: “Operation selection action as an example” New cup (crystal, 10cm, 5cm…) “The starting point for operational use is here”
The definition of the product that the operation delivers together with the product, design and technology around the business indicators: the possibility of the product, for example, the material may be glass or crystal, and then the operation decides according to the business indicators, inventory, cost and other conditions. What are the specific parameters of this operation?
Tools: Products and solutions
Once you have a theory, the second step is to build a set of tools to carry it on. Many companies can tell the theory. If we don’t have a good set of tools to carry on the theory and make it continue, evolve and evolve, the theory cannot be grounded. Therefore, we must have a set of tools behind the methodology.
Organizations change, businesses change, people leave and move on… Only through the productization of tools and technical capabilities can our system continue to stand on the shoulders of our predecessors. In ali scene, a set of basic front-end intelligent tool matrix is finally settled through practice, as shown in the figure below:
We will use these tools to serve various businesses within Ali. At the same time, we will also serve our merchants and users, inject vitality into the ecosystem of merchants and ISVS, reduce costs, and bring users predictable, tasteable and easy-to-use products.
For the promotion scenes, marketing products and channel business of the 11th Singles’ Day, the “promotion on a daily basis” can be realized uniformly by using UI intelligent assets: the planning, design and implementation costs of the promotion can be reduced, and the “daily promotion” can be realized uniformly: With rich scenes, people and goods yard, personalized user products to improve the efficiency of undertaking and business efficiency, the daily channel shopping business has become like a “market”. We will also launch businesses for alibaba Group’s global APP matrix, deeply combine with traffic scenes, and improve the frequency and effect of media access to users in the second and third ring roads. With the help of UI intelligent assets and related tool product system, we can build internal strength of agile business innovation, find our own direction and rhythm together with the business, and flexibly respond to market changes and threats from competitive products.
Organization: Make front-end intelligence really work
We need these tools to be integrated with the organization, to be integrated with alibaba’s existing environment. A simple example, like every double tenth before a battle of the research and development of “0” and whale power applications, the wisdom hall based on double tenth we want to be able to the scene, let everyone familiar with the product, design, produce technology and the application of digital operation ability, intelligent UI assets so as to better understand and consumption. Through continuous learning and application, the use of intelligent, efficient, high-quality and personalized problem solving will become a muscular response, as a natural part of work.
In addition, in daily business, the algorithm model can be used to automate the mapping and translation of concepts to reduce the introduction of new concepts, so that business, PD, designer, operation, technical and quality students can all be responsible for business indicators under their familiar concepts.
In addition to the methodology, tools and organization, to combine these three at the end of the day, we also need a system mechanism, whether it is business system, product system and the system design, the most valuable part often occurred in the intersection, the intersection in the supportive tissue of the methodology, tools, and the mutual restriction: any local innovation can be bound by other global system, and can be assigned to each other: Any local innovation can transfer value to other global systems, but also can incubate chemical reactions: cross-border, cross-field, cross-system global innovation.
Business, product, design, technology — the core customers of front-end intelligence
Do anything must have the customer’s perspective, know who service to know what problems to solve. The core customer of front-end intelligence is closely related to our development stage. In the early stage, it aims at technicians to build the ability of “design draft generation code” centering on “solving the r&d efficiency problem of c-end business”. In the mid – to late-stage, we have to transform technical capabilities into business capabilities. Business-facing people build the ability to “transparently, quickly and directly intervene in the business” around “solving the problem that the business lacks a user product grip”. PD oriented ability to build “WYSIWYG requirements annotation on code generated by design draft can be delivered online” centering on “solving problems of business agile innovation and rapid trial-and-error”. For designers to build the ability of “integration of design and production” around “solving the problems of difficult implementation of design specifications and high cost of design innovation and r&d”.
Operation and use has not been defined as core customers in our system, because today’s small second workbench led by the center of the circle not only solves investment promotion, product selection, delivery… A series of operational problems, build a complete digital drive operational technology system, also with the aid of PU, SOP arrangement way, help business according to their own needs to customize, sirman leadership “Noah” with “ark” in many years of experience in large to promote marketing activities on precipitation, further reducing the use cost of small 2 workbench, We only need to introduce “Noah” or other PU and SOP capabilities in the process of front-end intelligent enabling business to serve the operation well.
Quality assurance is not defined as a core customer in our system, because today, the quality assurance system led by Qingling has been cooperating with us in depth since the early stage of front-end intelligence. From ATS automatic test module quality to TMQ automatic test based on machine vision and algorithm, we have done very well. The technical system at the bottom and the open capacity at the top are sufficient to support us “out of the box”, ensuring the consistency of the functions and experience of the two systems, and facilitating quality acceptance and quality monitoring for students with good quality assurance.
Technical personnel: The efficiency of C-end business r&d
For our ali, technical personnel is a huge population, C side business is an end user oriented and complex scenes, cross-platform, requirements changes, the core issue of personalization is the complex scene, their common requirements “there is a technique to eliminate duplication of effort”, otherwise, can only rely on technical personnel itself to fill the hole. For technical personnel, cross-platform, demand change and personalization are not too big challenges and innovations for technology, but just a matter of time cost: adapting to different platforms, realizing changing requirements, and developing different personalized products one by one are the “repetitive labor” that need to be eliminated.
Only by delving into cross-platform business development issues can we accurately define RunCook’s capabilities and address the issue of adapting and degrading the business capabilities of the host environment during business development. Only by digging deeper into requirements changes can we accurately define the functionality of BizCook, imgCook, LogicCook, Only through NLP’s algorithm model can we understand how business-defined indicators are described as requirements by PD, how designers express PD’s requirements with design language, how design draft generates UI and interactive logic with imgCook, and how changes in UI content and state are derived from PaaS with LogicCook The glue layer code obtained and instantiated into Node FaaS is mounted on the UI and interaction logic. Only by in-depth research on consumers can we accurately define the functions of UICook and Whale Power, and reduce the cost of product functions developed by front-line business r&d personnel for users in different circles through the integration of intelligent design and production and code generation ability, so as to bring the ultimate product experience to consumers.
Business personnel:
When introducing the data center, Xiao Peng said: “To see, for example, at large by observers see macro business governance of wanda group, this is one of the biggest top node. Following the child nodes, is each business executives and business matrix, below each have their own business data logic and data of decision system. Following is every activity, or some nodes, It’s basically a decision-making system. I really believe in the idea that systematic decision-making is what makes the business effective today, but there’s something that gets in the way of that: execution.”
In technical products, there is a vivid metaphor: P9 strategy, P8 planning, P7 design, P5 implementation. Finally, a good idea and theory can not form a good technical product. Why not P9 strategy, P8 planning, design and execution? Because information gets lost in transmission. Popular understanding is: Aunt Zhang said you go home with female students after school, aunt Li said you fall in love with female students, uncle Wang said you married female students. In information theory, the loss of information is studied, because the signal will be disturbed when it is transmitted in the medium, and the fluctuation of the electric signal in the value range does not meet the signal voltage requirements, which leads to the loss of some codes of information. Aunt Li and Uncle Wang’s own bias is the interference that leads to the loss of information. Below, with the help of some information theory of knowledge about service business personnel in the process of how anti-interference?
Optimize the coding of business indicators
When it comes to information theory, it’s about coding. When it comes to information theory, it is inseparable from transmitting information. The process of transmitting information needs to encode information, and how to express the information to be transmitted with the least coding is the target of our research. Suppose two places communicate with each other, and the two places transmit four types of messages: A, B, C, and D all the time. What encoding method should be chosen to use as few resources as possible? If the occurrence of these four types of messages is equally likely, are
In this way, the optimal encoding mode can be achieved, and the average encoding length is
The average encoding length is
Business students decision based on data index have a lot of ambiguity and redundancy, which requires a set of encoding mechanism effectively to encode the business indicators, ensure that in the process of decision-making information is more accurate and effective, because, the greater the probability of loss of the more information redundancy, the effectiveness of the coding, coding the more redundant information. In BizCook, we reconstruct the index system of C-end business research and development and its coding method, and encode the information contained in the index effectively, which is the first step for the effective transmission of decision information.
Measure business metrics for loss
If there is a message sequence whose probability distribution satisfies q distribution but still uses the optimal encoding mode of P distribution, then its average encoding length is
Among them
Their different sizes are zero
KL divergence can be seen as the distance between two distributions, or it can be said to measure the degree of difference between two distributions.
For business students, there is also a distribution of Q and P of business indicators, which are decomposed layer by layer around their two or three core objectives. We can measure the actual situation on BizCook according to cross entropy: The deviation degree of product, design and technology in undertaking these indicators can also be used in BizCook to judge the information loss of decision-making indicators in the undertaking process.
Analyze the association of service indicators
Just like a single variable, if we have two variables X and Y, we can calculate their Joint Entropy.
When we already know the distribution of Y in advance, we can calculate Conditional Entropy
Some information may be shared between X and Y variables. We can think of information entropy as an information bar, as shown in the following figure
It can be seen that univariate information entropy
It can be seen that
Let’s make it a little bit thinner
Mutual information represents information about Y contained in X (or vice versa). The information that X and Y do not overlap is defined asVariation of InformationAnd remember to
For business students, it can be easily measured by mutual information: when products, design and technology undertake their own decisions, the correlation between their indicators and their own goals, BizCook can use mutual information and difference information to measure, so as to help business students solve the deviation problem of target implementation.
conclusion
Due to business decisions is encoded into business indicators, business production index is encoded into a product, design, technical indices, with the help of the generation of intelligent front-end technology system, decision-making and implementation will be unprecedented unity, and is no longer a fragmented, thus, BizCook solve the “Zhang Dama said you and female classmate go home together after school, Aunt Li said you fell in love with a female classmate, uncle Wang said you married a female classmate.” This problem of information loss.
Product and design people
Different from b-end business, in C-end business, design directly determines the visual and interaction of UI in most cases, which is a very core part of C-end business. Therefore, product and design define product functions together in most cases, which can be said together.
The tools for defining a product are requirements documents, and the tools for designing a product are design drafts and interaction drafts, both of which describe: what users? In what scene? See what? What to do? However, the requirements document is visually more abstract and functionally more concrete, while the design draft and interaction draft are visually more concrete and functionally more abstract. As a result, PD and designer combine together. Therefore, usually in the production process to get the design draft is mostly maozi:
This visual and functional dual description, how a “good” word? Simple and clear, the design and product function of the perfect integration together, at a glance. Since PD and designers like this kind of product definition description, why don’t we learn from it in the field of front-end intelligence? BizCook extends LogicCook’s business logic code generation and recommendation by inheriting imgCook’s design code generation. Let PD and designers complete MVP in the process of defining products and put gray scale online for small batch product verification.
As can be seen from the above figure, imgCook generated the content in the left preview area through the design draft. PD can select a block based on this to define what business this block accepts. What associated business metrics have been achieved? What are the functional constraints? How to provide personalized service? … The process of defining products is seamlessly linked to business decisions and business metrics, moving from defining business to sub-business, to page, and to a block.
Therefore, when BizCook serves the two core customers of product and design, the first principle is to conform to the working habits of PD and designers, and use the concepts familiar to PD and designers to describe the product definition and business functions, without introducing additional new concepts, and never using obscure technical concepts. This not only allows the product to be defined as “high fidelity”, but also greatly reduces the cost of understanding and use for PD and designers.
Summary and Outlook
From serving front-line research and development personnel at first, to serving product and design personnel later, and then to serving business personnel in the future, the thinking behind this route is that innovation in the field of technology centering on efficiency must gradually transition to production and then sublimate into the business field. Divorced from business, production simply in the field of technology to go round and round, is restricting most of the technical people to develop up the Wangwu and Taihang.
Looking up at the stars and then back down to earth, we must be aware that the transition from technology to production and even business is a complex and lengthy process. To enter the field of production requires an overall view and structural ability, and to enter the business field requires an overall view and commercial ability. If you do not change and improve yourself, you will end up painting a tiger’s skin rather than a bone.
Most of my colleagues, after hearing, seeing and even part of the practice, will still feel in a fog, at the beginning, feel sensible, when doing, feel a myriad of things. In general, I think what is missing is introspection and introspection. The so-called vistas require us to find the reason for ourselves. The truly valuable thing is the thinking behind the plan and achievement, which cannot be induced, perceived or understood through representation. It is just like two tuning forks, which strike one and the other resonates because the two tuning forks have the same texture. Most of the time when we can’t resonate with other people’s opinions is because our own texture is not consistent with the texture behind other people’s opinions. Just cultivate yourself. The so-called observation requires us to look for reasons at the foot of the road, not overly ambitious, ambitious, to be clear about their every step: light? Heavy? Is fast? Is slow? Even go backwards, revisiting every step of the past.
Today’s front-end intelligence, a little bit of technical success, but also a lot of problems. In today’s attempts to expand production and business, there are some small business effects in the results, but also many defects and deficiencies in the process and even in the source capacity. This requires us to look up and take a closer look. We should not only see the overall direction, trend and strategy, but also see small closed loops, small problems and small details. Focus on the big and start from the small, put the UI intelligent assets and front-end intelligent easy to learn, easy to use into practice, to solve the practical problems in the process of front-line business research and development.
To this end, we visualize the future development of some general direction, trend, strategy, at the same time, such as “out-of-the-box algorithm”, “business to achieve practical results”…… And so on a series of small closed loop, small problems, small details. Under the guidance of Professor zhan Yan, we set up the theme of “User experience Upgrade” in the new fiscal year to focus on the problems we need to solve and put front-end intelligence into practice:
Thus, we set our OKR (or discussion board) to constrain our direction and goals and give a clear path:
Intelligent visual design for scene and crowd
KR1: scenes based on UI interaction and crowd tags are expanded 2-3 times, and the accuracy rate reaches more than 80%. KR2: scenes based on UI interaction and crowd understanding algorithm are automatically generated and users undertake UI. KR3: Use design Center to open up the two systems of intelligent UI and intelligent design, and realize the automatic generation of high-quality schemes
Intelligent human-computer interaction
KR1: End-to-end model and server-side algorithm model can work together to realize personalized PRESENTATION of UI and improve user stickiness KR2: Based on the understanding of users by algorithms, user tasks drive UI assembly and interaction path decisions to improve the effectiveness of user interaction KR3: Develop new user interaction modes and user undertaking scenarios with the end-to-end algorithm model capability, and drive user volume growth with the new strange intelligent human-computer interaction
Lower the threshold for front-end applications in the preceding scenarios
KR1: achieve intelligent UI generation capability and intelligent UI recommendation algorithm, end intelligent model optimization and delivery out of the box KR2: Use PipCook to reduce the cost of front-end learning, application and deployment of machine learning algorithm KR3: The way of sending experts for guidance and co-construction, and integrating front-end algorithm training, drives the innovation of front-end application algorithm in the business of the Group
Looking forward to the new fiscal year, we can build our data, tags, algorithms, materials… UI intelligent assets can also continue to upgrade our production capacity. At the same time, with the help of PipCook, we can lower the threshold of front-end application and make front-end intelligence universal and open to each front-end of Ali Group. At the same time, also expect each participating intelligent front-end colleagues, can be in your own business scenarios, use your eye to find more for machine learning algorithm, the technology, engineering, and even the possibility of the business, with their own wisdom and intelligent front-end, together create more belong to ali front end value to the business! ‘!
