Author: Yue Fei

Product: Tao department front end team

preface

This paper will share with you the topic of how to design and realize PaaS Platform for middle and background building, mainly centering on iceluna, a middle and background building product of Flying ice series of products of Ali Tao Technology Department.

Topic is introduced

According to different roles, construction can be roughly divided into two types of construction products oriented to operation and r&d. Operational-oriented building products mainly carry out complete page building in the form of visual configuration (no-code), such as marketing campaign page building. Rd-oriented construction products mainly build “middle background system” or “wireless module” in the way of low-code development, such as merchant, primary 2 background system construction, wireless Rax module construction. The theme of this paper is the establishment of middle and background system, which is very different from the establishment of marketing activity pages in role-oriented and construction mode. Next, it mainly focuses on the two dimensions of “iceluna product” and “PaaS platform construction”.

The outline

The outline is shown in the figure above. The first part introduces iceluna as a whole, including its product background, positioning and current situation. The second part introduces the design ideas of middle and background building products from three aspects of architecture design, functional module design and r&d process design. Part 3 will focus on the back-end infrastructure dimension to see how ICELuna builds infrastructure. Part 4 will return to the focus of the PaaS platform and explain the core capabilities that need to be built in the PaaS platform. The last part is summary and prospect.

Iceluna product introduction

Product demonstration

Iceluna R&D center includes several main interfaces, such as the site home page, site block center, application R&D center and low-code construction editor. Above is iceluna’s low-code build editor, one of the core capabilities of the Iceluna product, with an item list page in the middle of the example.

Product background!

Iceluna product development has been going through three and a half years. There are a series of reasons behind the iterative evolution of the product to the present PaaS platform of low code development. Before we get into these reasons, we want to introduce a concept that we’ve been talking about today: “Low code development.” For iceluna general construction, has its own understanding, it is through the way of visualization, so that developers with different levels of experience through the graphical user interface, the use of drag-and-drop components and model-driven to build web page development mode.

With that in mind, let’s talk about the background of iceluna products:

  • The war of technology in the background: Amoy department in the whole business side of the consumer side, in the background of this side of the human input is weak, this is currently in the background of a dilemma. We have a large number of vendors or primary 2 operating systems, and the front-end manpower is quite scarce. A large number of systems rely on the back-end/outsourcing/ISV for r&d. Due to the complex front-end engineering environment, rapid technological iteration and high threshold, there are many problems in efficiency/quality/experience/maintainability, etc., how to enable? How can collaboration patterns be improved? Traditional source code mode can not meet the demand of business development, and the demand for low code development mode is increasingly strong.
  • The rise of low code development: According to Forrester market research, low code development can lead to multiple to more than 10-fold improvement in r&d efficiency. This is a business pressure facing the current situation of the middle and back, and it is a problem that we urgently need to solve, that is, how to improve efficiency. It gives us plenty of room to imagine. In recent years, major Internet companies have invested in the construction of low code or no code platform. In Ali’s words, the major BU is also heavily invested in this piece. This leads to the third problem, that is, the convergence and unification of technology after the overflow of construction.
  • Convergence and unification after the overflow of construction: it is conservatively estimated that there are more than dozens of low-code construction products constructed by BU in ** Ali for different business scenarios, with huge investment cost and relatively different degree of capability perfection. How to converge and unify the construction piece? It is imperative to improve the construction infrastructure and provide a unified operation and development environment for the construction service at the group level. Personally, as the person in charge of the group construction agreement, I also hope to solve this problem through iceluna product, so that ICELuna can evolve into a PaaS platform to provide the ability to build underlying services and serve the whole group’s construction products.

Product positioning

Currently, iceluna product has three levels of positioning:

  1. Universal building products in the middle and background: developed by the technology department of Tao Department, available to all research and development personnel, to create a universal building products in the middle and background. Since Amoy business is almost based on business, primary and secondary operation operating system, business logic, complex interaction, it is difficult to abstract fixed scene business template or visual configuration solutions. Therefore, it is necessary for our low code development middle and background building products to have strong versatility and expansibility, so as to cover 100% of the complex interaction of middle and background system page building.

  2. Full-link low-code development platform: a full-link low-code platform integrating front-end application project creation, development, debugging, release, and even page hosting. Shielding the complex front-end engineering system, the whole link through.

  3. PaaS platform: build infrastructure, produce materials based on standard building agreement, and provide operation and development environment of building services for various business scenarios. At present, iceluna’s PaaS platform mainly provides services in the following two modes:

  4. Platform mode: Business r&d enters Iceluna R&D center, the whole link is customized and run on iceluna platform, and business is hosted on Iceluna platform;

  5. Middle Taiwan mode: Separated from iceluna R&D center, iceluna editor ability and low code ability are provided externally in the form of NPM package, helping to incubate independent low code editor in various fields, and deployed independently.

Then, in goal setting, we should have the following three goals:

  1. Assigned to: Fu can is our first important goal, because in the present situation in the background of the development of the business down, said fu can exactly is by far the most can digest the backend business pressure is an important means, this is our trial and error in our business after 2-3 years come to a conclusion, under can assign by the backend, outsourcing, change with the front end of a production relations, To improve and enhance the productivity of research and development projects or the overall research and development team, so that our back-end outsourcing can cross the boundary of work, reduce some unnecessary dependence and cost.
  2. Improving performance: The goal is still to reduce r&d costs. Efficiency improvement has always been an eternal topic of our technology, but as far as iceluna is concerned, the efficiency improvement effect is not particularly ideal, which will be discussed in the next page.
  3. Building ecology: I hope to become a PaaS platform or middle platform to build products in the incubation field and form a product matrix to achieve higher research and development efficiency in various fields.

Product status

Iceluna is a general low-code development platform that provides services for the group. Based on product positioning and objectives, iceluna has achieved good results in enabling, improving efficiency and building ecology.

  • ** Enabling: ** active users 1000+, back-end 44%, front-end 39%, testing 11%, outsourcing 7%. As can be seen from the proportion, more than 60% of the users are non-front-end developers who use iceluna to build system pages. According to the data, ICELuna has a good effect on empowerment.
  • ** Efficiency improvement: there are 440+ online applications and 6000+ pages in **, covering middle and background application research and development of multiple departments of Ali. According to calculation of Halsted software complexity algorithm model (to be introduced in the following chapter), r&d efficiency per capita has increased by about 200%.
  • ** Construction Ecology: ** provides complete construction infrastructure services & PaaS platform services, and has incubated customized construction products for 8+ business scenarios.

Iceluna architecture design

Architecture design

The figure above is an architecture hierarchy diagram of iceluna products from the perspective of PaaS. It can be seen from the diagram that the core consists of five layers: back-end service, infrastructure building, PaaS service, R&D center and product building. The following describes the service capabilities of each layer.

  • Backend services: A Server layer implemented by the Midway framework based on Node.js, providing platform data interfaces and Socket services.
  • Setup Infrastructure layer: The goal is to provide a development environment for setup editors. The core includes the ability construction of four modules, including description protocol for middle and background construction, low code editor, plug-in ecology and material ecology.
  • PaaS service layer: provides the operating environment for building the editor and enables it to have complete ability of building supporting services.
  • R&d center layer: the main front of business R&D, providing cloud integrated R&D process. It consists of four functional modules: site center, application center, material center and data center.
  • Building products: The top layer is the building product layer, which is iceluna’s building products in various vertical fields targeted for incubation as PaaS platform or middle platform.

Functional module design!

Next, we further understand the overall function of ICELuna from the perspective of functional module design, or disassemble the core modules based on the three layers of architecture design core from the perspective of just PaaS.

Building infrastructure layer, including the construction of basic capabilities, including building protocols, visual specifications, engineering scaffolding and other source code level of some of the basic capabilities. Secondly, we built the editor kernel, including skeleton, theme package, plug-ins, controls for configuring the visual property panel, the other is my canvas and rendering engine two core modules, as well as the internationalization ability module. As for the plug-in ecological module, all the functional modules in the overall construction editor are in the form of plug-ins. For example, at the top, we will have model-driven, image recognition, data-driven, logic orchestration, process orchestration and other plug-ins. These plug-ins are currently our focus on promotion or research and development, as the core of the development mode to upgrade and improve efficiency. Secondly, plug-ins such as outline tree, attribute/event/style/data are more common in the editor. At present, visual enhancement is mainly used to improve the efficiency. In the aspect of material ecology, we hope to construct a complete material ecology, developing and building components, building blocks, building templates, component instances, and publishing and sharing materials through low code. The PaaS and R&D center modules will not be covered in detail.

R&d process design

The figure focuses on the R&D process design of ICELuna R&D Center, which can be divided into two authority roles:

  1. Station manager: generally responsible by the front end of professional services. After a site is approved, it can complete the basic information of the site, default configuration of editors, default configuration of applications, default configuration of materials and other information. This configuration story determines the application theme, build edit form, and material selection pool to be created under the site.
  2. Application administrator: After entering a site, you can create applications, create multiple branches, access the Setting up page or components in the Setting up editor, perform real-time online debugging, and publish and deploy a site with one click.

Middle and background building infrastructure construction

Understand iceluna from all dimensions of the design, I believe iceluna product design has a more comprehensive understanding; Then, from the implementation level, we take a look at the infrastructure capabilities built by iceluna to ensure the technological advancement of the platform.

Content from this page are our whole low code to build the infrastructure of a content spreads, we can see from the left to a material research and development process, students of professional front research source material, and settling into the center of the material, such a source material, through a parsing module, we can generate a building component description file, with the description file, The low build editor recognizes this component, generates a property configuration panel for this component, and has a good build editing experience. Second, the editor can publish an existing source code component to the material center and precipitate it into a build component. Multiple building product matrices, based on the same protocol standards, and upstream and downstream of the material center, so that materials can be circulated and reused, so as to form the material ecology. As you can see from the far right, front and back ends, outsourcing, and testing all use low code editors by means of scaffolding, and low code editors become the most core capability. How to guarantee the technical advancement of low code editor and the overall construction ideas are expounded from the following five directions:

  1. The first step is to define a standard construction description protocol that all construction products follow, so that materials can be circulated.
  2. We want to construct a build editor development ecosystem, iceluna as a PaaS platform or a medium platform to provide a capability to low-cost incubation of low code editors in various scenarios;
  3. The low code editor we provide is an appeal for different ends. We will include the construction of React, Rax, small programs and other technology stacks. The editor needs to be able to support multiple technology stacks and adapt to multiple ends.
  4. How does it stay technologically advanced compared to other low code editors, and what are its core capabilities.
  5. It is the plug-in ecology and material ecology that we want to create.

Build description protocol standard specification

How do I specify a standard specification to build describing protocols? My thoughts are elaborated from the following four aspects:

  1. Versioning, semantic, progressive description: version control, clear semantic, concise, readable; The essence of building is the process of nesting and combining source code components from small to large, combining components, blocks and pages in turn, and finally building applications through the cloud. Therefore, in building the basic protocol, we need to know how to progressively describe the four entity concepts of component, block, page and application.
  2. Do not introduce new concepts, can be exchanged with standard source code: do not introduce new grammar concepts, the code part of pure JS grammar, reduce the threshold to get started; The corresponding conversion relationship between each attribute and source code can be clearly defined to generate high-quality standard source code with no difference from hand-written;
  3. Extensible, liquid, multi-end: support the introduction of third-party NPM packages, enhance the scalability of protocol description capabilities, to cope with the complex and changeable needs of different applications, such as Lodash, moment.js and other third-party tool libraries; Products can be circulated in different building products, without any private data storage, unified standards, building material ecology. Not only React, but also applets and so on;
  4. Support internationalization

Low code editor development ecology

Iceluna, as a PaaS platform or construction medium platform, we hope to atomize and open all the capabilities of the bottom layer of the construction editor, so we take the following issues into consideration when we build the construction editor:

  1. Layered architecture: the whole framework is divided into four layers of capability construction, the innermost layer is to build the editor kernel (mainly message communication, state management and configuration of the analysis, skeleton loading, plug-in mechanism loading and other capabilities); The second is the rendering module, that is, the part of the rendering module, its input is in line with the Schema of the building description protocol, through this module can render the whole page; Next up is the layout module, mainly responsible for the canvas area of material drag, drill down editing, click, shortcut keys, multi-design mode and other operations, providing flexible expansion ability; The top layer is the framework of the entire editor, including the skeleton, theme and all the panels in the editor are integrated in the form of functional plug-ins.
  2. Module decoupling: Here the framework is layered, each with its own NPM package, providing atomized service capabilities to open. For example, we can open it to the scene as a whole with a whole low-code editor, or we can only open it as a choreography engine or a rendering engine, such as a preview of the material in the material center.
  3. Extended capabilities and development ecosystem: In addition to the existing capabilities, we provide a complete skeleton, plug-in and control development scaffolding and command-line tools to ensure that the entire low-code build editing and development mechanism is complete, and the entire skeleton and plug-in can be customized or customized across our platform.

Low code editor is versatile

At present, the fields that the editor is built for are not only the React system in the middle background, but also the Vue, small program and Rax system. Due to the difference of the underlying technology stack, there are great differences in component parsing and rendering in some systems, and the total page cannot be rendered by pure React rendering module. So, how do we adapt to multi-terminal? We need to achieve the corresponding rendering engine for different technology stacks. Through a very thin layer of adaptation layer, our construction editor can support the rendering of each technology stack, so as to achieve the purpose of multi-terminal adaptation. For example, ali form data report building, IMGCook consumer terminal building, Taobao small program building and so on.

Low code editor core capabilities

The fourth part is how we secure some core capabilities for building advanced editor technology.

  1. Out of the box: provides the whole link integration construction service, no need to go offline two times of development. Second, it supports custom build editors and custom business theme styles. At the same time, it supports multi-person collaboration and multi-branch parallel development on our platform, which can deal with large and complex projects. For example, tao marketing system, there will be dozens of people developing the same application in parallel at the same time, often establishing several branches to develop requirements in parallel. Therefore, for some large and complex systems like this, the middle and background building system does not have the ability of multi-person collaboration and multi-branch parallel development, which is basically impossible to be implemented in our business scenarios. So these two pieces of capacity building are very critical and important.
  2. Safety sandbox isolation: we conducted market research on many construction products in the industry and found that many construction editors did not do a good job in sandbox isolation. Iceluna has been developing for 3 years, iterating from isolation to non-isolation. Finally, all problems are completely solved and sandbox isolation is fully realized, so as to ensure that the page is completely isolated from the editor itself without interference, and independent theme setting is supported.
  3. Real-time debugging ability: our canvas is a true Runtime, it is not a simulator or incomplete rendering, many low code editor in the industry in a state is to build a pure UI rendering, the configuration interaction by low code data, or events, it can’t, need to preview or post link to debug, etc. However, the business logic of middle and background scenarios is very heavy and often requires high-frequency real-time debugging, which is different from other construction products and is an important ability to combine business scenario construction.

Low code editor material ecology

The universal building platform we provide has different demands for materials in different business scenarios. Today, our build platform serves more than 400 front-end applications, serves 20+ divisions of the group, or a larger organization, and the usability of the build platform is compromised by providing only one set of materials. We need to be able to quickly access different materials according to different BU business scenarios. The first point is to be able to quickly produce materials, the second point is to be able to quickly access existing materials, and the third point is to be able to make the material flow and become an ecological mechanism, just like Iconfont icon ecology. So Iceluna is also working on how to create an ecosystem of low-code scaffolders. Our core work in this area is as follows:

  • Unified construction material description protocol: unified standards, standardized production, improve the reuse of construction materials.
  • Low cost access of materials: Low-cost access to support the React component NPM package, don’t need to package the components in a 2 times or development, a form through simple configuration, components can be access to come in, and to ensure all attributes of the components in complete source code, in the property configuration panel can have the ability to complete visual configuration and no matter what is your property type, Array types, object types, and ReactNode types all have full visualization mechanisms to ensure a good editing experience.
  • Building material circulation: Building building material market, forming ecological mechanism similar to Iconfont.

Back on the left, our low code editor, it’s not just the ability to plug in components, the most important capability is the ability to produce components in low code. Why? Low code editors are aimed at a wider audience, such as backend and outsourcing students, who do not have much knowledge of source code or the engineering environment of source code, but they will also have the appeal of making components. Secondly, building the editor itself is a development mode to improve efficiency. Whether it is front-end or back-end, research and development pages or components, low code development also brings efficiency improvement on the development side. On iceluna, we also provide an exclusive construction editor for materials, which can build materials on our platform by means of construction and send the materials up to our material center, finally forming a material circulation mechanism.

PaaS platform construction to provide construction services

Let’s get back to the core of the theme — PaaS platform capacity building. PaaS: Platform-as-a-Service (PLATform-as-a-Service). Iceluna’s PaaS positioning is to provide the operation and development environment of the build editor as a Service to build products in different business scenarios.

Speak to the previous chapter to build infrastructure build editor is to provide the development environment, the upper side also need a better platform service capacity, to provide the editor for a good running environment, complete makes us with the integration of research and development ability, in the whole ability, on the basis of incubation building products in the field of the vertical.

We will introduce the capabilities of Paas services from the following six dimensions: r&d center editor customization, cloud build & Publish & Store, multi-player collaboration, multi-branch development, code rollback, and performance measurement.

Build the editor custom service

In the center of our site, we provide a visual way to make an editor. Different build editors can be built through cloud construction, such as the Iceluna build editor on the right and imgCook editor below, which are two different sites created in the R&D center. We just saw a configuration for the editor, so what can be configured for the editor?

  • ** Layout customization: ** editor for different fields, different scenes, do consumption pages and background pages, it is different for the size of the panel canvas and available area, so we have different demands for the whole build editor panel layout. We can quickly build different layouts through a layout customization, and then make different plug-ins to form a new build editor.

  • ** Theme customization: ** Build editor can be embedded in different scenes, such as Amoy department inside we can embed WebIDE with source code for a mutual ability to get through, you can switch to the source code development, you can also switch to the visual development, under WebIDE, it is a dark visual style, So we also provide a theme pack configuration capability on the platform, and then customize the editor for different theme styles.

  • Plug-in customization: We provide a complete plug-in mechanism, and all panels of the entire build editor are carried in the form of plug-ins. Currently, there are 26 plug-ins on iceluna editor (visible on the right of the figure), and we will deposit more and more public plug-ins in the plug-in ecosystem. And the plugin can be embedded into the build editor. If the pool doesn’t have the plug-ins you want, we also provide plug-in development scaffolding for you to implement a separate plug-in decoupled from the editor’s functionality, pluggable, and customizable.

Cloud building/publishing/DB storage services

Cloud construction service is one of the core links of iceluna low code development platform. At present, the ability of cloud construction is mainly to store the fragmented data of the Schema generated by the components in the database after the pages built in the process of application construction. To publish into application or component or editor resource package, we also push the source code of these applications and components constructed to GitLab for storage, and also to CDN, apply for CDN resources and finally push to the line. At present, cloud construction mainly supports application (daily/online publishing), component (Low code/ Procode publishing), editor (canvas/frame construction), and six forms of construction capabilities.

The cloud construction architecture diagram is divided into four layers: data layer, operation layer, communication layer and application layer, as shown on the left. Its core capabilities include the following:

  1. Editor decentralization: On our platform, each application created on our site has its own editor resource bundle, so we can customize each application’s theme and component extensions with versioning control.
  2. One-click release deployment: Rights control; Dynamic analysis of component dependencies; During branch publishing, you need to merge the trunk and resolve conflicts online if any. Webpack build and CDN publishing.
  3. Multi-system access: GitLab storage and code rollback mechanism through GitLab, followed by Tair to do concurrent lock during the build process, and finally, through ODPS to do build log analysis.

Multi-person collaboration service

In PaaS platform, multi-person collaboration is an indispensable ability, its main principle is through WebSocket connection plus a file lock mechanism, file lock in the platform currently includes page lock, component lock, application level public file lock these three dimensions. The main idea is to use WebSocket alive mechanism, and Tair maintain a heartbeat alive message communication. On the Tair side, a distributed optimistic lock with active failure is stored, and then the information about the lock is stored. The lock will fail if there is no new heartbeat within about 10 seconds. So once the client or Server is broken, the file lock will be released automatically, a mechanism to do multi-party collaboration services. We have also done some research on multi-person collaboration solutions in the industry. For example, the advanced OT technology in the industry, such as Pin Documents and Google Docs, is used to achieve relatively complex and more powerful functions. For low-code build editor scenarios, the ability to edit locks is sufficient. The points highlighted by the red line on the left iceluna editor are all locked.

Multi-branch parallel service

One of the most important problems solved by multi-branch parallelism is conflict resolution. For source code, conflict resolution is an existing capability, but it is a very difficult problem to solve for low-code cloud engineering systems. So far, the code of iceluna conflict resolution is still the code to build the Schema describing the protocol, which is relatively difficult to compare, which is one of the problems. Secondly, the total conflict resolution process, multi-branch parallel, including code rollback to the database DB, this part of the total mechanism construction is relatively complex. The overall process is shown in the figure above.

Code rollback service

The code rollback service makes use of the baseline synchronization mechanism, which ensures that we can specify any Commit hash for the editor to apply the code rollback. Because every release we make on a low-code platform synchronizes our code to Git, any release can be rolled back to our low-code editor. The biggest difficulty here is that “database fragmentation information” and “Git repository source engineering files” can have a one-to-one conversion relationship.

Set up efficiency measurement system

Low code platform performance measurement system, at first glance not very technical, but why talk about performance measurement system? We have been doing platform building for three years, and other brother teams are all faced with a problem, that is, they all say that building can improve efficiency, but there is no precise method or strategy to measure how much improvement efficiency, so we spent a long time studying how to have a strategy to measure whether building really improves efficiency. The specific implementation method and measurement standard is that we rely on the complexity measurement algorithm model of Halsted software in the industry. This algorithm model can take the Schema generated during page building as an input. It uses dozens of operators such as the number of expressions and code length in the Schema. You can calculate the complexity of the Schema and the estimated development time.

Of course, the estimated development time needs to be adjusted repeatedly, and a relatively accurate number will be obtained after more consistent with the real situation. In addition, low code set up relative to the source development, there is a benefit is that users are on our platform, the platform side can be buried, operation logs and other means to record every developers operating records on a page, in two operating interval length in 10 minutes to calculate effective development period, The sum of valid time periods is the actual development time. By calculating r&d performance = estimated development time/actual development time, you can know whether the user’s development performance is improving or decreasing. In the iceluna platform, there is a special performance center in the data center to feedback the overall per capita R&D performance and individual R&D performance of the total platform. So this is very valuable, so share it with you.

Summary & Prospect

  • Summary of the road ahead: The construction cost of general construction products in the middle and background is extremely high, which can well solve the problem of empowerment and collaboration, but the r&d efficiency is not up to several times or even 10 times the expectation. It is necessary to upgrade the new Nocode R&D mode such as model-driven and intelligent construction, or build product matrix in the construction field to achieve the goal of several times efficiency improvement.
  • Future vision: Focus on building iceluna as hpaPaaS (ultra high productivity platform) in the middle and back office space. If like-minded, looking forward to your joining!