Business practice of Ceramics, Youdao interactive content engine

Element representation. A button can be a press button or a switch button, which are used in different scenarios. We have customized the hook of usePress and the hook of useToggle to deal with the interaction between mouse and touch on the Web and mobile terminals in a unified manner, smoothen the differences between browsers and expose simple API. Based on usePress, the useButton component is encapsulated, indicating the ability of the button to be pressed; UseToggle and useButton encapsulates the useToggleButton component, which represents the basic capability of the switch button. Finally, the Button component is bound with useButton or useToggle to become Button component and ToggleButton component, that is, corresponding capabilities are given to the Button.

Figure 5: Button component interaction capability design

For example, drag and drop interaction, as shown in FIG. 6, is simply the behavior of dragging objects to the designated hot zone. True or false judgment is to compare objects in the hot zone with objects in the answer setting at the specified time.

Figure 6: Drag objects and hot areas

Both draggable and non-draggable objects are essentially the same powerless entities. The drag capability is encapsulated in a separate component to handle the movement behavior of mouse and touch and the collision detection of drag objects and hot areas. The draggable ability is given to the dragable object, the droppable ability is given to the target hot area, and the drag behavior can be realized by the subsequent logical processing of the interface exposed by the drag component.

In addition to the implementation of interactive capabilities, ECS design ideas are reflected in many modules in Ceramics, such as capabilities represented by custom hooks that allow for greater logic reuse. There are many custom hooks in Ceramics, such as useQuestion hooks, which integrate the process of doing questions, including checking answers, providing correct/false feedback based on the check results, and accumulating the number of errors. UseViewport hooks, which encapsulates mobile adaptation capabilities UseAnime hooks, which encapsulate animation operations such as play, pause, replay, etc. Using the same hooks from different boards can unify the code style, improve development efficiency, and make it easier for new developers to understand and learn from.

Figure 7: useQuestion hooks

3. Animation effect

Animation is an important part of interactive content. Lifelike and interesting animation effects can not only assist in displaying teaching content, but also help improve user immersion, attract attention, stimulate interest in learning and retain users better.

Common Web animation schemes mainly include sequence frames, CSS animation, JS animation, Canvas, WebGL, etc. Ceramics supports various animation forms by introducing common animation libraries, and carries out secondary encapsulation for business scenes and compatibility. For some simple animations, the development only needs a simple API call to achieve the effect. Some front-end to achieve high cost of complex dynamic effects, by the designer through AE, Spine and other tools, the development of the corresponding components introduced in Ceramics to achieve animation rendering and control.

3.1 Lottie

Lottie is a dynamic solution launched by Airbnb that can be applied to Web, Android, iOS and other platforms. After animators create animations with After Effects, the AE Bodymovin plugin can export json files that render animations on mobile and Web terminals. The front-end uses Lottie libraries to parse and render JSON files containing animation information. Lottie animation is simple to use, high efficiency and high performance, with strong effect reduction ability and good cross-platform support. It can be well presented in iOS, Android, Web and small programs.

Figure 8: Lottie animation

3.2, has been

Spine is a common tool for 2D bone animation production in game development. The animation produced by Spine can be exported as JSON + ATLAS + PNG file. Spine Runtime is an official library provided by Spine that can parse animation files exported by Spine. The HTML Canvas Runtime provides the ability to render Spine animation on the Web.

Spine animation can be used in different engines and languages, so designers can adapt a set of Spine animations to different applications that require cross-end collaboration to improve work efficiency and avoid duplication of effort.

Figure 9: Spine animation

3.3 Animation-effect library based on Anime.js

Animo.js is a lightweight Javascript animation library that animates CSS properties, SVG, DOM and Javascript objects. Animo.js’s built-in interlacing system makes complex animations and overlapping animations easy.

Based on animos.js, Ceramics has developed a dynamic effect library more in line with business requirements, which encapsulates the general dynamic effects of the board, such as:

1. Emphasize (zoom, change styles, shake, blink)
2. Element entry and exit animation (fade in, fade out, flash, drop)
3. Element entry and exit animation (fade in, fade out, flash, drop)

Dynamic effect library is also an independently published NPM package, which can serve multiple projects and improve development efficiency. The dynamic effect components in @ceramics-AE are directly introduced into the project, and the effect can be achieved by adding dynamic effect names and parameter configurations to DOM elements. Timeline can be used to series single-step animations, supporting single playback, circular playback, time axis and other functions.

import { Teeter } from "@ceramics-ae/core"
Teeter.play({ el }) / / jitter
StyleChange.play({el,config: {style: { color: "#F5636C".background: `center/100% no-repeat url(${OptionStyle[type].bg_wrong}) `}}}) // Change the CSS style
Copy the code

3.4, WEB 3 d

The most popular technology for implementing 3D effects on the Web is WebGL. WebGL is based on OpenGL, which is a cross-platform 2D and 3D drawing standard. WebGL combines OpenGL with JS to display 3D scenes and models on the Web.

WebGL is powerful and provides a large number of basic drawing apis, but it has a high threshold to get started and the code complexity is high, requiring developers to have pre-knowledge of graphics and mathematics.

As an open source WebGL engine, three. js encapsulates the underlying rendering details and complex data structures of WebGL. Developers can configure lighting, vertex shaders, slice shaders, automatic matrix generation, perspective control, ray, etc., through simple API calls, to quickly make 3D models. We made 3D effects such as Rubik’s cube and stereo graphics based on three. js, which are easy to develop and have high effect reduction degree, and can maintain good performance and compatibility on mobile terminal.

Figure 10: Web 3D

A group of high quality effect is often a vision, move together to the product of work division, front-end developer, the developer according to different application scenarios, material dependence, development cost, performance and compatibility choose different animation program, can express a rich animation effects in the lower cost of designers and r&d collaboration at the same time, improve the efficiency of team work.

4. Design of board components

As an interactive content engine, one of the goals of Ceramics is to efficiently produce topic templates for multiple businesses. We hope that each topic template can serve more businesses and be accessed more quickly when new businesses need it.

Based on this, we packaged each board into board components, developed independently, and independently published as NPM package. The board is decoupled from the service, and the board is decoupled from the board. The upper business side is responsible for controlling the combination and switching logic among multiple boards. The board components can also be introduced into other boards as sub-components for secondary packaging to improve the reuse of the board to a greater extent.

Figure 11: Call relationship between board components and business side

Usually used when the upper business topic board, need to collect the topic plate internal state to implement the business logic of the external, for example: topic plate loading successfully remove the transition animation and background music, the user switch when a mistake is included in the wrong topic this, right next question and so on, the problem also need according to the external data update status inside the plate, such as: When the user enters the topic in different game modes, the board should switch the corresponding interface and logic.

For the above business scenarios, the board component defines a common set of interfaces:

Figure 12: Board component interface

Upper-layer services install the NPM package, import the plugboard as required, and pass in props according to the interface definition to complete the access.

import { CeramicsProvider, DragCetegoryQuestion } from "@ceramics-math/core"
<CeramicsProvider>
  <DragCetegoryQuestion// Title template componentquestionConfig = {}// Configure datadeviceConfig = {} /> // Device configuration data
</CeramicsProvider>
Copy the code

The board components are independent of each other, but have similar logic and component dependencies. In order to better reuse code and manage project dependencies, Lerna framework was introduced for multi-package management. Multiple board components were merged into a parent project as sub-projects, and repeated code logic was also extracted into another independent sub-project under the parent project.

Figure 13: Structure of multi-package management The benefits of multi-package management are as follows:

• All subprojects can share the same package, test configuration;
• Internal subprojects can be introduced into other subprojects just like other dependency packages. Lerna will automatically identify and guide internal projects and synchronize updates in real time.
• It is more convenient to handle the dependency packages. Run the Lerna bootstrap command to install the dependency packages of all subprojects. During the dependency installation, the dependency packages of all projects are promoted to the root directory to avoid multiple installation of the same dependency packages in different projects.

Lerna commands can easily execute the build, test and component release processes of all components, update the corresponding dependent version numbers of components and release them, tag and generate Changelog after component release.

In addition, due to the different plate in the packaging configuration, interface definitions, rules of mobile terminal adapter, component dependence, etc. There are more similarities, in order to facilitate new colleagues to fit in to avoid interface omissions, unified code style and data structures, we also define the initial template, the introduction of automated build tools Plop assist quickly create board components. You can dynamically generate board components, including initialization code and configuration files, in a specified directory through interactive command line input.

Figure 14: Interactive command line

Figure 15: Initialization templates and generated files As described in this section, clear interface definition, convenient and fast access, multi-package management and the introduction of automatic construction make it possible for upper-layer business to quickly access the board component library, and the development, iteration and maintenance of board components are also easier.

Atomic components and plug-ins

In order to further improve the development efficiency, we have encapsulated many atomic components and functional plug-ins according to common business scenarios, which can be introduced by developers through simple component import and parameter transfer.

Figure 16: Atomic components and functional plug-ins

We think of the smallest elements in the interface as atomic components, such as buttons, titles, displays, options, progress bars, and so on, depending on the different configuration styles and interactions of props.

Figure 17: Atomic components

Functional plug-ins are composed of a number of atomic components, coupled with some business logic, common plug-ins such as handwriting board, calculator, abacus, and gamification of common entry, countdown, guide animation, etc.

Figure 18: Functional plug-ins

Figure 19: Keyboard components

A board component can have multiple atomic components and functional plug-ins. Atomic components and feature plug-ins may depend internally on other atomic components, as well as on generic hooks, global Context, animation, utils, and so on. Atomic components and functional plug-ins are also subprojects of the multi-package project and can be exported and published independently as NPM packages, which can be introduced not only by any board component, but also by other atomic components, and into other projects.

###6. Universal problem boards

In addition to the atomic components and plug-ins mentioned above, Ceramics also has built-in universal board components to further improve development efficiency.

General boards are some of the most basic question templates, such as general drag and drop, general select, general fill in the blank, general line, etc. The common board encapsulates the basic logic and interaction, and more business logic is expressed in the form of data. In this way, the reusability of a single topic template can be improved, and a common board can support N kinds of interaction and high style similarity requirements.

Through the classification and abstraction of multiple boards, we summarize the basic elements of several basic boards:

The multiple drag problems in the figure below are composed of canvas, hot zone, dragable object and undragable object.

Figure 20: Basic elements of a drag and drop problem

Similarly, the basic elements of the board are canvas, clickable object, and unclickable object. Fill in the blanks with canvas, object, blank box, option box; There are canvases, wired objects and unwired objects. Object types include pictures, animations, text, etc.

In addition to the basic elements, the feedback of the board is also similar: for example, the correct animation will appear after the correct answer, the wrong animation will appear after the wrong answer, the line matching is successful, or when the drag is correct, object A will disappear and be replaced by object B, etc.

Consistent with the ECS design mode mentioned above, basic interaction of a topic can be realized when an object is given drag-and-drop, clickable, wired and other interactive capabilities, and corresponding feedback is added to each interaction.

Based on this, we abstract the general board, in the plate processing elements in the rendering, user interaction and submit judgment logic, as for the page in which elements are click, which can drag and drop, the number of each element, style, location, and animation, and different timing of feedback, etc., can use the data to express, through the data driven view rendering.

We encapsulate some basic classes, such as Entity, Feedback, Area, and ShowCondition.

All elements in the page belong to the Entity class. EntityType represents the type of the object, including pictures, animations, text, titles, submit buttons, etc. Different types of objects have different configuration items and rendering methods.

Add different properties to an entity to give it different interactions, such as clickable for options that can be clicked, Draggable for objects that can be dragged, and Droppable for hot spots that can be placed. Droppable hot spots can also be configured with properties such as shape, capacity, and adsorption accuracy.

export interface Entity extends Rectangle {
  id: string | number // The unique id of the object
  name: string // The object nameentityType? : EntityType// Object typedraggable? : boolean/ / can drag and dropdroppable? : boolean/ / can be placedclickable? : boolean/ / clicklinkable? : boolean/ / can be wiredzIndex? : number// Level, 0-999, the larger the level, the higher the levelshow? : ShowCondition// When to display. The default is alwaysfeedbacks? : Feedback[] [key: string]: any }Copy the code

The Feedbacks field is used to configure a set of feedback events, defined primarily by timing and behavior.

export interface Feedback {
  type: FeedbackType | string // Feedback timing, FeedbackType, AND, OR string
  action: FeedbackAction // Feedback behavior
  data: FeedbackData // Feedback behavior needs data
}
Copy the code

The feedback time (type) includes: click, put into hot zone, hot zone is full, drag right hot zone, drag wrong hot zone, start connection, end of connection, animation play completed, correct overall judgment, wrong overall judgment, etc.

Feedback actions include: show objects, hide objects, play audio, replace pictures, check answers, etc.

The same action can trigger more than one feedback, and one feedback can be triggered at different times. With Feedbacks you can chain feedback steps together, much like an animation script, making a regular board richer for gameplay.

After the above disassembly, the general board design ideas are gradually clear up. Take the generic multiple-choice board in the following figure for example:

Figure 21: General choice board

The core interaction process of multiple choice questions is as follows:

1. Click on clickable elements to trigger click feedback;
2. Meet the submission conditions, trigger submission feedback, test, trigger positive/false feedback;
3. Meet the end of the question conditions, the end of the question can not answer, otherwise you can continue to answer.

There are three main opportunities to submit your answers:

1. When there is no submit button, click the clickable element to submit immediately;
2. When there is no submit button, submit immediately after clicking the number of elements equal to the number of allowed submissions;
3. When there is a Submit button button, click the Submit button to submit.

Multiple choice questions support single choice, multiple choice, partial choice, no correct choice (such as voting). So the conditions for judging the answer are different:

1. Single selection & multiple selection; The user’s answer is consistent with the correct answer of the question;
2. Partial selection: the user’s answer is consistent with the correct part of the question;
3. Option with no correct answer: Any option is correct.

Here is the basic data structure for the general selection:

interface QuestionConfig {
  canvas: Canvas / / the canvas
  answer: Answer // The correct answer
  entities: Entity[] / / objectsubmit? : SubmitEntity// Submit buttonaudio? : {/ / soundcorrect? : string wrong? : string click? : string } aeTemplate? : string// Animation templatemaxCheckNum? : number// Maximum number of errorsallowMultipleChoice? : boolean// Whether multiple selection is allowedallowPartChoice? : boolean// Whether partial selection is allowedshowHint? : boolean// Whether boot occurs
}
Copy the code

As mentioned above, the topic is composed of board and data. The main business logic in the general board includes processing the above rendering and interaction, defining the data structure of configuration data, and presenting and feedback accordingly according to the configuration data. For example, render entities, bind click events to clickable objects, and display feedback from entity feedbacks. Submit logic and render the submit button according to the Submit field; Determine the question according to the fields of answer, allowPartChoice, allowMultipleChoice and so on.

Through data-driven page rendering, a board can meet most of the teaching and research needs of the same type. Teaching and research can configure data in the supporting visual configuration interface of the board to create new boards without the intervention of RESEARCH and development. Developers can also carry out secondary packaging based on common boards. Common boards can be introduced into other boards as components, and multiple common boards can be combined to form new boards, which greatly reduces the development cost of new boards.

Iii. Summary and outlook

This paper mainly introduces the technical realization of Youdao interactive content engine Ceramics in the aspects of architecture design, user interaction, animation effect, component design, etc. It tells the production process of interactive content from the perspective of RESEARCH and development, and how the interactive engine makes the production link of interactive content run more efficiently.

Ceramics is still in the growth stage. In the future, we will continue to iterate and enrich the development tool chain. We will build a low code development platform, a patchwork visual development board, making it easier to write interactive logic; Open the conversion channel from design drawing to code, develop material library, optimize UI and r&d cooperation experience; Increasing efficiency and rendering capabilities, moving towards gamification, supporting more complex gameplay and cool animations; Increase the ability of real-time interaction and multiplayer battle, further improve the interactive and interesting scenes such as live classes.

Our vision is to build a high-performance Web interaction engine that enables front-end engineers to develop high-quality interactive content at a lower cost. In the future, we will continue to expand Ceramics’ influence and empower more businesses.