Oasis Engine (hereinafter referred to as Oasis) is a mobile first, Web first interactive Engine developed by RichLab Interactive Technology team of Ant Group. In this sharing, Chen Mo, lead architect of Oasis, will bring you Oasis summary of the past year, plans for 2022, and open plans for Oasis editor. Please enjoy. Class of 23, come to the bowl

Hello everyone, I’m Chen Mo, the architect of Oasis engine. Today I’m sharing Oasis Anniversary Review and Future Plan with you.

Today, I’m going to share three parts, the first is a review of open source one year, then Roadmap2022, and finally I’m going to share with you the Oasis editor open plan.

What did we do

We made four milestones in the past year, from 0.3 to 0.6. If you have been following us since last year’s open source launch, you can certainly feel our progress this year. These four milestones include engine improvement, official website revision, documentation improvement, case enrichment and open source construction.

A larger technology

Let’s take a look at the big picture of the engine. The features in this picture come in three different states. Among them, dark green and light green represent the existing functions. The light green is the key part to be redesigned and added in 2021. The hollow icon represents the functions that we are planning but have not realized at present. Here we highlight our focus features for 2021. The first is the node and component parts. We redesigned and added animation, mesh, physics, Sprite, SpriteMask, Spine, Lottie, etc.

We added background and ambient light to the environment part of the rendering, and we redesigned PBR, Unlit and BlinnPhong for the material part. In the assets section we have refactored the industry standard glTF and added private asset grids, materials, textures and SpriteAtlas. For Input interactions we have added Pointer interactions.

Mesh

We redesigned and streamlined the entire Mesh system. First we removed the original GeometryRenderer, leaving only the MeshRenderer. The MeshRenderer data source is reconfigured into two types, BufferMesh and ModelMesh. BufferMesh is mainly gPU-oriented, and data is based on the concept of Buffer, aiming at powerful, flexible and high-performance design. ModelMesh is model-oriented, with vertex as the concept of data and docking industry model standard. It takes easy-to-use and simple functions as the design goal.

The use case for ModelMesh is glTF, which becomes ModelMesh after being parsed. A good example of BufferMesh is our Spine ecosystem, which has been redesigned with BufferMesh to dramatically improve performance. If developers are familiar with GPU principles, they can use BufferMesh for high-performance geometry data manipulation. There are also common base grids such as Cube, Sphere and Plane. We have provided static PrimitiveMesh factory methods that can be used to create these primitives directly using a static method. The return type is also ModelMesh. We removed the presence of separate classes SphereMesh and PlaneMesh. For developers, one more class means one more concept, and one more concept means one more learning cost. In engine design we always pursue moderate abstraction, even subtraction.

Material

We recombed and optimized the materials, including UnlitMaterial, BlinnPhongMaterial, PBRMaterial and PBRSpecularMaterial. First, we unified the material interface of each renderer. We put the function of setting material for each renderer into the parent class of the renderer. So MeshRenderer and ParticleRenderer have the same texture-specific interface functionality. Second, we unified the rendering state of the material, such as transparency and blending mode. Finally, we unified the basic functional attributes of the four materials into baseColor, baseTexture and tilingOffset. Because there is a sense of primary texture and primary color in any material, Diffuse in BlinnPhong and albedo in PBR. Through the unification and sorting of material related apis, the ease of use is improved from the perspective of developers’ learning and use.

We’ve also enhanced material related features. The renderer has added multi-material capabilities and instance material capabilities. The multi-material capability is used to support typical submesh functionality, where a mesh can correspond to multiple materials. The main feature of instance materials is to avoid script modification of shared materials that can affect multiple models. You can modify the instanceMaterial interface to keep the material effects independent. We also made adjustments and optimizations to the PBR API and rendering stream.

Let’s take a look at the PBR rendering flow. PBR is not only its own material algorithm, but also includes ambient light and even linear rendering space. We first enhanced the ambient light of the scene. We used to have a solid color diffuse mode for ambient light, but now we have a spherical harmonic mode. The spherical harmonic mode is more delicate than the solid color, and we use the third order spherical harmonic. We use pre-baked textures for our specular reflections. We baked the reflection data of different roughness into different MIPmaps of the cube texture.

In fact, both diffuse reflection spherical harmonic and reflection texture need to be calculated, this is a considerable way in the effect and performance, before there is no open editor we provide an independent online baking tool. All you need to do is drag and drop the HDR environment map into the online baking tool to generate a binary of the ambient light. After generating, load the ambient light file directly. It will be easier to use in the future when we open up the editor, and the whole process will be almost senseless.

Then there is the improvement of linear space, using linear space both from the attenuation of light and the increase of brightness curve are more natural. After a series of optimizations and adjustments, PBR’s rendering flow has been improved to a certain extent in terms of effect and performance.

glTF

The glTF standard has been used as a standard model format for Oasis. GlTF is an industry standard and has been used as a runtime format in the past. We also designed our own private asset format, because private assets are needed for everything from scalability to extremely small file sizes to extremely fast loading and parsing. However, as an industry standard, glTF is relatively easy to use and has a certain standard value. Our support for glTF will be retained for a long time. GlTF works as an art bridge even if private assets are used.

The PARSE pipeline of glTF has been refactored to include pipelin-style parsing and support for custom extensions. In the middle is a list of our currently officially supported glTF plug-ins. If developers feel that these plugins are not functional enough, or even some are not implemented, they can implement custom glTF extensions by referring to our official documentation. Also while the editor is not open, we have a small but beautiful glTF viewer on the official website, where developers can just drag and drop glTF models into the viewer to preview how they will be rendered in Oasis. The overall stability of glTF has been greatly improved after last year’s reconstruction, and its functions and expansibility have also been improved to some extent.

Animation

Animation is actually very important to the engine and is one of the core systems. We did a major redesign and feature addition to the animation system last year. First of all, we added animation transition, which is more natural when switching between two actions. If there is no animation transition, animation switch is the effect of flash cutting, which is relatively stiff.

The next step is animation mixing, which can complete the combination mode of animation. For example, the separation of upper body and lower body is common in FPS games. For example, the upper body can shoot, and the lower body can run or squat, which can be completed by animation mixing.

We also tweaked the animation events. It turns out that our animation events are separate from our scripting system. There is now a deep integration of animation events and scripting system Script. Oasis is clear and methodical in its architectural design to developers, using scripts for both animation logic and update logic.

In addition to the Animator redesign, we also made BlendShape (Morph Target) animations. The principle of BlendShape animation is to interpolate and modify model vertices directly. BlendShape animation is more violent than skin animation, but BlendShape animation has some unique advantages, such as the expression of detailed animation, such as facial expression animation.

BlendShape can also be driven by the Animator, which is more like a director and can drive any element, as well as materials and skeletons. Skin animation is implemented by Animator to drive the skeleton. Animator has been redesigned to improve the animation performance of the entire engine by 30%. Animation system in the ease of use, performance, function of three aspects have a certain improvement.

Physical system.

Added the physical function, in view of the current physics engine available on the Web have a certain shortage of the current situation, we have done a lot of back-end physical system design. For example, although PhysX has the most powerful function, the size of The WASM runtime library of PhysX is relatively large, resulting in a long loading time. Bullet’s WASM runtime is decent in size, but less powerful. Therefore, we have made a number of back-end physical system architectures, and implemented two sets of dual-backend physics-PhysX and lightweight physics-Lite. Developers can switch the back-end physics engine at will, and some interfaces called in the front end remain unchanged.

In terms of physical function components, we provide two types of Collider, a StaticCollider and a DynamicCollider. Physical events are also deeply integrated in scripts. In keeping with the animation event logic mentality mentioned above, the physical logic is also written using Script. Currently, the physical system only implements trigger mode and does not include physical feedback. Non-trigger mode will be added this year, allowing developers to easily and quickly complete fun interactions with physical feedback.

scenario

We also added new features to the scene, mainly Background and AmbientLight. Background currently provides three modes: color mode, texture mode, and Sky mode. The color mode is a solid color effect, and the texture mode uses a texture as the background, which is commonly used in some interactive marketing or model presentation scenes. Developers can set different stretching modes for the texture. Finally, sky, which has two properties of Mesh and Material, has high flexibility. Developers can customize sky rendering, such as atmospheric scattering algorithm to customize shader, etc.

Finally, AmbientLight. AmbientLight is part of the PBR rendering flow mentioned earlier, and we added diffuse spherical harmonic mode and cube specular reflection textures, which we’ve already covered and won’t repeat here.

2D

Oasis aims to be an integrated 2D/3D interactive engine, and we will continue to build 2D capabilities. We redesigned SpriteRenderer and Sprite, adding a SpriteMask. The new SpriteMask includes an inner mask and an outer mask. Sprite has custom shaders and added small features like flip flipping. We also refactored Sprite’s underlying rendering pipeline, resulting in a 2.1-fold performance improvement. Also added is the SpriteAlas asset, SpriteAtlas, an industry standard 2D optimization tool that reduces the DC of sprites.

2 d ecological

Oasis has always been open to some of the ecology of the industry. We redesigned SpineAnimation and added LottieAnimation. SpineAnimation is connected to the BufferMesh introduced before. Almost all GPU functions can be flexibly used in BufferMesh without excessive encapsulation, and the performance can be extremely extreme. After the redesign, the performance of SpineAnimation has been improved by 2.1 times, and the functions have been enhanced to some extent, including the support of a full set of skin replacement, single accessory replacement, and mixing and matching replacement. We also added LottieAnimation, which uses the Sprite capabilities of the engine for access.

Input

The Input system has been added. In the early days of Input interaction, the framework layer generally differentiated between Mouse and Touch. We adopted a modern design that unified the mouse and touch, abstracted the concept of Pointer. Both the mouse and the touch point are abstracted as a pointer point. Interactive logic editor in our Script provides onPointEnter/onPointDown onPointUp events, etc. For the description of the click area, we use physical components to work with it. To make clicking on a Box turn green, simply add a physical component in the shape of a Box and write code in the onPointDown script that changes the material to green.

website

After introducing the engine functions, we introduce the improvement of the official website. The official website supports both Chinese and English, including documents and cases. Oasis engine is aimed at global developers in the industry. Both the official website homepage and the tutorial document have adopted a new design. There are 67 new cases, and there will be some demonstration cases and teaching demo cases this year to get developers up and running with Oasis.

Documents and Cases

The API documentation and tutorial documentation have also been greatly improved and supplemented. You are welcome to study them. By the way, the yellow duck in this article is not a static image, but a real-time interactive application, and you can even modify the code in real time to see the effect, which is a small advantage of the web3D engine.

Open source construction

Oasis is a team that is serious about open source. We don’t just throw our code on Github. We do all our development and development management on Github. We’ve done two things with transparency. The first one is transparency in development management, moving from an annual Roadmap to specific milestones. We will list some key functions as longitudinal long-term tracking of the Project, and then make horizontal cutting according to the phased achievements, thus forming the development progress of Milestone and Milestone. We will use Wiki for public management.

The second is transparency in engine design. In addition to the source code, the engine design documentation is also made public. This year we also added Sponser channel, and all donations will go towards the open source development of the engine. We hope to attract outstanding developers in the industry to participate in the open source construction of Oasis through healthy open source technology management and construction. We are serious about open source.

Roadmap 2022

Before explaining the specific plans for 2022, let’s reiterate Oasis’s design philosophy:

  • First of all, mobile first, Web first, enabling the new form of front-end business, the terminal side of the device has gradually diversified, including mobile phones, VR/AR glasses, and intelligent vehicles, etc., the user side has gradually increased the demand for experience, fun and even temperature sense, front-end is the main force of our business. The front end needs a set of creative platform that can support the integrated development of UI, 3D, XR and other technologies and the collaborative development of business requirements.
  • Engine pursues balance between function, performance and ease of use. As an integrated interactive creation platform, excessive tilt of one side of engine will bring negative effects on the other side. Oasis pursues the ultimate comprehensive experience in engine design.
  • Editor of pursuing a minimalist workflow and human-computer interaction, side + cloud service integration, different from engine side, the editor is more focused on workflow, and human-computer interaction, hope that through minimalism and humanized workflow to help the business development team to promote the development and cooperation efficiency, at the same time, we will use the + cloud service combination way bring extreme experience for the development team.
  • Functions can be flexibly disassembled and combined, and the engine can be used independently. Both the component system architecture and the subcontracting mechanism of the engine have verified this concept. Oasis makes an interactive creation platform to realize the lightweight of the architecture in the way of flexible disassembly and combination of functions, rather than only doing the lightweight of functions in a single dimension.
  • Oasis is committed to creating a one-stop service creation platform. The required functions of interactive development can form a self-closed loop, and at the same time maintain the embrace industry ecosystem and open up the upstream and downstream workflow of the industry.

Next, let’s take a look at the key themes for 2022. There are seven major themes, which are as follows:

  • Continue to build 2D capabilities of the engine (text, GUI, etc.) and build an integrated 2D/3D engine.
  • Perfect physical features, dimension developers to provide rich physical effects.
  • Improved engine rendering capabilities (PBR material, light, shadow, baking, etc.) for better graphics.
  • Improve animation and develop special effects to provide dynamic creative ability for designers.
  • Add batch merge pipeline to improve performance.
  • Explore XR and other technologies to lay out future business forms.
  • The editor is open to the public.

Let’s start with the physics highlights:

  • Physical collider adds a non-trigger mode with physical feedback, which can quickly realize interaction requirements with physical feedback.
  • Add physical joints, such as hinges, elastic joints, fixed joints, etc.
  • Added character collider, which can quickly simulate character physics.

We split the render into three parts, PBR, Light and Shadow:

  • PBR adds quality grading to PBR algorithms of different complexity, allowing developers to tilt the balance between effect and performance. Clear Coat, Sheen and more will be added.
  • In the Light section, we will redesign the lighting API and rewrite the Light management. For example, forward+ supports a large number of points and focuses on the mobile terminal. Lightingmap and local reflective probe baking are also supported.
  • For the shadow part, we will rewrite the shadow rendering, such as CSSM. CSSM is a stable split shadow, which can solve the shadow rendering of large scenes.

The reconstruction of the dynamic part includes BlendAnimation, Animator and Effect:

  • BlendAnimation will increase the maximum number of animations that can be mixed. Currently, BlendAnimation only supports up to four animations. Multiple frames are supported. Currently, only one frame is supported.
  • Animator will add animation state machine, add some small feature of animation rewind.
  • The Effect side adds the TrailRenderer, which supports trailing effects, and the ParticleRenderer. ParticleRenderer is relatively weak at present and will be completely rewritten later. Both in effect and performance will improve.

Highlights of 2D section:

  • Support text rendering basic functions. There will also be a new GUI framework, which will support basic features like Canvas, text and button first, with other features gradually added.
  • Oasis will tap into WebXR to build XR capabilities and deeply integrate XR capabilities. Oasis’s mission as an engine vendor is to provide an integrated API that effectively lowers the barriers for front-end business development in 3D and XR.
  • On the Shader side, we will abstract the Shader framework to simplify the cost of writing the Shader. With the Shader framework, we can decouple the Shader from the script and write it in a separate file. The Shader framework will also support multiple passes, mixing states and demotion. This year it’s just shader framework, but in the future there will be cross-compilation of shaders and cross-platform support.

Focus on assets:

  • Oasis has done its own proprietary asset design for extremely small and fast files, as well as functional extensions that don’t affect the engine. Last year we finished designing private assets for textures, Mesh and textures. This year is more about editor side asset access. In addition, we will implement scenes and Prefab assets, which allow developers to switch between scenes, and Prefab allows developers to load prefabricated template models.
  • Performance is always a topic for 2D/3D engines, this year we will do batch optimization, which is a performance improvement from the GPU principle. Oasis’ previous performance improvements at milestones were mostly architectural and algorithmic gains. The first is to support static batch processing, improve the performance of scene static rendering objects. Second, it supports GPU instance processing to improve the discharge of dynamic similar objects in scenes. In fact, all of these are conventional optimization means of graphics engine. In the future, Oasis can use modern graphics API to do more in-depth optimization after accessing WebGPU.

Finally, there are some small feature and optimization improvements. The first is to simplify the texture structure to facilitate the addition of Texture3D, TextureXXArray and other texture types. Input Interaction Last year we did Pointer interaction and this year we will support keyboard interaction. Next, optimize the Transform API to reduce the cost of using properties like Position. Then there are optimizations like adding sound components and simplifying the Engine’s initialization API.

Editor open plan

Lowering the threshold of front-end interactive business development requires a modern interactive workflow which cannot be separated from visual editing and clear division of responsibilities. We should define and recommend workflow guidance according to the actual business characteristics and team size of different companies. We need to think about how the roles of programmer, designer, product planner and so on can work together effectively and get the best out of each of them, rather than putting all the pressure on the program, logic development is just one part of the interactive business. In addition to the engine itself, the modern interactive workflow needs an excellent supporting editor to connect the division of labor and efficient development.

In December 2022, we will open the 1.0 registry for the editor, which will focus on stable base features, including 2D/3D scene editing, scripting, and project build previews. In 2023 to 2024, we will gradually open up animation editing, GUI editing, scene baking, multiplayer collaboration, asset and plug-in markets and desktop applications.

Thank you for listening. That’s all for today’s sharing.

The last

If you want to see the full video, see Oasis Anniversary Review and Future plans