The introduction

Recently, for t-level interaction, WE need to use 3D models. I believe that you, like me, will have some questions when you start:

  • 1. How to select the export format of 3D model
  • 2. How to optimize the model file
  • 3. What about compatibility in high-traffic projects

Let’s make a detailed exploration, investigation and precipitation through this article.


What is a glTF file

GlTF, Graphics Language Transmission Format, is a standard file Format for 3D scenes and models.

The glTF core is a JSON file that describes the entire content of a 3D scene. It consists of a description of the scene structure itself, provided by the hierarchy of nodes that define the scene diagram.

The 3D objects present in the scene are defined using meshes(grids) connected to the nodes. Materials define the appearance of an object. Animations describe how 3D objects transform 3D objects over time, and Skins define the way the geometry of the objects is deformed based on skeleton poses. Cameras describe the view configuration of the renderer.

In addition, it includes links with binary data and image files, as shown in the figure below.


2..GLTF and.glB

As you can see from blender file export:

GlTF files come in two extended forms,.gltf (JSON/ASCII) or.glb (binary). A.gltf file may be self-contained or reference external binary and texture resources, whereas a.glb file is completely self-contained (but using external tools you can save its buffer/texture as an embedded or separate file, as described later).

2.1. Cause of GLB file

GlTF provides two delivery options that can also be used together:

  • GlTF JSON points to external binary data (geometry, keyframes, skins) and images.
  • GlTF JSON embed the Base64-encoded binary data and use the data URI to inline the image.

For these resources, glTF requires separate requests or additional space due to Base64 encoding. Base64 encoding requires additional processing to decode and increase the file size (approximately 33% increase in encoding resources). While Gzip mitigated the increase in file size, decompression and decoding still added significant load times.

To solve this problem, a container format, Binary glTF, is introduced. In binary glTF, glTF assets (JSON,.bin, and images) can be stored in binary blobs, which are.glb files.

2.2 File Comparison

2.2.1 The same glTF file,.glb format is smaller than.gltf

  • Self-contained:

  • References to external binary and texture resources:

2.2.2. GLTF file Preview:

  • Self-contained:

  • References to external binary and texture resources:

2.2.3 GLB File Preview:

  • Self-contained:

  • References to external binary and texture resources:

As you can see from the figure, when not self-contained, the glTF file is requested along with the image file.

So, we can take advantage of this feature, and we can achieve some performance optimizations, so let’s move on.


Three, glTF file split

As mentioned above, the glTF file can be split into.gltf/.glb file + binary file + texture image, so we can split it out and compress the texture image separately to optimize the performance.

GLTF pipeLine can be used, which has the following functions:

  • GlTF and GLB conversion
  • Save the buffer/texture as an embedded or separate file
  • Convert the glTF 1.0 model to glTF 2.0(usingKHR_techniques_webglandKHR_blend)
  • Use Draco for grid compression

In this case, we are going to use the “save the buffer/texture as an embedded or separate file” feature.

Let’s take a look at the split file

Again, this is how the.glb file introduces external separate textures and binaries

So, just put the split files in the same path and import them like we did before.

  • Compression way
gltf-pipeline -i male.glb -o male-processed.glb -s
Copy the code
  • Usage (in three.js)

Just use it in a normal way, no different than unsplit, right

import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const loader = new GLTFLoader()
loader.load(MODEL_FILE_PATH, (gltf) = > {
 / /...
})
Copy the code
  • The performance comparison


Four, glTF file compression

As described above, glTF files include.gltf/.glb files,.bin files, and texture resources. GlTF2.0 related plug-ins are as follows:

So let’s take some of them and analyze them.


4.1 Mesh Compression

4.1.1 KHR_draco_mesh_compression

One of the most common mesh compression methods, using the open source Draco algorithm, is used to compress and discompress 3D mesh and point cloud, and may change the order and number of vertices in the mesh. Compressed makes the file much smaller, but requires additional decoding time on the client device.

  • Compression way

It can be compressed using the GLtf-pipelineGltf file optimization tool

gltf-pipeline -i male.glb -o male-processed.glb -d
Copy the code
  • Usage (in three.js)
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'
import { DRACOLoader } from 'three/examples/jsm/loaders/DRACOLoader'

const loader = new GLTFLoader()

// Create a decoder instance
const dracoLoader = new DRACOLoader()
// Set the decompression library file path
dracoLoader.setDecoderPath(DECODER_PATH)
// Load the decoder instance
loader.setDRACOLoader(dracoLoader)

loader.load(MODEL_FILE_PATH, (gltf) = > {
 / /...
})
Copy the code
  • Performance analysis and comparison

The original size of this GLB file is 3.2m, and the Draco compression is 1.8m, which is about 56% of the original file.

As you can see from the above code, creating the decoder instance requires the introduction of additional libraries for decoding, and setDecoderPath automatically requests the WASM file to do the decrypting. These two WASM files also increase the request time and the number of requests, so with these two files, the true compression rate is about 62.5%.

So, if a project needs to load multiple glTF files, you can create a DRACOLoader instance and reuse it. However, if the project only needs to load a glTF file, it is worth considering whether the Draco algorithm is “cost-effective”.

Use demo to compare performance:

It can be seen that the first loading and decryption time of Draco algorithm is longer than that of the original file. In real projects, the gap is even more pronounced, and occasionally a decrypt jam will occur, requiring re-entry to restore functionality.

In addition, there is a very intuitive problem, model quality loss is considerable.

Here’s what it looks like in the iPhone 12 and mi MIX2, respectively:

In a word, if Draco compression algorithm is to be applied to large-scale projects, the following comparison should be made in combination with actual projects:

  • (1) Request two files + decryption time, compared with its own GLB file compressed volume size, real performance comparison;
  • (2) Whether there will be loss of picture quality that the designer cannot accept.


4.1.2 KHR_mesh_quantization

Vertex attributes are usually stored using FLOAT type, converting the original initial floating point value into 16-bit or 8-bit storage to fit a uniform 3D or 2D grid. This is called quantization, and the plug-in is mainly used to vectify it.

For example, a static Pbr-ready grid typically requires each vertex POSITION (12 bytes), TEXCOORD (8 bytes), NORMAL (12 bytes), and TANGENT (16 bytes), for a total of 48 bytes. With this extension, you can use SHORT to store location and texture coordinate data (8 and 4 bytes respectively) and BYTE to store normal and tangent data (4 bytes each) for a total of 20 bytes per vertex.

  • Compression way

You can use the GlTFPack tool for compression

gltfpack -i male.glb -o male-processed.glb
Copy the code
  • Usage (in three.js)

Just use it in a normal way, no different than uncompressed, right

import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const loader = new GLTFLoader()
loader.load(MODEL_FILE_PATH, (gltf) = > {
 / /...
})
Copy the code
  • The performance comparison

The original file is 3.2m, and the compressed file is 1.9m, which is 59.3% of the original file, and the loading speed is much faster than the original model. In real projects, there is no loss of quality and long load times.

4.1.3 EXT_meshopt_compression

This plug-in assumes that the bufferView data is optimized for GPU efficiency — using quantization and GPU rendering with the best data order — and provides a compression layer on top of the bufferView data. Each bufferView is compressed independently, which allows the loader to extract the data directly into the GPU storage to the maximum extent possible.

In addition to optimizing the compression rate, the compression format has two features — very fast decoding (using WebAssembly SIMD, the decoder runs at about 1 GB/ SEC on modern desktop hardware), and byte storage compatible with general compression. That is, rather than reducing the encoding size as much as possible, the bitstream is built in such a way that a universal compressor can compress it further.

  • Compression way

You can use the GlTFPack tool for compression

gltfpack -i male.glb -o male-processed.glb -cc
Copy the code
  • Usage (in three.js)
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'
import { MeshoptDecoder } from 'three/examples/jsm/libs/meshopt_decoder.module.js'

const loader = new GLTFLoader()
loader.setMeshoptDecoder(MeshoptDecoder)
loader.load(MODEL_FILE_PATH, (gltf) = > {
 / /...
})
Copy the code
  • Performance analysis and comparison

The original file is 3.2m and the compressed file is 1.1m, which is 65.6% of the original file. The first loading time is much faster than the original model. In real projects, there is no loss of quality and long load times.

Five, multiple models of equipment and optimization comparison results

In order to avoid the aforementioned “Draco” compression which causes the model to be damaged, I found a few iPhone and Android phones to conduct a performance and compatibility test. Let’s see the results. PS: The company network in different time period network speed (such as morning and afternoon), may have a small impact on the number, but does not affect the file optimization horizontal comparison.

IPhone 12 (iOS 14.4, personal)

Huawei Mate 40 Pro (HarmonyOS, personal)

Xiaomi Mix2 (Android 8.0, beta)

IPhone 6SP (iOS 13.7, personal)

5.1 summarize

It can be seen that KHR_mesh_quantization or EXT_meshopt_compression is adopted for a small number of businesses that need to use models and only need to load one model, and then GLTF-PIPELINE is used for module differentiation and texture image compression. It is a better optimization scheme found at present.


Sixth, other

In fact, there are many performance optimized plug-ins, which are currently under debugging and investigation, and will be updated in the next iteration or if there is any new progress:

Mesh-optimized:

  • EXT_mesh_gpu_instancing

    Now Three js GLTFLoader yet support, Babylon. Js Babylon. GLTF2. Loader. Extensions support

There are also some plugins for texture optimization:

  • KHR_texture_basisu

  • EXT_texture_webp

Vii. Reference materials

  1. The Basic Structure of glTF

  2. GLB File Format Specification

  3. Extensions for glTF 2.0

  4. KHR_draco_mesh_compression

  5. DRACOLoader – three. Js docs

  6. CesiumGS/gltf-pipeline: Content pipeline tools for optimizing glTF assets.

  7. KHR_mesh_quantization

  8. 📦 gltfpack | meshoptimizer

  9. GLTFLoader

  10. EXT_meshopt_compression

  11. [Grid Compression evaluation] MeshQuan, MeshOpt, Draco