WebAssembly que

This article is based on WebAssembly’s exploration and research. Recently, I need to do a project related to encryption. I want to put the back-end encryption scheme directly into the front end for use. The advantage is that the encryption scheme code only needs to maintain one set, and the back-end scheme is closer to the bottom of the system, so it should be able to get better performance. Just happened to find WebAssembly, which was designed with the goal of portability in mind, meeting that need.

I have recorded all the problems I encountered in the process of researching WebAssembly and can share them with you in the future. At the end of this article, I have placed a reference article for you to read. This article is the first in a series that covers The basics of WebAssembly and its use.

An overview of the

The birth of WebAssembly
What is WebAssembly?
MAC installation Emscripten
WebAssembly is simple to use and analyze
conclusion

The birth of WebAssembly

When people say WebAssembly is faster, it’s generally compared to JavaScript.

JavaScript, introduced in 1995, wasn’t designed to be fast to execute, and for the first 10 years it wasn’t. Then the browser market began to compete fiercely. The much-publicized “performance wars” began in 2008. Many browsers have introduced the just-in-time compiler, or JIT. Based on the JIT model, JavaScript code runs faster over time. Thanks to the introduction of jIts, JavaScript performance reached a tipping point, with JS code executing 10 times faster.

As performance improves, JavaScript can be used in areas that were never thought of before, such as Node.js for back-end development. Performance improvements have greatly expanded the range of JavaScript applications.

But this is also slowly exposing JavaScript problems:

The syntax is too flexible to develop large Web projects;
The performance cannot meet the requirements of some scenarios.

In view of the above two defects, some ALTERNATIVE JS languages have emerged in recent years, such as:

Microsoft TypeScript improves the looser syntax of JS by adding static type checking to improve code robustness.
Google’s Dart introduces a new VIRTUAL machine for the browser to run the Dart application directly to improve performance.
Firefox asm.js is a subset of JS, and the JS engine is optimized for asm.js.

The above attempts have their own advantages and disadvantages, among which:

TypeScript only solves the problem of loose JS syntax, and it still needs to be compiled into JS to run without improving performance.
Dart only works in Chrome preview and is not supported by major browsers.
Asm.js syntax is too simple, very restrictive, and inefficient to develop.

The three browser giants came up with their own incompatible solutions, which went against the purpose of the Web; It is the standardization unification of technology that makes Web come to today, so it is urgent to form a new specification to solve the problems faced by JS.

Thus was born WebAssembly, a new bytecode format already supported by major browsers. Unlike JS interpretation execution, WebAssembly bytecode is similar to the underlying machine code and can be loaded and run quickly, thus greatly improving performance compared to JS interpretation execution. In other words, WebAssembly is not a programming language, but a bytecode standard. It needs to be compiled in a high-level programming language and put into a WebAssembly VIRTUAL machine to run. Browser manufacturers need to implement the virtual machine according to the WebAssembly specification.

What is WebAssembly?

WebAssembly(Wasm for short) is a binary instruction format based on the stack virtual machine. Designed with a portable goal in mind, Wasm can be used to compile high-level languages such as C/C+/RUST to enable client and server applications to be deployed on the Web.

The above paragraph is from the official definition.

An Assembly language for the Web, which performs low-level binary syntax through the Web. Instead of using assembly language directly, WebAssembly provides a catch-and-trade mechanism (LLVM IR) that compiles high-level languages (C, C++, and Rust) into WebAssembly so that it has a chance to run in a browser. You can see that it is actually a working mechanism, a new bytecode format (.wASM), not a new language.

Install Emscripten for MAC

If you want to compile a C/C++ program into a. Wasm file, you need a compilation tool to do this. The WebAssembly community recommends common tools:

Emscripten: can convert C and C++ code into WASM, ASm.js;
Binaryen: Provides more concise IR, converts IR to WASM, and provides wASM compile-time optimization, WASM virtual machine, WASM compression, etc.

1. Environment dependence

Git
CMake
brew install cmake
Python 2.7.x or later, installed by default

2. Compile Emscripten

Next, you need to compile an Emscripten yourself from the source code. Run the following commands to use the Emscripten SDK automatically.

Git clone https://github.com/juj/emsdk.git CD emsdk # compile the source code. / emsdk install SDK. The latest # activation/emsdk activate the latest Set the environment variable source. /emsdk_env.shCopy the code

When running the preceding command, the following problems may occur:

/emsdk install latest error:

likai@likaideMacBook-Pro:~/resource/emsdk$ ./emsdk install latest Installing SDK 'sdk-releases-upstream-7a7f38ca19da152d4cd6da4776921a0f1e3f3e3f-64bit'.. Installing tool 'node - 12.18.1-64 - bit.. Error: Downloading URL 'https://storage.googleapis.com/webassembly/emscripten-releases-builds/deps/node-v12.18.1-darwin-x64.tar.gz' : <urlopen error unknown url type: https> Warning: Possibly SSL/TLS issue. Update or install Python SSL root certificates (2048-bit or greater) supplied in Python folder or https://pypi.org/project/certifi/ and try again. Installation failed!Copy the code

Solutions:

This command calls the emsdk.py file, so use./emsdk.py install latest.

likai@likaideMacBook-Pro:~/resource/emsdk$ ./emsdk.py install latest Installing SDK 'sdk-releases-upstream-7a7f38ca19da152d4cd6da4776921a0f1e3f3e3f-64bit'.. Installing tool 'node - 12.18.1-64 - bit.. Downloading: / Users/likai/hisun/resource/emsdk/zips/node - v12.18.1 - Darwin - x64. Tar. Gz the from https://storage.googleapis.com/webassembly/emscripten-releases-builds/deps/node-v12.18.1-darwin-x64.tar.gz, 20873670 Bytes Unpacking '/ Users/likai hisun/resource/emsdk/zips/node - v12.18.1 - Darwin - x64. Tar. Gz' to '/ Users/likai hisun/resource/emsdk/node / 12.18.1 _64bit' Done installing tool 'node - 12.18.1-64 - bit. Installing tool 'python - 3.7.4-2-64 - bit.. Downloading: / Users/likai/hisun/resource/emsdk/zips/python - 3.7.4-2 - macos. Tar. Gz the from https://storage.googleapis.com/webassembly/emscripten-releases-builds/deps/python-3.7.4-2-macos.tar.gz, 25365593 Bytes Unpacking '/ Users/likai/hisun/resource/emsdk/zips/python - 3.7.4-2 - macos. Tar. Gz' to '/ Users/likai/hisun/resource/emsdk/python / 3.7.4-2 _64bit' Done installing tool 'python - 3.7.4-2-64 - bit. Installing tool 'releases-upstream-7a7f38ca19da152d4cd6da4776921a0f1e3f3e3f-64bit'.. Downloading: /Users/likai/hisun/resource/emsdk/zips/7a7f38ca19da152d4cd6da4776921a0f1e3f3e3f-wasm-binaries.tbz2 from https://storage.googleapis.com/webassembly/emscripten-releases-builds/mac/7a7f38ca19da152d4cd6da4776921a0f1e3f3e3f/wasm- binaries.tbz2, 69799761 Bytes Unpacking '/Users/likai/hisun/resource/emsdk/zips/7a7f38ca19da152d4cd6da4776921a0f1e3f3e3f-wasm-binaries.tbz2' to '/Users/likai/hisun/resource/emsdk/upstream' Done installing tool 'releases-upstream-7a7f38ca19da152d4cd6da4776921a0f1e3f3e3f-64bit'. Running post-install step: npm ci ... Done running: npm ci Done installing SDK 'sdk-releases-upstream-7a7f38ca19da152d4cd6da4776921a0f1e3f3e3f-64bit'.Copy the code

Also activate Emscripten with./emsdk.py activate latest

likai@likaideMacBook-Pro:~/resource/emsdk$ ./emsdk.py activate latest Setting the following tools as active: Node - 12.18.1-64 - bit python - 3.7.4-2-64 - bit releases - upstream - 7 a7f38ca19da152d4cd6da4776921a0f1e3f3e3f - 64 - bit Next steps: - To conveniently access emsdk tools from the command line, consider adding the following directories to your PATH: / Users/likai hisun/resource/emsdk/Users/likai/hisun/resource/emsdk/node / 12.18.1 _64bit/bin / Users/likai/hisun/resource/emsdk/python / 3.7.4-2 _64bit/bin/Users/likai hisun/resource/emsdk/upstream/emscripten - This can be done for the current shell by running: source "/Users/likai/hisun/resource/emsdk/emsdk_env.sh" - Configure emsdk in your bash profile by running: echo 'source "/Users/likai/hisun/resource/emsdk/emsdk_env.sh"' >> $HOME/.bash_profileCopy the code

source ./emsdk_env.sh

likai@likaideMacBook-Pro:~/resource/emsdk$ source ./emsdk_env.sh Adding directories to PATH: PATH += /Users/likai/hisun/resource/emsdk PATH += /Users/likai/hisun/resource/emsdk/upstream/emscripten PATH += / Users/likai/hisun/resource/emsdk/node / 12.18.1 _64bit/bin PATH + = / Users/likai/hisun/resource/emsdk/python / 3.7.4-2 _64bit/bin Setting environment variables: EMSDK = /Users/likai/hisun/resource/emsdk EM_CONFIG = /Users/likai/hisun/resource/emsdk/.emscripten EM_CACHE = /Users/likai/hisun/resource/emsdk/upstream/emscripten/cache EMSDK_NODE = / Users/likai/hisun/resource/emsdk/node / 12.18.1 _64bit/bin/node EMSDK_PYTHON = / Users/likai/hisun/resource/emsdk/python / 3.7.4-2 _64bit/bin/python3Copy the code

3. Verify

Emcc-v succeeds without reporting errors

likai@likaideMacBook-Pro:~/resource/emsdk$ emcc -v emcc (Emscripten gcc/clang-like replacement + linker emulating GNU Ld) 2.0.3 clang version 12.0.0 (/ b/s/w/ir/cache/git/chromium.googlesource.com-external-github.com - LLVM - LLVM - project A39423084cbbeb59e81002e741190dccf08b5c82) Target: x86_64 - apple - darwin19.4.0 Thread model: posix InstalledDir: /Users/likai/hisun/resource/emsdk/upstream/bin shared:INFO: (Emscripten: Running sanity checks)Copy the code

Get help emCC –help, too much content will not show.

The emCC version is 2.0.3

likai@likaideMacBook-Pro:~/resource/emsdk$  emcc --version

emcc (Emscripten gcc/clang-like replacement) 2.0.3 (43fcfd2938b72c57373a910ece897b27aa298852)
Copyright (C) 2014 the Emscripten authors (see AUTHORS.txt)
This is free and open source software under the MIT license.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
		
Copy the code

Simple use and analysis of WebAssembly

Now that the WebAssembly compiler is installed, let’s use two official examples to see how WebAssembly works.

There are many different options when compiling using Emscripten, and we’ll cover two of the main ones:

Compile into WASM and generate an HTML to run our code, adding all the “glue” JavaScript code that WASM needs to run in a Web environment.
Compile to WASM and use JavaScript to call methods in WASM.

Generate HTML and JavaScript

Find a directory to create the hello_world.c file

#include <stdio.h>

int main(int argc, char ** argv) {
  printf("Hello World\n");
}

Copy the code

Using the terminal you just configured, locate the hello_world.c file and execute the following command
```
emcc ./hello_world.c -s WASM=1 -o ./hello_world.html

Copy the code
```
- Emcc is an Emscripten compiler line command
- Hello_world.c is our input file
- -s WASM=1 specifies the WASM output form we want. If we don’t specify this option, Emscripten will only generate ASM.js by default. (see emCC –help parameter description)
- -o./hello_world.html Specifies that this option will generate an HTML page to run our code, and will generate the WASM module, along with the “glue” JS code needed to compile and instantiate the WASM module, so that we can use it directly in the Web environment.
```
likai@likaideMacBook-Pro:~/resource/emsdk/demo$ emcc ./hello_world.c -s WASM=1 -o ./hello_world.html
shared:INFO: (Emscripten: Running sanity checks)

likai@likaideMacBook-Pro:~/resource/emsdk/demo$ ls
hello_world.c    hello_world.html hello_world.js   hello_world.wasm
Copy the code
```
Three new files are generated after execution:
- Hello_world.wasm binary wASM module code, although the local open, but the browser can help translate.
- Hello_world.js is a JavaScript file that contains glue code to convert between native C functions and JavaScript/ wASM
- Hello_world.html An HTML file used to load, compile, instantiate your WASM code and output it to the browser display
Start the HTTP command and view the result

emrun –no_browser –port 8080 ./hello_world.html
```
likai@likaideMacBook-Pro:~/resource/emsdk/demo$ emrun --no_browser --port 8080 ./hello_world.html Web server root Directory: / Users/likai hisun/resource/emsdk/demo Now listening at http://0.0.0.0:8080/Copy the code
```
- The emrun command is also available in the EMSDK.
You can see that what was printed in the original helloWorld.c file is now in the browser. I wonder how the printout in C code ends up on the browser console. Emscripten does a lot of work for this seemingly simple operation. Click on the hello_world.js code to generate glue, and there are 2000 + lines of code in it, loading wASM, handling memory allocation, memory freeing, garbage collection, function calls, and encapsulating various methods. I put the compiled JS file in gihub, click to view hello_world.js and briefly analyze the content of the glue code, which will help us understand WebAssembly and will be very helpful for later use.

Wasm is a binary file that can not be opened. If you want to see the contents of wASM, you can also use the decompression tool WASm2Wast. Right-click the console –>Sources–>hello_world.wasm

“Hello_world. C”, “main”, “hello_world. C”, “main”, “hello_world.

The glue code hello_world.js is used to load the WebAssembly module (.wasm).

function instantiateArrayBuffer(receiver) { return getBinaryPromise().then(function(binary) { return WebAssembly.instantiate(binary, info); }).then(receiver, function(reason) { err('failed to asynchronously prepare wasm: ' + reason); abort(reason); }); } // Prefer streaming instantiation if available. function instantiateAsync() { if (! wasmBinary && typeof WebAssembly.instantiateStreaming === 'function' && ! isDataURI(wasmBinaryFile) && // Don't use streaming for file:// delivered objects in a webview, fetch them synchronously. ! isFileURI(wasmBinaryFile) && typeof fetch === 'function') { fetch(wasmBinaryFile, { credentials: 'same-origin' }).then(function (response) { var result = WebAssembly.instantiateStreaming(response, info); return result.then(receiveInstantiatedSource, function(reason) { // We expect the most common failure cause to be a bad MIME type for the binary, // in which case falling back to ArrayBuffer instantiation should work. err('wasm streaming compile failed: ' + reason); err('falling back to ArrayBuffer instantiation'); return instantiateArrayBuffer(receiveInstantiatedSource); }); }); } else { return instantiateArrayBuffer(receiveInstantiatedSource); }}Copy the code

We mainly did the following things:

Try to use WebAssembly. InstantiateStreaming () method to create wasm module instance;
If the stream creation fails, create the instance instead with the webassembly.instantiate () method;
After successful instantiation return values to the receiveInstantiatedSource () method.

ReceiveInstantiatedSource () code

function receiveInstance(instance, module) { var exports = instance.exports; Module['asm'] = exports; removeRunDependency('wasm-instantiate'); }... function receiveInstantiatedSource(output) { // 'output' is a WebAssemblyInstantiatedSource object which has both the module and instance. // receiveInstance() will swap in the exports (to Module.asm) so they can be called assert(Module === trueModule, 'the Module object should not be replaced during async compilation - perhaps the order of HTML elements is wrong? '); trueModule = null; // TODO: Due to Closure regression https://github.com/google/closure-compiler/issues/3193, the above line no longer optimizes out down to the following line. // When the regression is fixed, can restore the above USE_PTHREADS-enabled path. receiveInstance(output['instance']); }Copy the code

ReceiveInstantiatedSource () method calls the receiveInstance () method, the latter this instruction:

Module['asm'] = exports;
Copy the code

Pass the exported object of the WASM Module instance to asm, a child object of the Module. If we manually add the code to print the instance export object in the above function.

function receiveInstance(instance, module) { ... . Module['asm'] = exports; console.log(Module['asm']); //print instance.exports ... .Copy the code

As you can see, Module[‘ ASM ‘] holds the exported object of the WebAssembly instance — the exported function is the main entry point for external calls to the WebAssembly instance.

Let’s see if I understand this correctly, the WASM compiler compilers the C code, the WASM file, which is assembly code, contains the content of the C code, glue the code to load, the WASM file, provides the methods in the C code through the WebAssembly instance, and then uses javascript to call the C code. Finally, it feels like you can run C programs on your browser.

Let’s take a look at the official quote (translated) :

WebAssembly(Wasm for short) is a binary instruction format based on the stack virtual machine. Designed with a portable goal in mind, Wasm can be used to compile high-level languages such as C/C+/RUST to enable client and server applications to be deployed on the Web.Copy the code

Wasm is a stack virtual machine based binary instruction format, hello_world. Wasm local open is a binary instruction format.
It can be used to compile high level languages such as C/C+/RUST and hello_world.c files with Emscripten.
Enable client and server applications to be deployed on the Web. It did run in the browser.
Wasm is designed with a portable goal in mind. If so, I can compile the encryption tool into WASM and use the glue code to call it. Let’s work on that in the next article.

2. Compile to WASM and use JavaScript to call wASM methods.

This is easy to understand. At compile time, do not generate the default recommended HTML, just generate WASM, and then call WASM directly. This requires us to write our own glue code. Here is a simple example. The steps are as follows:

(1) Write a test.c file where you can add, subtract, multiply and divide.

(2) Compile to. Wasm file

(3) Write an HTML file and call the. Wasm file

Test. C file

char* toChar (char* str) {
  return str;

}

int add (int x, int y) {
  return x + y;

}

int square (int x) {
  return x * x;

}

Copy the code

Compile to a. Wasm file
```
emcc ./test.c -Os -s WASM=1 -s SIDE_MODULE=1 -o ./test.wasm

Copy the code
```
This command appears to be different from the above command.
- Emcc stands for Emscripten compiler,
- Test.c is our input file
- The Os indicates that this compilation needs to be optimized (optimization strategies can be specified. emcc –help)
- -s WASM=1 indicates the output of WASM files, because the default is to output ASM.js
- -s SIDE_MODULE=1 means only one module, don’t give me any other code
- -o test.wasm is our output file.

Write an HTML file and call the.wasm file. Test.html these two functions are key:

function loadWebAssembly (path, Imports = {}) {return fetch(path) // load file.then (response => response.arrayBuffer()) // convert to arrayBuffer. Then (buffer => WebAssembly.com running (buffer)). Then (module = > {imports. The env = imports. The env | | {} / / open the memory space imports. The env. MemoryBase = imports.env.memoryBase || 0 if (! imports.env.memory) { imports.env.memory = new WebAssembly.Memory({ initial: 256})} / / create a variable mapping table imports. The env. TableBase = imports. The env. TableBase | | 0 if (! Imports. Env. table) {// In the MVP version element can only be "anyFunc" imports. 0, element: 'anyfunc'})} return new WebAssembly.Instance(module, Imports)})} // Load wasm file loadWebAssembly('test.wasm'). Then (instance => {// call c const toChar = instance.exports.toChar const add = instance.exports.add const square = instance.exports.square console.log('return: ', toChar("12352324")) console.log('10 + 20 =', add(10, 20)) console.log('3*3 =', square(3)) console.log('(2 + 5)*2 =', square(add(2 + 5))) })Copy the code

Now that you have a sense of what this means, you create an instance of WebAssembly, return the WebAssembly export object, and call the test.c function. There’s a little bit of glue code syntax in there. MDN Web docs-WebAssembly

The results

test.wasm

As you can see from the optimized WASM file, these are the only functions left, and you can see that the functions from the test.c export are included.

Five, the summary

We are going to talk about WebAssembly today with two simple examples to further understand what WebAssembly is. The overall process looks like this:

Use Emscripten to compile C source code, generate.wasm files and glue code, and use javascript to call glue code or.wasm to make C programs run in the browser.

That’s all for this article. Next, WASM-based encryption tools.

This article refer to

Webassembly official website

MDN Web docs-WebAssembly

The original in Chinese

Netwarps is composed of a senior cloud computing and distributed technology development team in China, which has rich experience in the financial, power, communication and Internet industries. Netwarps has set up research and development centers in Shenzhen and Beijing, with a team size of 30+, most of which are technical personnel with more than 10 years of development experience, respectively from the Internet, finance, cloud computing, blockchain and scientific research institutions and other professional fields. Netwarps focuses on the development and application of secure storage technology products, the main products are decentralized file system (DFS), decentralized computing platform (DCP), is committed to providing distributed storage and distributed computing platform based on decentralized network technology, with high availability, low power consumption and low network technical characteristics. Applicable to scenarios such as the Internet of Things and industrial Internet. Public account: Netwarps