Developing front-end Applications with Go (3)

preface

While writing the first article in this series, some of you may not know what scenarios WASM can be used in. Although it was mentioned in the comments, there are no specific examples. This article uses a specific example to demonstrate how WASM can be used. This article will mainly talk about the following aspects:

An introduction to MD5
Use Go to calculate file MD5
Use js to calculate the MD5 of the file
The performance comparison
What is the implementation of file second transmission?

A brief introduction to MD5

Md5 simple said is that it is a kind of hash algorithm, in the past there are a lot of sites or system, are using md5 encryption, md5 is characterized by, arbitrary length of the input, it can give you to generate a 128 – bit result come out, and as long as the input, output the result must be different, now it is said that there will be a hash collision, But we won’t discuss it here. 128 bits is 32 bits in hexadecimal notation. However, in today’s system, it is no longer possible to use MD5 to encrypt passwords, because MD5 can be cracked using today’s hardware or machines.

For the details of the md5 algorithm can refer to: zh.wikipedia.org/wiki/MD5

File MD5 obtaining tool

On a MAC, you can run the MD5 command to obtain the MD5 value of a specified file:

Md5 File PathCopy the code

On Linux, you can use the md5sum command to obtain:

Md5sum File pathCopy the code

Use Go to calculate file MD5

Here’s how to calculate the MD5 value of a file in Go:

func main() { s := time.Now(); F, err := os.open ("1.mp4")
	iferr ! = nil {log.fatal (err)} defer f.close () // instantiate a Hash object h := md5.new ()if_, err := io.Copy(h, f); err ! = nil { log.Fatal(err) } e := time.Now(); fmt.Println(e.Sub(s)); // Computes, and prints the MD5 result of the given file fmt.println (fmt.s)printf("%x",h.Sum(nil)))
	
}
Copy the code

Directory structure:

Running results:

In the above code, we calculated the MD5 value of the 1.mp4 file and printed the calculation time of the MD5 value, which can be seen as 55ms. Take a look at the size of this file:

The file size is 37M.

Here is actually a Go calculation file MD5 demo, later will be used to compile into WASM, front-end interaction. I’m going to stop here for the moment.

Use js to calculate the MD5 of the file

As we know, JS can also calculate the MD5 value of a file. Here we use a mature open source library as an example.

The github address for the open source library is github.com/satazor/js-…

But here, for convenience, we directly use the library on the CDN:

<! DOCTYPE html> <html lang="en">
<body>
    <form method="POST" enctype="multipart/form-data" onsubmit="return false;"><input id="file" type="file" placeholder="select a file"></form>
</body>
<script src="//cdn.rawgit.com/satazor/SparkMD5/master/spark-md5.min.js"></script>
<script>
    var log=document.getElementById("log");
    document.getElementById("file").addEventListener("change".functionConst s = date.now (); var blobSlice = File.prototype.slice || File.prototype.mozSlice || File.prototype.webkitSlice, file = this.files[0], chunkSize = 2097152, //read in chunks of 2MB
            chunks = Math.ceil(file.size / chunkSize),
            currentChunk = 0,
            spark = new SparkMD5.ArrayBuffer(),
            frOnload = function(e){
                spark.append(e.target.result); // append array buffer
                currentChunk++;
                if (currentChunk < chunks)
                    loadNext();
                else{// There is no next chunk, console.log('time:',  Date.now() - s);
                }
                    
            },
            frOnerror = function() {};function loadNext() {
            var fileReader = new FileReader();
            fileReader.onload = frOnload;
            fileReader.onerror = frOnerror;
            var start = currentChunk * chunkSize,
                end = ((start + chunkSize) >= file.size) ? file.size : start + chunkSize;
            fileReader.readAsArrayBuffer(blobSlice.call(file, start, end));
        };
        loadNext();
    });
</script>
</html>
Copy the code

This example is actually from the open source project, I changed it, added a time-consuming output, removed other hints, the official demo address is: 9px. Ir /demo/increm…

Using the above code, we can also calculate the MD5 of the 1.mp4 file.

From the point of view of time, we found that using JS to calculate the MD5 value of the file takes about 8 times longer than using Go to calculate the MD5 value of the file. This is definitely unacceptable for C-side applications. So, that’s why I need to use WASM.

What is the implementation of file second transmission?

If we have used Baidu network disk or other similar network disk, there must be a second pass such a function. That net disk second transmission function is how to achieve it? Is it really fast upload speed, to achieve it? Definitely not, because maybe you upload a file of tens of gigabytes, can also achieve second transmission.

Pass from the implementation of the actually very simple, is to use the file md5 to do with the cloud file md5 contrast, if the same, you want to upload this file, the cloud already exists, so this time, there is no need to upload the logo directly upload finish line, later if you need to download, will offer you the cloud files to download. Isn’t that easy?

Let’s do an example, use Go to calculate the MD5 of the file, select the file through the front page, and then give the file to WASM compiled by Go to calculate, after the calculation, return to JS for use. Note that there is no file to achieve a pass of complete function, and because of the need to interact with the server, and get back to the md5, sent to the server, the server check if the file exists in the clouds, these steps, the article said no longer, because the back of the content, here at the front, as if nothing can go again. If you have any special questions, you can ask me again.

Rough implementation process

After simply drawing a sketch, the JS side selects the file, and then passes the file to the Go side. After the Go side calculates the MD5 value of the file, it returns the file to the JS side. After the JS side gets the result, it can do whatever it wants.

Js code implementation

<! DOCTYPE html> <html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="Width = device - width, initial - scale = 1.0">
    <title>Document</title>
</head>
<body>
    <form method="POST" enctype="multipart/form-data" onsubmit="return false;"><input id="file" type="file" placeholder="select a file"></form>
</body>
<script src="./wasm_exec.js"Var target = {// Go will call this method to pass the calculated result callback(md5) {// Print the result to console console.log(Md5 of file is:, md5);
        }
    }
    document.getElementById("file").addEventListener("change".function() {
        const file = this.files[0];
        const go = new Go()
        WebAssembly.instantiateStreaming(fetch('main.wasm'), go.importObject) .then(async result => { go.run(result.instance); Const buffer = await file.arrayBuffer(); Uint8Array const bytes = new Uint8Array(buffer) // Call the calcMd5 method on the target object (this method is provided by Go, Mount to target) target.calcmd5 (bytes); }); }); </script> </html>Copy the code

In the above code, we listen for the input change event, and in this event’s callback, we can access the selected file object through this.files. The following wASM initialization code for Go, described in the first article, is not covered here. Below:

Const buffer = await file.arrayBuffer(); Uint8Array const bytes = new Uint8Array(buffer)Copy the code

Why convert the ArrayBuffer object to a Uint8Array object? When Go receives data from JS, we need a method called CopyBytesToGo to copy data into the Go object, so that js data can be used in Go. This method requires that we must pass Uint8Array data to Go, otherwise the following error will be reported:

This mistake is fairly clear, isn’t it?

Last line of code:

// Call the calcMd5 method on the target object (provided by Go and mounted to target) target.calcmd5 (bytes);Copy the code

Here we call the calcMd5 method on the target object and pass bytes as the first argument. Note that the calcMd5 method is declared in Go and attached to the target object. You can see our js code. There is no place to declare a calcMd5 method on the target object.

The above code is the JS side implementation, the code is relatively simple, but if you are not familiar with Go wASM, it is easy to fall into the trap.

Here’s the code for the Go side:

Go code implementation

package main

import (
	"crypto/md5"
	"fmt"
	"syscall/js"
)

func main(a) {

	// Declare a function that is exported to the JS side for the js side to call
	calcMd5 := js.FuncOf(func(this js.Value, args []js.Value) interface{} {
		// Declare a slice of the same size as the file
		buffer := make([]byte, args[0].Length())
		// Copy the bytes data of the file into a slice, passing in a Uint8Array type
		js.CopyBytesToGo(buffer, args[0])
		// Calculate the MD5 value
		res := md5.Sum(buffer)
		// Call the method on the js side to return the structure to the JS side
		js.Global().Get("target").Get("callback").Invoke(fmt.Sprintf("%x", res))
		return nil
	})

	// Set target's calcMd5 method to the above calcMd5 implementation
	js.Global().Get("target").Set("calcMd5", calcMd5)

	// Prevent the go program from exiting, because the js side can no longer be called after exiting
	signal := make(chan int)
	<-signal
}

Copy the code

In the above code, we first declare a calcMd5 function. Note that this function is not quite the same as the normal go function, but needs to be wrapped with js.FuncOf.

buffer := make([]byte, args[0].Length())
Copy the code

In this line of code, we create a byte slice using the make function, which is a built-in function of go. Use the make function to create a slice called buffer, and then slice the size of the Uint8Array as long as the Uint8Array is passed to the JS end. We get the first argument by arg[0].

js.CopyBytesToGo(buffer, args[0])
Copy the code

This line of code copies the Uint8Array data passed in from the JS end into the buffer slice we created, so that it can be used in the later GO code, otherwise it cannot be directly used. If there is a CopyBytesToGo method, is there a copybyTesttestjs method? There must be. You can see it here: golang.org/pkg/syscall…

Res := md5.sum (buffer) // Call the method on the js side to return the structure to the js side js.global ().get ()"target").Get("callback").Invoke(fmt.Sprintf("%x", res))
Copy the code

Sprintf(“%x”, res) to convert res to hexadecimal string data. The md5.Sum method itself returns a byte slice. The following line of code calls the callback method in the target object declared on the JS side above, passing the MD5 value as an argument. In this way, the JS side can get the result of md5 calculation, to do the following things.

Finally, take a look at the execution result:

By contrast, the result of executing the MD5 command above is the same, indicating no problem.

conclusion

In this article, the main brief introduction of md5 is what, and then through the implementation of a demo, we should know, file second transmission is how to achieve, and, we also see, using WASM to calculate stationery MD5 speed, is much faster than JS calculation. This is why, in previous articles, I said that WASM is generally used for file uploading, computing, etc., but not for most scenarios.

Well, that’s all for this article, and there may be another article on using WASM to do calculations, such as using WASM for online Excel function calculations, if necessary.

If there is anything wrong in this article, please correct it at 🙏.

Reference links:

Golang.org/pkg/syscall…

Stackoverflow.com/questions/3…