1. Preparation

Emscripten environment construction, FFmpeg source download, FFmpeg Developer library.

The construction of the environment, download the source code class library and so on, the author will not tell about the online tutorial a lot.

2. Overall process

1. Get the uploaded video file from the browser and use FileReader to store the data in the Uint8Array byte stream.

2. Store the byte stream into memory using the _malloc method exposed by EMCC.

3. Use C/C++ to read the video data from memory, then use FFmpeg related interface to decode the video stream, take out the default YUV420P format data, and convert it into RGB24 type data.

4. Pass the RGB24 data and image size back to the browser, create createImageData container with Canvas, fill the container with RGB24, and render the image to canvas with putImageData.

5. Finally, use Canvas toDataURL method to export the image.

3. Core implementation

The front part

  const fileReader = new FileReader();
  fileReader.readAsArrayBuffer(files[0]);
  fileReader.onload = function () {
      const buffer = new Uint8Array(this.result);
      const buffPtr = Module._malloc(buffer.length);

      for(let i=0;i<buffer.length;i++) {
          Module.HEAP8[ptr + i] = buffer[i];
      }

      const rgbPtr = Module._getRgbData(buffPtr, buffer.length);

      ...

      Module._free_buf(buffPtr);
	  Module._free_buf(rgbPtr);
  }
Copy the code

C Language Section

FILE *fp_open = NULL; Void *opaque, uint8_t *buf, int buf_size){if(! feof(fp_open)){ int true_size=fread(buf,1,buf_size,fp_open); return true_size; }else{ return -1; AVFormatContext *avFmtCtx = avformat_alloc_context(); AVIOContext *avIoCtx = avio_alloc_context(buffer, length, 0, NULL, read_buffer, NULL, NULL); avFmtCtx->pb = avIoCtx; avformat_open_input(&avFmtCtx, NULL, NULL, NULL); . // frame -> yuv sws_scale(yuvSwsCtx, avFrame->data, avFrame->linesize, 0, avCodecCtx->height, yuvFrame->data, yuvFrame->linesize); // yuv -> rgb24 sws_scale(rgbSwsCtx, yuvFrame->data, yuvFrame->linesize, 0, avCodecCtx->height, rgb24Frame->data, rgb24Frame->linesize);Copy the code

4. Final remarks

With the continuous iteration and update of front-end technology, front-end is no longer the “front-end”, and there are bigger and more challenges waiting for us in the future.