1. Business scenarios

In the small program, the real-time frame data of Camera need to be acquired every 0.5s and converted into Base64

2, start

(1) Obtain Camera real-time frame data

// wxml <camera device-position="{{device? 'front':'back'}}" flash="off" binderror="error" style="width: 100%; height: 560rpx;" class="camera"></camera>Copy the code
// js

let listener

page({
  data: {
    device: true.speedMaxCount: 30
  },
  startTacking() {
    var _that = this;
    var count = 0;
    const context = wx.createCameraContext();
    if(! context.onCameraFrame) {var message = 'Base library 2.7.0 now supports'.';
      wx.showToast({
          title: message,
          icon: 'none'
      });
      return;
    }
    listener = context.onCameraFrame(async function (res) {
    // 60 frames per second
      if (count < _that.data.speedMaxCount) {
        count++;
        return;
      }
      count = 0;
      _that.stopTacking()
      // onCameraFrame retrieves unencoded raw RGBA image data, which is then converted to images
      _that.changeDataToBase64(res)
      
    });
    // start
    listener.start();
    console.log('startTacking'.'listener is start');
  },
  stopTacking() {
    if(listener) { listener.stop(); }},onReady: function () {
    this.startTacking(); }})Copy the code

(2) Transfer the obtained original image data to Base64

OnCameraFrame returns data in the following format and transfers the raw image data to Base64

Method 1: UPNG

Import upNG in the JS page

// js
const upng = require('.. /.. /.. /utils/upng/UPNG.js')
Copy the code

// js
changeDataToBase64() {
  let pngData = upng.encode([frame.data], frame.width, frame.height)
  let base64 = wx.arrayBufferToBase64(pngData)
  // Other operations after getting the data
}
Copy the code

Main problem: There was no problem in the developer tool test, real machine test and Android machine test, but the code transfer time of ios models was particularly long, basically more than 10s. This,,, do not need user feedback, the test will not pass

Method 2: Add live frame data to Canvas

Borrow canvas’s putImageData, add it to the canvas, turn it into a temporary file, and turn it into Base64

The data type of data written by canvasPutImageData in the small program is Uint8ClampedArray, but the data type of real-time frame data obtained is ArrayBuffer. Therefore, the data type of the ArrayBuffer needs to be changed to Uint8ClampedArray first

// js
changeDataToBase64(frame) {
  var data = new Uint8Array(frame.data);
  var clamped = new Uint8ClampedArray(data);
  let that = this
    wx.canvasPutImageData({
      canvasId: 'myCanvas'.x: 0.y: 0.width: frame.width,
      height: frame.height,
      data: clamped,
      success(res) {
        // Convert temporary files
        console.log(res)
        wx.canvasToTempFilePath({
          x: 0.y: 0.width: frame.width,
          height: frame.height,
          canvasId: 'myCanvas'.fileType: 'jpg'.destWidth: frame.width,
          destHeight: frame.height,
          // Change the precision
          quality: 0.8.success(res) {
            // Switch temporary files to base64
            wx.getFileSystemManager().readFile({
              filePath: res.tempFilePath, // Select the relative path for the image to return
              encoding: 'base64'.// Encoding format
              success: res= > {
                / / save base64
                let base64 = res.data;    
                // Other operations after getting the data}})},fail(res) {
            console.log(res)
            wx.showToast({
              title: 'Image generation failed, recheck'.icon: 'none'.duration: 1000
            })
            // If the frame is converted to a temporary file, it can be converted to a temporary file
            that.startTacking()
          }
        }, that)
      }
    })
 }
Copy the code

Main problem: There is no problem in the developer tool test, the real machine test, ios time is significantly shortened, about 1s. Android machine time is about 1s, in addition to some models for the first time to temporary file error, as long as the frame data again, transcoding there is no problem.

3, the end

Written at 17:27 dec 31, 2020

2020 is almost over and I don’t have a boyfriend