1. Business scenarios
In the small program, the real-time frame data of Camera need to be acquired every 0.5s and converted into Base64
2, start
(1) Obtain Camera real-time frame data
// wxml <camera device-position="{{device? 'front':'back'}}" flash="off" binderror="error" style="width: 100%; height: 560rpx;" class="camera"></camera>Copy the code
// js
let listener
page({
data: {
device: true.speedMaxCount: 30
},
startTacking() {
var _that = this;
var count = 0;
const context = wx.createCameraContext();
if(! context.onCameraFrame) {var message = 'Base library 2.7.0 now supports'.';
wx.showToast({
title: message,
icon: 'none'
});
return;
}
listener = context.onCameraFrame(async function (res) {
// 60 frames per second
if (count < _that.data.speedMaxCount) {
count++;
return;
}
count = 0;
_that.stopTacking()
// onCameraFrame retrieves unencoded raw RGBA image data, which is then converted to images
_that.changeDataToBase64(res)
});
// start
listener.start();
console.log('startTacking'.'listener is start');
},
stopTacking() {
if(listener) { listener.stop(); }},onReady: function () {
this.startTacking(); }})Copy the code
(2) Transfer the obtained original image data to Base64
OnCameraFrame returns data in the following format and transfers the raw image data to Base64
Method 1: UPNG
Import upNG in the JS page
// js
const upng = require('.. /.. /.. /utils/upng/UPNG.js')
Copy the code
// js
changeDataToBase64() {
let pngData = upng.encode([frame.data], frame.width, frame.height)
let base64 = wx.arrayBufferToBase64(pngData)
// Other operations after getting the data
}
Copy the code
Main problem: There was no problem in the developer tool test, real machine test and Android machine test, but the code transfer time of ios models was particularly long, basically more than 10s. This,,, do not need user feedback, the test will not pass
Method 2: Add live frame data to Canvas
Borrow canvas’s putImageData, add it to the canvas, turn it into a temporary file, and turn it into Base64
The data type of data written by canvasPutImageData in the small program is Uint8ClampedArray, but the data type of real-time frame data obtained is ArrayBuffer. Therefore, the data type of the ArrayBuffer needs to be changed to Uint8ClampedArray first
// js
changeDataToBase64(frame) {
var data = new Uint8Array(frame.data);
var clamped = new Uint8ClampedArray(data);
let that = this
wx.canvasPutImageData({
canvasId: 'myCanvas'.x: 0.y: 0.width: frame.width,
height: frame.height,
data: clamped,
success(res) {
// Convert temporary files
console.log(res)
wx.canvasToTempFilePath({
x: 0.y: 0.width: frame.width,
height: frame.height,
canvasId: 'myCanvas'.fileType: 'jpg'.destWidth: frame.width,
destHeight: frame.height,
// Change the precision
quality: 0.8.success(res) {
// Switch temporary files to base64
wx.getFileSystemManager().readFile({
filePath: res.tempFilePath, // Select the relative path for the image to return
encoding: 'base64'.// Encoding format
success: res= > {
/ / save base64
let base64 = res.data;
// Other operations after getting the data}})},fail(res) {
console.log(res)
wx.showToast({
title: 'Image generation failed, recheck'.icon: 'none'.duration: 1000
})
// If the frame is converted to a temporary file, it can be converted to a temporary file
that.startTacking()
}
}, that)
}
})
}
Copy the code
Main problem: There is no problem in the developer tool test, the real machine test, ios time is significantly shortened, about 1s. Android machine time is about 1s, in addition to some models for the first time to temporary file error, as long as the frame data again, transcoding there is no problem.
3, the end
Written at 17:27 dec 31, 2020
2020 is almost over and I don’t have a boyfriend