The front-end implementation of the browser side scan function 🌟

Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money.

background

Not long ago, I did the function of obtaining the browser camera and scanning code recognition. In this article, I sorted out the knowledge points involved and the specific code implementation, and organized them into the content of this article.

This paper mainly introduces, through the use of front-end development technology based on VUE technology stack, in the browser side to switch up the camera 📷, and scan code recognition function, to identify the TWO-DIMENSIONAL code to jump or other operations. This paper is divided into background introduction, implementation effect, technical introduction, code implementation, summary and other parts.

Implementation effect

In this example, there are two main page home page and scan code page, and the specific implementation effect is shown in the figure below.

Home page: ClickSCAN QRCODEButton to enter the scan page.
Scan page: enter for the first time, or pop upA dialog box is displayed asking you to obtain the camera access permission, click “Allow access”, and the page starts to load camera data and pick up TWO-DIMENSIONAL code. If the two-dimensional code is captured, the two-dimensional code analysis begins. After the analysis is successful, the popup window of loading recognition succeeds.

📸 Online experience: dragonir.github. IO /h5-scan-qrc…

📌 Tip: Portrait access is required in a browser with a camera device. Mobile phone horizontal and vertical screen detection small knowledge can go to my other article “50 tone small game in the front knowledge” for understanding.

Technology introduction

WebRTC API

WebRTC (Web Real-Time Communications) is a real-time communication technology that allows Web applications or sites to establish peer-to-peer connections between browsers without resorting to intermediaries. Implement the transmission of video streams and/or audio streams or any other data. WebRTC includes standards that make it possible to create peer-to-peer data sharing and teleconferencing without having to install any plug-ins or third-party software.

Three main interfaces:

MediaStream: Synchronizes video and audio streams through the device’s camera and microphone.
RTCPeerConnectionIs:WebRTCComponents for building stable, efficient flow from point to point.
RTCDataChannel: Enables browsers to establish a high throughput, low latency channel for transmitting arbitrary data.

🔗 head over to MDN to learn more: WebRTC_API

WebRTC adapter

Although the WebRTC specification is relatively robust, not all browsers implement all of its features, and some browsers need to prefix some or all of the WebRTC apis to work properly.

The WebRTC organization provides a WebRTC Adapter on Github to address the compatibility issue of implementing WebRTC across browsers. The adapter is a JavaScript shim that lets you write code as described in the WebRTC specification, without prefixes or other compatibility fixes in all browsers that support WebRTC.

🔗 Go to MDN for further study: WebRTC Adapter

The core of the API`navigator.mediaDevices.getUserMedia`

Web call camera needs to call getUserMedia API, MediaDevices getUserMedia () will prompt the user for permission to use the media input media input will produce a MediaStream, containing the request of the media type of orbit. This stream can contain A video track (from hardware or virtual video sources, such as cameras, video capture devices, screen sharing services, and so on), an audio track (also from hardware or virtual audio sources, such as microphones, A/D converters, and so on), or some other track type.

It returns a Promise object, and on success resolve calls back a MediaStream object; If the user denies permission, or if the desired media source is not available, Promise calls back a PermissionDeniedError or NotFoundError. (The returned Promise object may neither resolve nor reject, because the user does not have to choose to allow or reject.)

MediaDevices can usually be obtained using navigator.mediaDevices, for example:

navigator.mediaDevices.getUserMedia(constraints)
  .then(function(stream) {
    // 使用这个stream
  })
  .catch(function(err) {
    / / handle the error
  })
Copy the code

🔗 to MDN in-depth study: the navigator. MediaDevices. GetUserMedia

Two-dimensional code parsing library`JSQR`

JsQR is a pure JavaScript QR code parsing library that reads the raw image or camera and will locate, extract and parse any QR codes in it.

If you want to scan the webcam stream using jsQR, you need ImageData to be extracted from the video stream, which can then be passed to jsQR.

JsQR exports a method that takes four parameters: decoded image data, width, height, and optional objects to further configure scan behavior.

ImageData: Format is [R0, G0, B0, A0, R1, G1, B1, A1,…] Uint8ClampedArray (8-bit unsigned integer fixed array) rGBA pixel value of

const code = jsQR(imageData, width, height, options);
if (code) {
  console.log('Find the QR code! ', code);
}
Copy the code

🔗 head over to Github to learn more: jsQR

Code implementation

process

The entire code scanning process is as follows: During page initialization, check whether the browser supports mediaDevices related APIS. If the browser fails to call the camera, the callback fails. Call success, capture the video stream, and then scan code recognition, no scan to identify the two-dimensional code will continue to scan, scan successful pattern after successful draw and successful callback.

The following contents split the process to realize corresponding functions respectively.

Sweep code component`Scaner`

Page structure

Let’s take a look at the page structure, which is mainly composed of four parts:

Prompt dialog box.
Code box.
video: Shows the video stream captured by the camera.
canvas: Draws a video frame for TWO-DIMENSIONAL code recognition.

<template>
  <div class="scaner" ref="scaner">
    <! -- Prompt box: used to display prompt in incompatible browsers -->
    <div class="banner" v-if="showBanner">
      <i class="close_icon" @click="() => showBanner = false"></i>
      <p class="text">If the current browser cannot scan codes, switch to another browser</p>
    </div>
    <! -- Scan box: display scan animation -->
    <div class="cover">
      <p class="line"></p>
      <span class="square top left"></span>
      <span class="square top right"></span>
      <span class="square bottom right"></span>
      <span class="square bottom left"></span>
      <p class="tips">Put the QR code into the box and it will be scanned automatically</p>
    </div>
    <! -- Video stream display -->
    <video
      v-show="showPlay"
      class="source"
      ref="video"
      :width="videoWH.width"
      :height="videoWH.height"
      controls
    ></video>
    <canvas v-show=! "" showPlay" ref="canvas" />
    <button v-show="showPlay" @click="run">start</button>
  </div>
</template>
Copy the code

Method: Draw

Line drawing.
Frame (used to draw a rectangle after a successful sweep).

Draw a line / /
drawLine (begin, end) {
  this.canvas.beginPath();
  this.canvas.moveTo(begin.x, begin.y);
  this.canvas.lineTo(end.x, end.y);
  this.canvas.lineWidth = this.lineWidth;
  this.canvas.strokeStyle = this.lineColor;
  this.canvas.stroke();
},
/ / frame
drawBox (location) {
  if (this.drawOnfound) {
    this.drawLine(location.topLeftCorner, location.topRightCorner);
    this.drawLine(location.topRightCorner, location.bottomRightCorner);
    this.drawLine(location.bottomRightCorner, location.bottomLeftCorner);
    this.drawLine(location.bottomLeftCorner, location.topLeftCorner); }},Copy the code

Method: Initialize

Check whether it is supported.
Turn on the camera.
Successful Failure processing.

/ / initialization
setup () {
  / / whether the browser supports mounted in MediaDevices. GetUserMedia () method
  if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
    this.previousCode = null;
    this.parity = 0;
    this.active = true;
    this.canvas = this.$refs.canvas.getContext("2d");
    // Get camera mode, default setting is rear camera
    const facingMode = this.useBackCamera ? { exact: 'environment' } : 'user';
    // Camera video processing
    const handleSuccess = stream= > {
       if (this.$refs.video.srcObject ! = =undefined) {
        this.$refs.video.srcObject = stream;
      } else if (window.videoEl.mozSrcObject ! = =undefined) {
        this.$refs.video.mozSrcObject = stream;
      } else if (window.URL.createObjectURL) {
        this.$refs.video.src = window.URL.createObjectURL(stream);
      } else if (window.webkitURL) {
        this.$refs.video.src = window.webkitURL.createObjectURL(stream);
      } else {
        this.$refs.video.src = stream;
      }
      // If you don't want the user to drag the progress bar, you can just use the playsinLine property, webKit-PlaysinLine property
      this.$refs.video.playsInline = true;
      const playPromise = this.$refs.video.play();
      playPromise.catch(() = > (this.showPlay = true));
      // Perform periodic scan recognition when the video starts playing
      playPromise.then(this.run);
    };
    // Capture the video stream
    navigator.mediaDevices
      .getUserMedia({ video: { facingMode } })
      .then(handleSuccess)
      .catch(() = > {
        navigator.mediaDevices
          .getUserMedia({ video: true })
          .then(handleSuccess)
          .catch(error= > {
            this.$emit("error-captured", error); }); }); }},Copy the code

Method: Periodic scanning

run () {
  if (this.active) {
    // The browser loops the sweep method before the next redraw
    requestAnimationFrame(this.tick); }},Copy the code

Method: Successful callback

// Failed to identify the QR code
found (code) {
  if (this.previousCode ! == code) {this.previousCode = code;
  } else if (this.previousCode === code) {
    this.parity += 1;
  }
  if (this.parity > 2) {
    this.active = this.stopOnScanned ? false : true;
    this.parity = 0;
    this.$emit("code-scanned", code); }},Copy the code

Method: Stop


// Stop completely
fullStop () {
  if (this.$refs.video && this.$refs.video.srcObject) {
    // Stop the video stream sequence track
    this.$refs.video.srcObject.getTracks().forEach(t= >t.stop()); }}Copy the code

Method: Scanning

Draws video frames.
Scan code recognition.

// periodic scan recognition
tick () {
  // The video is in the ready stage and enough data has been loaded
  if (this.$refs.video && this.$refs.video.readyState === this.$refs.video.HAVE_ENOUGH_DATA) {
    // Start drawing video on canvas
    this.$refs.canvas.height = this.videoWH.height;
    this.$refs.canvas.width = this.videoWH.width;
    this.canvas.drawImage(this.$refs.video, 0.0.this.$refs.canvas.width, this.$refs.canvas.height);
    // getImageData() copies the pixel data of the rectangle drawn on the canvas
    const imageData = this.canvas.getImageData(0.0.this.$refs.canvas.width, this.$refs.canvas.height);
    let code = false;
    try {
      // Identify the qr code
      code = jsQR(imageData.data, imageData.width, imageData.height);
    } catch (e) {
      console.error(e);
    }
    // If the qr code is recognized, draw a rectangular box
    if (code) {
      this.drawBox(code.location);
      // Identify successful event handling
      this.found(code.data); }}this.run();
},
Copy the code

The parent component

Scaner’s parent component loads the page and displays callbacks to Scaner scan results.

Page structure

<template>
  <div class="scan">
    <! -- Navigation bar -->
    <div class="nav">
      <a class="close" @click="() => $router.go(-1)"></a>
      <p class="title">Scan QRcode</p>
    </div>
    <div class="scroll-container">
      <! -- Scan subcomponent -->
      <Scaner
        v-on:code-scanned="codeScanned"
        v-on:error-captured="errorCaptured"
        :stop-on-scanned="true"
        :draw-on-found="true"
        :responsive="false"
      />
    </div>
  </div>
</template>
Copy the code

Parent component method

import Scaner from '.. /components/Scaner';

export default {
  name: 'Scan'.components: {
    Scaner
  },
  data () {
    return {
      errorMessage: "".scanned: ""}},methods: {
    codeScanned(code) {
      this.scanned = code;
      setTimeout(() = > {
        alert('Scan code parsing success:${code}`);
      }, 200)},errorCaptured(error) {
      switch (error.name) {
        case "NotAllowedError":
          this.errorMessage = "Camera permission denied.";
          break;
        case "NotFoundError":
          this.errorMessage = "There is no connected camera.";
          break;
        case "NotSupportedError":
          this.errorMessage =
            "Seems like this page is served in non-secure context.";
          break;
        case "NotReadableError":
          this.errorMessage =
            "Couldn't access your camera. Is it already in use?";
          break;
        case "OverconstrainedError":
          this.errorMessage = "Constraints don't match any installed camera.";
          break;
        default:
          this.errorMessage = "UNKNOWN ERROR: " + error.message;
      }
      console.error(this.errorMessage);
     alert('Camera call failed');
    }
  },
  mounted () {
    var str = navigator.userAgent.toLowerCase();
    var ver = str.match(/cpu iphone os (.*?) like mac os/);
    // Failed to call the camera in iOS 10.3.3
    if (ver && ver[1].replace(/_/g.".") < '10.3.3') {
     alert('Camera call failed'); }}Copy the code

The complete code

🔗 lot: github.com/dragonir/h5…

conclusion

Application of extended

I think the following functions can be through the browser to call the camera and scan recognition to achieve, we think there are very wow 🌟 function applications can be achieved through the browser side scan code 😂?

🌏Link jump.
🛒Price enquiry.
🔒Login authentication.
📂File download.

compatibility

❗Even if it doesadapter.getUserMedia APISome browsers do not support it.
❗Earlier versions of browsers (e.gIOS 10.3Below),AndroidNiche browsers (e.gIQOONative browser) is not compatible.
❗ QQ,WeChatThe built-in browser cannot be called.

The resources

[1]. Taking still photos with WebRTC
[2]. Choosing cameras in JavaScript with the mediaDevices API
[3]. How to use JavaScript to access the front and rear cameras of the device

The front-end implementation of the browser side scan function 🌟

background

Implementation effect

Technology introduction

WebRTC API

WebRTC adapter

The core of the APInavigator.mediaDevices.getUserMedia

Two-dimensional code parsing libraryJSQR

Code implementation

process

Sweep code componentScaner

Page structure

Method: Draw

Method: Initialize

Method: Periodic scanning

Method: Successful callback

Method: Stop

Method: Scanning

The parent component

Page structure

Parent component method

The complete code

conclusion

Application of extended

compatibility

The resources

Related Posts

[Vue warn]: $attrs is readonly; Webpack externals configuration

JS Magic Hall: new ES6 feature – GeneratorFunction introduction

How does the React setState know what to do –Dan Abramov

The core of the API`navigator.mediaDevices.getUserMedia`

Two-dimensional code parsing library`JSQR`

Sweep code component`Scaner`