I am participating in the 2022 Spring Recruitment series – experience review, click here for more details.

The combination of HTML5, JavaScript, and modern browsers makes it possible for traditional Web applications to interact with users in a much richer way than in the past. The camera has now become a standard configuration of smart phones, and front-end Web applications also have more and more requirements for opening the camera of the machine and scanning bar codes and two-dimensional codes.

This article introduces two development techniques of using JavaScript to call device camera in browser environment.

Method 1: Based on the modern browser support WebRTC API implementation

I used this idea to implement a simple Web page, the full source code is maintained in the following Github repository:

Github.com/wangzixi-di…

Let’s start by looking at the effects of running the Web application.

Accessed from a laptop, the browser will pop up asking the user whether to allow the app to access the camera on the device:

After clicking Allow, the area below the app will display an image of the area my camera is facing in real time:

After clicking the “Take photo” button, the image displayed by the camera will be solidified below the button and automatically saved locally as a picture.

To access the link on my Samsung phone, the app must also be authorized to use the camera:

Take a picture of an alien hand in front of my computer:

Automatically generate a picture and save it to your phone:

Several key implementation points:

(1) the reason why JavaScript can identify the available camera of the device (including the available audio input and output devices) through the browser is that modern browsers support a set of APIS named WebRTC(Web Real Time Communication). This API enables Web application developers to implement rich real-time multimedia applications through simple JavaScript programming without learning digital signal processing of multimedia. Web application users also do not need to download additional plug-ins.

Use JavaScript to detect available multimedia devices. The core code is as follows:

navigator.mediaDevices.enumerateDevices().then(gotDevices).catch(handleError);
Copy the code

The code first half period of the navigator. MediaDevices. EnumerateDevices () is the native API browser support, this is an asynchronous invocation, returns a promise object:

Wait until after the asynchronous invocation result is available for applications, we through the chain then was introduced into the callback function called gotDevices is triggered, the input parameter is the navigator. MediaDevices. EnumerateDevices () call return values. Take a look at the details of the return values in the debugger:

From the debugger, enumerateDevices returns a list of available audio and video devices on my laptop. The information is consistent with the device information I saw in the operating system:

(2) IN my HTML page, I have defined a video label supported by HTML5, which is used to display images observed through the camera of the device.

But we also need to associate the device camera with the video tag. This is done by assigning a MediaStream object to the srcObject property of the DOM object for the tag.

Where does this MediaStream object come from? In the same way, through the chain called the navigator. MediaDevices. GetUserMedia (constraints) are:

(3) After clicking the photo button, the function of automatically generating pictures and downloading them to local is realized in the click event response function of the button. First, call the API drawImage corresponding to the Context of the Canvas tag to draw the current content displayed in the video tag of the camera content to the Canvas tag, and then use this content to generate a jpeg image and download it to the local.

Method 2: Use SAP UI5 custom controls

This approach is essentially the same as method one, but it is better encapsulation. The steps described in method 1 are encapsulated into a SAP UI5 reusable control, which is convenient for front-end programmers who develop the front-end framework of SAP UI5 to use directly.

For an introduction to SAP UI5, an enterprise-level front-end development framework, see my previous InfoQ article: A brief history of UI5 for enterprise-level front-end applications.

First recall the main points of technical implementation of method 1:

(1) In the HTML page of the Web application, define the native label of HTML5 for displaying videos: video

(2) Use THE API of WebRTC to obtain the MediaStream object corresponding to the camera of the device, and then assign this object instance to the srcObject attribute of the DOM object corresponding to the video tag.

After the above two steps are implemented, the video images we observe through the camera can be displayed in real time in the Video TAB of the Web application. As for saving the video content displayed in the video TAB at a point in time as an image and downloading it, the corresponding JavaScript code is not that different from any other front-end framework and is skipped in this article.

Therefore, with SAP UI5 development, we have to repeat the above two steps.

The most common view format for SAP UI5 applications is XML view. What happens if we just add the HTML native video or div tag to the XML view?

Error 404: UI5 framework cannot load div.js script file.

When I was working in THE CRM Fiori application development team of SAP Chengdu Research Institute, I once wrote a learning tutorial of SAP UI5 framework code. There were two articles in it, which introduced in detail the rendering principle of SAP UI5 XML view runtime:

  1. Why my formatter does not work? A trouble shooting example to know how it works
  2. How I do Self-study on a Given Fiori Control — Part 10

Simply put, there is an implementation of XMLTemplateProcessor.js in SAP UI5. At runtime, when the source file of an XML view is loaded and parsed into the DOM by the browser, it does a depth-first traversal of the DOM tree. For each UI5 tag it encountered, Load its implementation file (if in UI5 debug mode) and create an instance of the tag.

Going back to the example in this article, I wrote that the DIV tag in the SAP UI5 XML view is treated as a SAP UI5 XML control, so the UI5 framework automatically looks for the implementation file of the non-existent DIV control, which is nowhere to be found.

If you know what the problem is, you’ll have a solution. HTML5 native tag video package into UI5 control on the line?

The SAP UI5 open source community already has a packaged library:

Github.com/tiagobalmei…

Let’s look at this example on a laptop:

Click the content taken by the camera displayed on the page, can be automatically saved into a picture. What looks like on your phone:

Then we’ll look at how this Library works.

The hierarchical result of this camera custom UI5 control implementation is as follows:

The implementation of SAP UI5 custom controls includes three JavaScript files:

  • Library.js: defines the header level control information for this control, such as name, version number, dependencies, etc.

  • Camera.js: Implements the steps to bind MediaStream object instances obtained by the WebRTC API to the video element encapsulated by the control.

  • Camerarender.js: The tag that is responsible for placing this custom control in an XML viewCameraRender to a combination of native video and Canvas tags.

Each SAP UI5 control has a corresponding render class, which is used to convert UI5 tags in XML views to HTML5 native tags:

How do I use this custom control?

Use the following definition in the XML view:

<cam:Camera
    id="idCamera"
    width="800"
    height="600"
    snapshot=".onSnapshot" />
Copy the code

conclusion

This article introduces a solution based on WebRTC technology using JavaScript to call the native camera. WebRTC adds real-time communication capabilities to open standards-based applications, enabling video, voice, and generic data to be sent between peers, allowing developers to build powerful voice and video communication solutions.

WebRTC is available in all modern browsers as well as native clients for all major operating systems. The technology behind WebRTC is implemented as an open Web standard and can be used as a regular JavaScript API in all major browsers.

The two examples in this article are implemented and tested in Chrome for Windows 10 operating system. For native clients, such as Android and iOS applications, you can use the WebRTC library that provides the same functionality.

I hope this article will help you to use JavaScript to call the local camera in the browser environment. Thank you for reading.