This paper is based on the live broadcast of “How to achieve page recording and playback with JS” by Wang Lei, a senior front-end research and development engineer of Shence Data. The following is the text:

1. Business background

For R&D, there are always online issues to deal with. To B and To C companies have many similarities in the process of dealing with online problems, but there are also some differences.For To C enterprise, we usually obtain the abnormal state of the system through user feedback or online real-time monitoring system. Meanwhile, we integrate the obtained information of the user environment, system, configuration, version and other information To debug and reproduce locally. After successful reproduction, we carry out bugfix and release operations.

For ENTERPRISE To B, the overall process is similar To that of enterprise To B. However, the nature of ENTERPRISE To B determines that it is difficult for us To obtain the context and monitoring content of the problem, and customers, after-sales service, technical support or corresponding personnel on duty are often required To transfer information To assist r&d and locate the problem. At the same time, the customer environment of different types of enterprises is also a challenge To R&D. R&d cannot simulate the To B customer context like the universal To C environment, which hampers problem positioning To some extent.In order to improve the efficiency of locating online problems, we embarked on an investigation and implemented an internal anomaly diagnosis platform. You can use the diagnosis file provided by the customer to quickly locate the problem without entering the customer’s real environment. For front-end r&d engineers, in addition to environment, version and other information, operation path is often very important. In order to clearly and quickly reproduce customer operation path, we introduced the page recording function.

2. Abnormal diagnosis and investigation

Front-end r&d tends to rely on the operating process of customers when diagnosing anomalies. In order to reduce the communication cost with customers, we hope to directly record the operating process of users. To this end, we found in our research that there are two main approaches to Web recording in the industry. One is to record the screen represented by the native recording of WebRTC browser, and the other is to record the DOM structure of the page.

For the business scenario of Shenze data, we need to consider the customer’s data sensitivity (desensitization to the specified data), product size and other factors, and finally we choose the DOM recording with more flexibility and lower propagation to achieve the function. Through investigation, we found that RRWeb open source warehouse has very good support for DOM recording, and can efficiently convert the PAGE DOM structure into JSON data through the corresponding algorithm. For the data in JSON format with strong operability, we can achieve compression, encryption, sensitive data processing and other operations by ourselves. At the same time, the compressed file size is also very considerable, the overall can be controlled under 5M.

Three, page recording and playback

1. Recording principle

Rrweb for recording and playback mainly consists of RRWeb, RRWeb-snapshot, RRWeb-player, which has a good support for page recording and playback, the following figure shows the general operation process of recording and playback.

Rrweb takes the DOM snapshot of the first screen when recording, traverses the DOM Tree of the whole page and converts it into JSON structure data through nodeType mapping. At the same time, it synchronously converts the incremental changed data into JSON data for storage. During the whole recording process, unique ID will be generated to determine the DOM node corresponding to the incremental data, and timestamp is used to ensure the playback order. The following figure respectively corresponds to the first screen snapshot and recording schematic diagram.

For the incremental data update after the first screen snapshot, the DOM incremental change is obtained through mutationObserver, the method (property) is hijacked through global event listener, event (property) proxy, and the hijacked incremental change data is stored in JSON data.

2. Playback Principle

The principle of the playback function is similar to that of recording. Firstly, the DOM snapshot of the first screen is reconstructed, and the DOM structure of the first screen is reconstructed by mapping to different node construction methods through user-defined type while traversing THE JSON products.For the incremental data generated by recording, RRWeb internally calls different functions according to different incremental types and presents them on the page. At the same time, the time stamp generated by recording is used to ensure playback order, the Node ID is applied to the specified DOM Node, and the smoothness of page playback is ensured by the requestAnimationFrame.

Four, summary

Compared with video, screen recording by recording page structure and behavior is not only more operable (for example, YOU can encrypt, compress, segment, and delete JSON products), but also can be transmitted in multiple ways (for example, segment transmission and specified business scenario transmission). In addition, page recording can be used in a variety of scenarios, such as fault location, product usage analysis, automated test records, important information (operation) backup, and multi-party collaboration.

There are more good technology class when you come to 👇 registration address: school. Sensorsdata. Cn/active / 1828