One, foreword

Shence Data micro channel small program SDK[1] is a lightweight data acquisition burying point SDK for micro channel small program, including code burying point and full burying point functions. Among them, the full buried point function is realized by proxy wechat applet native App, Page, Component interface and corresponding life cycle function. The following will take SDK version V1.13.28 [2] as an example to introduce the architecture of wechat applets SDK.

Second, basic principles

Since open source, the SDK code and directory structure has been constantly adjusted and optimized. At present, there are product directory and other source files of different versions in the open source code, as shown in Figure 2-1:

Figure 2-1 source directory of wechat applets SDK

  • Among them, the introduction of several main directories is as follows:
  • /product: uncompressed source file;
  • / sensorsdata. Custom. Es6. Min. Js: support es6 import and export of grammar a custom version of the SDK source code files;
  • / sensorsdata. Custom. Min. Js: support the require syntax introduced a custom version of the SDK source code files;
  • /sensorsdata.min.es6.js: support es6 import/export syntax full embedded version SDK source file;
  • /sensorsdata.min.js: support the introduction of require syntax fully buried version SDK source file.

Data flow

Figure 3-1 shows the process from data collection to data transmission:

Figure 3-1 Data flow chart

As can be seen from the figure above, the data flow mainly includes the following steps:

  1. First of all, the uploaded events and attributes will do format verification;

  2. In the batch sending configuration, data is cached to the memory and local PC and sent over the network when the conditions are met. If the data is successfully sent, delete the data. If the data fails to be sent, try again.

  3. In the real-time sending configuration, data is first saved to the sending queue and then sent one by one. After the sending is complete (whether it succeeds or fails), the sent data is deleted.

  4. If the request duration exceeds the preset timeout period (3 seconds by default), the request will be canceled regardless of batch or real-time sending.

4. Architecture analysis

4.1 introduction

As mentioned above, Shence Data micro channel small program SDK is a lightweight data collection burying point SDK for micro channel small program, and data collection is the core element of building a data platform. Whether data collection is complete, accurate, timely and connected directly affects the application effect of the whole data platform. Therefore, shence Data micro channel small program SDK uses a good architecture to ensure data collection.

4.2 architecture diagram

Figure 4-1 shows the overall architecture of Shence Data micro program SDK:

Figure 4-1 overall architecture of wechat applets SDK

4.3 Architecture details

4.3.1 Initializing modules

The SDK initialization module mainly includes two functions: initialization parameter configuration and initialization of data cache queue.

  1. Initialize parameter configurations

After the introduction of shence Data micro program SDK, related configuration of SDK initialization parameters was carried out by calling setPara() interface. Common configuration items are as follows:

  • Server_url: address for receiving data. This parameter must be set. There is no default value.
  • AutoTrack: Full buried point control switch, autotrack. appLaunch, autotrack. appShow, Autotrack. appHide, Autotrack. pageShow, autotrack. mpClick,

AutoTrack. PageShare, autoTrack. MpFavorite

Respectively control the opening and closing of the corresponding full buried point events;

  • Show_log: indicates whether to print data on the console.

Among them, the code of full buried point control is as follows:

/** * Sensors Analytics SDK */ const sa = require('./sensorsdata.min.js'); Sa. setPara({server_URL: ", autoTrack: {appLaunch: true, // control whether to collect $MPLaunch appShow: $MPHide pageShow: true, $MPViewScreen pageShare: $mpClick favorite: true, $mpClick favorite: true, $MPAddFavortes}})Copy the code
  1. Initialize the data cache queue

Wechat small program SDK provides init() interface to let developers decide when the SDK initialization is complete. Only when the SDK initialization is complete, the event data triggered by the user will be sent out through the network. Otherwise, the data is stored in the initialization data cache queue. After the initialization is complete, the data in the queue is read and sent out. Figure 4-2 shows the initialization process.

Figure 4-2 Initialization flowchart

4.3.2 Data acquisition module

Data acquisition module is the core module, mainly responsible for accurate and complete data acquisition through buried points. The data collection module mainly provides the following functions: Code burying point: After SDK initialization, track interface burying point collecting events to be collected is called; Full buried point: all or most of the user’s behavior data can be collected automatically in advance without the developer writing code or writing a small amount of code;

Preset attributes: Basic attributes (such as the operating system type, operating system version, carrier information, application version, and device manufacturer) that are automatically collected by the EMBEDDED SDK and carried by any event. For the introduction of these functions, you can refer to the “Shence Data micro program SDK function introduction”, here is no longer repeated.

4.3.3 Data Storage Module

In order to ensure the accuracy and timeliness of event data, the data acquisition SDK is required to synchronize event data to the server as soon as possible. However, in some cases (for example, the user is disconnected from the network), the event data cannot be synchronized or the synchronization fails. When multiple events are triggered consecutively, the event triggered later may reach the server first, resulting in different behavior sequences.

Therefore, after collecting data, wechat small program SDK will first cache the event data in a queue for real-time transmission. Wait until the previous send request completes before sending the next data. In batch sending mode, data is cached in the memory and local storage, and the client time is used to record the event triggering time. After the event triggering condition is met, the event is sent.

4.3.4 Data transmission module

In the data storage module, how to store data to the client local or memory queue is introduced. It doesn’t make sense if the event data is always cached, we also need to synchronize the data to the server. Then the data can be stored, extracted, analyzed and displayed by the server to give full play to the value of the data. Therefore, the data sending module is responsible for synchronizing cached event data to the server.

In the development of wechat applets, the encapsulated official interface wx.request() provided by wechat applets can be directly used to send network requests. At present, wechat small program SDK supports two sending modes:

  • Real-time sending: Data stored in the cache queue is read and sent piece by piece. The latter piece of data is read and sent only after the former piece of data is sent. After the former piece of data is sent, it is deleted from the queue.

  • Batch sending: After a certain amount of data has accumulated in the cache queue, all data is sent via wx.request(). If the request fails to be sent, the failed data is sent with the new data in the next sending. If the request is sent successfully, the sent data is deleted from the cache.

By default, the wechat applets SDK uses real-time sending. In batch sending mode, you can set batch_send on the setPara() interface to set the interval and number of sending messages.

4.3.5 Tool module

Shenze data micro channel small program SDK provides a number of burying point related tool interface, these auxiliary tools can help the realization of burying point. For example,. GetUtmFromPage parses the parameters of the current page. The.getCurrentPath interface gets the current page path. The sa.store object provides the functions of creating, verifying, and storing user ids. The functions and implementation of these auxiliary tools will not be described, you can refer to the SDK source code.

4.3.6 Log module

If show_log is set to true in the initialization parameter, the log output function will be enabled, and the collected data log can be printed in the console of wechat applet developer tool, as shown below:

Five, the total knot

This paper gives a comprehensive introduction to the architecture of shence Data micro program SDK, aiming to give you a clear understanding of the architecture of SDK. The specific technical implementation of the SDK and other relevant knowledge will be introduced to you step by step in subsequent articles.

References:

[1] github.com/sensorsdata… [2] github.com/sensorsdata…

Article source: Shence Technology Community