Author of this article: IMWeb team original link

General introduction

Tencent Classroom is a product that provides users with online education services through live broadcast and on-demand. Since its establishment in 2014, it has accumulated 2.5 million videos, a total of 600 TB, and a total length of 1.5 million hours. Before is tencent video scheme has been adopted, but use the MP4 format, users get the play after the link is very easy to piracy, so taking on the trend of cloud, we will be video on demand moved to tencent cloud – the cloud on demand, this paper will talk about our overall solutions, Web access method and some of the problems.

Vod is divided into two parts: video uploading and video playing. The following table summarizes some data comparison before and after cloud uploading:

Tencent video Tencent cloud
Web video uploading success rate 92% 99.5%
Video transcoding speed (about two hours of video) > 60 minutes The < 20 minutes
Success rate – PC 99% 98.7%
Playback success rate – H5 97% 97.1%

It can be seen that the upload success rate and video transcoding speed have greatly improved, PC and H5 side of the playback success rate cloud and Tencent video basically equal.

The overall plan

Considering that there are too many videos in stock, it is impossible to migrate all videos from Tencent Video to Tencent Cloud in a short time, and users may continue to upload videos to Tencent Video in the old way during the migration process, so the whole voD cloud is divided into two phases:

  1. The first phase of the main work is tencent cloud access to the upload, transcoding and playback function, ensure that all new user upload video cloud process, at the same time, the background to upload new video to bypass a tencent video, both to front end users when they play video cloud failure relegated to tencent video playback, also a significant problem in convenient fast cut back to the old tencent video solutions.
  2. The second phase of work is to migrate all the stored Tencent videos to Tencent cloud, and access the AI function of the cloud to conduct pornography, violence and politics authentication. When the data on the live network is stable and meets expectations, the old scheme can be completely abandoned.

Video Uploading Process

The overall plan of video uploading is shown in the figure above, which mainly involves three parts:

  1. Obtain a signature from the service background
  2. Call the cloud SDK to upload videos
  3. The cloud server performs video transcoding

In the above three blocks, the most important and most likely to cause problems is the part of “calling SDK to upload”, which directly determines the success rate of upload. However, it is also easily affected by the user’s network status, so it needs to be focused on. It is recommended to record detailed user logs for problem location and troubleshooting.

In addition, in fact, the flow chart and tencent cloud document given the client upload guide slightly a little difference, lies in step 4 inform business background upload finished here, the official document is cloud background to inform, we actually is the Web side way used to inform, so as to avoid the Web side the backend interface error prompt the user to upload failed, The cloud background informs the service background of saving relevant data.

Video Playback Process

For various reasons, we did not encrypt the videos when we used Tencent Video, which resulted in some courses being maliciously stolen by others. At present, we use the HLS encryption scheme after going online. With the Key anti-theft chain and DRM (digital Rights Management) scheme provided by the cloud, we encrypt the video. Even if the video address is obtained, it cannot be stolen, which further cracks down on malicious behaviors and protects the copyright of teachers.

The main process of video playback by user browser is as shown in the figure above, where copyright protection is carried out by obtaining Token in step 1 and DK in step 3. Their functions are as follows:

  • The Token is used to prevent theft and can limit the expiration time of the URL and the maximum number of IP addresses allowed to play videos. The calculation method and authentication logic are customized by the service side.
  • DK is used to decrypt the encrypted slices of videos. The video fragments directly obtained by users are encrypted through AES-128, and their values are provided by Tencent Cloud Key Management Service (KMS).

Web access process

Upload video

Access method

Video upload mainly relies on voD-Js-SDK-v6 provided by the cloud, written in TypeScript, with relatively complete test cases and high code quality 👍 Its underlying dependence is coS-Js-SDK-V5, which is also provided by Tencent Cloud object storage capability.

The way to access the SDK is simple and involves only two aspects:

  1. Pass in the function that gets the signature to initialize the SDK, which will be called automatically when needed. Currently, the SDK gets a signature before, during, and after uploading.
  2. Call of the SDKuploadFunction uploads video.
import TCVod from 'vod-js-sdk-v6';

// Trigger with the signature function
const uploader = new TCVod({
  getSignature,
});

// Obtain the signature from the service background
function getSignature() {
  return fetch('FAKE_CGI_URL').then((result) = > {
    returnresult.sign; })}// Call SDK upload
function uploadVideo(videoFile) {
  const upVideo = uploader.upload({ videoFile });
  upVideo.on('video_progress', (info) => {
    // Get the upload progress here
    // For example, upload percentage, upload speed, etc
  });

  upVideo.done().then((result) = > {
    // Get the upload result here
    For example, fileId and CDN source file address
  }).catch((error) = > {
    // Upload failed
  });
}

uploadVideo(fileA);
uploadVideo(fileB);
Copy the code

While uploading the SDK is simple to use, but in the process of our gray, still have a few problems, therefore it is strongly recommended that add detailed report log in code, such as the DEMO above can join the log information includes: to get the signature of start, success and failure, the beginning of the file upload, success and failure, etc.

Problems encountered

1. Only the Chongqing storage area is enabled by default

After the launch, we found that the link to upload the video was xxx.cos.ap-chongqing.myqcloud.com. It doesn’t look right. Why are they all uploaded to Chongqing? Doesn’t it support the ability to upload locally? Later, we contacted our colleagues at the cloud and learned that since the bottom layer of the video cloud relies on the object storage (COS) of Tencent Cloud, COS guarantees the specific transmission direction and fast transmission, and relevant configurations need to be opened on the cloud console.

2. An error occurs in the SDK upload

At the initial stage of uploading, it was found that the upload success rate was 97%, and there was still a certain distance from the expected 99%. Through the joint investigation of both parties, it was finally found that two problems were mainly caused by:

  • If the local time of the user is inconsistent with the server time, cos-Js-SDK-V5 authentication error occurs, resulting in 403.
  • When the user’s network jitter occurs, the voD-Js-SDK-v6 of cloud video fails to process the signature, resulting in 403.

At present, the above problems have been solved in the latest version of VOD-JS-SDK-v6, and the success rate of upload is more than 99.5% after full upload.

PC & H5 video playback

The video playback process has been briefly mentioned before, so let’s explain it in detail here.

Introduction of the process

On-demand playback is actually very simple, simply put, is the following process:

Step 1: Get the M3U8 address

Step 2: Call the player to play

It’s that simple.

At this point we found a problem, with the M3U8 address, everyone can play. This M3U8 address can spread unbridled, anyone can get the link can play, there is no concept of paid classes. So we started to introduce the first technology mentioned earlier, which we call Key anti-theft. Anti-theft chain parameters are dynamically changing, and our process becomes:

After anti-theft link is added, links without anti-theft parameter will not play. Even if the m3U8 address with the anti-theft parameter is propagated, the link will fail after a while because of its timeliness.

At this time, smart partners should have found another problem, assuming that the anti-theft parameters before the failure of the M3U8 file download, also can be used to spread.

To solve this problem, we can briefly look at the format of m3U8.

Simply put, M3U8 is a text file that follows a format and contains an index of TS shards through which all video shards can be found.

Back to the topic of encryption, if every TS fragment is encrypted, is it not possible to download m3U8? HLS ‘normal AES encryption technology does just that. With the introduction of HLS plain encryption, the process looks like this:

For the sake of simplicity, we’re going to ignore the graph of the cosine CDN. Explain the image above:

The first is encryption, and encryption requires a key. At this time, KMS was introduced. We do not care about the internal implementation of KMS for the time being, and simply think that the work is to provide the key. After receiving the video encryption request initiated by the business background, Tencent Cloud will obtain the corresponding encryption key from KMS to encrypt the file. This is the blue text in the image above.

Then it is decryption. When the business front end gets the content of M3U8, it finds that it needs to decrypt TS, so it needs the decryption key, so it will request the business back end to get the decryption key. How does the business background think the request is legitimate? Of course, it needs to have the user’s identity information (cookie). Tencent cloud provides two ways, specific can see HLS ordinary encryption. The example above is the first one. Use an example to illustrate. Let’s look at an example of an M3U8 address:

https://1258712167.vod2.myqcloud.com/fb8e6c92vodtranscq1258712167/c896adc25285890789334843878/drm/voddrm.token.dWluPTt2b 2RfdHlwZT0yO2NpZD00MDY4NDQ7dGVybV9pZD0xMDA0ODUxNzc7cHNrZXk9O2V4dD0=.v.f3071.m3u8?t=5d2f1647&exper=0&us=77765851115272989 75&sign=195ed8bcbc08bb5e40f4823c49e71696

DWluPTt2b2RfdHlwZT0yO2NpZD00MDY4NDQ7dGVybV9pZD0xMDA0ODUxNzc7cHNrZXk9O2V4dD0 = here is to bring the business background of authentication token. Take a look at the contents of this file:

In m3U8 format, ext-X-key value is used for decryption. The cgi-bin/qcloud/get_dk in the figure above is the fifth step in our diagram, which carries the identity information and obtains the decryption KEY from the business background. After obtaining the decryption key, the TS file can be decrypted and played

Code implementation

Once you understand the flow, the code is pretty simple.

First: get the M3U8 address and concatenate the token.

async getM3U8List(fileId: string) {
  const { termId, onError } = this.props;
  try {
    // Get anti-theft parameters, corresponding to step 2 in the flowchart
    const urlParams = await getUrlToken({
      termId,
      fileId,
    });
    // Obtain the address of m3u8, corresponding to step 3 in the flowchart
    const videoInfo = await getPlayInfo(fileId, urlParams);
    // Get the M3U8 address after concatenating the token
    const m3u8List = getPlayListWithToken(videoInfo, {
      termId,
    });

    return m3u8List;
  } catch(e) { onError(e); }}Copy the code

Next, call the player, which can be referred to as super player or tcplayerlite. The documentation is more detailed, so I won’t go into it here. Step 4 is initiated by the player and step 5 is initiated by the browser itself.

Broadcast quality monitoring

As for monitoring, broadcast currently uses internal monitor + TDW + BADJS to report for monitoring.

Monitor Displays alarms and data accumulations.

TDW is used to generate statements, daily and weekly reports.

Badjs is used for troubleshooting when the playback fails.

Small program video playback

The applets side has two problems to solve:

  1. Tencent Cloud does not provide available cloud playback components for front-end use, so we need to package a component to provide cloud video playback capabilities;
  2. The applet does not have cookie, and the method of obtaining decryption key of M3U8 file is automatically completed by video, and the code cannot be controlled, so the applet can only use QueryString to transmit identity authentication information to authenticate;

Let’s first look at a basic process of Tencent cloud video playback of small program components:

  • In class, anti-theft chain and HLS encryption are enabled, so the above judgment process takes the green path;
  • TokenObj is an anti-theft token, which includes: expiration timestamp of play address, trial duration, link identifier, anti-theft signature. Reference Key anti-theft chain;
  • The drmToken is an authentication token used by M3U8 to obtain the decryption key. The encryption rules are specified by the front and back ends at the service layer. Refer to QueryString to pass authentication information;
  • <cloud-player-video />Components inside the playback or use a small program<video />Component, only provides the function to get the real play address through the parameters;
  • At present<cloud-player-video /\>It is a component developed by ourselves, and it is still in the process of continuous iterative optimization. In the future, it will add functions commonly used by players such as double speed switch and definition switch.
  1. The applet side obtains the corresponding fileId through the CGI of the business, and then obtains the corresponding tokenObj through the interface of getCloudUrlToken.
  2. The content obtained through the login interface is encrypted to generate a drmToken for authentication during decryption.
  3. Combined with the corresponding Tencent cloud businessappidAnd what we gettokenObj 、 drmToken 、 fileIdThese four key parameters are passed to the cloud playback component<cloud-player-video />;
  4. Appid, tokenObj and fileId can be used to get the encrypted M3U8 address from Tencent Cloud (through getPlayInfo), and then attach the drmToken information to the original M3U8 address (through getUrlToken).
  5. Pass the new M3U8 address to the video component of the applet, and the m3U8 file retrieved will be changed insidedrmTokenInformation is injected intoEXT-X-KEYIn the URI of the fieldQueryStringThe way to pass, eventuallydrmTokenWill be injected into the m3U8 file, the image has been pasted above, paste again

  1. The video component automatically reads the URI to get the decryption key to decrypt the TS file and play it back;

TokenObj and drmToken are obtained in the class mini program. Since the way of obtaining these two parameters is determined by business, the internal process will not be described here. The following step code is posted:

getCloudUrlToken(params)
.then(tokenObj= > {
  const drmToken = getDrmToken({ term_id: termId });
  this.setData({
    fileId,
    appId: '1258712167'.// pro
    drmToken,
    tokenObj,
  });
})
.catch(({ err_code, err_msg }) = > {
  // Downgrade playback
  this.init(this.properties.playInfo, null.true);
});
Copy the code

We then pass four key parameters to the component as follows:

<cloud-player-video
  player-id="course-video-player{{r}}"
  file-id="{{fileId}}"
  app-id="{{appId}}"
  token-obj="{{tokenObj}}"
  drm-token="{{drmToken}}"
  safety
  poster="{{poster && tools.renderUrl(poster)}}"
  bindplay="onPlay"
  bindpause="onPause"
  binderror="onVideoError"
  bindended="onEnded"
  bindmedianotsup="onMediaNotSup"! [] (http://imweb-io-1251594266.cos.ap-guangzhou.myqcloud.com/b645c306e5a3695be09104cfdb27183a.png)
></cloud-player-video>
Copy the code







FormatUrlWithToken is a method for attaching a drmToken to an address m3u8:

// The method to get the video playing address
getPlayInfo() {
  const {
    fileId,
    appId,
    safety,
    tokenObj: {
      t,
      us,
      sign,
      exper = 0,}} =this.properties;
  // The current version gets the address of playInfo by default
  let url = `https://playvideo.qcloud.com/getplayinfo/v2/${appId}/${fileId}`;
  // If anti-theft is enabled, add anti-theft information to queryString
  if (safety) {
    url += `? t=${t}&us=${us}&sign=${sign}&exper=${exper}`;
  }

  return request({ url });
}

// The method to attach the drmToken
formatUrlWithToken(m3u8 = ' ', drmToken) {
  const reg = /(\/drm\/)/g;
  let tokenUrl = m3u8.replace(/http:/.'https:');
  tokenUrl = tokenUrl.replace(reg, `$1voddrm.token.${drmToken}. `);
  return tokenUrl;
}
Copy the code

Write in the last

Although I encountered some problems in the process of going to the cloud, they were all solved smoothly, and the final product data and user experience have been improved than before. I hope more and more businesses can actively embrace the era of cloud!