In the era of short video, it should be considered for each product to provide short video capability, have good user shooting and watching experience, and make short video really help business development. How to make your app capable of short videos quickly, so that short videos in your app can have a better user experience? Today, we will sort out what technology-related issues we need to consider to build a complete short video service from the complete technical link. Let’s start with a picture:

The figure above shows a life cycle of a mobile short video, which includes the following key steps: mobile adaptation shooting + editing, video file uploading, video file processing (storage, transcoding, video understanding), video distribution (CDN acceleration), and short video playback. So let’s break down some of the issues to consider at each stage in a short video scenario.

How to select a short video SDK

Why choose an SDK instead of developing one? Since developing a short video SDK from scratch has a high requirement on the professional ability of audio and video developers, and many mature short video SDK can be found in the market today, so it is not recommended to develop by ourselves. But with so many SDKS out there, how do we choose? We can evaluate the advantages and disadvantages of an SDK in the following aspects:

1. Function richness

Currently, the main functions of short video SDK provided by the market, such as Kuaishou and Douyin, have been basically aligned, and each of them will provide some features of their own, such as Aliyun to provide real-time dynamic posts, photos and videos mixed import, etc. You can choose according to your own products;

2. Performance

Said to the mobile performance everyone usually from CPU usage, memory usage, power, etc, to make their own evaluation, but for a special scene short video SDK is users to quickly complete a short video filming and editing, we need to take full advantage of the performance of mobile phones including CPU, GPU hardware resources to complete The final video synthesis, As a result, CPU usage tends to be higher during actual composition. When we measure the performance of a short video SDK, we usually focus on two questions: first, who can shoot a video with a larger resolution and a higher frame rate, and second, who can import and synthesize a video with the same resolution, the same duration and the same frame rate in a shorter time? Here involves the decoding performance of short video, rendering performance, coding performance and overall multimedia framework scheduling performance which is better;

3. Video quality

Shooting a clear, smooth, and small enough file size video is what an SDK or a product needs to pursue.

4. Overall stability

The mobile environment is extremely complex, especially on the Android platform. The stable operation of SDK on various platforms and mobile phones is an important indicator of stability. Generally, the stability of an SDK is evaluated from the Crash rate. At present, few SDKS in the market publish their Crash rate. Users can make an assessment from the apps connected to the SDK.

5. Scalability

Is there a rich enough API offering to enable some personalized feature development?

6. Package size

The size of the entire application installation package directly affects users’ download, update, installation time and mobile phone storage space, and further affects the promotion cost of the application. Therefore, the package size of the introduced third-party SDK also needs to be paid close attention to.

At present, some SDKS directly use hard coding and hard decoding in short video SDK in order to blindly pursue the package size, so as to abandon ffMPEG and other third-party packages to reduce the package size. I think it is not advisable. First, there are many adaptation problems in hard coding and hard decoding, which will cause users directly unable to use the short video function. In addition, in some models, the ability of hard coding and hard decoding is not necessarily stronger than the ABILITY of CPU, so it is still necessary to choose a reasonable codec according to the model adaptation.

Price of 7.

Now the market generally has its own pricing strategies for short video SDK, mainly including the following: common functions bound to the cloud for free trial, and advanced functions for several hundred thousand fees. In addition, if there is a large consumption in the cloud, you can get a certain discount.

After comparing these dimensions, I believe I can quickly determine a suitable short video SDK.

Now let’s look at how we choose cloud services in the short video scenario. What issues do we need to focus on when making choices?

Second, cloud service selection

1. The store

Elastic expansion of capacity and processing capability, and security and reliability are required for a storage service. Therefore, a reliable cloud storage service is preferred.

Ali Cloud OSS: massive, secure, low-cost, highly reliable cloud storage service, providing 99.99999999999% data reliability, using RESTful apis can be stored and accessed from any location on the Internet, elastic expansion of capacity and processing capacity, a variety of storage types for comprehensive optimization of storage costs.

2. Media handling

In order to make the video generated from the client clearer and the processing capacity of the mobile terminal is limited after all, the video file we call the original is relatively large. When we upload the file to our cloud for a large number of users to watch, we need to pay attention to the following issues:

• 1. How to reduce the overall bandwidth traffic cost;
• 2. Are there any compatibility problems when playing videos generated on mobile terminals with complex environments?
• 3. How to improve users’ viewing fluency experience and reduce the lag rate in complex mobile networks?
• 4. How to reduce the traffic cost for end users?
• 5…

All of these problems make us need to do a unified video processing in the cloud to ensure a unified video format to improve the playback compatibility, and at the same time, we need to reduce the size of the file as much as possible without reducing the clarity.

Tip: general video processing (transcoding) need a certain time consuming, according to the video length varies from a few seconds to a few minutes, but in the short video scene often need a quick consumption, in this scenario is usually asynchronous processing can be used to do, let the user to watch glass, wait for after the completion of the transcoding to send under various definition of video to the audience.

Ali cloud point play service only requires a few simple configurations to complete all the above capabilities, in addition to these capabilities, voD service is a one-stop voD solution integrating powerful media asset management capabilities, video editing capabilities, detailed data statistics capabilities, distribution and acceleration.

Iii. Player

The final consumption of video is to have an excellent player. When choosing a player, in addition to paying attention to CPU usage, memory usage, power consumption and some traditional playing capabilities, there will be several special requirements in short video scenes, mainly including the following:

1. An immersive playback experience like Tiktok is inseparable from the player’s ability to start playback quickly (in seconds);
2. Short videos often have more demand for loop playback, need to consider whether the end of the loop is seamless and in order to save traffic player cache ability;
3. Playing behavioral data is very important for video products. Whether the player provides burying point, reporting, cloud analysis and final report presentation of behavioral data is also crucial.

Tip: There is a well-known open source player iJkPlayer, but the development of various optimizations and features requires professional development resources. In addition, the major cloud manufacturers have also provided their players. It is suggested that the terminal and cloud can be used together, so as to give full play to their advantages to achieve better playback experience;

After solving the above several technical problems, we can basically complete the development of the whole short video products.

4. Video AI

Many short video scenes are UGC business scenes, and users can shoot and upload their own short videos to the platform at will. And in domestic context of video for yellow, politics, wade, advertising, repeat the video review, in order to make video management and recommend is the basis of the need for multi-dimensional video content understanding and marking, these problems is to consider, in the early stage of the product under the circumstances of less amount of video review and video can be artificial operating personnel for marking. With the development of the product, it’s almost impossible to do these two things purely by hand in a large number of short video scenes. So video AI is an issue that has to be considered.

The original link