The SHORT video SDK mainly contains two core functions: “Video recording” and “video editing”.

Video recording includes: video collection, beauty, filter, camera switching, video and audio collection parameter setting and other functions;

Video editing includes video import, composition, special effects, filters, subtitles, stickers, music, playback, export and other functions.

I. Short video SDK framework

As shown in the figure above, the technical framework of short video SDK is divided into three levels, namely core processing layer, business logic layer and business interface layer:

  • Core processing layer: responsible for the technical implementation of the bottom layer of short video, including media combination, GPU data input, video transition, video effects, video pixels, adding subtitles/stickers, image to video, GPU data, audio and video processing, audio reverberation, audio adjustment, audio delay, video preview and media synthesis;
  • Business logic layer: responsible for the realization of short video business logic, including recording logic processing and editing logic processing two parts, including material timeline management, audio timeline management, transition effect management, subtitle layer management, sticker layer management and video effects management;
  • Service interface layer: Interconnects upstream services, including recording interfaces and editing interfaces.

Second, video recording

Video to record the implementation of the process as shown above, the first open the camera on the picture and sound collection, support to sampled respectively in the process of collecting data for video and audio processing, such as filter, noise reduction, add stickers, mixing, processing into video editing link, after the completion of the final through ffmpeg coding and generate the MP4 files. The whole video recording process uses GPU and hardware coding, which makes the CPU usage very low and can run well even on poor performance mobile phones.

Click on the”readingTo learn more about the short Video SDK

Video editing

Video editing refers to the content of the video for recording or selected for editing processing, the business process first import will need to edit the video content, and then respectively for video and audio processing, including adding filter, subtitles, stickers, video cutting and copying, video transmission and rotation, add background music, mixing, etc., finish processing of audio and video media can be instant preview, Finally, the edited media files are synthesized and exported.

1. Video processing

The realization process of video processing is as shown in the figure above. The input video content is clipped, segmented, copied, rotated, filtered and other video transition and special effects processing. In addition, subtitles and stickers can be added.

Video processing mainly uses FFMPEG for corresponding processing, decoding the original video, and then mapping the decoded YUV data to the TEXTURE of GPU for corresponding parameter adjustment, and then editing the adjusted parameter configuration.

2. Audio processing

The realization process of audio processing is as shown in the figure above. Firstly, the audio track needs to be separated from the video source to form the original audio track, and then the added audio track of background music is inserted into the original audio track. By adjusting the volume of the original and background music, the mixture of original audio and background music is formed. Finally, the mixed audio track is combined with the processed video track through media merging.

3. Media preview

The realization process of media preview is shown in the figure above. The variable content such as video content, audio content and background music is processed through the customized composite thread, including information transfer and transition, etc. Finally, the processed video and audio are instantly previewed through the player.

4. Media synthesis

Media, the realization of the synthesis process as shown in the above, the first read after processing according to the time chain need to merge multiple video and audio content after mixing content, and then began to merge the content of the video and audio content respectively, merger is implemented as to the content of video and audio content to do coding based on ffmpeg, incorporating encoded audio for the overall media files, Finally, the external output is merged into mp4 files.

4. Application scenarios of short video

1. Display of e-commerce products

The merchant side can quickly merge, cut and edit the videos or pictures shot by the short video SDK, and make short product demonstration videos with subtitles, stickers, music and other special effects to show products in a more vivid way and improve the shopping experience of users. Combined with JINGdong cloud vod products and player SDK, an end-to-end solution for short video can be formed.

2. Social creative sharing

Short videos help users express their opinions more vividly and deliver a more realistic scene. In addition, it can also make new videos by using special effects such as beauty, filter, subtitle, stickers and other functions such as merging and editing. It can combine personal creativity or focus content into a new short video to improve video hotspot and click rate.

conclusion

Above is the iOS side short video SDK main technical realization way, jingdong cloud short video video on demand product SDK with the clouds, can provide complete short video solutions for clients, help the user to focus on the business itself at the same time can have a short video business support ability quickly, welcome everybody in jingdong cloud website experience a short video download DEMO and integration.

Click on the”readingTo learn more about the short Video SDK

Welcome to”Jingdong cloud”Learn more