As an important way of carrying content, short video is the key to attract users. The content and experience of short video are directly related to whether users are willing to stay for a long time. Therefore, the optimization of experience is very important. In our last post, we shared how to optimize short videos on iOS. In this post, we’ll talk about how to optimize short videos on Android.

Author | Shaoyang

Review | Tai one

Before optimization, hema’s immersive short video playing page has a significant gap with mainstream short video apps in terms of body sense and fluency. There are two main problems: cover flash screen and slow flow rate. Therefore, the objective of optimization is to solve the existing shortcomings of Hema’s immersive short video and align it with the immersive short video experience of mainstream apps, such as Douyin and Handtao. Specific indicators are:

  • Meet the hard indicators: playback success rate, first frame duration, second open rate.
  • Satisfy the user’s body sense fluency. (In order to reflect users’ real experience in watching short videos, Hema added a second broadcast motion indicator: from the user to the video, to the first frame of the video playing time.)

Optimization effect comparison

First of all, let’s take a look at the effect comparison with other apps before and after optimization: v.youku.com/v_show/id_X…

The environment

  • Phone: Pixel 4
  • OS: Android 10
  • Player: Taobao player

Problem analysis

The homepage the splash screen

Hema originally added a cover image display that was hidden while playing to ensure that it was not a blank page when entering the screen. As can be seen from the motion indicator, even before optimization, the motion playback time is very short, only about 200ms (excluding the sliding process). During the sliding process, the cover will be displayed about 600ms before the video is officially played, and then the playing picture will be displayed quickly. At this time, users will still have strong screen flicker and frustration, and experience is very poor. ** Solution: display the first frame of the video in the sliding process, no longer display the cover, then play no longer produce a sense of frustration. ** The optimization here needs to be combined with the slow flow problem.

Slow discharge speed (slow motion of playback)

The service side: Slow outgoing traffic speed caused by the server, usually due to large files or poor network links. availableH.265 和 CDNTo accelerate the optimizationThe client: Experience required for client playbackDownload -> Load + Decode -> RenderThree steps, and the three steps are executed linearly. Therefore, it is necessary to prepare for about 1s before playing the picture in the window. Consider early execution hereDownload -> Load + decode.

Optimization scheme selection

In the early stage of optimization, we considered three optimization schemes.

Scheme 1: dual player + pre-download advantages: small memory, simple thinking. Disadvantages: limited optimization efforts, can not take into account both up and down.

Solution 2: Customization 3 Player management + Pre-download Advantages: Flip up and down pages at the same time, and experience is similar to Tiktok. Disadvantages: player management and recycling implementation is complex, easy to confusion; The memory usage is high.

Scheme 3:3 player (RecyclerView based caching mechanism) + pre-download advantages: both up and down page turning, experience close to Douyin, caching mechanism hosted by RecyclerView. Disadvantages: High memory footprint, frequent creation and destruction of players.

Finally, because of the cost performance factor, I chose the third plan.

Principle 3: Before turning pages

  • Current player starts playing video 1.
  • Pre and Next players load video data 0 and 2 respectively.
  • Meanwhile, video data 3-7 is added to the pre-download queue.

Principle 3: After page turning

  • Holder destruction player recycled by RecyclerView.
  • Create a new next player using holder in RecyclerView onBind.
  • Current player starts playing video 2.
  • Pre player seek 0 and pause, Next player creates and loads the video 3 file.
  • At the same time, the pre-download clears the unconsumed queue, and the video data 4-8 is added to the pre-download queue to start downloading (the cached videos here will not be downloaded repeatedly).

Specific optimization scheme

Multi-player transformation

In order to solve the problem of clunking and slow flow on the sense of motion, the use of multi-player combined with RecyclerView program for transformation, the steps are as follows:

  1. Set cache number: use RecyclerView to configure setItemViewCacheSize to ensure that there are 3 holders in memory (1 holder in cache, 1 holder pre-created, current Holder).

mRecyclerView.setItemViewCacheSize(1); // Set the number of caches

  1. Override Adapter onBindViewHolder to create the player and preload the decoded video content. The player controls parsing to the first frame and pauses. At this point onSurfaceCreated has not been called back and the screen has not been rendered to the screen.
  2. Listen onPageRelase controls the pause of the player about to remove the screen, and seekTo (0) facilitates immediate play when sliding back to the screen.
  3. Listen for onPageSelected to control the player that is about to enter the screen to start playing. Note: Since the decoding was completed during onBindViewHolder, entering the screen only 1px will trigger the Surface drawing immediately (only once), i.e. the first frame of the video will be displayed as the content enters the window.
  4. Rewrite Adapter onViewRecycled, because the current holder is about to move off the screen, the off-screen holder in the move direction will be recycled. Recycle and destroy the player at this point.

Multi-player + RecyclerView schematic diagram

The three player greatly improves the experience of immersive short video and mainly solves the following problems:

  • In the process of sliding up and down, the screen is the first frame of the video, and there is no visual pause.
  • Pre-create the player before formal playback, and load and decode, saving the preparation work before playing the video. (PS: The download process is also included here).
  • Due to loading and decoding in advance, Surface will be instantly rendered when entering the screen without visual perception. Therefore, the cover image is no longer needed before playing the video, avoiding the flash screen problem caused by inconsistency between the cover image and the first frame.

Pre-download optimization

As mentioned above, the multi-player is capable of page-turning and second-playing, which is a great improvement in the experience. However, as the pre-created player needs to download the video file when loading, the time for the next player to prepare the video will increase to about 1s. If the user slides to the video before the player has finished loading and decoding, the screen will appear obviously black, bringing a very bad experience.

Because the preloading time is too long, and there is no way to predict whether the user will swipe quickly. This requires pre-download and quick swipe detection.

The first thing we need to know about pre-downloads is the internal playback process. The local proxy here is implemented by the video caching mechanism, as described in the next section.Internal flow of player

Pre-download policy

Here, in order to save the process of requesting network data, we download the first frame of the video in advance before playing, adopting the following strategy:

  • File size: Download 1MB video file way to advance the first frame download. (PS: tested 1MB already includes the first frame, and the file is relatively small).
  • Advance volume: 5 downloads ahead (with a pageSize of 10).
  • Concurrency: Downloads are performed in a synchronous queue (to avoid bandwidth occupation caused by asynchronous downloads and delay of normally played videos).
  • Fast slide optimization: Fast slide clears download queues to avoid frequent downloads triggered during fast slide.
  • Download time: loadMore pushes the first 5 entries into the queue; When onPageSelected, five videos are pushed to the queue after the next one is skipped (the next video is automatically downloaded by a preloaded player, where repeated downloads will cause a screen-dead video).

Fast smooth definition

OnPageSelected does not fire when the user quickly turns the page (onPageSelected slips again before the onPageSelected call), but onPageRelease fires multiple times. In onPageRelease, if the difference between Release Position and current postion is > 1, it means that the user has at least one quick page turn, which is defined as the state of fast slide, and the pre-download and player preloading should be stopped. When onPageSelected returns, the user does not continue to turn pages and the fast slide status is cancelled. Start performing predownload and resume player preloading.Pre-download flow chart

Cache optimization

At present, hema uses a taobao internal player. The player itself does not have file caching and pre-download functions. After the player is recreated, even the same video will not be cached and will need to be downloaded again. An open source caching framework “com.danikula: Videocache” is introduced here.

The theory

The address played by the player is proxyed to the local cache service, which forwards data streams and saves files. For example, the video address is:https://****.mp4. The local proxy address is:http://127.0.0.1:8888(Assume the port number is assigned to 8888). The proxy address is local proxy address + video address (URL encoding). That is:http://127.0.0.1:8888/https%3A%2F%2F * * * *. Mp4

Future optimization prospect

There is a lot more that can be done to optimize multimedia. In addition to immersive seconds, we can also:

  1. Second play optimization for the player’s general scenes, such as the card video list on the home page. At present, the average first frame of the home page needs 550ms, and the longer frame needs more than 1000s, showing obvious sense of frustration. On the basis of an immersive solution, we can provide general pre-download capability to reduce the player’s download rendering time.
  2. Continuous broadcast and memory optimization. Continuance is another aspect that improves the experience. Users can intuitively feel whether it is consistent or not.
  3. Page single player hosting. In most scenarios, only one player is playing on a page, which allows page player reuse by managing a single player to optimize memory and experience.

In the next issue, we will continue to share the experience optimization practice of short video continuation on Hema iOS/Android terminal.

“Video cloud technology” your most noteworthy audio and video technology public account, weekly push from Ali Cloud front-line practice technology articles, here with the audio and video field first-class engineers exchange exchange. You can join ali Cloud video cloud product technology exchange group, and the industry together to discuss audio and video technology, get more industry latest information.