Source: bz.zzzmh.cn/

Author: Wang Yongliang

In the revised version 8.0 of netease Cloud Music, I received the demand that I could click the “small window” button to put the video in the Mini player bar and continue to play it. When I first received the demand, I was devastated. To know that the Mini player bar of netease Cloud Music is a View that may appear on any Activity in the App. When jumping between different activities, how to ensure that the video can be transferred from one Activity to another “seamlessly”?

MediaPlayer in tie

For simple video playback functions, I will use the system’s own VideoView. It only takes a few lines of code to make the video play. The system’s own VideoView inherits from SurfaceView, and calls the MediaPlayer, The bindings that include the Surface and MediaPlayer are wrapped in there, which has the advantage of being easy to use, but there are a few problems, the SurfaceView and MediaPlayer are completely tied together, One MediaPlayer can only correspond to one SurfaceView, and what we want to do with a small window is that MediaPlayer and SurfaceView can be one-to-one, and when the page switches MediaPlayer can bind to a new SurfaceView, It’s like a computer with multiple monitors. Our video playback frame solves this problem well, as shown in the picture below:

Since there are some scenes in the App that animate the video, TextureView is used in the frame. TextureView and MediaPlayer communicate using AIDL, as shown below:

As can be seen from the above two figures, all mediaplayers in the Video player frame are managed in a separate cache pool of the Video process. Those in use are stored in the Active pool and idle in the idle pool. When idle Mediaplayers exceed the upper limit, they will be recycled. When a new page is launched, VideoView can get the unused MediaPlayer from the pool of the Video process. Videoviews in other processes communicate with MediaPlayer in the Video process through AIDL.

This architecture makes it easy for videoViews in different activities to replace the MediaPlayer bound to them. Since the playback capabilities are all in MediaPlayer, So when MediaPlayer is unbound from TextureView, there is no interruption of playback. When a new page starts, just rebind the MediaPlayer that is playing with TextureView, and the new page will immediately display the image that is playing. In fact, the video playback frame was originally designed for the following reasons rather than serving seamless playback scenes:

  1. MediaPlayer is not as stable in its early days, and using multiple processes prevents player exceptions from affecting other functions of the main process
  2. Normal VideoView can only support one TextureView and one MediaPlayer, while video playback optimization requires additional video player capabilities to implement video preloading
  3. This reduces the memory usage of the main process and prevents important services such as audio playback from being affected by video playback
  4. Player reuse reduces object creation

Therefore, based on the above requirements, we designed this set of isolation scheme between MediaPlayer and TextureView. If it is a relatively simple scenario, singleton holding MediaPlayer can also be considered. Since MediaPlayer and TextureView are already well isolated, we can easily support the unbinding of VideoView and MediaPlayer by adding some methods to the MediaPlayer pool to get the player to be reused. From get the effect of seamless playback.

The specific process of replacing and binding MediaPlayer is as follows:

In the original Activity, if the player was to be reused, we saved the unique ID of the player and the ID of the playing resource in a global location as a sign that the player was reusable. When the new page is started, the VideoView of the new page is created. In the new page, the setDataSource of VideoView is called to set the content to be played. SetDataSource finds the current player in the pool based on the current player content and the saved global player ID, and sends the Surface to the reused MediaPlayer via AIDL for rebinding, so that without interrupting the current player, The video playback screen is seamlessly transferred to the new Activity. One thing to note is that Surface itself supports cross-process delivery:

public class Surface implements Parcelable
Copy the code

In addition, this solution uses the hashCode of the MediaPlayer object as the unique ID of the player. If you use this solution, you can also design the unique ID for your own situation.

The core of the binding solution is the rebinding of MediaPlayer and VideoView. The rebinding only requires the following two steps:

  1. Use the onPrepare and onPause callbacks defined on the current page to replace the original callbacks for the player
  2. Note that sometimes SurfaceTexture may not be ready immediately. If not, you can rebind SurfaceTexture in onSurfaceAvailable.

This solution can basically meet most of the requirements of seamless playback, but it is not without its disadvantages. This solution mainly has the following problems:

  1. When the video is paused, MediaPlayer rebinds TextureView and there is no content on the Surface. In this case, you can first cover TextureView with the video cover and then remove the cover when the video is replayed.
  2. In the original scheme design, AudioFocus acquisition was encapsulated in VideoView. When the new page started playing with MediaPlayer, the page that originally held MediaPlayer still held the reference of MediaPlayer. AudioFocus preemption () : pause MediaPlayer () : pause MediaPlayer () : pause MediaPlayer () : pause MediaPlayer () You might want to look for similar problems when you use this solution.
  3. The original player will be recycled after the page is destroyed, but it cannot be recycled in reuse scenarios. In this case, attention should be paid to avoid player leakage.

The final implementation effect is shown as follows:

“Fake” page switching scheme

In addition to the binding scheme, netease Cloud Music also has some other seamless playback scheme implementation. First of all, this paper introduces a relatively simple implementation, which is also a scheme used in netease Cloud earlier. The “fake” page switching scheme, as can be seen from the name, this scheme does not really jump between activities. Instead, it uses TextureView’s ability to move and animate like a normal View. It uses transition animations to make it look like it jumps from one page to another, as shown below:

In the video Feed stream of netease Cloud Music, when a video is playing, clicking the hot area can “expand” to the playing details page without pausing the video. The specific implementation method is to place the View of the video playing in the Fragment. The Fragment Container is placed at the top of the entire ViewTree. When clicking Play, the Fragment will be moved to the position where the video needs to be displayed and started to play. When clicking to enter the details page, you only need to perform pan and zoom animations for the Fragment. Add comments and other fragments under the Fragment of the video. Here you can refer to the Android native VideoView encapsulation idea to achieve:

public class VideoView extends SurfaceView
        implements MediaPlayerControl, SubtitleController.Anchor {
Copy the code

Reference VideoView source SurfaceView can be replaced with TextureView, corresponding processing next onSurfaceTextureAvailable callback again can. This scheme is widely used, such as jingdong, Taobao and other product details of the introduction video. Although this solution is simple, it has great limitations. It can only solve the scene in the same Activity. If the requirement is to seamlessly play and switch between different activities, this solution cannot be satisfied.

Player Seek scheme between different pages

Another solution to achieve seamless playback across activities is to use a new player to re-open resources in the new page when opening a new page, and then re-seek the resources according to the original saved progress. This solution does not guarantee the real “seamless” playback. After all, it takes a couple of hundred milliseconds for an Activity to start, but the biggest advantage of this solution is that some of the old logic can be seamlessly played with very little change. For example, in some pages that are not very important, the video playback function may already exist and the playback logic may be coupled with heavy business logic. At this time seek scheme is more appropriate.

This solution is simple but there are some caveats:

  1. Cache reuse improves experience. When playing online videos, you can use playback cache reuse to improve user experience. Video playback cache can be implemented in the form of URL proxy download. The general approach is to start the Local Http Server to proxy the video playback request to the Local Server. In the local server to store the video file to the specified location, this is also a common solution in video playback, the cache function can refer to AndroidVideoCache.
  2. The MediaPlayer provided by the system can only Seek out key frame problems. When the system player is used to reopen the video resource SEEK, sometimes the video cannot seek to the original playing position, or even directly jump to the beginning or end of the video playing position. The shorter the video, the higher the compression rate, the more obvious it will be, which is the problem of key frames. The solution to the keyframe problem is to implement the player based on MediaCodec, which can refer to or directly use Google open source ExoPlayer.

The key frame is called I frame, which can be regarded as an uncompressed frame, and there is no need to rely on other frames when decoding. There are compression frames such as B frame and P frame between the key frames, and the complete picture can be decoded by relying on other frames. The interval between two key frames is called a GOP. – Frame system players in GOP cannot seek directly.

4. View cross-activity reuse scheme

View reuse across activities refers to manually using the ApplicationContext to create a View that needs to be reused, and using the singleton Manager to hold the View. Adding and removing reusable views can be implemented in the Activity lifecycle functions. Example code is as follows:

object Manager : ActivityLifecycleCallbacks
    override fun onActivityStarted(activity: Activity) {
        ...
        removePlayerBarFromWindow(activity)
        addPlayerBarToWindow(activity)
    }

    override fun onActivityPaused(activity: Activity) {
        ...
        if (activity.isFinishing && getMiniPlayerBarParentContext() == activity) {
            removePlayerBarFromWindow(activity, true)
        }
    }
Copy the code
private fun getPlayerBar(activityBase: Activity): MiniPlayerBar { synchronized(this) { if (miniPlayerBar == null) { miniPlayerBar = MiniPlayerBar(activityBase.applicationContext) } ... return miniPlayerBar!! }}Copy the code

Theoretically, this is a more flexible solution, and there is no need to worry about leakage when using the Application as the Context of the View. However, due to the requirement of the small window, it involves the reuse of the old page and the new page, so it is not a unified playback View in many scenarios. Therefore, this scheme is not adopted. However, this scheme has been used in the Mini player bar of Netease Cloud Music App, and interested partners can also try it.

conclusion

The above is a summary of some seamless playback schemes in netease Cloud Music. It mainly introduces the realization ideas of several seamless playback capabilities in netease Cloud music for reference in scheme selection. If you have other schemes, you are also welcome to exchange them. The schemes in netease Cloud Music have gradually evolved from simple to complex. With the continuous iteration of the requirements, it is not necessary for me to pursue a large and comprehensive design scheme, but the one suitable for the current scene is the best. A good architecture not only depends on good design, but also on continuous improvement and optimization.

This article is published from netease Cloud Music big front end team, the article is prohibited to be reproduced in any form without authorization. Grp.music – Fe (at) Corp.Netease.com We recruit front-end, iOS and Android all year long. If you are ready to change your job and you like cloud music, join us!