Bytedance has developed an open source VIDEO animation special effects SDK (AlphaPlayer), which can make video materials separated from Alpha channel and then remix Alpha channel and RGB channel through OpenGL ES on the client, so as to play video with transparent channel on the client.
AlphaPlayer has starred 650 on Github and has 277 branches
Douyin, Douyin Volcano version, watermelon video and Toutiao have been connected.
The package for designers to significantly reduce the production cost of special effects, for the client has more reliable performance and stability, and compared with cocos2d engine has a lower entry barrier and maintenance costs, for the realization of complex animation provides a new way, a new complex animation will become simpler and more efficient.
The effect
AlphaPlayer
AlphaPlayer is a video animation SPECIAL effects SDK used in live broadcasting medium. It can make video materials separated from Alpha channel, and then remix Alpha channel and RGB channel through OpenGL ES on the client, so as to play video with transparent channel on the client.
The package for designers to significantly reduce the production cost of special effects, for the client has more reliable performance and stability, and compared with cocos2d engine has a lower entry barrier and maintenance costs, for the realization of complex animation provides a new way, a new complex animation will become simpler and more efficient.
background
The effect of the original gift animation in the live broadcast project is realized through the CoCOS engine. Most of the animations are combined by a series of rotation, translation and scaling, so the animation effect that can be realized is relatively simple and the development cost is high. In order to enrich the expression form of animation and reduce the development cost, we introduced the animation implementation scheme of AlphaPlayer.
Scheme comparison
At present, the most common animation implementation schemes include native animation, frame animation, GIF/WebP, Lottie /SVGA and COCOS engine. For the realization of complex animation effects, a simple comparison is made
In scenarios where complex animation effects are efficiently implemented, we have fewer alternatives and can focus our discussion on Cocos2d, Lottie, Webp, and this article’s AlphaPlayer.
Lottie
Lottie is a very preferred multi-platform animation effect solution. Its simple principle is to parse the JSON file exported from AE animation into each layer object, draw it into the corresponding Drawable, and finally display it on the View. When mask and mattes are not involved, the performance is optimized, and the main time is mainly focused on Canvas#draw(). When parsing JSON, the method of stream reading avoids OOM problems caused by loading all JSON data at one time.
But there are also some shortcomings:
- If the animation involves mask or mattes and other features, 2 to 3 temporary bitmaps need to be generated to achieve the animation effect, which is easy to cause memory jitter, and the involved canvas#saveLayer() and canvas#drawBitmap() will bring extra time.
- If an image is used directly in the animation, the first decoding of the image in ImageAssetManager is done in the main thread (it is understood that on the iOS version, using images leads to higher memory and CPU performance consumption).
- The main time is still in draw(), the larger the drawing area, the longer the time;
- The current AE features are still to be improved, and some features may not work on earlier versions, so you need to avoid designing resources. (Supported After Effect Features)
In practical use, the most common full-screen boot animation for the first start will basically contain the first three shortcomings mentioned above, and in this case, performance will deteriorate substantially. Therefore, Lottie is not a suitable implementation for complex special-effects animations in live scenes.
Cocos2d-x
Cocos2d-x supports a lot of game functions, such as sprites, actions, animation, particle effects, bone animation, etc. Developers can freely achieve a variety of posture animation effects, plus its own cross-platform ability and lightweight, and Lua as a development language. It can be said that it is very suitable to embed mobile terminal as the animation implementation of the game engine.
However, there will be many problems in actual use and maintenance:
- The volume is large, and the size is about 2M even after cutting. It is difficult to allow access if it is not required by the core scene.
- Have high requirements on the developer’s technology stack;
- Fast iteration cannot be satisfied;
- Maintenance is difficult, especially when it comes to compatibility with Android models.
Webp
Webp can reduce the size of PNG and JPEG formats by 25%. It also supports 24-bit RGB colors on mobile platforms. The disadvantage is that the resource volume is relatively large, and the soft solution efficiency is low, and the low-end machine has obvious stuck problem.
AlphaPlayer
Compared to the above solutions, AlphaPlayer has a very small access volume (only about 40KB), and a very high degree of restoration of animation resources. Resources are created without considering whether special effects are supported or not, and it is very friendly to developers and designers. By adding ExoPlayer or developing your own player, you can solve the compatibility problems that may exist in some system players and bring better decoding performance.
The basic principle of
AlphaPlayer mainly has two core parts. One is MediaPlayer, which is responsible for decoding each frame of the video and can be realized by the access party. The other is the VideoRenderer, which mixes each parsed frame into the alpha channel and outputs it to the Surface. View uses GLSurfaceView, which performs better than TextureView, but is limited to the top level.
The video resource exported by the designer will contain two parts of the content — transparent mask and the original video. The alpha value is then mixed by shader. See Frag.sh and Vertex.sh for details.
portal
Address: github.com/bytedance/A…