Effect:
Demo apk:
Github.com/zxzx74147/E…
I. Program Framework:
ExoPlayer + VP9 + WebM package
Ii. Usage Scenarios:
Gift effects, pop-up ads, dynamic stickers, etc
Iii. Comparison of schemes
1, WebM transparent video vs GIF, webP, APNG
GIF, WebP, APNG comparison between here skipped, from the coding quality to the platform support point of view (GIF quality is poor, APNG platform support is poor), which webP is comparable, WebM and WebP are Google, but one is a video, one is a picture. Webm is vp8 or VP9 encoding. Webp is VP8 encoding, and VP9 has a higher compression rate.
Compared to using WebP to implement related functions
This scheme has several advantages, one, support streaming playback, that is, side down broadcast, also do not need to load the full memory; Second, support audio track, do not need to add background sound to use an audio player, also do not have to consider the problem of audio and picture synchronization; 3. Video-based control logic, such as progress bar control, speed control, event callback and other mature interfaces. Fourth, there is no limit to the length of time and file size compared to the picture format. Disadvantages also have, that is, the video format is more “heavy”, the use of a little cumbersome.
2. Webm transparent video vs SVGA and Lottie
SVGA (YY), Lottie (Airbnb) and other drawing schemes based on Native Api are also a very mainstream animation implementation scheme. Compared with View, animation performance and development convenience will be improved substantially. But there are some problems in the process of our use.
SVGA and Lottie schemes have several shortcomings, which also prompted the birth of our transparent video scheme: first, the expressive force is not rich enough, some complex particle effects may cause stencin, and 3D effects cannot be realized (pseudo-3D which can only do affine transformation); Second, for the above problems, many animation effects use a large number of frames, resulting in large files. In addition, there is no video or WebP decoding and rendering mechanism, all frames have to be decoded into Bitmap, resulting in very serious memory problems and OOM probability increase; Third, the limitation for development tools is generally Adobe AE, and the feature support is not complete (Lottie is better).
Compared with SVGA and Lottie, the biggest disadvantage of WebM is that it cannot support dynamic content, such as dynamic picture replacement, dynamic text replacement, element editing and other high-level functions. The advantage is strong performance, no performance problems, as long as the effect can be made to the video format without brain playback.
3. Webm transparent video vs video split screen solution
There is no difference between these two schemes in essence. They are both original RGB information +Alpha information stored separately and then rendered. Alpha information is stored redundatively in extra RGB three-channel data, which is more or less wasteful. One of the advantages of webM scheme is that Alpha is stored as a payload, and the original video information remains unchanged. Even if you use the unmodified Exoplayer, you can play opaque videos. You don’t need to modify the logic of Render layer too much. It can be used directly in the Chrome kernel browser. The left and right split screen scheme of video is generally left RGB and right Alpha, which requires some Shader code to do rendering. Transparent video and non-transparent video are divided into two sets of rendering logic, which is a little cumbersome to use, and cannot be rendered correctly in non-modified player.
In order to ensure the principle of minimal modification, the current solution uses the VP9 extension of Exoplayer, which uses libVPx library to do decoding operations, and does not support hard decoding for the time being. So far, there is no problem with decoding performance. Follow-up can consider adding hard decoding to improve performance.
Four, use method
Github.com/zxzx74147/E…
Based on Exoplayer modification, no API call method has been changed. The modification point can be referred to the submission record. The change is not significant.
The AAR can be introduced by source compilation or directly generated based on the branch R2.11.8-Transparency.
Vi. Appendix:
1, ios13 + transparent video broadcast scheme based on hevc: developer.apple.com/videos/play…
2, ffmpeg conversion command (note the bold part) :
Mov -vf scale=640:360 -b:v 1.5m -c:v libvpx-vp9-vcodec libwebp /path/output/file.webp