I. Background introduction

1. VAP (Video Animation Player) is a Video Animation special effects SDK used in live broadcasting medium. It can make Video materials separated from Alpha channel and then remix Alpha channel and RGB channel through OpenGL ES on the client. In this way, the video with transparent channel can be played on the end.

Apps that have been accessed

The same principle applies to other apps

Douyin, Watermelon Video, Toutiao, Iqiyi, Bixin, etcCopy the code
2. Scheme comparison

At present, the most common animation implementation schemes are frame animation, GIF/WebP, Lottie /SVGA. For the realization of complex animation effects, a simple comparison is made

plan The file size Decoding way Special support Application performance
Lottie/SVGA Unable to export Soft solution Some complex effects are not supported Drawing time and memory jitter
gif/webp 4.6 M Soft solution Only 8 bit colors are supported File resource consumption is high
apng 10.6 M Soft solution Full support High resource consumption
vap 1.5 M Hard to decode Full support Good performance, good complex animation support
3. Operation effect

4. Main contents of this paper

This paper mainly introduces how VAP is implemented, from MediaCodec video decoding to OpenGL ES rendering RGB and Alpha channel, and finally output to TextureView.

Second, VAP implementation architecture

1. Required technology
  • OpenGL ES: Create textures and draw work
  • TextureView: Android UI component that holds SurfaceTexture, listens for SurfaceTexture size changes, and does display area updates
  • SurfaceTexture: Holds textures created by OpenGL ES, listens to the decoded frame data for frame updates, and tells OpenGL ES to draw
  • MediaCodec: Decodes, binding decoded data to SurfaceTexture
2. Implement flow charts

image

3. Overall workflow

To make it easy to see the responsibilities of each class and the sequential execution of the flow, the swimlane diagram is used here: image

Third, other

  • In this paper, the source code
  • Byte correspondence implementation
  • Flutter version of vap

End, scatter flower 🎉