Anton Garcia Diaz

Translation: Crazy geek

Original: www.freecodecamp.org/news/video-…

Reproduced without permission

Online video has always been hot. While it’s easy to embed Instagram and Youtube videos on a page, there’s a growing need — like in many e-commerce scenarios — for custom video delivery methods.

When setting up a video processing and transmission pipeline, the first consideration is the video format to be served. Aspects such as user experience, support (browser and system), compression efficiency, or coding speed may be relevant to this choice.

Based on my experience with web business media optimization, I will try to highlight the main aspects to consider. If you are looking for simple transcoding and optimization options for using FFMPEG, you can also check out this article.

Containers and codecs

It is important to be aware of the differences between containers and coding standards as compared to normal image formats. The file extension only tells us which container it belongs to, not which codec it uses. The coding standard followed determines whether the browser or system supports it.

For example, although Web video formats are generally encoded using the MP4 container and the H264 standard, not every MP4 file is universally supported because it may use a different standard encoding, such as H265.

It becomes even more sophisticated in terms of adaptive bit rate (ABR), which brings the best way to respond to the user’s network and device functions.

Let’s take a look at containers, the main combination of coding and delivery standards, and how they differ in support, compression efficiency, coding speed, and user experience.

Progressive video

H264/AVC

The king of video formats uses the MP4 container with H264/AVC encoding. Sometimes you’ll also see it in the M4V container (the default format in Handbrake), Apple’s MP4 derivative for H264 video with DRM protection.

Every browser and system — as well as native apps in iOS and Android — supports this format. This is a safe choice to avoid compatibility issues.

In addition, almost all desktop and mobile devices support H264’s hardware acceleration. Encoding and decoding speed is very fast. ,

All in all, this format is very simple to encode and use. As with images, you can simply plug in a video link using HTML5 and use it in any browser.

Around 2000 KBPS and a delay time of more than a few seconds may affect visual quality. Pauses and re-buffering may occur when viewed over mobile networks or during peak network hours. Using a scheme that reduces image quality will result in artifacts such as blurs, floaters, or clumps.

H265/HEVC

This is a powerful video format that uses the same container and is encoded with H265 HEVC, resulting in higher compression efficiency (about 50% volume reduction) with much less problems other than blurriness. The main problem with this format is that support is limited to Apple devices, including the high royalties that are included in their price. It’s almost exclusively available in Safari and iOS apps. If you have a lot of iPhone or Mac users, you can use it as a backup to H264. Their experience will be better.

Even with hardware acceleration (almost exclusively available on Apple devices), the higher complexity of this format means that encoding is significantly slower, so generating delivery files requires more computation and time.

VP9

This is a free, open source video format provided by Google. It uses the WebM container instead of mp4, which is basically the MKV container, but sets the encoding standard to VP8 or VP9. H265 offers similar benefits, perhaps less efficient but still much more than H264. Again, it allows for a reduction in size, with artifacts that are much smaller except for blur. Encoding speed is similar to H265, which is slow.

Note that while the previous version (VP8) has the same support, we don’t recommend it at all because it doesn’t bring any benefit to the already widely supported H264. Webm can only be used with VP9 encoding.

Of course, webM support is limited to the Google world. This means only Chrome and Android will support it.

AV1

The first stable version of the standard was released in March 2018, which included mapping between MP4 and MKV containers. It can offer similar or slightly higher compression efficiency gains than the H265 and is licensed free of charge. The final implementation also improved decoding speed compared to H265, and AV1 is a compelling alternative to Web video transmission

The Open Media Alliance, which helped create the format, is committed to providing extensive support in the near future.

Open Media Alliance partners participating in AV1

However, the implementations currently available should still be experimental, and the bottleneck remains coding speed. The lack of hardware acceleration is clearly a problem and the first solution is expected by the end of this year.

VVC

The committee responsible for H264 AVC and H265 HEVC is fast tracking the new standard, which is expected to be released in 2020. Preliminary tests of the currently considered method have shown a significant increase in performance compared to H265 and AV1. I include it here as a futuristic possibility, just to show that the video coding race seems far from over.

Adaptive bit Rate (ABR)

This is a very interesting alternative to the progressive format. It is based on HTTP – based media communication protocol. This method provides the video as the main playlist. Playlists offer options with different resolutions and bit rates for different viewport sizes, network bandwidths, and devices.

In addition, the video is divided into fragments or blocks so that clients can jump from one quality level to another. It can adapt to the user’s current conditions, namely network speed, as well as the viewport size, such as switching to full screen.

Abrs bring huge advantages to optimizing the user experience on mobile devices, avoiding pauses or re-buffering under the mobile network. If you’re looking for truly responsive behavior, this is the obvious approach to take. It has two main standards: HLS and MPEG-DASH.

Although there is a common belief that ABR only makes sense for very long videos, in my experience there are many cases where fairly short clips can benefit from this approach.

HLS

Developed by Apple, this ABR protocol relies on different renditions split in MP4 format. Originally H264, H265 is now supported. As a compromise, HOWEVER, I recommend sticking with H264 encoding for HLS, since it allows for better compatibility in a variety of client cases.

A key focus of this standard is support for recent Apple devices. For Safari or clients other than native iOS apps, you need a Viewer. But that’s not a big problem because there are good open source options like VideoJS. Or you may need to put some effort into customizing it and using it in your own front-end applications. There are also great transcoding and delivery services that make it easy for you to do all this.

Since each playback should be encoded at a constant bit rate, I recommend using HLS in conjunction with per-title encoding. That is, the bitrate of video playback is selected based on the content of the video.

MPEG-DASH

This is a codec-independent protocol for ABRS, so in addition to H264 and H265, it can be encoded with VP9 or even new alternatives such as AV1. The downside is its relative youth, which means less support compared to HLS. That’s why we don’t recommend it for most Web businesses.

conclusion

The advantages of H264 AVC compression have been evident for years, and new approaches are adding momentum to Web video. Competition in display size and resolution has led to the development of new formats that offer more content at the same bandwidth.

VP9 in WebM has a significant compression improvement (about 30%), no copyright issues, and is supported by Google Solutions (Chrome, Android). Furthermore, compared to H264, H265/HEVC uses only half the bit rate to achieve comparable subjective quality. Since none of them are universally supported, H264 is still needed, at least as a fallback.

Adaptive bitrate is a compelling alternative that offers an unmatched user experience. In this regard, HLS has enjoyed widespread support with the help of open-source Viewers. It may be the best choice for mid-sized networks. Open source initiatives such as VideoJS, as well as third-party services that offer very competitive prices, significantly reduce the complexity of the viewer requirements. If you take the last technical route, be sure to ask for per-title encoding.

Welcome to pay attention to the front-end public number: front-end pioneer, access to front-end engineering practical toolkit.