H.265
H.265 is a new video coding standard developed by ITU-T VCEG following H.264. The H.265 standard is built around the existing video coding standard H.264, retaining some of the original technologies, while improving some related technologies. The new technology uses advanced techniques to improve the relationship between code stream, code quality, latency, and algorithm complexity to achieve optimal Settings. The specific research contents include improving compression efficiency, improving robustness and error recovery ability, reducing real-time delay, reducing channel acquisition time and random access delay, and reducing complexity, etc. Due to algorithm optimization, H.264 can realize standard definition (resolution below 1280P720) digital image transmission at a speed of less than 1Mbps; The h.265 can deliver 720P (resolution 1280720) ordinary hd audio and video transmission with a transmission speed of 1~2Mbps.
It offers a number of qualitative improvements over H.264. For comparison, see the differences between H.265 and H.264. In summary, H.265, HEVC, a popular video compression method currently, can bring about an average width saving of nearly 50% compared to the well-known H.264.
The relevant knowledge
Video Player Architecture
A typical modern player can be divided into three parts: UI, multimedia engine, and decoder. The architectural model is shown below:
Hard decoding support
As 4K video becomes more and more popular, Apple’s latest operating system versions (Mac Hight Sierra and iOS 11) usher in HEVC (Efficient Video Coding, also known as H.265), a new industry standard [6]. Compared with the current H.264 video compression standard, its video compression rate can be up to 50% higher. Using H.265, video streaming media transmission is better while maintaining the same picture quality. At the same bit rate, the quality can be improved nearly twice. The image below shows two images of the same bit rate and resolution (400kpbs 1080p), the one on the left using H.265 and the one on the right using H.264.
Web soft solution
In addition to hard decoding schemes, software decoding also becomes an effective choice, because decoding h.265 video is a CPU intensive task with high performance requirements, the performance of decoders implemented by Web scripting languages is difficult to meet the requirements. Based on this, we can use flash-based H.265 decoding scheme, that is, through the FlasCC[11] compiler to compile the DECOder written in C language into SWC library, and then call the SWC library in the Flash player with Action Script.
Another solution is based on HTML5, that is, through WebAssembly technology, jinshanyun’s self-developed high-performance decoder is compiled into wASM library. Wasm files exist in binary form, which contains platform-independent virtual instructions (similar to assembly instructions). This is also the approach adopted by many mobile platforms.
Related HTML5 technology
The following figure shows the main modules and dependent background technologies of the player kernel.
Media Source Extensions(MSE)
Provides plug-in – free web-based streaming media. Using the MSE API(Media Source, Source Buffer, etc.), Media streams can be created in JavaScript and played through HTMLMediaElement elements (including video and audio elements). Internet Explorer 11(Win8 +) and other modern browsers are supported.
Streams
The standard provides a set of apis for creating and manipulating stream data, specifically including ReadableStream, WritableStream, and TransformStream. This allows us to process the data incrementally rather than having to cache all the data in memory for uniform processing. We can use the Fetch API to Fetch the video data, and the body returned is a ReadableStream object. This object represents a data source and maintains a queue internally to record underlying data sources that have not yet been read. Chunk data in the internal queue can be read through the getReader() interface of ReadableStream.
Web Workers
The single-threaded JavaScript has the ability of multi-threaded programming, so that the video player kernel can separate the demultiplexing, decoding, rendering, UI operation monitoring and other tasks into different threads, and process the computation-intensive tasks and interface display in parallel. Communication between workers is carried out through MessageChannel. It is supported by IE10+ and other modern browsers.
WebAssembly
Bytecode technology on the Web side defines a common, compact, fast loading binary format as the compilation target, which can exploit the performance of common hardware and run at a speed closer to native applications. Software decoding h.265-encoded video in a browser is a performance-challenging task that scripting languages such as JavaScript are not up to. Thus, a high-performance decoder library written in C/C++ can be compiled into bytecode and run through JavaScript calls. The technology is currently available in newer versions of Chrome, Firefox, Safari, and Edge browsers (Chrome57+, Firefox 52+).
H.265 Vs H.264
For more details on the differences between H.265 and H.264, visit the differences between H.265 and H.264.
The coding architecture of H.265/HEVC is generally similar to that of H.264/AVC, mainly including: Intra prediction, Inter prediction, Transform, quantization, Deblocking filter, and entropy encoding Coding, etc.
In HEVC coding architecture, it is divided into three basic units, namely coding unit (CU), prediction unit (PU) and Transform Unit (TU). H.265/HEVC provides many different tools to reduce bit rates than H.264/AVC. In terms of encoding units, each macroblock/MB in H.264 is a fixed size of 16×16 pixels, while h.265 encoding units can be selected from a minimum of 8×8 to a maximum of 64×64.
Meanwhile, h.265’s in-frame prediction mode supports 33 directions (h.264 only supports 8), and provides better motion compensation processing and vector prediction methods. Repeated quality comparison tests have shown that, with the same image quality, h.265 can reduce the video size by approximately 39-44% compared to H.264. Due to the different measurement methods of quality control, this data will change accordingly.
support
ios
Currently, according to apple’s official website, the support for HEVC can be explained by the following sentence: IOS 11 and macOS High Sierra Introduced support for these new, industry-standard media formats. That is, the following devices are supported.
- iPhone 7 or iPhone 7 Plus or later
- iPad (6th generation)
- The Pro (10.5 inch)
- IPad Pro 12.9-inch (2nd Generation)
Android
The browser
Currently, browsers are not very friendly to H.265 support:
In actual combat
Right now, HEVC isn’t taking off that quickly, but we can still try playing H265 videos elegantly on the Web.
Check whether playback is supported
H265 mimeType =”video/mp4; Codecs = hevc “. Such as:
var supportHEVC = function(video) {
if(typeof video. CanPlayType = = 'function') {var playable = em.canplayType ('video/mp4; codecs="hevc"');
if ((playable.toLowerCase() == 'maybe') || (playable.toLowerCase() == 'probably')) {
return true; }}return false;
};
Copy the codeIf the video cannot be played using H.265, it will be played using H.264, so we can set various formats through source:
<video controls autoplay>
<source src="your_video.mp4" type="video/mp4; codecs=hevc">
<source src="your_video.webm" type="video/webm; codecs=vp9">
<source src="your_video.mp4" type="video/mp4; codecs=avc1">
</video>
Copy the codeFor the Web, we use libde265.js, a library that decodes H.265 videos with JS by converting the video’s frame data to RGBA pixels and then drawing them to the Canvas. Here is an example:
<canvas id="canvas"></canvas>
<script src="./libde265.min.j"></script>
<script>
var VIDEO_URL = 'h265-test-640.mp4'
var video = document.getElementById('canvas')
// var fpsWrap = document.querySelector('.hevc-fps')
var status = document.querySelector('.hevc-status')
var playback = function (event) {
// event.preventDefault()
if (player) {
player.stop()
}
player = new libde265.RawPlayer(video)
player.set_status_callback(function (msg, fps) {
player.disable_filters(true)
console.log(msg);
switch (msg) {
case 'loading':
status.innerHTML = 'Loading movie... '
break
case 'initializing':
status.innerHTML = 'Initializing... '
break
case 'playing':
status.innerHTML = 'Playing... '
break
case 'stopped':
status.innerHTML = 'stopped'
break
case 'fps':
// fpsWrap.innerHTML = Number(fps).toFixed(2) + ' fps'
break
default:
status.innerHTML = msg
}
})
player.playback(VIDEO_URL)
}
playback()
</script>
Copy the codeTest address: events.jackpu.com/h265.
Reference: H.265 / HEVC WiKi H.265 vs. H.264