Vx Search for “gjZKeyFrame” Follow “Keyframe Keyframe” to get the latest audio and video technical articles in time.

(Basic logic of this paper: introduction of audio and painting original data analysis tools → introduction of coding data analysis tools → Introduction of package format analysis tools)

To do a good job, he must sharpen his tools. In audio and video development, in order to facilitate, fast, intuitive analysis of audio and video data, it is best to have some visual analysis tools to help us, this article will introduce the common visual audio and video analysis tools.

1. Audio and painting original data analysis tool

1.1, Adobe Audition

Adobe Audition[1] is a professional audio editing and processing tool developed by Adobe, which supports multiple audio tracks, multiple audio effects and multiple audio formats. Use it to analyze PCM, AAC and other formats of audio data naturally.

PCM data is bare audio data and does not contain attribute information of digital audio elements. Therefore, when opening the PCM data file, as shown in the following figure, corresponding sampling rate, number of sound channels, bit depth, byte order and other information must be specified to open it correctly.

The following is the interface after opening the sample PCM audio data, you can see the corresponding two-channel waveform:

Adobe Audition has very rich functions, there are a lot of professional introduction of information on the Internet, we will not do too much here.

1.2, YUVToolkit

YUVToolkit[2] is an open source cross-platform tool for playing and analyzing native YUV data. It has these features:

  • Supports most YUV and RGB formats. For example: I420, I422, I444, YV12, YV16, YV24, UYVY, YUY2, NV12, grayScale; RGB24, RGBX32, XRGB32.
  • Supports image resolution, frame rate, color model resolution from file name. For example, file nametest-640x480-30FPS-I420.yuv.
  • Render with Direct3D and OpenGL, up to 720P, 60FPS, 4 videos at the same time.
  • Support image comparison and frame by frame calculation of MSE and PSNR, and visualization of distortion.
  • Supports Javascript. For example, you can use scripts to open multiple files at once.
  • You can use plug-in extensions to support more video formats, quality assessment methods, and rendering engines.

The following figure shows playing two YUV data and comparing the calculated MSE and PSNR:

1.3, YUVView

YUVView[3] is an open source cross-platform YUV data playback and analysis tool based on QT development. It has the following functions:

  • Most YUV sampling formats are supported. For example: 4:4:4, 4:2:2, 4:2:0, 4:4:0, 4:1:0, 4:1:1, 4:0:0.
  • Supports 8-16 bit depth.
  • Support ITU-R.BT709, ITU-R.BT601, ITU-R.BT2020 color space conversion.
  • Chromaticity interpolation uses nearest neighbor interpolation or bilinear interpolation.
  • You can freely configure chroma position and UV plane sequence.
  • Support for compact YUV storage format.
  • Most RGB formats are supported.
  • H.265 (HEVC) files are supported.
  • Support video file analysis data generation and floating display.
  • Supports comparison and analysis of file differences.

The following figure shows the functional interface of YUVView:

Luma Intra Direction on a HEVC stream

The following figure shows how to compare the difference between the encoded data and the raw data:

For more information see: YUVView Introduction[4]

2. Coding data analysis tools

2.1, StreamEye

StreamEye[5] is a commercial media analysis software. Here are some of its features:

  • It provides the code stream view interface, HEX view interface, pixel view interface, information view interface and other visual interfaces.
  • Support reference file, image difference contrast, master/slave control mode.
  • You can view and analyze video code stream information, image frame information, block information, flag bit information, DPB information and many other data.
  • Supports H.264, H.265, VP9, AV1, AND VVC encoding formats.

Below is a visualization of navigation using bar charts, decoded image buffer (DPB), and block details and representation:

Below is a navigation using thumbnails, partitions, and motion vectors to illustrate the syntax structure of coding:

Below is the SAO filtering overlay, buffer analysis and pixel-level visualization:

The following figure shows ALF filtering overlay and image overview demonstration:

3. Package format analysis tools

3.1, MP4Box. Js

Mp4box.js [6] is an online MP4 format analysis tool. It supports the import of network and local videos, and visually displays the structure and data of the MP4 Box.

MP4 Box:

3.2, MediaParser

MediaParser[7] is an open source MP4 format analyzer with relatively simple functions. It can display MP4 Box and node data information in a tree structure, and also give data location information according to Sample.

Here’s what MediaParser looks like:

3.3, MediaInfo

MediaInfo[8] is a tool to display the basic information of MP4.

Here is the interface of MediaInfo:

3.4, FLVParser

FLVParser[9] is a tool that can parse online FLV streams and output the TAGS and details of FLV streams.

The following is the functional interface of FLVParser:

3.5, VLC

VLC[10] is an open source cross-platform multimedia player that can play most multimedia files and view media information.

Here is the functional interface of VLC:

3.6. Native HLS Playback

Native HLS Playback[11] is a Chrome browser plug-in for direct Playback of HLS/M3U8/TS streams on Chrome. This works with Chrome’s Inspect/Network feature to see details about HLS streams.

Here is the functional interface of Native HLS Playback:

3.7、Play HLS M3u8

Play HLS M3u8[12] is also a Chrome plugin that supports playing HLS/ M3u8 /TS streams directly on Chrome. This works with Chrome’s Inspect/Network feature to see details about HLS streams.

Here is the Play HLS M3u8 functional interface:

The resources

[1] Adobe Audition: www.adobe.com/products/au…

[2] YUVToolkit: github.com/svn2github/…

[3] YUVView: github.com/IENT/YUView

[4] YUVView Introduction: ient.github.io/YUView/

[5] StreamEye: www.elecard.com/zh/products…

[6] MP4Box. Js: gpac. Making. IO/MP4Box js/t…

[7] MediaParser: github.com/ksvc/MediaP…

[8] MediaInfo: mediaarea.net/en/MediaInf…

[9] FLVParser: github.com/imagora/Flv…

[10] VLC: www.videolan.org/vlc/

[11] Native HLS Playback: chrome.google.com/webstore/de…

[12] Play HLS M3u8: chrome.google.com/webstore/de…

(Through the introduction above, we know such as Adobe Audition, YUVToolkit, StreamEye, MP4box.js, FLVParser, VLC and many other visual audio and video analysis tools, which are very useful for our daily audio and video development work. We’ll explore other useful audio and video tools later, so stay tuned.)

Recommended reading

Audio Coding: PCM, AAC

Video Coding: H.264

Video Coding: H.265

Video Coding: H.266

MP4 Format

FLV Format

M3U8 Format

TS Format