This article is just some basic concepts related to M3U, M3U8, HLS, TS

Both “M3U” and “M3U8” files are the basis for the HTTP Live Streaming (HLS) protocol format used by Apple, which can be played on devices such as iphones and macBooks.

HLS solution

HTTP Live Streaming (HLS) is a real-time Streaming protocol defined by Apple. HLS is implemented based on HTTP. The transmission content consists of M3U8 description files and TS media files.

1. The advantages of HLS are adaptive streaming.

The effect is that the client automatically selects video streams at different bit rates based on network conditions, using high bit rates when conditions permit and low bit rates when the network is busy, and can automatically switch between the two at will. This is very helpful for ensuring smooth playback when the network condition of mobile device is unstable. The realization method is that the server side provides multi-bit rate video stream, and indicates in the list file, and the player adjusts automatically according to the playing progress and download speed.

2. Why TS instead of MP4

This is because the two TS fragments can be seamlessly spliced and the player can play them continuously, while the MP4 files cannot be seamlessly spliced due to the encoding method. When the PLAYER plays two MP4 files continuously, there will be broken sound and broken picture, which will affect the user experience. And if you want to jump from an hour-long video to a single VIDEO file in MP4 format, also using HTTP, you need a proxy server that supports HTTP Range Request to retrieve portions of the larger file. In this case, the proxy server performance requirements are high. However, HTTP Live Streaming only needs to find the corresponding TS fragment to download according to the timeline in the list file, without the need for Range Request, which has much smaller requirements on the proxy server. All proxy servers support efficient caching of small files.

3. AvailableFFmpegConvert MP4 to HLS(M3U8) file.

4. HSL treatment scheme

The core is to parse the m3U8 file and get the shard content binary via XHR, then encapsulate the buffer using MSE appendBuffer and merge it yourself.

M3U8

An M3U file is an indexed plain text file that can specify the location of one or more multimedia files with the file extension “M3U” or “M3U”. When opening it, the player software does not play it, but finds the network address of the corresponding audio and video file according to its index and plays it online. M3U files are typically used to create playlists that point to online streaming media that can be easily accessed.

M3U8 is the Unicode version of M3U, encoded in UTF-8.

The m3U8 file is essentially a playlist, which may be a Media playlist or a Master playlist.

When the M3U8 file is used as a Meida Playlist, its internal information records a series of media fragments, which can be played in sequence to display multimedia resources in a complete manner. For voD, the client only needs to download the fragment resources in sequence and play them in turn. For live broadcast, the client needs to periodically request the M3U8 file again to see if there is any new fragment data to download and play.

As a Master Playlist, M3U8 internally provides Variant Strean of the same media resource. Customers can choose the appropriate source of code stream according to different network states, and it is best to choose the appropriate resource content according to user preferences.

M3u8 files are either media playlists or main playlists. But no matter what kind of list it is, its effective composition is made up of two parts:

  • Tags starting with #EXT are used to further describe the media resource.
  • The remaining resource information is either the Media Playlist path or the M3U8 Master Playlist path.

M3u8 files follow the following conditions:

  • The M3U8 file must be encoded in UTF-8, cannot use Byte Order Mark (BOM) Byte Order, and cannot contain UTF-8 control characters (U+0000 ~ U_001F and U+007F ~ U+ 009F).

  • Each line of the 3u8 file is either a URI, an empty line, or a string beginning with #. Whitespace characters cannot appear, except to display declared elements.

  • The string beginning with # in the m3U8 file is either a comment or a tag. Tags start with #EXT and are case-sensitive.

M3u8 label

The tag is used to specify global parameters for the M3U8 file or some information for the slice file/media playlist behind it.

#EXTM3U // 表明该文件是一个 m3u8 文件，必须在文件的第一行
#EXT-X-VERSION:3   //  HLS 的协议版本号
#EXT-X-ALLOW-CACHE:NO
#EXT-X-MEDIA-SEQUENCE:1572999465  // M3U8直播时的直播切换序列，当播放打开M3U8时，以这个标签的值作为参考，播放对应的序列号的切片
#EXT-X-TARGETDURATION:6 // 该标签指定了媒体文件持续时间的最大值,，播放文件列表中的媒体文件在EXTINF标签中定义的持续时间必须小于或者等于该标签指定的持续时间。该标签在播放列表文件中必须出现一次。
#EXT-QQHLS-PIC-WIDTH:0
#EXT-QQHLS-PIC-HEIGHT:0
#EXT-QQHLS-CK:CRC_16
#EXT-QQHLS-BN:8
#EXT-QQHLS-SEGMENT-TYPE:0
#EXT-QQHLS-SEGMENT_RANGE:1572999448-1572999472
#EXT-X-PROGRAM-DATE-TIME:2019-11-06T10:11:57+08:00 // 片段取样时间
#EXT-QQHLS-MACHINEID:56448
#EXT-QQHLS-START-TIME:1573006317
#EXT-SID:1572999465
#EXT-SC:SZ=254364&CK=e633
#EXT-BC:SZ=31744&CK=782e
#EXT-BC:SZ=31744&CK=256a
#EXT-BC:SZ=31744&CK=7b04
#EXT-BC:SZ=31744&CK=30dd
#EXT-BC:SZ=31744&CK=6d89
#EXT-BC:SZ=31744&CK=46d
#EXT-BC:SZ=31744&CK=8f4
#EXT-BC:SZ=32156&CK=71b5
#EXTINF:5.000,
100204101-1572999465.ts?cdncode=/18907E7BE0798990/&HLSP2P=1&buname=qqlive&sdtfrom=10002212&vkey=F1FE87B5E6D84B6FEFBB683D7C2558C08C8CF7B6E752E05C689F1117D651C359E4C7460F44AF4CA4D54188D17A34DB8F3D1E1A9B03F5F5509AFFC93EEA5B90B835EB1E096C8EA81447313AD2761912137D4F692E8AEC2D52E0E3ACD02E09272F82A1FC8573388BDE57FBE56A424E01BA
// EXTINF为M3U8列表中每一个分片的duration,描述信息
#EXT-X-PROGRAM-DATE-TIME:2019-11-06T10:12:02+08:00
#EXT-QQHLS-MACHINEID:56448
#EXT-QQHLS-START-TIME:1573006322
#EXT-SID:1572999466
#EXT-SC:SZ=287264&CK=6abc
#EXT-BC:SZ=35840&CK=aa34
#EXT-BC:SZ=35840&CK=42f1
#EXT-BC:SZ=35840&CK=46f
#EXT-BC:SZ=35840&CK=3fd
#EXT-BC:SZ=35840&CK=75b2
#EXT-BC:SZ=35840&CK=af4f
#EXT-BC:SZ=35840&CK=52a1
#EXT-BC:SZ=36384&CK=86ed
#EXTINF:5.000,
100204101-1572999466.ts?cdncode=/18907E7BE0798990/&HLSP2P=1&buname=qqlive&sdtfrom=10002212&vkey=F1FE87B5E6D84B6FEFBB683D7C2558C08C8CF7B6E752E05C689F1117D651C359E4C7460F44AF4CA4D54188D17A34DB8F3D1E1A9B03F5F5509AFFC93EEA5B90B835EB1E096C8EA81447313AD2761912137D4F692E8AEC2D52E0E3ACD02E09272F82A1FC8573388BDE57FBE56A424E01BA
#EXT-X-PROGRAM-DATE-TIME:2019-11-06T10:12:07+08:00
#EXT-QQHLS-MACHINEID:56448
#EXT-QQHLS-START-TIME:1573006327
#EXT-SID:1572999467
#EXT-SC:SZ=289708&CK=749f
#EXT-BC:SZ=35840&CK=a209
#EXT-BC:SZ=35840&CK=abb9
#EXT-BC:SZ=35840&CK=13ab
#EXT-BC:SZ=35840&CK=40b7
#EXT-BC:SZ=35840&CK=2e10
#EXT-BC:SZ=35840&CK=3a00
#EXT-BC:SZ=35840&CK=62ae
#EXT-BC:SZ=38828&CK=65e5
#EXTINF:5.000,
100204101-1572999467.ts?cdncode=/18907E7BE0798990/&HLSP2P=1&buname=qqlive&sdtfrom=10002212&vkey=F1FE87B5E6D84B6FEFBB683D7C2558C08C8CF7B6E752E05C689F1117D651C359E4C7460F44AF4CA4D54188D17A34DB8F3D1E1A9B03F5F5509AFFC93EEA5B90B835EB1E096C8EA81447313AD2761912137D4F692E8AEC2D52E0E3ACD02E09272F82A1FC8573388BDE57FBE56A424E01BA
#EXT-X-PROGRAM-DATE-TIME:2019-11-06T10:12:11+08:00
#EXT-QQHLS-MACHINEID:56448
#EXT-QQHLS-START-TIME:1573006331
#EXT-SID:1572999468
#EXT-SC:SZ=523016&CK=97c
#EXT-BC:SZ=64512&CK=a072
#EXT-BC:SZ=64512&CK=b80e
#EXT-BC:SZ=64512&CK=13ee
#EXT-BC:SZ=64512&CK=2c70
#EXT-BC:SZ=64512&CK=992c
#EXT-BC:SZ=64512&CK=3c51
#EXT-BC:SZ=64512&CK=a079
#EXT-BC:SZ=71432&CK=a66
#EXTINF:5.000,
100204101-1572999468.ts?cdncode=/18907E7BE0798990/&HLSP2P=1&buname=qqlive&sdtfrom=10002212&vkey=F1FE87B5E6D84B6FEFBB683D7C2558C08C8CF7B6E752E05C689F1117D651C359E4C7460F44AF4CA4D54188D17A34DB8F3D1E1A9B03F5F5509AFFC93EEA5B90B835EB1E096C8EA81447313AD2761912137D4F692E8AEC2D52E0E3ACD02E09272F82A1FC8573388BDE57FBE56A424E01BA
#EXT-X-PROGRAM-DATE-TIME:2019-11-06T10:12:16+08:00
#EXT-QQHLS-MACHINEID:56448
#EXT-QQHLS-START-TIME:1573006336
#EXT-SID:1572999469
#EXT-SC:SZ=670032&CK=6479
#EXT-BC:SZ=82944&CK=dc72
#EXT-BC:SZ=82944&CK=e970
#EXT-BC:SZ=82944&CK=4ca4
#EXT-BC:SZ=82944&CK=3d1b
#EXT-BC:SZ=82944&CK=fa9b
#EXT-BC:SZ=82944&CK=e9d3
#EXT-BC:SZ=82944&CK=ee0b
#EXT-BC:SZ=89424&CK=b998
#EXTINF:5.000,
100204101-1572999469.ts?cdncode=/18907E7BE0798990/&HLSP2P=1&buname=qqlive&sdtfrom=10002212&vkey=F1FE87B5E6D84B6FEFBB683D7C2558C08C8CF7B6E752E05C689F1117D651C359E4C7460F44AF4CA4D54188D17A34DB8F3D1E1A9B03F5F5509AFFC93EEA5B90B835EB1E096C8EA81447313AD2761912137D4F692E8AEC2D52E0E3ACD02E09272F82A1FC8573388BDE57FBE56A424E01BA
#EXT-X-PROGRAM-DATE-TIME:2019-11-06T10:12:21+08:00
#EXT-QQHLS-MACHINEID:56448
#EXT-QQHLS-START-TIME:1573006341
#EXT-SID:1572999470
#EXT-SC:SZ=777756&CK=a13c
#EXT-BC:SZ=96256&CK=e95e
#EXT-BC:SZ=96256&CK=f459
#EXT-BC:SZ=96256&CK=33a6
#EXT-BC:SZ=96256&CK=bb7c
#EXT-BC:SZ=96256&CK=f3e4
#EXT-BC:SZ=96256&CK=635d
#EXT-BC:SZ=96256&CK=67e9
#EXT-BC:SZ=103964&CK=b938
#EXTINF:5.000,
100204101-1572999470.ts?cdncode=/18907E7BE0798990/&HLSP2P=1&buname=qqlive&sdtfrom=10002212&vkey=F1FE87B5E6D84B6FEFBB683D7C2558C08C8CF7B6E752E05C689F1117D651C359E4C7460F44AF4CA4D54188D17A34DB8F3D1E1A9B03F5F5509AFFC93EEA5B90B835EB1E096C8EA81447313AD2761912137D4F692E8AEC2D52E0E3ACD02E09272F82A1FC8573388BDE57FBE56A424E01BA
#EXT-X-PROGRAM-DATE-TIME:2019-11-06T10:12:27+08:00
#EXT-QQHLS-MACHINEID:56448
#EXT-QQHLS-START-TIME:1573006347
#EXT-SID:1572999471
#EXT-SC:SZ=479212&CK=c493
#EXT-BC:SZ=59392&CK=dec8
#EXT-BC:SZ=59392&CK=1ee8
#EXT-BC:SZ=59392&CK=b9bf
#EXT-BC:SZ=59392&CK=8b9c
#EXT-BC:SZ=59392&CK=29fe
#EXT-BC:SZ=59392&CK=493
#EXT-BC:SZ=59392&CK=ae5f
#EXT-BC:SZ=63468&CK=630
#EXTINF:5.000,
100204101-1572999471.ts?cdncode=/18907E7BE0798990/&HLSP2P=1&buname=qqlive&sdtfrom=10002212&vkey=F1FE87B5E6D84B6FEFBB683D7C2558C08C8CF7B6E752E05C689F1117D651C359E4C7460F44AF4CA4D54188D17A34DB8F3D1E1A9B03F5F5509AFFC93EEA5B90B835EB1E096C8EA81447313AD2761912137D4F692E8AEC2D52E0E3ACD02E09272F82A1FC8573388BDE57FBE56A424E01BA
#EXT-X-PROGRAM-DATE-TIME:2019-11-06T10:12:32+08:00
#EXT-QQHLS-MACHINEID:56448
#EXT-QQHLS-START-TIME:1573006352
#EXT-SID:1572999472
#EXT-SC:SZ=266396&CK=648
#EXT-BC:SZ=32768&CK=a085
#EXT-BC:SZ=32768&CK=8233
#EXT-BC:SZ=32768&CK=1831
#EXT-BC:SZ=32768&CK=4042
#EXT-BC:SZ=32768&CK=f0eb
#EXT-BC:SZ=32768&CK=cd3a
#EXT-BC:SZ=32768&CK=e245
#EXT-BC:SZ=37020&CK=f6e3
#EXTINF:5.000,
100204101-1572999472.ts?cdncode=/18907E7BE0798990/&HLSP2P=1&buname=qqlive&sdtfrom=10002212&vkey=F1FE87B5E6D84B6FEFBB683D7C2558C08C8CF7B6E752E05C689F1117D651C359E4C7460F44AF4CA4D54188D17A34DB8F3D1E1A9B03F5F5509AFFC93EEA5B90B835EB1E096C8EA81447313AD2761912137D4F692E8AEC2D52E0E3ACD02E09272F82A1FC8573388BDE57FBE56A424E01BA

Copy the code

Some other common tags:

  • Ext-x-discontinuity This label indicates that there is an interruption between the previous slice and the next slice. A classic use scenario of Ext-X-Discontinuity is to insert advertisements into video streams. Since video streams and advertising video streams are not the same resource, Ext-X-Discontinuity is used to indicate when switching between these two streams. When the client sees this label, It will handle this switch interruption and make the experience better.

  • The ext-X-key media fragment can be encrypted, and this tag specifies the decryption method.

  • Ext-x-playlist-type: indicates the streaming media TYPE. It takes effect globally. The label is optional. Possible values are as follows:

    • VODThat is, Video on Demand indicates that the Video stream is a vod source, so the server cannot change the M3U8 file.
    • EVENTIndicates that the video stream is a live stream source, so the server cannot change or delete any part of the file (but can add new content at the end of the file)

Note: VOD files usually carry the ext-X-endlist tag because they are on-demand sources and will not change; However, EVEVT files are not initialized with ext-X-EndList tags, indicating that new files will be added to the end of the playlist. Therefore, clients need to periodically obtain the M3U8 file to obtain new media fragment resources until the ext-X-EndList tag is accessed.

  • Ext-x-program-date-time This tag uses an absolute DATE/TIME to indicate when the first sample fragment was sampled.

  • Ext-x-media-sequence: indicates the SEQUENCE number of the first URL fragment file in the playlist. Each MEDIA fragment URL has a unique integer SEQUENCE number. Each media segment serial number is incremented by one in the order in which it appears. The sequence number of a media fragment is independent of the fragment file name. If this label is not specified, the default sequence number starts at 0.

  • Ext-x-endlist if the ext-X-endList tag appears, the M3U8 file will not generate any more slices, which can be interpreted as the M3U8 file has stopped updating, and the shard is played to this tag. M3U8 can not only be used as live streaming, but also as on-demand. Save all slice information in the M3U8 file and end with ext-X-endList. This M3U8 is called on-demand M3U8. The ext-X-Endlist tag may appear anywhere in the playlist file, but not twice or more.

  • Ext-x-stream-inf The ext-X-stream-INF tag appears in M3U8 files, mainly in multilevel M3U8 files, such as M3U8 containing child M3U8 lists, or main M3U8 containing multirate M3U8. This tag needs to be followed by some attributes. Here are some of the attributes:

    • BANDWIDTH:BANDWIDTHThe value of is the highest bit rate value, when broadcastPut the EXT - X - STREAM - INFMaximum bit rate (required) for M3U8.
    • AVERAGE-BANDWIDTH:AVERAGE-BANDWIDTHThe value is the average bit rate value when playingEXT-X-STREAM-INFAverage bit rate (optional) for M3U8.
    • CODECSThe value of: CODECS is used for declarationsEXT-X-STREAM-INFThe following corresponds to the audio and video coding and video coding information in M3U8 (optional parameters).
    • RESOLUTION: (Optional) Video width and height description in M3U8.
    • FRAME-RATE: Video frame rate (optional) in child M3U8.

Some criteria for client playback of M3U8:

  1. Sharding must be dynamic, sequences must not be the same, and sequences must be incremented
  2. When M3U8 doesn’t show upEXT-X-ENDLISTNo matter how many shards there are in the M3U8 list, shards will be played from the last shard. If there are less than three shards, shards should not be played. Of course, if some players are specially customized, they can not follow this principle.
  3. Refresh the M3U8 list by playing the current shard duration, and then perform the corresponding load action.
  4. If the playlist is the same as the previous list after refreshing, it is refreshed within half the time of playing the current sharding Duration.
  5. If there is a discontinuity between the preceding fragment and the following fragment, the playback may be wrongX-DISCONTINUTYTag to resolve the error.

Matters needing attention

  • There are two ways to request an M3U8 playlist: one is to request the URI of m3U8, then the file must end with.m3u8 or.m3u; 2 it is through the HTTP request, the request header content-type must be set to application/VND. Apple. Mpegurl or audio/mpegurl.

  • Empty lines and comment lines are ignored when parsing.

  • The total duration of stream resources in the Media Playlist is the sum of the duration of all slice resources.

TS file

Transport Stream (TS) is an audio and video encapsulation format, full name is MPEG2-TS. Mpeg-ts is mainly used for programs delivered in real time, such as television programs broadcast in real time.

TS files (streams) can be divided into three layers: TS (Transport Stream), PES (Packet Elemental Stream) and ES (Elementary Stream).

ES layer is the audio and video data, PES layer is the audio and video data with time stamps and other data frame description information, TS layer is added to PES layer data flow recognition and transmission necessary information. TS files (bit streams) are composed of multiple TS packets.

Hierarchical structure diagram of TS file (stream)

TS stream generation process

  1. After the original audio and video data is compressed, the compressed result forms a basic code stream (ES).
  2. ES (basic codestream) is packaged to form PES.
  3. Add timestamp information (PTS/DTS) to PES packages.
  4. The PES Packet contents are distributed into a series of fixed length transport packets (TS packets).
  5. Add timing information (PCR) to the transport packet.
  6. Add program specific information (PSI) to the transport packet.
  7. A continuous output transport packet forms an MPEG-TS stream with a constant bit rate.

TS flow parsing process

  1. TS packages are parsed from the multiplexed MPEG-TS stream.
  2. Get PAT and corresponding PMT from TS package;
  3. To obtain the PID of audio and video of a specific program;
  4. TS packages related to specific audio and video were screened by PID, and PES was resolved.
  5. PTS/DTS is read from PES, and basic code stream ES is resolved from PES.
  6. ES is given to the decoder to obtain the original audio and video data before compression.

Ts file Merge

Encrypted TS files cannot be merged or played directly. You need to decrypt each TS file using a key.

For details, see m3U8 encryption, TS file merge – author: Steed Jinlong

References:

  • Multimedia file format (3) : M3U8 format
  • Multimedia file format (4) : TS format
  • M3u8 file format details