Senior engineer of sound network, with professional experience from basic information storage service of Iaas layer to cloud service of paas layer, like python language, accustomed to using C#, familiar with business product architecture based on and combined with CDN, vod, live broadcast, cloud guide, etc. Like to explore problems and research innovation, has 5 national invention patents.

Live broadcast is the most common real-time audio and video scene, and RTMP is one of the most important protocols in this scene, which is what many developers who have initial contact with real-time audio and video need to know. This article will use Winshark to capture packets and analyze the basic principle of the RTMP protocol to help you understand it more easily.

First presents the original file of RTMP protocol www.adobe.com/devnet/rtmp… When needed, you can refer to ~.

The RTMP protocol is the most popular protocol for live streaming

  • RTMP is an application layer protocol that relies on the underlying reliable transport layer (TCP).
  • A protocol (usually TCP) used to ensure the reliability of information transmission. After the transport layer Connection is established, the RTMP protocol also requires the client and server to “shake hands” to establish the RTMP Connection over the transport layer Connection. To play an RTMP stream, you need to go through the following steps: shake hands, establish a network connection, establish a network stream, and play. Only one network connection can be established between the server and the client, but many network flows can be created based on that connection. Their relationship is shown below:

  • The RTMP protocol will format the data itself during transmission, and the Message in this format is called RTMP Message. In actual transmission, in order to better realize multiplexing, subcontracting and fairness of information, the sender will divide the Message into Chunk with Message ID. Each Chunk may be an independent Message or a part of Message. The receiving end will restore the Chunk to a complete Message according to the length of data contained in the Chunk, Message ID and Message length, so as to realize the sending and receiving of information. The detailed fields and procedures of the protocol itself are not explained in detail here, but the process of the protocol is mainly understood by looking at the package directly. For more details, you can refer to the official documents provided above.

RTMP steps:

1. Shake hands

To establish a valid RTMP Connection, a “handshake” is required: the client sends chunks OF C0,C1, and C2 (in order) to the server, and the server sends chunks of S0,S1, and S2 (in order) to the client before effective information transfer can take place. The RTMP protocol itself does not specify the transmission order of the six messages, but the implementers of the RTMP protocol need to ensure that:

  • The client cannot send C2 until S1 is received
  • The client can only send other information (control information and real audio and video data) after S2 is received.
  • The server waits until it receives C0 to send S1
  • The server must wait until C1 is received before sending S2
  • The server must wait until C2 is received before sending other information (control information and real audio and video data, etc.). The handshake starts when the client sends blocks C0 and C1. The server sends S0 and S1 after receiving C0 or C1. When S0 and S1 are collected, the client sends C2. When the server receives all C0 and C1, it starts to send S2. When the client and server receive S2 and C2 respectively, the handshake is complete.

In fact, the actual package is as follows:

We can see TCP’s three-way handshake, and RTMP is based on TCP’s reliable transmission.

After filtering the RTMPT protocol, the handshake process of RTMP is as follows. We find that the real packet is sent together with C0+C1. S0,S1,S2 together.

2. Establish a network connection.

A) The client sends the “connect” in the command message to the server, requesting to establish a connection with a service application instance.

Open the connect package, an OSI5 layer protocol model, and the last one is THE RTMP protocol, which sends the CONNECT link message. Check the content contains the address name of the push stream, but it can be observed that there is no stream name, and the address has the name of app.

Take a look at the RTMP header

StreamID is the unique identifier of each Message. When dividing Chunk into Chunk and restoring Chunk to Message, it is based on this ID to identify whether it is the Chunk of the same Message. 0 indicates that the Message is the initial 0 Message.

Chunk Stream ID: Message is divided into multiple chunks. The same Chunk stream ID must belong to the same message.

Message Type ID: indicates the type of the data to be sent. For example, 8 indicates audio data and 9 indicates video data.

Format: Refers to chunk type. A total of 4 kinds of different formats, including the first format field is zero, can represent the other three said all of the data, but the other three formats is based on the difference of the chunk before quantitative said, therefore can be more concise to represent the same data, the actual use of time should use less as far as possible the bytes of the same data. Since Type0 represents different data and the others are differences, you can imagine that there must be something wrong with the stream if the package of Type0 is not found. Format == 0.

B) After the server receives the connection command message, it sends the Window Acknowledgement Size protocol message to the client and simultaneously connects to the application mentioned in the connection command.

C) The server sends a message to set the bandwidth protocol to the client.

D) After the client processes the bandwidth protocol setting message, it sends the Window Acknowledgement Size protocol message to the server.

E) The server sends the “Stream Begin” message in the user control message to the client.

F) The server sends the “_result” in the command message to inform the client of the connection status. Figure b~f: In _result, we can see that the link has been established successfully

ReleaseStream agora is the stream name, so we can capture a stream address from connect and releaseStream.

  1. Set up a NetStream

Tip: A network stream represents a channel for sending multimedia data. Only one network connection can be established between a server and a client, and multiple network flows can reuse this one network connection.

A. The client sends a createStream request to the server.

B. After receiving the request, the server sends _result() to the client in response to the message that creates the flow. The NetStream has been created.

4. PLAY PLAY

A) The client sends the “play” command in the command message to the server.

B) After receiving the playback command, the server sends the ChunkSize protocol message.

C) The server sends “StreamBEGIN” in the user control message to inform the client of the stream ID.

D) If the command is successfully played, the server sends “response status” in the command message netstream.play. Start & netstream.play. reset to inform the client that the “Play” command is successfully executed.

E) After this, the server sends the audio and video data to be played by the client.

Notice that the audio type is 8 and the video is 9.

5. Push the PUBLISH flow

Publish netstream; publish netstream; publish netstream

If you have any questions about following the steps or reading this article, please click here to jump to the original text and talk directly with the author.