How to quickly implement RTSP/RTMP live broadcast on Windows platform

Some time ago, we in the blog.csdn.net/renhui1112/… This paper mainly introduces how to call SDK to realize RTSP/RTMP playback capability. This article takes the call of Daniu Live SDK as an example:

The demo illustrates

• SDK provides C++/C# two sets of interfaces, external 32-bit / 64-bit library, C++ and C# interface one by one correspondence, C# interface than C++ interface prefix NT_PB_;
• Win-playersdk-cpp-demo: indicates the Demo of the C++ interface corresponding to the PlayerSDK.
• Win-playersdk-csharp-demo: Demo of the C# interface corresponding to the PlayerSDK; The player SDK supports Win7 and above systems;
• This demo is developed based on VS2013;
• Related interfaces can be downloaded from Github.

Integrated description

1 the initialization

NT_SP_Init If you need to configure the log path, do the following before NT_SP_Init (the directory can be specified by yourself) : // Set the log path (ensure that the directory exists) //String log_path = “D:\playerlog”; //NTSmartLog.NT_SL_SetPath(log_path);

2 Open Generates a playback instance

3 Set the callback event

①NT_SP_SetEventCallBack: used to call back network link state, buffer state (start, buffer ratio, end), real-time bandwidth, etc. ②NT_SP_SetVideoSizeCallBack: set video resolution callback; ③NT_SP_SetVideoFrameCallBack: set YUV/RGB32 data callback, can be used for docking third-party video analysis, or draw; ④NT_SP_SetVideoFrameCallBackV2: set YUV/RGB32 data callback, and NT_SP_SetVideoFrameCallBack interface is different, spit out video data, can specify the width and height; (5) NT_SP_SetRenderVideoFrameTimestampCallBack: set rendering video frames, video frame timestamp callback; ⑥NT_SP_SetAudioPCMFrameCallBack: set audio PCM frame callback, PCM data out, currently each frame size is 10ms; ⑦NT_SP_SetUserDataCallBack: set user data callback, this interface needs to be used with the push side SDK, used to return the push side set real-time user data (such as time stamps, latitude and longitude and other extension instructions or information); ⑧NT_SP_SetSEIDataCallBack: Sets the video SEI data callback.

4 Set the RTMP/RTSP Url

5 Check whether D3DRender is supported

Set the handle of the rendering window (optional interface). ①NT_SP_IsSupportD3DRender ②NT_SP_SetRenderWindow ③NT_SP_GDIDrawRGB32: DrawRGB32 data using GDI

6 Set the callback PCM

7 Set RTMP/RTSP playback parameters

①NT_SP_SetBuffer: Sets the size of the video playback buffer, in milliseconds. ②NT_SP_SetMute: in the process of playing, mute or unmute in real time. It can be called before playing, or called in real time during playing. ③NT_SP_SetRTSPTcpMode: Sets the RTSP TCP mode. 1 is TCP and 0 is UDP. This interface is valid only for RTSP. ④NT_SP_SetRtspTimeout: Sets the RTSP timeout period. The timeout unit is second and must be greater than 0. NT_SP_SetRtspAutoSwitchTcpUdp: For RTSP, some may support RTP over UDP and some may support RTP over TCP. If udp cannot be played, SDK will automatically try TCP. If TCP cannot be played, SDK will automatically try UDP. Is_auto_switch_tcp_udp: If set to 1, THE SDK will try to switch playback between TCP and UDP. If set to 0, the SDK will not try to switch playback. ⑤NT_SP_SetFastStartup: set second on, 1 is second on, 0 is not second on, this interface is used for example, RTMP server cache GOP, appropriate use; ⑥NT_SP_SetLowLatencyMode: sets the low delay mode. The default mode is normal. Mode: 1 indicates the low delay mode and 0 indicates the normal mode. ⑦NT_SP_SetRotation: sets video View rotation and clockwise rotation. Degress: sets 0, 90, 180, 270 ° and other values are invalid. Note: other angles except 0 ° will consume more CPU. ⑧NT_SP_SetFlipVertical: set the video View to invert vertically; ⑨NT_SP_SetFlipHorizontal: Set video View horizontal reversal; ⑩NT_SP_SetReportDownloadSpeed: Sets the download speed to be reported. The download speed is not reported by default

• Is_report: switch for reporting, 1: table reporting 0: not reporting Other values are invalid.
• Report_interval: reporting interval (reporting frequency), expressed in seconds. The minimum value is once every second. If the value is less than 1 and the report function is set, the call fails
• Note: If set to report, set SetEventCallBack and handle the event in the callback function.
• The reported event is as follows: NT_SP_E_EVENT_ID_DOWNLOAD_SPEED NT_SP_GetDownloadSpeed: automatically obtains the download speed, and speed: returns the download speed, in Byte/s. Qiu NT_SP_SetParam: Universal interface for setting parameters, which can solve most problems; · If possible, NT_SP_GetParam is an all-purpose interface for obtaining parameters and solving most problems. 14. NT_SP_SetKey: Set the decryption key for RTMP encrypted streams. At present, it is only used to decrypt RTMP encrypted streams. 15 NT_SP_SetDecryptionIV: sets the decryption vector for RTMP encrypted streams, which is currently used only to decrypt RTMP encrypted streams; ⑯ persons NT_SP_SetSDKClientKey: Set the authorization Key, contact the official department for the formal authorization Key.

8 Check whether hard decoding is supported

		/* * Check whether H264 hard decoding is supported * if so, return NT_ERC_OK */
        [DllImport(@"SmartPlayerSDK.dll")]
		public static extern UInt32 NT_SP_IsSupportH264HardwareDecoder(a);

		/* * Check whether H265 hard decoding is supported * if so, return NT_ERC_OK */
        [DllImport(@"SmartPlayerSDK.dll")]
		public static extern UInt32 NT_SP_IsSupportH265HardwareDecoder(a);

		/* * set H264 hard solution * is_hardware_decoder: 1: hard solution, 0: no hard solution * reserve: reserve parameter, current pass 0 is good * success return NT_ERC_OK */
        [DllImport(@"SmartPlayerSDK.dll")]
        public static extern UInt32 NT_SP_SetH264HardwareDecoder(IntPtr handle, Int32 is_hardware_decoder, Int32 reserve);

		/* * set H265 hard solution * is_hardware_decoder: 1: hard solution, 0: no hard solution * reserve: reserve parameter, current pass 0 is good * return NT_ERC_OK */ on success
        [DllImport(@"SmartPlayerSDK.dll")]
        public static extern UInt32 NT_SP_SetH265HardwareDecoder(IntPtr handle, Int32 is_hardware_decoder, Int32 reserve);
Copy the code

9 RTMP/RTSP player – Starts playing

NT_SP_StartPlay

10 Change the size of the drawing window

11 RTMP/RTSP Video recording at the pull end

①NT_SP_SetRecorderDirectory: Sets the recording directory. ②NT_SP_SetRecorderFileMaxSize: sets the maximum size of a single file. ③NT_SP_SetRecorderFileNameRuler: sets the maximum size of a single file. Set the video file name generated rules (4) NT_SP_SetRecorderCallBack: set the video callback interface (5) NT_SP_SetRecorderAudioTranscodeAAC: Set audio to AAC encoding switch when recording, AAC is more common, SDK adds other audio encoding (such as SPEex, PCMU, PCMA, etc.) to AAC function. NT_SP_SetRecorderAudio sets whether to record a video or not. By default, if the video source has a video, it will record it. If the video source does not have a video, it will record it. By default, if the video source has audio, it will record, but if it does not have audio, it will not record. In some scenarios, it will not record audio, but only video, so add a switch ⑧NT_SP_StartRecorder: start recording ⑨NT_SP_StopRecorder: stop recording

12 RTMP/RTSP Pull End Snapshot (Real-time call)

13 RTMP/RTSP Quick URL switch (Real-time call)

NT_SP_SwitchURL: switches the URL. Switch_pos: Sets the playing location after switching to the new URL. The default value is 0

14 User data Callback

15 SEI data callback

NT_SP_SetSEIDataCallBack: Sets the video SEI data callback for receiving sei data callback

16 Set the video screen filling mode

        /* * Set the filling mode of the video picture, such as filling the whole drawing window or equal proportion filling the drawing window. If not set, the entire drawing window will be filled by default. * Handle: play handle * mode: 0: fill the whole drawing window; 1: Equal scale fill drawing window, default is 0 * return NT_ERC_OK */ on success
        [DllImport(@"SmartPlayerSDK.dll")]
        public static extern UInt32 NT_SP_SetRenderScaleMode(IntPtr handle, Int32 mode);
Copy the code

17 RTMP/RTSP Player – Stops playing

NT_SP_StopPlay

18 Close the playback instance

19 Uninit

NT_SP_UnInit

20 RTMP/RTSP Playback status Event callback

/ * event ID * /
        public enum NT_SP_E_EVENT_ID : uint
        {
            NT_SP_E_EVENT_ID_BASE = NTBaseCodeDefine.NT_EVENT_ID_SMART_PLAYER_SDK,

	        NT_SP_E_EVENT_ID_CONNECTING			= NT_SP_E_EVENT_ID_BASE | 0x2./* Connection */
	        NT_SP_E_EVENT_ID_CONNECTION_FAILED	= NT_SP_E_EVENT_ID_BASE | 0x3./* Connection failed */
	        NT_SP_E_EVENT_ID_CONNECTED			= NT_SP_E_EVENT_ID_BASE | 0x4./* Connected */
	        NT_SP_E_EVENT_ID_DISCONNECTED		= NT_SP_E_EVENT_ID_BASE | 0x5./* Disconnect */
             NT_SP_E_EVENT_ID_NO_MEDIADATA_RECEIVED = NT_SP_E_EVENT_ID_BASE | 0x8./* Failed to receive RTMP data */
 		  NT_SP_E_EVENT_ID_RTSP_STATUS_CODE   = NT_SP_E_EVENT_ID_BASE | 0xB./* RTSP status code Is reported. Currently, only 401 is reported. Param1 indicates status code*/
            NT_SP_E_EVENT_ID_NEED_KEY           = NT_SP_E_EVENT_ID_BASE | 0xC./* Decryption key is required to play */
            NT_SP_E_EVENT_ID_KEY_ERROR          = NT_SP_E_EVENT_ID_BASE | 0xD./* The decryption key is incorrect */

	        /* Start at 0x81 */
	        NT_SP_E_EVENT_ID_START_BUFFERING = NT_SP_E_EVENT_ID_BASE | 0x81./* Start buffer */
	        NT_SP_E_EVENT_ID_BUFFERING		 = NT_SP_E_EVENT_ID_BASE | 0x82./* In the buffer, param1 represents the percentage of progress */
	        NT_SP_E_EVENT_ID_STOP_BUFFERING  = NT_SP_E_EVENT_ID_BASE | 0x83./* Stop buffering */

	        NT_SP_E_EVENT_ID_DOWNLOAD_SPEED  = NT_SP_E_EVENT_ID_BASE | 0x91./* Download speed. Param1 indicates the download speed, expressed in Byte/s */

            NT_SP_E_EVENT_ID_PLAYBACK_REACH_EOS = NT_SP_E_EVENT_ID_BASE | 0xa1./* When the playback ends, the live stream does not have this event, the on-demand stream only has */
            NT_SP_E_EVENT_ID_RECORDER_REACH_EOS = NT_SP_E_EVENT_ID_BASE | 0xa2./* When the video is finished, the live stream does not have this event, the on-demand stream only has */
            NT_SP_E_EVENT_ID_PULLSTREAM_REACH_EOS = NT_SP_E_EVENT_ID_BASE | 0xa3./* Pull stream ends, live stream does not have this event, on-demand stream only have */

            NT_SP_E_EVENT_ID_DURATION = NT_SP_E_EVENT_ID_BASE | 0xa8./* Video duration. If the video duration is live, it is not reported. If the video duration can be obtained from the video source, it is reported. Param1 indicates the video duration in ms */
        }
Copy the code

21 RTMP/RTSP Playback Playback of audio and video data

 [StructLayoutAttribute(LayoutKind.Sequential)]
    public struct NT_SP_PullStreamVideoDataInfo
    {
        public Int32 is_key_frame_; /* 1: indicates a key frame, 0: indicates a non-key frame */
        public UInt64 timestamp_;	/* Decode timestamp in milliseconds */
	   public Int32 width_;	/* is usually 0 */
	   public Int32 height_; /* is usually 0 */
	   public IntPtr parameter_info_; /* Usually NULL */
	   public UInt32 parameter_info_size_; /* is usually 0 */
        public UInt64 presentation_timestamp_; /* Displays the timestamp, which must be greater than or equal to timestamp_ in milliseconds */
}
    /* * Pull out audio data, some related data */
  [StructLayoutAttribute(LayoutKind.Sequential)]
   public struct NT_SP_PullStreamAuidoDataInfo
   {
        public Int32 is_key_frame_; /* 1: indicates a key frame, 0: indicates a non-key frame */
        public UInt64 timestamp_;	/* The unit is milliseconds */
	   public Int32 sample_rate_;	/* is usually 0 */
	   public Int32 channel_; /* is usually 0 */
	   public IntPtr parameter_info_; /* This has a value if it is AAC, other encodings generally ignore */
	   public UInt32 parameter_info_size_; /* This has a value if it is AAC, other encodings generally ignore */
	   public UInt64 reserve_; / * * / reserved
    }
Copy the code

The above is the basic interface description and call flow, interested developers can refer to.