Many developers mentioned that RTMP player, do not know what the benchmark and test indicators, the following is about our experience, interested, can pay attention to Github:
1. ** Low latency: * * most RTMP broadcast live for scenario, if the delay is too big, the serious influence experience, so the low latency is very important to measure a good RTMP players, now Daniel live SDK RTMP live broadcast delay more excellent than open source players (Daniel live SDK delay in 1 seconds, open source players such as VLC, The delay is 5-7 seconds), and the playback end of DANIu Live SDK will not cause delay accumulation when it runs for a long time. Open source or third-party player is prone to delay accumulation when it runs for a long time.
Some servers will cache GOP to ensure that the first screen playback can be realized quickly without affecting the delay. In this regard, we designed a quick start interface to render the first frame quickly and catch up with the latest playback data:
/* * set the second on, 1 is second on, 0 is not second on */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetFastStartup(IntPtr handle, Int32 isFastStartup);
Copy the code2. Audio and video synchronization processing ** : ** In order to pursue low latency, most players do not even do audio and video synchronization, and directly play audio video, resulting in a/ V synchronization, and random timestamp jumping and other problems. The player provided by DANIu Live SDK has good timestamp synchronization and abnormal timestamp correction mechanism;
Note: In ultra-low delay mode, 0 buffer can be used without audio and video synchronization:
* mode: 1 indicates the low delay mode, 0 indicates the normal mode, and other parameters are invalid. * NT_ERC_OK */ is returned after the interface is called successfully
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetLowLatencyMode(IntPtr handle, Int32 mode);
Copy the code**3. Support for multiple instances: ** The RTMP live playback SDK provided by Daniu live streaming SDK supports multiple instances of RTMP stream data under the condition that the device performance allows. Most open source players are not friendly to multi-instance support;
With more conventional solid exceptions, such as domestic monitoring scenarios, although we CPU was already in the industry is very low, but a lot of manufacturers, not every road to full frame, according to the situation, we made a real-time playback only key frames and the whole frame of interface design, such as eight instance, one of the less important data a few road, You can set only key frames to play, and click full frame rate to play when you need to pay attention, which not only saves system overhead, but also realizes the purpose of multi-channel playback:
/* * set to decode only video keyframes * is_only_dec_key_frame: 1: decode only keyframes, 0: decode both, default is 0 * NT_ERC_OK */ is returned on success
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetOnlyDecodeVideoKeyFrame(IntPtr handle, Int32 is_only_dec_key_frame);
Copy the code** Support buffer time Settings: ** In some network jitter scenarios, players need to support buffer time Settings. Generally speaking, in milliseconds, open source players are not friendly enough to support buffer time Settings;
5. Real-time mute: ** For example, if multiple Windows play RTMP stream, if every audio plays out, the experience is very bad, so the real-time mute function is very necessary, open source player does not have real-time mute function;
6 **. Video View rotation: ** Many cameras have inverted images due to installation limitations, so a good RTMP player should support real-time video view rotation (0° 90° 180° 270°), horizontal reversal, vertical reversal, open source or third party players do not have this feature;
Is_flip: 1: indicates an inversion, 0: indicates no inversion */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetFlipVertical(IntPtr handle, Int32 is_flip);
Is_flip: 1: indicates reversal, 0: indicates no reversal */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetFlipHorizontal(IntPtr handle, Int32 is_flip);
/* * Set rotation, clockwise rotation * degress: 0, 90, 180, 270 degrees are valid, other values are invalid * Note: playing at any Angle other than 0 degrees costs more CPU * NT_ERC_OK */ is returned on success
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetRotation(IntPtr handle, Int32 degress);
Copy the code7 **. Support audio/video data output after decoding: ** Daniu Live SDK has contacted many developers, hoping to obtain YUV or RGB data for face matching algorithm analysis while playing, which is not available in open source player;
public void SDKVideoFrameCallBack(UInt32 status, NT_SP_VideoFrame frame)
{
if(cur_video_frame_.plane0_ ! = IntPtr.Zero) { Marshal.FreeHGlobal(cur_video_frame_.plane0_); cur_video_frame_.plane0_ = IntPtr.Zero; } cur_video_frame_ = frame;this.Invalidate();
}
public void SDKAudioPCMFrameCallBack(UInt32 status, IntPtr data, UInt32 size, Int32 sample_rate, Int32 channel, Int32 per_channel_sample_number)
{
// Get the PCM frame of the callback and do the related operations (e.g. play by yourself)
//label_debug.Text = per_channel_sample_number.ToString();
//release
Marshal.FreeHGlobal(data);
}
public void SetVideoFrameCallBack(IntPtr handle, IntPtr userData, UInt32 status, IntPtr frame)
{
if (frame == IntPtr.Zero)
{
return;
}
// To process RGB data directly, please refer to the following process
NT_SP_VideoFrame video_frame = (NT_SP_VideoFrame)Marshal.PtrToStructure(frame, typeof(NT_SP_VideoFrame));
NT_SP_VideoFrame pVideoFrame = new NT_SP_VideoFrame();
pVideoFrame.format_ = video_frame.format_;
pVideoFrame.width_ = video_frame.width_;
pVideoFrame.height_ = video_frame.height_;
pVideoFrame.timestamp_ = video_frame.timestamp_;
pVideoFrame.stride0_ = video_frame.stride0_;
pVideoFrame.stride1_ = video_frame.stride1_;
pVideoFrame.stride2_ = video_frame.stride2_;
pVideoFrame.stride3_ = video_frame.stride3_;
Int32 argb_size = video_frame.stride0_ * video_frame.height_;
pVideoFrame.plane0_ = Marshal.AllocHGlobal(argb_size);
CopyMemory(pVideoFrame.plane0_, video_frame.plane0_, (UInt32)argb_size);
if (playWnd.InvokeRequired)
{
BeginInvoke(set_video_frame_call_back_, status, pVideoFrame);
}
else{ set_video_frame_call_back_(status, pVideoFrame); }}public void SetAudioPCMFrameCallBack(IntPtr handle, IntPtr user_data, UInt32 status, IntPtr data, UInt32 size, Int32 sample_rate, Int32 channel, Int32 per_channel_sample_number)
{
if (data == IntPtr.Zero || size == 0)
{
return;
}
IntPtr pcmData = Marshal.AllocHGlobal((Int32)size);
CopyMemory(pcmData, data, (UInt32)size);
if (playWnd.InvokeRequired)
{
BeginInvoke(set_audio_pcm_frame_call_back_, status, pcmData, size, sample_rate, channel, per_channel_sample_number);
}
else{ set_audio_pcm_frame_call_back_(status, pcmData, size, sample_rate, channel, per_channel_sample_number); }}Copy the code8 **. ** Real-time ** snapshot: it is necessary to capture interesting or important pictures in real time. General players do not have the ability to snapshot, and open source players do not have this function.
/* * * file_name_UTF8: file name, UTF8 encoding * call_back_data: user-defined data when calling back * call_back: Callback function, which is used to notify the user screenshots have been completed or fail * successfully returns NT_ERC_OK * only in play sort of relief may be successful, the other is to use, return an error. * because generate PNG files are more time consuming, general need a few hundred milliseconds, to prevent excessive CPU, The SDK limits the number of screenshot requests, and when the number exceeds a certain point, * calling this interface returns NT_ERC_SP_TOO_MANY_CAPTURE_IMAGE_REQUESTS. In this case, delay for a while and try again after the SDK has processed some requests. */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_CaptureImage(IntPtr handle, IntPtr file_name_utf8, IntPtr call_back_data, SP_SDKCaptureImageCallBack call_back);
public void SDKCaptureImageCallBack(IntPtr handle, IntPtr userData, UInt32 result, IntPtr file_name)
{
if (file_name == IntPtr.Zero)
return;
int index = 0;
while (true)
{
if (0 == Marshal.ReadByte(file_name, index))
break;
index++;
}
byte[] file_name_buffer = new byte[index];
Marshal.Copy(file_name, file_name_buffer, 0, index);
byte[] dst_buffer = Encoding.Convert(Encoding.UTF8, Encoding.Default, file_name_buffer, 0, file_name_buffer.Length);
String image_name = Encoding.Default.GetString(dst_buffer, 0, dst_buffer.Length);
if (playWnd.InvokeRequired)
{
BeginInvoke(set_capture_image_call_back_, result, image_name);
}
else{ set_capture_image_call_back_(result, image_name); }}Copy the code9 **. Network jitter handling (such as network disconnection and reconnection) : ** Stable network handling mechanism and support, such as network disconnection and reconnection, etc. Open source player has poor support for network exception handling;
/ * event ID * /
public enum NT_SP_E_EVENT_ID : uint
{
NT_SP_E_EVENT_ID_BASE = NTBaseCodeDefine.NT_EVENT_ID_SMART_PLAYER_SDK,
NT_SP_E_EVENT_ID_CONNECTING = NT_SP_E_EVENT_ID_BASE | 0x2./* Connection */
NT_SP_E_EVENT_ID_CONNECTION_FAILED = NT_SP_E_EVENT_ID_BASE | 0x3./* Connection failed */
NT_SP_E_EVENT_ID_CONNECTED = NT_SP_E_EVENT_ID_BASE | 0x4./* Connected */
NT_SP_E_EVENT_ID_DISCONNECTED = NT_SP_E_EVENT_ID_BASE | 0x5./* Disconnect */
NT_SP_E_EVENT_ID_NO_MEDIADATA_RECEIVED = NT_SP_E_EVENT_ID_BASE | 0x8./* Failed to receive RTMP data */
NT_SP_E_EVENT_ID_RTSP_STATUS_CODE = NT_SP_E_EVENT_ID_BASE | 0xB./* RTSP status code Is reported. Currently, only 401 is reported. Param1 indicates status code*/
/* Start at 0x81 */
NT_SP_E_EVENT_ID_START_BUFFERING = NT_SP_E_EVENT_ID_BASE | 0x81./* Start buffer */
NT_SP_E_EVENT_ID_BUFFERING = NT_SP_E_EVENT_ID_BASE | 0x82./* In the buffer, param1 represents the percentage of progress */
NT_SP_E_EVENT_ID_STOP_BUFFERING = NT_SP_E_EVENT_ID_BASE | 0x83./* Stop buffering */
NT_SP_E_EVENT_ID_DOWNLOAD_SPEED = NT_SP_E_EVENT_ID_BASE | 0x91./* Download speed. Param1 indicates the download speed, expressed in Byte/s */
NT_SP_E_EVENT_ID_PLAYBACK_REACH_EOS = NT_SP_E_EVENT_ID_BASE | 0xa1./* When the playback ends, the live stream does not have this event, the on-demand stream only has */
NT_SP_E_EVENT_ID_RECORDER_REACH_EOS = NT_SP_E_EVENT_ID_BASE | 0xa2./* When the video is finished, the live stream does not have this event, the on-demand stream only has */
NT_SP_E_EVENT_ID_PULLSTREAM_REACH_EOS = NT_SP_E_EVENT_ID_BASE | 0xa3./* Pull stream ends, live stream does not have this event, on-demand stream only have */
NT_SP_E_EVENT_ID_DURATION = NT_SP_E_EVENT_ID_BASE | 0xa8./* Video duration. If the video duration is live, it is not reported. If the video duration can be obtained from the video source, it is reported. Param1 indicates the video duration in ms */
}
Copy the code**10. **** Long-term operation stability: ** The RTMP live broadcasting SDK provided by Daniu Live Broadcasting SDK is suitable for long-term operation, and the open source player has poor support for long-term operation stability;
** * Real-time download speed ** Feedback: ** Daniu Live SDK provides real-time download callback of audio and video streams, and can set the callback interval to ensure real-time download speed feedback, so as to monitor the network status, open source player does not have this ability;
/* * Set the download speed to report, which is not reported by default. * is_Report: switch for reporting, 1: table reporting 0: table reporting not reporting * report_interval: reporting interval (reporting frequency), expressed in seconds. The minimum value is once every second. If the value is less than 1, the call will fail * Note: If the value is set to report, set SetEventCallBack and process the event in the callback function. NT_SP_E_EVENT_ID_DOWNLOAD_SPEED * This interface must call * successfully before StartXXX returns NT_ERC_OK */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetReportDownloadSpeed(IntPtr handle, Int32 is_report, Int32 report_interval);
/* * Active download speed * speed: Returns the download speed in Byte/s * (note: this interface must be called after startXXX, otherwise it will fail) * NT_ERC_OK */ is returned on success
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_GetDownloadSpeed(IntPtr handle, ref Int32 speed);
Copy the code** If the playing process is interrupted, the player provided by DANIu Live SDK can call back the relevant state in real time to ensure that the upper module is aware of the processing, which is not well supported by the open source player;
/* * Set the event callback. If you want to listen for events, it is recommended to call this interface */ after the successful call to Open
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetEventCallBack(IntPtr handle, IntPtr call_back_data, SP_SDKEventCallBack call_back);
/* * Set the video size callback interface */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetVideoSizeCallBack(IntPtr handle, IntPtr callbackdata, SP_SDKVideoSizeCallBack call_back);
* FRAME_format: NT_SP_E_VIDEO_FRAME_FORMAT_RGB32, NT_SP_E_VIDEO_FRAME_FROMAT_I420 */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetVideoFrameCallBack(IntPtr handle, Int32 frame_format, IntPtr callbackdata, SP_SDKVideoFrameCallBack call_back);
/* * Set the video callback to spit out video data. You can specify the width and height of the spit out video * handle: Player handle * scale_width: zoom width (must be even, recommended multiples of 16) * scale_height: zoom height (must be even) * scale_filter_mode: [1, 3], the larger the value, the better the quality of the scale, but the more performance consumption * frame_format: Only NT_SP_E_VIDEO_FRAME_FORMAT_RGB32, NT_SP_E_VIDEO_FRAME_FROMAT_I420 * Returns NT_ERC_OK */ on success
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetVideoFrameCallBackV2(IntPtr handle, Int32 scale_width, Int32 scale_height, Int32 scale_filter_mode, Int32 frame_format, IntPtr call_back_data, SP_SDKVideoFrameCallBack call_back);
/* * Set the video frame timestamp callback when drawing a video frame * Note that if the current stream is pure audio, it will not be called back, this only works if there is video */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetRenderVideoFrameTimestampCallBack(IntPtr handle, IntPtr callbackdata, SP_SDKRenderVideoFrameTimestampCallBack call_back);
/* * Set the audio PCM frame callback to output PCM data. Currently, each frame size is 10ms. */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetAudioPCMFrameCallBack(IntPtr handle, IntPtr call_back_data, SP_SDKAudioPCMFrameCallBack call_back);
/* * Set user data callback */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetUserDataCallBack(IntPtr handle, IntPtr call_back_data, SP_SDKUserDataCallBack call_back);
/* * Set the video SEI data callback */
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetSEIDataCallBack(IntPtr handle, IntPtr call_back_data, SP_SDKSEIDataCallBack call_back);
Copy the code13. Set the video fill mode (equal proportion display) : In many cases, some scenes need to be played with full view, and some scenes can be set to equal proportion display in order to prevent video stretching;
/* * Set the filling mode of the video picture, such as filling the whole drawing window or equal proportion filling the drawing window. If not set, the entire drawing window will be filled by default. * Handle: play handle * mode: 0: fill the whole drawing window; 1: Equal scale fill drawing window, default is 0 * return NT_ERC_OK */ on success
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_SetRenderScaleMode(IntPtr handle, Int32 mode);
Copy the code14. D3D detection: Generally speaking, most Windows on the market support D3D, some niche, only support GDI mode drawing, so in order to better compatibility, this interface is very necessary.
*is_support: 1 if it is supported *is_support is 0 if it is not supported * NT_ERC_OK */ is returned on success of the interface call
[DllImport(@"SmartPlayerSDK.dll")]
public static extern UInt32 NT_SP_IsSupportD3DRender(IntPtr handle, IntPtr hwnd, ref Int32 is_support);
Copy the code