How to implement the Android for RTSP | RTMP push flow RTMP

Technical background

Recently, many developers came to us. When they are doing traditional industries such as smart home, they hope to pull local RTSP or RTMP stream from Android board, and then push RTMP out externally, or create a lightweight RTSP service internally to provide a media URL for external docking. In brief, the design architecture diagram is as follows:

Based on the appeal, we take the SmartRelayDemoV2 project of The Android terminal of Daniu Live SDK (official) as an example to roughly introduce the relevant implementation.

The overall design

1. Pull flow: through RTSP | RTMP live broadcast data callback interface of the SDK to the audio and video data;

2. Turn on: by live RTMP push SDK encoded data input interface, the correction of data, to live RTMP push module, realize the RTSP | RTMP data stream to the RTMP server forwarding;

3. Video: if you need a video, with the aid of RTSP | RTMP live broadcast the SDK, to audio and video data, direct storage MP4 files;

4. Snapshot: If a real-time snapshot is required, after the stream is pulled, the decoding invokes the snapshot interface at the player end to generate a snapshot. Snapshot involves video data decoding.

5. Pull stream preview: If you need to preview pull stream data, just call the playback interface of the player end to realize the pull stream data preview;

6. Data transfer and forwarding after AAC: Considering that the audio produced by many monitoring devices may be from PCMA/PCMU, if a more general audio format is needed, it can be transferred to AAC and pushed through RTMP;

7. RTMP real-time mute: Just add a judgment in the place where audio data is transmitted;

8. Pull-flow speed feedback: The real-time bit rate feedback event of the RTSP player can be obtained to obtain the real-time bandwidth usage;

9. Overall network status feedback: Considering that some cameras may be temporarily or abnormally shut down, as is the case with the RTMP server, you can view the overall network situation by calling back the event status of the push-pull stream, thus defining: whether the stream cannot be pulled, or the RTMP server cannot be pushed;

10. Data injection Lightweight RTSP service: Pull stream data, inject lightweight RTSP service, and provide RTSP URL externally.

The above first

Demo mainly realizes the following functions:

1. Set the URL of the RTMP and RTSP streams.

2. Set the URL for pushing RTMP.

3. In the process of real-time video broadcast |, mute, implementing snapshot in real time;

4. Real-time playback;

5. Real-time video recording;

6. Push the pulled stream data in real time, corresponding to “Start push stream”;

7. The pulled stream data is injected into the lightweight RTSP service. After starting the service, the RTSP stream is published and the accessible RTSP URL is provided externally.

Note: the above four functions of playing, recording, pushing RTMP and injecting lightweight RTSP service can work independently or start or stop relevant functions at any time without affecting each other.

Related code implementation

He began to pull on the flow

The purpose of pull stream is to start data callback. Note: Pull stream is not directly played out of the window, just take the data. If you want to preview the pull stream data locally, you can click “Start Playing”.

Note: Before “Start pushing stream” and “publish RTSP stream”, be sure to “start pulling stream” and get audio and video data.

	private boolean StartPull(a)
	{
		if ( isPulling )
			return false;

		if(! OpenPullHandle())return false;

		libPlayer.SmartPlayerSetAudioDataCallback(playerHandle, new PlayerAudioDataCallback());
		libPlayer.SmartPlayerSetVideoDataCallback(playerHandle, new PlayerVideoDataCallback());

		int is_pull_trans_code  = 1;
		libPlayer.SmartPlayerSetPullStreamAudioTranscodeAAC(playerHandle, is_pull_trans_code);

		int startRet = libPlayer.SmartPlayerStartPullStream(playerHandle);

		if(startRet ! =0) {
			Log.e(TAG, "Failed to start pull stream!");

			if(! isPlaying && ! isRecording && isPushing && ! isRTSPPublisherRunning) { libPlayer.SmartPlayerClose(playerHandle); playerHandle =0;
			}

			return false;
		}

		isPulling = true;
		return true;
	}
Copy the code

OpenPullHandle() encapsulates OpenPullHandle(), which is to start the investigation of the Open() interface of the Player, get the Player Handle, and then set the basic data interface, such as event callback, buffer time, TCP/UDP mode, pull flow URL, etc.

	private boolean OpenPullHandle(a)
	{
		if(playerHandle ! =0) {
			return true;
		}

		playbackUrl = "RTSP: / / admin: [email protected]:554 / h265 / ch1 / main/av_stream";

		if (playbackUrl == null) {
			Log.e(TAG, "playback URL with NULL...");
			return false;
		}

		playerHandle = libPlayer.SmartPlayerOpen(myContext);

		if (playerHandle == 0) {
			Log.e(TAG, "playerHandle is nil..");
			return false;
		}

		libPlayer.SetSmartPlayerEventCallbackV2(playerHandle,
				new EventHandePlayerV2());

		libPlayer.SmartPlayerSetBuffer(playerHandle, playBuffer);

		// set report download speed
		// libPlayer.SmartPlayerSetReportDownloadSpeed(playerHandle, 1, 5);

		// Set the RTSP timeout
		int rtsp_timeout = 12;
		libPlayer.SmartPlayerSetRTSPTimeout(playerHandle, rtsp_timeout);

		// Set the RTSP TCP/UDP mode to automatically switch
		int is_auto_switch_tcp_udp = 1;
		libPlayer.SmartPlayerSetRTSPAutoSwitchTcpUdp(playerHandle, is_auto_switch_tcp_udp);

		libPlayer.SmartPlayerSaveImageFlag(playerHandle, 1);

		// It only used when playback RTSP stream..
		//libPlayer.SmartPlayerSetRTSPTcpMode(playerHandle, 1);

		libPlayer.SmartPlayerSetUrl(playerHandle, playbackUrl);

		return true;
	}
Copy the code

Stop pulling flow

	private void StopPull(a)
	{
		if ( !isPulling )
			return;

		libPlayer.SmartPlayerStopPullStream(playerHandle);

		if(! isPlaying && ! isRecording && ! isPushing && ! isRTSPPublisherRunning) { libPlayer.SmartPlayerClose(playerHandle); playerHandle =0;
		}

		isPulling = false;
	}
Copy the code

Start playing

	private boolean StartPlay(a)
	{
		if(! OpenPullHandle())return false;

		// If the second parameter is set to null, pure audio is played
		libPlayer.SmartPlayerSetSurface(playerHandle, sSurfaceView);

		libPlayer.SmartPlayerSetRenderScaleMode(playerHandle, 1);

		// External Render test
		// libPlayer.SmartPlayerSetExternalRender(playerHandle, new
		// RGBAExternalRender());
		// libPlayer.SmartPlayerSetExternalRender(playerHandle, new
		// I420ExternalRender());

		libPlayer.SmartPlayerSetFastStartup(playerHandle, isFastStartup ? 1 : 0);

		libPlayer.SmartPlayerSetAudioOutputType(playerHandle, 1);

		if (isMute) {
			libPlayer.SmartPlayerSetMute(playerHandle, isMute ? 1
					: 0);
		}

		if (isHardwareDecoder)
		{
			int isSupportHevcHwDecoder = libPlayer.SetSmartPlayerVideoHevcHWDecoder(playerHandle, 1);

			int isSupportH264HwDecoder = libPlayer
					.SetSmartPlayerVideoHWDecoder(playerHandle, 1);

			Log.i(TAG, "isSupportH264HwDecoder: " + isSupportH264HwDecoder + ", isSupportHevcHwDecoder: " + isSupportHevcHwDecoder);
		}

		libPlayer.SmartPlayerSetLowLatencyMode(playerHandle, isLowLatency ? 1
				: 0);

		libPlayer.SmartPlayerSetRotation(playerHandle, rotate_degrees);

		int iPlaybackRet = libPlayer
				.SmartPlayerStartPlay(playerHandle);

		if(iPlaybackRet ! =0) {
			Log.e(TAG, "StartPlay failed!");

			if(! isPulling && ! isRecording && ! isPushing && ! isRTSPPublisherRunning) { libPlayer.SmartPlayerClose(playerHandle); playerHandle =0;
			}

			return false;
		}

		isPlaying = true;
		return true;
	}
Copy the code

Stop playing

	private void StopPlay(a)
	{
		if ( !isPlaying )
			return;

		isPlaying = false;

		libPlayer.SmartPlayerStopPlay(playerHandle);

		if(! isPulling && ! isRecording && ! isPushing && ! isRTSPPublisherRunning) { libPlayer.SmartPlayerClose(playerHandle); playerHandle =0; }}Copy the code

Start the video

	private boolean StartRecorder(a)
	{
		if(! OpenPullHandle())return false;

		ConfigRecorderFuntion();

		int iRecRet = libPlayer
				.SmartPlayerStartRecorder(playerHandle);

		if(iRecRet ! =0) {
			Log.e(TAG, "StartRecorder failed!");

			if(! isPulling &&! isPlaying && ! isPushing && ! isRTSPPublisherRunning) { libPlayer.SmartPlayerClose(playerHandle); playerHandle =0;
			}

			return false;
		}

		isRecording = true;
		return true;
	}
Copy the code

Stop the video

private void StopRecorder() { if ( ! isRecording ) return; isRecording = false; libPlayer.SmartPlayerStopRecorder(playerHandle); if ( ! isPlaying && ! isPulling && ! isPushing && ! isRTSPPublisherRunning) { libPlayer.SmartPlayerClose(playerHandle); playerHandle = 0; }}Copy the code

Began pushing flow

	private boolean StartPush(a)
	{
		if (isPushing)
			return false;

		relayStreamUrl = "RTMP: / / 192.168.0.211:1935 / HLS/stream1";

		if (relayStreamUrl == null) {
			Log.e(TAG, "StartPush URL is null...");
			return false;
		}

		if(! OpenPushHandle())return false;

		if( libPublisher.SmartPublisherSetURL(publisherHandle, relayStreamUrl) ! =0 )
		{
			Log.e(TAG, "StartPush failed!");
		}

		int startRet = libPublisher.SmartPublisherStartPublisher(publisherHandle);
		if( startRet ! =0)
		{
			Log.e(TAG, "Failed to call StartPublisher!");

			if(isRTSPPublisherRunning)
			{
				libPublisher.SmartPublisherClose(publisherHandle);
				publisherHandle = 0;
			}

			return false;
		}

		isPushing = true;

		return true;
	}
Copy the code

The OpenPushHandle() package is used as follows:

	private boolean OpenPushHandle(a)
	{
		if(publisherHandle ! =0)
		{
			return true;
		}

		int audio_opt = 2;
		int video_opt = 2;

		int videoWidth = 640;
		int videoHeight  = 480;

		publisherHandle = libPublisher.SmartPublisherOpen(myContext, audio_opt, video_opt,
				videoWidth, videoHeight);

		if (publisherHandle == 0 )
		{
			Log.e(TAG, "OpenPushHandle failed!");
			return false;
		}

		Log.i(TAG, "publisherHandle=" + publisherHandle);

		libPublisher.SetSmartPublisherEventCallbackV2(publisherHandle, new EventHandePublisherV2());

		return true;
	}
Copy the code

Stop pushing flow

	public void StopPush(a)
	{
		if(! isPushing)return;

		isPushing = false;

		libPublisher.SmartPublisherStopPublisher(publisherHandle);

		if(! isRTSPPublisherRunning && ! isRTSPServiceRunning) { libPublisher.SmartPublisherClose(publisherHandle); publisherHandle =0; }}Copy the code

Start the RTSP service

	// Start/stop the RTSP service
	class ButtonRtspServiceListener implements OnClickListener {
		public void onClick(View v) {
			if (isRTSPServiceRunning) {
				stopRtspService();

				btnRtspService.setText("Start RTSP service");
				btnRtspPublisher.setEnabled(false);

				isRTSPServiceRunning = false;
				return;
			}

			if(! OpenPushHandle()) {return;
			}

			Log.i(TAG, "onClick start rtsp service..");

			rtsp_handle_ = libPublisher.OpenRtspServer(0);

			if (rtsp_handle_ == 0) {
				Log.e(TAG, "Failed to create RTSP Server instance! Please check SDK validity.");
			} else {
				int port = 8554;
				if(libPublisher.SetRtspServerPort(rtsp_handle_, port) ! =0) {
					libPublisher.CloseRtspServer(rtsp_handle_);
					rtsp_handle_ = 0;
					Log.e(TAG, "Failed to create RTSP server port! Please check for duplicate ports or ports out of range!");
				}

				//String user_name = "admin";
				//String password = "12345";
				//libPublisher.SetRtspServerUserNamePassword(rtsp_handle_, user_name, password);

				// Generally speaking, unicast network devices support well, wifi multicast is not supported by many routers, the default unicast mode; If you want to use the multicast mode, ensure that the device supports it, open the comment code test
                /* boolean is_enable_multicast = true; if(is_enable_multicast) { int is_multicast = 1; libPublisher.SetRtspServerMulticast(rtsp_handle_, is_multicast); boolean is_enable_ssm_multicast = true; String multicast_address = ""; if(is_enable_ssm_multicast) { multicast_address = MakeSSMMulticastAddress(); } else { multicast_address = MakeMulticastAddress(); } Log.i(TAG, "is_enable_ssm_multicast:" + is_enable_ssm_multicast + " multiAddr: " + multicast_address); libPublisher.SetRtspServerMulticastAddress(rtsp_handle_, multicast_address); } * /

				if (libPublisher.StartRtspServer(rtsp_handle_, 0) = =0) {
					Log.i(TAG, "RTSP Server started successfully!");
				} else {
					libPublisher.CloseRtspServer(rtsp_handle_);
					rtsp_handle_ = 0;
					Log.e(TAG, "Failed to start RTSP Server! Please check whether the set port is occupied!");
				}

				btnRtspService.setText("Stop RTSP service");
				btnRtspPublisher.setEnabled(true);

				isRTSPServiceRunning = true; }}}Copy the code

Stop the RTSP service

	// Stop RTSP service
	private void stopRtspService(a) {
		if(! isRTSPServiceRunning)return;

		if(libPublisher ! =null&& rtsp_handle_ ! =0) {
			libPublisher.StopRtspServer(rtsp_handle_);
			libPublisher.CloseRtspServer(rtsp_handle_);
			rtsp_handle_ = 0;
		}

		if(! isPushing) { libPublisher.SmartPublisherClose(publisherHandle); publisherHandle =0;
		}

		isRTSPServiceRunning = false;
	}
Copy the code

Start publishing RTSP streams

	private boolean StartRtspStream(a)
	{
		if (isRTSPPublisherRunning)
			return false;

		String rtsp_stream_name = "stream1";
		libPublisher.SetRtspStreamName(publisherHandle, rtsp_stream_name);
		libPublisher.ClearRtspStreamServer(publisherHandle);

		libPublisher.AddRtspStreamServer(publisherHandle, rtsp_handle_, 0);

		if (libPublisher.StartRtspStream(publisherHandle, 0) != 0)
		{
			Log.e(TAG, "Failed to call publish RTSP stream interface!");

			if(! isPushing) { libPublisher.SmartPublisherClose(publisherHandle); publisherHandle =0;
			}

			return false;
		}

		isRTSPPublisherRunning = true;
		return true;
	}
Copy the code

Stop publishing the RTSP stream

	// Stop publishing RTSP streams
	private void stopRtspPublisher(a)
	{
		if(! isRTSPPublisherRunning)return;

		if(libPublisher ! =null) {
			libPublisher.StopRtspStream(publisherHandle);
		}

		if(! isPushing && ! isRTSPServiceRunning) { libPublisher.SmartPublisherClose(publisherHandle); publisherHandle =0;
		}

		isRTSPPublisherRunning = false;
	}
Copy the code

Gets the number of RTSP connection sessions

	// The number of current RTSP sessions dialog box is displayed
	private void PopRtspSessionNumberDialog(int session_numbers) {
		final EditText inputUrlTxt = new EditText(this);
		inputUrlTxt.setFocusable(true);
		inputUrlTxt.setEnabled(false);

		String session_numbers_tag = "RTSP service current client sessions:" + session_numbers;
		inputUrlTxt.setText(session_numbers_tag);

		AlertDialog.Builder builderUrl = new AlertDialog.Builder(this);
		builderUrl
				.setTitle("Built-in RTSP service")
				.setView(inputUrlTxt).setNegativeButton("Sure".null);
		builderUrl.show();
	}

	// Get the number of RTSP sessions
	class ButtonGetRtspSessionNumbersListener implements OnClickListener {
		public void onClick(View v) {
			if(libPublisher ! =null&& rtsp_handle_ ! =0) {
				int session_numbers = libPublisher.GetRtspServerClientSessionNumbers(rtsp_handle_);

				Log.i(TAG, "GetRtspSessionNumbers: "+ session_numbers); PopRtspSessionNumberDialog(session_numbers); }}};Copy the code

conclusion

Above is the general process, interested developers can refer to their own.