Java to achieve dahua video stream push stream live

preface

At present, camera live broadcast schemes mainly include RTSP, GB protocol, calling SDK of manufacturers to obtain video stream and so on. Dahua SDK supports the active registration of cameras to connect to the server. In this way, the video stream can be obtained remotely and the camera can be controlled. By pushing the video stream to the streaming media server, the video can be viewed and controlled in real time on mobile phones and web pages through FLV, WSFLV, HLS and other mainstream streaming media protocols.

Thanks to Banmajio for inspiring me with this article (blog.csdn.net/weixin_4077…

First, use the technology stack

springboot+javaCV+jna+nms

Two, implementation steps

2.1 Download Dahua netSDK from the official website

www.dahuatech.com/service/dow…

I mainly use the Java Win 64-bit version

2.2 Streaming Media Server Deployment

Github.com/illuspas/No…

I choose the open source version of Node-Media-Server, which is very convenient to deploy with Docker

Windows 10 can be deployed with a Docker Desktop installed

docker run --name nms -d -p 1935:1935 -p 1934:8000 illuspas/node-media-server
Copy the code

Deployment can log in after the completion of management page to view http://localhost:1934/admin/streams, the default user name password is admin

Port 1935 is the RTMP port, which is the port for pushing the flow, and 1934 is the address for pulling the flow

2.3 Downloading the Player

www.videolan.org/ Download the VLC player for later use

2.4 Project Construction

Create a new Spring Boot project

2.4.1 Introduction of DAhua NET SDK

According to the official website demo to introduce DAhua NET SDK, win64 library files into the resources directory (libraries in other environments according to the need to add), the following provides a general method of loading DLL files

NetSDKLib NETSDK_INSTANCE = (NetSDKLib)NetSDKLib.LoadHelper.loadDll("dhnetsdk", NetSDKLib.class);

	NetSDKLib CONFIG_INSTANCE = (NetSDKLib)NetSDKLib.LoadHelper.loadDll("dhconfigsdk", NetSDKLib.class);

    @Slf4j
    public static class LoadHelper {
        public synchronized static Object loadDll(String libName, Class
        className) {
            String libExtension = ".dll", systemType = "win";
            String libFullName = libName + libExtension;

            if (Platform.getOSType() == Platform.LINUX) {
                    libExtension = ".so";
                    systemType = "linux";
                    libFullName = "lib" + libName + libExtension;
            }

            String nativeTempDir = System.getProperty("java.io.tmpdir");

            InputStream in = null;
            BufferedInputStream reader = null;
            FileOutputStream writer = null;
            File extractedLibFile = new File(nativeTempDir + File.separator + libFullName);

            if(! extractedLibFile.exists()) {try {
                    systemType += System.getProperty("sun.arch.data.model");
                    Resource resource = new ClassPathResource("/libs/"+ systemType + "/" + libFullName);
                    in = resource.getInputStream();

                    if (in == null) {return null;
                    }
                    reader = new BufferedInputStream(in);
                    writer = new FileOutputStream(extractedLibFile);
                    byte[] buffer = new byte[1024];
                    while (reader.read(buffer) > 0) {
                        writer.write(buffer);    // Write DLL /so/dylib to temporary file
                        buffer = new byte[1024]; }}catch (IOException e) {
                    log.error(e.getMessage());
                } finally {
                    if(in ! =null) try {
                        in.close();
                    } catch (IOException e) {
                        log.error(e.getMessage());
                    }
                    if(writer ! =null) try {
                        writer.close();
                    } catch (IOException e) {
                        log.error(e.getMessage());
                    }
                }
            }
            String temp = extractedLibFile.toString();
            String dllName = temp.substring(0,temp.indexOf("."));
            if (Platform.getOSType() == Platform.LINUX) {
                dllName = temp;
            }
            returnNative.loadLibrary(dllName,className); }}Copy the code

The ToolKits provided by DAhua have included rich camera operation methods, which can be further packaged based on our own business, or added by jNA based on THE NETWORK SDK Development Manual. CHM of DAhua C edition.

2.4.2 Active Registration

To implement active registration, the server needs to implement the fServiceCallBack interface provided by NetSDK, register the callback class that implements disconnection and reconnection with the device with NetSDK, and bind port 9500 of the local device as the port that the device actively registers.

 @EventListener
    public void initNetSdk(ContextRefreshedEvent event) {
        if (Platform.getOSType() == Platform.WINDOWS) {
            try {
                // Initialize the SDK library
                netSdk.CLIENT_Init(DisConnectCallBack.getInstance(), null);
                // Set the callback function for successful disconnection
                netSdk.CLIENT_SetAutoReconnect(HaveReConnectCallBack.getInstance(), null);
                // Enable active registration
                ServiceCB servicCallback = ServiceCB.getInstance();
                netSdk.CLIENT_ListenServer(autoRegIp, 9500.1000, servicCallback, null);
            } catch (Exception e) {
                log.error(e.getMessage());
                log.error(this.netSdk.CLIENT_GetLastError()+""); }}}Copy the code

Log in to the configuration page of DAhua camera, configure the camera, enable active registration, and configure the active registration address (IP is the local address, which can be viewed through ipconfig) and device serial number (ensure that multiple devices do not duplicate).

In the invoke method of the implementation class, lCommand is judged to be EM_listen_Type.net_DVr_serial_RETURN, and the serial number is carried for the active registration of the device. The following is the information printing carried by the active registration of the device.

The ERROR com.zb.video.net SDK. Event. Register ServiceCB: 54-1 Device Info [Device address 192.168.1.200] [port 62801][DeviceID 6F06F7DPAJ3C924][lCommand 1]!Copy the code

And then through the camera’s serial number, login name and password to log in, the main call netSdk. CLIENT_LoginWithHighLevelSecurity method to log in.

NetSDKLib.NET_DEVICEINFO_Ex deviceInfo = new NetSDKLib.NET_DEVICEINFO_Ex();
Pointer deviceId = ToolKits.GetGBKStringToPointer(serNum);
/ / into the refs
NetSDKLib.NET_IN_LOGIN_WITH_HIGHLEVEL_SECURITY pstInParam=new NetSDKLib.NET_IN_LOGIN_WITH_HIGHLEVEL_SECURITY();
pstInParam.nPort=vsDeviceLoginInfo.getPort();
pstInParam.szIP=vsDeviceLoginInfo.getIp().getBytes();
pstInParam.szPassword=vsDeviceLoginInfo.getPassword().getBytes();
pstInParam.szUserName=vsDeviceLoginInfo.getUser().getBytes();
pstInParam.emSpecCap = 2;// Active registration
pstInParam.pCapParam=deviceId;
/ / the refs
NetSDKLib.NET_OUT_LOGIN_WITH_HIGHLEVEL_SECURITY pstOutParam=new NetSDKLib.NET_OUT_LOGIN_WITH_HIGHLEVEL_SECURITY();
pstOutParam.stuDeviceInfo=deviceInfo;
// Login handle
LLong loginHandle = netSdk.CLIENT_LoginWithHighLevelSecurity(pstInParam, pstOutParam);
Copy the code

To obtain the handle to login to 0 for the login successfully, otherwise you need to call ErrorCode. GetErrorCode (NetSDKConfig.net Sdk. CLIENT_GetLastError ()) to obtain login failure reason, After a successful login, you can invoke other interfaces through the login handle to perform various operations on the device.

2.4.3 Open preview, obtain dahua video stream and push it

After a successful login, the opening and closing of video stream can be controlled through THE HTTP interface. The opening of video stream includes calling netSDK real-time preview interface, and depackaging dahua video stream in video stream callback to obtain H264 naked code stream. The H264 stream is written to PipedOutputStream, and the corresponding PipedInputStream is passed as a parameter to the JavacV FFmpegFrameGrabber constructor, which pushes the stream through JavacV to port 1935 of the NMS. If you don’t know JAVACV, you can check the corresponding information online. I do this by consulting information and communicating with bloggers.

PipedInputStream and PipedOutputStream appear in pairs and need to be connected to work properly. You can establish a connection in either of the following ways:

PipedInputStream inputStream = new PipedInputStream(); PipedOutputStream outputStream = new PipedOutputStream(inputStream); PipedInputStream inputStream = new PipedInputStream(); PipedOutputStream outputStream = new PipedOutputStream(); inputStream.connect(outputStream);Copy the code

When starting the stream pushing interface, pay attention to determine whether the current camera has started to push the stream. If the current camera has started to push the stream, it will directly return the previous stream address. If it is not enabled, call dahua NetSDK interface first to obtain video stream:

Public void startRealplay(LLong loginHandle,Integer playSign,CameraPojo CameraPojo){// Default is the secondary stream int streamType = 3; If (stringutils.equals (camerapojo.getstream (),"main")){streamType=0; } / / open the preview LLong lRealHandle = NetSDKConfig.net Sdk. CLIENT_RealPlayEx (loginHandle, cameraPojo getChannel (), null, streamType); if(lRealHandle.longValue()! // Set the video stream callback if(NetSDKConfig.netSdk.CLIENT_SetRealDataCallBackEx(lRealHandle,RealDataCallBack.getInstance(),null, 31)){ } } catch (IOException e) { log.error(e.getMessage()); }}}Copy the code

In the video stream callback, dahua video stream needs to be encapsulated and processed by DAhua head to obtain H264 naked code stream. You can obtain the decoding library from the official website or query the corresponding method online. The processed raw code is exiled to the PipedOutputStream, and then the transwrap process of JavacV is called (mainly using FFmpegFrameGrabber and FFmpegFrameRecorder). Get the video stream from PipedInputStream encapsulate the VIDEO in H264 format into FLV format and push it to port 1935 of NMS. There are too many intermediate codes to paste one by one, which mainly provides an idea for everyone.

Maven introduces Javacv:

<dependency> <groupId>org.bytedeco</groupId> <artifactId> Javacv </artifactId> <version>1.5.3</version> </dependency> < the dependency > < groupId > org. Bytedeco < / groupId > < artifactId > ffmpeg < / artifactId > < version > 4.2.2-1.5.3 < / version > </dependency> <dependency> <groupId>org.bytedeco</groupId> <artifactId>ffmpeg</artifactId> <version>4.2.2-1.5.3</version> <classifier> Windows-x86_64 </classifier> --<classifier>linux-x86_64</classifier>--> </dependency>Copy the code

2.5 Effect

After configuring the camera and deploying the NMS, run the project and wait for the camera to register

Swagger calls the interface implemented to enable video stream and pushes the video of the main stream to the NMS

Copy the FLV address after successful push and play network stream in VLC for viewing

At the same time, it can also be played on other streaming media players, and the page can be played using FLv.js of station B. This video stream is also available on the NMS management page.

Third, summary

This paper provides a basic idea to realize real-time live broadcast of Dahua camera through Dahua NetSDK. If the cloud recording function is needed, it can also be developed based on JavacV. In addition, some simple video analysis can be achieved through JavacV periodic screenshots combined with Baidu ARTIFICIAL intelligence API. Based on this idea, you can combine JavacV to do more processing on the video stream obtained through NetSDK.