Elsampsa/Websocket-MSE-Demo Project Introduction:

It this demo we’re streaming live video from an RTSP camera to your HTML5 browser.

Video is streamed as H264 encapsulated in MP4. No transcoding takes place in the stream-to-browser pipeline. No browser-side flash or pure-javascript decoders required.

Video is decoded and presented in the browser using the W3 Media Source Extensions that is well supported by all major browsers by > now.

Only H264 is supported – browser MSE’s do not support H246 yet (and neither does libValkka).

The Real Time Streaming Protocol (RTSP) stream is played by MSE (Media Source Extensions). Have a hands-on experience

To prepare

# System Ubuntu 18

If you have conda virtual environment, please disable it
# conda deactivate

# fetch code
git clone --depth 1 https://github.com/elsampsa/websocket-mse-demo.git

# installation libValkka
sudo apt-add-repository ppa:sampsa-riikonen/valkka
sudo apt-get update
sudo apt-get install valkka

# to install nginx
sudo apt-get install nginx

# Install Python WebSockets
pip3 install websockets

# Prepare nginx user permissions
sudo addgroup nginx
sudo adduser $USER nginx
sudo chgrp -R nginx /var/lib/nginx /var/log/nginx
sudo chmod -R g+r+w+x /var/lib/nginx /var/log/nginx
# Change the user group
newgrp nginx
id

# check whether nginx works properly (no Permission denied)
cd websocket-mse-demo
nginx -p $PWD -c ./nginx.conf -g 'error_log error.log warn; '

# Set the camera to send H264 (this project only supports H264)
Copy the code

use

Sending an RTSP stream:

sudo snap install vlc
# allow camera permission
snap connect vlc:camera :camera

# Send ~/Videos/test.mp4 files (loop)
# https://archive.org/download/archive-video-files/test.mp4
vlc file:///$HOME/Videos/test.mp4 --loop \
--sout '#gather:transcode{vcodec=h264}:rtp{sdp=rtsp://:8554/test}' \
--network-caching=1500 --sout-all --sout-keep

/dev/video0 webcam
vlc v4l2:///dev/video0:chroma=yuyv:width=640:height=480:fps=30 \
--sout '#transcode{vcodec=h264,width=640,height=480}:rtp{sdp=rtsp://:8554/test}'

Test receiving RTSP streamsVLC RTSP: / / 127.0.0.1:8554 /test
Copy the code

Start procedure:

Killall -9 nginx python3 ws_serve_new.py RTSP ://127.0.0.1:8554/test
Copy the code

Open your browser:

xdg-open http://localhost:8089
Copy the code

More and more

Technology:

  • WebSocket
  • MediaSource: Builds a media stream for video/audio elements to play
  • WebRTC: it can directly capture client video streams and transmit them point-to-point, independent of server transfer
  • WebCodecs: Encoding and decoding audio and video
  • WebAssembly: C/C++/Rust etc compiled into the front-end use
    • ffmpegwasm/ffmpeg.wasm
    • GoogleChromeLabs/webm-wasm

Project:

  • MSE: Streams need to be constructed, usually in ISO BMFF (Fragmented MP4) = fMP4 container format
    • elsampsa/websocket-mse-demo: RTSP > fMP4
    • ChihChengYang/wfs.js: H.264 NAL > fMP4
    • bilibili/flv.js: FLV > fMP4
    • Garefield/RTSP-WS-Proxy: RTSP > fMP4
    • Streamedian/ html5_rtSP_player: RTSP > fMP4, Proxy charge
  • JS decoding
    • phoboslab/jsmpeg: MPEG1 Video & MP2 Audio Decoder
      • kyriesent/node-rtsp-stream
      • wanghaoxi3000/gin-rtsp
      • YE-Fan/JAVA-RTSP-JSMpeg
      • Aleckgt/rtsp-ws-proxy
      • k-yle/rtsp-relay
    • mbebenita/Broadway: H.264 decoder
      • 131/h264-live-player
  • Wasm decoding
    • sonysuqin/WasmVideoPlayer: H265 > YUV, WebGL
    • wupeng-engineer/decoder_wasm: H264,H265 > YUV, WebGL
    • gliese1337/h264decoder: H264 > YUV
    • mattdesl/mp4-h264: H264 > YUV/RGB
    • mattdesl/mp4-wasm: H264
  • other
    • HTML5 RTSP Player: Charging

GoCoding personal practice experience sharing, please pay attention to the public account!