GStreamer is a library for building diagrams of media processing components (also known as pipelines, or pipes). It supports a wide range of applications, from simple Ogg/Vorbis playback, audio/video streaming to complex audio (mixing) and video (non-linear editing) processing.

Applications can transparently take advantage of advances in codec and filter technology. Developers can add new codecs and filters by writing simple plug-ins based on a clean, generic interface.

GStreamer runs on all major operating system platforms such as Linux, Android, Windows, Max OS X, iOS, as well as most BSDs, commercial Unixes, Solaris, and Symbian. It has been ported to a wide range of operating system, processor, and compiler platforms. It runs on all major hardware architectures, including x86, ARM, MIPS, SPARC, and PowerPC, 32-bit and 64-bit, and on small and large tails.

GStreamer can bridge to other multimedia frameworks to reuse existing components (such as codecs) and use platform input/output mechanisms:

  • Linux/Unix: Openmax-IL (via GST-OMX)
  • Windows: : DirectShow
  • Mac OS X: QuickTime

GStreamer core framework

  • The graph-based structure allows you to build pipelines of any shape
  • Object-oriented design and inheritance based on GLib 2.0 object model
  • A compact core library of less than 500KB, with about 65K lines of code
  • Building multithreaded pipelines is easy and transparent
  • There are clean, simple and stable apis for plug-in and application developers alike
  • Extremely lightweight data processing means very high performance/low latency.
  • There is a complete debugging system for both core architecture and plug-in/application developers
  • Has a clock to ensure global interstream synchronization (A/V synchronization)
  • Quality of service (qos) to ensure optimal possible quality under high CPU load.

Smart Plug-in Architecture

  • Dynamically loaded plug-ins provide elements and media types that are loaded on demand through a registry cache, similar to ld.so.cache.
  • Element interfaces handle all known types of source, filter, and sinks
  • The empowerment system allows you to verify element compatibility using MIME types and media-specific properties
  • Auto – plug – in access allows the system to automate complex paths
  • You can visualize the pipeline by dumping it into a.dot file and creating a PNG image based on it.
  • Resource-friendly plug-ins do not waste memory

Extensive coverage of multimedia technology

The capabilities of GStreamers can be extended with new plug-ins. The features listed below are only a rough overview of GStreamer’s own plug-ins available when using GStreamers and do not include any third party offerings.

  • Container format: ASF, AVI, 3GP/MP4 / MOV, FLV, MPEG-PS/TS, MKV/webM, MXF, OGG
  • Streams: HTTP, MMS, RTSP
  • Codec: FFmpeg, various codec libraries, third party codec packages
  • Metadata: Local container format with common mappings between them
  • Video: various color Spaces, supporting progressive and interlaced video
  • Audio: Integer and floating point audio data with various bit depths and multi-channel configurations

Lots of development tools

  • The GST-launch command-line tool can be used for rapid prototyping and testing, similar to Ecasound
  • Extensive documentation, including a partially completed manual and plug-in writer’s guide
  • Numerous optional test programs and sample code in each module
  • The GStreamer API is accessible through a variety of programming languages

GStreamer is a flexible, fast and multi-platform multimedia framework.

GStreamer is an extremely powerful and feature-rich framework for creating streaming applications. Many of the benefits of the GStreamer framework come from its modularity: GStreamer can work seamlessly with new plug-in modules. But because modularity and power often come at the cost of greater complexity, writing new applications is not always easy.

GStreamer compilation

GStreamer is an open source multimedia framework, so compiling from source is very convenient. GStreamer provides compilation methods such as Meson and Cerbero. Here GStreamer is compiled on Ubuntu Linux using Meson.

The Meson build system is a fast and portable build system. It generates build instructions that can be executed by Ninja from build profiles, and the GStreamer project uses it as a build system for all of its subprojects. You can install Ninja using the following command:

sudo apt-get install ninja-build
Copy the code

To compile the latest version of GStreamer, Ubuntu’s meson source may be a bit old, such as Ubuntu 20.04 meson 0.53.2 installed via APT, Error: GStreamer Master Branch code is too old:

Hanpfei @ ubuntu: ~ / Data/opensource/gstreamer $meson build The meson build system Version: 0.53.2 Source dir: /home/hanpfei/Data/opensource/gstreamer Build dir: /home/hanpfei/Data/opensource/gstreamer/build Build type: native build meson.build:1:0: ERROR: Meson version is 0.53.2 but project requires >= 0.54 A full log can be found at /home/hanpfei/Data/opensource/gstreamer/build/meson-logs/meson-log.txtCopy the code

It is best to use Python’s PIP tool to install meson (if you have already installed meson via apt, remove it first) :

hanpfei@ubuntu:~/Data/opensource/gstreamer$ sudo apt install python3-pip
hanpfei@ubuntu:~/Data/opensource/gstreamer$ python3 -m pip install meson
Copy the code

Python’s PIP tool installs meson in a hidden directory in the user’s root directory, ~/. Local /bin, such as /home/hanfei/.local /bin. This PATH also needs to be added to the PATH environment variable.

Ninja can also be installed using Python’s PIP tool:

hanpfei@ubuntu:~/Data/opensource/gstreamer$ python3 -m pip install ninja
Copy the code

In addition, there are some dependencies that need to be installed before GStreamer can be compiled:

hanpfei@ubuntu:~/Data/opensource/gstreamer$ sudo apt-get install flex bison
Copy the code

In September 2021, all major GStreamer modules were merged into a single code repository. GStreamer’s separate repository is located in the main GStreamer Git repository, This is where all GStreamer development will take place for GStreamer version 1.19/1.20 and later.

Before this single repository merges different GStreamer modules in separate Git repositories, there is a separate meta-build project called GST-Build for downloading and building all subprojects. If you want to develop older stable branches, such as GStreamer 1.16 or 1.18, you should use it.

If you want to build or develop an upcoming development or stable branch, you should use the main branch of the GStreamer module contained in a single code repository. Gst-build works in much the same way as a single repository, except that it will download various GStreamer submodules.

To build the current GStreamer development version, which will become a 1.20 stable branch in the near future, clone the GStreamer repository:

git clone https://gitlab.freedesktop.org/gstreamer/gstreamer.git
cd gstreamer
Copy the code

Or if you have developer permission for this repository:

git clone [email protected]:gstreamer/gstreamer.git
cd gstreamer
Copy the code

Clone gst-build if you want to build a stable 1.18 or 1.16 branch:

git clone https://gitlab.freedesktop.org/gstreamer/gst-build.git
cd gst-build
Copy the code

The repository contains some notable scripts and directories:

  1. meson.buildIs the top-level build definition that recursively configures all dependencies. It also defines helper commands that allow you to have an uninstalled development environment or simply update your GStreamer module’s Git repository.
  2. subprojects/GStreamer is a directory containing GStreamer modules and a set of dependencies.

Configure one module (or multiple at a time at gST-build) by executing the following command:

meson <build_directory>
Copy the code

Where build_directory is where all build instructions and output will be placed (this is also known as an “output directory” build). If the directory is not already created, it will be created at this point. Note that calling meson does not require any command arguments but implicitly calls the meson setup command (such as performing the initial configuration of a project).

There is only one limitation to the location of build_directory: it cannot be the same as the source directory (for example, where you downloaded your module). Although it can be outside or under/inside the directory.

Once meson is configured, you can:

  1. Go to the specific build directory and run Ninja:
cd <build_directory>
ninja
Copy the code
  1. Or don’t perform it every time you want toninjaYou can specify only the build directory as an argument. The nice thing about this option is that you can do it from anywhere (instead of switching to the Ninja directory)
ninja -C </path/to/build_directory>
Copy the code

This will build everything for that module (and subprojects if building GST-build or a single repository).

Note: You don’t need to re-run Meson when modifying the source file, you just need to re-run Ninja. If the build/configuration file changes, Ninja will determine that Meson needs to be rerun and will run it automatically.

Hello, World

There’s no better way to get a first impression of a library than running an app that’s built on top of that library and prints “Hello World” on the screen. But here we’re dealing with a multimedia framework, which will play a media file instead.

The above build process will also build GStreamer project tests and sample applications, including GStreamer’s HelloWorld sample application. The code of the application in gstreamer/tests/examples/helloworld, compiled the generated binary executable file is located in the < build_directory > / tests/examples/helloworld/helloworld.

The source code for the HelloWorld sample application is as follows:

#include <gst/gst.h> static gboolean bus_call (GstBus * bus, GstMessage * msg, gpointer data) { GMainLoop *loop = (GMainLoop *) data; switch (GST_MESSAGE_TYPE (msg)) { case GST_MESSAGE_EOS:{ g_print ("End-of-stream\n"); g_main_loop_quit (loop); break; } case GST_MESSAGE_ERROR:{ gchar *debug; GError *err; gst_message_parse_error (msg, &err, &debug); g_printerr ("Debugging info: %s\n", (debug) ? debug : "none"); g_free (debug); g_print ("Error: %s\n", err->message); g_error_free (err); g_main_loop_quit (loop); break; } default: break; } return TRUE; } gint main (gint argc, gchar * argv[]) { GstElement *playbin; GMainLoop *loop; GstBus *bus; guint bus_watch_id; gchar *uri; gst_init (&argc, &argv); if (argc < 2) { g_print ("usage: %s <media file or uri>\n", argv[0]); return 1; } playbin = gst_element_factory_make ("playbin", NULL); if (! playbin) { g_print ("'playbin' gstreamer plugin missing\n"); return 1; } /* take the commandline argument and ensure that it is a uri */ if (gst_uri_is_valid (argv[1])) uri = g_strdup (argv[1]); else uri = gst_filename_to_uri (argv[1], NULL); g_object_set (playbin, "uri", uri, NULL); g_free (uri); /* create an event loop and feed gstreamer bus messages to it */ loop = g_main_loop_new (NULL, FALSE); bus = gst_element_get_bus (playbin); bus_watch_id = gst_bus_add_watch (bus, bus_call, loop); gst_object_unref (bus); /* start play back and listed to events */ gst_element_set_state (playbin, GST_STATE_PLAYING); g_main_loop_run (loop); /* cleanup */ gst_element_set_state (playbin, GST_STATE_NULL); gst_object_unref (playbin); g_source_remove (bus_watch_id); g_main_loop_unref (loop); return 0; }Copy the code

The application receives the URI of a media resource and plays the media resource. Specific usage is as follows:

$ build/tests/examples/helloworld/helloworld [URI]
Copy the code

If playing an AAC file:

$ build/tests/examples/helloworld/helloworld file:///home/hanpfei/aac.aac
Copy the code

When using this application to play media files, the plugin failed to find the error, such as:

$ build/tests/examples/helloworld/helloworld file:///home/hanpfei/aac.aac 
'playbin' gstreamer plugin missing
Copy the code

The HelloWorld application plays media files through the plugin “playbin”. In general, the physical form of a plug-in is a dynamically linked library, and the GStreamer framework looks for plug-ins in a specific directory during initialization. GStreamer can use the environment variable GST_PLUGIN_PATH to specify a plug-in search path for GStreamer, or programmatically ask GStreamer to search for plug-ins in a specific directory, such as:

GstRegistry *registry; registry = gst_registry_get(); Gst_registry_scan_path (registry, "/ usr/lib/x86_64 - Linux - gnu/gstreamer 1.0");Copy the code

The “playbin” plug-in, seen above, is a basic plug-in provided by the GStreamer project itself, and its code is located in the GST-plugins-base project. The location of the project is https://github.com/GStreamer/gst-plugins-base. For Ubuntu Linux, you can also install compiled binaries using the following command:

$apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev Gstreamer1.0 - plugins - base gstreamer1.0 - plugins - good gstreamer1.0 - plugins - bad gstreamer1.0, plugins, and ugly Gstreamer1.0 - LibaV gstreamer1.0- DOC gstreamer1.0- Tools gstreamer1.0-x gstreamer1.0- alSA gstreamer1.0- GL Gstreamer1.0 - gtk3 gstreamer1.0 - qt5 gstreamer1.0 - pulseaudioCopy the code

This command installs all of GStreamer’s libraries at once. Gstreamer1.0-plugins-base contains the plugin “playbin”, This plugin is typically installed in /usr/lib/x86_64-linux-gnu/ gstreamer-1.0/libgstPlayback.

With this in place, HelloWorld is up and running.

Now you can have fun exploring the concepts and operations of GStreamer.