This is the first day of my participation in the Gwen Challenge in November. Check out the details: the last Gwen Challenge in 2021

GStreamer Core

Communication (Communication)

Buffer Buffers

Buffer Buffers are used to pass flow data between Elements of a Pipeline. Buffers are usually used to flow from a downStream Source node to a Sink node

Events Events

Events are usually sent between Elements downstream or between the upstream Application and Elements. Events between downstream elements can be used to Synchronize Synchronize

News Messages

The Message is sent by Elements to the Pipeline’s Message Bus, where it is collected by the Application. Messages can be intercepted synchronously from the flow thread context of the Element that publishes the message, Element, but are typically processed asynchronously by the application from the main thread of the application. Messages are used to transfer errors, tags, state changes, buffered state, redirects, and so on from elements to applications in a thread-safe manner.

Query the Queries

Query allows applications to request information such as duration or current play location from the pipe. Queries are always answered synchronously. Elements can also use queries to request information from their counterparts, such as file size or duration. They can be used in pipelines in two ways, but upstream queries are more common.

Ogg Plyaer GStreamer communication flow

message/event/signal

Bus message – Used for interaction between Gstreamer and app. For example, when a file ends, Gstreamer will send an EOS message to GstBus. If the app is listening (function gst_bus_add_watch), The message is received in the callback function that processes the message.

Event — Used to send events between elements within gStreamer (or between pads), such as source Element data has ended, to send an EOS event along the pipeline to the downstream direction, After all sink elements are received and processed, gstreamer generates a GstMessage internally and posts it to GstBus. If APP is listening, it can know that the current playback has ended.

Signal — A mechanism for interaction between app and GObject, belonging to GObject architecture. In GStreamer, element itself is also a gobject, so you can associate an app with it via signal. It can signal the app when something happens to Element that it wants the app to know about, such as the dynamic creation of a Pad.

Each pipeline has a bus by default. So the application does not need to create any Buses. All the application has to do is set up a message handler on the bus. When MainLoop is running, the bus checks periodically for new messages and calls the callback function (message handler) if a message is available.

Bus

• Each Pipeline has a bus by default
• GST_OBJECT_LOCK (bus)/GST_OBJECT_UNLOCK (bus)
• Type message to bus
  • gboolean gst_bus_post (GstBus * bus, GstMessage * message);
    • Store the message in the bus. There are two main patterns
    • The GST_BUS_PASS mode is only responsible for queuing messages and updating the write flag (listening for read until it can be read).
    • GST_BUS_ASYNC mode takes the lock of the message and waits until the message is processed (exit the gST_bus_POST function).
• Fetch message from bus
  • GstMessage * gst_bus_peek (GstBus * bus);
    • Retrieves a message, or returns NULL if none is present
  • GstMessage * gst_bus_pop (GstBus * bus);
  • GstMessage * gst_bus_pop_filtered (GstBus * bus, GstMessageType types);
  • GstMessage * gst_bus_timed_pop (GstBus * bus, GstClockTime timeout);
  • GstMessage * gst_bus_timed_pop_filtered (GstBus * bus, GstClockTime timeout, GstMessageType types);
    • Wait/try/timeout for a particular type of message
  • GstMessage * gst_bus_poll (GstBus * bus, GstMessageType events, GstClockTime timeout);
• Processing the message
  • void gst_bus_set_sync_handler (GstBus * bus, GstBusSyncHandler func, gpointer user_data, GDestroyNotify notify);
    • Message processing is triggered during POST and is not added to polling. (Processed in the same thread as POST)
  • guint gst_bus_add_watch (GstBus * bus, GstBusFunc func, gpointer user_data);
    • Associate func with Signal_watch (the new GSource) in the polling of the default GMainContext.
  • void gst_bus_add_signal_watch (GstBus * bus); // g_signal_connect() // info, error, eof, ….
    • Call gst_bus_add_watch with gST_bus_async_signal_func to implement signal triggering inside GST_BUS_async_signal_func
• Bus can handle messages in two ways: sync_handler synchronously and watch/signal asynchronously.

• Conclusion:
  • Bus is thread-safe (ensured using multiple locking mechanisms) and internally uses queues for message receiving and forwarding processing
  • Message is suitable for point-to-point delivery and must be executed (or destroyed) once retrieved
  • Bus provides synchronous message processing, asynchronous message callback (or signal-triggered mechanism)

struct GstBus { GstObject object; struct GstBusPrivate { GstAtomicQueue *queue; Queue (length 32) GMutex queue_lock; // queue lock SyncHandler *sync_handler; Guint num_signal_watchers; // Synchronous message processing, first call when there is a new message (same context as POST) guint num_signal_watchers; // Mutually exclusive with signal_watch guint num_sync_message_emitters; // GSource *signal_watch; // Callback function (asynchronous) gboolean enable_async; GstPoll *poll; // Create a pair of nameless interconnected sockets using socketPair internally. // Poll handle (associated with poll handle)} *priv; gpointer _gst_reserved[GST_PADDING]; };Copy the code

struct GstMessage
{
  GstMiniObject   mini_object;

  GstMessageType  type;
  guint64         timestamp;
  GstObject      *src;
  guint32         seqnum;

  GMutex          lock;
  GCond           cond;
};
Copy the code

struct GstQuery
{
  GstMiniObject mini_object;

  GstQueryType type;
};
Copy the code

struct GstEvent {
  GstMiniObject mini_object;
  GstEventType  type;
  guint64       timestamp;
  guint32       seqnum;
};
Copy the code

struct GstBuffer {
  GstMiniObject          mini_object;

  struct GstBufferPool {
    GstObject            object;
    gint                 flushing;
    GstBufferPoolPrivate *priv;
    gpointer _gst_reserved[GST_PADDING];
  } *pool;

  GstClockTime           pts;
  GstClockTime           dts;
  GstClockTime           duration;

  guint64                offset;
  guint64                offset_end;
};
Copy the code

GMainLoop – GLib main event loop

Introduction to the

GLib’s main event loop manages any number of different types of event sources (Gsources), such as file descriptors (files, pipes, sockets), timeouts.

New event sources can be added by g_source_attach().

To allow multiple sets of independent event sources to be processed in different threads, each event source is associated with a GMainContext. One thread can only run one GMainContext, but other threads can add and remove event sources.

All functions running on GMainContext or the built-in GSource are thread-safe.

Each event source is assigned a priority. The default priority is G_PRIORITY_DEFAULT(0). A smaller value indicates a higher priority. The event source with a higher priority is processed first.

Idle functions are executed when no higher priority events are being processed.

The GMainLoop data type represents a main event loop. The GMainLoop object is created by g_main_loop_new(). G_main_loop_run () is executed after the initial event source is added, and the main loop continuously checks for new events generated by each event source and distributes them until a call to g_main_loop_quit() is triggered when an event from one of the event sources is processed to exit the main loop.

Note that if an event source is added to a GMainContext, it will be checked and distributed by all main threads associated with the GMainContext.

Custom event source types

Create a new event source type by inheriting the GSource structure. The new event source type generated by inheritance represents the GSource structure as the first element of the new event source type followed by other elements. Use the g_source_new() function to create a new instance of the event source type. The function takes the size of the new event source type as an argument. GSourceFuncs determines the behavior of the new event source type.

New event sources interact with GMainContext in two basic ways. The preparation function in their GSourceFuncs can set sleep events that determine when the main loop will next detect them. You can also use the g_source_add_poll() function to add file descriptors to GMainContext for detection.

Custom main loop iteration

To complete a single iteration of GMainContext, call g_main_context_Iteration (). In some cases, we might want to get more low-level control details about the main loop by calling a component function in g_main_context_Iteration () : G_main_context_prepare (), g_MAIN_context_query (), g_MAIN_context_check (), and g_main_context_dispatch().

The state of the Main Context

Custom event sources can operate only 3+1 phases (Prepare, Check, Dispatch, MainContext destruction, Finalize)

Event source memory management

There are two ways to manage the memory of user data passed to the GSource callback function. The user data here is the parameter passed in calls to g_timeout_add(), g_timeout_add_full(), and g_IDLE_add (). This data is usually owned by timeout or idle callbacks, such as a component or an implementation of a network protocol. Sometimes these callbacks will be called after the data has been destroyed, using memory that has been freed, which will result in an error.

• The first recommended method is to save the event source ID returned by g_timeout_add(), g_source_attach(), and then remove it from the GMainContext as displayed after the user data it maintains has been freed. This ensures that the user data is still valid when these callbacks are called. (g_source_remove (source_id))

• The second option is to save a reference to the user data object in these callbacks and then release it in the GDestroyNotify callback. This ensures that the data object remains valid until the event source is last called and then released. The GDestroyNotify callback is an input parameter to a variant of the GSource function, which is called when the event source is released.

Note: In the second method, if the main loop ends before the event source is called, the state of the user data object is maintained is uncertain.