GStreamer Multimedia Framework

Part 4

2

Contents

Description

Part 1 Introduction to GStreamer

Part 2 GStreamer Plugin Internals

Part 3 Advanced GStreamer Concepts

Part 4 Concepts of GStreamer Application

Part 5 An Example Application Code

The pipeline’s message busMechanism for interaction with Application

4

Introduction to GstBus

• GstBus is a facility provided by the GStreamer Pipeline which applications can use to receive messages from framework core.

• The messages delivered over GstBus are represented using the GstMessage data type.

• Messages are delivered on the bus in FIFO order. If application do not pick up, messages will remain in the bus.

• Example list of messages include: End of stream State change notification for each element in the pipeline Error, Info and Warning messages posted by elements in the pipeline Clock notifications, etc

• Applications would register a function (i.e. add a watch) with the pipeline’s bus. The registered function would get invoked whenever a message is to be delivered.

• The message delivery mechanism used by GstBus fits in to the generic event loop handling used in GTK based applications.

5

application

GstBus

pipeline Video dec video sink

queue

queue

GstBus and Application Threading Architecture

filesrc demuxer

audio dec audio sink

Application uses GStreamer core APIs to invoke pipeline functionality

Thread 1

Thread 2

Thread 3

App Thread

Messages from different streaming threads are accumulated by GstBus and delivered to the application in the context of application’s main loop itself.

SignalsElements interacting with application

7

GObject Signals

• Another method for applications to get triggered from GStreamer is through Signals.

• Signaling mechanism is implemented by GObject (i.e. GLib functionality) and GStreamer directly inherits it.

• Each GObject, and hence each GstElement, can raise a custom set of signals. Applications can register a callback function for each type of signal with the GstElement directly.

• Examples: “have-type” signal emitted by typefind element “pad-added” signals emitted by elements which support dynamic (i.e.

sometimes) pads.• The application’s callback function would be executed in the context

of GStreamer streaming thread itself.

8

When to use Bus Messages and When to use Signals?

• Bus Messages: Bus messages are usually informational in nature. The pipeline would

continue its functionality irrespective of whether the application processed a particular message or not.

Pipeline should not get blocked when the application is processing a message as it may affect the timing constraint of pipeline. Hence messages are processed by application in the context of its own thread (glib main loop).

If required, applications can register for synchronous delivery of messages where the pipeline streaming thread gets blocked till the application processes the message.

The frequency of messages delivered is more compared to the frequency of signals in a pipeline. For example, there would be a message delivered for every state transition of every element in the pipeline.

9

When to use Bus Messages and When to use Signals?

• Signals: Signals are more critical triggers than messages as the pipeline’s further

functionality would depend on what action would the application take for a particular signal. Examples:‐ When a “have-type” signal is raised, the application might decide not to support that

particular media type and hence tear down the complete pipeline.‐ In case of dynamic pipeline construction, the complete pipeline would be

constructed only after the “pad-added” signal is received by application. The application’s callback function would get invoked from the context of the

pipeline streaming thread itself. Hence the pipeline thread would be blocked till the application finishes processing the signal.

In a pipeline, generally, there would be very few signals that an application needs to handle.

GStreamer Application Structure

11

The Glib main loop

• GLib provides a generic event processing loop that applications can use straight away

• Event processing loop is typically used in GUI based applications, where the applications needs to process events from different sources (example, window events, keypad events, application backend events, etc)

• The messages transferred over GStreamer pipeline bus has support for usage of the glib event processing loop

• This way, a GUI based application which uses GStreamer framework can be very easily built

12

Phases of a GStreamer Application

Initialization of gstreamer elements

and pipeline

Registration of message and signal handlers

Glib Main Loop

Start

Deinitialisation

Stop

Exit from glib main loop has to be

triggered by one of the message handlers

(e.g., the message handler for End of Stream Message)

Thank You