Gstreamer: an Overview

GStreamer: An overview

Rodrigo Costa

Pontifical Catholic University of Rio de JaneiroTeleMı́dia Lab.

[email protected]

April 16, 2015

Rodrigo Costa (TeleMı́dia Lab.) GStreamer April 16, 2015 1 / 22

Overview

1 GStreamerOverviewPipelinesBasic ConceptsSynchronizationExamples


What is GStreamer?

A framework for multimedia applications

Linux, Windows, OS X, . . .

10+ years old project

Similar to DirectShow, QuickTime, etc.

Designed to make it easy to write applications that handle audiovisualcontents


GStreamer overview

What GStreamer NOT is?

A codec implementation

An application itself

Features

Fully extensible by plugins

Flexible

Low and high level API

Often wraps other libraries (decoders, encoders, filters, etc.)

Integration with other frameworks and projects

WebKit, Firefox, Gnome, Android, iOS, . . .

Pipeline-based

Bindings to multiple languages

C/C++, Python, Java, Ruby, Pearl, etc.


GStreamer overview


Design

The fundamental design comes from the video pipeline at OregonGraduate Institute: Infopipes

Infopipes (2001)

High-level abstraction for building blocks that handle informationflows

Goal: simplify the task of building distributed streamingapplications by providing basic elements such as pipes, filters, buffers,and pumps

Motivated by the lack of support of RPC-like technologies to developstreaming and processing of distributed information


Infopipes

Infopipe example:


GStreamer pipeline

GStreamer pipeline example:


Basic Concepts

Elements

The most important class of objects in GStreamer

Each one has a specific function: read a file, receive data fromnetwork, decode, multiplex, present the data, . . .

A chain of elements is created and linked together for the data toflow through it

GStreamer ships with a large set of elements


Basic Concepts

Elements (cont)

Source

Filters, converters, demuxers, muxers, codecs

Sink


Basic Concepts

Pads

Connection points between elements

Source pads produce data

Sink pads consume data

Data always flows from src to sink pads

Can operate in pull or push mode


Basic Concepts

Pads (cont)

Each pad have a predefined set of properties called Capabilities orCaps

Caps are used to validate the communication between elements (capsrestrict the type of data that flows through it)

A source pad can only be linked to a sink pad if their allowed datatypes are compatible

For instance, a source pad that produces a video stream cannot belinked to a sink pad that consumes audio streams

Elements can negotiate with each other on format to use (capsnegotiation)


Basic Concepts


Basic Concepts

Bins

Elements can be grouped into a container called bin

It can be manipulated as it were an element

It also manages the state of its children elements


Basic Concepts

Pipeline

A pipeline is a top-level bin

It provides a bus for communication purposes

It manages the playout synchronization

It runs in a separate thread


Basic Concepts

Communication:


Synchronization

GstClock objects provide clock time

GstClock always returns the absolute-time

The pipeline maintains a GstClock object and a base-time (time whenit goes to PLAYING state)

running-time = absolute-time - base-time

Sink elements are responsible for present the buffers in theirrespective presentation time

If a buffer is delayed, the sink drops it


Pipeline Example


Example 1


Example 2


Example 3


Example 4


Software

Gstreamer: an Overview