Gstreamer plugin development

Gstreamer Plugin Development

Session VII

Element

• Core of Gstreamer• Object derived from GstElement• Source elements provides data to stream• Filter elements acts on a data in stream• Sink elements consumes data of stream

Plugin

• Loadable block of code• Usually shared object or a dynamically linked

library• Contains implementation of many elements or just

a single one• Very basic functions reside in the core library• All other are implemented as plugins• Are only loaded when their provided elements are

requested

Pads

• Negotiate links and data flows between elements

• Is an port of an element – Output or Input Port• Specific data handling capabilities• Can restrict the data that flows through it.• Links are allowed between two pads when

data types of two pads compatible• Is similar to plug or jack of a physical device

Pads

• Source Pads– Data flows out of one element through one or

more source pads• Sink Pads– Accepts incoming data flow through one or more

sink pads

GstMiniObject

• Stream of data are chopped up into chunks that are passed from a source pad on one element to a sink pad on another element

• Structure used to hold these chunks of data• Exact type indicating what type of data• GstMiniObject Types– Events ( Control )– Buffers ( Control )

GstBuffer

• Contains any sort of data that two linked pads know how to handle

• Contains some sort of audio or video data that flows from one element to another.

• Metadata describing the buffer’s contents– Pointers to one or more GstMemory Objects– Ref Counted objects the encapsulate a region of memory– Timestamp indicating the preferred display timestamp of

the content in the buffer.

Buffer Allocation

• Able to store chunks of memory of several different types

• Most generic type of buffer contains memory allocated by malloc– Very convenient– Not always fast, since data often needs to be

copied

Specialized Buffers

• Many specialized elements create buffers that point to special memory. – filesrc element maps a file into address space of

application ( using mmap() )– Creates buffers that point into that address range– Created by filesrc act like generic buffers, except

that are read only• Buffer freeing code automatically freed by the

correct method of freeing

GstBufferPool• Element might get specialized buffers is to request them from a

downstream peer through a GstBufferPool• Downstream able to provide these objects, upstream can use them to

allocate buffers.• Accelerated methods for copying data to hardware or direct access to

hardware• These elements able to create a GstBufferPool for their upstream

peers.• Example Ximagesink

– Create a buffer that contain Ximages– Upstream peer copies the data into the buffer, it is copying directly into the

Ximage– Enables Ximagesink draws the image directly without copy

GstEvent

• Information on the state of the stream flowing between two linked pads

• Will only be sent if element explicitly support them, else core will try to handle automatically

• Events used to indicate ( for example )– A media type– End of media stream– Cache should be flushed

Events

• Subtype indicating the type of contained event• Other contents of event depend on the

specific event type

Media Types & Properties

• Uses type system to ensure that the data passed between the elements in recognized format.

• Type system is important for ensuring that the parameters required to fully specify the format match up correctly when linking pads between elements.

• Each link is made between elements has a specified type and optionally a set of properties.

• Capabilities Negotiation.

Basic TypesS.No Media Type Description Property Description

1 audio/* All audio types rate Sample Rate

2 channels integer Greater than 0 No of channels

3 audio/x-raw raw integer audio

Format String

4 audio/mpeg MPEG Audio encoding

mpegversion Integer

5 framed boolean 0 or 1 True => Frame available,

false => no frame6 layer Integer 1,2 or 3 Compression

layer7 bitrate integer Greater than 0

8 audio/x-vorbis Vorbis audio data

Building a plugin

• Example file element– Single input and single output pad– Simply pass Media Data and Event data from its

sink pad to its source pad without modification.• Including properties• Including signal handling• Examples/pwg/examplefilter

Plugin Template

• To check out gst template,– git clone git://anongit.freedesktop.org/gstreamer/gst-

template.git– cd gst-template/gst-plugin/src– ../tools/make_element MyFilter– Creates two files

• gstmyfilter.c• gstmyfilter.h

– autogen.sh– make && sudo make install

Examing Code

• Basic code structure• Element metadata• GstStaticPad• Constructor• Plugin_init• Specifying the pads

Plugin Header#include <gst/gst.h>/* Definition of structure storing data for this element. */typedef struct _GstMyFilter {

GstElement element;GstPad *sinkpad, *srcpad;gboolean silent;

} GstMyFilter;

/* Standard definition defining a class for this element. */typedef struct _GstMyFilterClass {

GstElementClass parent_class;} GstMyFilterClass;

/* Standard macros for defining types for this element. */#define GST_TYPE_MY_FILTER (gst_my_filter_get_type())#define GST_MY_FILTER(obj) (G_TYPE_CHECK_INSTANCE_CAST((obj),GST_TYPE_MY_FILTER,GstMyFilter))#define GST_MY_FILTER_CLASS(klass) (G_TYPE_CHECK_CLASS_CAST((klass),GST_TYPE_MY_FILTER,GstMyFilterClass))#define GST_IS_MY_FILTER(obj) (G_TYPE_CHECK_INSTANCE_TYPE((obj),GST_TYPE_MY_FILTER))#define GST_IS_MY_FILTER_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE((klass),GST_TYPE_MY_FILTER))

/* Standard function returning type information. */GType gst_my_filter_get_type (void);

Setup Object

#include "filter.h"G_DEFINE_TYPE (GstMyFilter, gst_my_filter,

GST_TYPE_ELEMENT);

Element Meta datagst_element_class_set_static_metadata (klass,

"An example plugin","Example/FirstExample","Shows the basic structure of a plugin","your name <your.name@your.isp>");

static void gst_my_filter_class_init (GstMyFilterClass * klass){GstElementClass *element_class = GST_ELEMENT_CLASS (klass);[..]

gst_element_class_set_static_metadata (element_klass,"An example plugin","Example/FirstExample","Shows the basic structure of a plugin","your name <your.name@your.isp>");

}

Pad Template

static GstStaticPadTemplate sink_factory =GST_STATIC_PAD_TEMPLATE ("sink",GST_PAD_SINK,GST_PAD_ALWAYS,GST_STATIC_CAPS ("ANY")

);

Plugin initstatic gbooleanplugin_init (GstPlugin *plugin){

return gst_element_register (plugin, "my_filter", GST_RANK_NONE, GST_TYPE_MY_FILTER);}

GST_PLUGIN_DEFINE (GST_VERSION_MAJOR,GST_VERSION_MINOR,my_filter,"My filter plugin",plugin_init,VERSION,"LGPL","GStreamer","http://gstreamer.net/"

)

Specifying Padsstatic voidgst_my_filter_init (GstMyFilter *filter){

/* pad through which data comes in to the element */filter->sinkpad = gst_pad_new_from_static_template (&sink_template, "sink");/* pads are configured here with gst_pad_set_*_function () */gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad);/* pad through which data goes out of the element */filter->srcpad = gst_pad_new_from_static_template (&src_template, "src");/* pads are configured here with gst_pad_set_*_function () */gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);/* properties initial value */filter->silent = FALSE;

}

Chain Function

• The function in which all data processing takes place

• In simple filter, mostly linear functions for each incoming buffers, one buffer will go out.

Simple Chain Implementationstatic GstFlowReturn gst_my_filter_chain (GstPad *pad, GstObject *parent, GstBuffer *buf);static void gst_my_filter_init (GstMyFilter * filter){/* configure chain function on the pad before adding the pad to the element */gst_pad_set_chain_function (filter->sinkpad, gst_my_filter_chain);}

static GstFlowReturn gst_my_filter_chain (GstPad *pad, GstObject *parent, GstBuffer *buf){

GstMyFilter *filter = GST_MY_FILTER (parent);if (!filter->silent)g_print ("Have data of size %" G_GSIZE_FORMAT" bytes!\n",gst_buffer_get_size (buf));return gst_pad_push (filter->srcpad, buf);

}