Upload
hayden-stanger
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Song and Dance and TWDusing GStreamer
Thomas Vander Stichele
Overview
● TWD and USP● TMS● VKC of GST● GPE prototyping● LAP for SAD celebrating FTA● Roadmap
software desktops
● Desktops have targets and philosophies– GNOME: less options, less bloat, HIG– KDE: full configurability– MacOS: eye candy, artistic personality– Windows: applications
● Desktops need a USP for better marketing● After-hours business school applied to
GNOME● TMS done by Korean firm
Thorough Market Study
GNOME KDE Mac Windows Other0
10
20
30
40
50
60
70
80
90
100
Which desktop are you currently using ?
which desktop ?
Thorough Market Study
GNOME KDE MacOS Windows0
10
20
30
40
50
60
70
80
90
100
What do you want in your desktop ?
More options
Less bloat
Stability
More appsKaraoke
Thorough Market Study
Yes No0
10
20
30
40
50
60
70
80
90
100
Would you switch to GNOME if it had karaoke ?
well, would you ?
Next year at LWE
Karaoke
● What do we need ?● Why use GStreamer ?● What is GStreamer ?● What needs to be done ?
What do we need ?
● Audio playback● Visualization● Video● Subtitling
Why use GStreamer for it ?
● Lot of elements already present● Framework is there for things you need
– Output– Synchronization– Mixing multiple formats
● Prototyping is easy– gst-launch for quick tests– gst-editor for experimenting with pipelines– Gst-python for testing application structure
GStreamer
● What is it ?● Why is it ?● Why is it not ?● Elements and pads● Graph-based pipelines
What is it ?
● Core: streaming media framework– Arbitrary filter graphs– Extensible pluggable architecture– Data-agnostic, basic elements (GObjects)
● Plug-ins : provide functionality– Audio : filters, codecs, analysis, visuals– Video : filters, codecs, (de)muxers– Sources/Sinks : video/audio/media in/out
What is it ?
● GStreamer is first of all a library– Compare it to GTK
● We provide a few applications– A player as a test bed for features (~ gtk-
demo)– An editor used in debugging (~ glade)
● We provide plug-ins● We provide documentation
Why is it ?
● Linux has bad media handling● many hacked single-purpose apps and libs,
code copies and forks, ...– avifile, Broadcast2000, ...
● other OS's have such a framework– DirectShow, Quicktime framework, Be
Media, RealMedia SDK● LGPL core gives companies the chance to
make binary plug-ins
What is it not ?
● It's NOT a sound server– one could be written with it (maybe)– not important for an app
● It's NOT Gnome-only● It's NOT 100% ABI-stable (partial)● It's NOT perfect
Elements and Pads●element has pads
●chain- or loop-based
●buffer of data comes in on the sink pad
●element processes data
●Processed buffer goes out on the source pad
●This one decodes Ogg/Vorbis audio
Graph-based pipelines
●Pipeline managed by a bin (not visible)
●Links elements by their pads●elements are chain-based or loop-based
●Data will flow left to right
What needs to be done ?
● Experiment– gst-launch– gst-editor
● Write elements● Fix bugs
– In elements– In applications like gst-editor
Let's get to it !
● Audio playback– Guadec-editor 1
● Visualization– Guadec-editor 2
● Subtitling– Guadec-editor 3
Combining the three
● Normal:– Guadec-editor 4
● Threaded:– Guadec-editor 5
Using video footage
● Threaded MPEG playback:– Guadec-editor 6
● With subtitles– -guadec-editor 7
Ideas for improvement
● Use pitchshifter so Stallman can sing● Use festival to replace Stallman
– Play: Guadec-editor 8– Save to vorbis: Guadec-editor 9– Use with subtitles: guadec-editor 10
● Personalise– Find video of friends: guadec-editor 11
Ideas for improvement
● Use live input– Mix in audience audio and video
● Use backing music– Mod files, midi files, ...
● Use color font rendering● Add a bouncing ball plugin
Gkaraoke
● Truth is stranger than fiction
Roadmap
● Actual stuff to work on● What can you do ?
Actual stuff to work on
● Media-info library:– Save and load metadata and streaminfo– Cache it (-> thumbnails)– Playback parameters (mixing, volume)– Shared by all apps
Actual stuff to work on
● Nautilus:– Hoverplay for audio and video– Further audio view improvement
● Encoding profiles– Used by recorder, sound-juicer, ...– Using human-readable names
● Editor:– Educational GStreamer tool
What can you do ?
● Users– Test our applications:
● Gst-totem, gst-player, gst-editor● Rhythmbox, quark, nautilus-media, sound-juicer● Use media files on http://gstreamer.net/media/
– Suggest new applications– Run the upcoming test suite on your computer– Harass developers
What can you do ?
● Developers– Help us fix bugs and implement new features– Document– Write applications– Write/fix elements– Test and use the bindings– Use the tools we provide
● Build scripts● Template tarballs
– Go to Ronald's talk