04/08/23 CSC533 - SAY 1

Multimedia ProgrammingCSC533

Chapter 3a: Chapter 3a:

Multimedia FrameworksMultimedia Frameworks

04/08/23 CSC533 - SAY 2

What is Software Framework?

In computing, a software framework provides “the skeleton of an application that can be customized by an application developer”

Like software libraries, software frameworks aid the software developer by containing source code that solves problems for a given domain and provides a simple API.

04/08/23 CSC533 - SAY 3

Types of software frameworks

A software framework may focus on building graphical editors for different domains like artistic drawing, music composition, and mechanical CAD.Another software framework can help build compilers for different programming languages and target machines.Yet another might help build financial modeling applications or decision support systemsFrameworks exist for multimedia, web applications, and even communicating between different systems.

04/08/23 CSC533 - SAY 4

What is Multimedia Framework?

A multimedia framework (MMF) is a software framework that handles media on a computer and through a network.

A good multimedia framework offers an intuitive API and a modular architecture to easily add support for new codecs, container formats and transmission protocols.

It is meant to be used by applications such as media players and audio or video editors, but can also be used to build Videoconferencing applications, media converters and other multimedia tools.

» Source from wikipedia.

04/08/23 CSC533 - SAY 5

Multimedia Framework Requirements

1.1. Economy of conceptsEconomy of conceptsMM framework should be based on small number of concepts, otherwise it becoming a mazeIdentify any general concepts that apply across media types

2.2. OpenOpenshould be possible to extend MM framework to incorporated new media types, new data representations, new h/ware capabilities

3.3. QueryableQueryableshould specify interfaces for querying environments concerning their capabilities. Applications can then recognize missing functionality and adapt their behavior.

04/08/23 CSC533 - SAY 6

Multimedia Framework Requirements

4. DistributionMM Framework should help applications in a way that facilitates distribution. In particular, objects within framework should correspond to easy-to-distribute units or subsystems.

5.5. ScalableScalableMM Framework should support scalable media representations.Once media are in digital form, improvements in quality and capability are tied to advances in h/w

Early version of processor might handle video at quarter screen resolution and few frames per second.

6.6. High-Level InterfacesHigh-Level InterfacesMM Framework should provide high-level interfaces for media synchronization, media composition, device control, database integration and concurrent media processing activities.Development is simplified if easy to use high-level interfaces are available

04/08/23 CSC533 - SAY 7

Multimedia Framework Example

Consists of four main class hierarchies: (Click Figure)1.1. Media classes Media classes

Correspond to audio, video and the other media types. Instances of these classes are particular media values (artifacts)

2.2. Transform classesTransform classes Represent media operations in a flexible and extensible manner. For example, many image editing programs provide a large number of filter operations with which to transform images. These operations could be represented by methods of an image class.

04/08/23 CSC533 - SAY 8

Multimedia Framework Example

3.3. Format classes Format classes Encapsulate information about external representations of media values. Can be defined for both file formats:

Image File format (such as GIF and TIFF) and ;Uncompressed digital video “stream” formats (CCIR 601 4:2:2)

4.4. Component classesComponent classes Represent h/w and s/w resources that produce, consume and transform media streams. For instance, a CD-DA player is a component that produces a digital audio stream (specifically, stereo 16 bit PCM samples at 44.1 kHz).

04/08/23 CSC533 - SAY 9

4 class hierarchies of a multimedia framework:

BACK

04/08/23 CSC533 - SAY 10

Different Operating Systemsand their Multimedia Support (Framework)

WindowsDirectX

.NET Framework

LinuxGstreamer

SDL

Mac OS XQuickTime

SymbiamMMF

Helix

JavaJMF

JSRs

04/08/23 CSC533 - SAY 11

Windows : DirectX

DirectX is the most important Multimedia Framework in Windows

First version was introduced in 1995 (Windows 95)

Current versions are 9.0 (Windows XP) and 10.0 (Windows Vista and Xbox 360)

04/08/23 CSC533 - SAY 12

Windows : DirectX

DirectX, formerly known as the Game SDK, is a term given for a collection of Windows APIs for easily handling tasks related to game programming Windows OS.

This set of development libraries for high performance games allows software developers "direct access" to the low-level functions of PC peripherals.

04/08/23 CSC533 - SAY 13

Windows : DirectX APIs

DirectX GraphicsDirectDraw (2D)

Direct3D

DirectInput

DirectPlay

DirectSoundDirectSound3D

DirectMusic

DirectSetup

DirectX MediaDirectAnimation

DirectShow

DirectX Video Acceleration

Direct3D Retained Mode

DirectX Transform

DirectX Media Objects

04/08/23 CSC533 - SAY 14

Windows : . NET Framework

Managed DirectX released to work with .NETIs a software technology that is available with several Windows OS. It includes a large library of pre-coded solutions to common programming problems, a runtime or virtual machine that manages the execution of programs written specifically for the framework, and a set of tools for configuring and building applications. The .NET Framework is a key Microsoft offering and is intended to be used by most new applications created for the Windows platform.

04/08/23 CSC533 - SAY 15

Linux : GStreamer

GStreamer is a framework for creating streaming media applications. It can be used to implement any kind of multimedia application too.

The framework is based on plugins that will provide the various codec and other functionality.

GStreamer is used, e.g., in Maemo environment (Nokia tablets N800 and 770) and GNOME desktop environment

http://gstreamer.freedesktop.org/data/doc/gstreamer/head/manual/html/chapter-gstreamer.html

04/08/23 CSC533 - SAY 16

Linux : SDL

Simple DirectMedia Layer (SDL) is an opensource, cross-platform multimedia library.

It gives access to audio, keyboard, mouse, joystick, 3D hardware (i.e., OpenGL), and 2D Video Frame buffer.

SDL supports Windows, Linux, Mac OS, etc.

http://www.libsdl.org/

04/08/23 CSC533 - SAY 17

Mac OS X : Quicktime

QuickTime is Apple's multimedia framework.

Supports Mac OS X, Windows and old Mac OS.

API for encoding and decoding audio and video.

QuickTime file format is a multimedia container file. Competitors: MS ASF, ogg, Matroska.

QuickTime for Java supports capturing, editing, playback, etc.

http://wapedia.mobi/en/QuickTime

04/08/23 CSC533 - SAY 18

Symbian : Multimedia Framework (MMF)

Symbian OS was originally built for handheld devices, with limited resources.A completely new Multimedia Framework for Symbian was introduced in Symbian 7.0sMMF provides a lightweight, multi-threaded framework for handling multimedia data.MMF enables:

Audio recording, playback, and streamingVideo recording, playback, and streamingImage related functionalities

MMF allows developers to write efficient and powerful plug-ins

04/08/23 CSC533 - SAY 19

Symbian : Helix Framework

Helix DNA is an open-source multimedia player project based on the RealPlayer.

The Helix Framework contains streaming server, players, codecs, and production tools.

Since May 2006, Helix was included in over a hundred mobile phones and was distributed more than 125 million times. It is also distributed within Maemo environment.

04/08/23 CSC533 - SAY 20

Java : Java Media Framework (JMF)

JMF allows use of multimedia in Java applicationsReal-time network protocolsMultiplexingCodecsPlayersEffectsCaptureControl

04/08/23 CSC533 - SAY 21

Java : Java Specification Request (JSR)

Third parties can participate in Java platform development by proposing JSRs.A Java Specification Request (JSR) describes a technology to be added to the Java platform.Most important Multimedia-related JSRs are:

JSR 135: Java Mobile Media API (for J2ME platform)JSR 32 and JSR 180: Java SIP APIsJSR 231 and 926: Java 3D graphic APIsJSR 184 and 239: Java 3D graphic APIs for J2MEJSR 226: Scalable 2D Vector Graphics API (J2ME)JSR 234: Java Advanced Multimedia API for J2MEJSR 272: Mobile Broadcast Service API for J2ME