Interactive Content Authoring for A/153 ATSC Mobile Digital Television Employing Open Source Tools

Brad Fortner – Ryerson University

bfortner@ryerson.ca

ATSC TechZone Pavilion

Booth # N4039

interactive content authoring?

Lack of books or

information on

subject that

provide

examples and

clarification.

evolution

ATSC M/H content group

Brad Fortner Tyler Pham Gregory Kon Steven James May

From Left To Right: Many Ayromlou, Victor Nguyen, Alireza

Tahmasbi, Raymond Padillo, Jeremy Littler

a decade later

a decade later- standards

A/153 part 5 set standards for the rich media environment for ATSC mobile dtv

a decade later- content

Steve Bratt W3C Technical Plenary 2003

"The Semantic Web: common framework that allows data to be shared and reused

across application, enterprise, and community boundaries."

a decade later- MPEG 4

MPEG-4 multimedia object based coding standard

Today: way more than a video compression format

H.264/MPEG-4 Part 10

a decade later- MPEG 4

MPEG-4 now 28 parts from which the principles of the

rich media environment for A/153 was derived

a decade later- learned?

=

Rich Media Application

Linked Data

Multimedia Objects

+

agenda Could you

provide some

background?

• background

• the three parts to rich media delivery

• A/153’s rich media environment

• what and why multimedia object coding is important

• A/153’s “push in advance of consumption” content model

• multimedia scenes

• presentation format

• SVG Tiny 1.2

• ECMAScript

• packaging format

• DIMS and carousels

• open source software available for prototype development

• pitfalls and presentation takeaways

rich media delivery

Let’s do a

traffic app!

presentation format

packaging format

transportation format

Rich Media Content Server(s)

Browser with Rich

Media Engine

Let’s do a

traffic app!

Browser with Rich

Media Engine

rich media environment

Let’s do a

traffic app!

presentation format

packaging format

transportation format

Browser with Rich

Media Engine

SVG Tiny 1.2

ECMA Script

Dynamic Interactive Multimedia

Scenes (DIMS)

A/153

Let’s do a

traffic app!

Browser with Rich

Media Engine

Scene Stream Server

scenes in MPEG-4

Server A

Server B

How’s the Weather

App progressing?

Server C

primitive multimedia objects

How’s the

Weather App

progressing?

rendered scene on browser

Scene Stream Server

scenes in MPEG-4

Server A

Server B

How’s the Weather

App progressing?

Server C

primitive multimedia objects

How’s the

Weather App

progressing?

rendered scene on browser scene description

scene description MPEG-4 primitive multimedia objects

How’s the

Weather App

progressing?

rendered scene on browser scene description

Environmental

Objects

Office Desk Chair

Businessman

Objects

Businessman Speech Bubble

Fragment Fragment Fragment Fragment Fragment Fragment Fragment Fragment

Hierarchy

scene description MPEG-4 primitive multimedia objects

How’s the

Weather App

progressing?

rendered scene on browser scene description

• places (arranges) media objects on interface

• transforms media objects attributes to alter appearance

o scale, transparency, color, animation etc.

• provides timing between media objects and streamed data

• provides for user interactivity

• called (microDOM) uDOM in SVG Tiny 1.2

+ =

MPEG-4 object coding primitive multimedia objects

How’s the

Weather App

progressing?

rendered scene on browser scene description

+ =

+ = primitive multimedia objects rendered scene on browser

scene description

presentation format

packaging format

transportation format

SVG Tiny 1.2

ECMA Script

Dynamic Interactive Multimedia Scenes

(DIMS) A/153

pushing multimedia objects

push content model

• return path not guaranteed in ATSC M/H

• return path

• device dependent

• network availability dependent

• forward, store, use content design model

• applications must be designed accordingly

“push content in advance of consumption model”

push content model “push content in advance of consumption model”

• six “Multimedia Scenes” are used in the weather app

• they are transmitted all at one time

• a small number of SVG fragments can be reused over a number of scenes

• judicious use of SVG fragments reduce transmission requirements

push content model “push content in advance of consumption model”

• ECMAScript is used to program the interactive buttons to switch between multimedia scenes

presentation format So how does one

create these multimedia

objects?

SVG tiny 1.2

=

• XML-based file format used for describing two-dimensional vector graphics • editable text • human-readable, machine-readable • can be searched, indexed, and scripted

raindrop object (fragment) markup in SVG Tiny 1.2

SVG tiny 1.2

But what about my

cute cat pictures?

JPEG and PNG

encoded to

base64

SVG tiny 1.2

3X size: above rendered at 1262 * 727 – originally designed for 416 * 240

• traffic object is originally a 293 * 220 bitmap • does not scale with other SVG objects in scene

SVG tiny 1.2 scene creation

scene description

SVG simultaneously creates multimedia objects

and scene description as one authors content

SVG Link to raindrop

fragment Raindrop fragment screen positioning and

sizing markup. This is used as

scene description.

SVG tiny 1.2 scene creation

SVG Tiny 1.2 rich media application displayed in Opera

browser

.svg

freely available software

Show Me

The Free!!!

• any text editor will edit SVG and XSLT

• integrated developer environment (IDE)

freely available software

Show Me

The Free!!!

• SVG vector graphics editors

“VISIO like” “Illustrator like” online

freely available software

Show Me

The Free!!!

• SVG vector graphics libraries

freely available software

SVG Tiny 1.2 rich media application displayed in Opera

browser

inkscape svg object

text inside svg

markup

icon svg objects from

openclipart.org

objects chosen for display

on client using ECMA

script

weather from rss feed

inserted using XSLT

inkscape svg object text inside svg

markup event handler buttons are

ECMAScript functions

software used weather app

packaging format

Scene Stream Server

scene

description

Server A

Server B MPEG-4 Terminal

Server C

3GGP DIMS object packaging

Server A

Server B

Server C

Scene Stream Server and

DIMS Packager

packaged

scene

description

and objects

highly constrained network and

devices

open source DIMS software

What! Open

Source DIMS

software. WOW!

“as is”

open source DIMS software

Understood.

GPAC is useful

for prototypes.

• GPAC Project on Advanced Content

• implementation of the MPEG-4 systems standard

• written in ANSI C (cross platform)

• tools for media playback, vector graphics, MPEG-4 authoring and distribution

• consists of

o multimedia packager called MP4Box

o some server functions (experimental)

o multimedia player called Osmo4

• MP4Box

o generates 3GPP DIMS packaging from SVG files

• Osmo4

o displays SVG Tiny 1.2 scenes packaged in 3GPP DIMS files

3GGP DIMS carousel types

object

0

object

1

object

2

object

3

object

0

object

1

object

2

object

3

static carousel 0

1

2 3

4

GPAC Experimental DIMS Stream Server

• static carousel

• repeatedly sends scene description and objects

• repeats because devices tune in at different times

3GGP DIMS carousel types

• dynamic carousel

• sends updates to screen elements

0

1

2 3

4

GPAC Experimental DIMS Stream Server

dynamic carousel (update)

object

0

object

1

object

2

object

3

object

4

object

0

object

1

object

2

object

3

object

4 object

5 object

5

change change change

GPAC DIMS static operation

object

0

object

1

object

2

object

3

object

0

object

1

object

2

object

3

static carousel 0

1

2 3

4

GPAC Experimental DIMS Stream Server

GPAC DIMS server command-line interface instructions are as follows

GPAC

Software

Packager

Destination

IP Address

Time

Between

Carousel

Sends

Initiates

DIMS

Packaging

Name Of File

To Be Packaged

And Sent

Streams

Scene

Live

GPAC OSMOS4 player

object

0

object

1

object

2

object

3

object

0

object

1

object

2

object

3

static carousel 0

1

2 3

4

GPAC Experimental DIMS Stream Server

GPAC OSMOS4 player command-line interface instructions are as follows

GPAC

OSMOS4

Player

Looks for Session

Description

Protocol from

DIMS server

GPAC dynamic updates

Primary Stream

GPAC Experimental DIMS Stream Server

0

1

2 3

4

Primary Stream

GPAC Experimental DIMS Stream Server

0

1

2 3

4

GPAC dynamic updates

Update Stream

Primary Stream

GPAC Experimental DIMS Stream Server

0

1

2 3

4

GPAC dynamic updates

Update

Main Carousel 0

1

2 3

4

GPAC Experimental DIMS Stream Server

Linked Data Server Linked Data Server

Update

GPAC dynamic updates

Update

Main Carousel 0

1

2 3

4

GPAC Experimental DIMS Stream Server

Linked Data Server Linked Data Server

Update

GPAC dynamic updates

Linked Data Server

• Linux BASH shell script to collect linked data and write the update

file (doScript) • BASH: wget --output-document=camera.jpg http://www.toronto.ca/trafficimages/loc9.jpg

• BASH/DIMS: a="<Replace ref=\"trafficImage\" attributeName=\"xlink:href\"

value=\"data:image/ jpeg;base64,"$(base64 camera.jpg)"\" />"

• BASH: echo $a > updateFile.svg

Update Stream

Primary Stream

GPAC Experimental DIMS Stream Server With BASH Script, Update File, XSLT

Program and Traffic Prototype SVG File

0

1

2 3

4

GPAC

Software

Packager

Destination

IP Address

Time

Between

Carousel

Sends

Initiates

DIMS

Packaging

Name Of File

To Be Packaged

And Sent

Name Of Update

File That GPAC

Watches

Streams

Scene

Live

GPAC dynamic operation

freely available software

Show Me

The Free!!!

Pitfalls – SVG tiny 1.2 viewer

GPAC OSMO4

Browser Render

(Packaged DIMS File)

OPERA

Browser Render

(SVG File)

Lack of a fully

implemented

freely available

SVG tiny 1.2

standard viewer

Update Stream

Primary Stream

GPAC Experimental DIMS Stream Server With BASH Script, Update File, XSLT

Program and Traffic Prototype SVG File

0

1

2 3

4

GPAC – experimental nature

• dynamic carousel

• Prototype crashes when primary and update streams sent too

close together

takeaways - moving objects

presentation format

packaging format

transportation format

SVG Tiny 1.2

ECMA Script

Dynamic Interactive Multimedia Scenes

(DIMS) A/153

takeaways - push

• return path not guaranteed in ATSC M/H

• return path

• device dependent

• network availability dependent

• forward, store, use content design model

• applications must be designed accordingly

“push content in advance of consumption model”

SVG simultaneously creates

multimedia objects and scene

description as one authors

content

Takeaways - SVG

takeaways - carousels

Primary Stream

GPAC Experimental DIMS Stream Server

0

1

2 3

4

Update Stream

GPAC Experimental DIMS Stream Server

0

1

2 3

4

free prototype software

more information

www.openmobiledtv.org

Brad Fortner – Ryerson University bfortner@ryerson.ca

ATSC TechZone Pavilion Booth # N4039

www.openmobiledtv.org

Ryerson University Booth North Hall - #N524

International Research Park