T.Sharon-A.Frank1

MultimediaNeed high Bandwidth

Analog vs. DigitalMulti-modalityReal-Time MM

T.Sharon-A.Frank2

Major Issues

1. Size of Data Frame2. Need High Bandwidth3. Digital vs. Analog Representation4. Multi-Modality Integration/Blending5. Soft Real-Time Constraints6. Need Group Communication7. Need Synchronization8. Need for Standards

T.Sharon-A.Frank3

2 .Need High Bandwidth

VOICE RIVER

IMAGE RIVER

BANDWIDTH RIVERBROADBAND

RAY

VIDEO RIV

ER

DATA STREAM

LAN CREEK

MICROBROOK PC R

UN

T.Sharon-A.Frank4

Channel Rates (Bits/Second)

1Gbps1Mbps1Kbps

Video conferencing

Graphical user interfaces

Video phone

Compressed videoPeripheral sharing

TV

Information, e-mail, voice mailRemote file systemInquiry

File transfer

HDTVCDAlarm, pager VoiceTelemetry Fax

T.Sharon-A.Frank5

Applications RequirementsSe

ssio

n D

urat

ion

Channel Rate

101

102

103

105

104

1 Min

1 Hr

1 Day

102 104 106 108 1010

1 KB/s 1 MB/s 1 GB/s

High SpeedData

Information Retrieval

Teleconferencing

CD Audio Entertainment Video

T.Sharon-A.Frank6

Increasing Throughput RequirementsIn

crea

sing

Bur

stin

ess

Bandwidth64K 1.5M 45M 150M 600M

FRAMERELAY

DEDICATED PRIVATE LINE

ISDNSWITCHEDT-SERIES

SMDS

B-ISDN

X.25PACKET

SWITCHING

DIAL-UP

T.Sharon-A.Frank7

LAN/WAN fit for MM?

MMLAN/WAN

Slow Fast

Need Fiber Optics, Broadband,High-Speed Networks (HSN)

T.Sharon-A.Frank8

Solutions

Use High Speed Networks (HSNs)

Use Lossy Compression

Show/See Flash!

T.Sharon-A.Frank9

3 .Digital vs. Analog Representation

Analog Sound Wave Digital Sample “Quantized” Sound Wave

Original Sound Digital Sampling Digital Playback

10

• Only certain signal values are valid:– Relatively immune to corruption by noise.– Do not degrade when copied or transmitted

over network.• Some information lost.• Undersampling

– Samples 'too far apart' so cannot accurately reconstruct original signal.

Digital Signals

T.Sharon-A.Frank11

Solutions

Use Standards

Use Special Hardware (AD/DA)

T.Sharon-A.Frank12

4 .Multi-Modality Integration/Blending

T.Sharon-A.Frank13

Multimedia Input

Eye Tracker

Speech Recognizer

Data Glove

Put that there!

T.Sharon-A.Frank14

Multimedia Output

Maps

0102030405060708090

MIG-27 MIG-29 MIG-31

Speed

MIG-31MIG-29MIG-27720650490SP

FashionedFulcrumFlugger-DMODEL

Data

Graphs

Natural Language

Aircraft Speed Name

MIG-27 490 Flogger-D

MIG-29 650 Fulcrum

MIG-31 720 Fashioned

Tables

Video

There are three types ofSoviet fighters: the MIG-27Flogger-D that travels 490miles per hour, the MIG-29Fulcrum that...

T.Sharon-A.Frank15

Modality Blending

Syntax

Semantics

Pragmatics

Application

Mode 1 Mode 2 Mode 3 Mode n...

T.Sharon-A.Frank16

Multimedia-Multimodal Interface

Syntax

Semantics

Pragmatics

Application

Mode 1 Mode 2 Mode 3 Mode n...

T.Sharon-A.Frank17

5 .Soft Real-Time Constraints

T.Sharon-A.Frank18

Hardware/Software Pipeline

ScheduleCompress Compression

Transmit InitiateTransmit

NetworkDriver Transmission

DigitizeScheduleDigitize

SoftwareHardware

16.5ms 33ms 16.5ms 20ms

20ms 1ms 16.5ms ??Depends on network load

Transmitting Video Frame

T.Sharon-A.Frank19

Solutions

Use Standards

Use Quality of Service (QoS)