Ch. 0: Course Overview Multimedia Systems - Classesclasses.engr.oregonstate.edu/eecs/spring2013/ece477/slides/0... · Ch. 0: Course Overview Multimedia Systems ... entertainment,

Ch. 0: Course Overview Multimedia Systems

Prof. Thinh Nguyen (based on Prof. Ben Lee’s slides) Oregon State University

School of Electrical Engineering and

Computer Science

Outline

What is multimedia?

Why multimedia?

Multimedia applications

Technical challenges

Course outline

Course logistics

Chapter 0: Course Overview 2

What is Multimedia?

Development, integration, and delivery of

any combination of text, graphics,

animations, sound, or video through a

computer.

3 Chapter 0: Course Overview

Why Multimedia?

Information can often be better

represented using audio/video/animation

rather than using text, images and

graphics alone.

Collaboration and virtual environments.

Potential for improving our lives (e.g.,

learning, entertainment, and work).

High Market Demand!

◦ Major driver of Computer Technology


Why Multimedia? (cont.)

Convergence of computers, telecommunication, and TV.

◦ Growth in computational capacity:

Dramatic increase in CPU processing power.

Dedicated compression engines for audio, video, etc.

◦ Rise in storage capacity:

Large capacity disks.

Increase in storage bandwidth, e.g., disk array technology.

◦ Surge in available network bandwidth:

High speed fiber optic networks - gigabit networks.

Fast packet switching technology.


Multimedia Mobile Devices


GSM/CDMA

WLAN

Video Audio

Image

Text

Embedded

System (RT)OS

Bluetooth

WiMAX/Bro

Satellite

Enabling Technologies

UWB

Games

GSM/CDMA

WLAN

Video Audio

Image

Text

Embedded

System (RT)OS

Bluetooth Satellite



WiMAX/Bro UWB

Wearable Computers

A wrist-worn Linux PC

CybergLog

Wearable headset system

GSM/CDMA

WLAN

Video Audio

Image

Text

Embedded

System (RT)OS

Bluetooth Satellite



WiMAX/Bro UWB

Residential Services

Desktop Multimedia

Conferencing

Video on Demand

Multimedia Home Shopping

+ Virtual Reality Self-Paced

Education

GSM/CDMA

WLAN

Video Audio

Image

Text

Embedded

System (RT)OS

Bluetooth Satellite



WiMAX/Bro UWB

Business Services

Multimedia Conferencing

Video on Demand

Distance Education

Cooperate Training

GSM/CDMA

WLAN

Video Audio

Image

Text

Embedded

System (RT)OS

Bluetooth Satellite



WiMAX/Bro UWB

Remote Presence Systems

The ability to project yourself to another

location and to move, see hear and talk as

though you were actually there.


InTouch Health RP-7 Robot

Immersive Environment

An artificial, interactive, computer created scene, or

“world” within which a user can immerse themselves.



Virtual Store

(Kimberly Clark Corporation)

Immersive Command & Control

(US Army)

3D Design Software

(Northrop Grumman) Projected Grid

(Los Alamos National Lab)

Space Shuttle Aerodynamics

(Los Alamos National Lab)

Other Multimedia Applications

Science and Technology

◦ Computational visualization and prototyping.

◦ Astronomy, environmental science: efficient

access to large number of satellite images.

Medicine

◦ Diagnosis and treatment - e.g., multimedia

databases that provide support for queries on

scanned images, X-rays, assessments, response,

etc.


Technical Challenges

Need to manage high volumes of data in a timely manner.

Need to process traditional media (text, images) as well as continuous media (audio/video).

Need to communicate multimedia data across networks that makes no promise about ◦ End-to-end delay,

◦ Variation of packet delay within a packet stream (i.e., jitters), or

◦ Bandwidth.

This puts a strain on all aspects of computer and network technologies: computation, communication, and storage.


High Data Volume

Characteristics BW Required

Speech 8000 samples/s,

1 byte/sample 8 Kbytes/s or 64 Kbps

CD Audio

44,100 samples/s,

2 bytes/sample,

stereo

176.4 Kbytes/s or 1.41 Mbps

Satellite Imagery 180180 km2

30 m2 resolution

600 Mbytes/image

(60 MB, compressed)

NTSC Video

30 fps,

640480 pixels,

3 bytes/pixel

27.6 Mbytes/s or 221.2 Mbps

(2-8 Mbps, compressed)

HD video

60 fps,

19201080 pixels,

3 bytes/pixel

373.2 Mbytes/s or 2.99 Gbps

(15-30 Mbps, compressed)


Course Outline

• Ch. 0: Course Overview

• Ch. 1: Audio/Image/Video Representation

• Ch. 2: Basic Coding & Compression Techniques

• Ch. 3: Image compression

• Ch. 4: Video compression

• Ch. 5: Audio compression

• Ch. 6: Multimedia Networking

• Ch. 7: Multimedia Embedded Systems


Ch.1: Audio/Image/Video

Representation • Computer Representation of Audio:

◦ Quantization

◦ Sampling

• Digital Image Representation:

◦ Color system

◦ Chrominance Subsampling

• Digital Video Representation

• Hardware Requirements


Ch. 2: Basic Coding &

Compression Techniques • Why compression?

• Classification

• Entropy and Information Theory

• Lossless compression techniques:

◦ Run Length Encoding

◦ Variable Length Coding

◦ Dictionary-Based Coding

• Lossy encoding


Ch. 3: Image compression

Introduction

JPEG Standard: ◦ YUV or YIQ conversion and subsampling

◦ DCT

◦ Quantization

◦ Zig-zag ordering and RLE/DPCM RLE on AC coefficients

DPCM on DC coefficients

◦ Variable Length Coding (entropy coding)

JPEG Header Format

JPEG Compression Modes


Ch. 4: Video compression

Introduction

MPEG overview

MPEG encoding:

◦ Motion compensation

◦ Frame encoding

MPEG video bit stream

MPEG video standards

H.264/AVC


Ch. 5: Audio Compression

Sampling techniques (PCM):

◦ Linear

◦ Non-linear

μ-Law

A-Law

Generic Coding Techniques:

◦ DPCM

◦ ADPCM

Psychoacoustic Coding

◦ MPEG


Ch. 6: Multimedia Networking

Introduction

Wireless LANs: ◦ What is the Internet?

◦ 802.11 Physical layer

Medium Access Control (MAC) protocol

Mobility

Dealing with packet jitters and losses: ◦ Issues

◦ Reducing Delay Jitters Fixed vs. Adaptive Playout

◦ Recovering from packet loss FEC

Interleaving


Ch. 7: Multimedia

Embedded System • Introduction

• Embedded Processors

• Classic 5-Stage RISC Processor

• The ARM Architecture


Other Topics Not Covered

OS support for multimedia

◦ Real-time scheduling, buffer handling, I/O subsystems, OS structure/architecture, etc.

Multimedia applications and services:

◦ Content authoring

◦ Content query

◦ Distributed collaboration

Multimedia synchronization

Mobile multimedia

…


Course Logistics(1)

Class web site

◦ http://classes.engr.oregonstate.edu/eecs/spring2013/ece477

Textbook:

◦ Fundamentals of Multimedia, by Li and Drew, 2004, Prentice

Hall.<http://www.cs.sfu.ca/mmbook/>

◦ Materials from archive journal, conference, and magazine papers.

See the course web page.

Reference books:

◦ Computer Networking: A Top-Down Approach Featuring the Internet,

4th Edition, by Kurose and Ross, 2008, Addison Wesley.

◦ Internetworking Multimedia, by Crowcroft, Handley, and Wakeman,

1999, Morgan Kaufmann Publishers


Course Logistics(2)

Office Hours: WFr 9:00-10:00 and by

appointment.

Grading Policy:

◦ Assignments (20%)

◦ 2 exams (50%)

◦ Final project and presentation (30%)


Final Project and Presentation

Work in groups.

Start early! Doing background work is more than half

the work.

30% of the grade!

Types of projects:

◦ Implementation/Demo

Project Proposal & Progress (mid) Report

Presentation will be done in the last week(s) of class.

◦ All group members must be involved in the presentation.

Final report due during finals week.


Some Project Topics

• Look at course web site for projects done in the past

• Video/Audio compression/processing

• OS support for multimedia

• Synchronization

• Multimedia applications

• Middleware for multimedia

• Hardware architecture

• Content authoring

• VoD

• VoIP

• QoS

• Security

• …


Documents

Ch. 0: Course Overview Multimedia Systems - Classesclasses.engr.oregonstate.edu/eecs/spring2013/ece477/slides/0... · Ch. 0: Course Overview Multimedia Systems ... entertainment,