Mobile, Multimedia and Beyond

How to Meet the DeadlineHow to Meet the Deadlinefor Packet Videofor Packet Video

How to Meet the DeadlineHow to Meet the Deadlinefor Packet Videofor Packet Video

Bernd GirodBernd Girod

Mark KMark KalmanalmanEric SettonEric Setton

Information Systems LaboratoryInformation Systems LaboratoryStanford UniversityStanford University

22B. Girod: Packet Video 2006

[Economist, September 2005]

THE MEANING OF FREE SPEECH

The acquisition by eBay of Skype is a helpful reminder to the world's trillion-dollar telecoms industry that all phone calls will eventually be free . . .

. . . Ultimately—perhaps by 2010—voice may become a free internet application, with operators making money from related internet applications like IPTV . . .

THE MEANING OF FREE SPEECH

The acquisition by eBay of Skype is a helpful reminder to the world's trillion-dollar telecoms industry that all phone calls will eventually be free . . .

. . . Ultimately—perhaps by 2010—voice may become a free internet application, with operators making money from related internet applications like IPTV . . .


IPTV is Becoming a Reality IPTV is Becoming a Reality

SBC (ATT)18M IPTV householdsby 2007

SBC (ATT)18M IPTV householdsby 2007

Verizon10M IPTV households

by 2009

Verizon10M IPTV households

by 2009

[IEEE Spectrum, Jan. 2005]


Why Is Internet Video Hard?Why Is Internet Video Hard?

Internet is a best-effort network . . .

Congestion Insufficient rate to carry all trafficPacket loss Impairs perceptual qualityDelay Impairs interactivity of services;

Zapping < 500 ms


How to Meet the Deadline for Packet VideoHow to Meet the Deadline for Packet Video


Internet




• Congestion, QoS, and “fair” sharing• Maximum-utility resource allocation for

multiple video streams• Example: video over wireless home

networks• Congestion-distortion optimized packet

scheduling (CoDiO)• Example: P2P multicasting of live video• Packet scheduling for multicast trees


Measuring CongestionMeasuring Congestion

Traffic flow

E[Delay]“Congestion”

Congestion in packet-switched network:

queuing delay that packets experience, •weighted by size of the packet

•averaged over all packets in the network


Congestion GrowsCongestion GrowsNonlinearly with Link UtilizationNonlinearly with Link Utilization

Congestion [seconds]

Rate R

Example: M/M/1 model

1 =

C-R

Example: M/M/1 model

1 =

C-R

C


1.22 MTUR

RTT p

1.22 MTUR

RTT p

How 1B Users Share the InternetHow 1B Users Share the Internet

maximum transfer

unit

roundtrip time

packetloss rate

data rate

[Mahdavi, Floyd, 1997]

[Floyd, Handley, Padhye, Widmer, 2000]

Rate R

Growing congestion

p0.0010.0001 0.10.01

TCP Throughput


QoS vs. Best EffortQoS vs. Best EffortReservation-ism

– Voice and video need guaranteed QoS (bandwidth, loss, delay)

– Requires admission control: “Busy tone” when network is full

– Best effort is fine for data applications

Best Effort-ism– Best Effort good enough for

all applications– Real-time applications can

be made adaptive to cope with any level of service

– Overprovisioning always solves the problem, and it’s cheaper than QoS guarantees


Simple Model of A Shared LinkSimple Model of A Shared Link• Link of capacity C is shared among k flows

• Fair sharing: each admitted flow uses rate R=C/k• Homogeneous flows with same utility function u(R)• Total utility

C

CU k k u R k u

k

[Breslau, Shenker, 1998]


Rigid ApplicationsRigid Applications• Utility u=0 below of

minimum bit-rate B

• Admit at most flows

• With sufficient overprovisioning, no admission control needed, since

u

C/kC

kB

B

1

Pr 0C

kB


Elastic ApplicationsElastic Applications

• Elastic applications: convex utility function u(R)

• All flows should be admitted: best effort!

R

u(R)


0 500 1000 1500 2000 2500 3000 3500 400024

26

28

30

32

34

36

38

40

42

44

Y-PS

NR

in d

B

encoding rate in kbps

mobile

foreman

QoS vs. Best Effort for VideoQoS vs. Best Effort for Video• H.264 video coding for 2

different testsequences• Video is elastic application

. . . above a certain minimum quality

• Bottleneck links: admission control and dynamic rate control combined

• Rate must be adapted to network throughput. How?

• Utility function depends on content: should use unequal rate allocation

Foreman

Mobile

Goodpicturequality

Badpicturequality


• Better than utility-oblivious “fair” sharing

• With rk>=0 Karush-Kuhn-Tucker conditions

Different Utility FunctionsDifferent Utility Functions

rk

ukEqual-slope

“Pareto condition”

Vilfredo Pareto1848-1923


Distribution of TV over WLANDistribution of TV over WLAN

[courtesy: van Beek, 2004]

5 Mbps

2 Mbps

11 Mbps

Home MediaGateway


Video over WLANVideo over WLAN

Decoder

Transcoder

Controller

802.11b

Wireless Terminal

NetworkInterface playout

buffer

Video encoded at higher rate

Receiver

[Kalman, van Beek, Girod 2005]


Video over WLAN with Multiple StreamsVideo over WLAN with Multiple Streams

DecoderTranscoder

Controller

Wireless terminals

NetworkInterfaceTranscoder

Transcoder

…

…

…

Decoder

Decoder

…

c0

c1

cM

0

1

M

0

1

M

Receiver

(Multi-Channel)

[Kalman, van Beek, Girod 2005]


1 2 3 4 55

10

15

20

25

30

35

40

45

time in seconds

Y-P

SN

R in

dB

Dynamic Estimation of R-D CurveDynamic Estimation of R-D Curve

Parameters track weighted average of last I-Frame, P-Frame and B-Frame

Scene cuts

[Stuhlmüller et al. 2000]

00

D DR R

0

0

D DR R

R-D Model

Rate


0 5 10 15 20 25 300

2000

4000

ch

an

ne

l ca

pa

city

in

kb

s

0 5 10 15 20 25 300

0.5

1

ch

an

ne

l-tim

e

allo

ca

tio

n

0 5 10 15 20 25 300

1000

2000

tra

nsco

de

d

rate

in

kb

ps

0 5 10 15 20 25 300

5

10

15

ba

cklo

g in

fra

me

s

0 5 10 15 20 25 3010

20

30

40

50

Y-P

SN

R

in

dB

time in seconds

Mean PSNR: 31 dB

802.11b Transmission of 2 Video Streams802.11b Transmission of 2 Video Streams

Linkrates

[kbps]

Channeltime

allocation

Transcoderbit-rate[kbps]

Backlogin frames

PSNRin dB


Video Distortion with SelfVideo Distortion with Self CongestionCongestion

GoodPicturequality

Badpicturequality

Bit-Rate [kbps]

Self congestioncauses late loss


Effect of Playout Delay and Loss Sensitivity Effect of Playout Delay and Loss Sensitivity

Foreman Salesman

Simulations over ns-2

Link capacity 400 kb/s



40% headroom 10%


1 sender

380 kbps, 36 dBHighest sustainable video quality

420 kbps, 33.7 dB

Simulation of 600 kbps link Latency 400 msec


Modeling Self-CongestionModeling Self-Congestionfor Packet Schedulingfor Packet Scheduling

Pro

bab

ility

d

istr

ibu

tio

n

delay

• Rate-distortion optimized packet scheduling (RaDiO) typically assumes independent delay pdfs for successive packet transmissions [Chou, Miao, 2001]

• Model delay pdf by exponential with varying shift

[Setton, Girod, 2004]


CoDiO Light SchedulerCoDiO Light Scheduler

I B B B P

B

IB

B

P

Pictures to sendSchedule

I P B BBP I BBB I

B


CoDiO Scheduling PerformanceCoDiO Scheduling Performance


Packet loss rate 2%

Bandwidth 400 kb/s

Propagation delay: 50ms


Packet loss rate 2%

Bandwidth 400 kb/s


30 %

25 %

Mother & Daughter News

Playout deadline (s) Playout deadline (s)


H.264/AVC @250 kb/sLink rate 400 kb/s, propagation delay 50 ms

2 % packet loss0.6 second playout deadline

CoDiO ARQ


Sequence: Foreman

Packet loss rate 2%



Sequence: Foreman

Packet loss rate 2%



60 %

Playout deadline (s)



CoDiO vs. RaDiOCoDiO vs. RaDiO


Video Multicast over P2P Networks Video Multicast over P2P Networks Challenges• Limited bandwidth • Delay due to multi-hop transmission• Unreliability of peersOur Approach [Setton, Noh, Girod, 2005]• Determine encoding rate as a function of

network bandwidth• Build and maintain complementary

multicast trees• Adapt media scheduling to network

conditions and to content• Request retransmissions to mitigate lossesRelated work• [Chu, Rao, Zhang, 2000]• [Padmanabhan, Wang and Chou, 2003]• [Guo, Suh, Kurose, Towsley, 2003]• [Cui, Li, Nahrstedt, 2004]• [Do, Hua, Tantaoui, 2004]• [Hefeeda, Bhargava, Yau, 2004]• [Zhang, Liu, Li and Yum, 2005]• [Zhou, Liu, 2005]• [Chi, Zhang, Packet Video 2006]

… …Video stream


Experimental SetupExperimental Setup• Network/protocol simulation in ns-2

– 300 active peers – Random peer arrival/departure

average life-time 5 minutes– Over-provisioned backbone– Typical access rate distribution– Delay: 5 ms/link + congestion

• Video streaming– H.264/AVC encoder @ 250 kb/s– 15 minute live multicast

[Sripanidkulchai et al., 2004]

Downlink Uplink Percentage

512 kb/s 256 kb/s 56% 3 Mb/s 384 kb/s 21%1.5 Mb/s 896 kb/s 9% 20 Mb/s 2 Mb/s 3% 20 Mb/s 5 Mb/s 11%

Downlink Uplink Percentage

512 kb/s 256 kb/s 56% 3 Mb/s 384 kb/s 21%1.5 Mb/s 896 kb/s 9% 20 Mb/s 2 Mb/s 3% 20 Mb/s 5 Mb/s 11%

[Setton, Noh, Girod, 2005]


Join and Rejoin LatenciesJoin and Rejoin Latencies

Simulations over ns-2, 300 peers

Number of trees: 4

Retransmissions enabled


Number of trees: 4

Retransmissions enabled[Setton, Noh, Girod, 2005]


CoDiO retransmissions No retransmissions

P2P Video Multicast: 64 out of 300 Peers

H.264 @ 250 kb/s2 second playout deadline for all streams


P2P Video Multicast: 64 out of 300 PeersP2P Video Multicast: 64 out of 300 Peers

H.264 @ 250 kb/s2 second playout deadline for all streams

CoDiO retransmissions No retransmissions


CoDiO Scheduling for Multicast TreesCoDiO Scheduling for Multicast Trees

Parent

PI B P B P B

DI DB DP3 DP2 DP1DB DB

min.E D min.Treesize E D [Setton, Noh, Girod, 2006]

Child

Child


Gain by Multicast CoDiOGain by Multicast CoDiO


Number of trees: 4



Number of trees: 4


30 %40 %

Foreman Mother & Daughter

Playout deadline (s) Playout deadline (s)

[Setton, Noh, Girod, 2006]


Sender-driven CoDiO light33.71 dB

Without prioritization30.17 dB

H.264 @ 250 kb/s0.8 second playout deadline for all streams

Average Video Sequence for 75 Peers


ConclusionsConclusions• Must avoid congestion for low latency• Video streaming over bottlenecks (IPTV, WLAN . . . ):

combine admission control and rate control• R-D-aware rate allocation better than fair sharing• Packet scheduling should consider congestion rather than rate• Low-complexity CoDiO scheduler• P2P video multicast possible with low latency• Retransmissions effective with application-layer multicast• CoDiO extended to packet scheduling for multicast trees

Cross-layer paradigm Media-aware transport superior system performance

The EndThe EndThe EndThe Endhttp://www.stanford.edu/~bgirod/publications.htmlhttp://www.stanford.edu/~bgirod/publications.html

Technology

Mobile, Multimedia and Beyond