Network Coding for the Internet Philip A. Chou, Yunnan Wu, and Kamal Jain * Microsoft Research & Princeton University Communication Theory Workshop,

Network Codingfor the Internet

Philip A. Chou*, Yunnan Wu†, and Kamal Jain*

*Microsoft Research & †Princeton University

Communication Theory Workshop, May 6-8, 2004

and EPFL, May 10, 2004

Outline

Introduction to Network Coding Practical Network Coding

Packetization Buffering

Internet Applications Live Broadcasting, File Downloading,

Messaging, Interactive Communication

Network Coding Introduction

Directed graph with edge capacities Sender s, set of receivers T Ask: Maximum rate to broadcast info from s to T ?

sender s

receiver t in T

Maximum Flow

Menger (1927) – single receiver Maxflow(s,t) ≤ Mincut(s,t) ≡ ht achievable

Edmonds (1972) – all nodes are receivers Maxflow(s,T) ≤ mint ht ≡ h achievable if T=V

sender s

receiver t in T

Network Coding Maximizes Throughput

Alswede, Cai, Li, Yeung (2000) NC always achieves h = mint ht

Li, Yeung, Cai (2003) Koetter and Médard (2003)

sender

receiver

optimal multicast

throughput = 1

network codingthroughput = 2

a

b

a

a

bb

a+ba+b

a+ba,b

a,b

a,b

coding node

b

b

a+b

a+b

a

a

b

b

b

a

a

a

Network Coding Minimizes Delay

Jain and Chou (2004) Reconstruction delay at any node t is no greater than

the maximum path length in any flow from s to t.

optimal multicastdelay = 3

network codingdelay = 2

a+b a+bba

ba

a

a

b

b

a

aa

a

a a

Network Coding Minimizes Energy (per bit)

Wu et al. (2003); Wu, Chou, Kung (2004) Lun, Médard, Ho, Koetter (2004)

optimal multicast

energy per bit = 5

network codingenergy per bit =

4.5

a

a a,b

a a ba,b b,a

Network Coding Applicable to Real Networks?

Internet IP Layer

Routers (e.g., ISP) Application Layer

Infrastructure (e.g., CDN) Ad hoc (e.g., P2P)

Wireless Mobile ad hoc multihop wireless networks Sensor networks Stationary wireless (residential) mesh networks

Theory vs. Practice (1/4) Theory:

Symbols flow synchronously throughout network; edges have integral capacities

Practice: Information travels asynchronously in

packets; packets subject to random delays and losses; edge capacitiesoften unknown, varying as competing communication processes begin/end


Some centralized knowledge of topology to compute capacity or coding functions

Practice: May be difficult to obtain centralized

knowledge, or to arrange its reliable broadcast to nodes across the very communication network being established


Can design encoding to withstand failures, but decoders must know failure pattern

Practice: Difficult to communicate failure pattern

reliably to receivers


Cyclic graphs present difficulties, e.g., capacity only in limit of large blocklength

Practice: Cycles abound. If A → B then B → A.

Our work addresses practical network coding in real networks

Packets subject to random loss and delay Edges have variable capacities due to

congestion and other cross traffic Node & link failures, additions, &

deletions are common (as in P2P, Ad hoc networks)

Cycles are everywhere Broadcast capacity may be unknown No centralized knowledge of graph

topology or encoder/decoder functions Simple technology, applicable in practice

Approach Packet Format

Removes need for centralized knowledge of graph topology and encoding/decoding functions

Buffer Model Allows asynchronous packets arrivals

& departures with arbitrarily varying rates, delay, loss

Standard Framework Graph (V,E) having unit capacity edges Sender s in V, set of receivers T={t,…} in V Broadcast capacity h = mint Maxflow(s,t)

y(e) = ∑e’ me(e’) y(e’) m(e) = [me(e’)]e’ is local encoding vector

s v

y(e’)

te

y(e’1) = x1

y(e’h) = xh

y(e’2) = x2

... ... ...

y(e1)

y(e2)

y(eh)

y(e)

Global Encoding Vectors

By induction y(e) = ∑hi=1 gi(e) xi

g(e) = [g1(e),…,gh(e)] is global encoding vector Receiver t can recover x1,…,xh from

h

t

hhhh

h

h x

x

G

x

x

egeg

egeg

ey

ey

11

1

1111

)()(

)()(

)(

)(

s v

y(e’)

te

y(e’1) = x1

y(e’h) = xh

y(e’2) = x2

... ... ...

y(e1)

y(e2)

y(eh)

y(e)

Invertibility of Gt

Gt will be invertible with high probabilityif local encoding vectors are randomand field size is sufficiently large If field size = 216 and |E| = 28

then Gt will be invertible w.p. ≥ 1−2−8 = 0.996

[Ho et al., 2003][Jaggi, Sanders, et al., 2003]

Packetization

Internet: MTU size typically ≈ 1400+ bytes y(e) = ∑e’ me(e’) y(e’) = ∑h

i=1 gi(e) xi s.t.

Nhhh

N

t

hNhh

N

h xxx

xxx

G

eyeyey

eyeyey

e

e

,2,1,

,12,11,1

21

112111

)()()(

)()()(

)(

)(

y

y

s

x1=[x1,1,x1,2,…,x1,N]

...

x2=[x2,1,x2,2,…,x2,N]

xh=[xh,1,xh,2,…,xh,N]

v

y(e’)

te

... ...

y(e1)

y(e2)

y(eh)

y(e)=[y1(e),y2(e),…,yN(e)]

Key Idea Include within each packet on edge eg(e) = ∑e’ me(e’) g(e’); y(e) = ∑e’ me(e’) y(e’)

Can be accomplished by prefixing i th unit vector to i th source vector xi, i=1,…,h

Then global encoding vectors needed to invert the code at any receiver can be found in the received packets themselves!

Nhhhh

N

t

hNhhhhh

Nh

xxx

xxx

G

eyeyeyegeg

eyeyeyegeg

,2,,

,12,11,1

211

11211111

10

01

)()()()()(

)()()()()(

Cost vs. Benefit Cost:

Overhead of transmitting h extra symbolsper packet; if h = 50 and field size = 28,then overhead ≈ 50/1400 ≈ 3%

Benefit: Receivers can decode even if

Network topology & encoding functions unknown Nodes & edges added & removed in ad hoc way Packet loss, node & link failures w/ unknown

locations Local encoding vectors are time-varying & random

Erasure Protection Removals, failures, losses, poor random

encoding may reduce capacity below h

Basic form of erasure protection:send redundant packets, e.g.,last hk packets of x1,…, xh are known zero

Nhhhh

Nhk

t

kNkkkhk

Nh

xxx

xxx

G

eyeyeyegeg

eyeyeyegeg

,2,,

,12,11,1

211

11211111

10

01

)()()()()(

)()()()()(

Priority Encoding Transmission (Albanese et al., IEEE Trans IT ’96)

More sophisticated form: partition data into layers of importance, vary redundancy by layer

Received rank k → recover k layers Exact capacity can be unknown1 2 3 4 5 Data layer h=6Global encoding vectors

...1 0 0 x1,10 0 0 x1,N...x1,2 x1,3 x1,4 x1,5 x1,6 x1,7 x1,8 x1,11 x1,12 Source packet 1

0 1 0 00 0 0 x2,2 x2,N...x2,3 x2,4 x2,5 x2,6 x2,7 x2,8 ... x2,11 x2,12 Source packet 2

0 0 1 0 00 0 0 x3,3 x3,N...x3,4 x3,5 x3,6 x3,7 x3,8 ... x3,11 x3,12 Source packet 3

1 0 0 0 00 0 0 00 x4,5 x4,N...x4,6 x4,7 x4,8 ... x4,11 x4,12 Source packet 4

1 0 0 0 0 0 0 00 0 0 0 0 x5,8 x5,N...x5,9 x5,10 x5,11 x5,12 ...

1 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 x6,Nx6,12 ... Source packet h=6

Asynchronous Communication

In real networks, “unit capacity” edges grouped Packets on real edges carried sequentially Separate edges → separate prop & queuing delays Number of packets per unit time on edge varies

Loss, congestion, competing traffic, rounding Need to synchronize

All packets related to same source vectors x1,…, xh

are in same generation; h is generation size All packets in same generation tagged with same

generation number; one byte (mod 256) sufficient

Buffering

random combinatio

n

Transmission opportunity: generate packet

buffer

node

arriving packets (jitter, loss, variable rate)

asynchronous

transmission

asynchronous reception

edge

edgeedge

edge

Decoding Block decoding:

Collect h or more packets, hope to invert Gt

Earliest decoding (recommended): Perform Gaussian elimination after each packet

At every node, detect & discard non-informative packets

Gt tends to be lower triangular, so can typically decode x1,…,xk with fewer more than k packets

Much lower decoding delay than block decoding Approximately constant, independent of block length h

Flushing Policy, Delay Spread,and Throughput loss

Policy: flush when first packet of next generation arrives on any edge Simple, robust, but leads to some

throughput loss

Time of arrival at node v

s vFastest path

Slowest path

...

...

... ...

...

...

Flushing time

X

X X X

Delay spread

Ih

(pkt/s) rate sending (s) spreaddelay

(s)duration generation

(s) spreaddelay (%) loss throughput

Interleaving Decomposes session into several

concurrent interleaved sessions with lower sending rates

Does not decrease overall sending rate Increases space between packets in

each session; decreases relative delay spread

0 0 0 201 1 1 1 2 2 23 3 3 3 ...

Simulations Implemented event-driven simulator in C++ Six ISP graphs from Rocketfuel project (UW)

SprintLink: 89 nodes, 972 bidirectional edges Edge capacities: scaled to 1 Gbps / “cost” Edge latencies: speed of light x distance

Sender: Seattle; Receivers: 20 arbitrary (5 shown) Broadcast capacity: 450 Mbps; Max 833 Mbps Union of maxflows: 89 nodes, 207 edges

Send 20000 packets in each experiment, measure: received rank, throughput, throughput loss, decoding

delay vs.sendingRate(450), fieldSize(216), genSize(100), intLen(100)

Received Rank

0 20 40 60 80 100 120 140 160 180 20040

50

60

70

80

90

100

Generation number

Re

ceiv

ed

ra

nk

Chicago (450 Mbps)Pearl Harbor (525 Mbps)Anaheim (625 Mbps)Boston (733 Mbps)SanJose (833 Mbps)

0 5 10 15 20 2540

50

60

70

80

90

100

Field size (bits)

Avg

. re

ceiv

ed

ra

nk


Throughput

400 450 500 550 600 650 700 750 800 850400

450

500

550

600

650

700

750

800

850

Sending rate (Mbps)

Th

rou

gh

pu

t (M

bp

s)


400 450 500 550 600 650 700 750 800 850400

450

500

550

600

650

700

750

800

850

Sending rate (Mbps)

Th

rou

gh

pu

t (M

bp

s)


Throughput Loss

20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

Generation Size

Th

rou

gh

pu

t Lo

ss (

Mb

ps)

Chicago (450 Mbps)Pearl Harbor (525 Mbps)Anaheim (625 Mbps)Boston (733 Mbps)San Jose (833 Mbps)

0 10 20 30 40 50 60 70 80 90 1000

50

100

150

200

250

300

Interleaving Length

Th

rou

gh

pu

t Lo

ss (

Mb

ps)


Decoding Delay

20 30 40 50 60 70 80 90 1000

20

40

60

80

100

120

140

160

180

200

Generation Size

Pa

cke

t de

lay

(ms)

Pkt delay w/ blk decodingPkt delay w/ earliest decoding

0 10 20 30 40 50 60 70 80 90 1000

20

40

60

80

100

120

140

160

180

200

Interleaving Length

Pa

cke

t de

lay

(ms)

Pkt delay w/ blk decodingPkt delay w/ earliest decoding

Network Coding for Internet and Wireless Applications

AkamaiRBN

ALM

Sp

litStr

ea

mC

oopN

et

Dig

ital

Fou

nta

in

Gn

ute

lla

Kaza

a

Bit

Torr

ent

Peer

Net

Win

dow

s M

ess

enge

r

Xbox

LiveAd Hoc

(P2P)

Infra-structure (CDN)

Inst

ant

Mes

sagi

ngwireles

sInterne

t Live

broa

dcas

t

Med

ia o

n

dem

and

Downl

oad

from

pee

r

Coded

downl

oad

Para

llel

downl

oad

Inte

ract

ive

Com

mun

icatio

n

(con

fere

ncin

g,

g

amin

g)

File download

Live Broadcast (1/2) State-of-the-art: Application Layer Multicast

(ALM) trees with disjoint edges (e.g., CoopNet) FEC/MDC striped across trees

Up/download bandwidths equalizeda failed node

Live Broadcast (2/2) Network Coding [Jain, Lovász, Chou (2004)]:

Does not propagate losses/failures beyond child

ALM/CoopNet average throughput: (1–ε)depth * sending rateNetwork Coding average throughput: (1–ε) * sending rate

failed node

affected nodes(maxflow: ht → ht – 1)

unaffected nodes(maxflow unchanged)

File Download State-of-the-Art: Parallel download (e.g.,

BitTorrent) Selects parents at random Reconciles working sets Flash crowds stressful

Network Coding: Does not need to reconcile working sets Handles flash crowds similarly to live broadcast

Throughput download time Seamlessly transitions from broadcast to download mode

Instant Messaging State-of-the-Art: Flooding (e.g., PeerNet)

Peer Name Resolution Protocol (distributed hash table) Maintains group as graph with 3-7 neighbors per node Messaging service: push down at source, pops up at

receivers

How? Flooding Adaptive, reliable 3-7x over-use

Network Coding: Improves network usage 3-7x (since all packets

informative) Scales naturally from short message to long flows

Interactive Communication in mobile ad hoc wireless networks State-of-the-Art: Route discovery and

maintenance Timeliness, reliability

Network Coding: Is as distributed, robust, and adaptive as flooding

Each node becomes collector and beacon of information Minimizes delay without having to find minimum delay

route

a+b a+b

Physical Piggybacking

Information sent from t to s can be piggybacked on information sent from s to t

Network coding helps even with point-to-point interactive communication

throughput energy per bit delay

a b

s t

Summary Network Coding is Practical

Packetization Buffering

Network Coding can improve performance in IP or wireless networks in infrastructure-based or P2P networks for live broadcast, file download, messaging,

interactive communication by improving throughput, robustness, delay,

manageability, energy consumption even if all nodes are receivers, even for point-to-point

communication

Documents

Network Coding for the Internet Philip A. Chou*, Yunnan Wu, and Kamal Jain* * Microsoft Research & Princeton University Communication Theory Workshop,

Network Coding for the Internet Philip A. Chou, Yunnan Wu, and Kamal Jain * Microsoft Research & Princeton University Communication Theory Workshop,