Accessing Multiple Mirror Sites in Parallel:Using Tornado Codes to Speed Up Downloads

John Byers, Boston University

Michael Luby, Digital Fountain, Inc.

Michael Mitzenmacher, Harvard

INFOCOM 99

The Problem

• Multicast: to save bandwidth.

Sender

Receivers

Senders

Receiver

• Parallel download: to improve speed.

Many-to-many Distribution

• Heterogeneous environment of senders and receivers.• Senders broadcast. • Receivers gather data as fast as possible from as

many sources as possible.

Senders

Receivers

Results

• A simple, robust, scalable solution for parallel downloads and many-to-many distribution using Forward Error Correction.

• Examination of tradeoffs– Speed vs. Goodput

Applications• Internet

– Connect to many mirror sites simultaneously.

• Multiple access media– ISDN and modem simultaneously.

• Mobile clients– Multiple access points.– Listen to multiple frequencies.

• Satellite networks– Ground user receives from many satellites.

Assumptions• Possible to create bottleneck-disjoint paths.

– Otherwise wasted bandwidth, more congestion.– Receiver should not be the bottleneck.– For people with big pipes.

Senders

Receiver

1 2

1 2

Senders

Receiver

1 2

2

1

dropped

BottleneckDisjoint

SharedBottleneck

Solutions without Coding• A protocol without codes:

– Initially receiver tells each of s senders to send disjoint 1/s parts of the file.

– If one sender finishes early, re-negotiate packets to be sent.– Continue until all packets arrive.

• Problems– Significant feedback.– Unsuitable for many-to-many.– Complexity.– No protection against losses.– Wait for last packet.

Forward Error Correction (FEC)

• Message of n packets encoded as cn packets.– A receiver decodes once enough packets arrive.

• FEC codes improve multicast scalability– Encoding packets can correct different losses

for different receivers.

– Reduces feedback, to even feedback-free solutions.

Tornado Codes

• Tornado Codes are FEC codes that are– Very fast (linear time).– Better for large files.– Information-theoretically slightly suboptimal.

Requires 1.055n packets to decode n packet message.

Ideal Solution: Digital Fountain

• Reconstruct file from any n packets, from any source.

• Feedback free: no need for receivers to acknowledge specific packets.

• Fountain metaphor: drink when the cup is full.• Approximate digital fountain solution using

Tornado codes.– Reception inefficiency due to overhead of Tornado

codes, duplicate packets.

Feedback Free Solution• Senders encode message the same way.

• Senders cycle through permutation of encoding

• When receiver obtain any 211 distinct packets, it can decode to obtain the message.

1 - 200

1 - 600

Original Message

Encoded Message

17 485 238 12 311 411 512...

216 156 7 128 415 238 333...

397 188 25 315 275 499 12...

Performance Metrics

• Speedup:

• Stretch factor (c): message of n packets encoded as cn packets.

• Reception inefficiency (z): zn packets arrive before decoding.– Code overhead– Duplicates

Download time now

Download time using single fastest sender

Tradeoffs• Increasing stretch c:

– Lessens duplicates: senders have more packets to send, so random collisions less likely

– Increases encoding/decoding time, memory requirements, and complexity. (Grow linearly in c.)

17 485 238 12 311 411 512...

216 156 7 128 415 238 333...

397 188 25 315 275 499 12...

Speedup vs. Stretch

0

0.5

1

1.5

2

2.5

3

3.5

4

2 3 4 5 6

Stretch Factor

Sp

eed

up

4 senders

3 senders

2 senders

Reception Inefficiency vs. Stretch

1

1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

2 3 4 5 6

Stretch Factor

Rec

epti

on

In

effi

cien

cy

4 senders

3 senders

2 senders

Speedup vs. Stretch

1

1.5

2

2.5

3

2 3 4 5 6

Stretch Factor

Sp

eed

up

2 senders, 1:1

2 senders, 2:1

2 senders, 4:1

Reception Inefficiency vs. Stretch

1

1.05

1.1

1.15

1.2

1.25

1.3

2 3 4 5 6

Stretch Factor

Rec

epti

on

In

effi

cie

ncy

2 senders, 1:1

2 senders, 2:1

2 senders, 4:1

Feedback Free Solution• Pros

– Simple– Loss protection– Good download speedups– No feedback, coordination– Solves many-to-many

• Cons– Extra bandwidth for Tornado codes (5.5%)– Extra bandwidth from packet duplicates

• depends on c, number of senders, variation in rates

• additional 5-25+ %

153421

341253

421354

Rare Feedback• Senders use same permutation of encoding.

• Receivers tell each of s senders to send 1/s of the encoding.– If c > s, each sender has 1 file worth of data.

• In rare cases, re-negotiate, or have senders send the rest in random order.

193672

193672

193672

Rare Feedback• Pros

– Simple– Loss protection– Rare feedback, minimal coordination– Extra bandwidth for Tornado codes only

• Cons– Does not solve multi-multi– Extra bandwidth for Tornado Codes

193672

193672

193672

Conclusions

• Fast parallel download and many-to-many distributions are practical.– Trade goodput for speed.

• FEC improves protocols.– Simpler.– Less feedback.– Loss protection.

• Deployment issues (fairness, bottleneck disjoint paths) still open.