1

Parallel TCP Sockets: Simple Model, Throughput

and Validation

Milan Vojnović

Microsoft ResearchUnited Kingdom

Bruno Tuffin

IRISA/INRIA

France

Dhiman Barman

UC Riverside

United States

Eitan Altman

INRIA

France

IEEE Infocom 2006 Talk, Barcelona, Spain, April 25, 2006

2

Motivation• Parallel TCP sockets used for bulk-data transfers

– Throughput improvements– Ex GridFTP

• Throughput characterization of AIMD (“Additive-Increase & Multiplicative Decrease”) connections

• Known: throughput-loss formulae of a single AIMD – Given a loss process (stochastic; stationary; ergodic)– SQRT, Altman, Avrachenkov, Barakat (2000)

• Our goal: Aggregate throughput of N AIMD connections competing for a bottleneck

3

Related Work

• Partial results for the same problem by Altman, El Azouzi, Ross, Tuffin (2004)– For two connections only (N = 2)– Unnecessary assumptions on loss process

• Experimental results by Hacker, Athey, Noble (2002)– Throughput increase exhibits diminishing-returns

with the number of connections

4

This Talk• One main result: throughput formula for parallel,

symmetric AIMD connections– For any given number of connections– For many loss polices (= assignment of congestion

signal to competing connections)

• Throughput second-moment for specific loss polices– Shows throughput higher-order statistic depends on

the loss policy

• Simulation & Internet experiment results

5

Origins of TCP throughput deficiency

• TCP window synchronization

• TCP window control– Congestion avoidance

• Receiver window limitation

0

c

0

c

0

cBottleneck capacity

Connection 1 send rate

Connection 2 send rate

6

Model• Single link of capacity c• Congestion event whenever the aggregate arrival

rate hits c– Model introduced by Baccelli & Hong (2002)

.

.

.

12

N

.

.

.

i

Bottleneck capacity c

AIMD connectionXi = send rate

7

Model (2)• Assumption ONE: at a congestion event, exactly

one connection undergoes multiplicative-decrease– TCP windows non synchronized

• Example: 3 AIMD connections

Time

X(1)

Congestion eventX(1) + X(2) + X(3) = c

X(2)

X(3)

X(1)

Slope

8

Main Result: Throughput Formula

• Consider N symmetric AIMD connections• = multiplicative-decrease factor• Assume ONE = exactly one connection

undergoes multiplicative-decrease per congestion event

The aggregate throughput is:

9

Implications of the Result

• Applies to a broad class of loss processes– A broad class of loss polices to select connection to

undergo a multiplicative-decrease at congestion events– Can depend in many ways on the observed past of

send rates, provided only the system is stable

• Throughput-invariance – Loss policy is irrelevant

• Special cases (for = ½; TCP like):– N = 1 : Utilization = 0.75 c– N = 2 : Utilization = 6/7 c 0.86 c

10

Implications of the Result (2)

• Throughput deficiency due to TCP window adaptation compensated with a few parallel connections

• Utilization – > 90% for N=3– almost 95% for N=6

N

Util

izat

ion

(N)

11

Proof Sketch

= 1 if connection i selected at the n-th congestion event, else 0

12

Proof Sketch (2)

• Palm inversion

• The latter follows by taking expectation on both sides of the recurrence for Xi

2(Tn)

const

13

Example 1: Aggregate Throughput is Invariant

• Connections: – RED (2) and BLUE (2)

• A BLUE connection chosen with probability

Aggregate throughput

Throughput of BLUE connections

Throughput of RED connections

Thr

ough

put (

)

14

Example 2: Loss Polices• Beatdown

– Pick a selected connection as long as possible

• Rate-independent– Pick at random a connection

with fixed probability

• Rate-proportional– Pick at random a connection

proportional to its send rate

• Largest-rate– Pick a connection with largest

send rate

= round-robin (in steady-state, with symmetric connections)

15

Throughput: Higher-order Statistics Depends on Loss Policy

• Result for N = 2 & = ½ • Loss policy: second moment

• Same type of result as in Altman, El Azouzi, Ross, Tuffin (2004)– But a correct version– Full proof; not symbolic math software solving

16

Validation

• ns-2 simulations– Single bottleneck c = 10 Mb/s– Queue discipline either Threshold Dropper or

DropTail or RED– N TCP connections

• Internet experiments– PlanetLab on various end-to-end paths

17

Validation: ns-2 simulations

• Queue discipline: threshold dropper– Drop only from a

selected connection for a fixed time interval Th

– Enforces assumption ONE

– b = buffer size (pkts)

Util

izat

ion

(N)

18

Simulations with DropTail

• Buffer size b = varying parameter

• Suggests buffer-size sensitivity

• Inspection suggests window synchronization

Util

izat

ion

(N)

N

N = 5

19

Simulations with RED

• Buffer size b = varying parameter

• Good conformance for some b and N

Util

izat

ion

(N)

N

N = 10

20

Internet Experiments

Util

izat

ion

(N)

Util

izat

ion

(N)

Util

izat

ion

(N)

Util

izat

ion

(N)

NN

UMASS-BerkeleyRTT = 97 ms

INRIA-StanfordRTT = 195 ms

Cornell-GreeceRTT = 338 ms

Fortiche-TiteufRTT = 1.6 ms

21

Internet Experiments (2)

N N N

Util

izat

ion

(N)

22

Conclusion• Obtained aggregate throughput formula of symmetric AIMD

connections– Under given bottleneck model– And: one congestion signal per link congestion event

• Throughput-invariance to loss policy which connection is signalled

• A few connections suffice to compensate for throughput deficiency due to window adaptation

• Higher-order throughput statistics is not invariant

• Simulation and Internet experiments suggest TCP window synchronization may be common