1

How to Identifythe Speed Limiting Factor

of a TCP Flow

E2EMON 2006Vancouver

Mark TimmerApril 3, 2006

Co-authors: Pieter-Tjerk de Boer and Aiko Pras

2

How to Identify the Speed Limiting Factor of a TCP Flow

• Motivation• Related work• Possible factors• Overview: identifying the limiting factors• One factor in-depth• Measurement results• Conclusions

E2EMON, April 3, 2006

Contents

3

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Motivation

• Surfnet: 1 Gbit 10 Gbit

• Upgrades still useful?

• Approach: Identify speed limiting factor

4

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Related work

• Y. Zhang, L. Breslau, V. Paxson, and S. Shenker, “On the characteristics and origins of internet flow rates”,Proceedings of SIGCOMM’02

• M. Siekkinen, G. Urvoy-Keller, E.W. Biersack, and T. En-Najjary, “Root cause analysis for long-lived tcp connections”,Proceedings of CoNEXT’05

5

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Speed limiting factors of a TCP flow

TCPTCP(sender)

network(network layer, linklayer, physical layer) network

application application

(receiver)(sender)

(receiver)

layer

TCP

application

layer

6

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Overview: identifying the limiting factors

Data repository of packet traces

• Information directly available:– Arrival time– IP header (source, destination)– TCP header (port numbers, sequence number, acknowledgment

number, size of receive window)

• Information calculated:– Number of outstanding bytes– Achieved throughput

7

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Application layer limitations (1)

Slow sender limitation

Identification– Measure idle period percentage– Careful when receive window is empty

ack

A B

datadata

idle p

eriod

data

datadatadata

ack

Note:We only consider flows measured at the sending side

8

B

ack (rwnd=30)

ack (rwnd=30)

data (length=30)

ack (rwnd=0)

data (length=30)

A

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Application layer limitations (2)

Slow receiver limitation

Identification– Measure how often the

receive window decreases

9

data (length=30)data (length=30)

ack (rwnd=150)

ack (length=0)ack (length=4)

ack (length=0)ack (length=4)

data (length=30)

data (length=30)data (length=30)

data (length=30)

A B

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Application layer limitations (3)

Application protocol limitation

Identification– Measure how often an ACK

contains data

Note:We only consider flows sending data one-way

10

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Network limitation (1)

Limited by loss / retransmission

time

#outs

tandin

g byte

s ‘TCP Friendly formula’MSS cBWRTT p

11

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

– Large buffers– Entire receive window stored– Injecting more packets more delay

Network limitation (2)

Limited by delay

extra datathroughput of bottleneck link extra delay

12

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

TCP buffers limitations The receive buffer limitation The send buffer limitation

data (length=10data (length=10)

ack (rwnd=30)data (length=10)

ack (rwnd=30)ack (rwnd=30)

ack (rwnd=30)

data (length=10)ack (rwnd=30)

data (length=10)data (length=10)

data (length=10)

rwnd

full

rwnd

full

BA

60 by

tes

data (length=20)data (length=20)data (length=20)

ack (rwnd=100)A B

ack (rwnd=100)data (length=20)

data (length=20)data (length=20)

ack (rwnd=100)

ack (rwnd=100)

ack (rwnd=100)

60 by

tes

13

time1 t2 t4t3

receivewindow

#outs

tandin

g byte

s

t

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

• Peaks

In-depth: the receive window limitation

data (length=10data (length=10)

ack (rwnd=30)data (length=10)

ack (rwnd=30)ack (rwnd=30)

ack (rwnd=30)

data (length=10)ack (rwnd=30)

data (length=10)data (length=10)

data (length=10)

rwnd

full

rwnd

full

BA

time1 t2 t4t3

receivewindow

#outs

tandin

g byte

s

t

14

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Measurement results

Category % flows % bytesUndetermined 20 – 40 10 – 20Network 20 – 30 30 – 40Send buffer 0 – 10 0 – 10Receive buffer 10 – 15 10 – 20Slow sender 20 – 30 20 – 40Slow receiver 0 – 5 0 – 5Application protocol 0 – 10 0 – 1

15

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow

Conclusions

• Upgrading network still useful• However, many flows limited by buffers

• Recommentations:– Perform statistical analyses– Develop tool to ‘fix’ TCP-buffer limited flows– Detect changes of limiting factor in time

16

Questions?

E2EMON, April 3, 2006How to Identify the Speed Limiting Factor of a TCP Flow