Masaki Hirabaru (NICT) and Jin Tanaka (KDDI)<masaki@nict.go.jp> <tanaka@kddnet.ad.jp>

Impact of Bottleneck Queue on Long Distant TCP Transfer

August 25, 2005NOC-Network Engineering Session

Advanced Network Conference in Taipei

APAN Requirements on Transport

Advanced ► High Speed

International ► Long Distant

Difficulty in Congestion Avoidance is in proportion to:

Bandwidth-Delay Product (BWDP)

Single TCP flowNo fairness considered

Long Distant Rover Control

(at least) 7 minutes one way delay

Image

Command

EarthMars

When operator saw collision, it was too late.

Long-Distance End-to-End Congestion Control

Merge (Bottleneck) A+B > C

Overflow

Sender(JP)

Receiver(US)

Feedback

BWDP: Amount of data sent but not yet acknowledged64Kbps x 200ms = 1600B ~ 1 Packet

1Gbps x 200ms = 25MB ~ 16700 Packets

200ms round trip delay

A

B

C

Analyzing Advanced TCP Dynamic Behavior in a Real Network(Example: From Tokyo to Indianapolis at 1G bps with HighSpeed TCP)

The data was obtained during e-VLBI demonstration at Internet2 Member Meeting 　 in October 2003.

Throughput

RTT

Window Sizes

Packet Losses

The graphs were generatedthrough web100.

ReceiverLinux TCP

Senderdummyne

tFreeBSD

5.1

GbE

RTT 200ms(100ms one-way)

GbE

Only 800 Mbps available

TCP Performance Measurement in Testbedfocus on bottleneck queue

overflow(loss)

queuing delay (q) + trip delay (t) 1/2RTT < t < RTT

1500B MTU

TCP Performance with Different Queue Sizes

TCP’s Way of Rate Control (slow-start)

RTT (200ms)

20ms 40ms 80ms 160ms

t

1Gbps

rate

average rate

150 Mbps average rate overflows with a 1000-packet queue

100Mbps

(a)

Hig

hSpe

ed

(b)

Sca

labl

e(c

) B

IC(d

) F

AS

TBottleneck bandwidth and queue size

TCP Burstyness

* set to 100M for measurement

Measuring Bottleneck Queue Sizes

Switch / Router Queue Size Measurement Result

ReceiverSenderCapacit

y C

packet train lost packetmeasured packet

Queue Size = C x (Delaymax – Delaymin)

DeviceQueuing

Delay (µs)Capacity (Mbps)

Estimated Queue Size (1500B)

FES12GCF 6161 100* 50p/75KB

GB9812T 22168 100* 180p/270KB

Summit1i 20847 100* 169p/254KB

GS4000 738 1000 60p/90KB

FI400 3662 1000 298p/447KB

M20 148463 1000 12081p/18MB

Pro8801 188627 1000 15350p/23MB

cross traffic injectedfor measurement

RouterSwitch

1Gbps(10G)100Mbps

(1G)

b-1)

Typical Bottleneck Cases

RouterSwitch

a)

Queue~100 Queue

~1000

VLANs

Switch/Router

10G LAN-PHYEthernet Untag

b-2)

9.5G WAN-PHY802.1q Tag

Solutions by Advanced TCPs

• Loss-Based ► AQM (Advanced Queue Management)

Reno, Scalable, High-Speed, BIC, …

• Delay-BasedVegas, FAST

• Explicit Router NotificationECN, XCP, Quick Start, SIRENS, MaxNet

How can wee foresee collision (queue overflow)?

Queue Management Methods

FIFO (First In First Out)

RED　（ Random Early　 Detection）

12

43

56

12345

6drop

full

12

43

56

12346

5drop

threshold

HOLB (Head of Line Blocking)

21111

2222

1

full

slow

fast

output queueinput queue

wait

blockedempty

Switch

Note: Ethernet flow-control (PAUSE frame in 802.3x) may produce HOLB (Head of line blocking),resulting in less performance at a backbone switch.

full

PAUSE

Summary

Add an interface to a router. Or,Use a switch with an appropriate interface queue.Let’s consider making use of AQM on a router.

Future Plan

10G bps congestion through

TransPAC2 and JGN II with large delay (>=100 m

s)