Investigating the Use of Synchronized Clocks in TCP Congestion Control

Investigating the Use ofSynchronized Clocks in TCP

Congestion Control

Investigating the Use ofInvestigating the Use ofSynchronized Clocks in TCPSynchronized Clocks in TCP

Congestion ControlCongestion Control

Michele Weigle

Dissertation Defense

May 14, 2003

Advisor: Kevin Jeffay

2

Research QuestionResearch QuestionResearch Question

Can the use of exact timinginformation improve TCPcongestion control?

3

ClaimClaimClaim

synchronized clocks

exact timing information

early congestion detection

less packet loss and shorter queues

better overall network performance

4

OutlineOutlineOutline

• Background

• Related Work

• Thesis Statement

• Sync-TCP

• Evaluation

• Conclusions

• Future Work

5

BackgroundQueuingBackgroundBackgroundQueuingQueuing

• Router queues are FIFO and finite– the longer the queue, the longer a packet at the end of the

queue is delayed– if queue is full, incoming packets are dropped

• Most queues are drop-tail– incoming packets are only dropped when the queue is full

X

6

BackgroundCongestionBackgroundBackgroundCongestionCongestion

• Sustained period where the incoming rate isgreater than the service rate

• Leads to increasedqueuing delays

• Leads to packet loss– leads to increased latency for TCP flows– leads to low throughput

7

BackgroundTCP Data TransferBackgroundBackgroundTCP Data TransferTCP Data Transfer

sender receiver

data 1

ACK 2

time time

data 2

RTT

OTT(data) congestion

window size(cwnd) = 1

throughput = cwnd / RTT

OTT(ACK)

8

BackgroundTCP Congestion WindowBackgroundBackgroundTCP Congestion WindowTCP Congestion Window

sender receiver

data 1

time time

data 2data 3

data 4

ACK 3

ACK 4

data 5data 6

cwnd = 3

ACK 2RTT throughput = cwnd / RTT

9

BackgroundTCP Congestion ControlBackgroundBackgroundTCP Congestion ControlTCP Congestion Control

• Available network bandwidth is unknown

• TCP probes the network by increasing thecongestion window when ACKs return

• TCP backs off by reducing the congestionwindow when loss is detected

10

BackgroundTCP Reno Loss DetectionBackgroundBackgroundTCP Reno Loss DetectionTCP Reno Loss Detection

• 3 Duplicate ACKs– reduce congestion

window by 50%

• RetransmissionTimeout– reduce congestion

window to 1 packettime

cong

estio

n w

indo

w

xx

duplicate ACKs

timeout

throughput = cwnd / RTT

11

BackgroundTCP Reno Data RecoveryBackgroundBackgroundTCP Reno Data RecoveryTCP Reno Data Recovery

sender receiverdata 1

time time

data 3

data 6

data 2

data 2X

data 4data 5

ACK 2

ACK 2

ACK 2

ACK 2

12

The ProblemTCP Congestion ControlThe ProblemThe ProblemTCP Congestion ControlTCP Congestion Control

• Overflows queues in search for moreresources

• Uses packet loss as its only indicator ofcongestion– relies on a binary signal of congestion

13

The ProblemCongestion ControlThe ProblemThe ProblemCongestion ControlCongestion Control

• TCP Reno: React to packet loss– reduce sending rate only when

packets are lost

– perform congestion control onlywhen it is time to retransmit lostpackets

• Goal: React to congestion earlyand avoid losses– congestion occurs before packets are

lost

– decouple congestion control andretransmission

time

cong

estio

n w

indo

w

xx

duplicate ACKs

timeout

time

cong

estio

n w

indo

w

14

Related WorkCongestion ControlRelated WorkRelated WorkCongestion ControlCongestion Control

• Router-based– drop-tail queues are the problem– active queue management (AQM)

router router

adaptation

Internet

router router

adaptationadaptation

Internet

• End-to-End– TCP Reno is the problem

15

Related WorkCongestion ControlRelated WorkRelated WorkCongestion ControlCongestion Control

• End-to-End– Delay-based congestion control [R. Jain, 1989]– TCP Vegas [Brakmo, O’Malley, Peterson, 1994]– TCP Santa Cruz [Parsa, Garcia-Luna-Aceves, 1999]– TCP Westwood [Mascolo, Casetti, Gerla, Sanadidi, Wang, 2001]– TCP Peach [Akyildiz, Morabito, Palazzo, 2001]– Binomial algorithms [Bansal, Balakrishan, 2001]

• Router-based– DECbit [Ramakrishnan, R. Jain, 1990]– Random Early Detection (RED) [Floyd, Jacobson, 1993]– Explicit Congestion Notification (ECN) [Floyd, 1994]– Adaptive RED [Floyd, Gummadi, Shenker, 2001]

16

Thesis StatementThesis StatementThesis Statement

Precise knowledge of one-way transittimes can be used to improve theperformance of TCP congestioncontrol.

17



• network-level metrics: packet loss andaverage queue sizes at congested routers

• application-level metrics: HTTPresponse times and goodput per HTTPresponse

18



• provide lower packet loss and lowerqueue sizes than TCP Reno

• provide lower HTTP response time andhigher goodput per HTTP response thanTCP Reno

19

My ApproachMy ApproachMy Approach

1. Exchange exact timing information

2. Detect congestion

3. React to congestion

4. Sync-TCP congestion control

5. Evaluate Sync-TCP vs. TCP Reno

20

Sync-TCPSynchronized ClocksSync-TCPSync-TCPSynchronized ClocksSynchronized Clocks

• Allow measurementof OTT

• Methods ofsynchronization– Global Positioning

System (GPS)

– Network TimeProtocol (NTP)

Internet

21

Sync-TCPTCP Header OptionSync-TCPSync-TCPTCP Header OptionTCP Header Option

• New option in the TCPheader– 14 bytes

source port # dest port #

application data(variable length)

sequence number

acknowledgment number

rcvr window size

ptr urgent datachecksum

FSRPAUheadlen

notused

options (variable length)

32 bits

timestamp

OTT (ms)type

echo reply

length

22

Sender’s Calculations

Sync-TCPExampleSync-TCPSync-TCPExampleExample

sender receiver[-1, 1, -1]

[1, 3, 1]

[1, 5, 3]

[2, 8, 5]

[OTT, timestamp, echo reply]

time data received = time datasent (echo reply) + OTT

time ACK delayed = time ACKsent (timestamp) - time datareceived

queuing delay = OTT - minimumOTT

time time

123456789

123456789

23

Sync-TCPCongestion DetectionSync-TCPSync-TCPCongestion DetectionCongestion Detection

• 50% of maximum-observed queuing delay(queuing delay = OTT – minimum-observed OTT)

• 50% of minimum-observed OTT

• Average queuing delay

• Trend analysis of queuing delays

• Trend analysis of the average queuing delay

24

Sync-TCPTrend Analysis of Average Queuing DelaySync-TCPSync-TCPTrend Analysis of Average Queuing DelayTrend Analysis of Average Queuing Delay

• Trend analysis for available bandwidth estimationadapted from [Jain and Dovrolis, 2002]

• Operation:– compute 9 average queuing delay samples

– split into 3 groups of 3 samples each

– compute median, mi , of each group

– trend is relationship of m1, m2, m3

25


• Every arriving ACK, compute smoothedaverage queuing delay from OTT

• Compute trend of average queuing delay– after first 9 ACKs

– afterwards, every 3 ACKs1 2 3 4 5 6 7 8 9

• Calculate the average queuing delay as apercentage of the maximum-observedqueuing delay– divide into 25% increments

10 11 12

26

Sync-TCPQueuing Delay at RouterSync-TCPSync-TCPQueuing Delay at RouterQueuing Delay at Router

queu

ing

dela

y (m

s)

time (s)

queuing delay at router100

80

60

40

20

0260 261 262 263 264 265

27

queuing delay at routeraverage computed queuing delay

decreasing trendincreasing trend


max

75%

50%

25%

0%

queu

ing

dela

y (m

s)

100

80

60

40

20

0260

time (s)261 262 263 264 265

28

Sync-TCPCongestion ReactionSync-TCPSync-TCPCongestion ReactionCongestion Reaction

• Decrease congestion window by 50% uponcongestion notification– same reaction as TCP Reno to packet loss

• Increase and decrease congestion windowaccording to congestion signal– intended to be used with trend analysis of average

queuing delay congestion detection

– operates the same as TCP Reno until 9 ACKshave been received

29

Sync-TCPCongestion Window AdjustmentSync-TCPSync-TCPCongestion Window AdjustmentCongestion Window Adjustment

max

75%

50%

25%

0%

increasing trend decreasing trend

aver

age

queu

ing

dela

y

decrease 50%

decrease 25%

decrease 10%

increase 1 packetper RTT

no change

increase 10%per RTT

increase 25%per RTT

increase 50%per RTT

time

30

Sync-TCPCongestion ControlSync-TCPSync-TCPCongestion ControlCongestion Control

• Trend analysis ofsmoothed averagequeuing delay

• 50% of maximumqueuing delay

• 50% of minimum OTT• Smoothed average

queuing delay• Trend analysis of

queuing delays

• Increase and decreasecongestion windowaccording to congestionsignal

• Decrease congestionwindow by 50% uponcongestion notification

Congestion Detection Congestion Reaction

31

• NS-2 network simulator– assume synchronized clocks

• FTP bulk-transfer traffic– examine the steady-state operation of the

mechanisms

• HTTP traffic– integrate traffic model developed at Bell Labs

into NS-2• main parameter is average number of HTTP requests

per second

– calibrate HTTP request rate to desired load level

EvaluationExperiment PlanEvaluationEvaluationExperiment PlanExperiment Plan

32

EvaluationHTTP Simulation EnvironmentEvaluationEvaluationHTTP Simulation EnvironmentHTTP Simulation Environment

• Sync-TCP and TCP Renoflows do not compete

• Two-way traffic– measure performance in one

direction only

• 70-150 new HTTP requestsgenerated per second

• 45-2,500 HTTPconnections activesimultaneously

• 250,000 HTTP request-response pairs completed

10 Mbps

webclients

webservers

webclients

webservers

requestrequest

responseresponse

33

EvaluationHTTP Experiment SpaceEvaluationEvaluationHTTP Experiment SpaceHTTP Experiment Space

• Sync-TCP congestion control mechanism– 50% max queuing delay detection and reduce by 50% reaction– trend analysis of average queuing delay detection and adjust

according to signal reaction

• TCP for comparison– TCP Reno, TCP SACK

• Queuing method for comparison– drop-tail, Adaptive RED, Adaptive RED with ECN

• End-to-end load (% of link capacity)– 50%, 60%, 70%, 80%, 85%, 90%, 95%, 100%, 105%

• Number of congested links– 1, 2 (75% total load, 90% total load, 105% total load)

34

EvaluationEvaluating HTTP PerformanceEvaluationEvaluationEvaluating HTTP PerformanceEvaluating HTTP Performance

• Network-level Metrics– packet loss at bottleneck router– queue size at bottleneck router

• Application-level Metrics– goodput per HTTP response

• bytes received per second at web client

– HTTP response times• time between sending the request and receiving the

entire response

35

TCP Reno

Sync-TCP

EvaluationAverage Packet Loss at BottleneckEvaluationEvaluationAverage Packet Loss at BottleneckAverage Packet Loss at Bottleneck

offered load

8

6

4

2

0

pack

et lo

ss %

3.5

K3.

5 K

00 8 K

8 K

00

21 K

21 K

300

300

45 K

45 K

5 K

5 K

100

K10

0 K

25 K

25 K

180

K18

0 K

90 K

90 K

400

K40

0 K

275

K27

5 K

50% 60% 70% 80% 85% 90% 95%

36

EvaluationAverage Queue Size at BottleneckEvaluationEvaluationAverage Queue Size at BottleneckAverage Queue Size at Bottleneck

80

60

40

20

0

queu

e si

ze (

pack

ets)

TCP Reno

Sync-TCP

50% 60% 70% 80% 85% 90% 95%offered load

37

EvaluationAverage Goodput per ResponseEvaluationEvaluationAverage Goodput per ResponseAverage Goodput per Response

160

120

80

40

0

good

put (

kbps

)

TCP Reno

Sync-TCP

offered load50% 60% 70% 80% 85% 90% 95%

38

Response Time CDFExampleResponse Time CDFResponse Time CDFExampleExample

100

80

60

40

20

0

cum

ulat

ive

prob

abili

ty

HTTP response time (ms)

~75% of the responsescompleted in 400 msor less

200 400 600 800 1000 1200 14000

39

Response Time CDF50% LoadResponse Time CDFResponse Time CDF50% Load50% Load


100

80

60

40

20

0

cum

ulat

ive

prob

abili

ty

200 400 600 800 1000 1200 14000

No large differencebetween uncongestedand congested

40



100

80

60

40

20

0

cum

ulat

ive

prob

abili

ty

200 400 600 800 1000 1200 14000

Sync-TCP performsslightly better thanTCP Reno

41



100

80

60

40

20

0

cum

ulat

ive

prob

abili

ty

200 400 600 800 1000 1200 14000

Sync-TCP performsbetter than both TCPReno and AQM

42



100

80

60

40

20

0

cum

ulat

ive

prob

abili

ty

200 400 600 800 1000 1200 14000

Sync-TCP performsbetter than both TCPReno and AQM

43

EvaluationEarly Congestion DetectionEvaluationEvaluationEarly Congestion DetectionEarly Congestion Detection

• Sync-TCP early congestion detection onlyoperates after 9 ACKs have been received– HTTP responses > 25 KB

• Only 7-8% of HTTP responses > 25 KB

• HTTP responses < 25 KB do not use Sync-TCP early congestion detection– use TCP Reno

44

Evaluation85% Load, 48 MB ResponseEvaluationEvaluation85% Load, 48 MB Response85% Load, 48 MB Response

0

time (s)900 1000 1100 1200 1300 1400 1500

68

34

068

34

cong

estio

n w

indo

w (

pack

ets)

1600

TCP Reno

x packet drop (952)(17 ms base RTT)

Sync-TCP

x packet drop (190)(47 ms base RTT)

45

ConclusionsConclusionsConclusions

• Sync-TCP performs better than TCP Reno– packet loss– average queue size– goodput per HTTP response– HTTP response time

• Sync-TCP has comparable performance to “best”TCP and AQM combination

• Limitations of delay-based congestion control– may not compete well with TCP Reno on same network– with many congested links, decrease in one queue could

mask increase in another queue

46

SummarySummarySummary

synchronized clocks

one-way transit times

early congestion detection

less packet loss and shorter queues

better overall network performance

Taking advantage ofsynchronized clocksin TCP can result inbetter networkperformance.

47

My ContributionsMy ContributionsMy Contributions

• Method for measuring a flow's OTT and returningthis exact timing information to the sender

• Comparison of several methods for using OTTs todetect congestion

• Sync-TCP: a family of end-to-end congestioncontrol mechanisms based on using OTTs forcongestion detection

48

Supporting WorkSupporting WorkSupporting Work

• Study of standards-track TCP congestion control anderror recovery mechanisms in the context of HTTPtraffic– Weigle, Jeffay, and Smith, “Quantifying the Effects of Recent

Protocol Improvements to Standards-Track TCP,” in submission.

• Additions to NS-2– integrated a state-of-the-art random number generator– integrated Bell Labs’ HTTP traffic model– developed a module for delaying and dropping packets on

a per-flow basis according to a given distribution

• Heuristics for determining appropriate run length forHTTP simulations

49

Future WorkFuture WorkFuture Work

• Further Analysis– accuracy of clock synchronization– multiple congested links– Sync-TCP with router support

• Extensions to Sync-TCP– improve congestion detection and reaction– ACK compression– ACK congestion control– improve fairness

• Uses for synchronized clocks in TCP– statistics for time-critical applications– wireless devices

50

Thank YouThank YouThank You

• Committee MembersKevin Jeffay Don Smith

Ketan Mayer-Patel Sanjoy Baruah

Bert Dempsey Jasleen Kaur

• UNC Department of Computer Science

• My parents, Mike & Jean Clark

• My husband, Chris

Technology

Investigating the Use of Synchronized Clocks in TCP Congestion Control