Fixing TCP in Datacenters

Costin RaiciuAdvanced Topics in Distributed Systems

2011

TCP Primer

• Loss recovery– Fast retransmit– Timeouts

• Congestion Control• Buffer sizing

TCP Incast

• Why does it happen?• How bad is it?

TCP Incast Kills ThroughputLab Setup, Artifical Synchronization

A datacenter example

How can we fix it?

• Application level– Add jitter– Reduce response size– Use aggregation

Jitter increases mean delay

Fixing Incast at the Transport Layer

• Quickly recover after timeouts

• Or just avoid the timeouts

Quickly recover the timeouts

• Remove RTOmin bound • Millisecond or lower time resolution• A whole paper about this in Sigcomm 2009– But is this enough?

Fixing Incast at Lower Layers

• Add more buffering to switches?– Expensive

• Add shared buffering?

Datacenter Traffic

Datacenter Traffic (2)

DCTCP

• Want to be robust to incast• Want to avoid interference between short and

long flows• Want to avoid buffer pressure

• We can do all this with small buffer usage

How might we do that?

• TCP shouldn’t blow out the buffer– Use delay?

How might we do that?

• TCP shouldn’t blow out the buffer– Use delay?

• Explicit Congestion Notification in switches– Switches tell you when to back off

How might we do that?

• TCP shouldn’t blow out the buffer– Use delay?

• Explicit Congestion Notification in switches– Switches tell you when to back off

• But TCP will underutilize the network if CWND<2*BDP

DCTCP

• Find out alpha fraction of packets that saw congestion

• Set cwnd = cwnd * (1-alpha/2)• Alpha is estimated using ECN signals– EWMA

DCTCP Convergence

• Consider what happens when a new connections starts

• How long does it take to reach equilibrium?– With TCP?– With DCTCP?

Conclusions

• TCP is heavily used in DCs– But sometimes its not ideal

• Simple changes can fix its shortcomings• The same problem (incast) can be fixed at

many layers

Your Presentations

• Read your article very carefully, several times• Tried to understand the “gist” of it– What differentiates it from previous work– What is good about it– What is less good

• Did they achieve their goals?• How would you design a solution to their

problem?

Your Slides

• Aim for 40-50 slides at most• Include as many animations as you can• Rehearse presentation at home a few times• Do not overcrowd your slides– 2-4 bullets per slide are ideal– Anything more is difficult to read

Do NOT

• Add outline slides very often (or at all)• Add a blank “Thank you” or “Questions” last

slide – Always finish your talk on a slide with content

• Read from slides• Stare at the screen• Put everything you have to say on the slide