Towards Low Latency Support for the Internetnetseminar.stanford.edu/seminars/04_02_15.pdf© Communication Systems Group (CSG), ETH Zurich Mirja Kühlewind | | Mirja Kühlewind (CSG,

||© Communication Systems Group (CSG), ETH Zurich Mirja Kühlewind

Mirja Kühlewind (CSG, ETH Zurich) <[email protected]>

April 2, 2015 Networking Seminar Stanford University

02.04.2015 1

Towards Low Latency Support for the Internet

||© Communication Systems Group (CSG), ETH Zurich Mirja Kühlewind - Towards Low Latency Support in the Internet 02.04.2015

The End-to-End Principle

TCP/UDP/X over IPNAT firewall

acceltunnel tunnel

TCP/UDP over IP TCP(/UDP?) over IPHTTP over TCP over IP ? over ?

Packet “mangling“ limits deployability of new services!

2


How can we de-ossify the Internet?

Research questions

How can we better support new services, e.g. with low latency requirements such as WebRTC?

How can we enable the deployment of new protocols and protocol extensions, such as MPTCP?

How can we better support new low-latency services?

How can we de-ossify the Internet?

3


▪ ECN as an enabler for a new service model ▪ Overview Low Latency Support and ECN ▪ ECN Deployment Status and Standardization Activities ▪ Using Data Center TCP (DCTCP) in the Internet?

▪ TCP SIAD: Congestion Control supporting High Speed and Low Latency ▪ Design Goals ▪ Scalable Increase Adaptive Decrease (SIAD) Principle

▪ Middle/End Signaling: Substrate Protocol for User Datagrams (SPUD) ▪ Requirements and Prototype ▪ SPUD-based Information to enable Low Latency Support

Outline

4


Configure small queuing buffers or use of Active Queue Management (AQM)

→ Might cause link underutilization (when using today’s congestion control)

Supporting Low Latency in the Network

5


Adaptive congestion control based on queue length/delay measurements

→ Use Explicit Congestion Notification (ECN) to migrate higher loss rate

Supporting Low Latency in the Host

6


▪ RFC 3168 (2001): “The Addition of Explicit Congestion Notification (ECN) to IP“ ▪ IP/TCP extension to signal Congestion Experienced (CE) without packet loss ▪ ECN defined as “drop equivalence” ▪ Largely implemented in most OSes but off by default for most clients (server-mode)

▪ Deployment issue(s) ▪ Initially middleboxes would break or drop ECN-enabled packets ▪ Today with networks optimized for low loss ECN only has low deployment gains

“Just avoiding loss is not good enough… ECN can do much better!“

Explicit Congestion Notification (ECN)

7


Current ECN measurement study with CSG and University of Aberdeen [PAM 2015]:

1. ECN deployment status at web servers 56.17% of Alexa 1M list by Sep 2014

2. Connectivity failure (without RFC3168 fallback) ~0.4% → risk of increased connection latency

3. ECN path failures2 legitimate CE marking…

but also marking without negotiation…?

ECN Deployment Status

8

p(E

CN

sup

port

in A

lexa

1M

)

0

0,15

0,3

0,45

0,6

Year

1999

2003

2007

2010

2014


Accurate ECN (accECN)

ECN provides only one congestion notification per RTT

→ Provide richer ECN feedback and sender can react to the extent of congestion

Immediate ECN Drop-based congestion notification smoothes feedback signal to avoid performance degradation in case of short-term overload

→ Signal congestion with ECN immediately and sender can react appropriately

ECN standardization activities in the IETF

9


Approach: Differentiate services based on ECN capability

1. Different set of RED parameters in one queue

→ Lower thresholds with an increasing amount of ECN-capable traffic

2. Two queues and Weighted Fair Queuing

→ Allows lower delays immediately but scheduling must be addressed (based on the number of active flows?)

→ More fine-grained congestion control needed at the endhost!

Enabling a new service model using ECN

10

Average queue occupancy

Pack

et m

ark

/dro

p p

robabilit

yM

ax_P

rob

RE

NO

Min_T

hresh

DCTCP

Max_T

hresh

RENO

1

0

Max_P

rob

DC

TC

P

Min_T

hresh

RENO

Max_T

hresh

DCTCP

DCTCP

TCP Reno


Goal: Keep buffer empty but maintain high utilization

1. Network node: ECN and RED with immediate step function

2. Sender: Adapt decrease function to extent of congestion α

cwnd ← cwnd – α/2 • cwnd [on congestion notification] α ← (1 – g) α + gF (updated once per RTT)

F: fraction of CE marked packets in last RTT g: weighting factor

3. Receiver: ECN feedback to reflect # of CE marks

Background: Data Center TCP (DCTCP) [Alizadeh et al. SIGCOMM 2010]

11


Goal: Configure existing switches and routers to allow DCTCP to co-exist with traditional TCP traffic → Dual AQM scheme (one shared queue approach)

Using DCTCP in the Internet

12


Pack

et m

ark

/dro

p p

robabilit

yM

ax_P

rob

RE

NO

Min_T

hresh

DCTCP

Max_T

hresh

RENO

1

0

Max_P

rob

DC

TC

P

Min_T

hresh

RENO

Max_T

hresh

DCTCP

DCTCP

TCP Reno


Pack

et m

ark

/dro

p p

robabilit

y

Max_P

rob

Max_T

hresh

1

0

Min_T

hresh

DCTCP

TCP Reno

K5 10 15 20 25

Time (seconds)

0

10

20

30

40

50

Congest

ion

win

dow

(MSS)

TCP Reno

DCTCP

TCP Reno drops

DCTCP markings



▪ TCP SIAD: Congestion Control supporting High Speed and Low Latency ▪ Design Goals ▪ Scalable Increase Adaptive Decrease (SIAD)


Outline

13


▪ Low Latency Support Enable operators to configure small buffers and keep utilization high!

▪ Scalability on High Speed Networks Proposed high speed schemes scale better but still depend on BDP!

▪ Per-user Congestion Policing TCP-friendliness should not be a requirement for congestion control! (Fairness should be enforce on a long-term per-user basis!)

TCP SIAD supporting Low Latency and High Speed

14


High Link Utilization independent of network buffer sizes

Avoid Standing Queue to minimize average queuing delay

Fixed Feedback Rate independent of bandwidth (when self-congested)

Speed-up for Bandwidth Allocation under changing network conditions

Configurable Aggressiveness to be used by higher layer control loop

TCP SIAD: Design Goals

15


▪ Adaptive Decrease adapts the decrease factor β to the queue size ▪ Scalable Increase adapts increase step 𝛂 to realize a fixed feedback rate

Scalable Increase Adaptive Decrease (SIAD)

16

Buffer size of 1 BDP Smaller buffer


Scalable Increase (SI) to receive the congestion feedback with a constant distance of 𝑁𝑢𝑚𝑅𝑇𝑇 RTTs

Adaptive Decrease (AD) to just empty queue without causing underutilization or a standing queue

[on congestion notification]

Algorithm Design: SIAD

17


• Additional Decreases during congestion epoch

to drain the queue

• Fast Increase phase above Linear Increment threshold 𝑖𝑛𝑐𝑡h𝑟𝑒𝑠h

to quickly allocate new bandwidth

• Trend calculation of 𝑐𝑤𝑛𝑑𝑚𝑎𝑥 to improve convergence

Algorithm Design: Further Components

18


▪ TCP SIAD provides same feedback rate on different bandwidth links

TCP SIAD’s Scalable Increase - Single Flow

19

Base RTT: 100 ms 𝑁𝑢𝑚𝑅𝑇𝑇: 20


▪ Queue drains completely with every decrease independent of queue size

TCP SIAD’s Adaptive Decrease - Single Flow

20

Base RTT: 100 ms 𝑁𝑢𝑚𝑅𝑇𝑇: 20


TCP SIAD with TCP Cross Traffic

21

Base RTT: 100 ms Link rate: 10 Mbps



▪ TCP SIAD: Congestion Control supporting High Speed and Low Latency ▪ Design Goals ▪ Scalable Increase Adaptive Decrease (SIAD)


Outline

22


Problem: Deployment of "new" transport protocols/extension limited by packet/flow modifications of middleboxes

that make implicit assumption about transport protocols or applications:

“TCP is congestion controlled” “UDP is attack traffic” “HTTP is web traffic”

Approach: New (UDP-based) transport encapsulation + in-band signaling to declare assumptions and intentions independent of the used transport or higher-layer protocol

Substrate Protocol for User Datagram (SPUD)

23


1. Information exposure is declarative no negotiation: path and endpoints expose properties independently

no assumption what action will follow

2. All entities may trust but verify no trust relationship needed: the best way to prevent cheating is to remove the incentives to do so

3. Information must be incrementally useful no support by all nodes should be required: you must assume you're not being understood

no minimal set of mandatory vocabulary: you must ignore (and not delete) what you don't understand

Constraints on information exposed

24


▪ IAB Workshop on Stack Evolution in a Middlebox Internet (SEMI), Jan 2015 https://www.iab.org/activities/workshops/semi/ ▪ Current work in the IETF needs to evolve the transport stack: low-latency services like interactive media

(RTCWEB), opportunistic security (TCPINC) ▪ Increasing pressure by increasing deployment of encryption

▪ SPUD BoF at IETF-92 in Dallas (last week) ▪ Strong interest in continuous work within the IAB Stack Evolution Program ▪ Potential for another (wg-forming) BoF

▪ Initial Use Case: UDP Firewall Traversal ▪ Use of identifiers (“tubes”) beyond the five-tuple to link packets together ▪ Explicit indication of tube start/end to middlebox (transport protocol independent)

SPUD Background

25

https://www.iab.org/activities/workshops/semi/


▪ Magic number: Constant to identify SPUD encapsulated in UDP

▪ Command: Data, Open, Close, Ack

▪ Application/Path Declaration: CBOR (RFC 7049) contains information for/from the path bound tube ID

SPUD Prototype

26


▪ Latency Sensitivity Flag to indicate low loss vs. low delay preference

▪ Yield to Other Tube to indicate lower priority of a given tube in relation to another tube sent by the same endpoint

▪ Maximum Single Queue Delay to indicate a maximum acceptable single-hop queueing delay per tube (in ms)

▪ Maximum Queue Length to be set by network node to the maximum queuing delay it might induce based on its configuration

SPUD-based Information to support Low Latency

27


▪ Introduce two services: Best-Effort and Low-Latency where endpoint can indicate preference for low loss vs. low latency

▪ ECN can be used … … as a service identifier … to provide more fine-grained congestion feedback than loss

▪ Scalable Increase Adaptive Decrease (SIAD) principle allows the operator to configure small buffers on high speed links

▪ SPUD implements middle-to-end and end-to-middle signaling that can provide additional information to support low latency requirements

Summary and Conclusion

28

Documents

Towards Low Latency Support for the Internetnetseminar.stanford.edu/seminars/04_02_15.pdf© Communication Systems Group (CSG), ETH Zurich Mirja Kühlewind | | Mirja Kühlewind (CSG,