The Case Against TCP for Transport Protocol Evolution

Naeem Khademi Networks and Distributed Systems Group Department of Informatics University of Oslo PhD Defense Trial Lecture – June 17, 2015 naeemk@ifi.uio.no

The Internet Transport Layer

3

17 June 2015 4

The Internet Transport Layer

• TCP/IP suite developed in (D)ARPANET (1970s) • TCP: The core transport protocol for the Internet’s operation since

then (RFC 675)

Reliable In-order delivery Stream-oriented Connection-based

TCP Application Transport (TCP/UDP)

Network (IP)

Data Link

Physical

TCP/IP Stack

Source: http://www.generalcomics.com

17 June 2015 5

The Internet Transport Layer

Multiplexing and Corruption handling (ports and checksum)

Connection handling (establishment and teardown)

Stream-based in-order delivery (sequence numbers)

Reliability (ACK and retransmission)

Flow control (overload protection of the receiver)

Congestion control (overload protection of the network, utilizing the

capacity)

Full duplex communication (ACK piggybacking)

TCP Functionalities

17 June 2015 6

The Internet Transport Layer

• A lot of major applications rely on TCP: Worldwide web (HTTP), file transfer (FTP, SCP), email (SMTP, IMAP, POP3,…), NTP, SSH, etc.

• TCP has gradually grown over the years as a set of patches

Robust Scalable (end-to-end design) TCP gets through most middleboxes

TCP and the Internet

17 June 2015 7

The Internet Transport Layer • TCP (RFC 675) or UDP (RFC 768) (by SOCK_STREAM or

SOCK_DGRAM in socket API)

• There are more transport protocols e.g. SCTP (RFC 4960), DCCP (RFC 4340), and a set of multicast transport protocols (e.g. NORM) – Also extensions of TCP or UDP (MPTCP, UDP-Lite, RUDP, µTP/LEDBAT)

1970 2015 ARPANET

1990 1980 2000 2005 2010

RTP 1996

DCCP 2006

MPTCP 2013

SCTP 2000

UDP-Lite 2004

RUDP 1999

UDP 1980

TCP (RFC 675) (1974)

Transport Protocols from IETF’s vantage point

17 June 2015 8

The Internet Transport Layer

• TCP has been the dominant transport protocol in the Internet

• UDP is sometimes blocked (by firewalls) – Hard to establish session/flow state: block rules are stateless

• Analysis of CAIDA traces show increasing trend of UDP usage by

P2P networks and IPTV (2002~2009) M. Zhang et al., ”Analysis of UDP Traffic Usage on Internet Backbone Links”, SAINT 2009

• What about other transport protocols (e.g. SCTP)? Not much or almost not used

• Why? We are going to see later

17 June 2015 9

Is TCP/UDP enough? • Yes

– It has been shown to work OK for so many years in the Internet

• In fact, no – Applications are evolving and so do the app requirements – More multimedia traffic: TCP/UDP was not designed for multimedia – Proof: the existence of other transport protocols

• What do applications expect from a transport? – Simply ”transport services” – Can be hard to identify/define but something like:

• What services applications may need that TCP cannot provide?

– Unordered reliable delivery: disk-over-LAN (e.g. iSCSI, RDMA), latency-sensitive multimedia (eliminates HOL-blocking)

– Multi-streaming: e.g. web: a jpeg file and html text multiplexed – Multi-homing: resilience to the link failure, load sharing – Authentication: e.g. use of cookies – Partial reliability: e.g. No need to retransmit TTL-expired packets – Low Latency: TCP’s 3-way handshake setup, retransmission

17 June 2015

10

Is TCP/UDP enough? (#2)

Transport Services Reliability Performance (e.g. delay, throughput)

Security (e.g. authentication, association, encryption)

Byte-stream, packet or message transfer

In-order delivery Multi-streaming …

17 June 2015 11

What do other transports provide? • Stream Control Transmission Protocol (SCTP)

– Message-oriented data transfer (header chunks) – Provides reliability and congestion control; connection-oriented – 4-way handshake on association establishment (exchange of cookies) – Provides multi-streaming, multi-homing, unordered reliable delivery – Provides partial reliability (optional) (RFC 3758) – Many other features…

Bits Bits 0 - 7 15-Aug 16 - 23 24 - 31 0 Source port Destination port 32 Verification tag 64 Checksum

96 Chunk 1 type Chunk 1 flags Chunk 1 length

128 Chunk 1 data … …

… Chunk N type Chunk N flags Chunk N length

… Chunk N data

SCTP packet header format

Common header

Client Server

SCTP association establishment 4-way handshake

17 June 2015 12

What do other transports provide? • Datagram Congestion Control Protocol (DCCP)

– Message-oriented and unreliable and unordered data transfer – Provides congestion control and ECN using ACKs – Reliable connection setup and teardown – Full-duplex bi-directional communication – Each endpoint can negotiate congestion control mechanism on connection setup

TFRC or TCP-like - Suitable for interactive multimedia and gaming

Prevents HOL-blocking and retransmission (of expired packets) of TCP Prevents congestion induced by UDP Session-based therefore trackable by middleboxes

DCCP = UDP + congestion control or DCCP = TCP – bytestream semantics – full reliability

DCCP

Source: http://heim.ifi.uio.no/michawe//teaching/cn2/cn2-3-6.pdf

17 June 2015 13

Why not use other transports? • SCTP/DCCP can offer various services that TCP/UDP does not

• But sometimes only TCP (http port 80 or https 443) gets through

middleboxes – UDP: not always

• SCTP or DCCP not (or rarely) supported by middleboxes

– NATs: ubiquitous in the network – Firewalls: rules based on TCP/UDP port numbers, DPI – Performance Enhancing Proxies (PEPs): improving TCP, splitters

Source: D. Ailawadi et al., “Minion—an All-Terrain Packet Packhorse to Jump-Start Stalled Internet Transports”, PFLDNeT 2010

17 June 2015

14

Why not use other transports? • Why middleboxes block anything other than TCP (and UDP)?

– Internet was not designed for security => disable/block everything that looks ”strange”

– TCP/UDP can be accepted (judging by port numbers) – => Application programmers do not use other transport protocols

• => Almost impossible to deploy new transports

Deployment impossibility Cycle Source: D. Ailawadi et al., “Minion—an All-Terrain Packet Packhorse to Jump-Start Stalled Internet Transports”, PFLDNeT 2010

What has been happening so far?

15

17 June 2015 16

What has been happening so far? • Applications have built solutions to work around the middlebox

problem

• Build abstraction on top of UDP – SPUD, LEDBAT (BitTorrent), QUIC (Google), Skype, RTMFP (Adobe) – User-space SCTP over DTLS over UDP for WebRTC data channel (draft-ietf-

rtcweb-data-channel-13)

• Build abstraction on top of TCP

– MPTCP, TCP Minion (partially in Apple)

17 June 2015 17

What has been happening so far? (The Minion Suite)

• Minion called by the authors as ”packet packhorse”

• Uses legacy TCP/UDP as ”substrate” (minions) for building new services for apps

– TCP Minion: unordered delivery in TCP

Source: D. Ailawadi et al., “Minion—an All-Terrain Packet Packhorse to Jump-Start Stalled Internet Transports”, PFLDNeT 2010

17 June 2015 18

What has been happening so far? (TCP Minion)

• TCP on the wire; offers datagram service to the apps (unordered delivery)

– Middleboxes cannot distinguish it from normal TCP except by DPI

– uTCP: One new TCP socket option for immediate delivery of received out-of-order segments: SO_UNORDERED via setsockopt()

– Minimizes delay for latency-sensitive apps

– Useful for VPN tunnels (solves TCP-over-TCP issues)

17 June 2015 19

What has been happening so far? (TCP Minion)

• Re-segmentation problem: TCP’s stream-oriented semantics do not preserve any application-relevant frame boundaries within a stream

– Datagram unrecognizable if fragmented by the network

• Use of delimeters (a certain byte) to mark the datagram boundaries

• Consistent Overhead Byte Stuffing (uCOBS): encode app.

datagrams to never include the ”delimiter” byte – Reserves a byte for delimiter – Max. 0.4% expansion – Insert delimiter before and after the datagram (max 0.8% overhead)

17 June 2015 20

What has been happening so far? (TCP Minion)

• Re-segmentation problem:

Source: http://www.ietf.org/proceedings/81/slides/tsvarea-0.pdf

17 June 2015 21

What has been happening so far? (TLS Minion)

• TCP Minion: delimiter can be detected using DPI by middleboxes

• TLS Minion to counter DPI by middleboxes: appears as SSL/TLS on the wire, but appears as HTTPS

17 June 2015 22

What has been happening so far? (MPTCP)

• Multi-Path TCP (MPTCP): multihoming, load sharing, multiplexing and resilience to the link failure (e.g. Wi-Fi and 3G/4G on mobiles)

– Works wherever regular TCP works – Combines one more of regular TCP subflows – Subflow: a regular TCP flow on the wire – Uses TCP options to carry info. about the multipath flow (e.g. a unique token) – Coupled CC: Linked Increases Algorithm (LIA) (RFC 6356), OLIA

Source: C. Paasch et al., “Multipath TCP”, ACM Queue, March 2014

Byte-stream No multi-streaming No unordered delivery No partial reliability Limited TCP options size Has Issues with reordering

in parallel paths …

MPTCP Still…

17 June 2015 23

What has been happening so far? (SPUD)

• Substrate Protocol for User Datagrams (SPUD): – It is a prototype: draft-hildebrand-spud-prototype

– Encapsulate ”new transports” on top of UDP

– Uses bi-directional UDP ”tubes” between endpoints

– Explicit endpoints-to-path communication instead of already implicit

communication (DPI)

– Give middleboxes hint about the ”new transport” => no DPI needed

– UDP tubes identifiable by middleboxes

– Also feedback from the middleboxes to the transport

17 June 2015 24

What has been happening so far? (QUIC)

• Quick UDP Internet Connections (QUIC): – Developed by Google, available since Chromium 29 – Works in browser (user-space) – Multiplexed in-order reliable stream transport (especially HTTP) over UDP – Low latency: fast handshake/encryption setup and data request (0-RTT) – Solves TCPs HOL-blocking of SPDY stream by pausing one stream only

Source: http://blog.chromium.org/2015/04/a-quic-update-on-googles-experimental.html

Internet Transport’s Evolution

25

17 June 2015

26

Zooming Out: Internet Transport’s Evolution • Current (ongoing) path: keep adding patches to TCP

– Addition of CC (1988): Tahoe -> NewReno -> BIC -> CUBIC (Linux) – Window scaling – MPTCP, TCP Minion – And so many other patches and bugfixes

Source: http://www.elliottspourhouse.com

Limited option space (max. 40-bytes) (SCTP has more)

Protocol overhead of x-over-TCP or x-over-y-over-TCP

(e.g. TCP Minion)

Middleboxes modify/remove TCP options

Middleboxes modify sequence numbers (e.g. MPTCP)

DPI (e.g. TCP Minion, etc.)

Some Middleboxes fragile to out-of-order packets

Segment spilitting and coalescing by middleboxes

Challenges ahead Extending TCP

M. Honda et al., “Is it Still Possible to Extend TCP?”, ACM IMC 2011

17 June 2015

Zooming Out+: Internet Transport’s Evolution

27

• Current (ongoing) path: keep relying on current transport interface – Narrow socket API (application chooses between TCP or UDP)

– Lack of abstraction e.g. like OSI model (ask for protocols instead of services)

– Impossible to request/define services

– Very difficult to change and deploy new transport protocols

– Very difficult to make use of existing protocols (e.g. SCTP or DCCP)

Mainly due to lack of middlebox support and DPI

17 June 2015 28

Internet Transport’s Evolution • Middleboxes are here to stay!

Approach B: B. Trammell et al., “Evolving Transport in the Internet”, IEEE Internet Computing, Sep. 2014

A. They will not change their behavior, bypass using legacy traffic encap. (Minion Suite)

B. Bypass them, but give them hints about the new transport to discourage DPI (SPUD)

C. Build a new transport API and try native new transports and fallback to TCP/UDP to

break the chicken-or-egg cycle (TAPS/NEAT)

Possible long-term approaches

Approach A: M. F. Nowlan et al, “Fitting Square Pegs Through Round Pipes: Unordered Delivery Wire-Compatible with TCP and TLS”, ACM NDSI 2012

17 June 2015

29

Internet Transport’s Evolution

• Approach (C): Transport Services (TAPS) – New flexible transport API – Apps ask for services instead of protocols (e.g. low latency, security, msg. priority) – Work in progress @ IETF TAPS WG – Work in progress @ NEAT EU project – Use “new transports” and fallback if no support on the path (happy eyeballs) – Can be orthogonal to or likely to include MPTCP, Minion and SPUD

Remember this? Deployment impossibility Cycle

Application designers

Want best performance at cheapest cost Upgrade apps for better performance

OS developers

Want best performance for mininal risk

Middlebox/firewall vendors/operators Want security and performance Blocking X degrades performance =>

unhappy customers!

Breaking the deployment impossibility cycle

17 June 2015 30

Conclusions

A. TCP is not well suited for the future of the Internet

B. Encapsulating X-over-TCP or Y-over-UDP will encourage DPI by middleboxes, adds overhead and/or may break end-to-end principles

C. Current transport API is narrow for extension of transport protocols

D. New backward-compatible transport API needed that 1. Provides services instead of protocols to the apps 2. Tries protocol X and falls back if not supported by the path 3. Breaks the deployment impossibility cycle

Q&A

31