Options or payload?

Costin RaiciuUCL

IETF 78, Maastricht

Outline

• We need explicit data ACKs• How do we encode Data ACK?– Data ACKs in the payload create problems– Data ACKs must be encoded in options

• How do we encode DSN mapping?

We need explicit data ACKs

10,1

Data ACK Window

Receive Window[ , ]

[1, ]

[1,2]

ACK 10,WND 1

20,2

ACK 20, WND 0

1 2

1 0

3 1 11,3

DropPacket loss =congestion

We need explicit data ACKs

10,1

Data ACK Window

Receive Window[ , ]

[1, ]

[2, ]

ACK 10,WND 1

20,2

ACK 20, WND 1

1 2

1 1

3 1 11,3

-----------------APP READ

20,3

Data ACKs must be explicit

• Otherwise flow control does not correctly work

• This may matter more often than we think– Stacks normally use as little buffering as possible

• How do we encode data ACKs?

Problems with Data ACKs in the payload

• Data ACKs are congestion controlled• Head of line blocking of Data ACKs due to

packet loss affect forward throughput– HoL blocking during Fast Retransmit, Timeouts

• Data ACKs are flow controlled – Potential deadlocks, can in principle be avoided

Experiment

• OpenVPN plaintext tunneling– TCP over TCP– TCP over UDP

• Baseline: – TCP over TCP tunneling: 93Mbps– TCP over UDP tunneling: 94Mbps

100Mbps,<1ms

OpenVPN Client OpenVPN ServerRouter

Experiment: ACK path congestion

TCP transfers

OpenVPN Transfer

1 2 4 8 160

10

20

30

40

50

60

70

80

90

TCP over UDPTCP over TCP

Thro

ughp

ut(M

b/s)

Number of Reverse PathTCP connections

Fixing Head of Line blocking

• Possible fix: use escape sequences instead of TLV– Overhead to escape payload• Need to scan all payload in kernel• No hardware acceleration (for now)

– Need to linearly search payload when HoL– Performance will be poor

• Better fix: use options

Problems with Data ACKs in payloadHead of Line Blocking

Congestion Control

Flow Control

Payload Poor performance Poor performanceFIX: do not cc signaling – but what if data and ACKs are mixed?

Possible deadlock:FIX: save last 1kb of receive buffer for signaling

Options No problem No problem No problem

What about Data Sequence Mapping?

• In payload– Can get away with simpler solutions for finding payload

length changes due to middleboxes– No space restrictions

• In options– We are using options anyways, we might as well keep the

payload unmodified– Space is tighter, but can fit multipath without removing

existing options– Future TCP extensions require extending the options space

Conclusions

• Options are the architecturally cleaner approach• Can get through middleboxes– Otherwise protocol detects and quickly falls back to

TCP• Data ACKs must be transmitted out of band– Options are the only feasible alternative

• Data Sequence Mapping may be sent in band– Slight advantage for payload– Which one do people prefer?

Backup Slides

Experiment: bidirectional transfer

TCP transfers

OpenVPN Transfer

1 2 4 8 1605

1015202530354045

TCP over UDPTCP over TCP

Thro

ughp

ut(M

b/s)

Number of TCP connections