MultiPath TCP: From Theory to Practice · S ebastien Barr e - Christoph Paasch - Olivier BonaventureMultiPath TCP: From Theory to Practice 16/17. MultiPath TCP - Architecture MultiPath

MultiPath TCP - ArchitectureMultiPath TCP - Receive-Buffer

MultiPath TCP - Congestion ControlMultiPath TCP - Live-Demo

MultiPath TCP: From Theory toPractice

Sebastien BarreChristoph PaaschOlivier Bonaventure

9 mai 2011

http ://inl.info.ucl.ac.be/mptcp/

Sebastien Barre - Christoph Paasch - Olivier Bonaventure MultiPath TCP: From Theory to Practice 1 / 17



MultiPath TCP

Mobile devices can connect to the Internet via different interfaces

3G

WiFi

Internet

Data-centers have a large redundant infrastructure

Dual-Homed Servers




MultiPath TCP

Mobile devices can connect to the Internet via different interfaces

3G

WiFi

Internet

Data-centers have a large redundant infrastructure

Dual-Homed Servers




MultiPath TCP

State of the Art

TCP is used for 95% of the Internet communications

A single TCP connection cannot be used across different interfaces.

MultiPath TCP (short MPTCP)

MPTCP allows a single data-connection to use several interfacessimultaneously.

Allows failover from one interface to another (e.g., mobile client).

Increases the bandwidth due to resource pooling.




MultiPath TCP : From Theory To PracticeS. Barre, C. Paasch, O. Bonaventure

We implemented MultiPath TCP in the Linux Kernel.

Solved several design challenges that needed to be considered.

Evaluated and measured the impact of our design choices in atestbed.

Live-Demo of MPTCP at the end of this presentation.




MultiPath TCP - Architecture - From Theory

standard Socket API

Transport Layer

MultiPath TCP

TCPsubflow

TCPsubflow

TCPsubflow

Application Layer

Network Layer

WiFi Wired3G




MultiPath TCP - Architecture - To Practice

Sending packets over MultiPath TCP

standard Socket API

Transport Layer

Application Layer

Network Layer

MultiPath TCPMeta-socket

send-queue

TCPsubflow

Mastersubsocket

TCPsubflow

Slavesubsocket

TCPsubflow

Slavesubsocket





Sending packets over MultiPath TCP

standard Socket API

Transport Layer

Application Layer

Network Layer

MultiPath TCPMeta-socket

send-queue

TCPsubflow

Mastersubsocket

TCPsubflow

Slavesubsocket

TCPsubflow

Slavesubsocket

MultiPath TCP Scheduler





Receiving packets over MultiPath TCP

Packets can be reordered at the data-level due to delay-differences.

MPTCP

MPTCP123

4567out-of-order queue

receive-queue

subflow 1

subflow 2

subflow 1

subflow 2

low-delay path

high-delay path

8






A loss at the subflow-level (or network-reordering) can also causereordering at the subflow-level

MPTCP

MPTCP123


receive-queue

subflow 1

subflow 2

subflow 1

subflow 2

low-delay path

high-delay path

8109

dropped packet






Subflow-level out-of-order queues are necessary to handle theretransmission at the subflow-level

MPTCP

MPTCP123


receive-queue

subflow 1

subflow 2

subflow 1

subflow 2

low-delay path

high-delay path

89

retransmitted packet

10

subflowout-of-order queue






standard Socket API

Transport Layer

Application Layer

Network Layer

MultiPath TCPMeta-socketreceive-queueofo-queuesend-queue

TCPsubflow

Mastersubsocket

ofo-queue

TCPsubflow

Slavesubsocket

ofo-queue

TCPsubflow

Slavesubsocket

ofo-queue





Interconnected testbed with two separate paths at 1Gbps

1Gbps

1Gbps

0

500

1000

1500

2000

1000 2000 3000 4000 5000 6000 7000 8000 9000

iperfg

oodp

ut(M

bps)

MSS (bytes )





regular TCP can only use one single path

1Gbps

1Gbps

regular TCP

0

500

1000

1500

2000

1000 2000 3000 4000 5000 6000 7000 8000 9000

iper

fgoo

dput

(Mbp

s)

MSS (bytes )

regular TCP





MultiPath TCP uses both paths simultaneously

1Gbps

1Gbps

MPTCP

0

500

1000

1500

2000

1000 2000 3000 4000 5000 6000 7000 8000 9000

iper

fgoo

dput

(Mbp

s)

MSS (bytes )

MultiPath TCPregular TCP




MultiPath TCP - Receive-Buffer - From Theory

Regular TCP : receive-buffer should be twice the BDP

Support a fast-retransmitSupport network-level reordering

MultiPath TCP : Higher reordering possible due todelay-differences of the paths.

MPTCP

MPTCP123


receive-queue

subflow 1

subflow 2

subflow 1

subflow 2

low-delay path

high-delay path

8




MultiPath TCP - Receive-Buffer - To Practice

⇒A subflow on a slow path (high RTT) may block a subflow on a fastpath from transmitting.

RTTmax






⇒We need to cope with one fast-retransmit and network-level reorderingon this slow path

RTTmax ∗ 2






⇒We need to cope with one fast-retransmit and network-level reorderingon this slow path

⇒During this time, all other subflows should be able to transmit at fullspeed (BWi = bandwidth of subflow i)

RTTmax ∗ 2 ∗∑

i∈subflows

BWi





Regular TCP only gets 100Mbps of goodput

100Mbps

100Mbps+ 0ms

regular TCP

40

60

80

100

120

140

160

180

200

2 4 6 8 10 12 14 16

iperfg

oodp

ut(M

bps)

rcvbuf (MB)

TCP (0 ms ec)





MultiPath TCP doubles the goodput

100Mbps

100Mbps+ 0ms

MPTCP

40

60

80

100

120

140

160

180

200

2 4 6 8 10 12 14 16

iperfg

oodp

ut(M

bps)

rcvbuf (MB)

MPTCP (0 ms ec , 0 ms ec)

TCP (0 ms ec)





10ms and 100ms of difference has not a huge influence

100Mbps

100Mbps+ 10ms

MPTCP

40

60

80

100

120

140

160

180

200

2 4 6 8 10 12 14 16

iperfg

oodp

ut(M

bps)

rcvbuf (MB)

MPTCP (0 ms ec , 0 ms ec)MPTCP (0 ms ec , 10 ms ec)

TCP (0 ms ec)





10ms and 100ms of difference has not a huge influence

100Mbps

100Mbps+ 100ms

MPTCP

40

60

80

100

120

140

160

180

200

2 4 6 8 10 12 14 16

iperfg

oodp

ut(M

bps)

rcvbuf (MB)


MPTCP (0 ms ec , 100 ms ec)

TCP (0 ms ec)





500ms of difference affects the goodput significantly

100Mbps

100Mbps+ 500ms

MPTCP

40

60

80

100

120

140

160

180

200

2 4 6 8 10 12 14 16

iperfg

oodp

ut(M

bps)

rcvbuf (MB)



TCP (0 ms ec)




MultiPath TCP - Congestion Control

Several subflows will ”eat up” regular TCP

33%

66%MPTCP

2 subflows

regular TCP shared Bottleneck

1 2 3Number of MPTCP-Subflows

0

20

40

60

80

100

Aver

age

Good

put i

n M

bps

TCPMPTCP-Reno




MultiPath TCP - Congestion Control

Our implementation of the Coupled Congestion Control 1 is fair to regularTCP across shared bottlenecks and avoids the paths with congested links.

50%

50%MPTCP

2 subflows

regular TCP shared Bottleneck

1 2 3Number of MPTCP-Subflows

0

20

40

60

80

100

Aver

age

Good

put i

n M

bps

TCPMPTCP-CCC

1. D. Wischik, C. Raiciu, A. Greenhalgh, M. Handley. ”Design, implementation andevaluation of congestion control for multipath TCP”. NSDI’2011




MultiPath TCP - Live-Demo

Access http ://inl.info.ucl.ac.be/mptcp to download ourimplementation (source-code, Live-CD,...).

Contributions are always welcome ! ! !

Setup for the Live-Demo :




MultiPath TCP - Congestion Control - Theory

Coupled Congestion Control 2

For each ack on subflow i :

cwndi = cwndi + min(α/cwndtot , 1/cwndi )

α = cwndtot

maxi (cwndi∗mss2

i

RTT 2i

)

(∑

icwndi∗mssi

RTTi)2

2. D. Wischik, C. Raiciu, A. Greenhalgh, M. Handley. ”Design, implementation andevaluation of congestion control for multipath TCP”. NSDI’2011




MultiPath TCP - Congestion Control - Practice

Congestion Control in the Linux Kernel

Limitations :

Congestion window is expressed in packets

Linux Kernel cannot handle floating point numbers

Coupled Congestion Control implementation

Count the number of acknowledged packets per subflow incwnd cnti .

Increase congestion window by one, ifcwnd cnti > max(totcwnd/α, 1/cwndi )

Calculate alpha by minimizing the number of divisions and scalingthe remaining ones :

α = cwndtotcwndmax ∗ scalenum

(∑

irttmax∗cwndi∗scaleden

rtti)2


Documents

MultiPath TCP: From Theory to Practice · S ebastien Barr e - Christoph Paasch - Olivier BonaventureMultiPath TCP: From Theory to Practice 16/17. MultiPath TCP - Architecture MultiPath