DOT – D istributed O penFlow T estbed

DOT Distributed OpenFlow Testbed

DOT Distributed OpenFlow Testbed

MotivationMininet is currently the de-facto tool for emulating an OpenFlow enabled network However, the size of network and amount of traffic are limited by the hardware resources of a single machineOur recent experiments with Mininet show that it can causeFlow serialization of otherwise parallel flowsMany flows co-exist and compete for switch resources as transmission rates are limited by the CPUProcess for running parallel iperf servers and clients is not trivial

2ObjectiveRun large scale emulations of an OpenFlow enabled networks andAvoid/reduce flow serialization and contention introduced by the emulation environmentEnable emulation of large amounts of traffic

3DOT EmulationEmbedding algorithm partitions the logical network into multiple physical hostsIntra-host virtual linkEembedded inside a single host Cross-host linkConnects switches located at different hostsGateway Switch (GS) is added to each active physical host to emulate link delay of the cross-host linksThe augmented network with GS is called physical networkSDN controller operates on the logical network4Embedding of Logical Network5

Two Physical MachinesCross-host linksEmulated NetworkPhysical Host 1Physical Host 2Embedding algorithm partitions the emulated network into several physical hosts. Our heuristic minimizes the number of physical hosts and cross-host links and considers the resource constraints. This embedding guarantees resource requirements like CPU, memory, and link bandwidth5

Embedding Cross-host Links6

Physical EmbeddingGateway switchesabbbaaVirtual Switch (VS)Each active physical host contains a Gateway Switch (GS).A cross-host link is divided into two segments. For example, cross-host link a is divided into two segments a and aEach segment is connected to the GS of its physical host. For example, a is connected to GS1

6SDN Controllers View7

Controllers ViewSDN Controller7Software Stack of a DOT Node8

Virtual InterfaceVirtual LinkPhysical LinkOpenFlow SwitchVMs are used for generating traffic. Hypervisor layer is responsible for provisioning VMs.VSs and GSs are instances of OpenFlow enabled virtual switch (e.g., OpenVSwitch)

8Gateway SwitchGateway SwitchA DOT componentOne gateway switch per active physical hostIs attached with the physical NIC of the machineFacilitates inter-physical host packet transferEnables emulation of delays in cross-host linksOblivious of the forwarding protocol used in the emulated network

9

Simulating Delay of the cross host links10

Emulated Network (Only the cross-host links are shown)Physical EmbeddingLink delayOnly one of the segments of a cross-host link will simulate delay10Simulating delay11

A->FB->ED->EScenario explains three packets are being sent over three different cross host links: A-F, B-E, and D-E. 11Simulating delay12

A->FB->ED->EWhen a packet is received at a Gateway Switch through its physical interface, it should identify the remote segment through which it was previously forwardedNow, GS2 has to forward the packet through particular link even if the next hop (e.g., B->E and D->E) is same. Solution of Traffic Forwarding at the Gateway SwitchMac RewritingTaggingTunnel with tag13Approach 1: MAC RewriteEach GS maintains IP to MAC address mapping of all VMsWhen a packet arrives at a GS through logical links, it replaces The source MAC with its receiving port MACThis enables the remote GS to identify the segment through which the packet has been forwardedThe destination MAC with the destination physical hosts physical NICs MACThis enables unicast of the packet through physical switching fabricWhen a GS receives a packet from the physical interfaceIt checks the source MAC to identify the corresponding segment through which it should forward the packetBefore forwarding, it replaces the source and destination MAC by inspecting the IP address field of the packet

14

Approach 1: MAC Rewriting15

MAC (src, dst)IP (src, dst)VM2, VM1VM2, VM1SDN Controller16

Approach 1: MAC RewritingSDN Controller

17

MACIPVM2, VM1VM2, VM1Approach 1: MAC RewritingSDN Controller

18

MACIPVM2, VM1VM2, VM1Approach 1: MAC RewritingSDN Controller

19

MACIPVM2, VM1VM2, VM1Approach 1: MAC RewritingSDN Controller

20

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCApproach 1: MAC RewritingSDN Controller

21

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCApproach 1: MAC RewritingGS1GS2Outward TrafficIf(receiving port PB) srcMacPB ,dstMacPM2If(receiving port PC) srcMacPC ,dstMacPM2

Output: PM1If(receiving port PD) srcMacPD ,dstMacPM1If(receiving port PE) srcMacPE ,dstMacPM1

Output: PM2Inward TrafficIf(srcMAC= PD) output: PBIf(srcMAC = PE) output: PCRestore MAC by inspecting IPIf(srcMAC= PB) output: PDIf(srcMAC = PC) output: PERestore MAC by inspecting IP

22

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCGS1GS2Outward TrafficIf(receiving port PB) srcMacPB ,dstMacPM2If(receiving port PC) srcMacPC ,dstMacPM2

Output: PM1If(receiving port PD) srcMacPD ,dstMacPM1If(receiving port PE) srcMacPE ,dstMacPM1

Output: PM2Inward TrafficIf(srcMAC= PD) output: PBIf(srcMAC = PE) output: PCRestore MAC by inspecting IPIf(srcMAC= PB) output: PDIf(srcMAC = PC) output: PERestore MAC by inspecting IPApproach 1: MAC Rewriting

23

Controllers ViewMACIPPD, PM1VM2, VM1PEPDPM2PM1PBPCApproach 1: MAC RewritingGS1GS2Outward TrafficIf(receiving port PB) srcMacPB ,dstMacPM2If(receiving port PC) srcMacPC ,dstMacPM2

Output: PM1If(receiving port PD) srcMacPD ,dstMacPM1If(receiving port PE) srcMacPE ,dstMacPM1

Output: PM2Inward TrafficIf(srcMAC= PD) output: PBIf(srcMAC = PE) output: PCRestore MAC by inspecting IPIf(srcMAC= PB) output: PDIf(srcMAC = PC) output: PERestore MAC by inspecting IP

24

Controllers ViewMACIPPD, PM1VM2, VM1PEPDPM2PM1PBPCApproach 1: MAC RewritingGS1GS2Outward TrafficIf(receiving port PB) srcMacPB ,dstMacPM2If(receiving port PC) srcMacPC ,dstMacPM2

Output: PM1If(receiving port PD) srcMacPD ,dstMacPM1If(receiving port PE) srcMacPE ,dstMacPM1

Output: PM2Inward TrafficIf(srcMAC= PD) output: PBIf(srcMAC = PE) output: PCRestore MAC by inspecting IPIf(srcMAC= PB) output: PDIf(srcMAC = PC) output: PERestore MAC by inspecting IP

25

Controllers ViewMACIPPD, PM1VM2, VM1PEPDPM2PM1PBPCApproach 1: MAC RewritingGS1GS2Outward TrafficIf(receiving port PB) srcMacPB ,dstMacPM2If(receiving port PC) srcMacPC ,dstMacPM2

Output: PM1If(receiving port PD) srcMacPD ,dstMacPM1If(receiving port PE) srcMacPE ,dstMacPM1

Output: PM2Inward TrafficIf(srcMAC= PD) output: PBIf(srcMAC = PE) output: PCRestore MAC by inspecting IPIf(srcMAC= PB) output: PDIf(srcMAC = PC) output: PERestore MAC by inspecting IP

26

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCApproach 1: MAC RewritingGS1GS2Outward TrafficIf(receiving port PB) srcMacPB ,dstMacPM2If(receiving port PC) srcMacPC ,dstMacPM2

Output: PM1If(receiving port PD) srcMacPD ,dstMacPM1If(receiving port PE) srcMacPE ,dstMacPM1

Output: PM2Inward TrafficIf(srcMAC= PD) output: PBIf(srcMAC = PE) output: PCRestore MAC by inspecting IPIf(srcMAC= PB) output: PDIf(srcMAC = PC) output: PERestore MAC by inspecting IP

27

Controllers ViewPEPDPM2PM1PBPCMACIPVM2, VM1VM2, VM1Approach 1: MAC RewritingSDN Controller

28

Controllers ViewPEPDPM2PM1PBPCMACIPVM2, VM1VM2, VM1Approach 1: MAC RewritingSDN ControllerAdvantagesPacket size remains sameNo change is required in the physical switching fabricLimitationsNeeds to maintain all IP to MAC address mapping in each of the GSs.Not scalable

29Approach 1: MAC Rewriting29Approach 2: Tunnel with TagAn unique id is assigned to each cross-host linkWhen a packet arrives at a GS through internal logical links It encapsulates the packet with any tunneling protocol (eg. GRE)The destination address is the IP Address of the physical host addressAn tag equal to the id of the cross-host link is assigned to the packet (using tunnel id field of GRE)When an GS receives a packet from the physical interfaceIt checks the tag (tunnel id) field to identify the outgoing segmentIt forwards the packet after decapsulating the tunnel header.30

31

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCApproach 2: Tunnel with TagSDN ControllerCross-host link id#1#2

32

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCApproach 2: Tunnel with TagSDN ControllerCross-host link id#1#2GS1GS2Outward TrafficIf(receiving port PB) tunnelID1 Use tunnel to Machine 2If(receiving port PC) tunnelID2 Use tunnel to Machine 2If(receiving port PD) tunnelID1 Use tunnel to Machine 1If(receiving port PE) tunnelID2 Use tunnel to Machine 1Inward TrafficIf(tunnelID=1) output: PBIf(tunnelID=2) output: PCIf(tunnelID=1) output: PDIf(tunnelID=2) output: PE

33

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCApproach 2: Tunnel with TagMACIPTIDPM1, PM2PM1, PM2#1SDN Controller#1#2Header for encapsulationOriginal PacketTID= Tunnel ID33

34

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCApproach 2: Tunnel with TagMACIPTIDPM1, PM2PM1, PM2#1SDN Controller#1#2TID= Tunnel ID34

35

Controllers ViewMACIPVM2, VM1VM2, VM1PEPDPM2PM1PBPCApproach 2: Tunnel with TagSDN ControllerCross-host link id#1#2GS1GS2Outward TrafficIf(receiving port PB) tunnelID1 Use tunnel to Machine 2If(receiving port PC) tunnelID2 Use tunnel to Machine 2If(receiving port PD) tunnelID1 Use tunnel to Machine 1If(receiving port PE) tunnelID2 Use tunnel to Machine 1Inward TrafficIf(tunnelID=1) output: PBIf(tunnelID=2) output: PCIf(tunnelID=1) output: PDIf(tunnelID=2) output: PEAdvantagesNo change is required in the physical switching fabricNo GS need to know IP-MAC address mappingRule set in GS is the order of cross-host linkScalable solutionLimitationsLowers the MTUDue to the scalability issue, we choose this solution36Approach 2: Tunnel with Tag36Emulating BandwidthConfigured for each logical link Using Linux tc commandMaximum bandwidth for a cross-host link is bounded by the physical switching capacityMaximum bandwidth of an internal link is capped by the processing capability of the physical host

37DOT: SummaryCan emulates OpenFlow network with Specific link delay BandwidthTraffic forwardingGeneral OpenVSwitchForwards traffic as instructed by the Floodlight controllerGateway SwitchesInstances of OpenVSwitchForwards traffic based on pre-configured flow rules38Technology used so farOpenVSwitch : Version 1.8Rate limit is configured in each portFloodlight Controller: Version 0.9Custom modules addedStatic Network Loader, ARP ResolverHypervisorQemu-KVMLink delays are simulated using tc (Linux traffic control)39