Building Virtual Networks for Experimentation and Profit

Nick Feamster, Georgia Tech

Andy Bavier, Lixin Gao, Mark Huang, Murtaza Motiwala, Jennifer Rexford, Larry Peterson, Yaping Zhu

2

Original Internet Design Goals

• Interconnection/Multiplexing (packet switching)• Resilience/Survivability (fate sharing)• Heterogeneity• Distributed management• Cost effectiveness• Ease of attachment• Accountability

DecreasingPriority

3

Internet Faces New Demands• High availability and performance

– Path diversity, low-latency paths, fast reaction to failures, convergence…

• Security guarantees– Defense against unwanted traffic

• Manageability– Fault detection and localization

• Scaling

• Mobility

4

Design Goal Redux

• Human costs, operational expenditure dominate– The network must provide support for manageability

• Accountability increasingly important– Mobility, security, etc. to the forefront

“…some of the most significant problems with the Internet today relate to lack of sufficient tools for distributed management, especially in the area of routing.”

—Dave Clark, The Design Philosophy of the DARPA Internet Protocols

“Dr Cerf admits that with hindsight, there are two things he regrets not including: authentication, to ensure that internet packets really do come from where they claim to have originated, and support for mobile devices.” ---The Economist, June 2006

5

New Protocols to the Rescue

• Addressing: IPv6, 8+8, HIP, NIRA

• Security: Secure BGP (S-BGP), soBGP, SPV

• Routing: HLP, RCP, Compact/Valiant Routing

• Naming: DOA

• Unwanted traffic: Capabilities, SANE, DoS-Resistant Internet Archictecture,

6

Two Hurdles

• Deployment requires feasibility• Feasibility requires deployment

Deployment Dilemma

Coordination Constraint

• Benefits only realized when large fraction of networks deploy

• No single network wants to deploy first

VINI: Realistic and controlled network experimentation.

Cabo: “Concurrent Architectures are Better than One”

7

Network Virtualization: Characteristics

• Multiple logical routers on a single platform• Resource isolation in CPU, memory, bandwidth,

forwarding tables, …

• Customizable routing and forwarding software• General-purpose CPUs for the control plane• Network processors and FPGAs for data plane

Sharing

Customizability

8

VINI Overview

• Runs real routing software• Exposes realistic network conditions• Gives control over network events• Carries traffic on behalf of real users• Is shared among many experiments

Simulation

Emulation

Small-scaleexperiment

Livedeployment

?VINI

Bridge the gap between “lab experiments” and live experiments at scale.

9

Goal: Control and Realism

• Control– Reproduce results– Methodically change or

relax constraints

• Realism– Long-running services

attract real users– Connectivity to real Internet– Forward high traffic

volumes (Gb/s)– Handle unexpected events

TopologyActual network

Arbitrary, emulated

TrafficReal clients, servers

Synthetic or traces

Network EventsObserved in operational network

Inject faults, anomalies

10

Fixed Physical Infrastructure

11

Shared By Many Parties

12

Supports Arbitrary Virtual Topologies

14

PL-VINI: Prototype on PlanetLab

• First experiment: Internet In A Slice– XORP open-source routing protocol suite – Click modular router

• Expose issues that VINI must address– Unmodified routing (and other) software on a virtual

topology– Forwarding packets at line speed– Illusion of dedicated hardware– Injection of faults and other events

15

PL-VINI: Prototype on PlanetLab

• PlanetLab: testbed for planetary-scale services• Simultaneous experiments in separate VMs

– Each has “root” in its own VM, can customize

• Can reserve CPU, network capacity per VM

Virtual Machine Monitor (VMM)(Linux++)

NodeMgr

LocalAdmin

VM1 VM2 VMn…PlanetLab node

17

XORP: Control Plane

• BGP, OSPF, RIP, PIM-SM, IGMP/MLD

• Goal: run real routing protocols on virtual network topologies

XORP(routing protocols)

18

User-Mode Linux: Environment

• PlanetLab limitation:– Slice cannot create new

interfaces

• Run routing software in UML environment

• Create virtual network interfaces in UML

• Challenge: Map these interfaces to the right tunnels

XORP(routing protocols)

UML

eth1 eth3eth2eth0

19

Click: Data Plane

• Performance– Avoid UML overhead– Move to kernel, FPGA

• Interfaces tunnels– Click UDP tunnels

correspond to UML network interfaces

• Filters– “Fail a link” by blocking

packets at tunnel

XORP(routing protocols)

UML

eth1 eth3eth2eth0

Click

PacketForwardEngine

Control

DataUmlSwitch

element

Tunnel table

Filters

21

Demonstration

planetlab1.csail.mit.edu

planetlab3.csail.mit.edu

planetlab6.csail.mit.edu

planetlab4.csail.mit.edu

1 2

1 3

22

Some Ongoing Experiments

• IGP convergence: data and control plane• RON and overlay experiments• iBGP convergence• Network troubleshooting• …

23

Two Hurdles

• Deployment requires feasibility• Feasibility requires deployment

Deployment Dilemma

Coordination Constraint

• Benefits only realized when large fraction of networks deploy

• No single network wants to deploy first

24

Overcoming the Coordination Constraint

• Key problem: Federation– No Internet Service Provider has control over an

entire end-to-end path– This makes deployment, troubleshooting,

accountability, etc. very dificult

• Idea: Make the infrastructure that supports the testbed the architecture itself– Separate providers of infrastructure and service

25

Concurrent Architectures are Better than One (“Cabo”)

• Infrastructure: physical infrastructure needed to build networks

• Service: “slices” of physical infrastructure from one or more providers

The same entity may sometimes play these two roles.

26

End-to-End Services

Online Banking Web Surfing

Routing Secure routing protocol (e.g., S-BGP)

Lowest common denominator

Addressing Self-certifying addresses(optimized for persistence)

Dynamic addresses(optimized for convenience)

More SecurityMore Complete

Reachability

• Today: Deployment logjam– Deployment requires consensus and coordination

• Instead: Adopt pluralist approach– Determined service provider leases infrastructure and deploys technology end-to-end

Example

27

Application-Specific Networks

Internal BGP Link-State Protocols

Dissemination Hierarchical, incremental Flooding

Computation BGP-style decision process Shortest Paths

Better ScalabilityFaster

Convergence

• Today: Optimize a single set of protocols• Instead: Parallel deployment

– Run multiple networks, each catered to specific applications

Example

28

Embedding

• Given: virtual network and physical network– Topology, constraints, etc.

• Problem: find the appropriate mapping onto available physical resources (nodes and edges)

• Many possible formulations– Specific nodes mapping to certain physical nodes– Generic requirements: “three diverse paths from SF to

LA with 100 MBps throughput”– Traffic awareness, dynamic remapping, etc.– On-the-fly creation of links in the substrate

29

Accounting and Provisioning• Problem: Brokering of physical infrastructure

– Discovery: Discovering physical infrastructure• Autodiscovery of components and topology• Decision elements that configure components

– Provisioning: Creating virtual networks• Requests to decision elements (initially out of

band), which name virtual network components• Turner et al., Substrate Control Metanet

– Creation: Instantiating virtual networks

30

Economic Refactoring

• Infrastructure providers: maintain physical infrastructure needed to build networks

• Service providers: lease “slices” of physical infrastructure from one or more providers

31

Examples in Communications Networks

• Packet Fabric: share routers at exchange points• FON: resells users’ wireless Internet connectivity

• Infrastructure providers: Buy upstream connectivity, broker access through wireless

• Nomads: Users who connect to access points• Service provider: FON as broker

Broker

32

Economic Refactoring: Challenges

• Being a service provider: a great deal– Opportunity to add value by creating new services

• Infrastructure providers– Can this enterprise be profitable?

• Who will become infrastructure providers?

http://www.cc.gatech.edu/~feamster/papers/cabo-tr.pdf

Need to understand whether this refactoring would occur

33

Summary

• Internet faces stronger demands from users– Not necessarily designed for those challenges– Difficult to deploy fundamentally new fixes

• Virtualization: moving beyond paper design– Testbed– Architectural foundation

• Challenges– Simultaneous operation– Substrate and interface

34

35

Supports Controlled Link Failures

36

Carry Traffic for Real End Users

s

c

37

Participate in Internet Routing

s

c

BGP

BGP

BGP

BGP

38

These Solutions Face Two Hurdles

• Deployment requires feasibility• Feasibility requires deployment

Deployment Dilemma

Coordination Constraint

• Benefits only realized when large fraction of networks deploy

• No single network wants to deploy first

39

Intra-domain Route Changes

s

c

1176

587 846

260

700

6391295

2095

902

548

233

1893

366

856

40

Ping During Link Failure

70

80

90

100

110

120

0 10 20 30 40 50

Pin

g R

TT

(m

s)

Seconds

Link down

Link up

Routes converging

41

Close-Up of TCP Transfer

2.1

2.15

2.2

2.25

2.3

2.35

2.4

2.45

17.5 18 18.5 19 19.5 20

Meg

abyt

es in

str

eam

Seconds

Packet receiv ed

Slow start

Retransmitlost packet

PL-VINI enables a user-space virtual networkto behave like a real network on PlanetLab

42

Attracting Real Users

• Could the experiment have been run in Emulab?– Maybe, though interconnection with real Internet

could provide some advantages– “Turning the dials” between realism and contro

• Goal: Operate our own virtual network– Carrying traffic for actual users– We can tinker with routing protocols

43

End-to-End Services

• Multi-provider VPNs• Paths with end-to-end performance guarantees

Today Cabo

Competing ISPs with different goals must coordinate

Single service provider controls end-to-end path

44

Can Other Industries Offer Clues?

• Infrastructure providers: Airports• Infrastructure: Gates, “hands and eyes”, etc.• Service providers: Airlines

SFOATL

BOS

ORD

45

Parallel Deployment: Questions

• Guaranteeing global reachability– Do we need an end-to-end global reachability

service?

• Proliferation of protocols and architectures– Is “low barrier to entry” a good thing for an

architecture?

• Security– Should parallel deployment imply isolation?– If so, how to implement it?