Hybrid Programmable Forwarding Planes: BoF Session

HYBRID PROGRAMMABLE FORWARDING PLANE HPFP BOF ONF Summit 2011.10

David Ward

2 Copyright © 2011 Juniper Networks, Inc. www.juniper.net

PURPOSE OF SLIDES

  These slides were used as a conversation starter at the BoF for people interested in discussing HPFP.

  The point of the BoF was to see if there was agreement on the problem space, desire to find solutions and understand if folks were willing to work at the ONF on HPFP.

  The outcome is that a charter is being proposed to the ONF board to form a WG.


WHY HAVE MULTIPLE APPS PROGRAM FWD PLANE?   Large networks of Layer-1-7 devices work today

  If it ain’t broke…

  Layer-3 device learns forwarding entries through multiple sources (IGPs, BGP, LSPs, manual configuration etc.)

  API-based programmable forwarding would extend a device’s capabilities:   Insert entries into the devices’ forwarding chain:

  Programmed prefixes/LSPs, together with match and modify actions   Firewall filter entries   QoS directives

  Read entries/status from the devices’ control plane / forwarding chain:   ALTO: read the content of the RIB

  Common API provided to external sources, creating interface for off-box programming entities

  OpenFlow Controller and/or higher level apps   PCE engines   ALTO   Node resident applications (if an SDK available)


ROUTER - SWITCH CONTROL PLANE

  Multiple roles:   Control & program the hardware

  Knobs to control the forwarding state   Discover & distribute topology & reachability information

  Distribution mechanisms: network protocols   Policies:

  Policy engines   Applications & Services

  Today: built-in, mostly hard-wired   E.g.: Flowspec, VPNs (in general – network virtualization), custom

statistics collection, Service chain control (Firewall, NAT, …), …

  Today – a closed system:   Vendor SDK may be available for a particular vendor platform


ROUTER: CONTROL AND DATA PLANES PACKET FORWARDING PIPELINE

Router

Router Control Plane

Ingress Egress

Route lookup

Output IFL Feature Exec

OFF Feature Execution

IFF Feature Execution

IFL Feature Execution

Packet Decap

Packet Encap

Routing Protocols

RIB, LIB,…

MPLS …

Router Router


ADD PROGRAMMABLE INTERFACES ….

  Replace the existing control plane and come at a low level   The least common denominator…

  Or

  Augment the existing control plane and   Utilize all functionality (control hardware, distribution mechanisms,

policy engines, …   Externalize applications   Come at different levels of abstraction (support different forwarding

paradigms: L2, L3, flexible)   Augment existing forwarding paradigms


ROUTER: ENTER OPENFLOW

Router


Ingress Egress

Route lookup





Packet Decap

Packet Encap

Routing Protocols

RIB, LIB, …

MPLS … OpenFlow 1.0/1.1

Controller

Abstraction level: data plane (low)

Control Plane


ROUTER: CONTROL AND DATA PLANES AUGMENT CONTROL PLANE, CONTROL PKT. FWDG

Router

Ingress Egress

Route lookup





Packet Decap

Packet Encap RIB, LIB, …


…

Router Control Plane Routing

Protocols MPLS OpenFlow

Controller PCE ALTO etc.

ALTO, BGP-TE OF PCEP

Abstraction level: data plane (low), control plane (high)


SHIPS IN THE NIGHT VS. INTEGRATED

Router Router

OpenFlow Control Plane

“Ships-in-the-Night”

•  A subset of ports controlled by OF, another subset controlled by router’s native CP – physical resources are partitioned

•  Some level of integration: “OF_NORMAL”: •  Implementer free to define what “normal” is •  May or may not be what router normally does

Control Plane OpenFlow

“Integrated”

•  Use OF for feature definition – augment the native control plane

•  No longer partitioning of resources •  Can operate at different abstraction levels

(low-level like OK1.0 or higher level)


SHIPS-IN-THE-NIGHT APPROACH

  Create one or more instances of virtual OF-controlled switch

  Network architecture: ships-in-night (physical partitioning) or overlay:   Overlay can still can utilize the underlying networking infrastructure

controlled by the “default” control plane

  The “default” control plane required for IP connectivity between switches and controllers (except where controller on the same subnet as the switch)

  App/Controller needs to set the entire OF switch state


SHIPS-IN-THE-NIGHT VS. INTEGRATED APPROACH Still can do ships-in-the-night, if so desired (multiple abstraction levels defined) Network architecture: logical partitioning or integrated network:

  Application / Controller only needs to set small subset of the overall state   Non-standard treatment (features, forwarding, service chains, …)

Apps can utilize control plane infrastructure: policy engines, state distribution (draft-marques-l3vpn-end-system-02) An app does not have to have to create & set the the entire forwarding state, just of the portion that it wishes to modify

Low level CP functions (ARP, Link bundling, loadsharing, …) provided by the node (app can focus on the goal it wishes to accomplish rather than re-implement control plane functions over and over gain)

Leverage the management plane and available tools Utilize useful CP infrastructure mechanisms & building blocks (state distribution, policy engines, databases, etc.)

Externalize built-in & hardwired applications for better scale; create new apps


ROUTER FORWARDING CHAIN   Multi-stage pipeline   May be distributed across multiple cards, chassis   Rich feature set that can be made available to external apps

  Forwarding model (L2, L3, flexible OF2.0)

  Applications coexist with the control plane:   Security / Access Control (“Sandbox” for apps)   Resource usage limits   Prioritization

Match-Action Table programming model (other Control Plane features will have different models)

  RIB/FIB entries   Features (ingres/egress), e.g. filters   Service chains   QoS


PROGRAMMABLE FORWARDING CHAIN

Packet Decap

IFL Lookup


Internal next_hop

Process next_hop

Output IFL Feature Execution

Output OFF Feature execution

Packet Encap

RIB •  IGP/BGP-Derived entries •  Manual entries •  Programmed entries –

flows, LSPs etc.

Programmed Entry Sources •  OF Controller •  PCE Engine •  Others

Route lookup


Ingress Egress



Packet Decap

IFL Lookup

Internal next_hop

Process next_hop



Packet Encap

Match operations •  Manual entries – e.g

Firewall filters, policers •  Programmed port/vlan-id

entries


Route lookup



Ingress Egress

Programming features



Packet Decap

IFL Lookup

Internal next_hop

Process next_hop



Packet Encap

Set operations •  Manual entries – e.g

Firewall filters, policy •  Programmed actions


Route lookup



Ingress Egress



Packet Decap

IFL Lookup

Internal next_hop

Process next_hop



Packet Encap

Set operations •  Manual entries – queuing,

shaping, policing •  Programmed actions


Route lookup



Ingress Egress


PROGRAMMING ENTITY / ROUTER INTERACTION

Programming Entity

Router responds to Entity with •  Port state •  V4 / V6 / MAC address / port resolution •  RIB & Label Table •  Programming support (match/action) •  Resource arbitration •  Counter reporting

Match operations: Ingress Port Source Mac Vlan ID Vlan Pri IPv4/v6 Src IPv4/v6 Dest MPLS Label IP Proto v4 DSCP /v6 Flow-label / MPLS EXP src / dest port

Action operations: Set next_hop Forward via port / Vlan ID MPLS impose/pop/swap Set src / dest address Set .1p bits Set src / dest port Set v4 DSCP / v6 Flow-label / MPLS EXP Forward via FIB match Drop


FORWARDING ZONES

  With PF-capable devices using a common API, we should be able to have multiple programming entities sharing the same Layer-3 devices enabling ‘forwarding zones’ on a device

  Layer-3 device could have   IGP/BGP zone (default)   OpenFlow zone   PCE/LSP zone   ALTO zone

  Only one zone permitted per logical port with ability to ‘drop through; to default zone

  Arbitration function necessary to ensure clean resource split – no deadlock states permitted


FORWARDING FLOW DIAGRAM

Packet received

PF enabled

on port?

Forward via IGP entry

Match PF

entry?

Modify & forward as

per PF entry

Fall through to IGP?

Forward via IGP entry

Drop Yes

No Yes

No

Yes

No

Programmed forwarding (PF) entries pushed into RIB/FIB Forwarding chain ‘check’ if programmed entry should be applied


USE CASE: OVERLAY NETWORK

PF

PF

PF

PF PF

PF

PF

Programming Entity

Mesh of tunnels (MPLS, GRE) Discontigous PF deployment •  PF-capable Router

programmed with entry •  Non-PF capable routers

forward traffic as normal •  Programming entity may have

a view of paths through the network from IGP (not a participant in the IGP) – not a requirement

IGP

PF

PF


USE CASE: INTEGRATED APPROACH FEATURE PROGRAMMING

Packet Decap

IFL Lookup

Internal next_hop

Process next_hop



Packet Encap

OpenFlow

Route lookup



Ingress Egress

Default Control Plane

Application


USE CASE: FLOW & NATIVE CP INTEGRATION INJECT PROGRAMMED STATE INTO THE NETWORK

Router

Control Plane

OpenFlow

Controller

1

2 LDP: Advertise Label

RSVP: Advertise Label

BGP: Advertise Prefix

Utilize network protocols to distribute state (which would otherwise have to be programmed into every node


FORWARDING IN OPENFLOW

  Openflow 1.0 architecture aimed at Layer2 Ethernet environments

  OF Controller provides the ‘brains’ to an OF Switch

  Switches are ‘dumb’ – require the Controller to determine what to do with an unknown packet, or the Controller to define actions to the performed by the switch when a packet is matched   No communication of state between switches   No communication of state between controllers

  Requires the controller to have a view across the entire switching domain to create end-to-end switching path

  Coordinated programming necessary if passing between switching domains


TOPOLOGY DISCOVERY

OF

OF

OF

OF OF

OF

OF

Controller

Layer-2 networks are often complex in their own right OF 1.0 controller must understand connectivity between OF switches


TOPOLOGY DISCOVERY

OF

OF

OF

OF OF

OF

OF

Controller

•  In a Layer 3 network, we don’t typically examine how traffic from one node gets to another, as long as it arrives (except in specific instances)

•  OF Controller listens to the control-plane to learn topology – not an active participant

Control-plane data


OPENFLOW 1.0 VS TRADITIONAL ROUTER PACKET OPERATIONS

OF 1.0 ‘modify’ operations Set Vlan ID Set .1q priority Modify src/dest MAC Modify src/dest v4 addr Modify v4 TOS Modify src/dest port

Routing ‘modify’ operations Decap/encap L2 headers TTL/Hop-limit decrement Fragmentation handling Protocol Operations eg – MPLS Push/Pop/Swap, set Next_hop etc. QoS marking L4 modifications

✗ ✗

✓

✓ ✓ ✓

OF 1.0 ‘modify’ functions have some of the functionality which would be required for traditional L3 routing – extensions would need to be implemented for support of protocols such as IPv6 and MPLS


CONTROL PLANE FOR L3 OPERATION

  Typical Layer-3 networks reply on distributed-state model

  Dynamic topology discovery handled by routing protocols

  Each participating devices responsible for computing its own results (RIB, FIB etc)

  Overlay control-plane functions such as label-distribution rely on underlying routing protocols

  Static configuration can be achieved (static routes, explicit path definitions, static label bindings) but are cumbersome at scale, requiring automated systems for management

Technology

Hybrid Programmable Forwarding Planes: BoF Session