Using Agilio SmartNICs for OpenStack Networking Acceleration

© 2016 NETRONOME

Johann Tönsing August 9, 2016

Using Agilio™ SmartNICs for OpenStack Networking Acceleration

© 2016 NETRONOME 2

Agenda

• Overview of Traditional OpenStack Networking Options OVS (with/without connection tracking), Contrail vRouter, SR-IOV, VirtIO… • Issues: Flexibility, Performance, CPU Utilization

• Accelerating and Offloading OpenStack Networking using Agilio™ SmartNICs • Results: Features, Throughput, CPU Utilization • Implementation Details: Hardware and Software Architectures

• Evolution of OpenStack Networking • Supporting P4, C and eBPF Programmability • Integration Activities

• Conclusions

© 2016 NETRONOME

Traditional OpenStack Networking Options

3

Forwarding /Virtual Switching Technology

OpenStack Control Plane

Forwarding Overlays (Tunnels) Security Groups (Microsegmentation, Stateful Firewalling)

SR-IOV with regular NIC

Vendor specific plugin Limited E.g. MAC/VLANbased directing

None None

© 2016 NETRONOME


3






None None

OVS

ML2 plugin (optional: SDN controller) OVN plugin

L2 / L3 VXLAN, GRE, (for OVN) GENEVE…Emerging: NSH (for NFV)

OVS 2.5+ can interface to Linux conntrack(Previously Linux bridge with iptables/nftables)

© 2016 NETRONOME


3






None None

OVS




Contrail vRouterContrail plugin (via Contrail controller)

L2 / L3 MPLS in UDP, MPLS in GRE,VXLAN…

Built in, via flow table

© 2016 NETRONOME


3






None None

OVS




Contrail vRouterContrail plugin (via Contrail controller)

L2 / L3 MPLS in UDP, MPLS in GRE,VXLAN…

Built in, via flow table

Additional features: load balancing, NAT etc.

© 2016 NETRONOME

OpenStack Networking Options Evaluated

4


Traditional Approach Agilio™ SmartNIC Accelerated Approach

SR-IOV

Limited expressiveness to direct traffic to VMs (no support for general match/action rules, tunnel termination, stateful firewalling) High throughputNo VM migration support

Full OVS or vRouter virtual switching incl. tunnel termination, stateless/stateful firewalling

and SR-IOV based data delivery to VMs High throughputVirtIO integration, supporting VM migration

OVSandContrail vRouter

High expressiveness - match/action, tunnels, stateless/stateful firewalling etc. Limited throughput High CPU utilization (e.g. 50% of cores)

Same expressiveness - match/action, tunnels, stateless/stateful firewalling etc.

Higher throughput (~5x higher) Lower CPU utilization (~10x lower)

© 2016 NETRONOME 5

Agilio™ CX SmartNIC Family

• Optimized for standard server based cloud data centers • Low Profile Half Length PCIe form factor, power < 25W • Based on Netronome’s NFP-4xxx silicon (72 C programmable cores, 8 threads each) • 2GB DRAM for lookup tables / state tables (millions of entries) • Dataplane fully implemented in software

1x 40GbE 2x 40GbE2x 10GbE New: 2x 25GbE

Also available: Agilio™ LX 2x40G / 1x100G with dual PCIe interfaces, 120 cores, 8GB DRAM…

© 2016 NETRONOME 6

OVS Throughput vs. Packet Size

OVS L2 Forward to VMs

Packet Size

Mill

ions

of P

acke

ts p

er S

econ

d

OVS VXLAN + L2 Forward to VMs

Packet Size

Mill

ions

of P

acke

ts p

er S

econ

d

© 2016 NETRONOME 7

OVS Throughput vs. Number of Rules

5

10

15

20

25

30

OVS in Kernel Space

OVS in User Space on DPDK

100 Wildcard Rules

1000 Wildcard Rules

10000 Wildcard Rules

64000 Wildcard Rules

Mill

ions

of P

acke

ts p

er S

econ

d

12 CPU Cores

12 CPU Cores

OVS Offloaded to Agilio™ CX-4000

1 CPU Core

5X Throughput Improvement + 90% CPU Savings

OVS L2/L3 Forwarding to 8 VMs with 64K Flows

© 2016 NETRONOME 8

Tested Scenario: Server CPU Core Allocation

Unaccelerated OVS (Kernel / User Mode)

Agilio™ OVSOVS

© 2016 NETRONOME 9

Efficiency: Throughput Per Server CPU Core

Throughput with single server CPU corededicated to network related processing

Mill

ions

of P

acke

ts p

er S

econ

d • 50x Efficiency vs. Kernel OVS

• 20x Efficiency vs. User Mode (DPDK) OVS=> Replace 3-6 racks with 1 rack!

Throughput / efficiency similar for Agilio™ Contrail vRouter

Benefit for your use case: search for

“netronome.com roi calculator”

© 2016 NETRONOME

Offload Model: Agilio™ OVS Acceleration

10

Open vSwitch Subsystem

OVS Agent

OpenFlow

Virtual MachineVirtual Machine

Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

OVS CLI Callable API

1 Configuration via controller, CLI, or Callable API

(Nova, Neutron)

Execute Action

OVSKernel DP Match/Act


© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



(Nova, Neutron)

Execute Action

Open vSwitch Datapath

Execute Action (e.g. Entunnel, Deliver to VM, Send to Port)



© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



(Nova, Neutron)

Execute Action





Miss

© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



(Nova, Neutron)

Execute Action





Miss

Miss

© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



2

2 OVS userspace agent populates kernel cache

(Nova, Neutron)

Execute Action





Miss

Miss

© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



2


(Nova, Neutron)

Execute Action



3 Offload datapath: copy match tables, sync stats

3



Miss

Miss

© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



2


(Nova, Neutron)

Execute Action




3

4 Flow tracking: per-microflow state learning

4

Self Learning Exact MatchFlow Tracker

Miss

Hit



Miss

Miss

© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



2


(Nova, Neutron)

Execute Action




3

Conn track

FTPSIP


4


Miss

Hit



Miss

Miss

© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



2


(Nova, Neutron)

Execute Action



Conn track


3

Conn track

FTPSIP


4


Miss

Hit



Miss

Miss

© 2016 NETRONOME


10


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



2


(Nova, Neutron)

Execute Action



Conn track


3

5 Offload connection tracking: synchronize state

5

Conn track

FTPSIP


4


Miss

Hit



Miss

Miss

© 2016 NETRONOME

Agilio™ vRouter Acceleration

11

vRouter Subsystem

vRouter Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

vRouter CLI

1 Configuration via controller or CLI: BGP over XMPP

(Nova, Neutron)

Execute Action

vRouter Flow

Classify

vRouter ForwardingvRouter

Forwarding

© 2016 NETRONOME

vRouter Datapath

vRouter Flow

Classify



Forwarding


11

vRouter Subsystem

vRouter Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

vRouter CLI


(Nova, Neutron)

Execute Action

vRouter Flow

Classify


Forwarding

© 2016 NETRONOME

vRouter Datapath

vRouter Flow

Classify



Forwarding


11

vRouter Subsystem

vRouter Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

vRouter CLI


(Nova, Neutron)

Execute Action

vRouter Flow

Classify


Forwarding

2

2 Offload forwarding: pre-emptively copied

© 2016 NETRONOME

vRouter Datapath

vRouter Flow

Classify



Forwarding


11

vRouter Subsystem

vRouter Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

vRouter CLI


(Nova, Neutron)

Execute Action

vRouter Flow

Classify

Miss


Forwarding

2


© 2016 NETRONOME

vRouter Datapath

vRouter Flow

Classify



Forwarding


11

vRouter Subsystem

vRouter Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

vRouter CLI


(Nova, Neutron)

Execute Action

vRouter Flow

Classify

Miss

Miss


Forwarding

2


© 2016 NETRONOME

vRouter Datapath

vRouter Flow

Classify



Forwarding


11

vRouter Subsystem

vRouter Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

vRouter CLI


3

3 Userspace agent populates kernel tables

(Nova, Neutron)

Execute Action

vRouter Flow

Classify

Miss

Miss


Forwarding

2


© 2016 NETRONOME

vRouter Datapath

vRouter Flow

Classify



Forwarding


11

vRouter Subsystem

vRouter Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

vRouter CLI


3


(Nova, Neutron)

Execute Action

4 Offload flows: copy tables, sync stats

4

vRouter Flow

Classify

Miss

Miss


Forwarding

2


© 2016 NETRONOME

vRouter Datapath

vRouter Flow

Classify



Forwarding


11

vRouter Subsystem

vRouter Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

vRouter CLI


3


(Nova, Neutron)

Execute Action

4 Offload flows: copy tables, sync stats

4

vRouter Flow

Classify

Miss

Miss


Forwarding

Notes: - Control protocol: BGP over XMPP - Tunnel protocols: VXLAN, MPLS over UDP / GRE - Distinct policy (flows - ACLs) and forwarding (IP/MPLS) - Flow table is used to implement stateful firewalling

2


© 2016 NETRONOME

SmartNIC Firmware: Pre-programmed or Custom

• SmartNIC with dynamically downloadable firmware

© 2016 NETRONOME



Contrail OVS

OpenStack ONOS ODL

Linux BSD

• OVS / Contrail / Linux eBPF datapath on host can be accelerated by SmartNICD

P D K

eBPF

© 2016 NETRONOME


Compiler Debugger

Run-Time

app.P4 app.C

Editor

• Firmware can be developed in P4 and/or C • TBD how to integrate custom programmed

datapaths (implemented in P4 / C etc.) into OpenStack


Contrail OVS

OpenStack ONOS ODL

Linux BSD

• OVS / Contrail / Linux eBPF datapath on host can be accelerated by SmartNICD

P D K

eBPF

© 2016 NETRONOME 13

SmartNIC Datapath “Worker” Software Architecture

• Load balancer distributes each packet to next available thread for optimum throughput • Hardware assisted reordering ensures packet order is maintained • Flow tracker statefully learns / tracks millions of sessions • Matching performed using DRAM-backed tables - capacity > 500k entries • Actions efficiently performed in on-chip memory

Parse Match ActFlow Tracker Learn microflows

Cache action

Datapath Run to completion

Load Balance

C Plugin

Re- order

Pool of worker threads on flow processing cores

net or PCIE

net or PCIE

= Ring / Work Queue (multi producer / consumer)


Example: P4 “main” implementing a simple NIC

header_type eth_hdr { fields { dst : 48; src : 48; etype : 16; } } header eth_hdr eth;

parser start { return eth_parse; }

parser eth_parse { extract(eth); return ingress; }

action drop_act() { drop(); }

action fwd_act(port) { modify_field(standard_metadata.egress_spec, port); }

table in_tbl { reads { standard_metadata.ingress_port : exact; } actions { fwd_act; drop_act; } }

control ingress { apply(in_tbl); }

© 2016 NETRONOME

SmartNIC Firmware (P4/C)

Example of Fully Customized Datapath (P4 / C)

15

Run-Time Interface

Server (x86 - Linux)

PCIe

Agilio™ SmartNIC

Virtual Machine 1

VNF Kernel Mode

(C)

netdev

P4 / C Development Environment

Edit - Debug

Control App

Populate tables, display

statistics

Security µVNF (C)

Timestamp µVNF (C)

Latency Stats µVNF (C)

Virtual Machine 2

VNF User Mode

(C)

DPDK

Timestamp µVNF (C)


Match Protocol

Meter

Other

TCP

Concepts:

• P4 and C running on SmartNIC implements datapath - e.g. defines protocols, match / actionbehavior

• Datapath steers traffic to VNFsrunning on x86 server and on SmartNIC

© 2016 NETRONOME

SmartNIC Firmware (P4/C)

Example of Fully Customized Datapath (P4 / C)

15

Run-Time Interface

Server (x86 - Linux)

PCIe

Agilio™ SmartNIC

Virtual Machine 1

VNF Kernel Mode

(C)

netdev

P4 / C Development Environment

Edit - Debug

Control App

Populate tables, display

statistics

Security µVNF (C)

Timestamp µVNF (C)


Virtual Machine 2

VNF User Mode

(C)

DPDK

Timestamp µVNF (C)


Match Protocol

Meter

Other

TCP

Concepts:

• P4 and C running on SmartNIC implements datapath - e.g. defines protocols, match / actionbehavior

• Datapath steers traffic to VNFsrunning on x86 server and on SmartNIC

To Discuss: OpenStackIntegration

© 2016 NETRONOME

Extending OpenStack Networking - P4/C Plugins

16


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



2


(Nova, Neutron)

Execute Action




3



Miss

Miss

© 2016 NETRONOME

Extending OpenStack Networking - P4/C Plugins

16


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK



2


(Nova, Neutron)

Execute Action




3


Datapath Extension or Plugin

P4 / C in Sandbox

DP Ext.

4 Datapath extension software

4

4


Miss

Miss

© 2016 NETRONOME

OVS “on” SmartNIC P4 Datapath

17


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

OVS CLI Callable API (Nova, Neutron)

Execute Action


P4 Matching

© 2016 NETRONOME


17


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK


Execute Action

P4 Generated Datapath

Execute P4

Action


P4 Matching

© 2016 NETRONOME


17


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK


Execute Action


Execute P4

Action


P4 Matching

Fallback

© 2016 NETRONOME


17


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK


Execute Action


Execute P4

Action


P4 Matching

Fallback

Fallback

© 2016 NETRONOME

P4 “into” OVS Datapath

18


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK


Execute P4 / OVS Action

P4 / OVS Matching

P4 / OVS Matching

© 2016 NETRONOME


18


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK





P4 / OVS Matching

P4 / OVS Matching

© 2016 NETRONOME


18


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK





P4 / OVS Matching

P4 / OVS Matching

Fallback

© 2016 NETRONOME


18


OVS Agent

OpenFlow


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK





P4 / OVS Matching

P4 / OVS Matching

Fallback

Fallback

© 2016 NETRONOME

P4 “instead of” OVS (Datapath and Control)

19


P4 Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

(Nova, Neutron)


Execute P4

Action

Execute Action

P4 or eBPF or XDP

P4 Matching

Fallback

Fallback

1 Protocol(s) to be discussed (could become callable API)

© 2016 NETRONOME

P4 “instead of” OVS (Datapath and Control)

19


P4 Agent


Virtual Machine

x86 Kernel

x86 Userspace

PCIe

Virtual Machine

SR-IOV / VirtIO VFs

SR-IOV / VirtIO VFs

Agilio™SmartNIC

Apps

Apps

1

netdev or DPDK

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

Apps

netdev or DPDK

(Nova, Neutron)


Execute P4

Action

Execute Action

P4 or eBPF or XDP

P4 Matching

Fallback

Fallback

1 Protocol(s) to be discussed (could become callable API)

Other open issues: - Downloading programs via OpenStack or other systems- Scheduling VMs to run on nodes with acceleration hardware (Nova)Acceleration discussion started at OpenStack Spring 2016 (Austin)


Integration / Open Sourcing Activities

Area ActivitiesLinux drivers VF netdev (kernel device

driver) upstreamed in kernel 4.5

Patches available for older kernels, or use VirtIO

Representative netdev (for fallback processing of traffic) proposals imminent

FreeBSD drivers Kernel device driver implemented

DPDK drivers Poll mode driver upstreamed in DPDK 2.2

Patches available for older DPDK versions

Open vSwitch accelerationintegration

Hooks for acceleration proposed in October 2014

Iteration in progress

OpenStack integration

Proposing plugins and agents to support virtual switching acceleration via RFE process

Integration for OVS in process - Mirantis, Ericsson CEE etc.

Integration for Contrail vRouter in process - Juniper etc.


Integration / Open Sourcing Activities

Area ActivitiesLinux drivers VF netdev (kernel device

driver) upstreamed in kernel 4.5

Patches available for older kernels, or use VirtIO

Representative netdev (for fallback processing of traffic) proposals imminent

FreeBSD drivers Kernel device driver implemented

DPDK drivers Poll mode driver upstreamed in DPDK 2.2

Patches available for older DPDK versions

Open vSwitch accelerationintegration

Hooks for acceleration proposed in October 2014

Iteration in progress

OpenStack integration

Proposing plugins and agents to support virtual switching acceleration via RFE process

Integration for OVS in process - Mirantis, Ericsson CEE etc.

Integration for Contrail vRouter in process - Juniper etc.

Also: P4 and protocol independent forwarding, switchdev, eBPF / XDP…


Next Steps

• Use Agilio™ SmartNICs with existing dataplanes • Use Agilio™ OVS (with / without Conntrack) • Use Agilio™ vRouter • Future: Agilio™ eBPF/XDP, etc.

• Program Agilio™ SmartNICs • Use APIs (on x86 servers) - with above dataplanes • Program in P4 and/or C (on SmartNIC / on x86)

• Improve performance + free up server resources!

© 2016 NETRONOME

More information: netronome.com and open-nfp.org

Thank You!

22

http://netronome.com

http://open-nfp.org