The path to data plane MicroservicesRay KinsellaFebruary 2018Email raykinsella [at] intelcomIRC mortderire

Contributions frombull Kuralamudhan Ramakrishnan (Intel) bull John DiGiglio (Intel)bull Dana Nehama (Intel)

FOSDEMrsquo18 2

Legal Disclaimer

General Disclaimer

copy Copyright 2018 Intel Corporation All rights reserved Intel the Intel logo Intel Inside the Intel Inside logo Intel Experience Whatrsquos Inside are trademarks of Intel Corporation in the US andor other countries Other names and brands may be claimed as the property of othersTechnology DisclaimerIntel technologiesrsquo features and benefits depend on system configuration and may require enabled hardware software or service activation Performance varies depending on system configuration No computer system can be absolutely secure Check with your system manufacturer or retailer or learn more at [intelcom] Performance DisclaimersCost reduction scenarios described are intended as examples of how a given Intel- based product in the specified circumstances and configurations may affect future costs and provide cost savings Circumstances will vary Intel does not guarantee any costs or cost reductionResults have been estimated or simulated using internal Intel analysis or architecture simulation or modeling and provided to you for informational purposes Any differences in your system hardware software or configuration may affect your actual performance

FOSDEMrsquo18 3

Agenda

bull Network Function Evolution

bull Containerized Network Functions

bull Cloud Native Network Functions

bull Summary

FOSDEMrsquo18 4

Network Appliances

2012+2000s+ 2017+

Monolith

Industry packaging

Software Architecture

2020+

Software DefinedNetwork Functions

Cloud NativeNetwork Functions

ContainerizedNetwork Functions

VirtualizedNetwork Functions

Network Function Evolution

Network Function Evolution

FOSDEMrsquo18 2

Legal Disclaimer

General Disclaimer

copy Copyright 2018 Intel Corporation All rights reserved Intel the Intel logo Intel Inside the Intel Inside logo Intel Experience Whatrsquos Inside are trademarks of Intel Corporation in the US andor other countries Other names and brands may be claimed as the property of othersTechnology DisclaimerIntel technologiesrsquo features and benefits depend on system configuration and may require enabled hardware software or service activation Performance varies depending on system configuration No computer system can be absolutely secure Check with your system manufacturer or retailer or learn more at [intelcom] Performance DisclaimersCost reduction scenarios described are intended as examples of how a given Intel- based product in the specified circumstances and configurations may affect future costs and provide cost savings Circumstances will vary Intel does not guarantee any costs or cost reductionResults have been estimated or simulated using internal Intel analysis or architecture simulation or modeling and provided to you for informational purposes Any differences in your system hardware software or configuration may affect your actual performance

FOSDEMrsquo18 3

Agenda

bull Network Function Evolution

bull Containerized Network Functions

bull Cloud Native Network Functions

bull Summary

FOSDEMrsquo18 4

Network Appliances

2012+2000s+ 2017+

Monolith

Industry packaging

Software Architecture

2020+

Software DefinedNetwork Functions

Cloud NativeNetwork Functions

ContainerizedNetwork Functions

VirtualizedNetwork Functions

Network Function Evolution

Network Function Evolution

FOSDEMrsquo18 3

Agenda

bull Network Function Evolution

bull Containerized Network Functions

bull Cloud Native Network Functions

bull Summary

FOSDEMrsquo18 4

Network Appliances

2012+2000s+ 2017+

Monolith

Industry packaging

Software Architecture

2020+

Software DefinedNetwork Functions

Cloud NativeNetwork Functions

ContainerizedNetwork Functions

VirtualizedNetwork Functions

Network Function Evolution

Network Function Evolution

FOSDEMrsquo18 4

Network Appliances

2012+2000s+ 2017+

Monolith

Industry packaging

Software Architecture

2020+

Software DefinedNetwork Functions

Cloud NativeNetwork Functions

ContainerizedNetwork Functions

VirtualizedNetwork Functions

Network Function Evolution

Network Function Evolution

FOSDEMrsquo18 5

What is driving Containers

Network Function Evolution

FOSDEMrsquo18 6

Network Function Evolution

https12factornetcopy Adam Wiggins 2017

Application Decomposition

The 12 Factor APP

FOSDEMrsquo18 6

Network Function Evolution

https12factornetcopy Adam Wiggins 2017

Application Decomposition

The 12 Factor APP

FOSDEMrsquo18 7

Compute

Core 0

Linux Scheduler

Core 1

NoisyAppCPU Manager

for Kubernetes

MemoryContainer

Kubernetes

Core 0

Linux Scheduler

Core 1

NoisyApp

HugepageEnhancementsfor Kubernetes

Core Isolation

Alsohellip bull Platform (node) feature

discoverybull Platform telemetrybull and more

Containerized Network Functions

FOSDEMrsquo18 8

Container IO

SR-IOV

What is itVirtual Function Passthroughfor Container

bull High Throughputbull Low Density

Virtio-User

What is itContainer Virtual Interface for Bare-metal deployments

bull Supportsbull Virtual Network Functions

bull Good Throughputbull Good Density

Master-VM

What is itContainer Virtual Interface for Virtualized deployments

bull Supportsbull Virtual Network Functionsbull Socket Based Applications

bull Good Throughputbull Good Density

Container

App

DPDK vSwitch

DPDK

Container

DPDK

App

Container

App

DPDK vSwitch

DPDK

Container

App

Virtual Machine

SocketsContainer

App

DPDK

ContainerApp

Sockets

Kubernetes

Multus

What is itCNI provider for Kubernetes to enable multiple interfaces per Container

bull Functional separation of control and data network planes

bull Link aggregation bull Network segregation

ContainerApp

DPDK

Kubernetes

Containerized Network Functions

FOSDEMrsquo18 9

Cloud Native Network Functions

ContainerEnabling

CPU Sharing IO Sharing

MicroservicesEnabling

Resource sharing API

Data plane Microservices

Memory Sharing

Same APP same DPDK API(s) deployment specific behaviour

Cloud Native Network Functions

FOSDEMrsquo18

Core Core

Service1

Service3

Service4

Service2

DPDK Scheduler

Core Core

Service1

Service3

Service4

Service2

Linux Scheduler

Core Core

Polling Polling

Compute Lightweight scheduling models

Multi-Process SchedulerIn-Process SchedulerMonolithic

Memory Lightweight memory models

Monolithic Dynamic Allocation 4K page allocation

Cloud Native Network Functions

FOSDEMrsquo18

Cloud Native IO

App1

Core Core

Polling Polling

App2

Accelerated Virtual SwitchScalable Virtual SwitchCurrent

Scalable and accelerated Virtual Switching

Cloud Native Network Functions

FOSDEMrsquo18

12

Data plane microservice models

ModelIn-process

MicroservicesMulti-process Microservices

Multi-nodeMicroservices

Why Highest Performance Multi-process scaling Multi-node scaling

Scheduling DPDK Scheduler Cooperative OS Cooperative OS

Memory Monolithic Dynamic Dynamic

Transport Mem RingMem Ring vSwitch HW

acceleratedRPC a over reliable protocol

High Availability No Yes Yes

Live Migration No No Yes

microserviceData plane microservice evolution

Cloud Native Network Functions

FOSDEMrsquo18 13

Summary

Containerized Network Functions are here

bull Support available on Intelrsquos github ( githubcomIntel-Corp ) and upstream in Kubernetes

bull See our ldquoExperience Kitrdquo ( intelly2rXMswf ) for related deployment guidelines source code app notes etc

Cloud Native Network Functions are coming

bull Emerging support in DPDK and FDio eg new memory model targeted for DPDK 1802 release

E-mail raykinsella [at] intelcom IRC mortderire

FOSDEMrsquo18 8

Container IO

SR-IOV

What is itVirtual Function Passthroughfor Container

bull High Throughputbull Low Density

Virtio-User

What is itContainer Virtual Interface for Bare-metal deployments

bull Supportsbull Virtual Network Functions

bull Good Throughputbull Good Density

Master-VM

What is itContainer Virtual Interface for Virtualized deployments

bull Supportsbull Virtual Network Functionsbull Socket Based Applications

bull Good Throughputbull Good Density

Container

App

DPDK vSwitch

DPDK

Container

DPDK

App

Container

App

DPDK vSwitch

DPDK

Container

App

Virtual Machine

SocketsContainer

App

DPDK

ContainerApp

Sockets

Kubernetes

Multus

What is itCNI provider for Kubernetes to enable multiple interfaces per Container

bull Functional separation of control and data network planes

bull Link aggregation bull Network segregation

ContainerApp

DPDK

Kubernetes

Containerized Network Functions

FOSDEMrsquo18 9

Cloud Native Network Functions

ContainerEnabling

CPU Sharing IO Sharing

MicroservicesEnabling

Resource sharing API

Data plane Microservices

Memory Sharing

Same APP same DPDK API(s) deployment specific behaviour

Cloud Native Network Functions

FOSDEMrsquo18

Core Core

Service1

Service3

Service4

Service2

DPDK Scheduler

Core Core

Service1

Service3

Service4

Service2

Linux Scheduler

Core Core

Polling Polling

Compute Lightweight scheduling models

Multi-Process SchedulerIn-Process SchedulerMonolithic

Memory Lightweight memory models

Monolithic Dynamic Allocation 4K page allocation

Cloud Native Network Functions

FOSDEMrsquo18

Cloud Native IO

App1

Core Core

Polling Polling

App2

Accelerated Virtual SwitchScalable Virtual SwitchCurrent

Scalable and accelerated Virtual Switching

Cloud Native Network Functions

FOSDEMrsquo18

12

Data plane microservice models

ModelIn-process

MicroservicesMulti-process Microservices

Multi-nodeMicroservices

Why Highest Performance Multi-process scaling Multi-node scaling

Scheduling DPDK Scheduler Cooperative OS Cooperative OS

Memory Monolithic Dynamic Dynamic

Transport Mem RingMem Ring vSwitch HW

acceleratedRPC a over reliable protocol

High Availability No Yes Yes

Live Migration No No Yes

microserviceData plane microservice evolution

Cloud Native Network Functions

FOSDEMrsquo18 13

Summary

Containerized Network Functions are here

bull Support available on Intelrsquos github ( githubcomIntel-Corp ) and upstream in Kubernetes

bull See our ldquoExperience Kitrdquo ( intelly2rXMswf ) for related deployment guidelines source code app notes etc

Cloud Native Network Functions are coming

bull Emerging support in DPDK and FDio eg new memory model targeted for DPDK 1802 release

E-mail raykinsella [at] intelcom IRC mortderire

FOSDEMrsquo18 9

Cloud Native Network Functions

ContainerEnabling

CPU Sharing IO Sharing

MicroservicesEnabling

Resource sharing API

Data plane Microservices

Memory Sharing

Same APP same DPDK API(s) deployment specific behaviour

Cloud Native Network Functions

FOSDEMrsquo18

Core Core

Service1

Service3

Service4

Service2

DPDK Scheduler

Core Core

Service1

Service3

Service4

Service2

Linux Scheduler

Core Core

Polling Polling

Compute Lightweight scheduling models

Multi-Process SchedulerIn-Process SchedulerMonolithic

Memory Lightweight memory models

Monolithic Dynamic Allocation 4K page allocation

Cloud Native Network Functions

FOSDEMrsquo18

Cloud Native IO

App1

Core Core

Polling Polling

App2

Accelerated Virtual SwitchScalable Virtual SwitchCurrent

Scalable and accelerated Virtual Switching

Cloud Native Network Functions

FOSDEMrsquo18

12

Data plane microservice models

ModelIn-process

MicroservicesMulti-process Microservices

Multi-nodeMicroservices

Why Highest Performance Multi-process scaling Multi-node scaling

Scheduling DPDK Scheduler Cooperative OS Cooperative OS

Memory Monolithic Dynamic Dynamic

Transport Mem RingMem Ring vSwitch HW

acceleratedRPC a over reliable protocol

High Availability No Yes Yes

Live Migration No No Yes

microserviceData plane microservice evolution

Cloud Native Network Functions

FOSDEMrsquo18 13

Summary

Containerized Network Functions are here

bull Support available on Intelrsquos github ( githubcomIntel-Corp ) and upstream in Kubernetes

bull See our ldquoExperience Kitrdquo ( intelly2rXMswf ) for related deployment guidelines source code app notes etc

Cloud Native Network Functions are coming

bull Emerging support in DPDK and FDio eg new memory model targeted for DPDK 1802 release

E-mail raykinsella [at] intelcom IRC mortderire

FOSDEMrsquo18

Core Core

Service1

Service3

Service4

Service2

DPDK Scheduler

Core Core

Service1

Service3

Service4

Service2

Linux Scheduler

Core Core

Polling Polling

Compute Lightweight scheduling models

Multi-Process SchedulerIn-Process SchedulerMonolithic

Memory Lightweight memory models

Monolithic Dynamic Allocation 4K page allocation

Cloud Native Network Functions

FOSDEMrsquo18

Cloud Native IO

App1

Core Core

Polling Polling

App2

Accelerated Virtual SwitchScalable Virtual SwitchCurrent

Scalable and accelerated Virtual Switching

Cloud Native Network Functions

FOSDEMrsquo18

12

Data plane microservice models

ModelIn-process

MicroservicesMulti-process Microservices

Multi-nodeMicroservices

Why Highest Performance Multi-process scaling Multi-node scaling

Scheduling DPDK Scheduler Cooperative OS Cooperative OS

Memory Monolithic Dynamic Dynamic

Transport Mem RingMem Ring vSwitch HW

acceleratedRPC a over reliable protocol

High Availability No Yes Yes

Live Migration No No Yes

microserviceData plane microservice evolution

Cloud Native Network Functions

FOSDEMrsquo18 13

Summary

Containerized Network Functions are here

bull Support available on Intelrsquos github ( githubcomIntel-Corp ) and upstream in Kubernetes

bull See our ldquoExperience Kitrdquo ( intelly2rXMswf ) for related deployment guidelines source code app notes etc

Cloud Native Network Functions are coming

bull Emerging support in DPDK and FDio eg new memory model targeted for DPDK 1802 release

E-mail raykinsella [at] intelcom IRC mortderire

FOSDEMrsquo18

Cloud Native IO

App1

Core Core

Polling Polling

App2

Accelerated Virtual SwitchScalable Virtual SwitchCurrent

Scalable and accelerated Virtual Switching

Cloud Native Network Functions

FOSDEMrsquo18

12

Data plane microservice models

ModelIn-process

MicroservicesMulti-process Microservices

Multi-nodeMicroservices

Why Highest Performance Multi-process scaling Multi-node scaling

Scheduling DPDK Scheduler Cooperative OS Cooperative OS

Memory Monolithic Dynamic Dynamic

Transport Mem RingMem Ring vSwitch HW

acceleratedRPC a over reliable protocol

High Availability No Yes Yes

Live Migration No No Yes

microserviceData plane microservice evolution

Cloud Native Network Functions

FOSDEMrsquo18 13

Summary

Containerized Network Functions are here

bull Support available on Intelrsquos github ( githubcomIntel-Corp ) and upstream in Kubernetes

bull See our ldquoExperience Kitrdquo ( intelly2rXMswf ) for related deployment guidelines source code app notes etc

Cloud Native Network Functions are coming

bull Emerging support in DPDK and FDio eg new memory model targeted for DPDK 1802 release

E-mail raykinsella [at] intelcom IRC mortderire

FOSDEMrsquo18

12

Data plane microservice models

ModelIn-process

MicroservicesMulti-process Microservices

Multi-nodeMicroservices

Why Highest Performance Multi-process scaling Multi-node scaling

Scheduling DPDK Scheduler Cooperative OS Cooperative OS

Memory Monolithic Dynamic Dynamic

Transport Mem RingMem Ring vSwitch HW

acceleratedRPC a over reliable protocol

High Availability No Yes Yes

Live Migration No No Yes

microserviceData plane microservice evolution

Cloud Native Network Functions

FOSDEMrsquo18 13

Summary

Containerized Network Functions are here

bull Support available on Intelrsquos github ( githubcomIntel-Corp ) and upstream in Kubernetes

bull See our ldquoExperience Kitrdquo ( intelly2rXMswf ) for related deployment guidelines source code app notes etc

Cloud Native Network Functions are coming

bull Emerging support in DPDK and FDio eg new memory model targeted for DPDK 1802 release

E-mail raykinsella [at] intelcom IRC mortderire

FOSDEMrsquo18 13

Summary

Containerized Network Functions are here

bull Support available on Intelrsquos github ( githubcomIntel-Corp ) and upstream in Kubernetes

bull See our ldquoExperience Kitrdquo ( intelly2rXMswf ) for related deployment guidelines source code app notes etc

Cloud Native Network Functions are coming

bull Emerging support in DPDK and FDio eg new memory model targeted for DPDK 1802 release

E-mail raykinsella [at] intelcom IRC mortderire