BRKACI-2001 - Integration and Interoperation of Existing Nexus Networks Into an ACI Architecture (2015 Melbourne)

7/24/2019 BRKACI-2001 - Integration and Interoperation of Existing Nexus Networks Into an ACI Architecture (2015 Melbourne)

1/115


2/115

#clmel

Integration and Interoperation ofExisting Nexus Networks into an ACI

Architecture

BRKACI-2001

Mike Herbert Principal Engineer INSBU


3/115

2014 Cisco and/or its affiliates. All rights reserved.BRKACI-2001 Cisco Public

Application Oriented Policy = Operational Simplicity

Introducing: Application Centric Infrastructu

Apps + Infrastructure

Physical + VirtOpen + Secure

On-Premises + Cloud


4/115


Control & Audit Connectivity(Security Firewall, ACL, )

IP Address, VLAN, VRF

Enable Connectivity

(The Network)

Application Requirements

IP Addressing

Application Requirements

Application Specific C

Dynamic provis

connectivity explicitthe applica

Application Requir

Application Requir

Redirect and Load Balance

Connectivity

IP Address, VLAN, VRF

ACI directly maps the

connectivity requiremen

network and service

Why Networks are ComplexOverloaded Network Constructs


5/115 2014 Cisco and/or its affiliates. All rights reserved.BRKACI-2001 Cisco Public

Why Network Provisioning is SlowApplication Language Barriers

Developers

Application

Tiers

Provider /

Consumer

Relationships

Infrastru

VLANs

Subnets

Protocols

Ports

Developer and infrastructure teams must translate between disparate la



ADC APPF/WADC WEB

HYPERVISORHYPERVISOR HYPERVISOR

CONNECTIVIT

Y POLICY

SECURITY

POLICIESQOS

APPLICATIO

L4..7

SERVICES

SLA

QoS

Security

Load

Balancing

APP PROFILE

Centralised Policy and Distributed Enforcem



Two Big Questions

Is ACI a Closed System?

Do I need to replace all of my existing

infrastructure to begin leveraging ACI?

ABSOLUTELY NOT !!!

Lets see WHY and HOW



Extend ACI to localhypervisors

vSwitch

Things we would like to understand how to do

Extend AWAN/

Interconnect to existingDC Networks

Let me just runmy network (butfix my Flooding,

Mobility,Configuration,

Troubleshootingchallenges)

AV

AVS

Extend ACI to toinstallations via

Switching Enableremote ACI P


9/115

ACI Policy Based Forwarding via anIntegrated Overlay



ACI FabricAn IP network with an Integrated OverlVirtual and Physical

Ciscos ACI solution leverages an integrated VXLAN based overlay

IP Network for Transport

VXLAN based tunnel end points (VTEP)

VTEP discovery via infrastructure routing

Directory (Mapping) service for EID (host MAC and IP address) to VTEP lo

PayloadIPVXLANVTEP

VTEP VTEP VTEP VTEP VTEP VTEP

vSwitch

vSwitc

hVTEPVTEP

IP Transport


11/115


VXLAN EnabledHypervisor

VTEP

VXLAN EnabledHypervisor

VTEP

Interworking with ACI

VXLAN enabled Hypervisor (FCS)

VXLAN Hardware VTEP (Nexus 9000 standalone, Nexus 3100/7000-F3, ASR9K, )

MP-BGP EVPN based control plane for external VTEP connectivity (post FCS)

ServASR

Multi-Fab

VXLAN BasedFabric

ACI Fabric Integrated OverlayConnecting and Extending the Overlay

VTEP

VT


12/115


ACI Fabric Integrated OverlayDecoupled Identity, Location & Policy

VXLANOuter

IP

NVGR

E

Outer

IP

802.1Q

ACI VXLAN (VXLAN) h

attributes of the applica

the fab

Policy attributes are car

External Identifiers are localised to

specific iLeaf or iLeaf ports (unless

external requirements for

consistency, e.g. downstream

networks)

VTEP

All Tenant traffic within the Fabric is

tagged with an ACI VXLAN (VXLAN)

header which identifies the policy

attributes of the application end pointwithin the fabric

Policy Group (source group)

Forwarding Group (Tenant, VRF,

Bridge Domain)

Load Balancing Policy

Telemetry Policy

At the ingress port the Fabric translates

an external identifier which can be used

to distinguish different application end

points via the internal VXLAN tagging

format

VNIDFlagsVTEPSRC

Group

AVS

VTEP


13/115


ACI leverages VXLANRFC Draft Next IETF Meeting


14/115


Forward based on destination IP Address for intra and inter subnet (Default Mode

Bridge semantics are preserved for intra subnet traffic (no TTL decrement, n

header rewrite, etc.)

Non-IP packets will be forwarded using MAC address. Fabric will learn MAC

packets, IP address learning for all other packets

Route if MAC is router-mac, otherwise bridge (standard L2/L3 behaviour)

IP Forwarding:

Forwarded using DIPi

address, HW learning of IPaddress

10.1.3.11 10.6.3.210.1.3.35 10.6.3.17

MAC

Forwar

addressMA

Overlay Elements Host Forwarding


15/115


10.1.1.10 10.1.3.11

10.6.3.2

Distributed Default Gateway

ACI Fabric supports full layer 2 and layer 3 forwarding semantics, no changes required to applica

stacks

ACI Fabric provides optimal forwarding for layer 2 and layer 3

Fabric provides a pervasive SVI which allows for a distributed default gateway

Layer 2 and layer 3 traffic is directly forwarded to destination end point

IP ARP/GARP packets are forwarded directly to target end point address contained within ARP/G

(elimination of flooding)

10.1.3.35 10.1.1.1010.1.3.1

110.6.3.2

Directed ARP Forw

Location Independent ForwardingLayer 2 and Layer 3


16/115


10.1.3.1

1

10.6.3.2

Pervasive SVI

Default Gateway can reside internal or external to the Fabric

Pervasive SVI provides a distributed default gateway (anycast gateway)

Subnet default gateway addresses are programmed in all Leaves with end points present

IP subnet

Layer 2 and layer 3 traffic is directly forwarded to destination end point

External Gateway is used when Fabric is configured to provide layer 2 transport only for a specif

10.1.3.35 10.1.1.1010.1.3.1

110.6.3.2

External Default G

Where is my default Gateway?

10.6.3.2

10.6.3.110.1.3.1


17/115


Overlay Elements - Mapping/Directory ProxyInline Hardware Mapping DB - 1,000,000+ hosts

10.1.3.11 fe80::462a:60ff:fef7:8e5e10.1.3.35 fe80::62c5:47ff:fe

The Forwarding Table on the Leaf Switch is divided between local (directl

global entries

The Leaf global table is a cached portion of the full global table

If an endpoint is not found in the local cache the packet is forwarded to th

forwarding table in the spine switches (1,000,000+ entries in the spine for

Local Station Table

contains addresses of

all hosts attached

directly to the iLeaf

10.1.3.11

10.1.3.35

Port 9

Leaf 3

Proxy A*

Global Station Table

contains a local cache

of the fabric endpoints

P

add

Proxy Proxy Proxy Proxy


18/115


Endpoint Repository (Proxy DataBase)


19/115


Proxy Database (Oracle)

Spine-1# show coop internal info global Spine-1# show coop internal event-history

You still have full access to all forwarding, adjacency, ..., information

CLI and debug commands when you want them


20/115


ACI Endpoint Tracker Application

Tracks all attachment, detachment,movement of Endpoints in ACI fabric

Stores activity in open source MySQLDatabase, allowing query capabilities

Provides foundation for visualisation andquery tools

Some questions that could be solved:

What are all the Endpoints on network?

Where is a specific Endpoint?

What was connected last Thursdaybetween 3:30am and 4:00am?

What is the history of a given Endpoint?


21/115


GitHub a resource for ACI scripts and tool

21

ACI Toolkit:http://datacenter.gith

https://github.com/da

ACI Diagramhttps://github.com/cg

ACI Endpoint Trhttp://datacenter.githuild/html/endpointtra
http://datacenter.github.io/acitoolkit/https://github.com/datacenter/acitoolkithttps://github.com/cgascoig/aci-diagramhttps://github.com/cgascoig/aci-diagramhttp://datacenter.github.io/acitoolkit/docsbuild/html/endpointtracker.htmlhttp://datacenter.github.io/acitoolkit/docsbuild/html/endpointtracker.htmlhttp://datacenter.github.io/acitoolkit/docsbuild/html/endpointtracker.htmlhttps://github.com/cgascoig/aci-diagramhttps://github.com/datacenter/acitoolkithttp://datacenter.github.io/acitoolkit/


22/115


ACIA Policy Based IP Network

PayloadIPVXLANVTEP

AVS

VTEP

A

D

I

Physical and Virtual L4-

Service Nodes

Physical and Virtual VTEPs

(Policy & Forwarding EdgeNodes)

Proxy (Directory)

Services

Physical and Virtual Endpoints

(Servers) & VMM (Hypervisor vSwitch)

VTEP

IP Network & Integrated

VXLAN

VTEP

VTEP

AVS

VTEP


23/115

Extending the Network

Transitions will and need to occur independently


24/115


Transitions will and need to occur independently

vPC

vPC

vPC

Policy Zone A Policy Zone B

Policy Zone C

EApp

ElementApp

ElementApp

ElemeA

EleE

Application Container

Operations

Evolution

Policy

Evolution

Component

Evolution


25/115


Interconnect existing network PODs with new ACI PODs vi

Layer 2 extensions (VLAN or VXLAN) or via standard Layer(OSPF, BGP)

Leverage an ACI policy/services block attached to any existCatalyst network to provide L4-L7 services and policy automexisting virtual and physical servers

Extend the ACI forwarding and distributed policy capabilitiesand physical leaf switches connected to an existing Nexus/Cbased network

Extending ACI in to Current Data CentresThree Stages

Extending ACI in to Current Data Centres


26/115


Backbone

vPC

vPC

vPC

Layer 2 and Layer 3 interoperation between ACI Fabric and Existing Data Centre builds

Layer 3 interconnect via standard routing interfaces,

OSPF, Static, iBGP (FCS)MP-BGP, EIGRP, ISIS (Post FCS_

Layer 2 interconnect via standard STP or via VXLAN overlays

vSwitch

Extending ACI in to Current Data Centre sInterconnect Existing to New ACI POD

Extend Layer 2 VLANs

where required

Interconnect at

Layer 3


27/115

2014 Cisco and/or its affiliates. All rights reserved.TECACI-2009 Cisco Public

Connecting/Extending ACI via Layer 2

Layer 2

Layer 2

Layer 2

Extend L2 domain beyond ACI fabric - 2 options

1. Manually assign a port to a VLAN which in turn mapped to an EPG. This extend EPG beyo

(EPG == VLAN)

2. Create a L2 connection to outside network. Extend bridge domain beyond ACI fabric. Allo

between EPG inside ACI and EPG outside of ACI

Lets Look at

the Links

ACI Fabric Integrated Overlay


28/115


VXLAN

VNID = 5789VXLAN

VNID = 11348

NVGRE

VSID = 7456

Any to Any

802.1Q

VLAN 50

Normalised

Encapsulation

Localised

Encapsulation

IP Fabric Us

VXLAN Tagg

VXLANVTEP

All traffic within the ACI Fabric is encapsulated with an extended VXLAN header

External VLAN, VXLAN, NVGRE tags are mapped at ingress to an internal VXLAN tag

Forwarding is not limited to, nor constrained within, the encapsulation type or

encapsulation overlay network

External identifies are localised to the Leaf or Leaf port, allowing re-use and/or translation

if required

VXLANOuter

IP

NVGREOuter

IP

802.1Q

Normalisat

Encap

ACI Fabric Integrated OverlayData Path - Encapsulation Normalisation

28

E t d th EPG


29/115


Extend the EPGOption 1

VLAN 30

Layer 2

100.1.1.3 100.1.1.5

EPG

100.1.1.7100.1.1.99

VLANs are localised to the leaf nodes

The same subnet, bridge domain, EPG can be configured as a different VLAN o

switch

In 1HCY15 VLANs will be port local

100.1.1.3

BDExisting

App

VL

Extend the EPG


30/115


Extend the EPGOption 1

Layer 2

VLAN 10

100.1.1.3 100.1.1.5

EPG

100.1.1.7100.1.1.99

Single Policy Group (one extended EPG)

Leverage vPC for interconnect (diagram shows a single port-channel which

BPDU should be enabled on the interconnect ports on the vPC domain

100.1.1.3

VLAN 30

VL

BExis

A

VLAN 10 VLAN 10 VLAN 10


31/115


Assign Port to an EPG With VMM integration, port is assigned to EPG by

APIC dynamically.

In all other cases, such as connecting to switch,

router, bare metal, port need to be assigned toEPG manually or use API

Use Static Binding undport to EPG

The example assigns tra

eth1/32 with vlan tagging100

Assign Port to EPG


32/115


Assign Port to EPGVLAN Tagging Mode

Tagged. Trunk mode

Untagged. Access mode. Port can only be in one

EPG 802.1P Tag. Native VLAN.

No Tagged and Untagged(for different port) config

for same EPG with current software

Assign port eth1/1 with VLAand port eth1/2 with VLAN 1EPG WEB is not supported

Use 802.1P Tag. Port eth1/eth1/2 vlan 100 902.1P Tag

VLAN to EPG mapping is sw

Extend the Bridge Domain


33/115


C

Extend the Bridge DomainOption 2

Layer 2

100.1.1.3 100.1.1.5100.1.1.7100.1.1.99

External EPG (policy between the L2 outside EPG and internal EPG)

Leverage vPC for interconnect (diagram shows a single port-channel which is a

BPDU should be enabled on the interconnect ports on the vPC domain

L2 outside forces the same external VLAN


34/115


L2 Outside Connection Configuration Examp

Step 1. Create L2 Outside

connection. Associate with BD.

Specify VLAN ID to connect to

outside L2 network

External Bridge Domain is a way

to specify the VLAN pool for

outside connection. It is NOT a Bridge Domain.


35/115



Step 2. Specify leaf node

and interface providingL2 outside connection


36/115



Step 3. Create external EPG

under L2 outside connection

Step 4. Create contract

between external EPG and

internal EPG

ACI Interaction with STP


37/115


ACI Interaction with STP

STP Root

Switch

Same L2 Outside

EPG

(e.g. VLAN 10)

No STP running within ACI fabric

BPDU frames are flooded betweenports configured to be members of the

same external L2 Outside (EPG) No Explicit Configuration required

Hardware forwarding, no interactionwith CPU on leaf or spine switchesfor standard BPDU frames

Protects CPU against any L2 floodthat is occurring externally

External switches break any potentialloop upon receiving the flooded BPDUframe fabric

BPDU filter and BPDU guard can beenabled with interface policy

ACI Fabric Loopback Protection


38/115


ACI Fabric Loopback Protection

LLDP Loop

Detection

Multiple Protection Mechanisms againstexternal loops

LLDP detects direct loopback cables

between any two switches in the samefabric

Mis-Cabling Protocol (MCP) is a new linklevel loopback packet that detects anexternal L2 forwarding loop

MCP frame sent on all VLANs on all Ports

If any switch detects MCP packet arriving ona port that originated from the same fabric theport is err-disabled

External devices can leverageSTP/BPDU

MAC/IP move detection and learningthrottling and err-disable

MCP Loop

Detection

(supported with

11.1 release)

Managing Flooding Within the BD


39/115


C

Managing Flooding Within the BD

Layer 2

100.1.1.3 100.1.1.5100.1.1.7100.1.1.99

In a classical network traffic is flooded with the Bridge Domain (within the VLAN

You have more control in an ACI Fabric but need to understand what behaviour

BD

MultiEPG

EPG

App 1

EPG

Outside

VLAN 30VLAN 10 VLAN 10 VLAN 10

VLAN 10

EP

App

VLAN 20

Managing Flooding Within the Fabric


40/115


Managing Flooding Within the FabricARP Unicast

ARP Flooding Disabled

(Default)

Disable ARP FloodingARP

forwarded as a unicast pack

fabric based on the host forw

On egress the ARP/GARP is

flooded frame (supports hos

downstream L2 switches)

Firewall Configured a

the Default Gateway

ARP



41/115


Managing Flooding Within the FabricARP Flooding

ARP Flooding Enabled

Enabling ARP FloodingAR

flooded within the BD

Commonly used when the d

external to the Fabric

Firewall Configured a

the Default Gateway

ARP



42/115


Managing Flooding Within the FabricUnknown Unicast Proxy Lookup

Unknown Unicast

Lookup via Proxy

Hosts (MAC, v4, v6) that arespecific ingress leaf switch a

one of the proxies for lookup

rewrite of VTEP address

If the host is not known by a

fabric it will be dropped at th

honeypot for scanning attac

Unknown

Unicast

Proxy

HW Proxy

Lookup



43/115


Managing Flooding Within the FabricUnknown Unicast Flooding

Hosts (MAC, v4, v6) that are

specific ingress leaf switch a

ports within the bridge doma

Silent hosts can be installed

in the proxy (flooding not req

hosts)

Unknown Unicast

Flooded

UnknownUnicast

Unk

Un

Flo



44/115


Managing Flooding Within the FabricUnknown Multicast Mode 1 (Flood)

Unknown Multicast traffic is

all ports in the BD on the sam

source server is attached to


ports in the BD on leaf node

router port

Unknown Multicast

Flooded

UnknownMulticast



45/115


Managing Flooding Within the FabricUnknown Multicast Mode 2 (OMF or Optimised Flood)


multicast router ports in this

Unknown Multicast

Optimised Flooding

UnknownMulticast



46/115


g g gScoping Broadcasts to a micro segment

100.1.1.3 100.1.1.5100.1.1.7100.1.1.99 100.1.1.3

EPG

BEPG

AEPG

C100.1.1.72

Traffic Type 11.0(x) Behaviour 11.1(x) Behaviour

ARP Flood or Unicast Flood or Unicast

Unknown Unicast Flood or Leverage Proxy Lookup Flood or Leverage Proxy

Unknown IP Multicast Flood or OMF Flood or OMF

L2 MCAST, BCAST, Link Local Flood Flood within the BD, Floo

Disable Flooding within t



47/115


g g gMulti Destination Flooding (Supported with 11.1(x) Q2CY15)

Link Level Traffic is either

Contained within the EPG

Contained within the Bridge Dom

Dropped

Security Segmentation for L

Link LevelBCAST

Manage

Flooding within

the BD

100.1.1.3 100.1.1.7100.1.1.99 100.1.1.52

EPG A

100.1.1.4

EPG AEPG B EPG B


48/115



49/115


g g gDisabling Flooding within the BD

100.1.1.3 100.1.1.5100.1.1.7100.1.1.99 100.1.1.3

EPG

BEPG

AEPG

C100.1.1.72

When desired each Bridge Domain can be configure to permit or di

flooding of Link Local, BCAST and L2 MCAST traffic


50/115

Connect Fabric to existing Network


51/115


Connect Fabric to existing Network

Functionally we are expanding the VLANs into ACI.

Existing Design

HSRP

Default GW

VLAN 10 / Subnet 10

P PVM VM VM

ACI Fabric

EPG-10 = VLAN 10

Configure ACI Bridge Domain settings


52/115


Configure ACI Bridge Domain settings

Temporary Bridspecific settings

using the HSRPthe existing net

Select ForwardCustom which

Enable Floodunknown unic

Enble ARP fl Disable Unica

Tenant Red

Context Red

Bridge Domain 10

Subnet 10 EPG-10

Migrate Workloads


53/115


Migrate Workloads

Existing Design

HSRP

Default GW

VLAN 10 / Subnet A

P PVM VM VM

APIC

EPG 10

P PVM VM VM

APIC point of view, the po

VMs will need to be connected to new Port

Group under APIC control (AVS or DVS).

Complete the Migration


54/115


Change BD settings back to normal for ACI mode

Change BD settings back to default. No Flooding

Unicast Routing enabled.

Migrating Default Gateway to the ACI Fabric


55/115


Change GW MAC address. By default, All

fabric and all BD share same GW MAC

Enable Routing and ARP flooding

Extension and ConnectingIt N t k ith VLAN A h


56/115


Its a Network with any VLAN Anywhere

Anycast Default Gateway

10.10.10.8 10.20.20.3210.10.10.9

10.20.20.33

10.10.

Any IP - Anywhere

Policy can be added gradually starting with what ytoday


57/115


Application

Client

Subnet

10.20.20.0/24

Subnet10.10.10.0/24

Subnet

10.30.30.0/24

Subnet

10.40.40.0/24

Subnet

10.50.50.0/24

External Networks

(Outside)

Redirect to Pre-

configured FW

Redirect to Pre-configured FW

Critical Users

(Outside)

Middle Ware

Servers

Web

ServersOracle

DB Contract

Redirect to dynamically

configured FW

NFS ConRedirect to

dynamically

configured FW

Default Users

(Outside)

NFS

Servers

Subnet

10.20.20.0/24

Subnet10.10.10.0/24

Permit TCP any

any

Redirect to Pre-configured FW

today

Simple Policy During Migration - Any-to-AnyConfiguration


58/115


Configuration

Contracts

Provided

Filter Contracts

Provided

Contract

consume

EPG VLAN 10 VLAN10 Default ALL ALL



ALLVLAN 10

VLAN 20

VLAN 30

I want to have a very open configuration with VLAt lki t thi (1)


59/115


talking to anything (1)

Create ContractALL if it doesnt existyet

Use filtercommon/default

I want to have a very open configuration with VLAt lki t thi (2)


60/115


talking to anything (2)

EPG VLAN 10provides andconsumes ALL

Extension and ConnectingD i Di t ib t d ACL


61/115


Dynamic Distributed ACLsPermit ACL is applied on

al l ports between VLAN

10, 20 & 30

10.10.10.8 10.20.20.3210.10.10.9

10.20.20.33

10.10.

All Subnets are allowed to communicate with this policy appl

Later if I want to put an ACL between VLAN 10 an


62/115


ALLVLAN 10

VLAN 20

VLAN 30

Contracts

Provided

Filter Contracts

Provided

Contracts

consumed

EPG VLAN 10 VLAN10 Default VLAN20



Extension and ConnectingD i ACL


63/115


Dynamic ACLsDynamic ACL is applied

between all endpoints

only allowing port 80

10.10.10.8 10.20.20.3210.10.10.9

10.20.20.33

10.10.

Traffic is controlled between VLAN 10 & 20 to HTTP (port 8


64/115

Integrating and Extending L4-7 Servi

Automate Service Insertion Through APIC


65/115


APPWEBEXTERNAL

APIC Policy Model

Endpoint Group (EPG): Collection of similar End Points identifying a particular

Application Tier. Endpoint could represent VMs, VNICs , IP, DNS name etc

Application Profile: Collection of Endpoint Groups and the policies that define w

Endpoint group communicate with each other

Application profile

PolicyPolicyPolicy

ACI Service Insertion via Policy


66/115


Automated and scalable L4-L7 service insertion

Packet match on aredirection rule sends thepacket into a servicesgraph.

Service Graph can be oneor more service nodes pre-

defined in a series.

Service graph simplifies andscales service operations

BeginStage

1

EPG

1Application

Admin

ServiceAdmin

ASA

5585

Ne

Chain

FW_ADC 1

Policy-baseRedirectio

Service Automation Through Device Packag


67/115


Open Device

Package

Policy

Engine APIC provides extendable policy model through

Configuration

Model

Device Interface: REST/CLI

APIC Script Interface

Call Back Scripts

Event Engine

APIC Policy Manager

Configuration

Model (XML File)

Call Back Script

Provider Administrator can upload a Device Pa

Device Package contains XML fine defining De

Device scripts translates APIC API callouts to

APIC

Extending ACI in to Current Data CentresStandard Architecture with Services


68/115


Backbone

vSwitch

Services Chas

vSwitch vSwitch

Services Chassis

Extension and ConnectingServices Switch


69/115


Services SwitchDynamic ACL is applied

between all endpoints

only allowing port 80

10.10.10.8 10.20.20.3210.10.10.9

10.20.20.33

10.10.

Start with the picture we now understand



70/115


Services Switch

10.10.10.8 10.20.20.32

Lets simplify things



71/115


Services Switch

10.10.10.8 10.20.20.32

Start with just a few switches



72/115


Services Switch

10.10.10.8 10.20.20.32

Lets connect them the way we should, redundantly (the previous slides showed a sijust as an example



73/115


Services Switch

10.10.10.8 10.20.20.32

Lets use a small spine switch

We are starting with a very small fabric



74/115


Same picture as seen if the fabric was a services switch



75/115


Attach the L4-7 Services

They can be physical and/or virtual



76/115


vSwitch vSwitch vSwitch

Add in the Core/WAN and the Servers and vSwitches



77/115


vSwitch vSwitch vSwitch

Activate the services, leverage the services chaining and dynamic provisioning

Leverage the fabric as the layer 3 gateway for all the other VLANs


78/115

Extending ACI - Application Virtual S

Hypervisor Integration with ACI


79/115


L/B

EPG

APP

EPG

DBF/W

EPG

WEB

Application Network Profile

VM VM VM

WEB PORT GROUP APP PORT GROUP DB PORT GROUP

APIC

ACI Fabric impleme

Virtual Networks by

Endpoints to EPGs

Endpoints in a Virtua

environment are rep

vNICs

VMM applies networ

by placement of vNI

Groups or VM Netwo EPGs are exposed t

1:1 mapping to Port

Networks

79

VMWare Integration Three Different Options


80/115

2014 Cisco and/or its affiliates. All rights reserved.BRKACI-2001 Cisco Public 80

Three Different Options

+

Distributed Virtual Switch

(DVS)vCenter + vShield

Application

(A

Encapsulations: VLAN

Installation: Native

VM discovery: LLDP

Software/Licenses:vCenter withEnterprise+ License

Encapsulations: VLAN,VXLAN

Installation: Native

VM discovery: LLDP

Software/Licenses:vCenter withEnterprise+ License,vShield Manager withvShield License

EncapsulVXLAN

Installatio

VUM or C VM disco

Software/vCenter wEnterprise


81/115

ACI Hypervisor Integration VMware DVS


82/115


Name of VMM Domain

Type of vSwitch (DVS o

Associated Attachable E

VLAN Pool

vCenter Administrator C

vCenter server informat

ACI Hypervisor Integration VMware DVS


83/115


Application Virtual Switch with OpFlex in AC

S f


84/115


AVS: First Virtual Leaf toimplement OpFlex

Network policy communicatedfrom APIC to AVS through N9kusing OpFlex

Increased control plane scalethrough APIC Cluster and LeafNode

APIC communicates with vCenterServer for Port Group creation

OpFlex

AVS

VMVM VVMVM VM VM

OpFlex OpF

Hypervisor Integration with ACI Endpoint Discovery


85/115


p y

DVS Host

Data Path

Virtual Endpoints arediscovered for reachability &

policy purposes via 2 methods: Control Plane Learning:

- Out-of-Band Handshake: vCenterAPIs

- Inband Handshake: OpFlex-enabled Host (AVS, Hyper-V,etc.)

Data Path Learning: Distributedswitch learning

LLDP used to resolve Virtualhost ID to attached port on leafnode (non-OpFlex Hosts) OpFlex Host

Control

(OpFlex)

Data Path

EPG


5

ACI Hypervisor IntegrationAVS


86/115


APIC Admin

VI/Server Admin Instantiate VMs,

Assign to Port Groups

L/BF/W

EPG

WEB

Create Application Policy

WWeb App

HYPERVISOR

Application Virtual Sw

WEB PORT GROUP APP PORT GRO

vCenter

Server

8

5

1

9ACI

Fabric

Automatically Map

EPG To Port Groups

Push Policy

Create AVS

VDS2

Cisco APIC and

VMware vCenter Initial

Handshake

6

DB

7Create Port

Groups

86

APIC

3

Attach Hypervisor

to VDS

4Learn loc

Host thr

OpFlex Agent

ACI Hypervisor Integration Cisco AVS


87/115


Name of VMM Domain

Type of vSwitch (DVS o

Associated Attachable EVXLAN Pool

vCenter Administrator C

vCenter server informa

Switching mode (FEX o

Multicast Pool

Extending ACI to Existing Virtual & Physical NetworVLAN & VXLAN Extension


88/115


Layer 2

AVS

Layer 2

AVS AVS

AVS AVS

AVS supports OpFlex to integrate with APIC

Supports a Full multi-hop Layer 2 Network between Nexus 9k and AVS: Investme

Layer 2 network is required to support OpFlex bootstrapping in this phase

OpFlex

OpFlex

Extending ACI to Existing Virtual & Physical NetworVLAN & VXLAN Extension


89/115


Layer 2

AVS

Layer 2

AVS AVS

AVS AVS

Supports VLAN and VXLAN for transport (Recommend VXLAN to automate new workload)

Existing Network need to have 1 Infrastructure VLAN for VXLAN transport

Multicast: Turn on IGMP Snooping

Recommend 1 L2 Multicast group per EPG

OpFlex

OpFlex

ACI - Investment protection for INTEGRATION / MIRemote VTEP (Virtual) via AVS


90/115


VLAN VXLAN VLAN

ESXi-1

ESXi-2

pepsi-d

b

N1k-

APP

N1k-

WEB

pepsi-a

pp

APIC APICAPIC

Pepsi-W

eb

Shar

ed_EPG

-1

N1Kv

lan-vS

witc

h-le

af1

ESXi-4 ESXi-3

Coke

-Web

Coke

-App

Spine 1 Spine 2

Leaf 1 Leaf 2 Leaf 3

ESXi-1

Fabric

(40Gbps)

Nexus 3K (L2)

ACI Hypervisor Integration VMware


91/115


ACI Azure Pack Integration

1


92/115


APIC Admin

(Basic Infrastructure)

Azure Pack TenantWeb WeWeb AppApp

3

6

ACI

Fabric

Push Network

Profiles to APIC

Pull Policy on leaf

where EP attaches

Indicate EP Attach

when VM starts1

2

DB

HYPERVISOR HYPERVISOR

APIC

Get VLANs allocated

for each EPG

Create Application

Policy

7

Azure Pack \ SPF

SCVMM PluginAPIC Plugin OpFlex Agent OpFlex Agent

Instantiate VMs

5

4

Create VM Networks

4

ACI OpenStack Integration

Create Application Network

Profile

EPG

Application Network Pro


93/115


2

ACI Admin

(manages physical

network, monitors tenant

state)

L/BF/WL/B

EPG

WEB

Application Network Prof

Create Application Policy

3

5ACI

Fabric

Push Policy

OpenStack Tenant

(Performs step 1,4) Instantiate VMs

Web Web AppApp4

1

DB

HYPERVISOR HYPERVISOR

NOVANEUTRON

Automatically Push

Network Profiles to

APIC

L/BF/W

L/B

EPG

WEB

APIC

Hypervisors vs. Linux Containers

Containers share the OS kernel of the host and thus are lightweight.

However, each container must have the same OS kernel.

C t i i l


94/115


Hardware

Operating System

Hypervisor

Virtual Machine

Operating

System

Bins / libs

App App

Virtual Machine

Operating

System

Bins / libs

App App

Hardware

Hypervisor

Virtual Machine

Operating

System

Bins / libs

App App

Virtual Machine

Operating

System

Bins / libs

App App

Hardware

Operating Syst

Container

Bins / libs

App App

C

Ap

Type 1 Hypervisor Type 2 Hypervisor Linux Containers

Containers are isola

OS and, where appr

bins.

Docker and ACI


95/115


http://www.cisco.com/c/en/us/solutions/collater

al/data-center-virtualization/application-centric-

infrastructure/white-paper-c11-732697.html

One Network for Everything - Logical
http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-732697.htmlhttp://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-732697.htmlhttp://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-732697.htmlhttp://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-732697.html


96/115


SECURITY

Trusted

Zone

DB

Tier

DMZ

External

Zone

APP DWEB

EXTERNALACI

Policy

ACI

Policy

ACI

Policy

96

FW

ADC

Virtual Machines Docker Containers Bare-Meta

HYPEHYPERVISORHYPERVISOR


One Network for Everything


97/115


ESX1 ESX2 ESX3

1/97 1/97 1/97

3/1

3/3

3/9

1/1 1/2 1/1 1/2 1/1 1/2

HyperV

1/3 1/4

BareMetal

1/3

Docker

LxC

KVMKVM


98/115

Extending ACI into Existing Data CenAdding Remote Switch Nodes

Integration of Existing DC Network AssetsExtending the ACI Overlay to Existing DC Assets


99/115


vSwitch

vSwitch

Maintain Existing Physical Network infrastructure and operatio

Extend the ACI policy based forwarding to Hypervisor environ

attached to the existing physical network

IP Core with a Hardware Directory Service

Transit Node

Multicast Root


100/115


Three data plane functions required within the core of an ACI fabric

Transit: IP forwarding of traffic between VTEPs

Multicast Root: Root for one of the 16 multicast forwarding topologies (used for optimisation of mu

balancing and forwarding)

Proxy Lookup: Data Plane based directory for forwarding traffic based on mapping database of E

Not all functions are required on all spine switches

Proxy Lookup

Cached EIDEntry

Proxy Lookup

Unknown EID

Multicast Root

ACI Spine Proxy == LISP Proxy Tunnel Router+ Map-Server


101/115


http://tools.ietf.org/html/draft-moreno-lisp-datacenter-deployment-00

Extension of the ACI Overlay to remote AVSACI Extended Overlay CY15 ACI VXLAN E

Overla


102/115


Direct Attach

EndpointsHypervisor AttachedEndpoints (VLAN or

VXLAN)

vSwitch

AVS

ACI Policy overlay can be extended over existing IPnetworks

Full ACI VXLASwitching Enabl

Hypervisor

L2 or L3

VTEP VTEP VTEP

VTEP

VTEP

Forwarding within the Extended OverlayAdding Remote Leaf Nodes, Nexus 9000

ACI VXLAN EOverla


103/115


vSwitch

AVS

VTEP VTEP VTEP

VTEP

VTEP

Nexus 9000 as a remote ACI Leaf

Support for full policy based forwarding, atomic counters,

zero touch install, health scores

VVM

10.2.4.710.9.3.37

VM

VM

10.2.4.32

VM

10.9.3

10.9.3.38

AVSVTEP

AVSVTEP

VM

10.2.4.1910.9.3.123

VM VTEPVTEP


VTEP Group


104/115


vSwitch

AVS

VTEP VTEP VTEP

VTEP

VTEP

Nexus 9000 as a remote ACI Leaf

Support for full policy based forwarding, atomic counters,

zero touch install, health scores

VVM

10.2.4.710.9.3.37

VM

VM

10.2.4.32

VM

10.9.3

10.9.3.38

AVSVTEP

AVSVTEP

VM

10.2.4.1910.9.3.123

VM VTEPVTEP

IP

pPolicy



105/115


vSwitch

AVS

VTEP VTEP VTEP

VTEP

VTEP

VM

10.2.4.710.9.3.37

VM

VM

10.2.4.3

2

VM

10.9.3.38

AVSVTEP

AVSVTEP

VM

10.2.4.1910.9.3.123

VM

Nexus 9000 as a remote ACI Leaf will support vSwitch

downstream (ESX/DVS, Hyper-V, OVS)

Leverage Existing Hypervisor implementations

VTEPVTEPVTEP



106/115


vSwitch

AVS

VTEP VTEP VTEP

VTEP

VTEP

VM

10.2.4.710.9.3.37

VM

VM

10.2.4.32

VM

10.9.3.38

AVSVTEP

AVSVTEP

VM

10.2.4.1910.9.3.123

VM

ACI 9300 performs both L2 and L3 forwarding (Remote Leaf

is a full member of the fabric)

Remote 9300 Leaf performs full local inter EPG policy

forwarding

VTEPVTEP


Unknow

Known

(Cache


107/115


vSwitch

AVS

VTEP VTEP VTEP

VTEP

VTEP

VM

10.2.4.710.9.3.37

VM

VM

10.2.4.32

VM

10.9.3.38

AVSVTEP

AVSVTEP

VM

10.2.4.1910.9.3.123

VM

First packet for an unknown MAC or IP host address passes

through the inline directory

Subsequent packets are forwarded directly to the target

VTEP (local switching)

VTEPVTEP



108/115


vSwitch

AVS

VTEP VTEP VTEP

VTEP

VTEP

VM

10.2.4.710.9.3.37

VM

VM

10.2.4.32

VM

10.9.3.38

AVSVTEP

ACI services insertion and full L4-L7 policy are supported via

the AVS

VTEPVTEP

Multi-Site Fabrics

Fabric A

VTEP

IPVNID Tenant Packet

Group

Policy


109/115


Host Level Reachability Advertised between Fabrics via BGP

Transit Network is IP Based

Host Routes do not need to be advertised into transit network

Policy Context is carried with packets as they traverse the transit IP Ne

Forwarding between multiple Fabrics is allowed (not limited to two sites

Web/AppDB

Multi-Site

Traffic

mBGP - EVPN

Applications will spread across existing and newinfrastructure

Outside

AppWeb


110/115


vPC

vPC

vPC

Outside

(Tenant

VRF)

QoS

Filter

QoS

Service

QoS

Filter

DB

Web

Outside

(Tenant VRF)

QoS

Filter

QoS

Service

QoS

Filter

DB Outside EPG

Directory/Proxy

Service Nodes

ACIIts an IP Network


111/115


IP Enabled Data CentreNetwork

AVS

AVS

Border Leave

APIC Policy

ControllerACI Leaf

Nexus 9000

ACI Virtual Leaf

(AVS)vSwitch

ACI Exte

via C

ACI Policy and Automation

Extended to Physical andExisting Virtual Servers via

Cisco Nexus 9000

ACI Enabled L4-7

Virtual and Physical

Services (Support forExisting and New

Services Instances)

Extending ACI Policy and Automation into the Existing Data Centre

Recommended Readings


112/115


Give us your feedback and receive a

Cisco Live 2015 T-Shirt!

Complete Your Online Session Evaluation


113/115


C sco e 0 5 S t

Complete your Overall Event Survey and 5 Session

Evaluations.

Directly from your mobile device on the Cisco Live

Mobile App

By visiting the Cisco Live Mobile Site

http://showcase.genie-connect.com/clmelbourne2015

Visit any Cisco Live Internet Station located

throughout the venue

T-Shirts can be collected in the World of Solutions

on Friday 20 March 12:00pm - 2:00pm

Learn online with Cis

Visit us online after the

access to session vide

presentations. www.Ci
http://showcase.genie-connect.com/clmelbourne2015http://www.ciscoliveapac.com/http://www.ciscoliveapac.com/http://showcase.genie-connect.com/clmelbourne2015


114/115


115/115

Documents

BRKACI-2001 - Integration and Interoperation of Existing Nexus Networks Into an ACI Architecture (2015 Melbourne)