Upload
trandat
View
224
Download
0
Embed Size (px)
Citation preview
Mastering ACI and OpenStack
Domenico Dastoli
Technical Marketing Engineer INSBU
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Spark
Questions? Use Cisco Spark to communicate with the speaker after the session
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
How
cs.co/ciscolivebot#BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI connects Virtual and Physical World
4BRKACI-3456
Agenda
• ACI, Virtualisation and VMM Domains
• ACI and OpenStack
• Options to Install OpenStack and ACI plugin
• Operate OpenStack: ML2 mode and GBP mode
• External Network connectivity
• Demo
• Q&A
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Remote PoD Multi-Pod / Multi-Site Hybrid Cloud Extension
ACI Anywhere
IP
WAN
IP
WAN
Remote Location Public CloudOn Premise
Security Everywhere Policy EverywhereAnalytics Everywhere
7BRKACI-3456
ACI Anywhere - VisionAny Workload, Any Location, Any Cloud
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Fabric
App DBWeb
WWW
QoS
Filter
QoS
Service
QoS
Filter
Cisco ACI – The basicsLogical Network Provisioning of Stateless Hardware
BRKACI-3456 8
Scale-Out Penalty Free Overlay
APIC
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hypervisor Interaction with ACITwo modes of Operation
• ACI Fabric as an IP-Ethernet Transport
• Encapsulations manually allocated
• Separate Policy domains for Physical and Virtual
VLAN 10 VLAN 10 VLAN 100
Non-Integrated Mode
• ACI Fabric as a Policy Authority
• Encapsulations Normalized and dynamically provisioned
• Integrated Policy domains across Physical and Virtual
APP WEB DB
Integrated Mode
DB
BRKACI-3456 9
APIC APIC
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
vCenter
DVS/AVE
Relationship (VMM Domain) is formed between APIC and Virtual Machine Manager (VMM)
Multiple VMMs likely on a single ACI Fabric
There is 1:1 relationship between a Distributed Virtual Switch and VMM Domain
Hypervisor Integration with ACI
SCVMM
10
OpenStack
BRKACI-3456
Kubernetes
APIC
RHEV
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
EP
EP
EP
EP
EP
EP EP
EP
EP
EP
EP
EP
EP
EP
VMM Domain 1
4K EPGsVMM Domain 2
4K EPGs
16M Virtual Networks VLAN ID only gives 4K EPGs (12 bits)
Scale by creating pockets of 4K EPGs
Map EPGs to VMM Domain based on scope of live migration
Place VM anywhere
Live migrate within VMM domain
Hypervisor Integration with ACIVMM Domains & VLAN Encapsulation
BRKACI-3456 11
APIC
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hypervisor Integration with ACIVMM Domains & VLAN Encapsulation
EP
EP
EP EP
VMM Domain 1
4K EPGsVMM Domain 2
4K EPGs
VLAN 5
VLAN 16
16M Virtual Networks
VNID 6032
VLAN ID only gives 4K EPGs (12 bits)
Scale by creating pockets of 4K EPGs
Map EPGs to VMM Domain based on scope of live migration
Place VM anywhere
Live migrate within VMM domain
BRKACI-3456 12
APIC
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpenStack Controller
ACI + OpenStack – With OpFlex SupportFull Policy Based Network Automation Extended to the Hypervisor
• Open Source OpFlex agent extends ACI into the host
• OpFlex Proxy exposes new open API in ACI fabric
OpFlex for OVS
OS nodes OVS OpFlex Agent
OpFlex Proxy
Solutions with Major OpenStack Distributions
14
APIC Unified Plugin
BRKACI-3456
APIC
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Why Cisco ACI and OpenStack?
Distributed, Scalable
Virtual Networking
• Full Neutron Node datapath replace
• Fully distributed Layer 2, anycast
gateway, DHCP, and metadata
• Distributed NAT and floating
IP address
Hardware-Accelerated
Performance
• Automatic VXLAN tunnels at top of
rack (ToR)
• No wasted CPU cycles for tunneling
• Optional use of SRIOV
Operations and
Telemetry
• Troubleshooting across physical and
virtual environments
• Health scores and capacity planning
per tenant network
Integrated Overlay
and Underlay
• Fully managed underlay network
through Cisco® APIC
• Capability to connect physical servers
and multiple hypervisors to overlay
networks
BRKACI-3456 15
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
What is the ACI Unified Plugin for OpenStack?
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Neutron ML2
The Modular Layer 2 (ml2) plugin is a framework allowing OpenStack Networking to simultaneously utilize the variety of layer 2 networking technologies.
Drivers within ml2 implement separately extensible sets of network types and of mechanisms for accessing networks of those types.
17
• Type DriversEach available network type is managed by an ml2 TypeDriver.TypeDrivers maintain any needed type-specific network state, andperform provider network validation and tenant network allocation.The ml2 plugin currently includes drivers for the local, flat, vlan, gre,opflex and vxlan network types.
• Mechanism DriversEach networking mechanism is managed by an ml2 MechanismDriver. The MechanismDriver is responsible for taking the information established by the TypeDriver and ensuring that it is properly applied given the specific networking mechanisms that have been enabled.
Neutron Server
ML2 Plug-in
Type Manager Mechanism Manager
API Extensions
GR
E
TypeD
river
Cis
co
AP
IC
VLA
N
TypeD
river
VX
LA
N
TypeD
river
Cis
co
Nexu
s
Mic
rosoft
Hyper-V
Layer 2
Popula
tion
Lin
ux
Brid
ge
Open
vS
witc
h
SR
-IOV
OpF
lex
TypeD
river
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI ML2 Mechanism Driver
When running the ACI integration, The following Type and Mechanism Drivers will be used:
• Type Driversopflex • Mechanism Driversapic_aim
18
Neutron Server
ML2 Plug-in
Type Manager Mechanism Manager
API Extensions
GR
E
Typ
eD
rive
r
VX
LA
N
Typ
eD
rive
r
Cis
co
Ne
xu
s
Mic
roso
ft
Hyper-V
La
ye
r 2
Po
pu
latio
n
Lin
ux B
ridg
e
Open
vS
witc
h
SR
-IOV
BRKACI-3456
VL
AN
Typ
eD
rive
r
Cis
co
apic
_aim
Op
Fle
x
Typ
eD
rive
r
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
APIC ML2 options
• Opflex mode allows creation of neutron networks based on
• VLAN
• VXLAN
• APIC AIM Mechanism driver enables the user to deploy OpenStack projects in:
• Neutron standard ML2 mode
• Group Based Policy (GBP) mode
19BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
ML2 vs GBP mode
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ML2 – APIC Mapping
Neutron Object APIC Object
Project Tenant
Network EPG + BD
Subnet Subnet
Router Contract
Security Group + Rule N / A
Iptables rules maintained per host
21
• With the ML2 Standard Neutron model, the following mapping happens.
• All the operations are done on OpenStack through Horizon, CLI or Heat
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with ML2 model
22BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with ML2 model
23BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with ML2 model
24BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with ML2 model
25BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with ML2 model
26BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with ML2 model
27BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with ML2 model
28BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
GBP – APIC Mapping
GBP Object APIC Object
Project Tenant
L3 Policy VRF
L2 Policy Bridge Domain + Subnet
Policy Group Endpoint Group
Policy Ruleset Contract
29
• With the GBP Model the following mapping happens.
• GBP offers much more granularity and flexibility compare to standard neutron.
• GBP comes with CLI, Heat and Horizon plugins
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with GBP model
30BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with GBP model
31BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with GBP model
32BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with GBP model
33BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with GBP model
34BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with GBP model
35BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with GBP model
36BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Tenant Creation with GBP model
37BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
GBP Policy Mapping
38BRKACI-3456
Bridge Domain
EPG WEB
EPG APP
EPG DB
EPG DHCP
subnet
dhcpserver
VRF
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
GBP L3 PolicyGBP Policy Mapping
39BRKACI-3456
Bridge Domain
EPG WEB
EPG APP
EPG DB
EPG DHCP
subnet
dhcpserver
VRF
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
GBP L2 Policy
GBP Policy Mapping
40BRKACI-3456
Bridge Domain
EPG WEB
EPG APP
EPG DB
EPG DHCP
subnet
dhcpserver
VRF
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
GBP
Policy Groups
GBP Policy Mapping
41BRKACI-3456
Bridge Domain
EPG WEB
EPG APP
EPG DB
EPG DHCP
subnet
dhcpserver
VRF
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
GBP Policy Mapping
42BRKACI-3456
Bridge Domain
EPG WEB
EPG APP
EPG DB
EPG DHCP
subnet
dhcpserver
VRF
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ML2 vs GBP model – what is best?
• GBP:
• Application Centric
• Security groups are created as ACI contracts AND OVS rules. So they are visible on ACI and will be enforced both in HW (ACI leaf) and SW (OVS).
• Introduces new REST APIs: if any existing templates, you will need to adapt
• ML2:
• Network Centric
• Standard way of creating neutron networks
• REST API will not change: any heat or CLI template will keep working
• Security Groups not visible in ACI: they are implemented by OS as OVS rules
43BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
What are the components and how do they work?
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
What’s installed on the controller and compute node?
[heat-admin@overcloud-controller-0 ~]$ sudo yum list | grep @aci-repo
aci-integration-module.noarch 0.6.0-162.el7 @aci-repo
agent-ovs.x86_64 1:1.5.0-63.el7.centos @aci-repo
apicapi.noarch 1.1.0-170.el7 @aci-repo
neutron-opflex-agent.noarch 2:6.1.0-26.el7 @aci-repo
openstack-dashboard-gbp.noarch 6.0.0-53.el7 @aci-repo
openstack-heat-gbp.noarch 6.0.0-53.el7 @aci-repo
openstack-neutron-gbp.noarch 6.2.0-53.el7 @aci-repo
[heat-admin@overcloud-controller-0 ~]$
[heat-admin@overcloud-compute-0 ~]$ sudo yum list | grep @aci-repo
agent-ovs.x86_64 1:1.5.0-63.el7.centos @aci-repo
neutron-opflex-agent.noarch 2:6.1.0-26.el7 @aci-repo
openstack-neutron-gbp.noarch 6.2.0-53.el7 @aci-repo
[heat-admin@overcloud-compute-0 ~]$
controller
compute
BRKACI-3456 45
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
A key component:
AIM Daemon
A new component:
ACI Integration Module
The AIM daemon is running on the Controller nodes and is responsible to configure ACI through REST API call based on the OpenStack policy model defined.
46BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Architecture: APIC Integration Manager
Description
• APIC Integration Database and APIC Integration
Manager (AIM) introduced as central point of
storing plugin configuration.
• AIM uses the OpenStack database.
• AIM continuously synchronizes with APIC using
APIC Integration Daemon (AID).
• Group-Based Policies are mapped into Neutron API
and then AIM. Neutron APIs are mapped to AIM
directly.
RouterSecurity
Group
Netwo
rkRule
Set
Policy
Group
Policy
Group
Group-Based
Policy
Neutron API
APIC Integration Database (AIM)
APIC Unified Plugin
AID
processes
OpenStack
Controller
BRKACI-3456 47
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpenStack Tenant
Instantiate VMs
Create Application Policy
Web WebWebWeb AppApp4
3
5ACI
Fabric
Automatically Push
Network Profiles to
APIC and keep it
sync
Push Policy
Create Network, Subnet,
Security Groups, PolicyNETWORK ROUTING SECURITY
1
2
DB DB
HYPERVISOR HYPERVISOR HYPERVISOR
NOVANEUTRON
OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH OPEN VIRTUAL SWITCH
The AIM Daemon at work: the workflow
APIC
48
C2EPG
APPEPG DB
C1EPG
WEB
Application Network Profile
AIM
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Neutron Opflex Agent
Neutron Opflex Agent
The Neutron Opflex Agent runs on both the compute and the controller. It is responsible to communicate with the neutron server.
49BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agent OVS
Agent OVS
The Agent OVS runs on the compute and controller nodes. It is responsible to communicate with the OVS and the leaf node to register to ACI fabric.
50BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpFlex Architecture
• Neutron-opFlex-agent:
Receives updates from Neutron
about new endpoints and
updates EP and Service files
• Agent-OVS: Runs OpFlex
protocol with ACI leaf proxy.
• Agent-OVS Programs open
vswitch via OpenFlow
OpFlex policy (ACI infra VLAN)
OpenFlow
Endpoint Files
EndpointInformation:RabbitMQ
Neutron-Opflex-Agent
Agent-OVS
Open vSwitch
Neutron Server(s)
OpenStack
Node
51BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
What is the Endpoint file?
52
Routing settings
VM IP address
Network Policy including floating IP if any assigned
VM name
For each VM, the Neutron Opflex Agent creates a .ep file local to the node with all the information of the VM.
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56BRKACI-3456
Distributed, Scalable
Virtual Networking
• Full Neutron Node datapath replace
• Fully distributed Layer 2, anycast
gateway, DHCP, and metadata
• Distributed NAT and floating
IP address
Hardware-Accelerated
Performance
• Automatic VXLAN tunnels at top of
rack (ToR)
• No wasted CPU cycles for tunneling
• Optional use of SRIOV
Operations and
Telemetry
• Troubleshooting across physical and
virtual environments
• Health scores and capacity planning
per tenant network
Integrated Overlay
and Underlay
• Fully managed underlay network
through Cisco® APIC
• Capability to connect physical servers
and multiple hypervisors to overlay
networks
Closer look to the enhancement with the ACI plugin
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Routing and Policy Enforcement is done on the host
Tenant Networks
ComputeHost
Neutron Server(s)
OVS
Management/API Network
• Traditionally in OpenStack the
routing is done on the servers
hosting neutron services only.
• With ACI integration the opflex-
agent is taking care of the
routing of the VMs. Since each
compute node has a opflex-
agent, the routing is done in a
distributed manner.
• Also, the opflex-agent performs
local policy enforcement through
OVS rules locally on the same
hypervisor where the instance
lives.
DESCRIPTION
57
Neutron L3
Agent
EndpointFile
Agent-OVS
OpenFlow
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
DHCP Function
Tenant Networks
ComputeHost
Neutron Server(s)
Agent-OVS
Management/API Network
DHCP Allocation and Options
DHCP DORA
Neutron DHCPAgent DNSmasq neutron-
opflex-agent
EndpointFile
• Traditionally VMs are
getting IP from Neutron
DHCP Server
• Agent-OVS learns info of
the VM from Endpoint
Files
• Agent-OVS responds to
the VMs with DHCP
responses
• DHCP allocation and
options passed back to
Neutron server.
DESCRIPTION
58BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Metadata Function
Tenant Networks
NovaComputeHostOpenStack Controller
Agent-OVS
Management/API Network
neutron-opflex-agent
Nova-APIMetadataService
neutron-metadata-agent
VM Metadata
ServiceFile
VMMeta-data
• Traditionally in OS VMs
get the meatadata
information from the
service running on
Neutron Server
• Neutron metadata agent is
reading the Service File
• Metadata agent locally
performs proxy
• Metadata agent updates
the neutron server with
VM Metadata
DESCRIPTION
59BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
L3-Out VRFRID: 7.7.7.7Tenant VRFs
Compute Node
Open vSwitchw/Local NAT
ACI Fabric External RouterWith IP routes to:SNAT: 10.1.2.0/24Floating: 10.1.3.0/24
SNAT Subnet IP:10.1.2.1/24
Floating Subnet IP:10.1.3.1/24
Link Subnet IP:10.1.1.2/30
Link Subnet IP:10.1.1.1/30
NAT/External Traffic
Non-NATTenant Traffic
NAT Function performed in the OVS locally
• Floating IP configured by
OpenStack Neutron using
standard mechanism
• OVS performs NAT
function using OpenFlow
rules from OpFlex agent
for Floating IP
DESCRIPTION
60BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
About the OpenStack Infrastructure network
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Required connectivity for hosts
• Typically there will be a number of networks required for OpenStack:
• Internal API Network (VLAN)
• Storage Network (VLAN)
• Storage Management Network (VLAN)
• Provisioning Network (Native VLAN)
• External Network (VLAN)
62
Note:
Controller node requires connectivity to the APIC controller. External Network can be used for this purpose.
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Physical connectivity with ACI
OpenStack node will need:
• At least two NICs per server configured as bond interface (for redundancy)
• One NIC for provisioning network
BRKACI-3456 63
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI Configuration for OpenStack Node connectivity
To provide connectivity between hosts, it is required to pre provision a tenant on ACI with the appropriate configuration.
This tenant could be either dedicated to the OpenStack infrastructure, or it could be shared with other infrastructure hosts.
Note that this infrastructure tenant will provide the underlay connectivity for the host, therefore it will be updated only if necessary to modify the OpenStack node connectivity (i.e. adding a node).
BRKACI-3456 65
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI OpenStack Infrastructure tenant
66BRKACI-3456
VRF: Main_VRF
BD_MGMT
L3 unicast enabled
Default GW IP defined
Tenant: OpenStack_Infra
EPG-ExternalNet
VLAN104EPG-InternalApi
VLAN105
EPG-StorageNet
VLAN106EPG-StorageMgmt
VLAN107
- Two BDs:
- BD_MGMT provides OOB connectivity (in this design this provides connectivity both
Internet and APIC)
- BD_OSP is only switching but we keep L3 enabled to learn IP from the hosts
- EPGs have static bindings to the interfaces of the host
EPG-Provisioning
Native-VLAN
BD_OpenStack_Infra
L3 Unicast Enabled
Limit IP Learning to Subnet: Disabled
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
EPG Static binding
EPG-
Provisioning
Native
EPG-
InternalApi
VLAN105
EPG-
StorageNet
VLAN106
EPG-
StorMgmt
VLAN107
EPG-
External
VLAN104
OpenStack nodes will have NIC interfaces statically bound to ACI End Point Group.
On ACI side an individual interface will be configured for Provisioning network. The bond interfaces of
the host will be connected to a VPC pair on ACI leaf switches.
bond0
BRKACI-3456 67
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Define the EPG and the specific static binding for each network required:
• ExternalNet
• InternalAPI
• StorageMgmt
• StorageNet
OpenStack Infrastructure Tenant
BRKACI-3456 69
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
OpenStack Tenant Network (VM datapath)
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpenStack Tenant Network
• Neutron provides each tenant with their own networks using either VLAN segregation (where each tenant network is a network VLAN), or tunneling (through VXLAN). Network traffic is isolated within each tenant network. Each tenant network has an IP subnet associated with it, and network namespaces means that multiple tenant networks can use the same address range without causing conflicts.
71BRKACI-3456
• ACI Plugin allows to use as encapsulation mode:
• VLAN
• VXLAN
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpenStack Tenant Network with ACI with VLAN
• In VLAN encapsulation mode:
• The user will define a pool of VLAN (VMM Domain Pool)
• Each OpenStack project network will automatically pick one VLAN from the pool
• The ACI Access policy of the Leaf ports will allow all the VLANs defined in the VLAN pool
72
The bond0 could be the same interface used for the OpenStack infra traffic.
However this could be also a dedicated bond for tenant traffic.
BRKACI-3456
VMM Domain Pool: 200-300
bond0
Tenant1
net1
VLAN 200
Tenant1
net2
VLAN 201
Tenant2
net1
VLAN 220
Tenant3
Net1
VLAN 230
VLAN trunk
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpenStack Tenant Network with ACI with VLAN: Scalability
• If you need to scale more than 4k VLANs:
1. You can use VXLAN
2. You can create multiple VMM domain and assign nodes to those:Allows you to use multiple VMM Domains with potentially overlapping VLAN pool ranges in a single OpenStack deployment
73
VMM Domain1 Pool: 200-300
bond0
VLAN trunk VLAN trunk
VMM Domain2 Pool: 200-300
Compute-1
BRKACI-3456
Compute-2
bond0
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpenStack Tenant Network with ACI with VXLAN
• In VXLAN encapsulation mode:
• The ACI Access policy of the Leaf ports will allow the ACI Infra VLAN.
• Each OpenStack project network will automatically pick one VXLAN
• The VXLAN will be encapsulated into a tunnel using the ACI infra VLAN
74
The bond0 could be the same interface used for the OpenStack infra traffic or could be a dedicated bond for tenant traffic.
For better performance, server NICs should be capable of VXLAN offload
Blade systems are not supported with VXLAN encapsulation
BRKACI-3456
VMM Domain: VXLAN
bond0
Tenant1
net1
VXLAN 200
Tenant1
net2
VXLAN 201
Tenant2
net1
VXLAN 220
Tenant3
Net1
VXLAN 230
VXLAN are encapsulated into tunnel
using ACI Infra VLAN
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
What if you want to provision VLAN to 3rd party?
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hierarchical port binding (HPB)
opflex main segment
(VLAN or VXLAN)
ACI Leaf
Switch
3rd party opflex
Compute Host
non-opflex
using local vlan segment
Hierarchical Port Binding allows to create different network types:
• Opflex networks would be created onto ACI
• vlan or other network types can be created to bind special 3rd party agent or mech driver asks for vlan port binding
BRKACI-3456 76
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SR-IOV support with ACI
VNF
OpenStack Controller
Group-Based Policy
VNFVNF VNF VNF
NIC NICSRIOVSRIOV
ML2
VLANs
Another use case for HPB is SR-IOV enabled hosts:
• GBP or ML2 options
• GBP – Reintroduces security policies via groups / rulesets in the fabric
• Can mix opflex and SR-IOV on the same physnet
BRKACI-3456 78
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Many OpenStack customers are interested connecting VMs to both IPv4 and IPv6 networks
• This feature adds support in the ACI OpenStack plugins for dual stack operation
• OpenStack neutron address scopes are automatically mapped to ACI VRFs
• Each IPv4 address scope maps to a unique VRF in ACI. A IPv6 address scope may include multiple IPv4 address scopes will be provisioned on these VRFs
IPv6 Dual Stack
BRKACI-3456 79
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
OpenStack Support
• Cisco is committed to provide support to the main OpenStack distributions:
• Other distributions is supported with specific agreements with the 3rd party vendor, i.e.
81BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Installation of OpenStack and ACI Plugin
• On Cisco.com:
• https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html#OpenStack_Installation_Guides
• Manual installation:
• Prone to errors and discouraged. Moving forward we will limit the support for production environments while documentation will be always provided.
• RHEL OSP Director – full support for automated installation and upgrade
• Canonical Juju Charms – full support for automated installation and upgrade
82BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ML2 – APIC Mapping
Neutron Object APIC Object
Project Tenant
Network EPG + BD
Subnet Subnet
Router Contract
Security Group + Rule N/A
Iptables rules maintained per host
85
• With the ML2 Standard Neutron model, the following mapping happens.
• All the operations are done on OpenStack through Horizon, CLI or Heat
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Create a new OpenStack project:
[stack@dom-undercloud ~]$ openstack project create --description "tenant Cisco Live Europe
Barcelona" CiscoLive
+-------------+------------------------------------+
| Field | Value |
+-------------+------------------------------------+
| description | tenant Cisco Live Europe Barcelona |
| enabled | True |
| id | 97390b780c7545d393d9314d34e69cfa |
| name | CiscoLive |
+-------------+------------------------------------+
[stack@dom-undercloud ~]$ openstack role add --project CiscoLive --user admin admin
+-----------+----------------------------------+
| Field | Value |
+-----------+----------------------------------+
| domain_id | None |
| id | cd3c4088da8d40778e93efc2d8d8ce6c |
| name | admin |
+-----------+----------------------------------+
[stack@dom-undercloud ~]$
BRKACI-3456 86
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Create a new OpenStack network:
[stack@dom-undercloud ~]$ openstack network create net101
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | UP |
| dns_domain | None |
| id | 96c4644f-a63a-4b15-b36f-b00dfe71bc38 |
| is_default | None |
| name | net101 |
| port_security_enabled | True |
| project_id | 97390b780c7545d393d9314d34e69cfa |
| provider:network_type | opflex |
| provider:physical_network | physnet1 |
| provider:segmentation_id | None |
| qos_policy_id | None |
| revision_number | 3 |
| router:external | Internal |
| segments | None |
| shared | False |
| status | ACTIVE |
| subnets | |
+---------------------------+--------------------------------------+
[stack@dom-undercloud ~]$
BRKACI-3456 87
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Attach a subnet to the network[stack@dom-undercloud ~]$ openstack subnet create --network net101 --gateway 192.168.200.254 --subnet-range
192.168.200.0/24 subnet101
+-------------------+--------------------------------------+
| Field | Value |
+-------------------+--------------------------------------+
| allocation_pools | 192.168.200.1-192.168.200.253 |
| cidr | 192.168.200.0/24 |
| description | |
| dns_nameservers | |
| enable_dhcp | True |
| gateway_ip | 192.168.200.254 |
| host_routes | |
| id | 96c4644f-a63a-4b15-b36f-b00dfe71bc38 |
| ip_version | 4 |
| name | subnet101 |
| network_id | f816ceaa-af05-47ce-83b9-f06dc5ed9f5b |
| project_id | 97390b780c7545d393d9314d34e69cfa |
| revision_number | 2 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
+-------------------+--------------------------------------+
[stack@dom-undercloud ~]$
BRKACI-3456 89
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• A new Tenant is created and a new EPG and unicast disabled BD is created
• Unicast routing will stay disabled until a router is created in OS
• The BD is attached to a generic unroutedVRF created in common tenant
What happens on ACI
BRKACI-3456 90
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Add a router and attach the subnet to it
[stack@dom-undercloud ~]$ openstack router create CLrouter
+-------------------------+--------------------------------------+
| Field | Value |
+-------------------------+--------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | None |
| availability_zones | None |
| description | |
| distributed | False |
| external_gateway_info | None |
| flavor_id | None |
| ha | False |
| id | 0cbf9e21-f6f9-40c2-9c98-6f04a0ff6268 |
| name | CLrouter |
| project_id | 97390b780c7545d393d9314d34e69cfa |
| revision_number | None |
| routes | |
| status | ACTIVE |
+-------------------------+--------------------------------------+
[stack@dom-undercloud ~]$ openstack router add subnet CLrouter subnet101
[stack@dom-undercloud ~]$
BRKACI-3456 91
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Security Groups with ML2 model
BRKACI-3456
• In ML2 policy mode the router created corresponds to a permit any contract in ACI.
• Security groups are defined in OpenStack and controlled there.
• They will be reflected in policy defined in OVS rules.
92
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
GBP – APIC Mapping
GBP Object APIC Object
Project Tenant
L3 Policy VRF
L2 Policy Bridge Domain
Policy Group Endpoint Group
Policy Ruleset Contract
95
• With the GBP Model the following mapping happens.
• GBP offers much more granularity and flexibility compare to standard neutron.
• GBP comes with CLI, Heat and Horizon plugins
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Create a new GBP VRF:
[stack@dom-undercloud ~]$ gbp l3p-create Main_VRF --ip-pool
192.168.0.0/16 --subnet-prefix-length 24
Created a new l3_policy:
+----------------------------+--------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------+
| address_scope_v4_id | 059fed59-1f07-4907-bece-8f260cb0bb86 |
| address_scope_v6_id | |
| id | b7b638f7-7fbd-4594-9ef8-4a560961a26c |
| ip_pool | 192.168.0.0/16 |
| ip_version | 4 |
| l2_policies | |
| name | Main_VRF |
| proxy_ip_pool | 192.168.0.0/16 |
| proxy_subnet_prefix_length | 28 |
| routers | ac49f46d-f08e-4fe2-9016-35b81dc56942 |
| shared | False |
| status | BUILD |
| status_details | |
| subnet_prefix_length | 24 |
| subnetpools_v4 | d68b01f2-992b-4743-8dbf-a7f3a8c00313 |
| subnetpools_v6 | |
| tenant_id | 5b8945dba07a43e0b32efea4f1bc3fdf |
+----------------------------+--------------------------------------+
This pool is where I’ll be taking my tenant
subnets from during network creation.
BRKACI-3456 96
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Create a new GBP L2 Policy (Bridge Domain):
[stack@dom-undercloud ~]$ gbp l2policy-create --l3-policy Main_VRF l2pnet101
Created a new l2_policy:
+----------------------+--------------------------------------+
| Field | Value |
+----------------------+--------------------------------------+
| description | |
| id | 6c905c23-a4b7-4960-8959-6b8d16088ce3 |
| inject_default_route | True |
| l3_policy_id | b7b638f7-7fbd-4594-9ef8-4a560961a26c |
| name | l2pnet101 |
| network_id | eb2269dc-4e43-44f8-a96a-7b060d942d98 |
| policy_target_groups | autof6c8bb08ac721e02feae6f27a57a1444 |
| project_id | 5b8945dba07a43e0b32efea4f1bc3fdf |
| shared | False |
| status | ACTIVE |
| status_details | |
| tenant_id | 5b8945dba07a43e0b32efea4f1bc3fdf |
+----------------------+--------------------------------------+
[stack@dom-undercloud ~]$
BRKACI-3456
The subnet is carved out from the VRF /16 defined before
This EPG contains DHCP instance for the L2 policy.
97
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Create GBP groups (ACI EPGs):
[stack@dom-undercloud ~]$ gbp group-create epg101 --l2-policy l2pnet101
BRKACI-3456
[stack@dom-undercloud ~]$ gbp l2policy-create --l3-policy Main_VRF l2pnet102
[stack@dom-undercloud ~]$ gbp group-create epg102 --l2-policy l2pnet102
[stack@dom-undercloud ~]$ gbp group-create epg103 --l2-policy l2pnet102
I can add more EPG, both in the same Bridge Domain, or in others:
98
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
In order to allow communication, I need to create policy actions, classifiers, rules and rulesets within GBP.
GBP
Policy Classifier
Policy Rule
Policy Ruleset
ACI
Filter Entry
Filter
Contract
BRKACI-3456 99
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
First, create a Policy Action to define the behaviour:
[stack@dom-undercloud ~]$ gbp policy-action-create allow --action-type allow
Created a new policy_action:
+--------------+--------------------------------------+
| Field | Value |
+--------------+--------------------------------------+
| action_type | allow |
| action_value | |
| description | |
| id | c9333baf-aa23-4a32-806c-11d1e16eabeb |
| name | allow |
| shared | False |
| tenant_id | 5b8945dba07a43e0b32efea4f1bc3fdf |
+--------------+--------------------------------------+
BRKACI-3456 100
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Then define a Policy Classifier:
[stack@dom-undercloud ~]$ gbp policy-classifier-create icmp-traffic --protocol icmp --direction bi
Created a new policy_classifier:
+-------------+--------------------------------------+
| Field | Value |
+-------------+--------------------------------------+
| description | |
| direction | bi |
| id | 5947db25-6c2e-4091-b012-ea1b86a0fb53 |
| name | icmp-traffic |
| port_range | |
| protocol | icmp |
| shared | False |
| tenant_id | 5b8945dba07a43e0b32efea4f1bc3fdf |
+-------------+--------------------------------------+
BRKACI-3456 101
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Next, define a Policy Rule, referencing the classifier created in the last step:
[stack@dom-undercloud ~]$ gbp policy-rule-create ping-policy-rule --classifier icmp-traffic --actions allow
Created a new policy_rule:
+--------------------------+------------------------------------------------------------------------------------------+
| Field | Value |
+--------------------------+------------------------------------------------------------------------------------------+
| apic:distinguished_names | {"Forward-FilterEntries": ["uni/tn-common/flt-pr_3ecd614d-717b-483c-8e5c-c5f335d40a88/e |
| | -os-entry-0"], "Reverse-FilterEntries": ["uni/tn-common/flt-reverse-pr_3ecd614d-717b- |
| | 483c-8e5c-c5f335d40a88/e-os-entry-1", "uni/tn-common/flt-reverse-pr_3ecd614d-717b-483c- |
| | 8e5c-c5f335d40a88/e-os-entry-2", "uni/tn-common/flt-reverse-pr_3ecd614d-717b-483c-8e5c- |
| | c5f335d40a88/e-os-entry-3", "uni/tn-common/flt-reverse-pr_3ecd614d-717b-483c-8e5c- |
| | c5f335d40a88/e-os-entry-4"]} |
| description | |
| enabled | True |
| id | 3ecd614d-717b-483c-8e5c-c5f335d40a88 |
| name | ping-policy-rule |
| policy_actions | 2070e9ff-4de9-46ea-a81e-772906982adf |
| policy_classifier_id | 88c8e3c0-d9c5-4e6b-9992-a56b539e0b98 |
| project_id | 5b8945dba07a43e0b32efea4f1bc3fdf |
| shared | False |
| status | BUILD |
| status_details | |
| tenant_id | 5b8945dba07a43e0b32efea4f1bc3fdf |
+--------------------------+------------------------------------------------------------------------------------------+
102
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Now define a Policy Ruleset to tie everything together:
[stack@dom-undercloud ~]$ gbp policy-rule-set-create icmp-policy-rule-set --policy-rules ping-policy-rule
Created a new policy_rule_set:
+------------------------+--------------------------------------+
| Field | Value |
+------------------------+--------------------------------------+
| child_policy_rule_sets | |
| description | |
| id | 29c3654d-6c9f-446b-8461-62eea3f6c050 |
| name | icmp-policy-rule-set |
| parent_id | |
| policy_rules | 9be64bdb-1d86-4577-bd3f-0bad2e9c0758 |
| shared | False |
| tenant_id | 5ab060d7c812478b904203d7901c1356 |
+------------------------+--------------------------------------+
BRKACI-3456 103
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Let’s now apply these rules to our EPGs:
[stack@dom-undercloud ~]$ gbp group-update epg101 --provided-policy-rule-sets "icmp-policy-rule-set=true"
Updated policy_target_group: epg101
[stack@dom-undercloud ~]$ gbp group-update epg102 --consumed-policy-rule-sets "icmp-policy-rule-set=true"
Updated policy_target_group: epg102
BRKACI-3456 104
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
We can now ping between the two networks!
Compute
Node1
ACI Fabric
epg101 epg102
epg101 EPG Epg102 EPGContract
Compute
Node2epg102
OVS rules do the routing andenforcement on the host
Inter host enforcement isdone on ACI leaf switches.
BRKACI-3456 105
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Shared L3 out
External Connectivity
• Connectivity for a tenant can be either shared or dedicated.
• A shared external network is visible by all OpenStack projects.
• A dedicated connectivity for the OpenStack project.
• It would be possible to have a mixed environment both with shared and dedicated external connectivity.
107BRKACI-3456
net1 net2 net3 net4
Tenant Pasta&Co Tenant Pizza&Co
Dediacated
L3 out
net1 net2 net3 net4
Tenant Pasta&Co Tenant Pizza&Co
Dediacated
L3 out
WWW WWW WWW
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Physical Layout
108BRKACI-3456
Compute2Controller
APIC
Compute1
• L3out is defined on ACI.
• The external router is defined with a dynamic or static protocol
L3Out
OSPF/
BGP/
static
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
How to create the L3out on ACI
• Although the OpenStack plugin could create automatically an L3out on ACI, the best practice is to create it manually
• Defining manually an L3out supports all the L3out features:
• VPC
• Dynamic routing protocols
• Route engineering
• Etc.
• The L3out can be created with XML templates or in any ways you are familiar with.
• Once the L3out is available, ACI AIM plugin on OpenStack can import it and start controlling the L3out.
109BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 110
Dedicated Tenant External Network
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Creation of the L3out Dedicated
111BRKACI-3456
• A dedicated L3out must be created in the OpenStack created tenant.
• In the L3out creation, it must be specified:
• Interfaces and their IP information
• Dynamic routing if any
• External EPG
• You should NOT add any contract as they will be added later automatically by the plugin.
• If you require SNAT or FIP, the L3 out must be defined in a different VRF from the one created by OpenStack!
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Query ACI for external networks
• Through the ACI Integration Module (AIM) controller, it is possible to query ACI for the existing and available external networks.
112BRKACI-3456
[heat-admin@overcloud-controller-0 ~]$ aimctl manager external-network-find
+--------------------------------------+-------------------+--------+
| tenant_name | l3out_name | name |
|--------------------------------------+-------------------+--------|
| common | l3out1 | extEpg |
| prj_4ec99ec19a0f4f00808f18d82d7032af | l3out1-DefaultVRF | extEpg |
| prj_5d0431309d5d45a1836dfa0a8beb6ef0 | l3out1-DefaultVRF | extEpg |
| prj_97390b780c7545d393d9314d34e69cfa | externalNet | extEpg |
+--------------------------------------+-------------------+--------+
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Import External Networks from ACI to OpenStack
113BRKACI-3456
[heat-admin@overcloud-controller-0 ~]$ aimctl manager external-network-get
prj_97390b780c7545d393d9314d34e69cfa externalNet extEpg
+-------------------------+--------------------------------------------------------------------------+
| Property | Value |
|-------------------------+--------------------------------------------------------------------------|
| tenant_name | prj_97390b780c7545d393d9314d34e69cfa |
| l3out_name | externalNet |
| name | extEpg |
| monitored | True |
| consumed_contract_names | [] |
| provided_contract_names | [] |
| dn | uni/tn-prj_97390b780c7545d393d9314d34e69cfa/out-externalNet/instP-extEpg |
+-------------------------+--------------------------------------------------------------------------+
+---------------+-----------------------------------------+
| Property | Value |
|---------------+-----------------------------------------|
| resource_type | ExternalNetwork |
| resource_root | tn-prj_97390b780c7545d393d9314d34e69cfa |
| sync_status | synced |
| health_score | 100 |
| id | 3e368bc8-e83d-4c8a-b269-6c7873464def |
+---------------+-----------------------------------------+
• AIM controller manager will import the external network
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Create OpenStack External Network
114BRKACI-3456
[stack@dom-undercloud ~]$ neutron net-create external-net-CL --router:external --apic:distinguished_names type=dict
ExternalNetwork=uni/tn-prj_97390b780c7545d393d9314d34e69cfa/out-externalNet/instP-extEpg
+----------------------------+----------------------------------------------------------------------------------------+
| Field | Value |
+----------------------------+----------------------------------------------------------------------------------------+
| admin_state_up | True |
| apic:distinguished_names | {"ExternalNetwork": "uni/tn-prj_97390b780c7545d393d9314d34e69cfa/out-externalNet |
| | /instP-extEpg", "BridgeDomain": "uni/tn-prj_97390b780c7545d393d9314d34e69cfa/BD-EXT- |
| | externalNet", "VRF": "uni/tn-prj_97390b780c7545d393d9314d34e69cfa/ctx-DefaultVRF", |
| | "EndpointGroup": "uni/tn-prj_97390b780c7545d393d9314d34e69cfa/ap-OpenStack/epg-EXT- |
| | externalNet"} |
| apic:nat_type | distributed |
| apic:synchronization_state | synced |
| id | f085fe67-42e1-4b3c-8951-e5d9932222ca |
| is_default | False |
| name | external-net-CL |
| port_security_enabled | True |
| provider:network_type | opflex |
| provider:physical_network | physnet1 |
| provider:segmentation_id | |
| revision_number | 4 |
| router:external | True |
| shared | False |
| status | ACTIVE |
| subnets | |
+----------------------------+----------------------------------------------------------------------------------------+
Creating neutron external network bound to the L3out imported with the aimctl manager.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
External SNAT or Floating IP Pool Definition
115BRKACI-3456
[stack@dom-undercloud ~]$ neutron subnet-create external-net-CL 10.104.21.0/24 --name ext-subnet --disable-dhcp
--gateway 10.104.21.1 --apic:snat_host_pool True
Created a new subnet:
+----------------------------+--------------------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------------------+
| allocation_pools | {"start": "10.104.21.2", "end": "10.104.21.254"} |
| apic:distinguished_names | {} |
| apic:snat_host_pool | True |
| apic:synchronization_state | N/A |
| cidr | 10.104.21.0/24 |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 10.104.21.1 |
| host_routes | |
| id | 5344832d-dd03-40d7-a4d2-3f04c86fbb9d |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | ext-subnet |
| network_id | f085fe67-42e1-4b3c-8951-e5d9932222ca |
| revision_number | 2 |
| service_types | |
| subnetpool_id | |
| tenant_id | 97390b780c7545d393d9314d34e69cfa |
+----------------------------+--------------------------------------------------+
[stack@dom-undercloud ~]$ openstack router set --external-gateway external-net-CL CLrouter
Creating neutron external network SNAT pool and attaching the router to the external net.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SNAT Pool
116BRKACI-3456
Each Hypervisor will be assigned with one IP from the pool and the VMs will be NATted with the IP of the hypervisor.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
The External network in ACI
117BRKACI-3456
The External Network EPG will be created in the tenant itself.
A contract to allow connectivity between the EPG and the L3out will be created automatically.
1. VM traffic reaches OVS
2. OVS applies NAT rules
3. The NATted IP in ACI is represented by the external EPG
4. Traffic is sent to external router through ACI
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 118
Shared Tenant External Network
BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• The shared external network must be defined in the Common tenant in ACI
Create L3 out on ACI – Shared
BRKACI-3456 119
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Same as before, you use the aimctl manager to import the external network
120BRKACI-3456
[heat-admin@overcloud-controller-0 ~]$ aimctl manager external-network-find
+--------------------------------------+--------------+--------+
| tenant_name | l3out_name | name |
|--------------------------------------+--------------+--------|
| common | l3out1 | extEpg |
| prj_11fa0c41388f4d3fbf3f2f6d6184f687 | externalNet | extEpg |
+--------------------------------------+--------------+--------+
[heat-admin@overcloud-controller-0 ~]$ aimctl manager external-network-get common l3out1 extEpg
+-------------------------+---------------------------------------+
| Property | Value |
|-------------------------+---------------------------------------|
| tenant_name | common |
| l3out_name | l3out1 |
| name | extEpg |
| nat_epg_dn | |
| display_name | |
| monitored | True |
| consumed_contract_names | [] |
| provided_contract_names | [] |
| dn | uni/tn-common/out-l3out1/instP-extEpg |
+-------------------------+---------------------------------------+
[heat-admin@overcloud-controller-0 ~]$
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
External Network
121BRKACI-3456
[stack@dom-undercloud ~]$ neutron net-create external-net-CL --router:external --shared --apic:distinguished_names
type=dict ExternalNetwork=uni/tn-common/out-l3out1/instP-extEpg
+----------------------------+----------------------------------------------------------------------------------------+
| Field | Value |
+----------------------------+----------------------------------------------------------------------------------------+
| admin_state_up | True |
| apic:distinguished_names | {"ExternalNetwork": "uni/tn-common/out-l3out1/instP-extEpg", "BridgeDomain": "uni/tn- |
| | common/BD-osp11_s2_EXT-l3out1", "VRF": "uni/tn-common/ctx-external_vrf", |
| | "EndpointGroup": "uni/tn-common/ap-osp11_s2_OpenStack/epg-EXT-l3out1"} |
| apic:external_cidrs | 0.0.0.0/0 |
| apic:nat_type | distributed |
| apic:synchronization_state | synced |
| availability_zone_hints | |
| availability_zones | |
| id | b90bfad9-4ed3-477f-996a-4222ae0768dd |
| is_default | False |
| name | external-net-CL |
| port_security_enabled | True |
| project_id | 11fa0c41388f4d3fbf3f2f6d6184f687 |
| provider:network_type | opflex |
| provider:physical_network | physnet1 |
| provider:segmentation_id | |
| revision_number | 4 |
| router:external | True |
| shared | True |
| status | ACTIVE |
| subnets | |
+----------------------------+----------------------------------------------------------------------------------------+
Creating neutron external network bound to the L3out imported with the aimctl manager.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
External SNAT or Floating pool definition
122BRKACI-3456
[stack@dom-undercloud ~]$ neutron subnet-create external-net-CL 10.104.21.0/24 --name ext-subnet --disable-dhcp
--gateway 10.104.21.1 --apic:snat_host_pool True
Created a new subnet:
+----------------------------+--------------------------------------------------+
| Field | Value |
+----------------------------+--------------------------------------------------+
| allocation_pools | {"start": "10.104.21.2", "end": "10.104.21.254"} |
| apic:distinguished_names | {} |
| apic:snat_host_pool | True |
| apic:synchronization_state | N/A |
| cidr | 10.104.21.0/24 |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 10.104.21.1 |
| host_routes | |
| id | 5344832d-dd03-40d7-a4d2-3f04c86fbb9d |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | ext-subnet |
| network_id | f085fe67-42e1-4b3c-8951-e5d9932222ca |
| revision_number | 2 |
| service_types | |
| subnetpool_id | |
| tenant_id | 97390b780c7545d393d9314d34e69cfa |
+----------------------------+--------------------------------------------------+
[stack@dom-undercloud ~]$ openstack router set --external-gateway external-net-CL CLrouter
Creating neutron external network SNAT pool and attaching the router to the exterlan net. Same way as before with dedicated network.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SNAT Pool
123BRKACI-3456
Same as before, each Hypervisor will be assigned with one IP from the pool and the VMs will be NATtedwith the IP of the hypervisor. This time the SNAT IP will appear in the Common Tenant in ACI.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Using Floating IP
124BRKACI-3456
[stack@dom-undercloud ~]$ neutron subnet-create external-net-CL 10.104.31.0/24 --name ext-subnet-FIP --allocation-pool
start=10.104.31.10,end=10.104.31.100 --disable-dhcp --gateway 10.104.31.1
+----------------------------+---------------------------------------------------+
| Field | Value |
+----------------------------+---------------------------------------------------+
| allocation_pools | {"start": "10.104.31.10", "end": "10.104.31.100"} |
| apic:distinguished_names | {} |
| apic:snat_host_pool | False |
| cidr | 10.104.31.0/24 |
| enable_dhcp | False |
| gateway_ip | 10.104.31.1 |
| host_routes | |
| id | d9bb7111-b668-4823-932d-68fa211aa69b |
| ip_version | 4 |
| name | ext-subnet-FIP |
| network_id | b90bfad9-4ed3-477f-996a-4222ae0768dd |
| project_id | 11fa0c41388f4d3fbf3f2f6d6184f687 |
| service_types | |
| tenant_id | 11fa0c41388f4d3fbf3f2f6d6184f687 |
+----------------------------+---------------------------------------------------+
[stack@dom-undercloud ~]$
Creating floating IP is as simple as adding another subnet to the external network.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Floating IP in ACI
125BRKACI-3456
Floating Subnet will be visible in ACI and when you assign a FIP to a VM this will appear in the operational tab of the external EPG.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 127
Creation of opflex networks.
BRKACI-3456
Demo Time!
Binding of OpenStack VMs to those networks.
Adding connectivity to a bare metal server and a vSphere virtual machine.
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
controller compute1compute2 ESXi Bare Metal
BRKACI-3456 128
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
APIC
APIC
APIC
controller compute1compute2 Bare Metal
EPG green-OS
EPG Orange-OS
Bridge Domain Orange 192.168.100.254/24
Bridge Domain Green 192.168.200.254/24
ESXi
BRKACI-3456 129
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
APIC
APIC
APIC
controller compute1compute2 ESXi Bare Metal
EPG green-OS
EPG Orange-OS
EPG green-mixed
Bridge Domain Green 192.168.200.254/24
Bridge Domain Orange 192.168.100.254/24
BRKACI-3456 130
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
APIC
APIC
APIC
controller compute1compute2 ESXi Bare Metal
EPG green-OS
EPG Orange-OS
EPG green-mixed
Bridge Domain Green 192.168.200.254/24
Bridge Domain Orange 192.168.100.254/24
BRKACI-3456 131
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
APIC
APIC
APIC
controller compute1compute2 ESXi Bare Metal
EPG green-OS
EPG Orange-OS
Bridge Domain Green 192.168.200.254/24
Bridge Domain Orange 192.168.100.254/24
Contract allow-ICMP
Allow ICMP
Contract allow-SSH
Allow TCP:22
EPG green-mixed
BRKACI-3456 132
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Are we there yet?
134BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
ACI connects Virtual and Physical World
135BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Documentation
• APIC OpenStack Plugin Installation Guides:
• http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Architectural_Overview.html
• http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Deployment_Guide_for_Red_Hat.html
• http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/openstack/b_ACI_with_OpenStack_OpFlex_Deployment_Guide_for_Ubuntu.html
137BRKACI-3456
For YourReference
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Documentation (Cont.)
• APIC GBP Plugin Datasheet:
• http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/openstack-at-cisco/datasheet-c78-734181.html
• APIC OpenStack Plugin Datasheet:
• http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/openstack-at-cisco/datasheet-c78-732353.html
• GBP WhitePaper:
• http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-733126.html
• GBP wiki:
• https://wiki.openstack.org/wiki/GroupBasedPolicy
For YourReference
BRKACI-3456 138
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco Spark
Questions? Use Cisco Spark to communicate with the speaker after the session
1. Find this session in the Cisco Live Mobile App
2. Click “Join the Discussion”
3. Install Spark or go directly to the space
4. Enter messages/questions in the space
How
cs.co/ciscolivebot#BRKACI-3456
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Please complete your Online Session Evaluations after each session
• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt
• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations
Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.
Complete Your Online Session Evaluation
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Tech Circle
• Meet the Engineer 1:1 meetings
• Related sessions
141BRKACI-3456