Upload
gseltzer
View
1.072
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Annual Top Gun: DFA
Citation preview
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 1
Dynamic Fabric Automation
February 2014
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 2
DFA – Agenda
DFA Requirements and Functions
Optimized Network
Virtual Fabrics
Fabric Management
Workload Automation
Platform Support
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 3
What is DFA?
If you ask 10 People, you get 11 answer!
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 4
What is DFA?
DFA is Evolution NOT Revolution!
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 5
What is DFA?
DFA are Enhancements to- Simplify- Optimize- Automate
the Unified Fabric!
Most likely your customers already use DFA
Or at least one function of it
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 6
#1: Fabric Management
Advantages• Device Auto-Configuration
• Cabling Plan Consistency Check
• Common point of fabric access
• Network, Host & Workload Visibility
TFTP Services
DHCP Services
XMPP Server
LDAP
Message Broker
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 7
#2: Workload Automation
Advantages• Any workload, anywhere,
anytime
• Open orchestration integration
• Automated scalable provisioning
• Policy based Provisioning
Network Services Controller
Flow Controller
Fabric MgmtProvisioning
Open APIs
Published Schemas
Network & Network Services Policies
Cloud Stacks
Compute & Storage Policies
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 8
#3: Optimized Networking
Advantages• Optimized for L2-L7
• Reduced failure domains
• Extensible scale & resiliency
• Interoperability with other architectures
Any/all subnets on any leaf
Any/all leaf distributed default gtwy
N Spines/Paths + scale-out model
STP/VPCEnvironments
Nexus 2K, 3K, 5K, 6K, 7K
FC/FCoEEnvironments
MDS, Nexus 5K, 7K
Nexus 2K, 5K, 6K, 7K
FabricpathEnvironments
L3 Environments
Nexus 3k, 5K, 6K, 7K
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 9
#4: Virtual Fabrics
Advantages• Any VLAN / Subnet
Anywhere
• Scalable secure virtual fabrics
• Virtual fabric tenant visibility
• Physical-virtual integration
HR Finance
Manufacturing Sales
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 10
Programmable Fabric
Orchestration
Cisco UCS Director Openstack Custom Built
Multi-Hypervisor Support
Physical
Integrated Management
One Controller
Hyper-V
N1KV
KVM XEN ESX
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 11
Hardware & Software Requirements
Product Function Software Version
Nexus 6000Leaf, Border-Leaf, Spine, Route-Reflector 7.0(0)N1(0.513)
Nexus 7000Leaf: F3**, Border-Leaf: F3**Spine: F2, F2e, F3*Route-Reflector*
6.2(6)
Nexus 7700Leaf:F3**, Border-Leaf: F3**Spine: F2, F2e, F3*Route-Reflector*
6.2(6)
DCNM (CPOM)
Fabric Management incl. DHCP, TFTP, XMPP 7.0.(1.S23)
Nexus 1000vVirtual Switch with VDP-Signaling(FCS: VMWare vSphere, other Hypervisor coming soon)
4.2(1)SV2(2.2)
*requires NX-OS 6.2(6) / **requires NX-OS 7.1(x) planned for Q2 CY’14
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 12
License Requirements
Nexus 7000 / 7700> Enhanced Layer-2
(ENHANCED_LAYER2_PKG )
> Enterprise Services (LAN_ENTERPRISE_SERVICES_PKG)
Nexus 6000> Enhanced Layer-2
(ENHANCED_LAYER2_PKG)
> Layer-3 Base (LAN_BASE_SERVICES_PKG)
> Layer-3 Enterprise (LAN_ENTERPRISE_SERVICES_PKG)
Nexus 5500> Enhanced Layer-2
(ENHANCED_LAYER2_PKG)
It is required to install the Switch Feature Licenses before configuring DFA!
Please ensure that Nexus 6000 Layer-3 Base & Enterprise License is installed!
n6k# show license usageFeature Ins Lic Status Expiry Date Comments Count--------------------------------------------------------------------------------FCOE_NPV_PKG No - Unused -FM_SERVER_PKG No - Unused -ENTERPRISE_PKG No - Unused -FC_FEATURES_PKG No - Unused -VMFEX_FEATURE_PKG No - Unused -ENHANCED_LAYER2_PKG Yes - In use Never -LAN_BASE_SERVICES_PKG Yes - In use Never -LAN_ENTERPRISE_SERVICES_PKG Yes - In use Never ---------------------------------------------------------------------------------n6k#
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 13
Dynamic Fabric Automation – Modular Building Blocks
Fabric Management
Workload Automation
Virtual FabricsOptimizedNetworking
Bundled functions are Modular, Flexible and follows your Choice of Integration
and Speed of Adoption!
Optimized Networking
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 15
Optimized Networking
Advantages• Any subnet, anywhere,
rapidly
• Reduced Failure Domains
• Extensible Scale & Resiliency
• Profile Controlled Configuration
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 16
FabricPath – An Ethernet Fabric
Connect a group of switches using an arbitrary topology With a simple CLI, aggregate them into a Fabric:
An open protocol based on Layer 3 technology provides Fabric-wide intelligence and ties the elements together
FabricPath
N7K(config)# interface ethernet 1/1N7K(config-if)# switchport mode fabricpath
Flexible Topologies, Easy Configuration
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 17
Optimal, Low Latency Switching
Single address lookup at the ingress edge identifies the exit port across the fabric
Traffic is then switched using the shortest path available Reliable L2 connectivity any to any
(as if it was the same switch, no STP inside)
FabricPathe1/2
A B
s3 s8e1/1
MAC IF
A e1/1
… …
B s8, e1/2
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 18
High Bandwidth, High ResiliencyEqual Cost Multipathing
Mutipathing (up to 256 links active between any 2 devices)
Traffic is redistributed across remaining links in case of failure, providing fast convergence
e1/2
A B
e1/1 s3 s8
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 19
Unmatched ScaleConversational Learning
Per-port mac address table only needs to learn the peers that are reached across the fabric
A virtually unlimited number of hosts can be attached to the fabric
FabricPath
A B
s3 s8
MAC IF
A s1,e1/1
… …
B e1/2
MAC IF
… …
s5
MAC IF
A e1/1
… …
B s8, e1/2
e1/1 e1/2
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 20
Automatic VLAN PruningReducing Admin Workload and Mistakes
V10 V10 V10V20 V20V30 V30V30
FabricPath
V10 V20 V30
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 21
Multiple TopologiesVirtual Fabrics within a Fabric
Topologies are used for static VLAN pruning, security, traffic engineering
Topology: a group of links in the Fabric By default, all links are part
of topology 0 Links can be assigned to
several topologies An IS-IS routing table is built
per topology A VLAN is mapped to a
unique topology
Topology 0
Topology 1
Topology 2
FabricPath
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 22
Legacy L2 IntegrationvPC+
Allows extending vlans with no limitation (no risks of loop)
Devices can be attached active/active to the fabric using IEEE standard port channels and without resorting to STP
Legacy L2 device support not limited to Cisco products
FabricPath
A
s3 s8s7
B
s4
VLAN XVLAN YVLAN Z
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 23
Edge Devices IntegrationHosts can leverage multiple L3 default gateways
Hosts see a single default gateway
The fabric provide them transparently with multiple simultaneously active default gateways
Allows multipathing to extend from inside the fabric to the L3 domain outside the fabric
FabricPath
A
s3
dgdgL3
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 31
Scaling with FabricPathExample: 2,048 x 10GE server design
16X improvement in bandwidth performance 6 to 1 consolidation (from 74 managed devices to 12 devices) 2X+ increase in network availability Simplified IT operations (fewer devices, vlans anywhere)
Traditional Spanning Tree Based Network FabricPath Based Network
Fu
lly No
n-B
lockin
g
2, 048 Servers8 Access Switches64 Access Switches
Blocked Links
Ove
rsu
bsc
rip
tio
n 1
6:1
8:1
2:1
4 Pods
FabricPath
2, 048 Servers
Optimized Networking
Beyond FabricPath
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 34
FabricPath vs DFA
IS-IS feature support FabricPath DFA
Control Plane Yes Yes (Switch connectivity only)
Host connectivity MAC learning based BGP based
ARP flooding Yes IP-MAC binding carried by IS-IS
GM LSP announcement All FP enabled VLANs Only legacy VLANs
Emulated switch Yes Yes
Configurable Multi-destination trees
Yes Yes (for base topology)
Anycast HSRP Yes Support exists
Multi-topology Yes Not for current release
Software Any version that supports FP code since Cairo release
Iluka release onwards
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 35
Connecting Switches for DFA
WAN
RR RR
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
Reference Topology
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 36
Connecting Switches for DFA
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
WAN
RR RR
Scale-Out to fit your needs
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 37
Connecting Switches for DFA
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
WAN
RR RR
WAN
RR
RR
Flexible Topologies
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 38
CLOS
= DFA-Spine = DFA-Leaf = DFA-BorderLeaf
WAN
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 39
Fabric Interfaces
= DFA-Spine = DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
WAN
RR RR
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 40
Host Interfaces
= DFA-Spine = DFA-Leaf = Host Interfaces
= DFA-BorderLeaf
WAN
RR RR
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 41
Fabric Control Plane
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
IS-IS as Fabric Control Plane
IS-IS for Fabric Link-State distribution
Fabric Node reachability for overlay Encapsulation (FabricPath)
Building Multi-Destination Trees for Multicast/Broadcast traffic
Quick reaction to Fabric Link/Node failure
Enhanced for mesh topologiesWAN
RR RR
IS-IS
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 42
Fabric Control Plane
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
WAN
RR RR
MP-BGP
Host and Subnet Route Distribution
Host Route Distribution decoupled from the Fabric link state protocol
Use MP-BGP on the leaf nodes to distribute internal Host/Subnet routes and external reachability information
MP-BGP also used to distribute IP multicast groups information
MP-BGP enhancements to carry up to 100s of thousands of routes and reduce convergence time
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 43
Fabric Control Plane
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
Host Originated Protocols
ARP, ND, IGMP, LLDP, DHCP originated on servers are terminated on Leaf nodes
Contain floods and failure domains, distribute control packet processing
> Unknown unicast is dropped on leaf: fabric knows all hosts/subnets (DFA Proxy-Gateway)
> Non ARP broadcast is flooded in the segment: ACL override can drop them
> Per VNI/Segment override to allow ARP flooding: handles silent servers (DFA Anycast-Gateway
WAN
RR RR
ARP ,ND , IGMP, LLDP, DHCP with endpoints
PIM, IGP, eBGP
toward the L3 Network
Domain
Terminate PIM, OSPF, eBGP from external networks on Border Leafs
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 44
Connecting Switches with DFA
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
= Distributed Gateway
WAN
RR RR
Distributed Gateway exists on all DFA-Leaf where VLAN/Segment-ID is active
There are different DFA Forwarding Modes for the Distributed Gateway:
Proxy-Gateway (Enhanced Forwarding)> Leverages proxy-ARP
> Intra- and Inter-Subnet forwarding based on Routing
> Contain floods and failure domains to the Leaf
Anycast-Gateway (Traditional Forwarding)> Intra-Subnet forwarding based on FabricPath
> Layer-2 lookup is performed at the leaf
> Data-plane based conversational learning for endpoints MAC addresses
> ARP is flooded across the fabric
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 45
Connecting Switches for DFA – Proxy-Gateway
WAN
RR RR
interface vlan 123 vrf member Coke fabric forwarding mode proxy-gateway ip address 10.1.1.1/24 ip dhcp relay address 200.200.200.100 no shutdown
vlan 123 mode fabricpath vn-segment 30000
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
= Distributed Gateway
Proxy-Gateway (enhanced Forwarding)> Leverages proxy-ARP
> Intra- and Inter-Subnet forwarding based on Routing
> Contain floods and failure domains to the Leaf
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 46
Connecting Switches for DFA – Anycast-Gateway
WAN
RR RR
interface vlan 123 vrf member Coke fabric forwarding mode anycast-gateway ip address 10.1.1.1/24 ip dhcp relay address 200.200.200.100 no shutdown
vlan 123 mode fabricpath vn-segment 30000
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
= Distributed Gateway
Anycast-Gateway (Traditional Forwarding)> Intra-Subnet forwarding based on FabricPath
> Layer-2 lookup is performed at the leaf
> Data-plane based conversational learning for endpoints MAC addresses
> ARP is flooded across the fabric
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 48
DFA Forwarding Modes Comparison
Proxy-Gateway Anycast-Gateway Non-DFA Mode*
VLAN/Subnets stretched between
leaves✓ ✓ ✓
(requires anchor Leaf)
Common Anycast GW IP across leaves ✓ ✓ ✗
Common Anycast GW MAC across leaves ✓ ✓ ✗
Use Proxy-ARP/ND✓
(respond to ARP/ND only if the destination is available in
the RIB)
✗ ✗
ARP Flooding in Layer-2 Domain ✗ ✓
(floods also across DFA Fabric)
✓(local flood only)
Intra-Subnet forwarding
Always routed(TTL decrement)
Bridged Bridged
Silent Host Discovery ✗ ✓ ✓
* VLANs/IP Subnets are only locally defined behind a DFA leaf (or a pair of vPC peer leaves)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 49
Interface Consideration
= DFA-Spine RR = DFA Route-Reflector
= DFA-Leaf = Fabric Interface
= DFA-BorderLeaf
WAN
RR RR
e1/5
e1/5 e1/5
e1/6
e1/5 e1/5
e1/7e1/8
Consistently use the Fabric facing Interfaces to maximize simplification during POAP Definition
On DFA-Leafs use the same Interfaces for Fabric Interfaces, VPC Peer-Link, Host Interfaces and FEX uplinks
On DFA-Spine, use the same Interfaces for Fabric Interfaces
Optimized Networking
DFA and the Nexus 5500
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 51
DFA and Nexus 5500 (co-existence)
H1: 10.1.1.11/24 H2: 192.168.11.22/24 H4: 10.1.1.44/24H3: 192.168.11.33/24
= Host Interface= L2-only DFA-Leaf
= Fabric Interface
= Full DFA-Leaf
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 52
DFA and Nexus 5500 (co-existence)
= Host Interface= L2-only DFA-Leaf
= Fabric Interface
= Full DFA-Leaf
H1: 10.1.1.11/24(VLAN 101)
H4: 10.1.1.44/24(VLAN 101)
vlan 101 mode fabricpath
Intra-Subnet Forwarding based on FabricPath only
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 53
DFA and Nexus 5500 (co-existence)
= Host Interface= L2-only DFA-Leaf
= Fabric Interface
= Full DFA-Leaf
H1: 10.1.1.11/24(VLAN 101)
H4: 10.1.1.44/24(VLAN 101)
H2: 192.168.11.22/24(VLAN 221)
Inter-Subnet Forwarding over Anchor-Leaf(Single or VPC+ Domain)
Full DFA-Leaf act as Anchor-Leaf with Gateway for all L2-only DFA-Leaf (e.g. N5k)
vlan 101 mode fabricpathvlan 221 mode fabricpath
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 54
DFA and Nexus 5500 (co-existence)
= Host Interface= L2-only DFA-Leaf
= Fabric Interface
= Full DFA-Leaf
H1: 10.1.1.11/24(VLAN 101)
H4: 10.1.1.44/24(VLAN 101)
H2: 192.168.11.22/24(VLAN 221)
interface vlan 101 vrf member Coke fabric forwarding mode anycast-gateway ip address 10.1.1.2/24 no shutdown hsrp version 2 hsrp group 101 ip 10.1.1.1 mac-address 2020.0000.00AA
vlan 101 mode fabricpathvlan 221 mode fabricpath
Anchor-Leaf requires static Configuration with HSRP in addition to the “fabric forwarding mode anycast-gateway”
Anycast-MAC required for Distributed Gateway
Maximum of 2 Anchor-Leaf per VLAN with vPC+ Configuration for Active/Active Gateway
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 55
DFA and Nexus 5500 (co-existence)
= Host Interface= L2-only DFA-Leaf
= Fabric Interface
= Full DFA-Leaf
H1: 10.1.1.11/24(VLAN 101)
H4: 10.1.1.44/24(VLAN 101)
H2: 192.168.11.22/24(VLAN 221)
interface vlan 101 vrf member Coke fabric forwarding mode anycast-gateway ip address 10.1.1.2/24 no shutdown hsrp version 2 hsrp group 101 ip 10.1.1.1 mac-address 2020.0000.00AA
vlan 101 mode fabricpathvlan 221 mode fabricpath
interface vlan 221 vrf member Coke fabric forwarding mode anycast-gateway ip address 192.168.11.2/24 no shutdown hsrp version 2 hsrp group 221 ip 192.168.11.1 mac-address 2020.0000.00AA
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 56
DFA and Nexus 5500 (co-existence)
= Host Interface= L2-only DFA-Leaf
= Fabric Interface
= Full DFA-Leaf
H1: 10.1.1.11/24(VLAN 101)
H2: 192.168.11.22/24(VLAN 3001)
H4: 10.1.1.44/24(VLAN 101)
H3: 192.168.11.33/24(VLAN 3005)
vlan 101 mode fabricpath
interface vlan 101 vrf member Coke fabric forwarding mode anycast-gateway ip address 10.1.1.2/24 ip dhcp relay address 200.200.200.100 no shutdown
vlan 3001 or 3005 mode fabricpath vn-segment 30531
interface vlan 3001 or 3005 vrf member Coke fabric forwarding mode proxy-gateway ip address 192.168.11.1/24 ip dhcp relay address 200.200.200.100 no shutdown
Co-Existence of Enhanced- and Traditional-Forwarding allowed on a per SVI base
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 57
DFA and Nexus 5500 (co-existence)
= Host Interface= L2-only DFA-Leaf
= Fabric Interface
= Full DFA-Leaf
H1: 10.1.1.11/24(VLAN 101)
H2: 192.168.11.22/24(VLAN 3001)
H4: 10.1.1.44/24(VLAN 101)
H3: 192.168.11.33/24(VLAN 3005)
Vlan 101 mode fabricpath vn-segment 54321
interface vlan 101 vrf member Coke fabric forwarding mode proxy-gateway ip address 10.1.1.1/24 ip dhcp relay address 200.200.200.100 no shutdown
vlan 3001 or 3005 mode fabricpath vn-segment 30531
interface vlan 3001 or 3005 vrf member Pepsi fabric forwarding mode proxy-gateway ip address 192.168.11.1/24 ip dhcp relay address 200.200.200.100 no shutdown
After last L2-only DFA-Leaf has been removed, proxy-gateway mode could be used
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 58
DFA and Nexus 5500 (co-existence) No default gateway presence on
L2-only DFA-Leaf (Nexus 5500)
No Segment-ID support> All Nexus 5500 involved VLANs are
non-Segment-ID enabled across all DFA-Leafs
Reverts back to traditional FabricPath for forwarding
L2 lookup is performed at the L2-only DFA-Leaf
> Data-Plane based conversational learning for endpoints MAC addresses
ARP is flooded across the fabric
Routing performed at Anchor-Leaf which could be every Full DFA-Leaf in Forwarding-Mode “Anycast-Gateway” (maximum 2 per VLAN)
RR
H1: 10.1.1.11/24(VLAN 101)
H3: 192.168.11.33/24
Optimized Networking
Connecting Servers for DFA
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 61
Connecting Servers and/or FEX for DFA
Valid Server connection models are:> Single-Homed Server with single Link to one
DFA-Leaf (1)
> Single-Homed Server with Port-Channel to one DFA-Leaf (2)
> Dual-Homed Server with Active/Standby Link to two DFA-Leafs (3)
> Dual-Homed Server with virtual Port-Channel to two DFA-Leafs (vPC+ Domain) (4)
> All Options with either Access-Port or 802.1q Trunk
Valid FEX connection models are*:> Single-Homed FEX with single Link to one
DFA-Leaf (5)
> Single-Homed FEX with Port-Channel to one DFA-Leaf (6)
> Dual-Homed FEX with virtual Port-Channel to two DFA-Leafs (vPC+ Domain) (7)
> eVPC with FEX A/A and Dual-Homed Server with Active/Standby or Port-Channel (8)
Always connect Servers to DFA-Leaf or hybrid DFA-Leaf/BorderLeaf only (not Spine!)
WAN
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 62
Connecting Servers and/or FEX for DFA
WAN
1 2 3 4
5 6 7 8
Virtual Fabrics
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 64
Virtual Fabrics
Advantages• Any workload, any
vFabric, rapidly
• Scalable Secure vFabrics
• vFabric Tenant Visibility
• Routing/Switching Segmentation
HR Finance
Manufacturing Sales
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 67
What is a Segment-ID?
FabricPath Frame Format
Integrated Fabric Frame Format
Segment-ID = 802.
1Q802.1Q
Traditionally VLAN space is expressed over 12 bits (802.1Q tag)
> Limits the maximum number of segments in a datacenter to 4096 VLANs
The Segment-ID solution consists in using a double 802.1Q tag for a total address space of 24 bits, allowing for the support of ~16M L2 segment
Segment-ID is added/removed by the DFA Leaf nodes and is part of the Layer-2 Header
DFA Spines usually forward traffic based on FabricPath Switch-ID values, but can prune multi-destination traffic by parsing the segment-ID field
> Segment-ID is hardware-based innovation offered by DFA leaf and spine nodes
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 68
802.1Q Tagged Traffic to Segment-ID Mapping
Segment-IDs are utilized for providing isolation at Layer-2 and Layer-3 across the DFA Fabric
802.1Q tagged frames received at the Leaf nodes from edge devices must be mapped to specific Segments
The VLAN-Segment mapping can be performed on a Leaf device level
VLANs become locally significant on the Leaf node and 1:1 mapped to a Segment-ID
Segment-IDs are globally significant, VLAN IDs are locally significant
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 69
Virtual Fabrics – L2 Flows
1. H1 sends a packet to H2 traffic between the vSwitch and the Leaf is tagged with a local VLAN-ID 10
2. L2 lookup is performed by L1 in the MAC Table for the Segment-ID associated to VLAN 10 (5000)
3. L1 adds the L2 and FP headers before sending the packet into the fabric. The Segment-ID associated to VLAN 10 is added inside the L2 header
4. L4 receives the frame and performs the L2 lookup by looking at the Segment-ID value. It then sends it to H2 using a local VLAN-ID 20
H1 H2
vSwitchvSwitch
L1 L4
e1/1
SMAC→ H1_MAC
DMAC→ H2_MAC
SSID→ L1
DSID→ L4
[Segment-ID = 5000]
3
SMAC→ H1_MAC
DMAC→ H2_MAC
[VLAN = 10]
1
VLAN 10 <-> Segment-ID 5000
H2_MAC L4 SW_ID
2
SMAC→ H1_MAC
DMAC→ H2_MAC
[VLAN = 20]
VLAN 20 <-> Segment-ID 5000
H2_MAC e1/1
4
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 70
How are Segment-IDs Utilized?
Each IP Subnet defined at the Leaf of the DFA Fabric is associated to a Layer-2 Domain, which is represented by a Segment-ID
Multiple Segments can be defined for a given Tenant, mapped to a Layer-3 VRF and uniquely identifying that Tenant
A dedicated Segment-ID value uniquely identifies each VRF defined in the DFA Fabric
Note: Every Segment-ID will always be mapped to a traditional VLAN on a Leaf Switch. The “system fabric core-vlans” range will be used for the Layer-3 VRF Segment-Id
Blue TenantVRF: BlueSegment-ID 6000
Green TenantVRF: GreenSegment-ID 6001
Segment-ID 500010.0.0.0/24
Segment-ID 500111.1.2.0/24
Segment-ID 5002192.168.12.0/24
Segment-ID 502010.0.0.0/24
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 71
Virtual Fabrics – Fabric Routed Flows
H110.10.10.10
H210.10.10.20
vSwitch
1. H1 sends a packet to H2 traffic between the vSwitch and the Leaf is tagged with a local VLAN-ID 10
2. L3 lookup is performed by L1 in the context of the Blue VRF
3. L1 adds the L2 and FP headers before sending the packet into the fabric. The Segment-ID identifying the Blue VRF is added inside the L2 header
4. L4 receives the frame and associates it to the Blue VRF by looking at the Segment-ID value. It then sends it to H2 using a local VLAN-ID 20
Note: this behavior applies to all fabric routed flows (intra-subnet or inter-subnet)
vSwitch
L1 L4
e1/1
SMAC→ L1_MAC
DMAC→ L4_MAC
SIP→ 10.1.1.10
DIP→ 10.1.1.20
SSID→ L1
DSID→ L4
[Segment-ID = 6000]
3
SMAC→ H1_MAC
DMAC→ G_MAC
SIP→ 10.1.1.10
DIP→ 10.1.1.20
[VLAN = 10]
1
Blue_VRF <-> Segment-ID 6000
10.10.10.20 NH L4_IP
2
SMAC→ G_MAC
DMAC→ H2_MAC
SIP→ 10.1.1.10
DIP→ 10.1.1.20
[VLAN = 20]
Blue_VRF <-> Segment-ID 6000
10.10.10.20 e1/1
4
DFA - Platform Support
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 74
Cisco Dynamic Fabric Automation – Platform Support
High Density Spine
Medium Density Spine
Fabric Extenders
ToR LeafsHigh Density
Leafs
Network
Services Controller
Compute &
Storage
Network Services
DCNM/CPoM
Nexus 7X00 (F2/F2e/F3)
Nexus 6004
Nexus 6001
Nexus 2x00
Nexus 6004
Fabric Extenders
Nexus 2x00
Cloud Stacks & Orchestration Tools
Leaf / Border Leaf
Nexus 7X00 (F3)
Nexus 1kv, OVSVirtual
Networking
Virtual SwitchNexus 1000v OVS
Nexus 5596/5548
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 75
DFA availability at FCS
L3
L2
vSwitchN1kv
N7k-S1 N7k-S2 N6k-S3 N6k-S4
N5k-1 N6k-4N6k-2 N6k-3 N6k-6
N2k
Nexus 7000 (F2/F2e) and Nexus 6000 as Full DFA-Spine – Full Co-Existence Support!
Nexus 6000 as Full DFA-Leaf; supporting all the Functionalities
Nexus 2000 FEX Support at every kind of DFA-Leaf (Full or L2-only)
Nexus 5500 as L2-Only DFA-Leaf (no Segment-ID support)
Nexus 1000v enhancing Virtual Workload with VDP-Signalization
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 76
Platform to DFA-Pillar Support at FCS?
PlatformFabric
ManagementWorkload
AutomationOptimized
NetworkingVirtualized
Fabrics
Nexus 6000 ✓ ✓ ✓ ✓
Nexus 5500 ✓ ✗ ✓(1,3) ✓(1,3)
Nexus 7000 (M) ✓ ✗ ✗ ✗
Nexus 7k/7.7k (F2/F2e)
✓ ✗ ✓(2) ✓(2)
Nexus 3000 ✗ ✗ ✗ ✗Nexus 1000v ✓ ✓ ✓ ✗
1No Segment-IDs 2Spine 3Layer-2 only
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 77
DFA - Openstack and UCS Director Support
UCS Director support – work in progress • FCS Target 1HCY2014
Openstack support – work in progress• EFT2 will support openstack plugin
• Work in progress to upstream openstack plugin DFA support
• Work in progress to have Canonical and Redhat to support openstack plugin for DFA
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 78
Cisco Dynamic Fabric Automation - Roadmap
• DFA Spine, Leaf, Border Leaf (VXLAN Encap)
• FCoE Support with DFA• Workload Automation Support on
N5K/N6K for Fabricpath/vPC deployments
Nex
us 6
KN
exus
700
0
1HCY2014 (Commit Pending)
2HCY2014+(Commit Pending)• DFA Leaf, Border Leaf (VXLAN
Encap)• DFA Spine (NVGRE Encap)
• MAC Learning via Control Plane (evpn support)
• Workload Automation for existing Fabricpath/vPC deployments
• DFA Leaf, Border Leaf (Fabricpath Encap)
• DFA Spine (VXLAN Encap)• Fabric Management support for
Fabricpath/vPC
• DFA Leaf, Border Leaf (Fabricpath Encap)
• DFA Spine (VXLAN Encap)• Fabric Management support for
Fabricpath/vPC
• DFA Leaf, Border Leaf (NVGRE Encap)
• MAC Learning via Control plane (evpn support)
• DFA Spine, Leaf, Border Leaf (Fabricpath Encap)
• Fabric Management support on N5K/N6K for vPC/Fabrcipath
deployments
• F3-Series card • DFA Spine with F2/F3 (Fabricpath
Encap)• PoAP support for traditional
Fabricpath deployments
• F3-Series card • DFA Spine with F2/F3 (Fabricpath
Encap)• PoAP support for traditional
Fabricpath deployments
4QCY2013(EFT Starting soon)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 79
DFA – Services Roadmap
OpenStack
DCNM
Neutron
PNSC (Services)N1kv
vSphere
Q4CY13 Q1CY14 Q2CY14 2HCY14
UCSD CIAC
• Turnkey Mgt OVA • Automation API
(REST)• POAP• Auto-Configuration• OpenStack Ph. 1• VCD Agent• Services [PNSC] VSG, ASA1000V, CSR100V, VPX/1000v• VM/Segment Tracking
• OpenStack Ph. 2 (Community + Canonical)
• Unified OpenStack Plug-In• [DFA + Nexus 1000v]• Services Ph. 2 [PNSC] vASA, Citrix SDX, ASA55xx, Framework• DCI support• REST Updates• Scale
ESXi
• Additional Services [F5]
• Device Updates
Orchestration
Controller
Hypervisors
Network & Service
VCD
VSG ASA1000V CSR1000vVPX1000v
SCVMM
HyperV
OpenStack
KVM
Components
Roadmap
N6k/N7k/5k
SC-OM CloudStack
Fabric Management
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 81
Hardware, Software & License Requirements
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 82
Hardware & Software Requirements
Product Function Software Version
Nexus 6000Leaf, Border-Leaf, Spine, Route-Reflector 7.0(0)N1(0.513)
Nexus 7000Leaf: F3**, Border-Leaf: F3**Spine: F2, F2e, F3*Route-Reflector*
6.2(6)
Nexus 7700Leaf:F3**, Border-Leaf: F3**Spine: F2, F2e, F3*Route-Reflector*
6.2(6)
DCNM (CPOM)
Fabric Management incl. DHCP, TFTP, XMPP 7.0.(1.S23)
Nexus 1000vVirtual Switch with VDP-Signaling(FCS: VMWare vSphere, other Hypervisor coming soon)
4.2(1)SV2(2.2)
*requires NX-OS 6.2(6) / **requires NX-OS 7.1(x) planned for Q2 CY’14
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 83
Nexus 7000 / 7700> Enhanced Layer-2
(ENHANCED_LAYER2_PKG )
> Enterprise Services (LAN_ENTERPRISE_SERVICES_PKG)
Nexus 6000> Enhanced Layer-2
(ENHANCED_LAYER2_PKG)
> Layer-3 Base (LAN_BASE_SERVICES_PKG)
> Layer-3 Enterprise (LAN_ENTERPRISE_SERVICES_PKG)
Nexus 5500> Enhanced Layer-2
(ENHANCED_LAYER2_PKG)
It is required to install the Switch Feature Licenses before configuring DFA!
Please ensure that Nexus 6000 Layer-3 Base & Enterprise License is installed!
License Requirements
DCNM- Advanced License in NOT required….but recommended
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 84
Components of DCNM OVA for Enhanced Fabric Management
The DCNM OVA (CPOM), is intended to
be a new product from the DCNM family of products.
There is no upgrade path from DCNM
4.x,5.x, and 6.x train to this
DCNM version.
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 85
Components of DCNM OVA for Enhanced Fabric Management
What is an OVA and what is a DCNM OVA ?
A virtual appliance, in general, is a pre-built software solution, comprised of one or more virtual machines that is
packaged, maintained, updated, and managed as a unit.
The DCNM virtual appliance has a pre-installed operating system (CentOS 6.3) with pre-installed DCNM (Cisco Data Center Network Manager). It also provides an option to install additional packages to manage Enhanced Fabric architecture on demand.
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 86
Centralized Point of Management (CPOM)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 87
Common Point of Fabric Access
DFA Centralized Point of Management (CPOM)
> DCNM Fuji Release (7.0)> DHCP-Server> TFTP> XMPP> LDAP> Message Broker
Virtual Appliance for vSphere
All Functions packaged and pre-installed in ONE single OVA!
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 88
Common Point of Fabric Access
Welcome Screen provides easy access to
LicensingPOAP
Performance CollectionDocumentation
Menu structure with access to CPOM Functions, Configuration and Administration
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 89
Common Point of Fabric Access
Health Status and Event Overview
Summary Dashboard showing all Health, Inventory, Topology and Performance
Collection Information
Automatic Discovered Topology with Load and
Health information
Detailed Performance Collection for Top Access-Port, ISL/Trunk-Port & CPU
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 90
Common Point of Fabric Access
Detailed Port Information
available on Mouse-Over
DFA Dashboard showing Leaf/Spine Topology incl. Status and active Links
Selected Node with all active Links and
Status
Search for Switch and discovered
Server (virtual and physical)*
Pull-down to change view to selected virtual
Fabric
*requires VDP
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 91
Common Point of Fabric Access
Detailed Port Information available on Mouse-Over
DFA Dashboard showing Leaf/Spine Topology incl. Status and active Links
Selected Node with all active
Links and Status
Search for Switch and discovered
Server (virtual and physical)
Pull-down to change view to selected
virtual Fabric
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 92
Connecting DCNM (CPOM) for DFA
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 93
The management connectivity for DFA must come through the NXOS device management interface (mgmt0)
The management port for any given switch are connected to a the same management subnet that will include the DCNM (CPOM) “Fabric Management” interface
During Power-On Auto Provisioning (POAP) the device makes a DHCP request that is locally scoped and the DCNM (CPOM) DHCP server responds with a temporary (bootstrap) IP address
Preparation for DCNM (CPOM) deployment
mgmt0
con0
hostname
Fabric Manageme
nt
Serial Console Access
DCNM (CPOM)D
HC
P,TF
TP,S
CP,L
DA
P,XM
PP
DCNM
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 94
DCNM Access for User-Access to the DCNM (CPOM) WebUI or DCNM via Fat-Client
> Configuration of this Interfaces requires:
> IP Address
> Netmask
> Gateway
> DNS-Server
Fabric Management for Access to the Network Switch Out-of-Band Management (mgmt0)
> Configuration of this interface requires:
> IP Address
> Netmask
> DNS-Server
Connecting DCNM (CPOM) for DFA
Fabric ManagementDCNM (CPOM)DCNM
Access
Access for DCNM (CPOM) Users
Fabric Management
Access to CPOM/DCNM managed Fabric
DCNM
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 95
Connecting Switches for Fabric Management
Console Connection is recommended but not
required
con0
Fabric Management - Out-of-Band (OOB) Network
mgmt0
con0
DCNMAccess
Access for DCNM (CPOM) Users
Fabric ManagementDCNM (CPOM)
mgmt0
DCNM
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 96
Connecting Switches for Fabric Management
con0
mgmt0
con0
DCNMAccess
Access for DCNM (CPOM) Users
Fabric ManagementDCNM (CPOM)
mgmt0DHCP,TFTP,SCP,LDAP,XMPP,SNMP,SSH,TELNET
DHCP,TFTP,SCP,LDAP,XMPP,SNMP,SSH,TELNET
DCNM
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 97
The following Settings have to be verified or enabled after DCNM (CPOM) Setup and the Fabric bring-up (POAP)
Install the necessary Licenses> Admin(General) -> License
Verify if all Switches are shown as a Data sources
> Admin(General) -> Data Sources -> LAN
Add your vCenter to DCNM (CPOM) for additional Visibility
> Admin(General) -> Data Sources -> VMware
Enable Performance Collection for Trunks, Access-Ports and Error&Discards (Requires DCNM Advanced)
> Admin(Performance) -> Collections
Verify DFA Health in DCNM (CPOM)> Dashboard -> Dynamic Fabric
Automation -> DFA Health
DFA management portion is intended to be DCNM Web-UI only
> The use of the DCNM Java-Client (aka Thick-Client) is not a validated option for Nexus 1000v management within DFA
> DFA is not managing any Virtual Machine Manager (e.g. Vmware vCenter, Microsoft SCVMM etc)
Note
Fabric Management IP
Management Access IP
FQDN of CPOM (cpom.domain.tld)
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 98
DFA Device Auto-Configuration (POAP)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 99
Device Auto-Configuration (POAP)
Full DCNM (CPOM) integrated POAP Engine
DHCP Scope-Definition> Own DHCP-Daemon
Image & Configuration Repository
> Embedded TFTP- & SCP-Server
Pre-Defined as well as fully scriptable Configuration Templates
Easy POAP Switch Definition Workflow
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 100
PoAP Flow
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 101
Connecting Switches for POAP
mgmt0
DCNMAccess
Access for DCNM (CPOM) Users
Fabric ManagementDCNM (CPOM)
mgmt0
Switch Boots-Up without Configuration
1
Switch asks for IP Address via DHCP
2
DCNM (CPOM) answers to DHCP request and answers with IP Address and POAP
specific Boot-Options (TFTP)
3
IP: 192.168.12.142 /
24tftp://dcnm/
tftpboot/boot.py
DCNM
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 102
DHCP Use open source ISC DHCP Server 4.1.1-P1
It only supports the following DHCP options:• IP Address & netmask• Default gateway• Domain name server• Lease Time• TFTP server• bootscript
By default, the TFTP server option has the value of DCNM IP address
By default, the bootscript has value of poap_dcnm.py which is a PYTHON script provided by DCNM team)
IP address is allocated to the device temporarily. Once POAP process is complete, the IP address will be release back to the DHCP server
DCNM DHCP server only listens to interface eth1
It is required that the POAP switches and DCNM DHCP server has L2 adjacency.
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 103
Device Auto-Configuration (POAP)
Pre-Defined DHCP Scope, derived from “Enhanced Fabric Management” IP
Address
Edit the Scope to define the Temporary IP Address
Range for Bootstrapping the Switches
DHCP Scope of DCNM (CPOM) for POAP
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 104
Connecting Switches for POAP
mgmt0
DCNMAccess
Access for DCNM (CPOM) Users
Fabric ManagementDCNM (CPOM)
mgmt0
Switch asks for NX-OS Image and Configuration
File
1
DCNM (CPOM) answers with NX-OS Image-Download path and
Configuration-Files (SCP)
2
Boot with image: 6.0(2)
Use Configuration: Spine
Hostname: Spine-4IP: 192.168.12.4 / 24
…
DCNM
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 105
Image & Config Servers Allow customer to define the file server and the directory(path) where
the images, the uploaded or generated configuration will be stored.
By default, DCNM will create a system-defined “Default_SCP_Repository” repository located at /var/lib/dcnm to store the image & configuration.
No GUI support for image upload. It is the customer responsibility to upload the desired device images to the file server
Only SCP protocol is used by the devices to download images or configuration
If external file server is used, the provided access credential should have permission on directory creation, file read & write
Device images are always stored at the top level directory (e.g. /var/lib/dcnm)
Each device configuration (uploaded or generated) is stored under the device corresponding serial number directory (e.g. /var/lib/dcnm/TB03030000B/device-config)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 106
Device Auto-Configuration (POAP)
Pre-Defined SCP Server, listening on both
Interfaces of DCNM (CPOM)
Path for accessing the Image- and Configuration-Files (Default Directory on DCNM (CPOM) is: /var/lib/dcnm)
Image and Config Server of DCNM (CPOM)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 107
Connecting Switches for POAP
mgmt0
DCNMAccess
Access for DCNM (CPOM) Users
Fabric ManagementDCNM (CPOM)
mgmt0
Switch Boots-Up with defined NX-OS Image and
Startup-Configuration
1
DCNM (CPOM) Discovers new Switch and adds it to
Inventory
2
DCNM
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 108
Connecting Switches for POAP
mgmt0
DCNMAccess
Access for DCNM (CPOM) Users
Fabric ManagementDCNM (CPOM)
mgmt0
Power on Auto Provisioning (PoAP) communicates over the out-of-band network interface mgmt0 with the DHCP-, TFTP and SCP-Server. DCNM (CPOM) combines the function of DHCP-, TFTP- and SCP-Server in his installation and will
listen and serve it over the “Fabric Management” Network.
Note: Please ensure that only one single DHCP-Server serves in the “Fabric Management Network”; either the one of
DCNM (CPOM) or another delegated one.
Fabric Management - Out-of-Band (OOB) Network
DCNM
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 109
POAP Definition
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 110
POAP Definition – pre-requisitePre-requisite:
• Desired device images are manually copied to the image & config servers
• The desired POAP templates exists (DCNM does provide some system defined templates for DFA) or existing configuration is available
• DHCP server is assigned the correct IP address range, up and running
• User has the serial numbers of the POAP switches (use the command “show license host-id” to find out the serial number)
• The switches and DCNM server is Layer 2 adjacency.
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 111
POAP Definition Features1. Allow user to upload device startup
configuration or generate one
2. Allow batch creating of multiple POAP device definitions(bulk edit does not support)
3. Allow user to associated device images with the device.
4. Automatically import the POAP device into inventory system for discovery
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 112
POAP Definition Features (cont.)5. Allow user to create device
configuration basing on template.
6. Allow user to reuse the fill-in template values (setting feature)
7. Provide template form with field validation (support bulk config generation) and CLI preview
8. Real-time update of the POAP bootscript execution status (including error)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 113
POAP Definition Features (cont.)9. Send write erase & reload command to
device to POAP bootup
10. Automatically determine the switch id/name and management IP address from the uploaded configuration
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 114
Device Auto-Configuration (POAP)
Choose from pre-defined DFA Templates
Parameter Values can be saved for later purpose
Form, automatically created from the Templates; list and range Values supported to
accommodate multiple Switches
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 115
POAP Definition -- actionSteps to create a POAP Switch Definition:
1. Determine whether uploading the static configuration or generating one basing on template
2. Fill the information regarding device serial number, kickstart & system images, which image& config server to use, what devices group it belongs to, the access credential of the device
3. If upload the start up config, please upload it
4. If using template to generate config, please select the desired template and fill in the template form
5. Click “publish” button
6. Boot up the device in POAP mode
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 116
POAP Template
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 117
DCNM Template Feature
Existing DCNM template builder is enhanced
system defined templates for DFA (leaf, spine, etc) are provided
Support Cloning template
Only template marked as POAP and Published will be used as POAP Templates
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 118
DCNM Template Feature Annotation feature is added to the template builder
• Display the description or hint of the template form field• Provide the default value, allowed format, allowed min, max value• Provide tagging to allow application to extract data from the filled template
form• Which field represents management IP address, switch id, etc
“stuct” data type is added to support grouping of variables
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 119
Device Auto-Configuration (POAP)
Generate new POAP Definition for a single or
multiple Switches
Upload existing Startup Config for a given Switch
Workflow to for POAP-Definitions
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 120
Device Auto-Configuration (POAP)
Enter the Switch Serial-Number, multiples comma-separated or upload a CSV-
File
Define the Switch Type (N5k, N6k, N7k etc.)
Define the Switch repository (where are your images; default
is local SCP repository (var/lib/dcnm)
Choose Kickstart- and System-Image for Switch; list view of
images in repository
Configuration Repository to use
during POAP process
Username and Password for accessing the Switches through
CLI, SNMP, etc.
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 121
Device Auto-Configuration (POAP)
Choose from pre-defined DFA Templates
Parameter Values can be saved for later purpose
Form, automatically created from the Templates; list and range Values supported to
accommodate multiple Switches
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 122
The following Task have to be completed before using DCNM (CPOM) with it’s integrated POAP-Engine
Verify the DHCP-Scope, if it matches your Setup. Have a close focus on the IP Address Range, which are temporary IP Addresses during the POAP Process
> Config -> Power-On Auto Provisioning (POAP)
Upload the required NX-OS Kickstart- and System-Images to the chosen Repository-Server. If DCNM (CPOM) is your Repository-Server, you have to upload the images to DCNM (CPOM) via SCP or SFTP.
Via SCP and TFTP, the following Folder is exposed for your NX-OS Kickstart- and SystemImages: /var/lib/dcnm
Note
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 123
DFA pre-defined POAP Templates Repository & Editor
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 124
DFA pre-defined POAP Templates – Repository/Editor
Pre-Defined Configuration Template
Repository
Template Creator supporting scripting Language and Form-
Creation
Templates coveringSwitch Name,
Management, VPC, FEX, DFA, everything …..
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 125
DFA pre-defined POAP Templates – Repository/Editor
Select pre-defined Template for Open, Edit or Save-As
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 126
DFA pre-defined POAP Templates – Repository/Editor
Integrated Template Editor
Including check for Syntax Validation
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 127
DFA pre-defined POAP Templates – Repository/Editor
Pre-Defined Configuration
Template Repository
Placeholder defined with “$$$”
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 128
DFA pre-defined POAP Templates – Repository/Editor
Templates coveringSwitch Name,
Management, VPC, FEX, DFA,
everything …..Placeholder and definition of valid entries defined in Template Header
Detailed Description
available within Template Editor
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 129
DFA pre-defined POAP Templates Detailed Description
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 130
DFA POAP Base Templates – Template Parameter
General & Out-of-Band Configuration
In-Band Configuration
Fabric: Layer-3 Control-Plane(BGP & BGP Route-Reflector)
Fabric: Manageability and Cable Plan
*VPC+ Domain Configuration
Interface, Port-Channel & FEX-Configuration
*Distributed Gateway & Host Mobility specifics
Fabric: Layer-2 Control-Plane(Fabric & FabricPath)
*Leaf / Border-Leaf only
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 144
Cable Management and Consistency Check
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 145
Cabling Plan Consistency Check Detects Cabling anomalies
> Incorrect Connectivity (ErrC)> Link Not present (Unkn)> Unexpected Connections (Enp)
Flexible > supports DFA and Non-DFA platforms> Cable plan can be deployed global or > device-specific > Enforcement on one side
Auto Generation, Import, Export
Granular – Per port Validation
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 146
Consistency Check – Why?
= DFA-Spine(Tier 2)
= DFA-Leaf(Tier1)
2 2
✓
1 1 1 1
2 2
✗ ✗1 1 1 1
Consistency Check OK based on Cable Plan/Tier
Definition
Consistency Check FAILED based on Cable
Plan/Tier Definition
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 147
Cable Plan & Consistency Check Configuration
nexus# dir bootflash:/// | include cableplan.xml
906 May 28 06:43:52 2011 cableplan.xml
nexus#
Individual Cable-Plan-File generated and uploaded thru DCNM (CPOM)
Configuration already done in Pre-Defined POAP-Templates; you can chose if the Cable-Pan should be
enforced or not
2 2
1 1 1 1= DFA-Spine
(Tier 2)= DFA-Leaf
(Tier1)
feature cable-managementfeature lldp
!fabric connectivity tier 2fabric connectivity cable-
plan enforce
feature cable-managementfeature lldp
!fabric connectivity tier 1fabric connectivity cable-
plan enforce
errdisable recovery interval 300
errdisable detect cause miscabling
no errdisable recovery cause miscabling
Error Disable detect on per DefaultError Disable recovery OFF per
Default
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 148
2 2
✗ ✗
1 1 1 1
Consistency Check – Show & Log
2011 May 31 02:37:40 n6k-leaf-2018 %$ VDC-1 %$ %CMM-2-MISCBL_TIERERR: Miscabling: Port Ethernet1/47 Error detected on peer tier check. Local: Tier 1
System n6k-leaf-2018 Chassis 002a.6a27.27d6 Port Eth1/47 Neighbor: Tier 1 System n6k-leaf-2017 Chassis 002a.6a22.a416 Port Eth1/47
Log Message on Cable Plan Consistency Check failureError detected on peer tier check
n6k-leaf-2018# show fabric connectivity neighbors ------------------------------------------------------------------------
-------Local System:
Device Tier Config: Enabled Device Tier Level: 1
Mismatch Delay Config: Disabled Mismatch Delay Timeout: 0
Cable-Plan Enforce: Enabled DeviceID: n6k-leaf-2018 ChassisID: 002a.6a27.27d6
------------------------------------------------------------------------
------- Codes: (Ok) Normal, (ErrT) Tier error , (ErrC) Cable-Plan
error, (V) VPC Peer connection, (S) Stale entry, (Unkn)
Unknown, (Enp) Entry not present in Cable-Plan, (Tl) Tier level
Neighbor Table: ------------------------------------------------------------------------
-------Local DeviceID PortID Tl Cable-Plan
Status Intf Entry
Eth1/37 n6k-spine-2016 Eth1/37 2 n6k-spine-201,Eth1/37
Ok Eth1/38 n6k-spine-2015 Eth1/38 2 n6k-spine-201,Eth1/38
Ok Eth1/47 n6k-leaf-2017 Eth1/47 1 Enp
ErrT,S
Total entries displayed: 3
n6k-leaf-2018# show interface eth1/47
Ethernet1/47 is down (Miscabled)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 149
Why Looking at the CLI?
DCNM (CPOM) Shows same information:- Failure on Node and how many
- Interface Miscabling- Interface Status
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 150
DFA XMPP Chat Demo with Pidgin
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 151
Perquisite is a successful installed DCNM with XMPP-Server
Pidgin is installed on your Client
> Pidgin is a Opensource XMPP capable Chat-Client
> http://pidgin.im/
Configure your Pidgin to XMPP-Server Connection in the Pidgin-Client
> Accounts -> Manage Accounts -> Add
Add Buddy to Pidgin (Buddy List)
> Buddies -> Add Buddy
Name your Buddy (Buddy’s Username)
> This is the FQDN of your Switch; hostname@dcnm-fqdn
> Authorize the Switches when got asked in Pidgin
> Repeat this step for every Switch you want to import in to Pidgin
Pidgin Connection to DCNM XMPP-Server
Protocol is XMPP
Username is either the pre-defined or one you did create
with the appmgr-tool
This is the FQDN of the
DCNM-Server
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 152
XMPP Chat Demo with Pidgin Switches will appear as Buddies
The Status of the Switches will be shown
You can now IM to a Switch sending NX-OS CLI command
> Double-click the Buddy Name to open a Instant Message session
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 153
Create a Chat-Room in Piding> Buddies -> Add Chat
Name your Chat-Room> Note: the Room-Name has to be
configured in the Switches
> fabric access group group1 group2
> Every other setting is predefined from your Pidgin XMPP-Connection
You can now IM to a Group of Switches sending NX-OS CLI command
> Double-click the Buddy Name to open a Instant Message session
> Note: wait until all participants joined the room (# people in the room)
XMPP-Group Chat Demo with Pidgin
Cisco Confidential© 2010 Cisco and/or its affiliates. All rights reserved. 154
Automated Network Provisioning
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 155
Full-Automated Network Provisioning
DCNM (CPOM)
N1kv/OVS
VDP*
DHCP/ARP-ND
Physical Machines Virtual Machines
Auto-Config Triggers
Data Packet Driven
Programmatic
*VDP (VSI Discovery and Configuration Protocol) is IEEE 802.1Qbg Clause 41
Network & Services
Orchestration
Compute & Storage
OrchestrationOrchestration
StackUCS Director
(Cloupia), OpenStack, vCloud
Director
For Information on how to integrate Orchestrator into DFA, please refer to the “DCNM 7.0 OVA Installation
Guide” and the “DFA Fabric Management Whitepaper”
MAC Learning
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 156
Orchestration Administrator defines logical Organization Network
> Mapping the Auto-Config Definition “Name” to the logical Organization Network
> Name-Space (Segment-IDs) resources are administrated within the Orchestrator
> Orchestrator (for example vCD, Openstack) directly interacts with the Virtual Switch
Network Administrator prepares Auto-Config Definition in DCNM (CPOM)
> Virtual Switch are configured through Orchestrator (like in vCD) or pre-populated Port-Groups/Port-Profiles
When new Virtual-Machine get created and Network DCNM (CPOM) gets polled for Auto-Config Definition
> Based on MAC learn or VDP signalization Network gets instantiated
> Dynamic VLAN gets chosen and mapped to the Segment-ID (based on Dynamic VLAN range and Segment-ID Namespace, managed by Orchestrator)
> Auto-Config Definition gets installed (VLAN, SVI, VRF, Segment-ID)
> VLAN ID gets exchanged via VDP to the Virtual Switch (no, not VTP)
> Leaf receives 802.1q tagged frames and associates them to the segment-ID
Full-Automated Network Provisioning
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 157
DFA enables Network Auto-Configuration with no Workload&Network Orchestrator
Semi-Automated Network Provisioning
DCNM (CPOM)
N1kv/OVS
VDP*
DHCP/ARP-ND
Physical Machines Virtual Machines
Auto-Config Triggers
Data Packet Driven
Programmatic
*VDP (VSI Discovery and Configuration Protocol) is IEEE 802.1Qbg Clause 41
MAC Learning
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 158
Network Administrator prepares Network Auto-Configuration Definition in DCNM (CPOM) & Virtual-Switch Port-Profiles/Port-Groups
> Virtual Switch configuration is manual with VDP and Mobility-Domain
> Non VDP*-capable Servers (physical or virtual) need to belong to a Mobility-Domain
> external entity responsible for VLAN Namespace management (e.g. vCenter, UCS-Director, Openstack)
> no dynamic VLAN assignment by DFA-Leaf as per no Fabric-Global synchronization
On Workload start, VDP* or MAC learn does trigger instantiation of Network Auto-Configuration Definition on connected DFA-Leaf
> DFA-Leaf downloads the Network Auto-Config Definition for the given Segment-ID or VLAN
> DCNM (CPOM) provides LDAP Database with Network Auto-Config Definitions
DCNM (CPOM) provides Fabric bring-up, DFA-Leaf Network Auto-Config and Monitoring
> Auto-Config Definitions configured in DCNM (CPOM) and stored in integrated LDAP Database
> VDP* as Bottom-Up signalization for Auto-Config trigger
> MAC learn as alternative trigger for non-VDP* capable Devices (requires Mobility-Domain)
Semi-Automated Network Provisioning
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 159
DFA enables Optimized Networking with no Auto-Config or/and Workload&Network Orchestrator
Non-Automatic Configuration (Manual)
DCNM (CPOM)
N1kv/OVS
Physical Machines Virtual Machines
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 160
Network Administrator Configures Manual the physical Network> VLAN, SVI, Forwarding-Mode and the VLAN to Segment-ID mapping
No Automatic trigger to enable the configuration> pre-defined as per a traditional Operating Model or pulled from DCNM
(CPOM) repository (LDAP)
DCNM (CPOM) provides Switch bring-up and Monitoring functionality
Non-Automatic Configuration (Manual)
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 161
Note Control-Plane based – VDP Signalization
> Nexus 1000v on vSphere* & OVS> Bare-Metal Server with VDP capable CNA (only Data VLANs)
Packet based – MAC Learn> Every Bare-Metal or virtualized Server with Mobility Domain> Requires 802.1q Trunk between Server/Virtual-Switch and DFA-Leaf
CLI based – Manual Download of Auto-Config Definition to Leaf-Switch> Every Bare-Metal or virtualized Server
Static Configuration> Every Bare-Metal or virtualized Server
Note: Your Server can have Static or Dynamic IP Addressing> DCNM (CPOM) offers DHCP service for non-overlapping IP Address Scopes!
*Other virtualized Switches tbd (Nexus 1000v on other Hypervisors)
Summary
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 198
DFA is Evolution NOT Revolution!
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 199
DFA is a Happy Meal!
You pick and choose!
This presentation contains materials that are either copyright © 2011 IBM Corporation or copyright © 2011 Cisco Systems, Inc. All rights reserved. 200
Dynamic Fabric Automation – Summary of Facts
DFA is an evolution of Unified Fabric• It enhances Unified Fabric in four major areas:
- Simplify the management of Unified Fabric- Optimize the network for L2-L3 Services and extend any VLAN/Subnet anywhere inside /
across DC- Large scale Multi-tenancy- Automate L2-L3 network policies (VLAN, VRF, ACL etc.) and network services policies (L4-L7
Services)
• Overall it should simplify large and small scale virtualization deployments• DFA leverages and builds upon existing standards that are proven in Industry:
- MP-BGP for Segmentation- Proxy ARP and Anycast GW to support workload mobility- DHCP/TFTP for POAP- XMPP for multi-device management- LDAP for policy based provisioning of network services- LLDP for topology discovery- VDP (optional) for VM Discovery