View
217
Download
1
Tags:
Embed Size (px)
Citation preview
1© 2005 Cisco Systems, Inc. All rights reserved.
High AvailabilityCampus Networks
Tyler Creek
Consulting Systems Engineer
Southern California
222© 2005 Cisco Systems, Inc. All rights reserved.
Agenda
• Campus High Availability Design Principles
• Foundation Services
• Multi-Layer Design
• Routed Access Design
• Summary Architectural FoundationHierarchical Campus Design
Security
Mobility Convergence
Availability
Flexibility
SiSi SiSi
333© 2005 Cisco Systems, Inc. All rights reserved.
What Is High Availability?
DPM—Defects per Million
Availability Downtime Per Year (24x365)
99.000%
99.500%
99.900%
99.950%
99.990%
99.999%
99.9999%
3 Days
1 Day
53 Minutes
5 Minutes
30 Seconds
15 Hours
19 Hours
8 Hours
4 Hours
36 Minutes
48 Minutes
46 Minutes
23 Minutes
DPM
10000
5000
1000
500
100
10
1
$ 205$1,010,536Average
$ 107$ 668,586Transportation
$ 244$1,107,274Retail
$ 370$1,202,444Insurance
$1,079$1,495,134Financial Institution
$ 134$1,610,654Manufacturing
$ 186$2,066,245Telecommunications
$ 569$2,817,846Energy
Revenue/ Employee-
HourRevenue/HourIndustry Sector
To achieve five-nines availability or better, seconds or even milliseconds count
More than just revenue impactedRevenue lossProductivity lossImpaired financial performanceDamaged reputationRecovery expenses
444© 2005 Cisco Systems, Inc. All rights reserved.
Automation and local
action
SOFTWARERESILIENCY
EMBEDDEDMANAGEMENT
NETWORK LEVELRESILIENCY
SOFTWARERESILIENCY
Cisco IOS Software features for
faster network convergence, protection,
and restoration
INVESTMENT PROTECTION IS A KEY COMPONENT
HARDWARERESILIENCY
SOFTWARERESILIENCY
Cisco IOS® Software functionality that
mitigates the impact of faults
Reliable, robust hardware designed for
high availability
SYSTEM LEVELRESILIENCY
Systematic, End-to-End Approach:Targeting Downtime
555© 2005 Cisco Systems, Inc. All rights reserved.
System Level Resiliency Overview
Control/data plane resiliency
• Separation of control andforwarding plane
• Fault isolation and containment
• Seamless restoration of Route Processor control and data plane failures
Link resiliency
• Reduced impact of line card hardware and software failures
Planned outages
• Seamless software and hardware upgrades
Eliminate single points of failure for hardware and software components
Micro-Kernel
Lin
e C
ard
Lin
e C
ard
Lin
e C
ard
Lin
e C
ard
AC
TIV
E
ST
AN
DB
Y
CONTROL PLANE
MA
NA
GE
ME
NT
PL
AN
E
FORWARDING/DATA PLANE
666© 2005 Cisco Systems, Inc. All rights reserved.
Hierarchical Network Design
• Scalability to expand/shrink without affecting network behavior
• Predictable performance under normal conditions and failure conditions
Convergence and Self-Healing
• Reduce convergence times for major network protocols—EIGRP, OSPF,IS-IS, BGP
• Leverage in network wherever redundant paths exist
Intelligent Protocol Fabric
• Embed NSF intelligence network-wide in Service Provider and Enterprise networks
Service Provider Core
Service ProviderPoint of Presence
EnterpriseEdge
EnterpriseCampus Core
Campus Distribution
Layer
CampusAccessLayer
Data CenterBuilding Block
Network Level Resiliency Overview
777© 2005 Cisco Systems, Inc. All rights reserved.
• Commonality:
Intelligent switching
Simplified configuration
Reduced network complexity
Improved network availability
Reduced management complexity
• Commonality:
Intelligent switching
Simplified configuration
Reduced network complexity
Improved network availability
Reduced management complexity
Ω
Cisco Campus ArchitectureOne Architecture with Multiple Design Options
Enterprise Campus
Intelligent Switching
FutureCampusDesignOptions
FutureCampusDesignOptions
Routed Campus
Design
Routed Campus
Design
Cisco Campus Architecture
Cisco Campus Architecture
Multi-LayerDesign
Multi-LayerDesign
Intelligent Switching(Hybrid of L2 + L3 features)Intelligent Switching(Hybrid of L2 + L3 features)
888© 2005 Cisco Systems, Inc. All rights reserved.
Hierarchical Campus DesignWithout a Rock Solid Foundation the Rest Doesn’t Matter
SpanningTreeRouting
VLANs
GLBP
Trunking
LoadBalancing
Access
Distribution
Core
Distribution
Access
SiSi SiSi
SiSiSiSi
SiSi SiSi
Data Center
999© 2005 Cisco Systems, Inc. All rights reserved.
Hierarchical Campus DesignBuilding Blocks
Data CenterWAN Internet
SiSi SiSi SiSi SiSi SiSi SiSi
SiSi SiSi
SiSi SiSiSiSi SiSi
SiSi SiSi
Access
Distribution
Core
Distribution
Access • Offers hierarchy—each layer hasspecific role
• Modular topology—building blocks• Easy to grow, understand, and
troubleshoot• Creates small fault domains—clear
demarcations and isolation• Promotes load balancing and
redundancy• Promotes deterministic traffic patterns• Incorporates balance of both Layer 2 and
Layer 3 technology, leveraging the strength of both
• Can be applied to all campus designs; Multi-Layer L2/L3 and Routed Access designs
101010© 2005 Cisco Systems, Inc. All rights reserved.
Multi-Layer Reference DesignLayer 2/3 Distribution with Layer 2 Access
• Consider fully utilizing uplinks via GLBP• Distribution-to-distribution link required for route summarization• STP convergence not required for uplink failure/recovery• Map L2 VLAN number to L3 subnet for ease of use/management• Can easily extend VLANs across access layer switches when required
10.1.20.010.1.120.0
VLAN 20 DataVLAN 120 Voice
VLAN 40 DataVLAN 140 Voice
10.1.40.010.1.140.0
HSRP or GLBPVLANs 20,120,40,140
HSRP or GLBPVLANs 20,120,40,140
ReferenceModel
Layer 3SiSiSiSi
Layer 2
Access
Distribution
111111© 2005 Cisco Systems, Inc. All rights reserved.
Routed Campus Design Layer 3 Distribution with Layer 3 Access
• Move the Layer 2/3 demarcation to the network edge
• Upstream convergence times triggered by hardware detection of link lost from upstream neighbor
• Beneficial for the right environment
10.1.20.0
10.1.120.0
VLAN 20 Data
VLAN 120 Voice
VLAN 40 Data
VLAN 140 Voice
10.1.40.0
10.1.140.0
EIGRP/OSPF EIGRP/OSPF
GLBP Model
Layer 3
Layer 2
Layer 3
Layer 2EIGRP/OSPF EIGRP/OSPF
SiSiSiSi
121212© 2005 Cisco Systems, Inc. All rights reserved.
Optimal Redundancy
• Core and distribution engineered with redundant nodesand links toprovide maximum redundancy and optimal convergence
• Network bandwidth and capacity engineered to withstand nodeor link failure
• Sub-Second converge around most failure events
Access
Distribution
Core
Distribution
Access
Data CenterWAN Internet
SiSi SiSi SiSi SiSi SiSi SiSi
SiSi SiSi
SiSi SiSiSiSi SiSi
SiSi SiSi
Redundant
Nodes
131313© 2005 Cisco Systems, Inc. All rights reserved.
Campus Network ResilienceSub-Second Convergence
Worst Case Convergence for AnyLink or Platform Failure Event
Sec
on
ds
00.20.40.60.8
11.21.41.61.8
2
Multi-LayerL2 Access
OSPF Core*
Multi-LayerL2 Access
EIGRP Core
RoutedCampus
OSPFAccess*
RoutedCampusEIGRPAccess
L2 Access (Rapid PVST+ HSRP)
L3 Access
*OSPF Results Require Sub-Second Timers
Convergence Times for Campus Best Practice Designs
141414© 2005 Cisco Systems, Inc. All rights reserved.
ESE Campus Solution Test BedVerified Design Recommendations
Data CenterWAN Internet
SiSi SiSi SiSi SiSi SiSi SiSi
SiSi SiSi
SiSi SiSiSiSi SiSi
SiSi SiSi
Total of 68 Access Switches, 2950, 2970, 3550, 3560, 3750, 4507 SupII+, 4507SupIV, 6500
Sup2, 6500 Sup32, 6500 Sup720 and 40 APs (1200)
6500 with Redundant Sup720s
Three Distribution Blocks6500 with Redundant Sup720
4507 with Redundant SupV
Three Distribution Blocks 6500 with Redundant Sup720s
7206VXR NPEG1
4500 SupII+, 6500 Sup720, FWSM, WLSM, IDSM2, MWAM
8400 Simulated Hosts10,000 RoutesEnd-to-End Flows: TCP, UDP, RTP, IPmc
151515© 2005 Cisco Systems, Inc. All rights reserved.
Agenda
• Campus High Availability Design Principles
• Foundation Services
• Multi-Layer Design
• Routed Access Design
• SummaryArchitectural Foundation
Hierarchical Campus Design
Security
Mobility Convergence
Availability
Flexibility
SiSi SiSi
161616© 2005 Cisco Systems, Inc. All rights reserved.
Best Practices—Layer 3 Routing Protocols
• Used to quickly re-route around failed node/links while providing load balancing over redundant paths
• Build triangles not squares for deterministic convergence
• Only peer on links that you intend to use as transit
• Insure redundant L3 paths to avoid black holes
• Summarize distribution to core to limit EIGRP query diameter or OSPF LSA propagation
• Tune CEF L3/L4 load balancing hash to achieve maximum utilization of equal cost paths (CEF polarization)
• Utilized on both Multi-Layer and Routed Access designs
Data CenterWAN Internet
Layer 3 Equal Cost Link’s
Layer 3 Equal Cost Link’s
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi
171717© 2005 Cisco Systems, Inc. All rights reserved.
Best Practice—Build Triangles Not SquaresDeterministic vs. Non-Deterministic
• Layer 3 redundant equal cost links support fast convergence
• Hardware based—fast recovery to remaining path
• Convergence is extremely fast (dual equal-cost paths: no need for OSPF or EIGRP to recalculate a new path)
Triangles: Link/Box Failure Does NOT Require Routing Protocol Convergence
Model A
Squares: Link/Box Failure Requires Routing Protocol Convergence
Model B
SiSi
SiSiSiSi
SiSiSiSi
SiSiSiSi
SiSi
181818© 2005 Cisco Systems, Inc. All rights reserved.
Best Practice—Passive Interfaces for IGPLimit OSPF and EIGRP Peering Through the Access Layer
Limit unnecessary peering
Without passive interface:• Four VLANs per wiring closet,
• 12 adjacencies total
• Memory and CPU requirements increase with no real benefit
• Creates overhead for IGP
Routing
Updates
OSPF Example:
Router(config)#router ospf 1Router(config-router)#passive-interface Vlan 99
Router(config)#router ospf 1Router(config-router)#passive-interface defaultRouter(config-router)#no passive-interface Vlan 99
EIGRP Example:
Router(config)#router eigrp 1Router(config-router)#passive-interface Vlan 99
Router(config)#router eigrp 1Router(config-router)#passive-interface defaultRouter(config-router)#no passive-interface Vlan 99
Distribution
Access
SiSiSiSi
191919© 2005 Cisco Systems, Inc. All rights reserved.
CEF Load BalancingAvoid Underutilizing Redundant Layer 3 Paths
• The default CEF hash‘input’ is L3
• CEF polarization: In amulti-hop design, CEFcould select the same left/left or right/right path
• Imbalance/overload could occur
• Redundant paths are ignored/underutilized
Redundant PathsIgnored
SiSiSiSi
SiSi SiSi
SiSi SiSi
L
L
R
R
DistributionDefault L3 Hash
CoreDefault L3 Hash
DistributionDefault L3 Hash
AccessDefault L3 Hash
AccessDefault L3 Hash
202020© 2005 Cisco Systems, Inc. All rights reserved.
CEF Load BalancingAvoid Underutilizing Redundant Layer 3 Paths
• With defaults, CEF could select the same left/left or right/right paths and ignore some redundant paths
• Alternating L3/L4 hash and default L3 hash will give us the best load balancing results
• The default is L3 hash—no modification required in core or access
• Use:
mls ip cef load-sharing full
in the distribution switches to achieve better redundant path utilization
SiSiSiSi
SiSi SiSi
SiSi SiSi
RL
RDistributionL3/L4 Hash
CoreDefault L3 Hash
DistributionL3/L4 Hash
L
RL
Left Side Shown
AccessDefault L3 Hash
AccessDefault L3 Hash
L
All PathsUsed
212121© 2005 Cisco Systems, Inc. All rights reserved.
Single Points of TerminationSSO/NSF Avoiding Total Network Outage
• The access layer and other single points of failure are candidates for supervisor redundancy
• L2 access layer SSO
• L3 access layer SSO and NSF
• Network outage until physical replacement or reload vsone to three seconds
Core
Distribution
Access
SiSi SiSi SiSi SiSi SiSi SiSi
SiSi SiSi
L2 = SSOL3 = SSO/NSF
222222© 2005 Cisco Systems, Inc. All rights reserved.
Campus MulticastWhich PIM Mode—Sparse or Dense
“Sparse mode Good! Dense mode Bad!”
Source: “The Caveman’s Guide to IP Multicast”, ©2000, R. Davis
232323© 2005 Cisco Systems, Inc. All rights reserved.
PIM Design Rules for Routed Campus
• Use PIM sparse mode
• Enable PIM sparse mode on ALL access, distribution and core layer switches
• Enable PIM on ALL interfaces
• Use Anycast RPs in the core for RP redundancy and fast convergence
• Define the Router-ID to prevent Anycast IP address overlap
• IGMP-snooping is enabled when PIM is enabled on a VLAN interface (SVI)
• (Optional) use garbage canRP to black-hole unassigned IPmc traffic
IPmc Sources
WAN Internet
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi SiSi SiSiSiSiSiSi
IP/TV ServerCall Managerw/MoH
RP-Left10.122.100.1
RP-Right10.122.100.
1
242424© 2005 Cisco Systems, Inc. All rights reserved.
AccessAccess
Multicast in the Campus
MSDPMSDP
Distribution-A Distribution-B
DistributionLayer 2/3
CoreLayer 3
Core-A Core-B
SiSi SiSiip pim rp-address 10.0.0.1!interface Y description GigE to Access/Core ip address 10.122.0.Y 255.255.255.252 ip pim sparse-mode !<snip>
ip pim rp-address 10.0.0.1!interface Y description GigE to Access/Core ip address 10.122.0.Y 255.255.255.252 ip pim sparse-mode !<snip>
IGMP snooping on by default
SiSiSiSi
interface loopback 0 ip address 10.0.0.1 255.255.255.255
interface loopback 1 ip address 10.0.0.3 255.255.255.255!ip msdp peer 10.0.0.2 connect-source loopback 1ip msdp originator-id loopback 1!interface TenGigabitEthernet M/Y ip address 10.122.0.X 255.255.255.252 ip pim sparse-mode !ip pim rp-address 10.0.0.1
interface loopback 0 ip address 10.0.0.1 255.255.255.255
interface loopback 1 ip address 10.0.0.2 255.255.255.255!ip msdp peer 10.0.0.3 connect-source loopback 1ip msdp originator-id loopback 1 !interface TenGigabitEthernet M/Y ip address 10.122.0.X 255.255.255.252 ip pim sparse-mode!ip pim rp-address 10.0.0.1
252525© 2005 Cisco Systems, Inc. All rights reserved.
Best Practices—UDLD Configuration
• Typically deployed on any fiber optic interconnection
• Use UDLD aggressive mode for best protection
• Turn on in global configuration to avoid operational error/“misses”
• Config example
Cisco IOS Software: udld aggressive
CatOS: set udld enableset udld aggressive-mode enable <mod/port>
Data CenterWAN Internet
Layer 3 Equal Cost Link’s
Layer 3 Equal Cost Link’s
Fiber Interconnections
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi
272727© 2005 Cisco Systems, Inc. All rights reserved.
Best Practices—EtherChannel Configuration
• Typically deployed in distribution to core, and coreto core interconnections
• Used to provide link redundancy—while reducing peering complexity
• Tune L3/L4 load balancing hash to achieve maximum utilization of channel members
• Match CatOS and Cisco IOS Software PAgP settings
• 802.3ad LACP for interop if you need it
• Disable unless neededCatOS: set port host
Cisco IOS Software: switchport host
Data CenterWAN Internet
Layer 3 Equal Cost Link’s
Layer 3 Equal Cost Link’s
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi
282828© 2005 Cisco Systems, Inc. All rights reserved.
EtherChannel Load BalancingAvoid Underutilizing Redundant Layer 2 Paths
• Network did not load balance using default L3 load balancing hash
Common IP addressing scheme
72 access subnets addressed uniformly from 10.120.x.10 to 10.120.x.215
• Converted to L4 load balancing hash and achieved better load sharing
cr2-6500-1(config)#port-channel load-balance src-dst-port
Link 0 load—68%
Link 1 load—32%
Link 0 load—52%
Link 1 Load—48%
L3 Hash
L4 Hash
SiSi
SiSiSiSi
SiSi
292929© 2005 Cisco Systems, Inc. All rights reserved.
PAgP TuningPAgP Default Mismatches
Matching EtherChannel Configuration on Both Sides Improves Link Restoration Convergence Timesset port channel <mod/port> on/off
0
1
2
3
4
5
6
7
Tim
e t
o C
on
ve
rge
in
Se
co
nd
s
PAgP Mismatch PAgP Off
6500 (CatOS)
4006 (CatOS)
As Much as Seven Seconds of Delay/Loss Tuned Away
303030© 2005 Cisco Systems, Inc. All rights reserved.
Unauthorized Switch
Enterprise Server
Unauthorized Switch
Cisco SecureACS
Enterprise Server
PROBLEM: SOLUTION:
Mitigating Plug and PlayersProtecting Against Well-Intentioned Users
• Well-intentioned users place unauthorized network devices on the network possibly causing instability
• Cisco Catalyst® switches support rogue BPDU filtering: BPDU Guard, Root Guard
Incorrect STP Info
BPDU Guard
Network Instability
Authorized Switch
Authorized Switch
Root Guard
313131© 2005 Cisco Systems, Inc. All rights reserved.
BPDU GuardPrevent Loops via WLAN (Windows XP Bridging)
• WLAN AP’s do notforward BPDU’s
• Multiple Windows XPmachines can create aloop in the wired VLANvia the WLAN
• BPDU Guard configuredon all end station switch ports will prevent loopfrom forming
Win XPBridgingEnabled
Win XPBridgingEnabled
BPDU GuardDisables Port
STP LoopFormed
BPDUGenerated
BPDUDiscarded
PROBLEM:
SOLUTION:
323232© 2005 Cisco Systems, Inc. All rights reserved.
Cisco Catalyst Integrated Security Features
• Port security prevents MAC flooding attacks
• DHCP snooping prevents client attack on the switch and server
• Dynamic ARP Inspection adds security to ARP using DHCP snooping table
• IP source guard adds security to IP source address using DHCP snooping table
IP Source Guard
Dynamic ARP Inspection
DHCP Snooping
Port Security
333333© 2005 Cisco Systems, Inc. All rights reserved.
Cisco Catalyst 6500 High Availability LeadershipMaximizing Uptime
Physical Redundancy• Redundant supervisors, power supplies,
switch fabrics, and clocks
Non-Stop Forwarding/ Stateful Switch Over (NSF/SSO)
• Traffic continues flowing after a primary supervisor failure
• Sub-second recovery in L2 and L3 networks
Generic Online Diagnostics(GOLD)• Proactively detect and address
potential hardware and software faults in the switch before they adversely impact network traffic
Catalyst 6500
Cisco IOS Software Modularity• Subsystem In-Service Software Upgrades (ISSU)• Stateful Process Restarts• Fault Containment, Memory Protection
New!
343434© 2005 Cisco Systems, Inc. All rights reserved.
Catalyst 6500 with IOS ModularityInfrastructure Enhancements
• IOS with Modularity
Protected Memory
Fault Containment
Restartable Processes
– 20+ independent processes
– Remaining feature subsystems live in IOS Base process
Subsystem ISSU
• Embedded Event Manager
Create TCL policy scripts to program the Catalyst 6500
– When detect event X, then do action Y
• Generic On-Line Diagnostics (GOLD) supports pro-active diagnosis of faults before they become a problem…
Catalyst 6500 Data Plane
Network Optimized Microkernel
Bas
e
Ro
uti
ng
TC
P
UD
P
EE
M
FT
P
CD
P
INE
TD
etc
High Availability Infrastructure
Cisco IOS Software Modularity
353535© 2005 Cisco Systems, Inc. All rights reserved.
Software ModularityMinimize Unplanned Downtime
If an error occurs in a modular process…
• HA subsystem determines the best recovery action
Restart a modular process
Switchover to standby supervisor
Remove the system from the network
• Process restarts with no impact on the data plane
Utilizes Cisco Nonstop Forwarding (NSF) where appropriate
State Checkpointing allows quick process recovery
Traffic forwarding continues during unplanned process restarts
Cisco Catalyst 6500 Data Plane
Network Optimized Microkernel
Bas
e
Ro
uti
ng
TC
P
UD
P
EE
M
FT
P
CD
P
INE
TD
etc
High Availability Infrastructure
Cisco IOS Software Modularity
363636© 2005 Cisco Systems, Inc. All rights reserved.
Software ModularitySimplify Software Changes
Traffic forwarding continues during planned software changes
If the software needs to be upgraded (for example, to protect against a new security vulnerability)…
• The change can be made available as an individualpatch which reduces code certification time
• Subsystem In-Service Software Upgrade (ISSU)* allows the change to be applied with no service disruption
Time
Code Certification Code DeploymentCatalyst 6500 Data Plane
Network Optimized Microkernel
Bas
e
Ro
uti
ng
TC
P
UD
P
EE
M
FT
P
CD
P
INE
TD
etc
High Availability Infrastructure
Cisco IOS Software Modularity
Ro
uti
ng
Patch
*for all modularized processes
373737© 2005 Cisco Systems, Inc. All rights reserved.
Agenda
• Campus High Availability Design Principles
• Foundation Services
• Multi-Layer Design
• Routed Access Design
• SummaryArchitectural Foundation
Hierarchical Campus Design
Security
Mobility Convergence
Availability
Flexibility
SiSi SiSi
383838© 2005 Cisco Systems, Inc. All rights reserved.
Why Multi-Layer Campus Design?
• Most widely deployed campus design• Supports the spanning of VLANs and Subnets across multiple
access layer switches• Leverages the strength of both Layer 2 and Layer 3
capabilities• Supported on all models of Cisco Catalyst Switches
Access
Distribution
Layer 2
Layer 3
SiSiSiSi
Layer 2
Layer 2
SiSiSiSi
Non-Looped Looped
393939© 2005 Cisco Systems, Inc. All rights reserved.
Multi-Layer DesignBest Practices—Spanning VLANs
• ONLY when you have to!
• More common in thedata center
• Required when a VLAN spans access layer switches
• Required to protect against ‘user side’ loops
• Use Rapid PVST+ for best convergence
• Take advantage of the Spanning Tree Toolkit
Data CenterWAN Internet
Layer 3 Equal Cost Link’s
Layer 3 Equal Cost Link’s
Layer2 Loops
Same VLAN Same VLAN Same VLAN
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi
404040© 2005 Cisco Systems, Inc. All rights reserved.
PVST+ and Rapid PVST+, MSTSpanning Tree Toolkit, 802.1d, 802.1s, 802.1w
• 802.1D-1998: Classic Spanning Tree Protocol (STP)
• 802.1D-2004: Rapid Spanning Tree Protocol (RSTP = 802.1w)
• 802.1s: Multiple Spanning Tree Protocol (MST)
• 802.1t: 802.1d Maintenance, 802.1Q: VLAN Tagging (Trunking)
• PVST+: an instance of STP (802.1D-1998) per VLAN + Portfast, Uplinkfast, BackboneFast, BPDUGuard, BPDUFilter, RootGuard, and LoopGuard
• Rapid PVST+: an instance of RSTP (802.1D-2004 = 802.1w) per VLAN + Portfast, BPDUGuard, BPDUFilter, RootGuard, and LoopGuard
• MST (802.1s): up to 16 instances of RSTP (802.1w); combining many VLANS with the same physical and logical topology into a common RSTP instance; additionally Portfast, BPDUGuard, BPDUFilter, RootGuard, and LoopGuard are supported with MST
A
B
414141© 2005 Cisco Systems, Inc. All rights reserved.
Spanning Tree Toolkit
• PortFast*: Bypass listening-learning phase for access port
• UplinkFast: Three to five seconds convergence after link failure
• BackboneFast: Cuts convergence time by Max_Age for indirect failure
• LoopGuard*: Prevents alternate or root port to become designated in absence of BPDUs
• RootGuard*: Prevents external switches from becoming root
• BPDUGuard*: Disable PortFast enabled port if a BPDU is received
• BPDUFilter*: Do not send or receive BPDUs on PortFast enabled ports
Wiring ClosetSwitch
DistributionSwitches
Root
F
F
F F
F
XB
SiSi SiSi
* Also Supported with MST and Rapid PVST+
424242© 2005 Cisco Systems, Inc. All rights reserved.
Layer 2 HardeningSpanning Tree Should Behave the Way You Expect
• Place the Root where you want it
Root Primary/Secondary Macro
• The root bridge should stay where you put it
RootguardLoopguardUplinkFast (Classic STP only)UDLD
• Only end station traffic should be seen on an edge port
BPDU GuardRoot GuardPortFastPort-security
SiSiSiSi
BPDU Guard or RootguardPortFast
Rootguard
Loopguard
UplinkFast
STP Root
Loopguard
434343© 2005 Cisco Systems, Inc. All rights reserved.
0
5
10
15
20
25
30
35
Tim
ed t
o C
on
verg
e in
S
eco
nd
s
PVST+ Rapid PVST+
To Access To Server Farm
Optimizing Convergence: PVST+ or Rapid PVST+802.1d + Extensions or 802.1s + Extensions
• Rapid-PVST+ greatly improves the restoration times for any VLAN that requires a topology convergence due to link UP
• Rapid-PVST+ also greatly improves convergence time over backbone network fast for any indirect link failures
30 Seconds of Delay/Loss
Tuned Away
444444© 2005 Cisco Systems, Inc. All rights reserved.
Multi-Layer DesignBest Practices—Trunk Configuration
• Typically deployed on interconnection between access and distribution layers
• Use VTP transparent mode to decrease potential for operational error
• Hard set trunk mode to on and encapsulation negotiate off for optimal convergence
• Change the native VLAN to something unused to avoid VLAN hopping
• Manually prune all VLANS except those needed
• Disable on host ports:CatOS: set port host
Cisco Cisco IOS: switchport host
Data CenterWAN Internet
Layer 3 Equal Cost Link’s
Layer 3 Equal Cost Link’s
802.1q Trunks
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi
454545© 2005 Cisco Systems, Inc. All rights reserved.
0
0.5
1
1.5
2
2.5
Tim
e to
Co
nve
rge
in S
eco
nd
s
Trunking Desirable Trunking Nonegotiate
3550 (Cisco IOS)
4006 (CatOS)
4507 (Cisco IOS)
6500 (CatOS)
Optimizing Convergence: Trunk TuningTrunk Auto/Desirable Takes Some Time
• DTP negotiation tuning improves link up convergence time
CatOS> (enable) set trunk <port> nonegotiate dot1q <vlan>
IOS(config-if)# switchport mode trunk
IOS(config-if)# switchport nonegotiate
Voice Data
Two Seconds of Delay/Loss Tuned Away
SiSi
464646© 2005 Cisco Systems, Inc. All rights reserved.
Multi-Layer DesignBest Practices—First Hop Redundancy
• Used to provide a resilient default gateway/first hop address to end stations
• HSRP, VRRP, and GLBP alternatives
• VRRP, HSRP and GLBPprovide millisecond timersand excellent convergence performance
• VRRP if you need multi-vendor interoperability
• GLBP facilitates uplink load balancing
• Tune preempt timers to avoid black-holed traffic
Data CenterWAN Internet
Layer 3 Equal Cost Link’s
Layer 3 Equal Cost Link’s
1st Hop Redundancy
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi
474747© 2005 Cisco Systems, Inc. All rights reserved.
Optimizing Convergence: HSRP TimersHSRP Millisecond Convergence
• HSRP = default gateway redundancy; effects trafficout of the access layerinterface Vlan5description Data VLAN for 6k-accessip address 10.1.5.3 255.255.255.0ip helper-address 10.5.10.20no ip redirectsip pim query-interval 250 msecip pim sparse-modelogging event link-statusstandby 1 ip 10.1.5.1standby 1 timers msec 200 msec 750standby 1 priority 150standby 1 preemptstandby 1 preempt delay minimum 180
Data CenterWAN Internet
Layer 3 Equal Cost Link’s
Layer 3 Equal Cost Link’s
Layer 2 Link’s
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi
484848© 2005 Cisco Systems, Inc. All rights reserved.
0
5
10
15
20
25
30
Tim
e to
Co
nve
rge
in S
eco
nd
s
No Preempt Delay Prempt Delay Tuned
3550 (Cisco IOS)
2950 (Cisco IOS)
4506 (CatOS)
4507 (Cisco IOS)
6500 (CatOS)
6500 (Cisco IOS)
Optimizing Convergence: HSRP Preempt DelayPreempt Delay Needs to Be Longer Than Box Boot Time
More Than 30 Seconds of
Delay/Loss Tuned Away
Without Increased Preempt Delay HSRP Can Go Active Before Box Completely Ready to Forward Traffic L1 (Boards), L2 (STP), L3 (IGP Convergence)standby 1 preempt delay minimum 180
Test Tool Timeout—30 Seconds
494949© 2005 Cisco Systems, Inc. All rights reserved.
First Hop Redundancy with GLBPCisco Designed, Load Sharing, Patent Pending
• All the benefits of HSRP plus load balancing of default gateway utilizes all available bandwidth
• A group of routers function as one virtual router by sharing one virtual IP address but using multiple virtual MAC addresses for traffic forwarding
• Allows traffic from a single common subnet to go through multiple redundant gateways using a single virtual IP address
GLBP AVG/AVF,SVF GLBP AVF,SVF
R1- AVG; R1, R2 Both Forward Traffic
IP: 10.0.0.254MAC: 0000.0c12.3456vIP: 10.0.0.10vMAC: 0007.b400.0101
IP: 10.0.0.253MAC: 0000.0C78.9abcvIP: 10.0.0.10vMAC: 0007.b400.0102
IP: 10.0.0.1MAC: aaaa.aaaa.aa01GW: 10.0.0.10ARP: 0007.B400.0101
IP: 10.0.0.2MAC: aaaa.aaaa.aa02GW: 10.0.0.10ARP: 0007.B400.0102
IP: 10.0.0.3MAC: aaaa.aaaa.aa03GW: 10.0.0.10ARP: 0007.B400.0101
SiSiSiSi
Access-a
Distribution-AGLBP AVG/AVF, SVF
Distribution-BGLPB AVF,SVF
R1
505050© 2005 Cisco Systems, Inc. All rights reserved.
0
0.2
0.4
0.6
0.8
1
1.2
Longest Shortest AverageTim
e in
Se
co
nd
s t
o C
on
ve
rge
VRRP HSRP GLBP
SiSiSiSi
50% of Flows Have ZERO
Loss W/ GLBP
Optimizing Convergence: VRRP, HSRP, GLBP Mean, Max, and Min—Are There Differences?
• VRRP does not have sub-second timers and all flows go through a common VRRP peer; mean, maximum, and minimum are equal
• HSRP has sub-second timers; however all flows go through same HSRP peer so there is no difference between mean, maximum, and minimum
• GLBP has sub-second timers and distributes the load amongstthe GLBP peers; so 50% of the clients are not effected by anuplink failure
GLBP Is 50% Better
Distribution to Access Link FailureAccess to Server Farm
515151© 2005 Cisco Systems, Inc. All rights reserved.
• Both distribution switches act as default gateway• Blocked uplink caused traffic to take less than optimal path
VLAN 2VLAN 2
F 2
F 2
B 2B
2 F: Forwarding
B: Blocking
Access-b
SiSiSiSi
Core
Access-a
Distribution-AGLBP Virtual MAC 1
If You Span VLANS Tuning RequiredBy Default Half the Traffic Will Take a Two Hop L2 Path
Distribution-BGLBP Virtual MAC 2
Access Layer 2Access Layer 2
DistributionLayer 2/3
CoreLayer 3
525252© 2005 Cisco Systems, Inc. All rights reserved.
VLAN 3VLAN 2
Access-b
SiSiSiSi
Core
Access-a
Distribution-AGLBP Virtual MAC 1
Distribution-BGLBP Virtual MAC 2
B x
STP Port Cost
Increased
GLBP + STP turningChange the Blocking Interfaces + VLAN per Access
1. Force STP to block the interface between the distribution switches2. Use the fewest possible VLANs per access switch
Access Layer 2Access Layer 2
DistributionLayer 2/3
CoreLayer 3
535353© 2005 Cisco Systems, Inc. All rights reserved.
VLAN 2VLAN 2
Asymmetric Routing (Unicast Flooding)
• Affects redundant topologies with shared L2 access
• One path upstream and two paths downstream
• CAM table entry ages out on standby HSRP
• Without a CAM entry packet is flooded to all ports in the VLAN
DownstreamPacket
Flooded
Upstream PacketUnicast to Active
HSRP
AsymmetricEqual CostReturn Path
CAM Timer Has Aged out on
Standby HSRP
VLAN 2 VLAN 2
SiSiSiSi
545454© 2005 Cisco Systems, Inc. All rights reserved.
VLAN 2
Best Practices Prevent Unicast Flooding
• Assign one unique voice and as few data VLAN’s as possible to each access switch
• Traffic is now only flooded downone trunk
• Access switch unicasts correctly;no flooding toall ports
• If you have to:Tune ARP and CAM aging timers; CAM timer exceeds ARP timer
Bias routing metrics to remove equal cost routes
DownstreamPacket
Flooded on Single Port
Upstream PacketUnicast to Active
HSRP
AsymmetricEqual CostReturn Path
SiSiSiSi
VLAN 3 VLAN 4 VLAN 5
555555© 2005 Cisco Systems, Inc. All rights reserved.
Keep Redundancy Simple
“If Some Redundancy Is Good, More Redundancy Is NOT Better”
• Root placement?
• How many blocked links?
• Convergence?
• Complex fault resolution
565656© 2005 Cisco Systems, Inc. All rights reserved.
VLAN 2VLAN 2
But Not Too Simple…What Happens if You Don’t Link the Distributions?
• STP’s slow convergence can cause considerable periods of traffic loss
• STP could cause non-deterministic traffic flows/link load engineering
• STP convergence will causeLayer 3 convergence
• STP and Layer 3 timers are independent
• Unexpected Layer 3 convergence and re-convergence could occur
• Even if you do link the distribution switches dependence on STP and link state/connectivity can cause HSRP irregularities and unexpected state transitions
F 2 F 2 B
2
STP Secondary Root and HSRP Standby
F 2
Access-b
SiSiSiSi
Core
Hellos
Traffic
Dropped Until
HSRP Goes Active
Access-a
STP Root and HSRP Active
Traffic Dropped Until
MaxAge Expires Then Listening and
Learning
Traffic Dropped Until Transition to Forwarding; As much as 50 Seconds
575757© 2005 Cisco Systems, Inc. All rights reserved.
• Aggressive HSRP timers limit blackhole #1
• Backbone fast limits time (30 seconds) to event #2
• Even with Rapid PVST+ at least one second before event #2
What If You Don’t?Black Holes and Multiple ‘Transitions’…
• Blocking link on access-b will take 50 seconds to move to forwarding traffic black hole until HSRP goes active on standby HSRP peer
• After MaxAge expires (or backbone fast or Rapid PVST+) converges HSRP preempt causes another transition
• Access-b used as transit for access-a’s traffic
STP Root andHSRP Active
F 2
F 2
B 2
STP Secondary Root and HSRP Standby
F 2
HSRP Active (Temporarily)
MaxAge Seconds Before Failure Is Detected….Then Listening and Learning
F: Forwarding
B: Blocking
Access-b
SiSiSiSi
Core
Hellos
Traffic
Dropped Until
HSRP Goes
Active
F 2
Access-a
Access Layer 2Access Layer 2
DistributionLayer 2/3
CoreLayer 3
VLAN 2VLAN 2
585858© 2005 Cisco Systems, Inc. All rights reserved.
• 802.1d: up to 50 seconds
• PVST+: backbone fast 30 seconds
• Rapid PVST+: address by the protocol (one second)
VLAN 2VLAN 2
What If You Don’t?Return Path Traffic Black Holed…
• Blocking link on access-b will take 50 seconds to move to forwarding return traffic black hole until then
F 2
F 2
B 2F
2 F: Forwarding
B: Blocking
Core
Hellos
Traffic
Dropped Until
MaxAge
Expires Then
Listening and
Learning
F 2
STP Root andHSRP Active
Access-bAccess-a
STP Secondary Root and HSRP Standby
SiSiSiSi
Access Layer 2Access Layer 2
DistributionLayer 2/3
CoreLayer 3
595959© 2005 Cisco Systems, Inc. All rights reserved.
Layer 2 Distribution InterconnectionRedundant Link from Access Layer Is Blocked
• Use only if Layer 2 VLAN spanning flexibility required
• STP convergence required for uplink failure/recovery
• More complex as STP root and HSRP should match
• Distribution-to-distribution link required for route summarization
Trunk
Layer 2
VLAN 20 Data
VLAN 120 Voice
10.1.20.0
10.1.120.0VLAN 40 Data
VLAN 140 Voice
10.1.40.0
10.1.140.0
HSRP Active
and STP Root
VLAN 20,140
HSRP Active
and STP Root
VLAN 40,120
STP Model
Layer 2 Links
Layer 2 Links
SiSiSiSi
Access
Distribution
606060© 2005 Cisco Systems, Inc. All rights reserved.
Layer 3 Distribution InterconnectionNo Spanning Tree—All Links Active
• Recommended ‘best practice’—tried and true
• No STP convergence required for uplink failure/recovery
• Distribution-to-distribution link required for route summarization
• Map L2 VLAN number to L3 subnet for ease of use/management
Layer 3
VLAN 20 Data
VLAN 120 Voice
10.1.20.0
10.1.120.0VLAN 40 Data
VLAN 140 Voice
10.1.40.0
10.1.140.0
HSRP ActiveVLAN 20,140
HSRP ActiveVLAN 40,120
HSRP Model
Layer 2 Links
Layer 2 Links
SiSiSiSi
Access
Distribution
616161© 2005 Cisco Systems, Inc. All rights reserved.
Layer 3 Distribution InterconnectionGLBP Gateway Load Balancing Protocol
• Fully utilize uplinks via GLBP
• Distribution-to-distribution required for route summarization
• No STP convergence required for uplink failure/recovery
Layer 3
VLAN 20 Data
VLAN 120 Voice
10.1.20.0
10.1.120.0VLAN 40 Data
VLAN 140 Voice
10.1.40.0
10.1.140.0
GLBP Model
Layer 2 Links
Layer 2 Links
SiSiSiSi
GLBP ActiveVLAN 20,120,40,140
GLBP ActiveVLAN 20,120, 40, 140
Access
Distribution
626262© 2005 Cisco Systems, Inc. All rights reserved.
VLAN 120 Voice10.1.120.0/24
Layer 3 Distribution InterconnectionReference Design—No VLANs Span Access Layer
• Tune CEF load balancing
• Match CatOS/IOS Etherchannel settings and tune load balancing
• Summarize routestowards core
• Limit redundant IGP peering
• STP Root and HSRP primary tuning or GLBP to load balance on uplinks
• Set trunk mode on/nonegotiate
• Disable Etherchannelunless needed
• Set Port Host on access layer ports:
Disable TrunkingDisable EtherchannelEnable PortFast
• RootGuard or BPDU-Guard
• Use security features
P-t-P Link
Layer 3Layer 3
VLAN 20 Data10.1.20.0/24
VLAN 140 Voice10.1.140.0/24
VLAN 40 Data10.1.40.0/24
SiSi SiSi
SiSi SiSi
Access
Distribution
Core
636363© 2005 Cisco Systems, Inc. All rights reserved.
VLAN 250 WLAN10.1.250.0/24
Layer 2 Distribution InterconnectionSome VLANs Span Access Layer
• Tune CEF load balancing
• Match CatOS/IOS Etherchannel settings and tune load balancing
• Summarize routes towards core
• Limit redundant IGP peering
• STP Root and HSRP primary or GLBP and STP port cost tuning to load balance on uplinks
• Set trunk mode on/nonegotiate
• Disable Etherchannel unless needed
• RootGuard on downlinks
• LoopGuard on uplinks
• Set port host on access Layer ports:
Disable trunkingDisable EtherchannelEnable PortFast
• RootGuard or BPDU-Guard
• Use security features
VLAN 120 Voice10.1.120.0/24
Trunk
VLAN 20 Data10.1.20.0/24
VLAN 140 Voice10.1.140.0/24
VLAN 40 Data10.1.40.0/24
SiSi SiSi
SiSi SiSi
Layer 2
Access
Distribution
Core
646464© 2005 Cisco Systems, Inc. All rights reserved.
Agenda
• Campus High Availability Design Principles
• Foundation Services
• Multi-Layer Design
• Routed Access DesignRouted Access Design
EIGRP Design DetailsEIGRP Design Details
OSPF Design DetailsOSPF Design Details
• SummaryArchitectural Foundation
Hierarchical Campus Design
Security
Mobility Convergence
Availability
Flexibility
SiSi SiSi
656565© 2005 Cisco Systems, Inc. All rights reserved.
Why Routed Access Campus Design?
• Most Cisco Catalyst routers support L3 switching today
• EIGRP/OSPF routing preference over spanning tree
• IGP enhancements; stub router/area, fast reroute, etc..
• Single control plane and well known tool setTraceroute, show ip route, show ip eigrp neighbor, etc…
• It is another design option available to you
Layer 2
Layer 3
SiSiSiSi
SiSiSiSi Access
Distribution
666666© 2005 Cisco Systems, Inc. All rights reserved.
Ease of Implementation
• Less to get right:No STP feature placement coreto distribution
LoopGuardRootGuardSTP Root
No default gateway redundancy setup/tuning
No matching of STP/HSRP priority
No L2/L3 multicast topology inconsistencies
676767© 2005 Cisco Systems, Inc. All rights reserved.
Ease of Troubleshooting
• Routing troubleshooting tools
Show IP route
Traceroute
Ping and extended pings
Extensive protocol debugs
Consistent troubleshooting; access, dist, core
• Bridging troubleshooting tools
Show ARP
Show spanning-tree, standby, etc…
Multiple show CAM dynamic’s to find a host
• Failure differences
Routed topologies fail closed—i.e. neighbor loss
Layer 2 topologies fail open—i.e. broadcast and unknowns flooded
686868© 2005 Cisco Systems, Inc. All rights reserved.
Routed Campus Design Resiliency Advantages? Yes, with a Good Design
• Sub-200 msec convergence for EIGRP and OSPF
• OSPF convergence times dependent on timer tuning
• RPVST+ convergence times dependent on GLBP/HSRP tuning
A B
SiSiSiSi
SiSiSiSi
00.20.40.60.8
11.21.41.61.8
2
MultilayerRPVST+
Routed AccessOSPF
Routed AccessEIGRP
Upstream
Downstream
Sec
on
ds
696969© 2005 Cisco Systems, Inc. All rights reserved.
Routed Access Considerations
• Do you have any Layer 2 VLAN adjacencyrequirements between access switches?
• IP addressing—Do you have enough addressspace and the allocation plan to support arouted access design?
• Platform requirements;
Cisco Catalyst 6500 requires an MSFC in the access to get all the necessary switchport and routing features
Cisco Catalyst 4500 requires a SUP4/5 for EIGRP or OSPF support
Cisco Catalyst 3500s and 3700s require an enhanced Cisco IOS Software image for IGRP and OSPF
707070© 2005 Cisco Systems, Inc. All rights reserved.
EIGRP vs. OSPF as Your Campus IGPDUAL vs. Dijkstra
• Convergence:
Within the campus environment, both EIGRP and OSPF provide extremely fast convergence
EIGRP requires summarization
OSPF requires summarization and timer tuning for fast convergence
• Flexibility:
EIGRP supports multiple levels of route summarization and route filtering which simplifies migration from the traditional Multi-Layer L2/L3 campus design
OSPF area design restrictions need to be considered
• Scalability:
Both protocols can scale to support very large enterprise network topologies
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
OSPF OPSF 12.2S EIGRP
Upstream
Downstream
717171© 2005 Cisco Systems, Inc. All rights reserved.
Routed Access DesignHigh-Speed Campus Convergence
• Convergence is the time needed for traffic to be rerouted to the alternative path after the network event
• Network convergence requires all affected routers to process the event and update the appropriate data structures used for forwarding
• Network convergence is the time required to:
Detect the event
Propagate the event
Process the event
Update the routing table/FIB
SiSiSiSi
SiSiSiSi
727272© 2005 Cisco Systems, Inc. All rights reserved.
Agenda
• Campus High Availability Design Principles
• Foundation Services
• Multi-Layer Design
• Routed Access DesignRouted Access Design
EIGRP Design DetailsEIGRP Design Details
OSPF Design DetailsOSPF Design Details
• SummaryArchitectural Foundation
Hierarchical Campus Design
Security
Mobility Convergence
Availability
Flexibility
SiSi SiSi
737373© 2005 Cisco Systems, Inc. All rights reserved.
Strengths of EIGRP
• Advanced distance vector
• Maps easily to the traditional Multi-Layer design
• 100% loop free
• Fast convergence
• Easy configuration
• Incremental update
• Supports VLSM and discontiguous network
• Classless routing
• Protocol independentIPv6, IPX and AppleTalk
• Unequal cost paths load balancing
• Flexible topology design options
747474© 2005 Cisco Systems, Inc. All rights reserved.
EIGRP Design Rules for HA CampusSimilar to WAN Design, But…
• EIGRP design for the campus follows all the same best practices as you use in the WAN with a few differences
No BW limitations
Lower neighbor counts
Direct fiber interconnects
Lower cost redundancy
HW switching
• WAN stability and speed
• Campus stability, redundancy, load sharing, and high speed
SiSiSiSi
SiSiSiSi
757575© 2005 Cisco Systems, Inc. All rights reserved.
EIGRP in the CampusConversion to an EIGRP Routed Edge
• The greatest advantages of extending EIGRP to the access are gained when the network has a structured addressing plan that allows for use of summarization and stub routers
• EIGRP provides the ability to implement multiple tiers of summarization and route filtering
• Relatively painless to migrateto a L3 access with EIGRP if network addressing scheme permits
• Able to maintain a deterministic convergence time in very large L3 topology
10.10.0.0/1710.10.128.0/17
10.10.0.0/16
SiSi SiSi SiSi SiSi
SiSi SiSi
767676© 2005 Cisco Systems, Inc. All rights reserved.
EIGRP Design Rules for HA CampusLimit Query Range to Maximize Performance
• EIGRP convergence islargely dependent on query response times
• Minimize the number of queries to speed up convergence
• Summarize distribution block routes upstream to the core
Upstream queries are returned immediately with infinite cost
• Configure all access switches as EIGRP stub routers
No downstream queries areever sent
SiSiSiSi
SiSiSiSi
777777© 2005 Cisco Systems, Inc. All rights reserved.
EIGRP NeighborsEvent Detection
• EIGRP neighbor relationships are created when a link comes up and routing adjacency is established
• When physical interface changes state, the routing process is notified
Carrier-delay should be set as a rule becauseit varies based upon the platform
• Some events are detected by therouting protocol
Neighbor is lost, but interface is UP/UP
• To improve failure detectionUse Routed Interfaces and not SVIsDecrease interface carrier-delay to 0Decrease EIGRP hello and hold-down timers
Hello = 1Hold-down = 3
SiSiSiSi
interface GigabitEthernet3/2 ip address 10.120.0.50 255.255.255.252 ip hello-interval eigrp 100 1 ip hold-time eigrp 100 3 carrier-delay msec 0
Hello’s
RoutedInterface
SiSi
SiSi
SiSi
L2 Switchor VLAN Interface
787878© 2005 Cisco Systems, Inc. All rights reserved.
EIGRP Query ProcessQueries Propagate the Event
• EIGRP is an advanced distant vector; it relies on its neighbor to provide routing information
• If a route is lost and no feasible successor is available, EIGRPactively queries its neighbors forthe lost route(s)
• The router will have to receive replies back from ALL queried neighbors before the router calculatessuccessor information
• If any neighbor failsto reply, the queried routeis stuck in active and therouter resets the neighborthat fails to reply
• The fewer routers and routesqueried, the faster EIGRP converges; solution is to limit query range SiSiSiSi
Query
SiSiSiSi
SiSiSiSi
Query
Query
Query
Query
Query
Query
Query
Query
Reply
Reply
Reply
Reply
Reply
Reply
Reply
Reply
Reply
Traffi
c Dro
pped Until
EIGRP C
onverges
Access
Distribution
Core
Distribution
Access
797979© 2005 Cisco Systems, Inc. All rights reserved.
No Queries to Rest of Network
from Core
EIGRP Query ProcessWith Summarization
SiSiSiSi
SiSiSiSi
Query
Query
Query
Reply
Reply
Reply
Reply
∞
Reply
∞
interface gigabitethernet 3/1ip address 10.120.10.1 255.255.255.252ip summary-address eigrp 1 10.130.0.0 255.255.0.0
SummaryRoute
SummaryRoute
Traffi
c Dro
pped Until
EIGRP C
onverges
• When we summarize from distribution to core for the subnets in the access we can limit the upstream query/reply process
• In a large network this could be significant because queries will now stop at the core; no additional distribution blocks will be involved in the convergence event
• The access layer is still queried
808080© 2005 Cisco Systems, Inc. All rights reserved.
EIGRP Stubs
• A stub router signals (through the hello protocol) that it is a stub and should not transit traffic
• Queries that would have been generated towards the stub routers are marked as if a “No path this direction” reply had been received
• D1 will know that stubs cannot be transit paths, so they will not have any path to 10.130.1.0/24
• D1 simply will not query the stubs, reducing the total number of queries in this example to 1
• These stubs will not pass D1’s advertisement of 10.130.1.0/24 to D2
• D2 will only have one path to 10.130.1.0/24
D2D1
Distribution
Access
SiSi SiSi
10.130.1.0/24
“Hello, I’m a Stub…”
“I’m Not Going to Send You Any Queries Since You Said That!”
818181© 2005 Cisco Systems, Inc. All rights reserved.
No Queries to Rest of Network
from Core
EIGRP Query ProcessWith Summarization and Stub Routers
• When we summarize from distribution to core for the subnets in the access we can limit the upstream query/reply process
• In a large network this could be significant because queries will now stop at the core; no additional distribution blocks will be involved in the convergence event
• When the access switches are EIGRP stub’s we can furtherreduce the query diameter
• Non-stub routers do not query stub routers—so no queries will be sent to the access nodes
• No secondary queries—and only three nodes involved in convergence event
SiSiSiSi
SiSiSiSi
Query
Reply
Reply
∞
Reply
∞Stub Stub
SummaryRoute
SummaryRoute
Traffi
c Dro
pped Until
EIGRP C
onverges
828282© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
SiSiSiSi
EIGRP Route Filtering in the CampusControl Route Advertisements
• Bandwidth is not a constraining factor in the campus but it is still advisable to control the number of routing updates advertised
• Remove/filter routes from the core to the access and inject a default route with distribute-lists
• Smaller routing table in access is simpler to troubleshoot
• Deterministic topology
router eigrp 100 network 10.0.0.0 distribute-list Default out <mod/port>
ip access-list standard Default permit 0.0.0.0
838383© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
SiSiSiSi
EIGRP Routed Access Campus DesignSummary
• Detect the event:
Set hello-interval = 1 second and hold-time = 3 seconds to detect soft neighbor failures
Set carrier-delay = 0
• Propagate the event:
Configure all access layer switches as stub routers to limit queries from the distribution layer
Summarize the access routes from the distribution to the core to limit queries across the campus
• Process the event:
Summarize and filter routes to minimize calculating new successors for the RIB and FIB
SummaryRoute
Stub
848484© 2005 Cisco Systems, Inc. All rights reserved.
Agenda
• Campus High Availability Design Principles
• Foundation Services
• Multi-Layer Design
• Routed Access DesignRouted Access Design
EIGRP Design DetailsEIGRP Design Details
OSPF Design DetailsOSPF Design Details
• SummaryArchitectural Foundation
Hierarchical Campus Design
Security
Mobility Convergence
Availability
Flexibility
SiSi SiSi
858585© 2005 Cisco Systems, Inc. All rights reserved.
Open Shortest Path First (OSPF) Overview
• OSPFv2 established in 1991 with RFC 1247
• Goal—a link-state protocol more efficient and scaleable than RIP
• Dijkstra Shortest Path First (SPF) algorithm
• Metric—path cost
• Fast convergence
• Support for CIDR, VLSM, authentication, multipath and IP unnumbered
• Low steady state bandwidth requirement
• OSPFv3 for IPv6 support
868686© 2005 Cisco Systems, Inc. All rights reserved.
Hierarchical Campus DesignOSPF Area’s with Router Types
Data CenterWAN InternetBGP
SiSi SiSi SiSi SiSi SiSi SiSi
SiSi SiSi
SiSi SiSiSiSi SiSi
SiSi SiSi
Area 0
Area 200
Area 20 Area 30Area 10
BackboneBackbone
ABR’s ABR’s
Internal’sInternal’s
Area 0
ABR’s
Area 100
ASBR’s
ABR’s
ABR’s
Area 300
Access
Distribution
Core
Distribution
Access
878787© 2005 Cisco Systems, Inc. All rights reserved.
OSPF Design Rules for HA CampusWhere Are the Areas?
• Area size/border is bounded by the same concerns in the campus as the WAN
• In campus, the lower number of nodes and stability of local links could allow you to build larger areas however…
• Area design also based on address summarization
• Area boundaries should define buffers between fault domains
• Keep area 0 for core infrastructure; do not extend to the access routers
Data CenterWAN Internet
SiSi SiSi SiSi SiSi SiSi SiSi
SiSiSiSi
SiSiSiSi
SiSi SiSiSiSiSiSi
Area 100 Area 110 Area 120
Area 0
888888© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
Backbone Area 0
Area 120
Area Border Router
Regular AreaABRs Forward All LSAs from Backbone
An ABR Forwards the Following into an Area
Summary LSAs (Type 3)ASBR Summary (Type 4)Specific Externals (Type 5)
Access Config:router ospf 100 network 10.120.0.0 0.0.255.255 area 120
Distribution Configrouter ospf 100 summary-address 10.120.0.0 255.255.0.0 network 10.120.0.0 0.0.255.255 area 120 network 10.122.0.0 0.0.255.255 area 0
External Routes/LSA Present in Area 120
SiSiSiSi
898989© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
Backbone Area 0
Area 120
Area Border Router
Stub AreaConsolidates Specific External Links—Default 0.0.0.0
Stub Area ABR ForwardsSummary LSAsSummary 0.0.0.0 Default
Access Config:router ospf 100 area 120 stub network 10.120.0.0 0.0.255.255 area 120
Distribution Configrouter ospf 100 area 120 stub summary-address 10.120.0.0 255.255.0.0 network 10.120.0.0 0.0.255.255 area 120 network 10.122.0.0 0.0.255.255 area 0
Eliminates External Routes/LSA Present in Area (Type 5)
SiSiSiSi
909090© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
SiSiSiSi
Backbone Area 0
Area 120
Area Border Router
A Totally Stubby AreaABR Forwards
Summary Default
Totally Stubby AreaUse This for Stable—Scalable Internetworks
Access Config:router ospf 100 area 120 stub no-summary network 10.120.0.0 0.0.255.255 area 120
Distribution Configrouter ospf 100 area 120 stub no-summary summary-address 10.120.0.0 255.255.0.0 network 10.120.0.0 0.0.255.255 area 120 network 10.122.0.0 0.0.255.255 area 0
Minimize the Number of LSA’s and the Need for Any External Area SPF Calculations
919191© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
SiSiSiSi
Backbone Area 0
Area 120
Area Border Router
ABR’s ForwardSummary 10.120.0.0/16
Summarization Distribution to CoreReduce SPF and LSA Load in Area 0
Access Config:router ospf 100 area 120 stub no-summary network 10.120.0.0 0.0.255.255 area 120
Distribution Configrouter ospf 100 area 120 stub no-summary summary-address 10.120.0.0 255.255.0.0 network 10.120.0.0 0.0.255.255 area 120 network 10.122.0.0 0.0.255.255 area 0
Minimize the Number of LSA’s and the Need for Any SPF Recalculations at the Core
929292© 2005 Cisco Systems, Inc. All rights reserved.
OSPF Default Route to Totally Stubby Area
• Totally stubby area’s are used to isolate the access layer switches from route calculations due to events in other areas
• This means that the ABR (the distribution switch) will send a default route to the access layer switch when the neighbor relationship is established
• The default route is sent regardless of the distribution switches ability to forward traffic on to the core (area 0)
• Traffic could be black holed until connectivity to the core is established
A B
Traffi
c Dro
pped Until
Connectivity
to C
ore
Established
Defau
lt
Route
SiSi
SiSi
SiSi
SiSi
Note: Solution to this anomaly is being investigated.
939393© 2005 Cisco Systems, Inc. All rights reserved.
SiSi
SiSi
OSPF Timer TuningHigh-Speed Campus Convergence
• OSPF by design has a number of throttling mechanisms to prevent the network from thrashing during periods of instability
• Campus environments are candidates to utilize OSPF timer enhancements
Sub-second hellos
Generic IP (interface) dampening mechanism
Back-off algorithm for LSA generation
Exponential SPF backoff
Configurable packet pacing
SiSi
SiSi
Reduce Hello Interval
ReduceLSA and SPF
Interval
949494© 2005 Cisco Systems, Inc. All rights reserved.
Subsecond Hello’sNeighbor Loss Detection—Physical Link Up
• OSPF hello/dead timers detect neighbor loss in the absence of physical link loss
• Useful in environments where an L2 device separates L3 devices (Layer 2 core designs)
• Aggressive timers are needed to quickly detect neighbor failure
• Interface dampening is recommended if sub-second hello timers are implemented
Traffi
c Dro
pped Until
Neighbor Loss
Detection O
ccurs
OSPF Processing
Failure(Link Up)
A B
SiSi
SiSi
SiSi
SiSi
Access Config: interface GigabitEthernet1/1dampening ip ospf dead-interval minimal hello-multiplier 4
router ospf 100area 120 stub no-summarytimers throttle spf 10 100 5000timers throttle lsa all 10 100 5000timers lsa arrival 80
959595© 2005 Cisco Systems, Inc. All rights reserved.
Access Config:interface GigabitEthernet1/1ip ospf dead-interval min hello-multi 4
router ospf 100area 120 stub no-summarytimers throttle spf 10 100 5000timers throttle lsa all 10 100 5000timers lsa arrival 80
OSPF LSA Throttling
• By default, there is a 500ms delay before generating router and network LSA’s; the wait is used to collect changes during a convergence event and minimize the number of LSA’s sent
• Propagation of a new instance of the LSA is limited at the originator
timers throttle lsa all <start-interval>
<hold-interval> <max-interval>
• Acceptance of a new LSAs is limited by the receiver
timers lsa arrival <milliseconds> Traffic Dropped U
ntil
LSA G
enerated and
Processed
A B
SiSi
SiSi
SiSi
SiSi
969696© 2005 Cisco Systems, Inc. All rights reserved.
OSPF SPF Throttling
• OSPF has an SPF throttling timer designed to dampen route recalculation (preserving CPU resources) when a link bounces
• 12.2S OSPF enhancements let us tune this timer to milliseconds; prior to 12.2S one second was the minimum
• After a failure, the router waits forthe SPF timer to expire before recalculating a new route; SPF timer was one second
Traffi
c
Dropped U
ntil
SPF Timer
Expires
Access Config:interface GigabitEthernet1/1ip ospf dead-interval min hello-multi 4
router ospf 100area 120 stub no-summarytimers throttle spf 10 100 5000timers throttle lsa all 10 100 5000timers lsa arrival 80
A B
SiSi
SiSi
SiSi
SiSi
979797© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
SiSiSiSi
OSPF Routed Access Campus DesignOverview—Fast Convergence
• Detect the event:Decrease the hello-interval and dead-interval to detect soft neighbor failures
Enable interface dampening
Set carrier-delay = 0
• Propagate the event:Summarize routes between areasto limit LSA propagation acrossthe campus
Tune LSA timers to minimize LSA propagation delay
• Process the event:Tune SPF throttles to decrease calculation delays
StubArea120
BackboneArea
0
989898© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
SiSiSiSi
OSPF Routed Access Campus DesignOverview—Area Design
• Use totally stubby areas to minimize routes in Access switches
• Summarize area routes to backbone Area 0
• These recommendations will reduce number of LSAs and SPF recalculations throughout the network and provide a more robust and scalable network infrastructure
Area Routes Summarized
router ospf 100 area 120 stub no-summary summary-address 10.120.0.0 255.255.0.0 network 10.120.0.0 0.0.255.255 area 120 network 10.122.0.0 0.0.255.255 area 0
router ospf 100 area 120 stub no-summary network 10.120.0.0 0.0.255.255 area 120
Configured asTotally Stubby
Area
999999© 2005 Cisco Systems, Inc. All rights reserved.
SiSiSiSi
SiSiSiSi
OSPF Routed Access Campus DesignOverview—Timer Tuning
• In a hierarchical design, the key tuning parameters are SPF throttle and LSA throttle
• Need to understand other LSA tuning in the non-optimal design
• Hello and dead timers are secondary failure detection mechanism
Reduce Hello Interval
Reduce SPF andLSA Interval
router ospf 100 area 120 stub no-summary area 120 range 10.120.0.0 255.255.0.0 timers throttle spf 10 100 5000 timers throttle lsa all 10 100 5000 timers lsa arrival 80 network 10.120.0.0 0.0.255.255 area 120 network 10.122.0.0 0.0.255.255 area 0
interface GigabitEthernet5/2 ip address 10.120.100.1 255.255.255.254 dampening ip ospf dead-interval minimal hello-multiplier 4
100100100© 2005 Cisco Systems, Inc. All rights reserved.
Agenda
• Campus High Availability Design Principles
• Foundation Services
• Multi-Layer Design
• Routed Access Design
• Summary Architectural FoundationHierarchical Campus Design
Security
Mobility Convergence
Availability
Flexibility
SiSi SiSi
101101101© 2005 Cisco Systems, Inc. All rights reserved.
Campus High AvailabilityNon-Stop Application Delivery
Data CenterWAN Internet
SiSi SiSi SiSi SiSi SiSi SiSi
SiSi SiSi
SiSi SiSiSiSi SiSi
SiSi SiSi
Access
Distribution
Core
Distribution
Access
Hierarchical, systematic approach• System level resiliency for switches and routers• Network resiliency with redundant paths• Supports integrated services and applications• Embedded management
102102102© 2005 Cisco Systems, Inc. All rights reserved.
VLAN 120 Voice10.1.120.0/24
Multi-Layer Campus DesignReference Design—No VLANs Span Access Layer
• Tune CEF load balancing
• Match CatOS/IOS Etherchannel settings and tune load balancing
• Summarize routestowards core
• Limit redundant IGP peering
• STP Root and HSRP primary tuning or GLBP to load balance on uplinks
• Set trunk mode on/nonegotiate
• Disable Etherchannelunless needed
• Set Port Hoston access layer ports:
Disable TrunkingDisable EtherchannelEnable PortFast
• RootGuard or BPDU-Guard
• Use security features
P-t-P Link
Layer 3
VLAN 20 Data10.1.20.0/24
VLAN 140 Voice10.1.140.0/24
VLAN 40 Data10.1.40.0/24
SiSi SiSi
SiSi SiSi
Access
Distribution
Core
103103103© 2005 Cisco Systems, Inc. All rights reserved.
VLAN 250 WLAN10.1.250.0/24
Multi-Layer Campus DesignSome VLANs Span Access Layer
• Tune CEF load balancing
• Match CatOS/IOS Etherchannel settings and tune load balancing
• Summarize routes towards core
• Limit redundant IGP peering
• STP Root and HSRP primary or GLBP and STP port cost tuning to load balance on uplinks
• Set trunk mode on/nonegotiate
• Disable Etherchannel unless needed
• RootGuard on downlinks
• LoopGuard on uplinks
• Set Port Hoston access Layer ports:
Disable trunkingDisable EtherchannelEnable PortFast
• RootGuard or BPDU-Guard
• Use security features
VLAN 120 Voice10.1.120.0/24
Trunk
VLAN 20 Data10.1.20.0/24
VLAN 140 Voice10.1.140.0/24
VLAN 40 Data10.1.40.0/24
SiSi SiSi
SiSi SiSi
Layer 2
Access
Distribution
Core
104104104© 2005 Cisco Systems, Inc. All rights reserved.
Routed Access Campus DesignNo VLANs Span Access Layer
• Use EIGPR or OSPF
• Use Stub routers or Stub Areas
• With OSPF tune LSA and SPF timers
• Summarize routes towards core
• Filter routes towards the access
• Tune CEF load balancing
• Disable Etherchannelunless needed
• Set Port Hoston access layer ports:
Disable TrunkingDisable EtherchannelEnable PortFast
• RootGuard or BPDU-Guard
• Use security features
VLAN 120 Voice10.1.120.0/24
P-t-P Link
Layer 3
VLAN 20 Data10.1.20.0/24
VLAN 140 Voice10.1.140.0/24
VLAN 40 Data10.1.40.0/24
SiSi SiSi
SiSi SiSi
Access
Distribution
Core
105105105© 2005 Cisco Systems, Inc. All rights reserved.
Cisco Campus Architecture Multiple Design Options supporting Integrated Services
Enterprise CampusThe Right Design for Each Customer
Virtualizatio
n
Future Services
Integrated Security
WLAN Integration
Intelligent Switching(Hybrid of L2 + L3 features)Intelligent Switching(Hybrid of L2 + L3 features)
FutureCampusDesignOptions
FutureCampusDesignOptions
Multi-LayerCampusDesign
Multi-LayerCampusDesign
High Availability
• High Availability
• IP Communications
• WLAN Integration
• Integrated Security
• IPv6
• Virtualization
• Future Services
IP Communications
Cisco Campus Architecture
IPv6
RoutedCampusDesign
RoutedCampusDesign
106106106© 2005 Cisco Systems, Inc. All rights reserved.
RST-203111207_05_2005_c1