View
1.945
Download
1
Embed Size (px)
Citation preview
#ATM16
Building the right DC FabricPhilippe Michelet, Senior Director, GPLM Data CenterHPE ArubaMarch 2016
@ArubaNetworks |
2#ATM16
Agenda– Data Center Fabric: a Definition and an introduction
– HPE Aruba Data Center Fabric: a Flexible Approach– Layer 2 / Layer 3 / Overlay– LAN/SAN Convergence- In between multiple Data Centers – Data Center Interconnection
- The Foundation for Network Virtualization
- New !!! OpenSwitch on Altoline – the Most Open Network Operating System in the industry
- Conclusion
3#ATM16
Data Center FabricA Definition
4#ATM16
Data Center FabricDefinition/Goals
– How do you optimally interconnect 1000, 10 000, 100 000 or more end points (servers, storage)– E/W traffic vs N/S traffic
– Servers to servers inside the DC– Clients to servers entering/servers to clients leaving the DC
– Full bisectional bandwidth– Every end point has equal BW (TX/RX) with every other peer in the fabric
– Minimizing the number of connections– Cables/fiber/transceivers can represent 50% of the cost of a Data Center
– Minimizing the number of hops to reach out any other peer in the fabric– Latency impact
– Providing Redundancy when any node or any link fails– Failure will happen – it’s just a question of time
– Being Flexible– Can’t rewire a complete Data Center when going from 10000 to 10001 or 100000 to 100001 end points
5#ATM16
How the Graph Theory can help you...Seven Bridges of Konigsberg, Leonhard Euler, 1736
– What is the most efficient way to connect n nodes
– Large scale/Tier 1: >100 000 nodes– Typically using [32-48] port switches– Choice of Different Topologies
–Full Mesh–Line (1-dimension cube)–Rings–Cubes,–Etc.
– With Metrics–Switch Radix–Max Hops–Total Switches–Total Ports–Total Links
6#ATM16
Standard Enterprise DC Deployment?CLOS/Fat Tree
Note: Connectivity shown for some path only for clarity
– In a Fat Tree, there are as many edge ports as spine portsNE = NS
– Now the most deployed solution up to ~5000 edge ports
– Advantages:
– Non-blocking architecture
– Constant Latency
– Fat Trees have a constant bisectional bandwidth
– Bisection bandwidth scales linearly with the number of nodes
– There are alternate paths for resiliency
– FormulasFT (m,h): each node has m/2 children and m/2 parentsm = number of ports per switchh = tree levelmachines = m*(m/2)^(h-1)switches = (2h-1)* (m/2)^(h-1)example: m =24, if h=2 -> 2048 ports if h=3 -> 3456 ports
7#ATM16
Example of a CLOS Network Design One of the “Big Seven”
T3 T3 T3 T3
T2 T2 T2 T2 T2 T2 T2 T2
T1 T1 T1 T1 T1 T1 T1 T1
ToR-1 ToR-20
T2 T2 T2 T2
T1 T1 T1 T1 T1 T1 T1 T1
ToR-1 ToR-20
T2 T2T2 T2
8#ATM16
HPE Aruba Flexible ApproachSoftware Defined vs Hardware Defined
9#ATM16
So, which Data Center Fabric are you building in 2016?There is not a single answer
L R
x N
L R
x N
L R
x N
L R
x N
Ethernet & VLANS IP routing domain
L R
x N
L R
x N
L R
x N
L R
x N
L R
x N
L R
x N
L R
x N
L R
x N
L2 Fabric L3 Fabric Overlay
10#ATM16
Why using Layer 2 in 2016?
– Legacy support– It works … “If it ain’t broke, don’t fix it”
– Some customers just don’t have the bandwidth to make big changes
– Some applications can’t be easily rewritten so Layer 2 extension remains a must have– And introducing overlays is a no go considering the complexity (data/control and management planes completely different)
– From a pure technical perspective, Layer 2 is not dead– Standardization around distributed ling aggregation is done – 802.1AX-2014 giving all vendors an opportunity to still
use Layer 2, but without any STP like solution (or STP used an “insurance” policy)– HPE Aruba on the FlexFabric side is working on this implementation
– Proprietary implementations have been working for many years– HPE Aruba Intelligent Resilient Fabric (IRF) – more than 5 years with the big chassis and ToR
– If TRILL remains a niche (supported by FlexFabric products), PBB/SPB provides an elegant alternative– Compatible with all Ethernet L2 protocols– Has a native OAM (Operation & Maintenance) – something that overlays are struggling with– Simpler in nature than new overlays like VXLAN– Has been used in production very successfully by HP IT since 2011
11#ATM16
Why using Layer 3 in 2016?
– Technically sounder than Layer 2– No STP (always very complex to manage …)– Broadcast/Security concerns– Proven at scale / Reduces size of failure domains
– The choice made by the majority of big cloud providers– Draft RFC on this subject (Facebook/Microsoft): “Use of BGP for routing in large-scale data centers” here– Limited and controller number of applications – different from vast majority of Enterprise IT
– Simpler by nature if you know BGP– Simpler control plane protocols– eBGP (different AS for each ToR) /ECMP (Equal Cost Multi Path) / BFD (Bi Directional Forwarding Detection)
– Power of BGP, combined to ECMP for load distribution and BFD for protocol convergence acceleration (~200ms)
– Considered more “secure” even if more work will be required
– All protocols completely supported by FlexFabric portfolio and the foundation of the first release of OpenSwitch
12#ATM16
Why using Overlays in 2016?
– Scalability – goes beyond the 802.1Q VLAN limitations (12bits/4096 IDs)– Typically 16M services/tenants
– Essentially driven by VM mobility – L2 extension– VXLAN as de-facto solution by VMware (NSX as part of their SDDC initiative)
– Encapsulation over IP – ability to cross L3 boundaries
– The fabric becomes a big L3 domain with L2 processing (encapsulation / de-capsulation) at the edge (NIC or Leaf/ToR)– Separation between “underlay” (L3 Fabric per previous slide) from the “overlay” (Hypervisor/Leaf/Tunnel instantiation)
– All DC fabric vendors do have an overlay solution today – including HPE Aruba with FlexFabric
– But keep in mind that careful attention is required– Different data plane (additional header) makes Jumbo Frames a must have and will continue to evolve …– Standardization around control plane is still work in progress (even if BGP EVPNs are here)– Management is still a big issue – how do you quickly identify the root cause of the problem
– Is it the underlay? Is it the overlay?
13#ATM16
L2/L3/Overlays – Pros/Cons - My PerspectiveL2 L3 Overlays Comments
Maturity High/Very high High/Very high Low (VXLAN)If the concept of overlay is not new, VXLAN is quite
recent
Interoperability Well understood Well understood Control Plane still work in progress BGP EVPN
Tenants (Scalability) 4K – 802.1Q16M – PBB (802.1ah)
SW implementation (VRF/VPNs) 16M VXLAN/VNIDs 24 bits / 16M tenants
Stability LowThe key issue !!! High Jury is still out …
Convergence Time 45s (standard STP timers)3 ~ 5 s (RSTP)
BFD ~200ms (*)OSPF/BGP ~5 to 10s
Real dependencies on Underlay ..
Security (broadcast) Low Medium/High Medium
Multicast Well understood Well understood Lot already done, but still work in progress
OAM Ethernet OAM Protocol by Protocol Still many improvements required … Area of innovation
14#ATM16
FlexFabric – What you change is the SW – Same HWDC network Design Option 1 Option 2 Option 3
Traditional 3 Tier IRF MSTP PVST/PVST+
Layer 2 Spine & Leaf IRF MSTP PBB/SBP (IS-IS)
Layer 3 BGP (v4/v6) OSPF (v4/v6) IS-IS (v4/v6)
L3 Overlay IS-IS with centralized L3 GW
EVPN with centralized L3 GW
EVPN with distributed L3 GW
LAN/SAN Converged Separate document available on hpe.com describing use case
DC Interconnect EVI (MACoverGRE) EVI 2.0 (VXLAN/EVPN) MPLS/VPLS
DCN with ToR (59xx) Supported
OpenStack with ToR (59xx) Supported
NSX-v with 59xx On-going certification with VMware
15#ATM16
HPE Data Center FlexFabric for Spine/Leaf DeploymentThe industry’s best field tested and tried Ethernet fabric
Modular network OS with Intelligent Resilient Fabric
1/10/40GbE L2/L3 and converged switches25GbE/100GbE (5950/Modular 5950)
HP
IMC
Man
agem
ent Core switches
Spine
HP Comware Network OS
L2/L3 IPv4/v6MPLS/VPLS
VXLAN
Top of Rack (TOR)
Leaf switches
12900E 7910 7904
HP
Tec
hnol
ogy
Ser
vice
sH
P C
onsu
lting
Ser
vice
s
SDNNetVirt
5900 5900CP 5930/5940 Modular 5930/5940
16#ATM16
HPE Data Center FlexFabric for LAN/SAN ConvergenceIdeally positions existing FC/FCoE customers to transition to IP Storage
Native FC Ethernet FCoE
Ethernetswitches
NIC HBA
Server
FCswitches
FC SAN Ethernet LAN
Server
NIC HBA CNA
Server
Convergedswitches
CNA
Server
FC SAN Ethernet LAN
FCoE Now: 50% CAPEX reduction, 66% OPEX reductionBefore: multiple networks
17#ATM16
Data Center Interconnect and EVI• HPE Ethernet Virtual Interconnect (EVI) can be deployed for active/active DC over any Network• EVI provides additional benefits such as:
− Transport agnostic− Up to 16 Active/Active DCs− Active/Active VRRP default gateways for VMs− STP outages remain local to each DC− Improves WAN utilization by dropping unknown frames and providing ARP suppression
Virtual Overlay VXLAN tunnels
Physical Underlay Network
Active Data Center (DC) 1
L2 or L3
Virtual Overlay VXLAN tunnels
Physical Underlay Network
Active Data Center (DC) 2
L2 or L3
L3 WANVM VM VMVM VM VM
HypervisorVM VM VMVM VM VM
HypervisorVM VM VMVM VM VM
HypervisorVM VM VMVM VM VM
Hypervisor
EVI tunnel
18#ATM16
Orchestrate network fabrics
Complete the SDN architecture with management
Accelerates deployment of services and applications
•Unified IRF/PBB/SPB/TRILL, fabric management
•Manages across geographically dispersed locations (DCI/EVI)
•VMware vMotion playback•Unified DCB, FCoE management
•Configuration, monitoring & policy management for all SDN layers
•OpenFlow switch management•SDN controller performance management
•One application for managing SDN and traditional environments
• “Just right” network services tuned to business requirements
•Simplifies provisioning, monitoring and troubleshooting of applications
•Eliminates manual provisioning of network service parameters
•Easy to use service modeling tool with drag and drop UI
IMC Orchestration for Data Center
19#ATM16
HPE FlexFabricThe Foundation for SDN/Network Virtualization
20#ATM16
HPE Network Virtualization SolutionsIndustry’s most complete portfolio - addressing varied/complex use cases
20
Virtual Cloud Network HPE-VMware NSX Distributed Cloud Network
Enhanced Neutron Networking
Open source, DevOps Virtualized VMW Enterprises Service Provider/Telcos
De-risk the journey with HPE Trusted Network Transformation Services
DCNVCN
Multi DC NFV
Powered by HPE
21#ATM16
New !!! HPE Altoline / OpenSwitchThe Most Open NOS in the Industry
22#ATM16
Lower TCONo vendor Lock inFaster Time to Service
Customer choiceAgile and scalable Affordable capacity
HPE Altoline trusted open network switches solution Accelerate disaggregation of cloud data center networks
• Optimized for scalable and agile cloud deployments
• Faster provisioning & time to service
• HPE worldwide service & support• HPE Technology Services
expertise
• HPE Altoline switches for open networking
• Open source, or commercial Linux OS license from HPE
• Global component buying options• Global HPE support and services
• Direct sales and purchasing Lower CapEx and OpEx
• Open source, or commercial Linux tools and resources
• Consistent automation and SDN
23#ATM16
HPE Altoline deployment modelsTop-of-rack spine-leaf switches for cloud data centers
Spine 6940 switch
Leaf 6920 switch
HPE Altoline 6940 Spine TOR32*40GbE ports
HPE Altoline 6920 Leaf TOR48*10GbE ports + 6*40GbE ports
Servers
AltolineToR Switch
24#ATM16
What is OpenSwitch? More details on openswitch.net
Community Based
•Launched with 8 charter contributors•Over 90 non-HPE people / 30 companies on mailing lists•Active weekly IRC chats•Sample story:•LinkedIn said on IRC that they would be interested, but want to see Ansible support•Ansible jumped in, saying they want to help design the Ansible integration•Several IRC chats and open email discussions since then•Ansible looking to use OpenSwitch as template for native Ansible support
Open Source
•All HPE code for OpenSwitch is in publicly available git, mirrored to github•All under Apache 2.0 (except some leveraged projects e.g. Quagga)•Anyone can download the source, tinker, build for all supported platforms•All HPE development for TOR is done upstream first•Leverage Yocto build recipes and linux menuconfig: build only the components you need
Full Feature NOS
•L3: using Quagga, with significant investment to further enhance. BGP, OSPF, …•L2: open-sourcing internally developed protocols. MSTP, mLAG, …•Classic Management: CLI, SNMP•GUI: Web-UI•Programmatic Management: REST, Ansible, Direct OVSDB, …•Open vSwitch DB used for all state•Highly available, per-service restartable
25#ATM16
HPE Data Center SolutionsOpen StandardsOpen APIsOpen EcosystemOpen Source
26#ATM16
HPE Data Center Solutions – Built to Win
High Performance DC Fabric
High Density / High Performance / Highly Scalable / Highly Resilient
Composable Infrastructure / SDN Network Virtualization
“Instantiating open, complex networks and associated policies in minutes vs. weeks”
Zero Touch Provisioning- DC Fabric
IMC Platform (ZTP / Fabric Manager)DevOps (Python, Ansible …)
Hybrid Cloud Integration Helion/Openstack/CSA integration
27#ATM16
Customers & Analysts trust us … Will you be next?Solution: Entire HPE solution (Server/Storage/Networking/Technology Services)“We chose HPE and got more than what we asked for. We wanted to standardize our infrastructure and go with a single vendor to build our data center and reduce management complexity” Wahid S. Hammami, CIO,Ministry of Petroleum and Mineral Resourceshttp://h20195.www2.hp.com/V2/GetDocument.aspx?docname=4AA6-3550ENW&cc=us&lc=en
Solution: DC Core 12916“We have the largest capacity flagship core switch HPE sells, and with that comes all the flexibility we’ll ever need. With a 16-slot chassis, and 720 10Gb Ethernet ports, it’s really a remarkable network core that will support whatever we want to do for the next 10 years.” Bruce Allen, Director, Max Planck Institute for Gravitational Physicshttp://h20195.www2.hp.com/V2/GetDocument.aspx?docname=4AA5-9943ENW
Solution: 2 data centers and launch new services in 8 months “HP Networking solutions gave us the ability to rapidly scale our ShoreTel Skyvoice communications capacity from 130,000 users to more than a million usersin less than 8 months—and that’s a huge thing for us.” Dennis Schmidt, VP Network Engineering, ShoreTel
More References here
28#ATM16
Join Aruba’s Titans of Tomorrow force in the fight against network mayhem. Find out what your IT superpower is.
Share your results with friends and receive a free superpower t-shirt.
www.arubatitans.com
Thank [email protected] – 1 408 504 9514