Data Center Network Design - clnv.s3.amazonaws.com · BRKARC-3222 Cisco Nexus 9000 Architecture Tim Stevenson Check on CiscoLive App ... • Cisco alternatives for campus design:

Data Center Network Design for Midsize Enterprises

Fausto Vaninetti, Data CenterEMEAR DAT Consultant Systems Engineer

BRKDCT-2218

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKDCT-2218


Start small

… then grow

…until you get to something you’re happy with

Hint: leverage yet expand your skill set (even in IT)

EEEeducation exposure experience

BRKDCT-2218 4

• Introduction

• Beyond STP, from Networks to Fabrics

• Scaling Brownfield and Greenfield Data Center Networks

• Data Center Interconnect Solutions

• Feature-Specific Considerations

• Programmability, Automation & Orchestration

Agenda


LOTS of Related Sessions, My Recommended ListSession ID Title Presenter Date / Time

BRKACI-2125 ACI Multi-Site Architecture and Deployment Max Ardica Check on CiscoLive App

BRKDCT-2949 Building Data Center networks with VXLAN BGP-

EVPN Part I

Lukas Krattiger Check on CiscoLive App

BRKDCT-3378 Building Data Center networks with VXLAN BGP-

EVPN Part II

Lukas Krattiger Check on CiscoLive App

BRKACI-2003 Cisco ACI MultiPod Design and Deployment John Weston Check on CiscoLive App

BRKARC-3222 Cisco Nexus 9000 Architecture Tim Stevenson Check on CiscoLive App

BRKDEV-1368 Effectively Understand and Leverage YANG with

NETCONF and RESTCONF for Model Driven

Programmability

Hank Preston Check on CiscoLive App

BRKACI-2508 Nexus standalone fabrics migration to ACI Ricardo Trentin Check on CiscoLive App

BRKDCN-2657 Design and Implementation of DCI Networks Yves Louis Check on CiscoLive App

BRKDCT-2218 6

• Introduction






Agenda

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8BRKDCT-2218

Some European Statistics

Midsize enterprise:

• 50 < employees < 250

• € 10M < Revenue < € 50M

Depends on Geography

Out of 21M enterprises

http://ec.europa.eu/index_en.htm

Micro Enterprises

Small Enterprises

Medium Enterpries

Large Enterprises


Defining “Midsize”

Require dedicated DC Switches, moving away from collapsed core with campus

Mostly virtualized, some physical servers

Scalability

Size for current needs, reuse components in larger designs

Design Options

Feature choice + priority = tradeoffs

No such a things as the perfect recipe’

Where the industry is going:

Programmability, Automation (SDN buzz)

Designing Data Center Networks for Midsize Enterprises

FC

FCoE

iSCSI / NAS

L3-----------

L2

Campus

Client Access

WAN / DCI

BRKDCT-2218 9


What are you ready for?

Decisions will depend on where you draw the line:

• Want to stay with existing toolsets for config & management?

• Capable of consuming a new set of tools in the hope of some efficiency?

• New or traditional operational model?

• Resist/accept pressure from other teams or business owners?


Design Goals

Flexible

Reliable

Easy

© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12

Network Topologies

Ring Star Fully Connected Bus

N-Tiered Spine LeafTree

BRKDCT-2218


Which network model would YOU choose?Example: 350 VM’s; 25 Baremetal Servers @ 1/10GE; HA; IP Based Storage

13

……

Collapsed/Single Tier

VXLAN

Spine Leaf

L3

L2

L3

96 front

facing ports

L296 front

facing ports

Downlink ports: 48p Uplink/Breakout ports: 6-12p

1RU

BRKDCT-2218


Single-Tier, Dual-Tier, Spine/Leaf

Small Spine/Leaf

VXLAN

Dual Tier DC

Single Layer DC

Scalable Spine/Leaf DC Fabric

VXLAN

Scale by Tiering

Scale Out


Connectivity & Features Drive Design Choices

VM VMVM

FCoE

iSCSI

FC

NFS/

CIFS

VM VMVM

Connectivity Model

– 25 or 10 or 1-GigE Server ports

– NIC/HBA Interfaces per-server

– Copper vs Fiber cabling to ToR

Virtual Networking Requirements

– vSwitch/DVS/OVS/Nexus1Kv/AVS

Programmabiltiy/Automation/Orchestration

– Complete abstraction

– Device or fabric level programmability

Computing Form Factor

– Unified Computing Fabric

– 3rd Party Blade Servers

– Rack Servers (Non-UCS Managed)

Storage & Storage Protocols

– Native Fibre Channel

– Unified Ports, FCoE

– IP-based storage (iSCSI, NAS)

BRKDCT-2218 15


Data Center Network Needs

• Server virtualization is here

• Higher P2V ratio, access port bandwidth up

• Oversubscription ratio from 20:1 to 1:1

• Horizontal workload mobility

• Any VLAN everywhere, no physical boundaries

• Highly distributed applications

• Hadoop and microservices

• More server to server communication

• 80/20 traffic pattern reversed?

• Increasing high availability requirements

• Where did my maintenance window go?

• More execution speed, less human errors

16BRKDCT-2218

EAST – WEST TRAFFIC

NO

RT

H-

SO

UT

HT

RA

FF

IC

FC

FCoE

iSCSI / NAS

Server/Compute

Site BEnterprise

Network

PublicCloud

Internet

DATA CENTER

NETWORK

Mobile

Services

Storage

Orchestration/

Monitoring

Offsite DC

API


From Networks To Fabrics

• Flexibility: allows workload mobility, VLANs everywhere

• Robustness: reduce L2 failure domains, L2/L3 boundary on leafs, anycast gateway

• (Virtual) Network services: moving out to border leafs, policy-based service chaining

• Performance: full cross sectional bandwidth (any-to-any) with ECMP, avoid oversubscription

• Latency: deterministic at scale, single hop away

• Scalability: add end nodes, maintain oversubscription

• Cost: fixed switches vs modular switches

17BRKDCT-2218

EAST – WEST TRAFFIC

NO

RT

H-

SO

UT

HT

RA

FF

IC

FC

FCoE

iSCSI / NAS

Server/Compute

Site BEnterprise

Network

PublicCloud

Internet

DATA CENTER FABRIC

Mobile

Services

Storage

Orchestration/

Monitoring

Offsite DC

API

Definition: ensemble of switches that behave and get configured like a single giant switch


Network Implications of NVM/NVMe/NVMe-oF

• Rule of thumb: 1M IOPS need 40G of bandwidth

• 4x 10Gbps (or equivalent FC bandwidth) connectivity required in 2018

• 18x 40Gbps (or equivalent FC bandwidth) in 2022

• Making leaf-spine networks a necessity? More high-speed ports on access switches?

NVM storage as a key force driving the transition to higher bit rates

BRKDCT-2218 18

• Introduction






Agenda


History Lesson: Spanning tree

• Spanning Tree introduced around 1985

• 32 years ago, we also saw:

• Windows 1.0

• DNS come out of academia

• First Nintendo Entertainment System

• STP prevents loops, but blocked ports and slow re-convergence

• Successfully deployed for some time…

• …but since a few years, most DC Designs built to work around STP

BRKDCT-2218 20

Host or

Switch


Virtual Port Channel (VPC)

• VPC invented to overcome STP limitations

• IEEE standard in 2000 (802.3ad)

• Not perfect, but a good workaround

• STP is still there on every link

• Human error, misconfiguration, bug can still cause issues

• Cisco alternatives for campus design: VSS, StackWise Virtual

BRKDCT-2218 21

Host or

Switch

vPC Domain


Virtual Port Channel (VPC) “Mini-Fabric”

• VPC Northbound & Southbound

• More efficient than native STP

• STP is still running

• Another good workaround

• Configuration can become complex as switch counts grow

• vPC makes two switches look as one….but what about 4 switches?

22BRKDCT-2218

Host or

Switch

vPC Domain 1

vPC Domain 2

Dual sided vPC

Back-to-Back vPC

“mini-fabric”


L3-Based Fabrics

• Every link forwarding

• L3 ”routing” fast convergence

• VXLAN overlay (MAC in UDP)

• Flood and learn vs VXLAN BGP-EVPN control plane

• STP might still exist on edges, but not within the “fabric”

• VPC still needed at edge

• Spine/Leaf: • Flexible and efficient design

• Consistent hop count & latency

BRKDCT-2218 23

Host or

Switch

VXLAN

Cisco Live Session: BRKDCT-2949

Building Data Center networks with VXLAN BGP-EVPN Part I

Cisco Live Session: BRKDCT-3378

Building Data Center networks with VXLAN BGP-EVPN Part II


Why Spine-Leaf Design? Flexibility and Efficiency

96 ports

2x48 10G (960 Gbps total)

BWFabric

10G host ports

40G fabric ports

BWEdge

M fabric links per leaf

Oversubscription at leafs if: M x BWFabric < N x BWEdge

N ports

Scale ports and bandwidth independently



Need even more

host ports?

Add another leaf

To speed up FCT,

boost the backplane

by spreading load

across more spines

BWFabric

192 ports

4x48 10G (1920 Gbps total)P

er

Sp

ine

Utiliz

atio

nF

CT

FC

T

FC

T

10G host ports

40G fabric portsFCT = Flow

Completion Time

BWEdge



N portsmore ports




Need even more

host ports?

Add another leaf

To speed up FCT,

boost the backplane

by spreading load

across more spines

Lower FCT = FASTER

APPLICATIONS

BWFabric

192 ports

4x48 10G (1920 Gbps total)P

er S

pin

e

Utiliz

atio

nF

CT

FC

T

FC

T

10G host ports

40G fabric portsFCT = Flow

Completion Time

BWEdge



N ports

more bandwidth

more ports


reduce failure domain


Spine/Leaf DC Fabric ≅ Large Non-Blocking Fabric

Host

1

Host

3

Host

4Host

5

Host

7

Host

2Host

6

BRKDCT-2218 27


Host

1

Host

3

Host

2

Host

4H

ost

5

Host

7

Host

6

Spine/Leaf DC Fabric ≅ Large Non-Blocking Fabric

Host

1

Host

3

Host

4Host

5

Host

7

Host

2Host

6

BRKDCT-2218 28


Spine/Leaf DC Fabric ≅ Large Modular Switch

Host

1

Host

3

Host

2

Host

4H

ost

5

Host

7

Host

6

Lin

e

Ca

rd

Lin

e

Ca

rd

Lin

e

Ca

rd

Lin

e

Ca

rd

Lin

e

Card

Lin

e

Ca

rd

Lin

e

Ca

rd

Lin

e

Ca

rd

Lin

e

Ca

rdL

ine

Ca

rd

Fabric

Module

Fabric

Module

Fabric

Module

BRKDCT-2218 29

CLOS

Fabric


Impact of Link Speed – the Drive Past 10G Links

20×10Gbps

Downlinks

20×10Gbps

Uplinks

20×10Gbps

Downlinks

2×100Gbps

Uplinks

200G

Aggre

gate

Ba

nd

wid

th

200G

Ag

gre

ga

te

Ba

nd

wid

th

20×10Gbps

Downlinks

5×40Gbps

Uplinks

• 40 & 100Gbps fabric provide very similar performance for fabric links

• 40G provides performance, link redundancy, and low cost with BiDi


1 2 3 4 5

Statistical Probabilities of Efficient Forwarding

1 2

1 2 20

Probability of 100% throughput ≅ 3%



20×10Gbps

Uplinks2×100Gbps

Uplinks

11×10Gbps flows

(55% load)

5×40Gbps

Uplinks

BRKDCT-2218 31


Lower

FCT is

Better

Impact of Link Speed on Flow Completion Times

0

2

4

6

8

10

12

14

16

18

20

30 40 50 60 70 80

FC

T

(no

rmalize

d t

o o

pti

mal)

Load (%)

Avg FCT: Large (10MB,∞) background flows

Series1

Series2

Series3

Series4

BRKDCT-2218 32

Lower

FCT is

Better


0

2

4

6

8

10

12

14

16

18

20

30 40 50 60 70 80

FC

T

(no

rmalize

d t

o o

pti

mal)

Load (%)


Series1

Series2

Series3

Series4

• 40/100Gbps fabric links: ~ same FCT as non-blocking OQ-switch

• 10Gbps fabric links: FCT 40% worse than 40/100Gbps

Lower

FCT is

Better


0

2

4

6

8

10

12

14

16

18

20

30 40 50 60 70 80

FC

T

(no

rmalize

d t

o o

pti

mal)

Load (%)


Series1

Series2

Series3

Series4

• 40/100Gbps fabric links: ~ same FCT as non-blocking OQ-switch

• 10Gbps fabric links: FCT 40% worse than 40/100Gbps

Flow Completion Time dependent on

queuing and latency

40G is not just about faster ports &

higher bandwidth, it’s about

shorter Flow Completion Time


MM Fiber PlantMMF LC

Patch cord

MMF LC

Patch cord

SFP-10G-SR SFP-10G-SR

Used Fiber Pair

QSFP-40G-SR-BD

MM Fiber PlantMMF LC

Patch cord

MMF LC

Patch cordUsed Fiber Pair

Distance up to 125m with OM4

QSFP-40G-SR-BD

MM Fiber Plant

MP

O

MP

O

Used Fiber PairUsed Fiber PairUsed Fiber PairUsed Fiber Pair

QSFP-40G-SR4 QSFP-40G-SR4

35BRKDCT-2218

40G BiDi Optics Preserve Existing MM 10G Cabling

• Reuse of existing

cable plant

• Price of 40G BiDi

QSFP ~ price for 10G

SR SFP

• WDM technology

over MM fibers

• Excellent DC reach

• Can we do the same

at 100G?

Highlights:


Networking Underlays and Overlays

Flexible Overlay Virtual Network

• Mobility – Track end-point attach at edges

• Scale – Reduce core state, distribute to edge

• Flexibility/Programmability – Less touch points

• Physical + Virtual; L2 & L3 Connectivity

Robust Underlay/Fabric

• High Capacity Resilient Fabric

• Intelligent Packet Handling

• Full Cross Sectional Bandwidth

• Programmable & Manageable

IP Fabric

Do you really want to deal with two networks? Consider Cisco ACI insteadBRKDCT-2218


Why VXLAN Overlays?

VXLAN provides a Network with Segmentation, IP Mobility, and Scale

• “Standards” based Overlay (RFC 7348)

• Leverages Layer-3 ECMP underlay – all links forwarding

• Increased Name-Space to 16M identifier

• Integration of Physical and Virtual

• It’s the pillar of SDN

• Works better with a control plane

Classical Ethernet Frame

CRC (new)

VxLAN (8)

UDP (8)

IP (20)

Original CE Frame 50 bytes

Outer MAC (14)

VNI

DMAC SMAC 802.1Q Etype CRC Payload

DMAC SMAC 802.1Q optional

Etype Payload

ags

8 bits 24 bits 8 bits 24 bits

Reserved Reserved VNI

BRKDCT-2218 37


Network Centric Deployment

• Leverage known networking constructs.

• VLANs, IP address / subnets, Flood domains etc.

VLAN 100:

10.1.1.0/24

Existing network – Nexus, Catalyst, etc

VLAN 200:

20.1.1.0/24 VXLAN EVPN Fabric

MAC_A,IP_A: 10.1.1.2

VNI L2: 100100

VLAN 100 VLAN 200

MAC_B,IP_B:20.1.1.2

VNI L2: 100200

VRF VNI: 300100 VRF VNI: 300100

VLAN 100:

10.1.1.0/24

ACI Fabric

BD: Blue

10.1.1.0/24

EPG: Blue-100

Existing network – Nexus, Catalyst, etc

VLAN 200:

20.1.1.0/24

BD: Red

20.1.1.0/24

EPG: Red-200

TENANT

PRIVATE NETWORK (VRF)

BRKDCT-2218 38


Application Centric Deployment (ACI only)

• Leverage grouped constructs – Network Level; Application Level

• Abstraction with Application Network Profiles, Policy oriented

• Automation of network services

• APIC Controller: declaration of intention, translate to network constructs via policy

Web App DB

FW / SLB

External

Connectivity

EPG

Web

EPG

App

EPG

DB

FW / SLB

L3 Outside

App Profile

Cisco Live Session: PSOACI-4592

Why ACI: an overview and a customer (BBVA) perspective

BRKDCT-2218

APIC

39


Controlling, Managing & Automating the Fabric

BRKDCT-2218

Leaf

Spine

WAN / DCI

ACI Fabric

Application Policy

Infrastructure

Controller Client Access

APIC APICAPIC

Leaf

Spine

WAN / DCI

VXLAN EVPN

Client Access

Programmable Fabric

Variety of Nexus modelsApplication Centric Infrastructure (ACI)

Nexus9000 only

Creation Expansion

Fault MgmtReporting

Connection

Datacenter

Network Manager

Network Centric only Network Centric

Policy Centric

40


Remote PoD Multi-Pod / Multi-Site Hybrid Cloud Extension

ACI AnywhereAny Workload, Any Location, Any Cloud

ACI ANYWHERE

IP WAN

IP WAN

Remote Location Public CloudOn Premise

Security Everywhere Policy EverywhereAnalytics Everywhere

BRKDCT-2218 41

• Introduction






Agenda


Scaling a VPC-based DC design

L3

L2

Access

Layer

VLANs

100-150 Host Host Host



Access

Layer

VLANs


Access

Layer

VLANs

151-200Host Host Host

L3

L2

Consolidated

Core/Agg

Layer



Distributed or

Consolidated

Core/Agg

Layer

Access

Layer

VLANs


Access

Layer

VLANs

151-200Host Host Host

L3

L2

BRKDCT-2218 45


Integrating ACI with an existing network

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/migration_guides/migrating_existing_networks_to_aci.html

BRKDCT-2218

Cisco Live Session: BRKACI-2508

Nexus standalone fabrics migration to ACI

46


ACI Fabric

(VXLAN based)

47BRKDCT-2218


Spine

Layer

ACI PodNew DC

Row Upgrade

New Application

Access

Layer

VLANs

100-150

Access

Layer

VLANs

151-200Host

Distributed or

Consolidated

Core/Agg

Layer

L3

L2

L3

L2

Host

Access

Layer

VLAN

201-250

ACI Border Leafs

L3

L2Dual sided

vPC

Host


ACI Fabric

(VXLAN based)

48BRKDCT-2218


Spine

Layer

ACI PodNew DC

Row Upgrade

New Application

Access

Layer

VLANs

100-150Host

Distributed or

Consolidated

Core/Agg

Layer

L3

L2

L3

L2

Host

L3

L2

Host

ACI

Leafs

and Border Leafs


Remote Leaf Concept

IP WAN

IPN

Remote site contains Nexus 9300 connected

to IP Network via uplink ports and fully

managed by primary site APIC cluster

L2 / L3

APIC and Spine Nodes (Proxy)

remain at primary site(s)

Hypervisor

vSwitch

Remote Leaves

logically connect

to Spines of one

POD in Main DC

Supported from ACI

3.1(1) – Dec 2017

Any router with OSPF,

DHCP relay and 9150

MTU support.

Tested with 500Mbps on WAN,

but the more the better

All local traffic is switched directly between

endpoints, both virtual and bare metal

Specific traffic requires use of spine proxy.

Supported hardware and scale

- All hardware from -EX onwards is supported

- 40 RL (20 sites, 2 RL per site)

1/10G ports

BRKDCT-2218 49


ACI: Small Footprint Option Under Consideration

Leaf 48 ports

Leaf 48 ports

Spine 32 ports

Spine 32 ports

Physical Server

VM

VM

• Full featured ACI Fabric

• Reduced initial port count

• Optimized scalability for

mid size deployments

• Lower entry cost (-30%)

• Reduced footprint (-40%)What is your

opinion?

BRKDCT-2218 50

• Introduction






Agenda


IT Trends - Distributed Data CentersBuilding the Data Center Cloud

Distributed Data Center Goals:

• Seamless workload mobility between multiple datacenters

• Distributed applications closer to end users

• Pool and maximize global compute resources

• Ensure business continuity and disaster avoidance with workload mobility, distributed deployments and clustered applications

• Midsized enterprises often rely on two datacenters

Geographically

Dispersed Data Centers

BRKDCT-2218

Cisco Live Session: BRKDCN-2657

Design and Implementation of DCI Networks

52


OTV

Scalable Pod

vPC, N-tier design

Classical Pod

Spanning Tree ProtocolLeaf Spine Pod

VXLAN, ACI

NX-OS Overlay Transport Virtualization TechnologyExtend VLANs Across Datacenters

Benefits:

• Field proven, very mature

• Enable L2 elasticity across DC’s

• Simplify Virtual Machine Mobility

• Extend Layer 2 without the risks of large fault domain

• Simple – 3 easy commands !

Feature:

• Simplified Layer 2 / VLAN Extensions

• Restricted fault domain (STP domain), loop prevention

• Optimized Multicast replication

• Dual homing

• Works over dark fibre, MPLS or IP

In the news:

• M3 linecards with NX OS 8.0

• OTV Loopback Join Interface for Multicast-based OTV control-plane

• Wire-rate 256-bit AES MACsec on all ports at all speeds along with OTV

BRKDCT-2218 53


OTV options for DCI

L3-----------

L2

Campus

Client Access

WAN / DCI

VM VMVMVM VMVM

Virtualized Servers, Nexus

1000v, vPath, CSR 1000v

Virtual DC

Services in

Software

L3-----------

L2

WAN / DCICampus

Client Access

VM VMVMVM VMVM

Virtualized Servers, Nexus

1000v, vPath, CSR 1000v

Virtual DC

Services in

Software

CSR1000v


OTV options for DCI

L3-----------

L2

Campus

Client Access

WAN / DCI

VM VMVMVM

Physical or

Virtual

Workloads

and Services

L3-----------

L2

WAN / DCICampus

Client Access

VMVM VMVM

Physical or

Virtual

Workloads

and Services

CSR1000v

ASR1000

ASR1000

N7K


VXLAN as a Data Center Interconnect?

• VXLAN can absolutely fit into a DCI architecture…if you handle it CAREFULLY

• Protection at edge as well as control plane required

• Storm Control, BPDU Guard, HMM Route Tracking

• Control-Plane with MAC-learning, ARP suppression

• VXLAN is just an encapsulation technique

• DCI is an architectural discussion

Building

• OTV is a well-proven technology that gets you fully covered

DCI with ACI Multi-Fabric OptionsSingle APIC Cluster/Single Domain (BC-DA) Multiple APIC Clusters/Multiple Domains (DR)

Site 1 Site 2

ACI Fabric

Stretched Fabric

Inter Pod

Network@10G

Site ‘A’ Site ‘n’

MP-BGP - EVPN

Multi-Site (new)

Pod ‘A’ Pod ‘n’

APIC Cluster

MP-BGP - EVPN

Multi-Pod

Inter Pod

Network@10G

…

ACI Fabric NACI Fabric 1

Multi-Fabric Connected (L2 and L3 Extension)

L2/L3


ACI MultiPOD Design and Deployment

40G

More scale

More flexibility

More automation


ACI Multi-Site Architecture and Deployment

• Introduction






Agenda


What Features Matter The Most to You? (as of Jan 2018)

BRKDCT-2218 59

Feature 3000 3500 5500 5600/

6000

7000/

7700

9300 9500

Unified Ports

FCoE

FEX

VXLAN Bridging and Routing

VXLAN F&L

VXLAN BGP EVPN

DCI - OTV

DCI – VXLAN

ISSU

VDC

LISP, MPLS

ITD

RISE

Extreme low latency

IPv6 support

Automation ACI

Automation DCNM


Encapsulate Fibre Channel frames into Lossless Ethernet • Standards based

• Operationally same as existing LAN /SAN

• Transparent to OS and Apps

• Fewer Cables

• Fewer switches

• Fewer adapters

• Overall less power

Eth

ern

et

He

ad

er

FC

oE

He

ad

er

FC

He

ad

er

FC Payload CR

C

EO

F

FC

SByte 0 Byte 2229

FCoE and Unified FabricFibre Channel with simpler infrastructure and lower cost

Individual

Ethernets

Ethernet Carrier, DCB

Individual

Storage

(IP, Eth, FC)

FCoE Benefits

BRKDCT-2218 60


Nexus Fabric Switches Evolution (FC/FCoE View)

BRKDCT-2218 61


Repurposing of any switch port as either Fibre Channel or Ethernet/FCoE avoidsCAPEX lock-in with specific port-types

Reduces physical management points

Extend adoption of FCoE in the access with associated savings

Nexus 9300 Unified PortsCompelling Business Value and Flexibility

BUSINESS

AGILITY

OPEX

CAPEX

Simplifies the purchasing decision, makes port-type decisions a thing of the past

Allows IT to respond faster to changing business needs

Native Fibre

Channel16/32G

FC Eth

Lossless

Ethernet:1/10/25GbE, FCoE,

iSCSI, NFS/SMB

Unified

Port

BRKDCT-2218 62


Introducing 16/32G FC on Nexus 93180YC-FX

First Nexus 9300 switch with Fibre Channel CapabilitiesSupports NX-OS and ACI Mode.

Unified Ports: 48p 1/10/25G or 16/32G FC SFP 6p 40/100G QSFP

Seamless transition

from N5K

Multi-protocol

FC

FCoE

IP Storage

UNIFIED PORT Interop with Cisco

MDS and Nexus 5K

Cloud Scale

technologies

MACSec

Shipping Milestone Timeline

FCoE NPV mode

(NX-OS and ACI)Oct, 2017NX OS 7.0(3)I7(1)

16G Fibre Channel

NPV mode (NX-OS)

Nov, 2017

NX OS 7.0(3)I7(2)

32G Fibre Channel

NPV mode (NX-OS)

Soon

16/32G Fibre Channel

NPV mode (ACI)

Work in progress

EX only supports FCoE NPVFX supports both FCoE NPV (Oct 2017) and FC NPV (Nov 2017)

BRKDCT-2218 63


Example: Small Deployment For All Flash Array

10G FCoE access ports

16G FC uplink ports

(32G future)

32G FC storage ports

NPV mode

NPIV

feature

LAN

Servers

All Flash Array

Peer links (Ethernet)Nexus 93180YC-FX Nexus 93180YC-FX

MDS 9132T MDS 9132T

NX OS mode: shipping

ACI mode: work in progress

BRKDCT-2218 64


What is ITD ?

• Traffic distribution and redirection

• ASIC based solution (HW-switched)

• Caters to multi-terabit traffic

• Works on Nexus switches – 5/6/7/9k,

similar but not identical capabilities

Intelligent Traffic Director

ITD does L3-L4 traffic distribution,

does not replace Layer-7 Load-balancers


Where to use ITD ?

Clients Servers

ITD to load-balance to the destination

Example: Server-Load Balancing#1

Health checking supported


Where to use ITD ?

Clients

Firewalls/other appliances

ITD for In-line traffic redirection

Example: Firewalls, IDS, Web Caches etc.

Possible WCCP replacement? Possible MS NLB replacement?

#2

Destination


ITD Benefits

Line-rate

traffic-distribution

Ease of deployment,

reduced configuration

Automatic

failure handling

No service-module

or external

appliance required

Intelligent Traffic Director

• Introduction






Agenda


Why Network Programmability?

More Flexibility More InnovationMore Speed More Repeatability

Fewer Mistakes

BRKDCT-2218


Programmability, Automation, Orchestration

BRKDCT-2218 71


Programmable Network Programmable Fabric

VxLAN-BGP EVPN

standard-based

3rd party controller support

Modern NX-OS with enhanced

NX-APIs

DevOps toolset used for Network

Management(Puppet, Chef, Ansible etc.)

Cisco Data Center Network Automation TypesProviding Choice in Automation and Programmability

Cisco Controller for software

overlay provisioning and

management across N2K-N9K

DCNM

Application Centric Infrastructure

DB DB

Web Web App Web App

Turnkey integrated solution with

security, centralized management,

compliance and scale

Automated application centric-policy

model with embedded security

Broad and deep ecosystem

Tools, APIs, Controllers and Automation


Nexus Programmability

Protocols and

Data Models

NetConf (SSH/XML) Shipping Shipping Shipping

NX API CLI (JSON/XML) Shipping Shipping Shipping

NX API REST Future No Shipping

NetConf / YANG Future No Shipping

RESTconf/gRPC Future No Shipping

Streaming Telemetry Future No Shipping

Provisioning &

Orchestration

Puppet/Chef/Ansible Shipping Shipping Shipping

PoAP Shipping Shipping Shipping

OpenStack Shipping Shipping Shipping

Programmatic

Interfaces

Native Python Shipping Shipping Shipping

Linux container Shipping Shipping Shipping

Docker containers Future No Future

Guest Shell Future No Shipping

OpenFlow Shipping No Shipping

Nexus 7K Nexus 5K / 6K Nexus 9K

BRKDCT-2218 73

Cisco Live Session: BRKDEV-1368

Effectively Understand and Leverage YANG with NETCONF and

RESTCONF for Model Driven Programmability


Programming a Fabric

• A lot of work is being done to provide customers maximum flexibility in programming & automation interfaces

• Free Open Programmability book:• http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/nexus9000/sw/open_nxos/program

mability/guide/Programmability_Open_NX-OS.pdf

• New community site dedicated to NXOS programmability:• https://opennxos.cisco.com

• A lot of work has been done to increase available knowledge on network programming across all Cisco products

• DevNet: If you haven’t visited, please do so

• https://devnet.cisco.com• SANDBOX! – FREE 24 X 7 hosted labs

74BRKDCT-2218

http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/nexus9000/sw/open_nxos/programmability/guide/Programmability_Open_NX-OS.pdf

https://opennxos.cisco.com/

https://devnet.cisco.com/


Programmable Networking

Programmable Fabric

VxLAN-BGP EVPN

standard-based



NX-APIs








DB DB

Web Web App Web App








DCNM


CLI

Basic ElementManager

Scripting to the CLI and/or API

ACI

Along the Spectrum from CLI to ACIA New Way To Do Fabric Management

BRKDCT-2218 76


Fabric Management FunctionalitiesCommon Customer Asks

• Fabric management automation – high interest

• Many Fabrics are based on things like VXLAN, BGP/EVPN, IS-IS

• New protocols, new configurations, new things to learn

• Simple tool to ease burden of adoption

• Simplified interaction – GUI and API

• CVD/Best practices – done for you!

77BRKDCT-2218


Data Center Network Manager (DCNM) 10

78

• Network management system tool for

• SAN management (Nexus and MDS)

• LAN (full Nexus family)

• Programmable Fabrics (NX OS mode)

• Flow controller for IP Media Networks

• Advanced feature support (STP, VXLAN, POAP,

OTV, templates, topology, configuration backup)

• Underlay/Overlay manager for VXLAN-EVPN (IP)

Fabrics in NX-OS Mode

BRKDCT-2218


Functions Dashboard

Health Monitoring,

Inventory &

Diagnostics

Visualization &

Troubleshooting

Configuration/Backup Automation

Alert/Notifications Storage Management

Trend Analysis & VM

Analytics

Professional Media

Net Controller

Host / Endpoint

Monitoring

BRKDCT-2218 79


Programmable Networking

Programmable Fabric

VxLAN-BGP EVPN

standard-based



NX-APIs








DB DB

Web Web App Web App








DCNM


Programmable Network Programmable FabricApplication Centric

Infrastructure

Integrated stack

Or

A-la-carte Automation

Streamlined Workflow Management

Modern NX-OS with enhanced NX-APIs

DevOps toolset used for Network Management(Puppet, Chef, Ansible etc.)

Custom Script based Operations and Workflows

Turnkey integrated solution with security, centralised

management, compliance and scale

Automated application centric-policy model with

embedded security


FCAPS and Automation

Fault

Accounting

Performance

Security

Configuration

External

Tools Integrated

Tools

External

Tools

Integrated

Tools

BRKDCT-2218

DCNM

81


Config

Config

Config

Config

Ops

Ops

Ops

Ops

82BRKDCT-2218

Effort vs Time – Traditional IT Build/Run


Config

Config

Config

Config

Ops

Ops

Ops

Ops

Effort vs Time – Traditional IT Build/Run

BRKDCT-2218 83


Config

Config

Config

Config

Ops

Ops

Ops

Ops

Effort vs Time – Utilizing Abstraction & Automation

BRKDCT-2218 84

Plan Implement

Ops


Cisco UCS Director for Compute; Network & StorageInfrastructure consumption made easier

85BRKDCT-2218

Converged Stack

Control Panel

Policy-Driven

Provisioning

Secure Cloud

Container

VMsComputeNetwork Storage

UCS Director

OS and

Virtual

Machines

Storage

Network

Compute

Tenant

BTenant

CTenant

A

Virtualized and Bare-Metal

Compute and Hypervisor

B CA

Network and Services

VM VMBareMetal

Single Pane of Glass for

Virtual AND Physical

Speed with Accuracy

More efficient use of People

& Time

Consistency, Less Error in

Repetitive Tasks


Provision new SP from template, provision zoning/masking/boot LUN, boot from SAN, install hypervisor (ESX image on PXE server), associate VMWare Host Profile for vSwitchsettings, more customization

UCS Director – Example Use Cases for SMB

BRKDCT-2218 86

------------

------------

------------

------------

------

Create VM Disk, install and format in OS, then create windows share or linux mount point

Start

---------

-

---------

-

---------

---------

---------

---

Start


Cisco has many options for building data center networks, you own the key

All solutions can start small and grow

No Cisco solution has to be a “rip and replace”

Spine-Leaf does not have to be expensive

Programmable fabrics provide new tools for simplified operations

Automated fabrics provide new methods of managing DC Networking

Key Takeaways

BRKDCT-2218 87


Check out the sessions listed at beginning of presentation at www.ciscolive.com

Visit the World of Solutions and look for “Data Center”

Meet the Engineer

Visit the DevNet Zone and see what they have to offer!

Call to Action

BRKDCT-2218 88


Cisco Spark

Questions? Use Cisco Spark to communicate with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

cs.co/ciscolivebot#BRKDCT-2218


• Please complete your Online Session Evaluations after each session

• Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt

• All surveys can be completed via the Cisco Live Mobile App or the Communication Stations

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/.

Complete Your Online Session Evaluation

http://ciscolive.com/Online

http://www.ciscolive.com/global/on-demand-library/


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Tech Circle

• Meet the Engineer 1:1 meetings

• Related sessions

91BRKDCT-2218

Thank you

Documents

Data Center Network Design - clnv.s3.amazonaws.com · BRKARC-3222 Cisco Nexus 9000 Architecture Tim Stevenson Check on CiscoLive App ... • Cisco alternatives for campus design: