Dubrovnik, Croatia, South East Europe 20-22 May, 2013 Service Oriented Virtual DC Design · © 2012 Cisco and/or its affiliates. All rights reserved. 6Cisco Connect Virtualized Multi

© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Connect 1 1 © 2012 Cisco and/or its affiliates. All rights reserved.

Service Oriented Virtual DC Design

Višnja Milovanović

Consulting Systems Engineer

Data Center & Virtualization

Dubrovnik, Croatia, South East Europe

20-22 May, 2013

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 2

Setting the Stage: What’s the Meaning of Virtual?

• If you can see it and it is there

It’s real

• If you can’t see it but it is there

It’s transparent

• If you can see it and it is not there

It’s virtual

• If you can not see it and it is not there

It’s gone


3

Setting the Stage: Delivering a Service (With End-to-End Automation)

Design It

Where Can We Put It?

Procure It

Install It

Configure It

Secure It

Is It Ready?

Architect It

Design It

Where Can We Put It?

Procure It

Install It

Configure It

Secure It

Is It Ready?

Architect It

Before After

• Machine-oriented

• Manual provisioning

• Hard to control utilization

• Service-oriented

• Self-service; automated provisioning

• Elasticity (capacity-on-demand)

• High provisioning & ops cost

• Extended provisioning time

• Configuration risk

• Optimized provisioning & ops cost

• Rapid provisioning

• Increased Resiliency and Availability

Manual

Automated

Self-Service

On-Demand

Cloud


Database

Use Case: Acquisition of New Sales Force, Consolidation and Rapid Provisioning via Templates

Storage Template

Compute Template

Fabric Template

Application Template

Create 500 Virtual Desktops to Enable New Team to Access Corporate Information and Applications

Increase Database Capacity to Support Sales Consolidation Effort

Create 500 New Mailboxes

Virtual

Desktops

Email

Storage Template

Compute Template

Fabric Template


Storage Template

Compute Template

Fabric Template



Agenda • Data Center Virtualization Overview

• Front-End Data Center Virtualization

Core Layer

Aggregation Layer

Networking Services

Access Layer

• Server Virtualization

Unified Computing System

Virtual Access Layer

Virtualized Services

Server IO Virtualization

• Back-End Virtualization

Unified Fabric & FCoE

SAN & Storage

• Q&A


Virtualized Multi Service Data Center Infrastructure

Nexus 7000

10 GE Aggr

Network

Services

Layer 3

Layer 2 - 1GE

Layer 2 - 10GE

10 GE DCB

10 GE FCoE/DCB

4/8 Gb FC

FC

SAN A FC

SAN B

vPC+

FabricPath

Nexus 7000

10 GE Core

Catalyst 6500

End-of-Row

Nexus 5500 10GE

Nexus 2248

End-of-Row

CBS 31xx

Blade switch Nexus 7000

End-of-Row

Nexus 5500 FCoE

Nexus 2232

Top-of-Rack

UCS FCoE Nexus 3000

Top-of-Rack

Nexus 4000

FIP-Snoop.

IBM Blade

Center

1 GbE Server Access & 4/8Gb FC via dual HBA (SAN A // SAN B) 10Gb DCB / FCoE Server Access or 10 GbE Server Access & 4/8Gb FC via dual HBA (SAN A // SAN B)

L3

L2

MDS 9500

SAN

Director

B22

FEX

HP Blade

C-class

FC

SAN A

FC

SAN B

MDS 9200 /

9100

Nexus

5500

FCoE


Virtualized Network Infrastructure: Evolution and Considerations

Typical DC Challenges

L2 Fate-sharing

VLAN Location

L2 Adjacency

Higher Scale

L3 Access

App Environments

What are the implications…

Dynamic “routing protocol” for L2 (e.g.: IS-IS)

Any VLAN anywhere resonates well

Lower oversubscription

Larger subnet sizes

Global VLANs

Specific app environments Designs

Access Ports in

management domain

Density and

Capabilities of

access switch

Density & quantity of

aggregation

switches

Classic Pod Modern Pod


Leaf

Layer

Sp

ine L

ayer

Converged FCoE link

Dedicated FCoE link

FC

1 / 10GE

DR Data Center

OTV: Layer 2 Extension

Virtualized Data Center Infrastructure (2 Layers)


Virtualized Data Center Infrastructure

• FabricPath enabled for LAN traffic

• Dual Switch core for SAN A & SAN B

• All Access and Aggregation switches are FCoE FCF switches

• Dedicated FCoE links between switches are VE_Ports

• Storage VDC (Nexus 7000 only) for additional operation separation at high function agg/core (aka spine)

SAN can utilize higher performance, higher density, lower cost Ethernet switches (F2 module on Nexus 7000 and unified ports on Nexus 5500)

(*) FC connectivity to storage only available on Nexus 5000/5500. FCoE target and NAS / iSCSI target connectivity to any Nexus switch.

L2

L3

CNA FC (*) FCoE

Fabric ‘A’

Fabric ‘B’

FCF

FCF

FCF

FCF

VE

Converged FCoE link

Dedicated FCoE link

FC

1 / 10GE

FabricPath

NAS

iSCSI

Leaf

Layer

Sp

ine L

ayer

Maintaining Dual SAN Fabrics with Layer2 Multipath (Fabric Path)


Converged FCoE link

Dedicated FCoE link

FC

1 / 10GE

FabricPath

L3

Campus Access WAN

L2

VM

#4

VM

#3

VM

#2

Nexus 5500 with L3 module and

vPC+

Nexus 2200

FC FCoE NAS

iSCSI

ASA 5500

ACE 4710

DC @ SMB Customer

Virtualized Data Center Infrastructure Virtualized Data Center Switch — 1 Layer?




Core Layer

Aggregation Layer

Networking Services

Access Layer








SAN & Storage

• Q&A


Front-End: LAN Core Layer

LAN Core

Layer


FIB TCAM

Size 128K - 1M

ACL TCAM

Size: 64K - 128K

FIB TCAM

Size: 128K – 1M

FIB TCAM

Size 128K – 1M (from M1-F1 combination)

FIB TCAM

Size 32K

VDC-1

IP routes: 20K

ACL entries: 10K

VDC-2 IP routes: 100K

ACL entries: 50K

ACL TCAM

Size: 64K - 128K

VDC-3 Storage VDC

E.g.:FCoE on F1 linecard(s)

ACL TCAM

Size: 16K - 192K

ACL TCAM

Size: 1K - 16K

Linecard 1 (M1) Linecard 2 (M1)

Linecard 3 (F1) Linecard 4 (F2)

Virtual Device Contexts (VDC) Separate Resource Allocation Domains (Layer 3)

VDC-4

F2 linecard(s)

M1/F1

VDC

F2

VDC

M1

F2

F2

M1 F1

M1

VDC

F1

F1 Storage

VDC


Aggregation

Layer

Front-End: Aggregation Layer


vPC is a Port-channeling concept extending link aggregation to two separate physical switches

Allows the creation of resilient L2 topologies based on Link Aggregation.

Eliminates the need for STP in the access-distribution Layer

Enable seamless VM Mobility, Server HA Clusters

Scale Available Layer 2 Bandwidth

Dual-homed server operate in active-active mode

Simplify Network Design

Available on Nexus 7000 and Nexus 5000 / 5500

Bi-sectional BW with vPC

L2

SiSi

Non-vPC vPC

SiSi

Virtual Port Channel

Physical Topology Logical Topology

Virtual Port Channel (VPC) Active — Active Layer 2 Links


S10 S20 S30 S40

S100 S101 S201

FabricPath

Server

L1 L2 L4 L3

L5 L6 L7 L8

L9 L10 L11 L12

Unified Computing System (UCS)

Virtual Access Switch POD

(Nexus 7000 / 5x00 + Nexus 2200)

Virtual Blade Switching (VBS)

vPC+ allows dual-homed connections from edge ports into FabricPath

domain with active/active forwarding. E.g.: Classical Ethernet switch, Layer 3

router, dual-homed server, etc.

vPC+ vPC+

S200

Logical View with FabricPath: Distributed Topology without L2 loops


Front-End: Networking Services

Networking

Services


v5

v105

v6 v7

v107

v2081

v2082

v2083

...

v206 v207

v206

BU-4 BU-2 BU-3

v105

v108

BU-1

1

2

3

4

* vX = VLAN X

**BU = Business Unit

VRF

VRF

VRF VRF VRF

v208

“Front-End” VRFs (MSFC)

Firewall Module Contexts

ACE Module Contexts

“Back-End” VRFs (MSFC)

Server Side VLANs

v207

3

4

v8

Data Center Virtualized Services Physical Appliances/Modules Context Combination Example

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 19 © 2013 Cisco and/or its affiliates. All rights reserved.

VMDC Sample Containers Flexible framework allows variations as need be

Silver Gold Palladium Expanded Bronze

L2

L3

FW

LB

LB

Public Zone

Private Zone

L2

L3

FW

LB

LB

vFW

vFW

FW

Protected

Back-End

Protected

Front-End

L2

L3

L3

vFW

LB

L2

L3

L3

vFW

FW

L2

L3

L3

vFW

LB

• Predefined containers provide examples for different types of deployments

• Automated provisioning and management logic for each container type is pre-

defined in the Management and Orchestration software

• Customers can choose from existing models or define their own customized

models

© 2013 Cisco and/or its affiliates. All rights reserved.

© 2012 Cisco and/or its affiliates. All rights reserved. Cisco Connect 20 © 2013 Cisco and/or its affiliates. All rights reserved.

Business Continuance and Disaster Recovery?

Benefits

• Workload balancing across data centers and clouds

• Proactive response to disruptions – mitigates risks of Approaching disasters, viz.

hurricanes, floods, etc., Power grid maintenance, Data center maintenance and

migrations

• Planned events scheduled over a period of time

• Backup and Disaster Recovery aaS

Cisco Catalyst

6500 Series

Cisco Nexus

7000 Series Cisco Nexus

7000 Series

• Cisco Data Center

Interconnect (DCI)

Solution based on

OTV, VPLS,

EoMPLS, LISP

• NetApp’s FlexCache-

enabled Intelligent

Storage Arrays

• EMC’sVPLEX +

Avamar

• VMware vSphere &

VMotion Technologies

The Solution: • VMotion enabled

over Cisco DCI

solution

© 2013 Cisco and/or its affiliates. All rights reserved.


Front-End: Access Layer

Access

Layer

(LAN & SAN)


Distributed High Density

Edge Switching System

(up to 4096 virtual Ethernet

interfaces)

+

Cisco Nexus® 2000 FEX

Cisco Nexus® 5500

Cisco Nexus® 2000 FEX

Cisco Nexus® 7000

+

Cisco FEXlink: Virtualized Access Switch Nexus 2200 Fabric Extender (FEX)


To2R: Nexus 2200 Deployment Example

Rack 1 Rack 2

Access Layer

Rack 1 Rack 2Rack 1 Rack 2

Aggregation LayerNexus 7000 Nexus 7000

Nexus 5500 Nexus 5500

Nexus 2200 Nexus 2200 x4 x4x4x4

x4 x4x4x4

Rack 1 Rack 2Rack 1 Rack 2 Rack 12 Rack 1 Rack 2 Rack 12

vPC

vPC

Rack 13 Rack 14 Rack 24


IEEE 802.1BR: Bridge Port Extension Fully specifies a Port Extender (FEX Equivalent)

Extends ports of a switch to lower entities in a network

Port Extenders are not individually managed

Their ports become ports of the controlling switch

Cascading Port Extenders

Allows one to choose the appropriate controlling switch

Frame replication supported for efficient multicast / flooding

Traffic from each “Extended Port” is reliably segregated to an E-channel and identified by a tag containing an E-channel identifier (ECID)

Does not require prior knowledge of MAC addresses; switch performs standard learning functions

Works with all devices including VEBs, VEPAs, individual VMs, physical services, and devices providing transparent services

Controlling Bridge + PE = Extended Bridge

Single Point of Management

PE

Bridge

PE

PE

PE Port Extender

PE

vFW

Server

VM1

PE

Controlling

Bridge

Extended Bridge

ECID


Representation of Port Extender & Fabric Extender

Switch

Eth Port Extension 802.1BR

Port

Extender

PE Tag

802.1BR

Server

Hypervisor

VM VM VM VM VM VM

Adapter

PE Tag

802.1BR

1 2 3 4 5

Port n

vNIC

3

vNIC

2

vNIC

1

vNIC

5

vNIC

4

Port 0

Nexus 2200 (FEX)

2 3

1

NIV Capable Adapter

IEEE Bridge Port Extender = Cisco FEX (Fabric Extender)

1

5

1 2 3 4 5 6 7 8

Nexus 5500

Port 1 …

Eth


• Dual 4x 10 GE port-channels to a single server slot

• Host connectivity PCIe Gen2 x16

• PCIe Gen 2 x16 bandwidth limit is 32 Gbps

• HW Capable of 256 PCIe devices

• OS restriction apply

• PCIe virtualization OS independent (same as M81KR)

• Single OS driver image for both M81KR and 1280 VIC

• FabricFailover supported

• Eth hash inputs : Source MAC Address, Destination MAC Address, Source Port, Destination Port, Source IP address, Destination IP address, and VLAN

• FC Hash inputs: Source MAC Address, Destination MAC Address, FC SID and FC DID

Dual 4x 10 GE (80 Gb per host) VM-FEX scale, up to 112 VM interfaces /w ESX 5.0

Customer benefits

Feature details

UCS 1280 Virtual Interface Card (VIC)

UCS 1280 VIC

UCS 2208 IOM

Side A Side B

256 PCIe devices

UCS 2208 IOM




Core Layer

Aggregation Layer

Networking Services

Access Layer








SAN & Storage

• Q&A


Front-End: Servers Layer

Servers

Layer


29

LAN Any IEEE Compliant LAN SAN B

Any ANSI T11 Compliant SAN

Mgmt SAN A

Any ANSI T11 Compliant SAN

One Logical Chassis to Manage*

LAN Connectivity

SAN Networking

Blade Chassis’

Server Blades

Rack Servers

Server Identity Management

Monitoring, Troubleshooting

etc.

*architectural limit of 320 servers with 160 servers supported as of UCS release 2.0

Cisco Unified Computing System (UCS) Form Factor Independence


Network Has Complete Visibility to Servers

•UCS Service Profiles Capture more than MAC & WWN

MAC, WWN, Boot Order, Firmware, network & storage policy

•Stateless compute where network & storage see all movement

Better diagnostics and QoS from network to blade, policy follows

SAN

LAN Chassis-1/Blade-5

Chassis-9/Blade-2

Server Name: SP-A

UUID: 56 4d cd 3f 59 5b 61…

MAC : 08:00:69:02:01:FC

WWN: 5080020000075740

Boot Order: SAN, LAN

Service Profiles deliver Service Agility

regardless of Physical or Virtual Machine


Total Server Deployment

14 Servers

Reduction of 4 Servers

22% CapEx Savings

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Web Servers

Oracle RAC VMware

Old Deployment:

Blade

Blade

HA Spare Burst Capacity

Cisco’s Deployment:

• Resources provisioned based on

business need

• Still HA with fewer spares

Cisco UCS Deployment: (still 18 Service Profiles)

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Blade

Web Servers

Oracle RAC VMware

Stateless Computing @ UCS


Cisco VN-Link What is that and which problems does it solve ?

• VN-Link (or Virtual Network Link) is a term that refers to a VM specific link that is created between the VM and Cisco switch.

Logical equivalent & combination of a NIC, a Cisco switch interface and the RJ-45 patch cable that hooks them together.

Hypervisor

VNIC VNIC

VETH VETH


Virtual Access Layer @ Virtualized Servers

Virtual

Access

Layer

Nexus 1000v


34

1. Nexus 1000V automatically enables port groups in vCenter via API

2. Server Admin uses vCenter to assign vnic policy from available port groups

3. Nexus 1000V automatically enables VM connectivity at VM power-on

Port Profiles “How to”

1.

VMW ESX

Server 1

Nexus 1000V - VEM

VM

#1

VM

#4

VM

#3

VM

#2

Available Port Groups

WEB Apps HR

DB Compliance

2.

Nexus 1000V

VSM vCenter

3.

“WEB Apps” Port Profile:

PVLAN 108, Isolated

Security Policy = Port 80 and 443

Rate Limit = 100 Mbps

QoS Priority = Medium

Remote Port Mirror = Yes


VM VM VM VM

Nexus

1000V

VEM

VM VM VM VM

Nexus

1000V

VEM

Nexus 1000V

VSM

Windows 8 Hyper-V Nexus 1000V VSM

VMware vSphere

VMware vCenter SCVMM

Consistent architecture, feature-set & network services ensures operational transparency across multiple hypervisors.

Cisco Nexus 1000v multi-hypervisor support


Nexus 1000V

• Distributed switch

• NX-OS consistency

VSG

• VM-level controls

• Zone-based FW

ASA 1000V

• Edge firewall, VPN

• Protocol Inspection

vWAAS

• WAN optimization

• Application traffic

WAN

Router

Servers

Tenant A ASA

1000V

Cloud

Firewall

Nexus 1000V Physical Infrastructure

Virtualized/Cloud Data Center

vWAAS

Cisco Virtual

Security

Gateway

6000+ Customers Shipping Shipping Shipping

CSR 1000V (Cloud Router)

• WAN L3 gateway

• Routing and VPN

Beta

Switches

Ecosystem Services

• Citrix NetScaler VPX virtual ADC

• Imperva Web App. Firewall

Cloud Network Services

Citrix

NetScaler

VPX

Imperva

SecureSphere

WAF Cloud

Services

Router

1000V

Zone A

Zone B

vPath VXLAN

Multi-Hypervisor (VMware, Microsoft*, RedHat*, Citrix*)

2013


Securing Tenant Edge with ASA 1000V

• Proven Cisco Security…Virtualized

Physical – virtual consistency

• Collaborative Security Model

VSG for intra-tenant secure zones

Virtual ASA for tenant edge controls

• Seamless Integration

With Nexus 1000V & vPath

• Scales with Cloud Demand

Multi-instance deployment for

horizontal scale-out deployment

Tenant B Tenant A VDC

vApp

vApp

vSphere

Nexus 1000V

vPath

VDC

Virtual Network Management Center (VNMC)

vCenter

VSG VSG VSG

VSG

Virtual ASA Virtual ASA


VM VM

Enterprise DC

Enterprise Branch

Nexus

HW Switches

Physical

Services

Virtual

Services

Virtual

Services

vPath

Cloud

Service

Router

ISR

ASR 1K/9K

vPath

Enterprise Campus

VM VM VM

L2-VPC

vPath Nexus

1000V

Remote

Access

Cloud

Manager

Secure

L2 Extension

(Cloud GW)

Nexus 1000V vPath vPath

Public Cloud

Kumo Benefits:

Security * Consistency * Control

Cisco Nexus 1000v InterCloud




Core Layer

Aggregation Layer

Networking Services

Access Layer








SAN & Storage

• Q&A



Unified

Fabric


iSCSI

Appliance

File System

Application

SCSI Device Driver iSCSI Driver

TCP/IP Stack

NIC

Volume Manager

NIC

TCP/IP Stack

iSCSI Layer

Bus Adapter

iSCSI

Gateway

FC

File System

Application

SCSI Device Driver iSCSI Driver

TCP/IP Stack

NIC

Volume Manager

NIC

TCP/IP Stack

iSCSI Layer

FC HBA

NAS

Appliance

NIC

TCP/IP Stack

I/O Redirector

File System

Application

NFS/CIFS

NIC

TCP/IP Stack

File System

Device Driver

Block I/O

NAS

Gateway

NIC

TCP/IP Stack

I/O Redirector

File System

Application

NFS/CIFS

FC

NIC

TCP/IP Stack

File System

FC HBA

FCoE SAN

FCoE

SCSI Device Driver

File System

Application

Computer System Computer System Computer System Computer System Computer System

Block I/O File I/O

Ethernet Ethernet

Block I/O

NIC

Volume Manager Volume Manager

FCoE Driver

Unified Fabric Flexibility and Serialized Re-Use

Ethernet Ethernet Ethernet

Ability to re-provision any compute unit to leverage any access method to the data stored on the ‘spindle’

Serialized Re-Use – Boot from SAN and Run from NAS

Virtualization requires that the Storage Fabric needs to exist everywhere the IP fabric does


Unified I/O Architecture Consolidation Initial Goal

Ethernet FC

LAN SAN B SAN A

No Consolidated IO I/O Consolidation with FCoE

SAN B LAN SAN A

FCoE

Nexus

5000


Example: Embedded FCoE at Cisco UCS

From ad hoc and inconsistent…

…to structured, but siloed, complicated and costly…

…to simple, optimized and automated


Thank you.

Documents

Dubrovnik, Croatia, South East Europe 20-22 May, 2013 Service Oriented Virtual DC Design · © 2012 Cisco and/or its affiliates. All rights reserved. 6Cisco Connect Virtualized Multi