© 2006 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice

HP Blade System Virtual Connect

Technology Workshop

Glyn Pollington

UK Technical Services

2 6 January, 2010

side by side for redundancy

Virtual Connect environment – 3 Key Components to a VC Infrastructure

• Connect selected server Ethernet ports to specific data center networks.

• Supports aggregation/tagging of uplinks to data center.

• ‘LAN-safe’ for connection to any data center switch environment (Cisco, Nortel, HP ProCurve, etc.)

Virtual Connect Ethernet Modules

Virtual Connect Fibre Channel Modules• Selectively aggregate multiple server FC HBA ports

(Qlogic/Emulex) on a FC uplink using NPIV.

• Connect enclosure to Brocade, Cisco, McData, or Qlogic data center FC switches.

• Doesn’t appear as a switch to the FC fabric.

+

Virtual Connect Manager (embedded!)• Manage server connections to the data center without

impacting the LAN or SAN

• Move/upgrade/change servers without impacting the LAN or SAN

+

3 6 January, 2010

16x 1Gb Ethernet–Connects to one NIC in each HH blade server bay

1x 10Gb–Cross-link between adjacent VC-Enet modules

Mgmt Interfaces to Onboard Administrator (Enet & I2C)

HP Virtual Connect Ethernet ModuleHP 1/10Gb VC-Enet Module

MidPlane

8x 1Gb Ethernet (RJ45)Usable as data center links or as stacking links.

‘Next’ button (unused in initial release firmware)

Port Number & Status IndicatorsIndicates whether a data center link (green),stacking link (amber), or highlighted port (blue).

2x 10Gb Ethernet (CX4)Usable as data center links or as stacking links.

Recessed module reset button

4 6 January, 2010

16x 1Gb Ethernet–Connects to one NIC in each HH blade server bay

1x 10Gb–Cross-link between adjacent VC-Enet modules

Mgmt Interfaces to Onboard Administrator (Enet & I2C)

HP Virtual Connect Ethernet ModuleHP 1/10Gb-F VC-Enet Module

MidPlane

‘Next’ button (unused in initial release firmware)

Port Number & Status IndicatorsIndicates whether a data center link (green),stacking link (amber), or highlighted port (blue).

1x 10Gb Ethernet (CX4)*2x 10Gb Ethernet (XFP module)*

Recessed module reset button

2x 1Gb Ethernet (SFP modules)*

4x 1Gb Ethernet (RJ45)*Usable as data center links

or as stacking links.

5 6 January, 2010

16x 4Gb FC – Connects to one HBA port in each HH blade server bay

Mgmt Interfaces to Onboard Administrator

HP Virtual Connect FC Module

4x 1/2/4 Gb FC (SFP)Connects to data center FibreChannel switches.

Port LEDsLink and Activity

Port Number & Status IndicatorsIndicates whether a port is configured (green)or highlighted (blue).

MidPlane

Faceplate

Module LEDsHealth and UID

Note: VC-FC requires VC-Ethernetalso installed in same VC domain

6 6 January, 2010

Virtual Connect Manager

7 6 January, 2010

Why Virtual Connect?

•Overcome Interconnectivity Issues

•Simplify Management

•Separate Network and Server Management

•Manage Change

8 6 January, 2010

Too many cablesProblem: Error prone & costly

32 cables

64 cables

128 cables

Datacenter

9 6 January, 2010

Too many cablesProblem: Error prone & costly

128 cables

Datacenter

512 cables!!!!

10 6 January, 2010

Too many switchesProblem: 26X the amount of work

Cisco director-class -- 425 downlinks

425 servers

1 switch to manage

Cisco 3020 -- 16 downlinks

425/16 > 26

26 switches to manage for the same number of downlinks!

11 6 January, 2010

Too many switchesProblem: SAN fabrics are limited

Brocade 4Gb SAN Switch -- 16 downlinks

Each vendor has upper limit to number of switch domains:

McData � 24

Cisco FC � 40

Brocade � 56

At only 16 downlinks per switch,

that makes for a small fabric!

12 6 January, 2010

Blade Servers

1 NIC

HBA

World Wide Name

MAC address

SANFibre Channel Switch

2

NIC

HBA

MAC

WWN

3

NIC

HBA

MAC

WWN

4

NIC

HBA

MAC

WWN

5

NIC

HBA

MAC

WWN

LANEthernet Switch

Traditional server-network connectivityEach server needs unique MAC and WWN assignments

13 6 January, 2010

HP BladeSystem Enclosure

1 NIC

HBA

2

NIC

HBA

3

NIC

HBA

4

NIC

HBA

5

NIC

HBA

Virtual Connect

MAC address pools

SANFibre Channel Switch

LANEthernet Switch

LAN Uplinks

SAN Uplinks

• Reduces cabling• Provides constant connections to LANs/SANs• Server changes have no impact on networks

WWN address pools

HP BladeSystem and Virtual ConnectSimplifies blade server connectivity

14 6 January, 2010

Keep the fabric outside the enclosure

Server 1

Server 2

Server 3

Server 16

SWITCH …

…

SWITCH …

Enclosure

Brocade,Cisco, orMcDataSwitch

…

Fabric

Server 1

Server 2

Server 3

Server 16

HP Virtual ConnectModule

…

… HP Virtual ConnectModule

…

Enclosure

Brocade,Cisco, orMcDataSwitch

…

Fabric

StandardBlade LAN or SANConfigurationToday

Blade LAN or SANConfigurationwith VirtualConnect

Brocade,Cisco, orMcDataSwitch

…

Brocade,Cisco, orMcDataSwitch

…

15 6 January, 2010

SAN

Storage Administrator

traditional servers

everyone must follow the moving server!

Server Administrator

A

B

C

Spare

LAN

FC Switch

Network Administrator

NIC

HBA

NIC

HBA

NIC

HBA

NIC

HBA

ENET Switch

WWN A

MAC A

WWN B

MAC B

XLAN Administrator

SAN Administrator

Server Administrator

Find the new portConfiguration of the new port: VLAN, Security, STP, etc

LUN investigationZoning modification based on new WWN

LUN masking modification

**ANIMATED SLIDE**VIEW IN SLIDE SHOW MODE

16 6 January, 2010

SAN

Storage Administrator

Server Administrator

A

B

C

Spare

LAN

Network Administrator

LAN Administrator

SAN Administrator

NIC

HBA

NIC

HBA

NIC

HBA

NIC

HBA

HP Virtual Connect Modules

Server AdministratorWWN A

MAC AX

Virtual Connect solves these& makes IT change-ready

LUN investigationZoning modification based on new WWN

LUN masking modification

**ANIMATED SLIDE**VIEW IN SLIDE SHOW MODE

ENET Switch

FC Switch

Find the new portConfiguration of the new port: VLAN, Security, STP, etc

17 6 January, 2010

Physic

al S

erv

er A

Physic

al S

erv

er A

Logical Server MobilityVirtual and Physical Server Comparison

Physic

al S

erv

er B

Host Server B(VMware ESX)

External Network Fabric

SSN: CCDD

UUID: 3344CD

MACs: A2 & B2

WWNs: C2 & D2

Host Server A

(VMware ESX)

Hypervisor

OS Image

SSN: AABB

UUID: 1122AB

MACs: A1 & B1

WWNs: C1 & D1

VMotion

Hypervisor

OS Image

Guest

Server 3

OSImage

VLAN 4Fabric B

FC Fabrics A&B

VLANs 1-5

SSN: AABB

UUID: 1122AB

MACs: A1 & B1

WWNs: C1 & D1

Guest

Server 1

OSImage

Guest

Server 2

OSImage

VLAN 3Fabric A

FC Fabrics A&B

VLANs 1-5

VLAN 1Fabric A

Physic

al S

erv

er B

SSN: CCDD

UUID: 3344CD

MACs: A2 & B2

WWNs: C2 & D2

External Network Fabric

Virtual Connect Interconnect Fabric

FC Fabrics A& B; VLANs 1- 5

Host Server 1(Windows, ESX, Linux, etc)

OS Image(Boot from SAN)

VC SSN: VC1DD

VC UUID: 5566EF

VC MACs: A3 & B3

VC WWNs: C3 & D3

FC Fabric A

VLANs 1 & 2

**ANIMATED SLIDE**VIEW IN SLIDE SHOW MODE

18 6 January, 2010

Virtual Connect Manager

• For single rack datacenters

• Each domain configured and managed independently

• Multiple consoles and addresspools

• For multiple racks of enclosures /domains

• Configure and manage groups of enclosures & domains centrally

• Single console and address pool

• Import existing Virtual Connect domains

Scalable for small and large infrastructures

Virtual Connect Enterprise Manager

• LAN Addresses (MAC)• SAN Addresses (WWN)• Server Connection Profiles

19 6 January, 2010

Multi-enclosure VC Domains

• Up to 4 c-Class enclosures

• Up to16 VC-Enet Modules

• Up to16 VC-FC Modules

• Requires− One ‘base enclosure’

− OAs & VC modules must be on same management network (IP subnet)

− All Ethernet modules interconnected

− All enclosures must have the sameVC-FC configuration (no stacking)

• No merge-domain feature (yet)

58 5/1/2007 HP Confidential -- INTERNAL USE ONLY

Enclosure Stacking (Ethernet)Any server NIC can access any data center uplink

• If 4 enclosures, 64 servers in Virtual Connect Domain

• Every server in domain has access to any data center LAN connection

• Every server has fault-tolerant access to every uplink port

• With this stacking method, every Virtual Connect module still has 8 -1GbE and 1 -10GbE uplink ports available

Virtual Connect Ethernet Modules

c-Class Enclosure

10 GbE Internal Cross-Connect

10GE ports still available as additional stacking links or data center uplinks

20 6 January, 2010

“Stacking” up to four VC Enclosures

DefinitionInterconnecting up to 4 VC enclosures to

be managed as a single VC domain

pNIC pNIC

Server Blade 2

Enclosure 1

ServerBlade 1

pNIC pNIC pNIC pNIC

Server Blade 3

LAN 2(Backup)

pNIC pNIC

Server Blade 2

Enclosure 2

ServerBlade 1

pNIC pNIC pNIC pNIC

Server Blade 3

pNIC pNIC

Server Blade 2

Enclosure 3

ServerBlade 1

pNIC pNIC pNIC pNIC

Server Blade 3

vNet_1

LAN 1(Production)

vNet_2

Spanning linksStacking links

21 6 January, 2010

Traditional Switch and Virtual Connect?

Switch

• Part of LAN or SAN network

• Managed from network

• Server directly connected, so any server change impacts network

Virtual Connect• Part of server system• Managed with servers

• Layer between servers and network, so external network isn’t negatively impacted server changes

• Allows flexibility and control of system resources

• Ideal for virtualization environments

What’s the difference?

Server

Server

Server

Server

LAN

SAN

Server

Server

Server

Server

LAN

SAN

Virtual Connect

Server

LAN\SAN

Server LAN\SAN

22 6 January, 2010

VMware Switch

VM VM VM VM

VMWARE VSWITCH

Datacenter LAN

VIRTUAL CONNECT

NIC

NIC

NIC

NIC

assigned MACs

assigned MACs

Is Virtual Connect different?

23 6 January, 2010

Virtual Connect Fibre Channel

Fibre Channel VC−Methodology

−Management

24 6 January, 2010

HP Virtual Connect FC Module uses N_Port_ID Virtualization

• Defined by the T11 FC standards− FC Device Attach (FC-DA) Specification, Section 4.13

− FC Link Services (FC-LS) Specification

• “NPIV provides a Fibre Channel facility for assigning multiple N_Port_IDs to a single N_Port, thereby allowing multiple distinguishable entities on the same physical port.”

• Where is it used?− When implemented within a FC HBA, allows unique WWNs & IDs

for each virtual machine within a server

− Can be used by a transparent HBA aggregator device (like HP VC-FC module) to reduce cables in a vendor-neutral fashion.

25 6 January, 2010

N_Port_ID Virtualizationwith an HBA aggregator

Server #1(WWN A)

HBAAggr.

(WWN X)

Server #2(WWN B)

Server #3(WWN C)

FLOGI X

FullFC SwitchwithNPIVSupport

ACC A

1. Fabric Login using HBA aggregator’s WWN (WWN X).Sets up the buffer credits for the overall link.Receives overall Port ID.

2-4a Server HBA does normal fabric login with their WWNs.

2-4b Server HBA fabric logins translated to ‘FDISC’.

5. Traffic for all 4 N-port IDs carried on the same link.

1

FLOGI A 2a

5

ACC X

ACC A

ACC B

ACC C

ACC B

ACC C

FDISC A2b

FDISC B3b

FDISC C4b

FLOGI B 3a

FLOGI C 4a

As usedby HP VC-FC

module

26 6 January, 2010

N_Port_ID Virtualizationwith an HBA aggregator

Server #1(WWN A)

HBAAggr.

(WWN X)

Server #2(WWN B)

Server #3(WWN C)

FLOGI X

FullFC SwitchwithNPIVSupport

ACC A

1

FLOGI A 2a

5

ACC X

ACC A

ACC B

ACC C

ACC B

ACC C

FDISC A2b

FDISC B3b

FDISC C4b

FLOGI B 3a

FLOGI C 4a

As usedby HP VC-FC

module

Uses standard FC HBAs(Qlogic, Emulex)No changes to server OS or device drivers

Works with a wide range of Brocade, McData, or Cisco FC Switches (with recent firmware)

Once server is logged in, the FC frames pass through unchanged!

Doesn’t interfere with server/SAN compatibility.

27 6 January, 2010

Using NPIV with a Brocade SwitchHow does it look from the switch management?

HBA #110:00:00:00:c9:38:96:3f

HBAAggr.

HBA #210:00:00:00:c9:39:5f:64

BrocadeSilkWorm200E

N-port

N-port (with NPIV)

N-port

Example Configuration

28 6 January, 2010

Using NPIV with a Brocade SwitchHow does it look from the switch management?

Identifies port as NPIV Switch tools show the port with the same WWN that the server tools see.

Uses same fabric port

29 6 January, 2010

VC-FC Uplink MappingWhy did we do it that way?

30 6 January, 2010

VC-FC uplink usage – 1 active uplink

Uplink 1

Uplink 2

Uplink 3

Uplink 4