1© Copyright 2011 EMC Corporation. All rights reserved.

VMAX Best Practices for VMware Environments

David Robertson vSpecialist

Itzik ReichvSpecialist

2© Copyright 2011 EMC Corporation. All rights reserved.

VMAX FA Flag Settings for vSphere• SPC-2

• SCSI 3 (Optional) No effect on ESX anymore.

• Unique WWN (UWN)

• Common Serial Number (C)

• OS2007 (Optional)

Screenshot from within VirtualCenter using the VSI4

3© Copyright 2011 EMC Corporation. All rights reserved.

Connectivity Considerations with VMAX

• VMware ESX Servers should have multiple physical HBA• VMware Servers should be connected to multiple

directors– Directors 7 and 8 in single engine configurations– Connections to different directors in different engines

in multiple engine configuration

• Connect each HBA to a minimum of two ports on different directors– Not a requirement but strongly recommended– I/O intensive workload will benefit– Servicing of the array is less impacting

4© Copyright 2011 EMC Corporation. All rights reserved.

Connectivity Considerations with VMAX− Cont.

HBA1 HBA2 HBA1 HBA2 VMware vSphere Servers

MinimumConfigurationIdealConfiguration

5© Copyright 2011 EMC Corporation. All rights reserved.

Connectivity Considerations with VMAX− Cont.

HBA1 HBA2 HBA1 HBA2 VMware vSphere Servers

6© Copyright 2011 EMC Corporation. All rights reserved.

Path Management with VMAX

• PowerPath/VE is strongly recommended for vSphere environment– PowerPath/VE is optimized for multi-controller

storage (i.e. VMAX)

• NMP policy is available with vSphere– Use Round Robin policy for Symmetrix arrays

• esxcli nmp satp setdefaultpsp -P VMW_PSP_RR -s VMW_SATP_SYMM

– We recommend setting the Round Robin IOPS parameter to 1 (default is 1000)

– Use Fixed Policy for Gatekeepers

7© Copyright 2011 EMC Corporation. All rights reserved.

Using the VSI to configure Multipathing policy

8© Copyright 2011 EMC Corporation. All rights reserved.

VMAX FA Configuration for SPC-2

• Turned on per Fibre Channel port or per initiator• Do not activate in a live system if not

previously set• Default from 5773 (DMX-4)

9© Copyright 2011 EMC Corporation. All rights reserved.

Performance and Storage Layout – vSphere with VMAX

• Use Virtual Provisioning– Always provides optimal balance in VMware

environments – FAST VP

• Physical Disk Size and Protection– Depends on the IO characteristics of the workload

• LUN Layout– Avoid using same set of disks for applications with

different I/O characteristics

• Configuration for I/O intensive application data– Follow best practices recommendations for a physical

server

10© Copyright 2011 EMC Corporation. All rights reserved.

Partition Alignment (VMFS and Guest OS)• Intel-based systems are misaligned due to metadata written by the BIOS to handle LBA to CHS translation– Partition Misalignment affects VMFS and Guest OS partitions

• Host-based Partition Utilities can be used to Align Partitions:

– For Linux and VMFS Alignment use fdisk• Offset partition to 64 KB boundary

• Aligned VMFS partitions are now automatically created by vSphere Client

– For Windows Operating Systems (2003 in particular) use the diskpart utility• Create partition aligned on 64 KB boundary

• Use diskpar for versions of pre-Windows 2003 System at SP1– Earlier versions of diskpart will show partitions as aligned, even if

they are not

• For Metavolumes (MetaLUNs) only the base device needs to be aligned

11© Copyright 2011 EMC Corporation. All rights reserved.

Effects of Partition Misalignment• Symmetrix VMAX uses 64K track size• In an Aligned System, the 64 KB write would be serviced by

a single drive• File-system misalignment affects performance in two ways:

– Misalignment causes disk crossings: I/O broken across two drives

– Misalignment causes stripe crossings: I/O broken across stripe elements

• Even if disk operations are buffered by cache, there is performance impact

12© Copyright 2011 EMC Corporation. All rights reserved.

VMAX Virtual Provisioning in VMware vSphere Environments

• vSphere provides native thin provisioning– Either one can be used– Both features can be used simultaneously but doing so

increases risk

• VMAX Virtual Provisioning simplifies drive and DA workload distribution– Provides additional benefits besides optimizing storage

use– Ensure enough paths and TDEVs to support the workload

• VMAX Virtual Provisioning provides additional benefits– Zero Reclaim and Rebalancing

13© Copyright 2011 EMC Corporation. All rights reserved.

VMAX Virtual Provisioning for vSphere–

Performance Considerations• RAID protection of data devices– Balance between performance versus

resiliency

• Fully allocate Virtually Provisioned devices if– Applications sensitive to latency– The risk of oversubscription is too high

• Optimum performance when IOs are aligned– VMFS is aligned on 64 KB boundary (use

vCenter)– Virtual disks should be aligned

• Including boot volumes (Linux, pre-win2k8/windows7)

14© Copyright 2011 EMC Corporation. All rights reserved.

VMAX Virtual Provisioning for vSphere– Performance Considerations – Cont.

• Striped versus Concatenated thin metavolumes– Depends on type of workload

• Small block versus large block• Random versus sequential• Reads versus writes

– Influenced by presence or absence of SRDF– Concatenated thin metavolumes can be grown

• Frequently exploited feature in VMware environments• Striped metas can be grown as of 5875 and SE 7.2

• Most VMware environments have small block random read workload

15© Copyright 2011 EMC Corporation. All rights reserved.

VMAX Zero Space Reclamation

• Reclaims thin pool storage by deallocating unnecessary track groups– Scans each track group and discards those containing all zeros– Deallocated tracks are presented as all zeros by Symmetrix to host

• Primary use is post migration from “thick” to “thin”– Migration performed using TimeFinder/Clone or Open Replicator for

Symmetrix

• Reclamation should be run prior to configuring any replication relationships– Thin devices in active TimeFinder or SRDF relationships will be

skipped

Thin

Pool

100110010001111000000000000000010011001000111100000000000000000000000000000000

16© Copyright 2011 EMC Corporation. All rights reserved.

VMAX Zero Space Reclamation – Cont.• Very useful tool in VMware environments– Relevant if customer upgraded to vSphere

VMware Thin

Zeroedthick (VMware Thick)

Eagerzeroedthick

17© Copyright 2011 EMC Corporation. All rights reserved.

VMAX Virtual ProvisioningAutomated Pool Rebalancing• Rebalances allocated tracks across data devices contained

within thin pool• Levels out imbalances caused by thin pool expansion

– Or unbinding thin devices from the thin pool

• Scheduled process that runs at given intervals– User defines imbalance as a percentage utilization difference

within the pool (user configurable 1% to 50%)

• In VMware environments, storage requirements can increase rapidly– Mass VM deployment (VDI, testing environments, etc…)

• Virtual environments are very dynamic– Adding datastores, removing datastores

• Automated Pool Rebalancing maintains performance and gives best TCO

Th

in P

ool

18© Copyright 2011 EMC Corporation. All rights reserved.

VMware vSphere and FAST VP

• FAST VP operates at the sub-LUN level• Analyzes/moves data in groups of 10 extents (1 extent

= 768k)– Great value in vSphere environments– RDMs, dedicated datastores

• Considerations are the same as that for physical servers

• Configure a single standard size of device on all tiers– Same size TDEV’s recommended

• For optimal system performance use Optimizer in concert with FAST VP– Optimizer will balance load within a tier

• Utilize EMC Virtual Storage Integrator to identify the Storage Type

19© Copyright 2011 EMC Corporation. All rights reserved.

EMC FAST in ActionEMC Storage with an active ESX Cluster

VMware VMware VMware VMware

All Fibre ChannelDisk Drives

Disk Resources are ~80% Busy

20© Copyright 2011 EMC Corporation. All rights reserved.

EMC FAST in ActionAdd Flash Drives and Apply FAST Policy

68% Less Disk I/O Contention

2.5X Faster Disk Response Time

VMware VMware VMware VMware

Tiered Storage4% Flash Drives96% FC Drives

21© Copyright 2011 EMC Corporation. All rights reserved.

VMware vStorage API for Array Integration (VAAI)Available on VMAX with Enginuity 5875

Block Zero

SCSI write – many blocks from host to array

Without API

SCSI Write Same command – one block from host to array and repeatedly written

With API

Reduce host I/O traffic Higher throughput for guest

applications

Benefits

Without

With

22© Copyright 2011 EMC Corporation. All rights reserved.

VMware vStorage API for Array Integration (VAAI)Available on VMAX with Enginuity 5875

Full Copy

SCSI Read – data from array to host SCSI Write – data from host to array

Without API

SCSI Extended Copy – data moved within the array

With API

Reduces time to migrate Reduces time to deploy

Benefits

Without

With

23© Copyright 2011 EMC Corporation. All rights reserved.

VMware vStorage API for Array Integration (VAAI)Available on VMAX with Enginuity 5875

Hardware Assisted Locking

Reserves the complete LUN Requires several SCSI commands

Without API

Works at a block level – no LUN lock

SCSI Compare Swap command

With API

Reduces SCSI reservation issues

Allows more changes to metadata

Benefits

Without

With

vmdk1vmdk2

vmdk1vmdk2

24© Copyright 2011 EMC Corporation. All rights reserved.

Thank You!

• EMC Symmetrix VMAX Enginuity 5875: FAST VP, VAAI–Making The Best Array even Better: http://itzikr.wordpress.com/2011/01/17/emc-symmetrix-vmax-enginuity-5875-fast-vp-vaai-making-the-best-array-even-better/

• SRM Automated Failback Using EMC Symmetrix VMAX: http://itzikr.wordpress.com/2011/01/10/srm-automatic-failback-using-emc-symmetrix-vmax/

• Increase Scalability for Virtual Desktops with EMC Symmetrix FAST VP and VMware VAAI: http://www.emc.com/collateral/hardware/white-papers/h8062-scalability-vmax-vaai-wp.pdf

• April 28-EMC Live Webcast: VNX Best Practices for VMware Environments: http://info.emc.com/mk/get/DBM10690-17709_raf_lp?reg_src=PA_Vmware

• Tuning NMP IOPS Parameter for Symmetrix Devices: http://www.emc.com/collateral/hardware/white-papers/h8119-tuning-vmware-symmetrix-wp.pdf

• Using EMC Symmetrix Storage in VMware vSphere Environments: http://www.emc.com/collateral/hardware/solution-overview/h2529-vmware-esx-svr-w-symmetrix-wp-ldv.pdf

• White Paper: Using VMware vStorage APIs for Array Integration with EMC Symmetrix VMAX: http://www.emc.com/collateral/hardware/white-papers/h8115-vmware-vstorage-vmax-wp.pdf

25© Copyright 2011 EMC Corporation. All rights reserved.

VMware Site Recovery Manager Failover/ Failback DEMO

26© Copyright 2011 EMC Corporation. All rights reserved.

Storage Pool Management feature for VSIDEMO