EMC Celerra VSA and VMware SRM Complete Setup and Configuration Guide Revision 1.0.1doc

EMC Celerra Virtual Storage Appliance (VSA)

&

VMware Site Recovery Manager

Building a low-cost, self-contained, learning, testing and development environment using EMC Celerra Virtual Storage

Appliances (VSAs) with VMware Virtual Infrastructure and VMware Site Recovery Manager (SRM)

Revision 1.0.1

January17, 2009

Bernie Baker Sr. VMware Specialist

EMC | VMware Affinity Team (339) 293-2320

[email protected]

Source Material:

http://virtualgeek.typepad.com/virtual_geek/2008/08/celerra-virtual.html

http://virtualgeek.typepad.com/virtual_geek/2008/11/celerra-virtual.html

http://virtualgeek.typepad.com/virtual_geek/2008/11/celerra-virtual-appliance-howto-301---replicating-between-two-celerra-vsas.html

Administering VMware Site Recovery Manager 1.0, Mike Laverick

VMware Site Recovery Manager 1.0 Evaluator Guide

VMware Site Recovery Manager 1.0 Update 1 Administration Guide

EMC® Celerra® NS Series iSCSI, EMC Celerra Replicator™ Adapter for VMware Site Recovery Manager, Version 1.0 Release Notes

EMC P/N 300-007-023

Configuring iSCSI Targets on EMC Celerra EMC P/N 300-004-153

Using EMC® Celera Replicator™ (V2)

P/N 300-004-188

On-Demand Web Replay: EMC Celerra Replicator Delivers Advanced IP Storage Protection

http://www.userlocal.com/helpvi.php

THE CONTENTS OF THIS WORK ARE NOT MEANT TO BE A COMPLETE INSTALLATION AND CONFIGURATION GUIDE BUT AN INITIAL CONFIGURATION VIABLE TO SUPPORT LEARNING, TESTING AND DEVELOPMENT OF THE CELERRA VIRTUAL STORAGE APPLIANCE (VSA) IN A NON-PRODUCTION VMWARE VIRTUAL INFRASTRUCTURE ENVIRONMENT WITH VMWARE SITE RECOVERY MANAGER (SRM).

NOTE: THIS CONFIGURATION, SPECIFICALLY RUNNING CELERRA VSAS ON VMWARE ESX, NOT SUPPORTED IN PRODUCTION BY EMC OR VMWARE. BEST EFFORT SUPPORT PRTOVIDED BY FORUMS.EMC.COM, VIRTUALGEEK.TYPEPAD.COM AND THE VMWARE AFFINITY TEAM WITHIN EMC.

� HTTP://FORUMS.EMC.COM

� HTTP://VIRTUALGEEK.TYPEPAD.COM

ALSO, NOTE THAT YOU CANNOT USE THIS FOR ANYTHING BEYOND TESTING AND DEVELOPMENT WITHOUT VIOLATING THE LICENSE AGREEMENT

THANKS TO MY COLLEAGUES ON THE VMWARE AFFINTY TEAM AT EMC FOR TESTING, PROOFING AND SUPPORTING THIS EFFORT

A SPECIAL THANK YOU TO STEPHEN SPELLICY FOR HIS EFFORTS IN THE CREATION OF THIS PAPER

EMC Celerra Virtual Storage Appliance (VSA) & VMware Site Recovery Manager

Page 3 of 146

Table of Contents

Overview.................................................................................................................................................5

Diagram ................................................................................................................................................5

Required Server and Network Hardware .........................................................................................5

Required Software ..............................................................................................................................6

Required Infrastructure ......................................................................................................................6

Required Commitment .......................................................................................................................7

General Comment ...............................................................................................................................7

Small Request ......................................................................................................................................7

My Network ...........................................................................................................................................8

Section 1: The Basics .........................................................................................................................9

Assumptions.........................................................................................................................................9

Step 1: Download and Import Celerra Virtual Storage Appliance (VSA) ..................................10

Step 2: Configuring the Celerra VSA VM .......................................................................................13

Step 3: Configuring the Celerra VSA TCP/IP addresses..............................................................14

Step 4: Configuring the Celerra VSA to be Unique......................................................................18

Step 5: Licensing...............................................................................................................................27

Step 6: Configuring cge IP addresses ...........................................................................................32

Section 2: Adding Physical Storage ............................................................................................33

Step 1: Add an new VMDK to your Celerra VSA VM....................................................................33

Step 2: Configure the Celerra VSA to use the new storage .......................................................34

Step 3: Adding “Physical” Disk to the Celerra VSA ......................................................................37

Step 4: Configure the Celerra VSA to use the new storage .......................................................39

Section 3: Review..............................................................................................................................40

Section 4: Configuring Celerra Replication .............................................................................41

Step 1: Configuring NTP (Network Time Protocol)......................................................................42

Step 2: Correcting the Celerra VSA Replication Database ..........................................................44

Step 3: Configuring Replication Using the Celerra Manager GUI ..............................................48

Step 4: Preparing ESX Servers for iSCSI Targets and LUNs .......................................................72

Section 5: Site Recovery Manager Installation......................................................................79

Step 1: Site Recovery Manager Database Connectivity ..............................................................80

Step 2: Copying Site Recovery Manager and the SRA ................................................................88

Step 3: Installing Site Recovery Manager .....................................................................................88

Step 4: Installing the Storage Replication Adapter......................................................................91

Step 5: Installing the SRM Plug-in for Virtual Center ..................................................................91


Page 4 of 146

Section 6: Site Recovery Manager Configuration .................................................................95

Step 1: Configuring the Connection...............................................................................................96

Step 2: Configuring the Array Managers.......................................................................................99

Step 3: Configuring Inventory Mappings (Optional) .................................................................104

Step 4: Configuring Protection Groups........................................................................................106

Step 5: Configuring Recovery Groups .........................................................................................109

Step 6: Testing (at long last…) .....................................................................................................113

Appendix A: Configuring the Replication Target (Command Line Interface) ..........119

Configuring Replication ..................................................................................................................122

Appendix B: iSCSI and NFS Discrete LUN Creation & Assignment ...............................124

Step 1: Creating iSCSI Targets......................................................................................................124

Step 2: Configuring iSCSI LUNs and presenting them to ESX as VMFS Datastores ...........126

Step 3: Configuring NFS LUNs and presenting them to ESX as NFS Datastores..................137

Appendix C: Basic VI Commands...............................................................................................144

Appendix D: Troubleshooting Tips............................................................................................146


Page 5 of 146

Overview There are two goals associated with this initiative. The first is this installation and configuration of “geographically separate” Celerra VSA instances complete with bi-direction replication. The second goal leverages these Celerra VSA for the purpose or testing and developing a working knowledge of VMware’s Site Recovery Manager (SRM).

The ultimate goal is to build and environment similar to the one depicted in the diagram below.

Diagram

Required Server and Network Hardware

In order to accomplish this task, the following physical equipment is required:

• Two physical servers: (in my lab environment I used Dell PowerEdge 1850s

o Intel or AMD class, single processor or better, 4GB RAM (Minimum) 6-8GB preferred (the more you have the more testing you are able to accomplish e.g., more virtual machines), Internal disk with at least 100GB of free capacity or access to an external shared storage device, two physical GbE NICs per server

• GbE network (a simple GbE switch will suffice, flow control and trunking would be nice)

Need to build servers on a budget? Check out the following link:

http://virtualgeek.typepad.com/virtual_geek/2008/06/building-a-home.html

VMware: ESX1 ESX 3.5 U2Build 82663

Evaluation LIcense

VMware: ESX2ESX 3.5 U2Build 82663

Evaluation License

Virtual Center 2.5Update 3

Build 119598

RecoverySite

Protected

Site

Virtual Center 2.5Update 3

Build 119598

VirtualCenter VirtualCenter

TCP/IP Network

Array ReplicationArray Replication

Celerra Replication

Asynchronous

SRM 1.0 Update 1SRM 1.0 Update 1

VCPROD1 VCDR1

VMW Virtual Center 1 – Non ReplicatedCelerra VSA (csprod1)MS Active Directory (1)MS Windows XP (2)MS SQL Server 2005

VMW Virtual Center 2 – Non ReplicatedProtected Site VM Staging Area

Celerra VSA (csdr1)

DellPowerEdg 1850

Two Intel 2.8GHz4 GB RAM

Two GbE (Broadcom) AdaptersPERC4 (RAID Controller)Two 73GB SCSI RAID1

Dell Remote Access Card (DRAC)

DRAC IP:xxx.xxx.xxx.xxx

DellPowerEdg 1850

Two Intel 2.8GHz4 GB RAM

Two GbE (Broadcom) AdaptersPERC4 (RAID Controller)Two 73GB SCSI RAID1

Dell Remote Access Card (DRAC)

DRAC IP:xxx.xxx.xxx.xxx

Site-Site SRM Communications via TCP/IP

VMware Demo Environment

Site Recovery Manager (SRM)

Site RecoveryManager


These Disks are

provisioned from within the Celerra VSA VMs. The disk

used to setup the Celerra VSA VMs is provisioned

from the storage internal to

the physical server ESX is running on


Page 6 of 146

Required Software

• VMware ESX 3.5 Update2 (Evaluation License: 60 days by default from VMware download site)

• VMware Virtual Center 2.5 Update 3 (Evaluation License as per above)

• VMware Site Recovery Manager 1.0 Update 1 (Evaluation License as per above)

• The EMC Celerra Virtual Storage Appliance (VSA)

• Microsoft Windows XP or Windows 2003 Server R2 Enterprise Edition for use with the VMware Virtual Center Server

• Microsoft Windows XP for use as a utility device as required. It makes sense to use several of these as for SRM

• VMware SCSI Floppy image for installing XP in a VM on ESX

• Windows 2003 Server R2 Standard Edition or Enterprise for various servers you may want to create and test against. e.g., Active Directory, DNS, SQL, Exchange, etc.

• Other Operating Systems such as Solaris x86, Red Hat Enterprise Linux, CentOS or Ubuntu

• EMC Celerra Replicator Adapter for VMware Site Recovery Manager

• Microsoft SQL Server 2005 Management Studio Express (SSMSEE)

• Microsoft SYSPREP (optional) – provides for customization of VMs deployed from a template

Required Infrastructure

There are several components in this lab that work better if you have Active Directory, Domain Name Server (DNS), Windows Internet Name Service (WINS) and DHCP enabled. These don’t have to be large elaborate implementations. In fact, all of these services could be setup in a single VM on one of your ESX hosts. This about simple domain names like lab.com or vsa-srmlab.com. Create a DHCP scope to hand out TCP/IP addresses, a default gateway, DNS and WINS information. Have the client devices register host information automatically with DNS. This is a checkbox in the TCP/IP properties of the Ethernet adapter for the defined virtual machines. For those devices with static IP address add them to the DNS tables as “A” record, host, entries. Don’t forget to click the PTR (reverse lookup) check box.

A quick note about disk space; you should plan on allocating disk capacity for the primary and DR site Celerra VSAs as follows:

� Primary site: as required/desired to support your environment. Default, required minimum, is 40GB. Add space as required.

� DR Site: plan to allocate a file system that is 2.5-3 times larger that the replication target LUN. With out this space you will have issues creating snapshots required for Site Recovery Manager.


Page 7 of 146

Required Commitment

The task you are about to embark on will take some time to setup and configure. In my estimation, you should be able to accomplish the complete implementation in a day or two.

The first time I went through the task this document didn’t exist. I took me and a colleague two days to complete all of the required tasks leveraging a multitude of different documents and subject matter experts. Now that we have done it several times we can have a full environment up and running within 8 hours. That assumes, two people working from scratch, installing ESX, setting up active directory, test machines, Celerra VSA installs and SRM. There is a lot of work to accomplish but if you stick with it, the benefits will be instantly recognizable.

General Comment

If you need any assistance setting up your VMware ESX infrastructure I suggest reviewing these documents:

� Release Notes: ESX Server 3.5 Update 2 and VirtualCenter 2.5 Update 2

� Quick Start Guide

� Basic System Administration

Small Request

This document represents a great deal of effort on my behalf and that of several of my peers on the EMC VMware Affinity Team. We appreciate your desire to test VMware and EMC in your environment. If you have any questions or comments please forward them to my email address at [email protected] or [email protected]. Please let me know if you find any problems or glaring omissions. I have tried to ensure completeness and accuracy.


Page 8 of 146

My Network

VMware: ESX2

ESX 3.5 U2

Build 82663

Evaluation License

Virtual Center 2.5

Update 3

Build 119598

RecoverySite

Protected

Site

Virtual Center 2.5

Update 3

Build 119598

VirtualCenter VirtualCenter

iSCSI/TCP/IP Network

Array Replication

EMC Celerra VSA

Asynchronous

Replication

(v2)

SRM 1.0 Update 1SRM 1.0 Update 1

VCPROD1

IP Address: xxx.xxx.xxx.xxx

VCDR1

IP Address: xxx.xxx.xxx.xxx

VMW Virtual Center 1 w/ License Server and SRM

Server – Non Replicated

Celerra VSA (csprod1)

MS Active Directory (2)

MS DNS (2)

MS DHCP (1)

MS Windows XP (Clients) (4)

VMW Virtual Center 2 w/ License Server and SRM

Server – Non Replicated

Celerra VSA (csdr1)

Protected Site VM Staging Area

Qty 2

Dell

PowerEdge 1850

Two Intel 2.8GHz

8 GB RAM

Two GbE Adapters

PERC4 (RAID) Controller

Two 73GB 15K SCSI RAID 1

VMware DRS/HA Cluster

(vMotion Enabled)

Dell Remote Access Card (DRAC4)

DRAC IP: xxx.xxx.xxx.xxx (192.168.1.0 Net)

VMware ESX2 IP:

eth0: xxx.xxx.xxx.xxx (192.126.1.0 Net)


VMware ESX3 IP:



EMC Celerra VSA VM IP:

eth0: xxx.xxx.xxx.xxx (General/NFS/CIFS)

– 192 Net

eth1: xxx.xxx.xxx.xxx (iSCSI)

– 10 Net

eth2: xxx.xxx.xxx.xxx (Control Station)

– 192 Net

cge0: xxx.xxx.xxx.xxx (Data Mover NFS/CIFS target)

– 192 Net

cge1: xxx.xxx.xxx.xxx (Data Mover iSCSI target)

– 10 Net

Qty 2:

Dell

PowerEdge 1850

Two Intel 2.8GHz

8 GB RAM

Two GbE AdaptersPERC4 (RAID) Controller

Two 73GB 15K SCSI RAID 1

VMware DRS/HA Cluster

(vMotion Enabled)

Dell Remote Access Card (DRAC4)

DRAC IP: xxx.xxx.xxx.xxx (192.168.1.0 Net)

VMware ESX3 IP:



VMware ESX4 IP:



EMC Celerra VSA IP:

eth0: xxx.xxx.xxx.xxx (General/NFS/CIFS)

-192 Net

eth1: xxx.xxx.xxx.xxx (iSCSI)

-10 Net

eth2: xxx.xxx.xxx.xxx (Control Station)

-192 Net

cge0: xxx.xxx.xxx.xxx (Data Mover NFS/CIFS target)

-192 Net

cge1: xxx.xxx.xxx.xxx (Data Mover iSCSI target)

-10 Net

Site-to-Site SRM Communications via TCP/IP

EMC Celerra VSA and

VMware Site Recovery Manager

(SRM) Lab Environment



These Disks are provisioned

from within the Celerra VSA

VMs. The disk used to setup

the Celerra VSA VMs is

provisioned from the storage

internal a physical server on

which ESX is running: Server

Name is ESX1 (not pictured)

All server and Celerra VSA TCP/IP addresses are static.

The client TCP/IP assignments accomplished via MS DHCP server running as a VM on ESX1

DNS Domain: baker-iss.com

VMware: ESX1

ESX 3.5 U2

Build 82663

Evaluation LIcense


Page 9 of 146

Section 1: The Basics Important Note:

This document makes mention of a second Celerra VSA for use with VMware’s Site Recovery Manager. All of Section 1 and Section 2 will need to be completed for two Celerra VSA instances on separate VMware ESX hosts. This document assumes the following names for these instances: csprod1 and csdr1. Where csprod1 represents a mock production environment (A.K.A. the protected site) and csdr1 is the disaster recovery partner (A.K.A. the recovery site). I assume you, as the reader and implementer, will create the recovery site Celerra (csdr1) using the same steps required to configure the protected site (csprod1). This will be accomplished without specific instructions for the recovery site since they are identical to the protected site with name and network changes applied. This text will also cover the specific steps necessary to configure replication components required for the Celerra VSAs. Lastly, we will discuss, in detail, the steps required to setup and configure VMware Site Recovery Manager on both the protected and recovery sites.

With the intention of streamlining the installation and configuration process, this document focuses on the core steps required to build out multiple Celerra VSAs to support replication and disaster recovery using the Site Recovery Manger (SRM) from VMware. As mentioned on page 2, some of the source material contained within was provided via the Virtual Geek web blog. I’ll reference the blog on a few occasions throughout this document. Please note the main URL for this site is: http://virtualgeek.typepad.com

Assumptions

• I have assumed some core competency with respect to Linux, VMware ESX and VMware Virtual Center. I have added tips on specific Linux commands and tried to remain consistent with those tips throughout

• Active Directory implementation to support the environment (I setup two AD virtual machines. One on each ESX host)

• I also assume you have two preconfigured VMware ESX 3.5 Update2 servers under the management of VMware Virtual Center 2.5 Update3 implemented as virtual machines on the ESX servers each VC manages

• You will need full administrative access to all of the platforms discussed

• Need a little help with vi? See Appendix C – Basic vi Commands • One last thing… The Celerra VSA is implemented in a service know as “blackbird” and is

sometimes referred to as a Celerra Simulator.

So have you had enough of the blah blah blah? Let’s get started with the Celerra VSAs!!


Page 10 of 146

Step 1: Download and Import Celerra Virtual Storage Appliance (VSA)

Ensure you have the latest version of the Celerra Virtual Storage Appliance (VSA). It is available at the following location:

• Virtual Geek, a blog maintained by Chad Sakac, Sr. Director, EMC VMware Strategic Alliance

THIS IS AN EXTREMELY LARGE FILE (approximately 1.4GB).

ALLOW ADEQUATE TIME FOR THE DOWNLOAD TO COMPLETE

Once downloaded:

� unzip the file � make note of the location � start the Virtual Infrastructure Client and attach to your “production”

Virtual Center instance (in my case it was vcprod1)

� Don’t forget to come back and perform these steps again for the “recovery” site. You will attach to the “recovery” site Virtual Center instance (mine was vcdr1) and configure the Celerra VSA in the same manner as the “production” site


Page 11 of 146

From Virtual Center, Click “File”, “Virtual Appliance”, “Import”

Click on the Import from File radio button, Click “Browse”, Browse to the .OVF file and Click “Open” Click “Next” when returned to the Import Location panel.


Page 12 of 146

Identify the Name of the VM (csprod1 and csdr1) and the target datastore. In most cases I will choose to put the Celerra VSA on the local datastore of the ESX hosting the virtual machine. If and when you add additional storage to the Celerra VSA the source is inconsequential. It can come from a local source (internal server disk) or a network based source such as a physical EMC Celerra. Setup network mappings (leave as default for now)

• Summary, Finish, Import process started. Typically the import process takes about 15 minutes to complete. Note: OVF is approximately 1.4GB. When finished importing the file will be in eagerthick VMDK format and approximately 40GB.

• Once imported Click Close The Celerra VSA VM will be displayed in the Virtual Center Inventory.

Prior to configuring anything else; clone the Celerra VSA VM to Template from Virtual Center for future use. Right Click on the Celerra VSA VM and Select Clone to Template. (Note: The Celerra VSA VM needs to be powered off in order to complete this step) Give it a name and datastore location; identify the cluster on which to store the template. Store it in Compact format.


Page 13 of 146

Step 2: Configuring the Celerra VSA VM

Edit the Virtual Machine (VM) settings Right Click on the Celerra VSA VM and Select “Edit Settings” Change the Memory allocated from the default to 2048 (min for initial testing) or 3072 (replication or SRM). If you have the extra memory set this to 4096 as I did for my configuration.

Configure Networks to map appropriately to your ESX configuration – This configuration requires two Physical NICs in your ESX host. If you have more than two that is ok. This document assumes only two.

eth0 – vSwitch0 (connected to LAN Segment for NFS) eth1 – vSwitch1 (Dedicated to iSCSI traffic) eth2 – vSwitch0 (Maps to the Celerra Control Station Interface)

Click “OK” when you have finished making these changes. This screen shot also shows additional hard disks assigned to the Celerra VSA VM. Your configuration may or may not require these. For now, take the defaults on the disk configuration. We will discuss additional storage later in this document.

Power ON the Celerra VM. Right Click on the Celerra VM name and Select “Power On”


Page 14 of 146

Step 3: Configuring the Celerra VSA TCP/IP addresses

The following steps are specific to the Celerra VSA only. A physical Celerra offers a configuration wizard for these steps

Right Click on the Celerra VSA VM. Select “Open Console”

Please note: You will need to click inside the VM console window in order to interact with the command line. If the screen is blank, click inside the window and hit the Enter key. If you need to interact with some other window type CTL-ATL together to release the cursor from the VM window. This is required as VMware Tools is not installed on the VM.

Login to the Control Station; User: root password: nasadmin

Type the following command: (do not type [root@localhost ~]# as this is the system prompt)

[root@localhost ~]# ifconfig | less The less command will allow for scrolling up and down within the command output for the purpose of viewing all of the interfaces.


Page 15 of 146

The virtual NICs (vNICs) are as follows:

� dart-eth0: (Internal Interface for DART OS Data Mover – Celerra VSA service)

� dart-eth1:(Internal Interface for DART OS Data Mover – Celerra VSA service)

� eth0: connected to first vNIC (by default this maps to cge0 and cge1 in the Celerra Manager but this needs to be changed – we’ll do that later and explain what cge0 and cge1 are used for)

� eth1: connected to the second vNIC (by default not connected to any cge ports but

it should be)

� eth2: management interface for the Celerra VSA Control Station eth2 may not be listed as this point. If not, don’t worry, we’ll add it momentarily.

� lo: loopback


Page 16 of 146

Configure the NICs

The following set of commands will configure the TCP/IP address information and reset the Virtual NICs accordingly.

Type the following command: [root@localhost ~]# netconfig -d eth0

netconfig will launch a pseudo GUI to accomplish this task: Enter the appropriate values for each interface. eth0 will act as the NFS interface, eth1 will handle the iSCSI traffic and eth2 will be the Celerra VSA Control Station interface. This is the interface you will use for SecureShell (SSH) sessions. It is also the interface for the Celerra Manager GUI.

Repeat the process for eth1 and eth2

[root@localhost ~]# netconfig -d eth1 [root@localhost ~]# netconfig -d eth2

Once completed enter the following commands in ordered succession.

[root@localhost ~]# ifdown eth0

[root@localhost ~]# ifdown eth1 [root@localhost ~]# ifdown eth2 [root@localhost ~]# ifup eth0 [root@localhost ~]# ifup eth1 [root@localhost ~]# ifup eth2


Page 17 of 146

Alternatively you could reboot the Celerra VSA (but that takes more time) using the command: reboot –n from the system prompt: [root@localhost ~]#

Validate the TCP/IP address information has been assigned correctly:

[root@localhost ~]# ifconfig | less (check the IP addresses for proper configuration)

Ping each of the interface addresses including the DART interfaces and ensure a positive response:

[root@localhost ~]# ping <eth0 ipaddress> [root@localhost ~]# ping <eth1 ipaddress> [root@localhost ~]# ping <eth2 ipaddress> [root@localhost ~]# ping 128.221.252.2 Note this is the simulated data mover [root@localhost ~]# ping 128.221.253.2 Note this is the simulated data mover


Page 18 of 146

Step 4: Configuring the Celerra VSA to be Unique

These steps are required for Celerra VSA Replication and VMware Site Recovery Manager to function properly.

In these steps we will change the host name and correct the fact that the MAC addresses have changed

You should still be logged in as root. If not, login to the Control Station (eth2 IP Address); via SSH or use the Open Console selection from the right click menu on the Celerra virtual machine. If you prefer an SSH client one is freely available from http://www.putty.org user: root password: nasadmin

[root@localhost ~]# cd /opt/blackbird/tools

[root@localhost ~]# ls –l (this displays the contents of the /opt/blackbird/tools directory)

Enter the following command: [root@localhost tools]# ./init_storageID

This regenerates the Celerra VSA Serial number to match the VM UUID – The Celerra VSA was created from a clone. This removes the clone Serial Number and replaces it with a unique identifier. This is a critical step as we will setup replication in a later step and each Celerra VSA instance in a replication partnership needs a unique ID

Once completed we need to update the host file using the following command:

[root@localhost tools]# vi /etc/hosts (see Appendix C for assistance with vi commands)


Page 19 of 146

Insert an entry below the localhost entry as follows: 192.168.1.100 csprod1 csprod1.mydomain.com csprod1

Use the IP address you specified for the Celerra VSA Simulator Control Station interface (eth2)

save the file by hitting <Esc> then typing :wq!

Now for the network file in /etc/sysconfig. This will ensure the host boots with the proper host name.

[root@localhost tools]# vi /etc/sysconfig/network

Add the following lines to the bottom of the existing file: DOMAINNAME=mydomain.com (in my case: baker-iss.com) HOSTNAME=csprod1 save the file by hitting <Esc> then typing :wq!


Page 20 of 146

In order to save a reboot, issue this command:

[root@localhost tools]# hostname csprod1 This changes the host name immediately. The previous two file edits (/etc/hosts and /etc/sysconfig/network) will ensure host name retention over a reboot. [root@localhost tools]# exit (may need to repeat this a couple of times to get back to the login prompt)

csprod1 login: Notice the host name has changed, it used to look like this; localhost login:

We will now login with the nasadmin user account. This account is used for all activities not related to the configuration of the Linux operating environment but to the actual Celerra VSA environment.

Login to the Control Station (eth2 IP Address); user: nasadmin password: nasadmin


Page 21 of 146

Adjustments required for the new NIC MAC address:

The Celerra VSA service (blackbird) stores its interface MAC address so we need to change them. This is due the fact we created this Celerra VSA instance from an .OVF file which was created from a physical EMC Celerra platform. The blackbird service is aware of the previous physical network interface MAC addresses and we need to change them to match the MAC address of the Celerra VSA Virtual Machine’s virtual NICs (vNICs). We’ll accomplish this with the following set of commands:

[nasadmin@csprod1 ~]$ cd /opt/blackbird/tools

[nasadmin@csprod1 ~]$ ls –l (this displays the contents of the /opt/blackbird/tools directory) [nasadmin@csprod1 tools]$ ./configure_nic server_2 –l This lists the NICs in their current configuration

Notice that both cge0 and cge1 map to eth0; we will delete and re-add them to resolve this problem

[nasadmin@csprod1 tools]$ ./configure_nic server_2 -d cge0 (Delete cge0) [nasadmin@csprod1 tools]$ ./configure_nic server_2 -d cge1 (Delete cge1)

As of these commands execute the output shows the remaining configured cge interfaces. Since we have run this command twice the output from the second execution shows an empty list.


Page 22 of 146

As a second confirmation execute the same command with the list option:

[nasadmin@csprod1 tools]$ ./configure_nic server_2 –l

The output should confirm the deletion and non-existence of the previously defined cge interfaces.

Now that the cge interfaces have been deleted and the NAS_DB has been exported, we need to reboot the Celerra VSA. When the system comes back up we will re-add these interfaces and map them to the appropriate eth interfaces. [nasadmin@csprod1 tools]$ export NAS_DB=/nas This is the Linux Switch User command. By default su assumes the current user is switching to the root user account. The “-“ as in su – forces the system to reload the new user’s profile information. [nasadmin@csprod1 tools]$ su –

Password: nasadmin

This command will shutdown the Linux operating system, warm reset the virtual machine and restart the Linux operating system and Celerra VSA [root@csprod1 tools]# reboot –n


Page 23 of 146

Post Reboot #1:

Login to the Control Station (eth2 IP Address); user: nasadmin password: nasadmin

Note the boot process take approximately 20 minutes to complete. The system will appear to be up but the Celerra VSA (blackbird) service may not be fully operational. To check the status of the service issue the following command: [nasadmin@csprod1 ~]$ /nas/sbin/getreason

When the status matches the above the Celerra VSA is up and fully operational. We now need to add the cge interfaces back to the configuration and map them to their appropriate eth interface.

[nasadmin@csprod1 ~]$ cd /opt/blackbird/tools [nasadmin@csprod1 ~]$ ls –l (this displays the contents of the /opt/blackbird/tools directory) [nasadmin@csprod1 tools]$ ./configure_nic server_2 -l Notice the list is empty. It should look just as it did prior to the reboot. [nasadmin@csprod1 tools]$ ./configure_nic server_2 -a eth0 (maps eth0 to cge0) [nasadmin@csprod1 tools]$ ./configure_nic server_2 -a eth1 (maps eth1 to cge1)


Page 24 of 146

[nasadmin@csprod1 tools]$ ./configure_nic server_2 -l

Notice the list now reflects appropriate mapping between cge0/eth0 and cge1/eth1

[nasadmin@csprod1 tools]$ export NAS_DB=/nas

We need to reboot:

[nasadmin@csprod1 tools]$ su - Password: nasadmin [root@csprod1 tools]# reboot -n


Page 25 of 146

Post Reboot #2:

Login to the Control Station (eth2 IP Address); user: nasadmin password: nasadmin Remember to wait for the blackbird service to finish loading. To check use the following command:

[nasadmin@csprod1 ~]$ /nas/sbin/getreason Once the service had completed loading issue the following commands:

[nasadmin@csprod1 ~]$ nas_cel -list

Output refers to the hostname (name) as “localhost” and not “csprod1” the net_path may be pointing to the lo (loopback) interface as 127.0.0.1. We’ll correct this using the next series of commands. First thing we need to do is su to the root user as we discussed a moment ago. What is different now is that we will not pass the “-“ argument to the su command as we want to maintain the nasadmin profile, specifically the file system pathing.

[nasadmin@csprod1 ~]$ su (This is not a typo. The command here is su not su -) Password: nasadmin

Now that the network interfaces are properly configured and the host name is set properly, the internal Celerra database must be updated. To do so execute the next command.

[root@csprod1 ~]# nas_cel -update id=0 Operation in progress (not interruptible)

Ignore the warning about the loopback interconnect and not that this step may take a minute or two to complete

[root@csprod1 ~]# nas_cel -list This verifies the change was completed


Page 26 of 146

As we continue these housekeeping tasks we need to update the SSL certificates associated with the Celerra VSA to avoid errors on the Celerra VSA's web management interface. To do this, enter the following commands:

[root@csprod1 ~]# /nas/sbin/nas_config -ssl Do you want to proceed? [y/n]: y [root@csprod1 ~]# /nas/sbin/js_fresh_restart The new SSL certificate has been generated and installed successfully.

Time to reboot the VM again. Since we executed the su command earlier we don’t need to do it again here. Simply type the reboot command and hit Enter.

[root@csprod1 ~]# reboot -n


Page 27 of 146

Step 5: Licensing

Login to the Control Station (eth2 IP Address); user: nasadmin password: nasadmin Remember to wait for the Celerra VSA service to finish loading. To check use the following command:

[nasadmin@csprod1 ~]$ /nas/sbin/getreason

First we need to initialize the Celerra Licensing Database [nasadmin@csprod1 ~]$ nas_license -init done

Now we can enable the specific licenses available with in the Celerra Manager. It is important to note that EMC does not require the actual purchase or installation of a license. The Celerra VSA is enabled for all protocols and advanced features. Please recall using this platform for anything other than learning, testing or development is in violation on the license agreement. Launch each of these commands in succession. Each command will signify a successful application of the feature with the message: done

[nasadmin@csprod1 ~]$ nas_license –l

The nas_license –l command will output the site_key number only. Next time we run this command it will list the site_key and all of the applied licenses. See the graphic on the next page

[nasadmin@csprod1 ~]$ nas_license –c advancedmanager [nasadmin@csprod1 ~]$ nas_license –c nfs [nasadmin@csprod1 ~]$ nas_license –c cifs [nasadmin@csprod1 ~]$ nas_license –c iscsi [nasadmin@csprod1 ~]$ nas_license –c snapsure [nasadmin@csprod1 ~]$ nas_license –c replicatorV2


Page 28 of 146

[nasadmin@csprod1 ~]$ nas_license –l


Page 29 of 146

As an option, you could have performed the licensing task from the Celerra Manager GUI. if you are so inclined the steps are as follows:

Open a web browser to the running Celerra VSA VM Management interface: https://csprod1

Best Practice: Add your Celerra VSAs to a DNS server in your environment. If DNS is not an option, define your machine name is short and FQDN format in your host file. Do the same for the Celerra VSA use as the disaster recovery target in the recovery site.

The Celerra Manager requires Java. Java will be installed for you if it is not present on your machine. Of course this requires a connection to http://www.sun.com for the installation bits. From my personal experience Internet Explorer 7 will always indicate an issue with the SSL certificates issued during connection even after the SSL configuration has been modified. Internet Explorer 6 works great as does Firefox as long as Java is installed properly and operational. I used Firefox 3.0.4 in most cases.

Login with the following credentials; Username: nasadmin Password: nasadmin


Page 30 of 146

If you get an HTTP 503 Error: logout, wait a few moments and log back in. This error is caused as a result of incomplete initialization of web services within the Celerra VSA. Give it a couple of minutes to complete. If you get a cookie error there may be a DNS issue. Resolve the DNS issues or use a Fully Qualified Domain Name (FQDN) or TCP/IP addresses. Example: FQDN: http://csprod1.baker-iss.com TCP/IP: http://192.168.1.246 - this was the TCP/IP address of my csprod1 If you receive a “secure connection failed” error while using the Firefox Web Browser you may need to delete the cached certificate. This may happen if you run thru the document more than once or connect to a Celerra VSA with the same name. (It may happen in a lab. I happen to me and at least on of the document testers. Navigate to Tools, Options, Advanced, Encryption. Click View Certificates, Authorities. Scroll to the certificate in question, Select said certificate and Click “Delete”. Click “OK”


Page 31 of 146

When presented with the Celerra Message of the Day (MOTD) Click “OK”. You have the option to not see the additional MOTD pop-ups until the message changes. There is a check box on the panel.

From the Celerra Manger Main Page: Select the Licenses tab

Select the licenses one at a time. Click “Apply”, Click “OK” Celerra Manager – Advanced Edition License NFS Licensed CIFS Licensed iSCSI Licensed SnapSure Licensed (CIFS, NFS, iSCSI Snapshots) ReplicatorV2 Licensed (iSCSI and Fibre Channel Only – NFS not supported)


Page 32 of 146

Step 6: Configuring cge IP addresses

This step configures the “real” data mover IP addresses (cge0/1) and configures the initial iSCSI target

Note: cge breaks down as “c” (Copper) “ge” (Gigabit Ethernet) Using the Web Management interface: https://csprod1 Login with the following credentials; Username: nasadmin Password: nasadmin

Navigate to and Click on the “Network” Folder (left side panel, middle of the list); you'll notice two el3x (where x = 0 or 1) interfaces. These interfaces facilitate Control Station to Data Mover communications. We will configure cge0/1 with TCP/IP address

Click “New”

In the presented panel, configure cge0: Select Data Mover (server_2), Device Name (cge0) and add the TCP/IP address information you want to assign. This will be the NFS export address on your test network should you choose to configure and enable NFS.

The TCP/IP address can not be the same as eth0 even though they bind to the same physical interface

Perform these steps again for cge1. (This will map to eth1; your iSCSI interface)


Page 33 of 146

Section 2: Adding Physical Storage This section walks through the process of adding physical storage to the Celerra VSA. This includes modifications to the VM with the addition of a new VMDK, configuring the storage within the Celerra VSA host OS i.e., partitioning, formatting, and mounting the new storage and then reviews the steps required to "plug in" the “new” enclosures. Note: This is not explicitly required but most folks want to play with more storage the 25GB available in the Celerra VSA by default. I’ve added these steps so you have the option if you choose to add capacity. Of course you must have available physical capacity to support the additional virtual capacity in the Celerra VSA. This storage can be in the form of Direct Attached/Internal Server Storage (DAS) or iSCSI/FibreChannel (SAN) based storage.

Step 1: Add an new VMDK to your Celerra VSA VM

• Power Down the Celerra VSA M This can be accomplished via CLI within the VM or via Virtual Center

• Take a snapshot of the Celerra VSA VM from Virtual Center

• Right Click on the Celerra VSA VM and Select “Edit Settings”

• Add a new VMDK to the Celerra VSA VM, make it an appropriate size

Best practice: add many small VMDKs vs. one large one. Each VMDK file will be associated with a physical disk partition. If you every need to run files system check (fsck) on one of these physical partitions the process will be less intrusive and will typically require less time. Note: Maximum size of a physical partition/virtual disk file is 2,048 GB (2 TB)

• Power On Celerra VSA VM


Page 34 of 146

Step 2: Configure the Celerra VSA to use the new storage

From the Celerra Console

Login to the Control Station (eth2 IP Address); user: nasadmin password: nasadmin Remember to wait for the Celerra VSA service to finish loading. To check use the following command:

[nasadmin@csprod1 ~]$ /nas/sbin/getreason [nasadmin@csprod1 ~]$ su - When prompted enter the root user’s password: nasadmin [root@csprod1 ~]# dmesg | grep sd This command will search device message and provide an indication of the added storage device address. We need to create a partition on the newly added disk. To do this, enter the following command: [root@csprod1 ~]# fdisk /dev/sdb


Page 35 of 146

The device number/letter (/dev/sd#) will change based on the number of attached “disks” At the prompt; Command (m for help): Type “n” for new partition Enter “p” for primary partition (without the quotes) Enter “1” for partition number Take the defaults for First Cylinder and Last Cylinder Command (m for help): w (write partition table to disk) The next command assumes the existence of the directory /mount. If /mount doesn’t exist issue the following command first; mkdir /mount

[root@csprod1 ~]# mkdir /mount/<insert your desired name here> i.e., mkdir /mount/csprod1-disk or mkdir /mount/sdb1

[root@csprod1 ~]# mkfs.ext3 /dev/sdb1 This process will take a few minutes to complete. The operating system is formatting the disk Remember this will change based on your device name. In this case it is /dev/sdb1 Repeat the previous three steps for each new disk added to the virtual machine. Subsequent device addressed would be /dev/sdc, /dev/sdd, etc.


Page 36 of 146

Edit the fstab file: [root@csprod1 ~]# vi /etc/fstab Add the following line to the bottom of the existing file and save the file (<esc> :wq!) /dev/sdb1 /mount/sdb1 auto defaults 0 0 (these are zeros) These may also be added for other drives added to the virtual machine. These are not needed if you only added one additional disk. Furthermore, only add one entry per added disk /dev/sdc1 /mount/sdc1 auto defaults 0 0 /dev/sdd1 /mount/sdd1 auto defaults 0 0

[root@csprod1 ~]# mount –a

[root@csprod1 ~]# df –h (verify the newly mounted file system is listed)

[root@csprod1 ~]# export NAS_DB=/nas


Page 37 of 146

Step 3: Adding “Physical” Disk to the Celerra VSA


[root@csprod1 ~]# export NAS_DB=/nas

[root@csprod1 ~]# /nas/sbin/setup_clariion –init The initial output from this command presents the serial number of the virtual CLARiiON backend storage array. Please note this number: The format starts with BB Note your specific serial number here: _____________________________

Select “q” to quit the setup_clariion command Because the Celerra VSA was created from another virtual device we need to ensure the serial number and the associated log .xml files match this number. To accomplish this execute the following commands: Note the BBxxx…xxx.xml file used in the example below should match that of the serial number just noted above.

� [root@csprod1 ~]# cp /nas/log/backend_status.BBxxxxxxxxxxxx.xml /nas/log/backend_status.<INSERT_YOUR_SERIAL_HERE>.xml

� [root@csprod1 ~]# cp /nas/log/backend_resume.BBxxxxxxxxxxxx.xml /nas/log/backend_resume.<INSERT_YOUR_SERIAL_HERE>.xml

If the previous two copy commands are not executed the following steps will fail.

Tip: Use the up arrow to load the second command from the CLI history and just change the words “status” to “resume”. Of course leave the quotes out.


Page 38 of 146

[root@csprod1 ~]# /nas/sbin/setup_clariion –init Notice the initial configuration defaults include two virtualized disk storage shelves. This graphic shows a third as well. I previously configured the new shelf but forgot to take the screen shot so we actually see three here.

• Select Option “1”. Add New Enclosure • Accept defaults for Enclosure [2_0], Physical Disk Size [146] and Physical Disk Type [FC] • Select Option “3”, Continue to Diskgroup Template Menu • Select Option “1”, CX_Standard_Raid_5 • Type “yes” to continue when asked: Do you want to continue and configure as shown? • When prompted to enter pathname for disk storage location enter the mount point you

defined earlier: i.e., /mount/sdb1 • Next prompt, Select “1. Create LUNs using available storage size in this path (xx MB in

total)” • Type “no” when asked about zeroing all LUNS

Note: there is a slight performance benefit when zeroing the LUNS up front but it takes a really long time to complete.

The Celerra VSA will now create the enclosures, hot spares, LUNs and disk groups

• Reboot the Celerra VSA (reboot –n)


Page 39 of 146

Step 4: Configure the Celerra VSA to use the new storage


Login to the Control Station (eth2 IP Address); user: nasadmin password: nasadmin Remember to wait for the Celerra VSA service to finish loading. To check, use the following command:

[nasadmin@csprod1 ~]$ /nas/sbin/getreason

[nasadmin@csprod1 ~]$ server_devconfig server_2 –p –s –a You will notice a number of LUNs where the val= -99: This indicates an unsigned LUN

[nasadmin@csprod1 ~]$ server_devconfig server_2 –c –s –a

This may take several minutes

[nasadmin@csprod1 ~]$ server_devconfig server_2 –p –s –a

The new LUNs are now ready for use. We see these new LUNs as we configure file systems on top of these in a few moments. [nasadmin@csprod1 ~]$ logout


Page 40 of 146

Section 3: Review At this point you should have two fully configured Celerra VSAs installed and configured with additional disk, iSCSI and NFS LUNs. You may recall from the opening statements that we would not walk thru DR site Celerra VSA implementation. The DR Celerra VSA is a mirror image of the primary site Celerra. The only differences relate to the networking configuration. Using the previous steps in Section1 and Section 2 and replacing all references to csprod1 with csdr1 you will successfully configure the DR Celerra VSA. Remember the DR Celerra VSA is installed on the ESX host designated for DR in your test environment. In the next section, Configuring Celerra Replication, we will connect the two Celerra VSAs and replicate LUNs between them. Once we complete the replication configuration we will install and configure Site Recovery Manager, setup Protection and Recovery Groups and exercise the tool in test and failover scenarios in Section 5.

If you have not already done so, please setup, install and configure the Celerra VSA for the

recovery site.

Proceed to Section 4: Configuring Celerra Replication when complete


Page 41 of 146

Section 4: Configuring Celerra Replication

Celerra Replicator is a very sophisticated remote replication facility. It supports the following features: (this is not an all inclusive list)

• 1:n and n:1 replication fan-in/fan-out replication relationships

• Support cascading topologies (i.e. site one replicates to site two with a given frequency, and then site 2 replicates the data from site 1 to site 3 at a different frequency)

• It has sophisticated QoS mechanisms (i.e. you can setup different bandwidth use for different parts of the day)

• You can replicate all sorts of configurations - CIFS/NFS/iSCSI.

• Full support for thin provisioning at the source or the target • And of course - most interestingly of all - it's fully integrated with VMware Site Recovery

Manager so you can use this functionality to build your own SRM test bed. Site Recovery Manager (SRM) doesn't support NFS yet, so this section is focused on iSCSI - but it's very easy to see how you would do CIFS/NFS by selecting the appropriate radio button during configuration. Contained within this section are the following items:

• Configuring NTP • Correcting the Celerra VSA Replication Database

The ID associated with the Replication Database needs to be unique. As a result of the import process the database still maintains the original ID it had prior to cloning. Therefore, two Celerra VSAs replicating between each other would potentially have the same ID which will inhibit replication setup. The process of correcting/changing the ID will avoid this problem

• Configuring an iSCSI LUN that will be used as a replication target • Configuring Celerra Replicator

My configuration was completed in a local datacenter. All connections and communications where limited to that location. If you want to perform these steps over your WAN or to a network protected by a firewall you should be aware of the following:

• Port 8888 is used by Celerra Replicator for transferring data between Celerra VSAs • The HTTPS connection between the source and destination Data Movers take place on port 5085 • The HTTPS connection between the Control Station on the source site and the Control Station on the destination site happens on port 443


Page 42 of 146

Step 1: Configuring NTP (Network Time Protocol)

Celerra Replication requires the system time on the source and target devices to be within 10 minutes of each other. If not replication may fail in addition to issues with authentication and security.

From the Celerra Manager GUI

(Using the Web Management interface: https://csprod1) Login as root PW: nasadmin Click “Control Station Properties” tab Modify the Current Date and Time, Current Time zone and NTP Servers: fields accordingly Note: Changing the Timezone may require a reboot of the Celerra VSA


Page 43 of 146

Open the Data Movers Folder and Right Click, Properties on server_2 Verify the date and time are in sync with the control station If the time skew is too great to sync with the NTP server you specified in the GUI, follow this sequence:

o Login to the Celerra VSA Console as root o [root@csprod1 ~]# date (verify current date and time) o [root@csprod1 ~]# pgrep ntpd (ensure the NTP daemon is running) o [root@csprod1 ~]# chkconfig ntpd on (ensures the NTP starts with the VSA) o [root@csprod1 ~]# service ntpd stop o [root@csprod1 ~]# ntpdate –u <NTP server IP address>

o Here are a couple of Internet based NTP server addresses for reference:

� 129.6.15.28 (time-a.nist.gov) � 129.6.15.29 (time-b.nist.gov) � 128.138.140.44 (utcnist.colorado.edu)

o [root@csprod1 ~]# service ntpd start o [root@csprod1 ~]# date (verify current date and time are correct) o [root@csprod1 ~]# logout

• Now perform the same process for the data mover

o Login to the Celerra VSA Console as nasadmin PW: nasadmin o [nasadmin@csprod1 ~]$ server_date server_2 timesvc delete ntp o [nasadmin@csprod1 ~]$ server_date server_2 timesvc start <NTP server IP

address> o [nasadmin@csprod1 ~]$ server_date server_2 o [nasadmin@csprod1 ~]$ logout

If the times are still out of sync after following these steps reboot the entire Celerra VSA

Repeat these steps for the recovery site Celerra VSA (csdr1)


Page 44 of 146

Step 2: Correcting the Celerra VSA Replication Database

The Celerra VSA has several small databases for the maintenance of critical information. As a result of the cloning process the replication database has the wrong serial number. The following process will correct this issue.

From the Celerra VSA Console

Login as nasadmin: PW: nasadmin [nasadmin@csprod1 ~]$ nas_cel –list (note the CMU is incorrect) [nasadmin@csprod1 ~]$ su (This is not a typo. The command here is su not su -) Password: nasadmin [root@csprod1 ~]# cd /nas/dos/slot_2 [root@csprod1 slot_2]# vi boot.cfg Scroll to bottom of the file and comment the dpinit line (comment character = #) Use the Insert Command (i) from the beginning of the dpinit line and type the hash (#) key. Save the file: (<esc> :wq!)


Page 45 of 146

• [root@csprod1 slot_2]# cd /nas/server/slot_2/ (notice this is a different directory off of /nas)

• [root@csprod1 slot_2]# vi eof Scroll to middle of the file and comment the dpinit line (comment character = #) Use the Insert Command (I) from the beginning of the dpinit line and type the hash (#) key. Save the file: (<esc> :wq!)

• [root@csprod1 slot_2]# server_cpu server_2 –reboot now

• [root@csprod1 slot_2]# /nas/sbin/getreason (will delver status on the data mover reboot)

Wait for slot_2 to be in a “contacted” state


Page 46 of 146

• [root@csprod1 slot_2]# server_dbms server_2 –db –delete icon_db (deletes the existing replication database)

• [root@csprod1 slot_2]# .server_config server_2 –v “dpinit” (creates a fresh replication database – Don’t forget the “.” at the beginning of this command. It is not a typo)

• [root@csprod1 slot_2]# nas_cel –list

• [root@csprod1 slot_2]# nas_cel –update id=0 (zero) (The CMU now correctly correlates to the Celerra VSA serial number)


Page 47 of 146

• [root@csprod1 ~]# cd /nas/dos/slot_2 • [root@csprod1 slot_2]# vi boot.cfg

Scroll to bottom of the file and uncomment the dpinit line (remove the #) Save the file: (<esc> :wq!)

• [root@csprod1 slot_2]# cd /nas/server/slot_2/ • [root@csprod1 slot_2]# vi eof

Scroll to middle of the file and comment the dpinit line (remove the #) Save the file: (<esc> :wq!)


Page 48 of 146

Step 3: Configuring Replication Using the Celerra Manager GUI

Replication involves a source and a target - if you're replicating file systems, then they are file systems. If you are replicating LUNs then they are LUNs. This step shows how to configure an iSCSI LUN that will be a replication target. To be a replication target, it needs to be the same size as the source, it needs to be configured as “read only”, and the file system has to be bigger than the LUN itself. Therefore, you need a bit of "space reservation". This "space reservation" is common to many storage use cases involving snapshots. One nice thing is that EMC has always philosophically chosen that an "out of space condition" should cause snap/replica LUNs to fail, not production LUNs. This step assumes the existence of a second Celerra VSA configured as per the contents of this document. That said, on the target Celerra VSA, there needs to be a LUN designated as a read-only LUN that the primary Celerra VSA will use as a replication target. This will be created as part of the configuration process. I was just pointing out the face that it needs to be there as some point. These next several pages will walk thru the GUI version of all these steps. The process is a little slower than the CLI but it combines several sub-steps in a single Wizard interface. If you want to see the CLI version of these steps please refer to Appendix A: Configuring the Replication Target (Command Line Interface).


These steps will configure the Celerra VSAs with the following components:

� iSCSI Target � iSCSI LUN � Replication Interconnects � Replication


Page 49 of 146

Create file system for use with the iSCSI LUN

Click on “Wizards”, “New File System” Best Practice: The size of the file system should be at least two times the size of the source LUN you will be replicating. This will accommodate the creation of Snap LUNs during SRM testing and failover. If not you will run into issues including replication failure.

Click “Next” to Select server_2 (default), Click “Next” to Select Storage Pool (default)


Page 50 of 146

Select the Storage Pool capable of providing capacity for the iSCSI LUN. In my case, I am using clar_r5_economy. Click “Next”

Enter a name for the file system. I used iscsi_fs1_replicated as I plan to replicate this LUN to csdr1. Enter the size of the LUN in MB (1024MB = 1GB). Ensure the Slice Volumes check box is selected. Click “Next”


Page 51 of 146

For the purposes of this exercise, take the defaults on the Enable Auto Extension Panel and the Default Quota Settings Panel by clicking “Next” Click “Finish” to create the file system:


Page 52 of 146

When completed. The Panel will look similar to this one. Click “Close” Repeat this process for csdr1. This will ensure the Celerra VSAs are configured for bi-directional replication which makes the process of fail back and reconfiguration that much easier. If you choose not to do this than proceed to the next step. One you have completed the file system creation on csdr1 we’ll add an iSCSI LUN and iSCSI target on csprod1. We’ll repeat the process on csdr1


Page 53 of 146

Create the iSCSI target and LUN(s)

Click “Wizards”, Click “New iSCSI Lun”


Page 54 of 146

During this step the Create iSCSI LUN wizard will create the iSCSI target and the iSCSI LUN associated to said target. Click “Next” to select server_2 (default), on the Select/Create Target panel, Click “Create Target” Enter a Target Alias Name (I am using csprod1_rep_target1), Ensure the Auto Generate Target Qualifier Name check box is checked. Click “Next”


Page 55 of 146

Select the iSCSI interface created earlier (cge1), Click “Add”, Click “Next” Click “Submit”, Click “Next”


Page 56 of 146

Select the iSCSI target just created, Click “Next” On the Select/Create File System panel; select the file system created for replication earlier. Click “Next”


Page 57 of 146

Enter the LUN information. Click “Next” Note: When configuring this step on csdr1 the LUN must be defined as read-only Here is the same screen for csdr1. Notice the LUN sizes are identical.


Page 58 of 146

Add the iSCSI initiators for the ESX servers that will access the replicated LUN Enable Multiple Access if more than one ESX will access this LUN Click “Add New”, Add the IQN number from your “production” ESX server, Click “OK” Repeat this for each ESX server accessing this LUN When completed, Click “Next” on the main Main LUN Masking panel Click “Next” on the CHAP Access (Optional) Panel


Page 59 of 146

Assuming you have not added any iSCSI LUNs to this Celerra VSA you will be prompted with the following panel: Ensure the iSCSI service check box is selected, Click “Next” Review the Summary in the Overview / Results panel, Click “Finish” Click “Close” once the task completes

Of course this needs to be completed on csdr1.

When completed move on to Configuring Replication


Page 60 of 146

Configuring Replication

Select “New Replication” from the “Wizards” folder


Page 61 of 146

All of our conversation thus far has centered on iSCSI. To that, we will select the iSCSI LUN radio button and Click “Next” Click “New Destination”


Page 62 of 146

Enter the TCP/IP address and the user credentials of csdr1 The username is nasadmin; password is nasadmin Click “Next”

Enter a Celerra Network Server Name: Your choice Enter a passphrase: Your choice Click “Next”


Page 63 of 146

Click “Submit” and view the following screen… Acknowledge the successful completion of Control Station configuration, Click “Next”


Page 64 of 146

Select the Celerra VSA Destination just created. In my case it network name was cns. Click “New Interconnect”


Page 65 of 146

Name the data mover interconnect. I used csprod1-to-csdr1, Click “Next” The next panel is asking you to define an interconnect for the peer data mover. This is the interconnect the allows for reverse communication between the Celerra VSAs I used csdr1-to-csprod


Page 66 of 146

Take the defaults on the next screen, Click “Next”

Ensure the time on the Celerra VSA control stations is within 10 minutes of each other.

Double-check prior to clicking submit.

Are you sure? Click “Submit”


Page 67 of 146

Review the results of the changes and Click “Next” Select the newly created Data Mover Interconnect, Click “Next”


Page 68 of 146

Select the iSCSI interfaces for each of you Celerra VSAs, Click “Next” The next couple of items will name the replication session and select the specific LUNs involved in this session. Since this session will facilitate replication for VMware Site Recovery Manager, I named the session “srm_replication”. Notice the iSCSI target and iSCSI LUN selections. These were there by default. Double-check the entries for accuracy prior to clicking “Next”


Page 69 of 146

If the Target Destination information is correct Click “Next”. Otherwise select the correct parameters and then Click “Next” Take the defaults on the Update Policy Panel. Click “Next”


Page 70 of 146

Click “Finish” Review the results, Click “Close”


Page 71 of 146

Once completed. Navigate back to the Celerra Manager Main Panel. Click on the “Replications” folder. Do this from both sides of the replication relationship. csprod: csdr1:


Page 72 of 146

Step 4: Preparing ESX Servers for iSCSI Targets and LUNs

From Virtual Center:

Click on the “Configuration” tab of the ESX hosts you want to configure. Click on “Networking”

Click on “Properties” for the vSwitch configured for iSCSI

Select vSwith, Click “Edit”


Page 73 of 146

Click on the “Security” tab

Change Promiscuous Mode to Accept

Note: This is NOT required for the Celerra Simulator as it will run either way. This is a best practice as it related to most simulators

Click “OK”, Click “Close” From the main panel: Click on the Security Profile. Ensure that the Software iSCSI Client is enabled


Page 74 of 146

Move to the Storage Adapters Tab and scroll to the iSCSI Software Adapter.

Note: Two items: 1) the iSCSI Initiator IQN, 2.) Targets: currently 0 (zero)

Click “Properties”, Click “Configure”, Select Enabled

Click “OK”

Click on the “Dynamic Discovery” tab, Click “Add”

Enter the TCP/IP address of the iSCSI target you created earlier. Ensure the port number is 3260 (default)

Click “OK” (This may take a moment or two...) Click “Close”


Page 75 of 146

When prompted to Rescan the host Click “Yes” Accept the defaults and Click “OK”


Page 76 of 146

Notice the new Target. Repeat this step for each ESX server masked to the replicated iSCSI Target To this point all we have done is allow the ESX servers to see the iSCSI LUNs. We must

establish a VMFS file system so they are usable within the Virtual Infrastructure.

The steps on the proceeding two pages are for

the “Production” side ESX servers only!


Page 77 of 146

Click on the “Configuration” tab, Select Storage, Click “Add Storage”

Select the “Disk/LUN” radio button, Click “Next”

Select the LUN just added

Click “Next”


Page 78 of 146

Review the current disk layout and notice the message: “The hard disk is blank.” Click “Next” Enter a Name in the “Datastore Name” field:

Use something obvious like “csprod1_iscsi_replicated_lun10” Click “Next” Review VMFS Parameters Click “Next” Click “Finish” Verify the LUN was added to the ESX host

Important: Add or migrate at least one virtual machine you want to protect with SRM to the replicated datastore. If not the SRM configuration will not display any summary information in the Display Replicated Datastores later in this configuration. Please add at least one VM to this new datastore. Also only one LUN is replicating between the Celerra VSAs. You can create additional LUNS following the process above.


Page 79 of 146

Section 5: Site Recovery Manager Installation In order to complete the implementation and configuration of SRM we need to ensure the following components are in place.

• ESX Server managed by Virtual Center for each location Of course these locations can be in the same data center or home office or where ever you want them. The key is a working TCP/IP connection between the two for the purposes of TCP/IP based Celerra Replication failover and SRM server connectivity.

ESX Server version must be 3.5 Update 2 because we are using SRM 1.0 Update 1 Virtual Center must be version 2.5 Update 3

• You will need to following software components:

VMware SRM 1.0 Update 1 Microsoft SQL Server Management Studio Express (SQLServer2005_SSMSEE) If you haven’t already done so, download the SSMSEE and install it on the Virtual Center Server. It is a straight-forward Next, Next, Next, Finish type of installation EMC Celerra Storage Resource Adapter (SRA)


Page 80 of 146

Step 1: Site Recovery Manager Database Connectivity

In this step we will perform a number of steps required prior to the actual installation of the SRM server instance on the Virtual Center Server. Specifically we will

o Create a new Active Directory user for use with the SRM database o Create the SRM data base using the SSMSEE application o Assign the new user to the database as the db_owner o Setup and test an ODBC connection to the new database

Within your Active Directory create a new user: I used srmprotected-db. Notice I selected the “User cannot change password” and “Password never expires” options. Click “Create” Once created launch the SSMSEE application: Start> all Programs> Microsoft SQL Server 2005 > SQL Server Management Studio Express The application may or may not display the following panel:


Page 81 of 146

From the SQL Server 2005 Connect to Server Panel, Click “Connect”

Make note of the Server Name: in my case VMWARE-VC\SQLEXP_VIM. You will need this information when you create an ODBC connection to the SRM database. For some reason it will not be on the drop down list so I copied it to my clipboard. In the main panel Right Click Databases and Select New Database. Name the Database as you wish. I used srmprotected-db. Click “OK”


Page 82 of 146

Create a new login: Right Click on the “Logins” Folder under “Security”

Enter the login ID you created for this purpose. Enter the name in domain\user format: i.e., BAKER-ISS\srmprotected-db Select the database you created from the Default Database Drop Down


Page 83 of 146

Next select “User Mappings” from the “Select a Page” panel. Select the database you user will be mapped to (i.e., srmprotected-db), Select db_owner and leave public checked. Click “OK” Now that the database has been created and the user mapped to it, we need to create an ODBC connection to the new database. Navigate to the Control Panel on the Virtual Center Server. Double Click on the Data Sources (ODBC)

shortcut. Click “Add”

(If the ODBC setup icon is not in the Control Panel it will be in the Administrative Tools folder. You may need to enable this via the start menu properties.)


Page 84 of 146

Select “SQL Native Client”, Click “Finish” Name the ODBC Connection to the SRM database a name and enter the server name on the Server: field as noted above i.e., in my case VMWARE-VC\SQLEXP_VIM, Click “Next” Select the “With Integrated Windows authentication” radio button, Click “Next”


Page 85 of 146

Ensure “Change the default database to:” selection is checked Select the SRM database you created earlier from the dropdown box. Keep the rest of the selections at their defaults. Click “Next” On the next panel take the defaults and Click “Finish” Once completed you will be presented with the following panel: Click “Test Data Source”


Page 86 of 146

If everything was configured properly you should see the following panel (a small reward for the work so far!). Click”OK” When returned to the original panel Click “OK”


Page 87 of 146

Click OK again when presented with the ODBC Data Source Administrator Panel

Notice the addition of the VMware SRM Data Source

At this point you should configure SQL on the recovery site Virtual Center Server.


Page 88 of 146

Step 2: Copying Site Recovery Manager and the SRA

Copy the SRM and SRA installation media to your Virtual Center Desktop. The best was to accomplish this (if you are remote to the console) is to map a drive to the administrative share, C$, on the Virtual Center server. Start> Run> \\192.168.1.14\c$ Enter your credentials as required

Step 3: Installing Site Recovery Manager

Double Click on the VMware-srm-1.0.1-128004 installation icon Accept the ELUA and Select a Destination Folder, Fill in the appropriate information in the SRM registration panel.


Page 89 of 146

Acknowledge the Security Warning and accept the certificate’s thumbprint by Clicking “Yes” Select the “Automatically generate a certificate” radio button and Click “Next” Enter the appropriate (anything you like) information in the Organization and Organizational Unit fields and Click “Next” Fill in the requested information and Click Next. Do not modify the SOAP or HTTP ports


Page 90 of 146

Enter the database specifics in the next panel and Click “Next” Click “Install” The installation will proceed without any additional input. Once completed Click “Finish” and proceed to the next step


Page 91 of 146

Step 4: Installing the Storage Replication Adapter

Double Click on the EMC_Celerra_Replicator_Adapter_for_VMware_Site_Recovery_Manager installation icon. Click “Next”, Accept the License, Click “Install”, Click “Finish” From the services MMC restart the VMware Site Recovery Manager Service No need for screen shots! (yeah!)

Step 5: Installing the SRM Plug-in for Virtual Center

If not already running, launch the VI Client and attach to your Virtual Center Server. Click “Plugins” from the Menu Bar


Page 92 of 146

Click the “Download and Install” button for Site Recovery Manager At this point a separate installer will launch to install the plug-in. This is a standard windows install similar to the installs completed in Steps 2 and 3 of this section. Click “Next”, Accept the license, “Install”, “Finish”


Page 93 of 146

Shortly after the completion of the add-in install, you will be taken back to the plug-in manager for Virtual Center. Notice the plug-in has been installed but still needs to be enabled. Click on the “Installed” tab and Click “Enabled” in the Site Recovery Manager panel, Click “OK” Click on the new “Site Recovery” Manager Icon


Page 94 of 146

This panel offers us the ability to start configuring the actual linkages between SRM servers and their associated storage devices. Prior to that however, the SRM installation needs to be completed on the recovery, disaster recovery site. Please complete Section 5 on the recovery site Virtual Center server before proceeding to Section 6.


Page 95 of 146

Section 6: Site Recovery Manager Configuration

Welcome Back! Hopefully the SRM implementation on the recovery site went well. At this point we have completed several tasks each a critical component in the grand scheme of SRM. Here however, is where the preverbal rubber hits the road. In this section we will configure each of the SRM servers to interact. We will also create protection and recovery groups and test the failover capability of the SRM servers. When we finished Section 5 we were left with the following screen shot. Notice the information in the Local Site panel. This information matches the information in the paired site providing you are looking at this screen from the perspective of the protected site. Of course this will change when you open this panel this from the recovery site SRM server. The Paired site will always reflect the “replication partner.” That said, if logged into the protected site you will see the recovery site information in the Paired Ssite panel. If you are looking at this from the Recovery site, the paired site will be that of the Protected site. Crystal clear? It will all make sense as we work through the next several steps. Click “Configure” in the “Protection Setup” panel


Page 96 of 146

Step 1: Configuring the Connection

All of these tasks are completed from the protection site unless otherwise indicated. Enter the TCP/IP Address of the remote (recovery) site Virtual Center Server, the Port number defaults to 80. Acknowledge the Certificate Panel, Click “OK”


Page 97 of 146

Enter the Administrative Credentials for the Virtual Center Server. You may be asked to authenticate to both the remote and the local servers. Once Authenticated the system will establish a reciprocal connection between the Virtual Center Servers. Acknowledge any Security Certificate warnings (you may not see any)

Click “Finish”


Page 98 of 146

Check the status of the connection. It should be “Connected”


Page 99 of 146

Step 2: Configuring the Array Managers

Now that the connection between the two sties has been established the arrays can be configured. You recall that we installed the EMC Celerra SRA and the Solution Enabler earlier. This step will not work if those components did not install properly. Click “Configure” next to Array Managers: SRM main panel Add the Protection Site Celerra VSAs, Click “Add”


Page 100 of 146

Enter the requested information in the “Add Array Manger” Panel. The display name should be the same as your protection site Celerra VSA. Manager Type is Celerra iSCSI Native, Control Station IP is the csprod1 eth2 IP address, User Name and Password are nasadmin. Click “Connect” Once added, the replicated LUNs on csprod1 should be displayed in the Protection Arrays Panel Click “OK”


Page 101 of 146

The Add Array Manager process will return to the Configure Array Manager main panel. Notice he Protection Arrays: panel. It lists the Celerra VSA as “NS-Simulator”, shows the Peer array at the remote site and the number of LUN(s) configured for replication Click “Next”, to configure the recovery side array, Click “Add”


Page 102 of 146

Enter the requested information in the “Add Array Manger” Panel. The display name should be the same as your protection site Celerra VSA. Manager Type is Celerra iSCSI Native, Control Station IP is the csdr1 eth2 IP address, User Name and Password are nasadmin. Click “Connect” Click”Next”


Page 103 of 146

Review the Replicated Datastores, Click “Finish”

Remember: If you did not add at least one virtual machine to the replicated datastore this panel will be blank… That is OK but don’t think something is wrong if the datastore summary is blank.


Page 104 of 146

Step 3: Configuring Inventory Mappings (Optional)

Moving forward… the Inventory Mappings need to be completed. This puts us one step away from configuring the Protection and Recovery Groups. Note that these mappings set the general defaults for virtual machine networks on the recovery site. These can be overridden during the process of creating protection groups. Click “Configure” next to the “Inventory Mappings:” entry in the “Protection Setup” Panel Use this panel to map source site resources to recovery site resources. These are the recovery site resources used during testing and failover. Specifically, you are mapping network, resource pools and data center locations.


Page 105 of 146

Example:


Page 106 of 146

Step 4: Configuring Protection Groups

There are only a few steps left prior to testing our SRM configuration. Specifically, we need to create protection and recovery Groups. Protections Groups as their name infers are created at the protection/”production” site and will define which virtual machines are protected via the SRM facility. Recovery Groups wrap policy and procedure around the protection groups and become the basis of the recovery plan. The recovery Groups are created on the recovery/DR site. Click “Create” next to “Protection Groups” in the “Protection Setup” Panel Give the Protection Groups a Name and Click “Next” The name should give some insight to the VMs that will be contained within.


Page 107 of 146

Select a Datastore Group and Click “Next” (as discovered by the Array Manager)

Select a recovery site datastore for the shadow/“placeholder” Virtual Machines, Click “Finish”


Page 108 of 146

We now see the created Protection Group


Page 109 of 146

Step 5: Configuring Recovery Groups

The final step in the configuration process centers on the Recovery Site and the configuration of Recovery Groups. The following steps will be executed from the recovery site Virtual Center server. Click “Create” in the “Recovery Setup” Panel Name the Recovery Plan – use something indicative of the Plan’s scope (entire site, windows machines, Linux machines, web servers, etc.), Click “Next”


Page 110 of 146

In the “Create Recovery Plan - Protection Groups” dialog select the Protection Group(s) created in the previous step. Click “Next” In the next panel, “Response Times”, leave the defaults and Click “Next”


Page 111 of 146

The “Configure Test Networks” dialog provides the ability to select a test network for use when testing your recovery plan. By default this is set to Auto. Auto will dynamically create a “bubble” network during a running test and tear it down when the test completes. Note: Any virtual machines dependant on non-protected machines running DNS, AD and DHCP may not be able to communicate with each other once on the bubble network. Make provisions for these services in the bubble or configure an isolated network with these services for testing purposes. Make the appropriate selections. Click “Next”


Page 112 of 146

It is likely, during a test, that you may need to free-up some of the available DR site resources (CPU and Memory) until the test cycle is completed. SRM provide this capability via the recovery plan. If you choose to, SRM will suspend selected virtual machines. Click the check box next to the VMs you wish to suspend. Click “Finish”


Page 113 of 146

Step 6: Testing (at long last…)

Providing everything to this point has been configured properly, we are ready to push the test button. Let’s get right to it. From the Recovery Site Virtual Center Server, Click the “Site Recovery” button on the ribbon menu. Expand the Recovery Plans, and Click on the Recovery Plan created earlier. Click the “Recovery Steps” tab.


Page 114 of 146

The steps listed here have been automatically created via the Create Recovery Plan. Take a few moments to review the steps in the recovery plan. You have many options to tweak the plan but that is slightly outside the scope of this document. When you are ready to proceed… Click on the Green “Test”’ Button (The Red “Run” Button is reserved for the actual failover event!) Acknowledge the following dialog


Page 115 of 146

SRM will start by configuring the storage components. Without a long dissertation, SRM will ready the remote storage and present a writeable snapshot to the ESX server in the DR site. Notice the status in the recent task panel at the bottom of the main Virtual Center window.


Page 116 of 146

The next step in the process will suspend any Virtual Machines (if they where selected to do so during the creation of the Recover Plan)


Page 117 of 146

Next the plan starts recovering, in order of priority, the virtual machines (these are the place holder VMs created during the Protection Groups configuration)


Page 118 of 146

Once the VM have been recovered and all of the recovery plan steps have been completed, the plan will pause for user input. During this pause the virtual machines are running and “connected” to the network previously defined in the recovery plan. Remember, by default this network is referred to as the “bubble” network and maintains no connectivity to other physical networks and does not provide for network-based services such as AD, DNS or DHCP. This is unless you have made provisions ahead of time. Testing complete? Great! Click “Continue”

Site Recovery Manager will clean up the storage, place the VMs back into a powered off state in the placeholder datastore and return to the Recovery Plan main panel. You have just executed a successful Site Recovery Manager test using Celerra VSAs. How did it go? It is a lot of work but this document in conjunction with the Celerra VSAs allow for the creation of an environment fully capable of testing SRM and a very real manner. Please let me know if you found this document useful. I would also welcome feedback on the accuracy of the content. Thanks for taking time to look at the Celerra VSA and testing VMware Site Recovery Manager in this manner.


Page 119 of 146

Appendix A: Configuring the Replication Target (Command Line Interface)

Replication involves a source and a target - if you're replicating file systems, then they are file systems. If you are replicating LUNs then they are LUNs. This step shows how to configure an iSCSI LUN that will be the replication target. To be a replication target, it needs to have the same size as the source, it needs to be configured to be read only, and the file system has to be bigger than the LUN itself. Therefore, you need a bit of "space reservation". This "space reservation" is common to many storage use cases involving snapshots. One nice thing is that EMC has always philosophically chosen that an "out of space condition" should cause snaps/replicas to fail, not production. This step assumes the existence of a second Celerra VSA configured as per the contents of this document. That said, on the target VSA, there needs to be a LUN designated as a read-only LUN that the primary Celerra VSA will use as a replication target. These next several pages will walk thru the CLI version of all these steps. The process is streamlined and much faster that using the Celerra Manager GUI. If you want to see the GUI version of these steps please refer to Section 4, Step 3 Configuring the Replication Target via the Celerra Manager GUI.


These commands will configure the Celerra VSAs with the following components:

� iSCSI Target � iSCSI LUN � Replication Interconnects � Replication

Create and Mount a file system for use with the iSCSI LUN

csprod1:

[nasadmin@csprod1 ~]$ nas_fs -name fs1-csprod1 -type uxfs -create size=45G pool=clar_r5_performance -option slice=y

[nasadmin@csprod1 ~]$ server_mount server_2 fs1-csprod1 /nas/fs1


Page 120 of 146

csdr1:

[nasadmin@csdr1 ~]$ nas_fs -name fs1-csdr1 -type uxfs -create size=45G pool=clar_r5_performance -option slice=y

[nasadmin@csdr1 ~]$ server_mount server_2 fs1-csdr1 /nas/fs2

Start the iSCSI service

csprod1:

[nasadmin@csprod1 ~]$ server_iscsi server_2 -service -start

csdr1:

[nasadmin@csdr1 ~]$ server_iscsi server_2 -service -start

Create an iSCSI target that will be used to serve LUNs to each Celerra VSA

csprod1:

[nasadmin@csprod1 ~]$ server_iscsi server_2 -target -alias target1-csprod1 -create 1:np=172.16.1.150 (This is the iSCSI interface on you created in Section 1-eth2)

csdr1:

[nasadmin@csdr1 ~]$ server_iscsi server_2 -target -alias target1-csdr1 -create 1:np=172.16.1.160 (This is the iSCSI interface on you created in Section 1-eth2)

Create an iSCSI LUN Note that the LUN on the secondary needs to be set as “read-only”

csprod1: (Primary)

[nasadmin@csprod1 ~]$ server_iscsi server_2 -lun -number 1 -create target1-csprod1 -size 10G -fs fs1-csprod1

csdr1: (Secondary)

[nasadmin@csdr1 ~]$ server_iscsi server_2 -lun -number 1 -create target1-csdr1 -size 10G -fs fs1-csdr1 -readonly yes


Page 121 of 146

Note: In order to setup reverse replication, from secondary to primary, a second set of LUNs needs to be created. This time the target replication LUN on the Primary must be set to read-only. The source LUN on the secondary is configured normally as a read/write LUN.

Set the iSCSI LUN masking for each ESX host

csprod1:

[nasadmin@csprod1 ~]$ server_iscsi server_2 -mask -set target1-csprod1 -initiator iqn.1998-01.com.vmware:esx2-29d419e2 -grant 1

csdr1:

[nasadmin@csdr1 ~]$ server_iscsi server_2 -mask -set target1-csdr1 -initiator iqn.1998-01.com.vmware:esx3-017f4fc5 -grant 1

Note: The IQN number is associated with the software iSCSI interface on the ESX host. You will need to retrieve this from the ESX server via Virtual Center as noted earlier.


Page 122 of 146

Configuring Replication

Set up mutual trusts between Celerra VSAs This trust needs to be setup in both directions; primary to secondary and secondary to primary. The passphrase must be the same in both cases:

csprod1:

[nasadmin@csprod1 ~]$ nas_cel -create cel1 -ip 192.168.1.160 –passphrase nasadmin

csdr1:

[nasadmin@csdr1 ~]$ nas_cel -create cel1 -ip 192.168.1.150 –passphrase nasadmin

Configure the Data Movers to converse and share replicated data

csprod1:

[nasadmin@csprod1 ~]$ nas_cel -interconnect -create cel1-to-cel2 -source_server server_2 -destination_system cel2 -destination_server server_2 -source_interfaces ip=172.16.1.150 -destination_interfaces ip=172.16.1.160

csdr1:

[nasadmin@csdr1 ~]$ nas_cel -interconnect -create cel2-to-cel1 -source_server server_2 -destination_system cel1 -destination_server server_2 -source_interfaces ip=172.16.1.160 –destination_interfaces ip=172.16.1.150

Note: Perform this interconnect from both directions; primary to secondary and secondary to primary. If you only set it from one side replication will only work in

one direction.

The interconnect must be the same in both cases.


Page 123 of 146

Establish Replication from the primary Celerra VSA to the secondary Celerra VSA

csprod1:

[nasadmin@csprod1 ~]$ server_iscsi server_2 –target –l This will provide the Celerra VSA IQN number for the next command

csdr1:

[nasadmin@csdr1 ~]$ server_iscsi server_2 –target –l This will provide the Celerra VSA IQN number for the next command csprod1 (only):

[nasadmin@csprod1 ~]$ nas_replicate -create rep1 -source -lun 1 -target iqn.1992-05.com.emc:bb0050568a59e40000-3 -destination -lun 1 -target iqn.1992-05.com.emc:bb0050569a3d070000-3 -interconnect cel1-to-cel2 -source_interface ip=172.16.1.150 -destination_interface ip=172.16.1.160 -overwrite_destination

csprod1: [nasadmin@csprod1 ~]$ nas_replicate -l

csdr1: [nasadmin@csdr1 ~]$ nas_replicate –l


Page 124 of 146

Appendix B: iSCSI and NFS Discrete LUN Creation & Assignment

Some people may have downloaded this document hoping for a “I just need to add an iSCSI LUN or NFS export to my ESX server from my existing Celerra VSA or physical Celerra” type of document. This appendix discussed just that. It falls outside the scope of the SRM discussion earlier but nevertheless provides insight to some of the basic functions of the Celerra VSA. This appendix assumes that all LUNs and file systems are created for non-replicated uses. This appendix also assumes that you have completed all of the steps in Section 1 of this document. If you need to create LUNs greater than 20GB in capacity you may also want to complete Section 2.

Step 1: Creating iSCSI Targets

Login to your Celerra VSA as nasadmin, Password: nasadmin Navigate to the Wizards Folder

Click “New iSCSI Target”, Click “Next” to configure an iSCSI target on server_2

Enter the Target Alias Name: csprod1_target1 and select Auto Generate Target Qualified Name Note: This will generate a Unique IQN number for this target.


Page 125 of 146

Assign the target to cge1 (your iSCSI Interface), Click “Next”

Click “Finish”

This graphic represents the complete target configuration as viewed from the iSCSI folder, Targets tab.


Page 126 of 146

Step 2: Configuring iSCSI LUNs and presenting them to ESX as VMFS Datastores

You may not need to complete this step (outlined on pages 118-121) if you already configured your environment for iSCSI replication. These steps have already been completed. If you did not set up replication please continue.


Click on the “Configuration” tab of the ESX hosts you want to configure. Click on “Networking”

Click on “Properties” for the vSwitch configured for iSCSI


Page 127 of 146

Select vSwith, Click “Edit”

Click on the “Security” tab Change Promiscuous Mode to Accept

Note: This is NOT required for the Celerra Simulator as it will run either way. This is a best practice as it related to most simulators

Click “OK”, Click “Close”


Page 128 of 146

From the main panel: Click on the Security Profile. Ensure that the Software iSCSI Client is enabled


Page 129 of 146

Move to the Storage Adapters Tab and scroll to the iSCSI Software Adapter.

Note: Two items: 1) the iSCSI Initiator IQN, 2.) Targets: currently 0 (zero)

Click “Properties”, Click “Configure”, Select Enabled Copy the iSCSI Name field to the clipboard. You may want to write it down as well. Just in case you copy something else to the clipboard.

Click “OK” Click on the “Dynamic Discovery” tab, Click “Add”

Enter the TCP/IP address of the iSCSI target you created earlier. Ensure the port number is 3260

Click “OK” (This may take a moment or two...) Click “Close” When prompted to Rescan the host Click “No” Repeat this step for each ESX host that will attach to this target.


Page 130 of 146


(Using the Web Management interface: https://csprod1) Navigate to the Wizards Folder

Click “New iSCSI LUN”, Click “Next” to configure an iSCSI target on server_2


Page 131 of 146

Pick the iSCSI target created earlier (csprod1_target1), Click “Next”

Click “Create File System” (remember, like all filers, iSCSI LUNs are files in file systems)

Select the Storage Pool radio button. On the next screen select the specific Storage Pool you want to use. Two pools are defined by default; one for Performance and one for Economy.


Page 132 of 146

Click “Next”, Enter the name for the new file system: iscsifs1

Enter the size of the new file system: enter an appropriate value in MB Click “Next”, Click “Submit” wait for the job to complete, review the summary results Click “Next”


Page 133 of 146

Pick the file system just created from the pick list windows (it may be highlighted)

Click “Next”

There are several options presented on this page: execute the options required for your specific use case. For the purpose of this document we'll focus on creating a single LUN

Enter the Number: (This is the LUN number assigned to the newly created LUN) i.e., 10

Enter the size of the new LUN


Page 134 of 146

Click “Next”

Click “Enable Multiple Access” check box so it is selected Click “Add New” Paste the ESX Host IQN number you copied earlier.

Repeat for all ESX Hosts, Click “Next”, Click “Next” on the CHAP Access Panel

Click “iSCSI Service” check box so it is selected. Click “Next” Click “Finish” Wait for the job to process, review the summary for errors

Click “Close”


Page 135 of 146


Select one of the ESX host(s) just masked to the iSCSI target.

Click on the “Configuration” tab, Select Storage Adapters. Select the iSCSI software adapter Click “Rescan”, Accept the defaults and Click “OK”

Notice the “Targets” count has incremented by the number of LUNs associated with the target and masked to this server. In this case, maybe one or two LUNs..

Perform these steps for all ESX hosts in the cluster.

To this point all we have done is allow the ESX servers to see the iSCSI LUNs. We must establish a VMFS file system so they are usable within the Virtual Infrastructure.


Page 136 of 146

Click on the “Configuration” tab, Select Storage, Click Add Storage

Select the “Disk/LUN” radio button, Click “Next” Select the LUN just added (i.e., LUN10 which may be display as vmhba32:0:10) Click “Next” Review the current disk layout and notice the message: “The hard disk is blank.” Click “Next” Enter a Name in the “Datastore Name” field: Use something obvious like “csprod1_iscsi_lun10” Click “Next” Review VMFS Parameters Click “Next” Click “Finish” Verify the LUN was added to the ESX host.


Page 137 of 146

Step 3: Configuring NFS LUNs and presenting them to ESX as NFS Datastores

Note: This step is not required for SRM but is included here for completeness of configuration and protocol options.


Click Security Profile for the ESX host(s). Enable the NFS Client


(Using the Web Management interface: https://csprod1) Click “Wizards”. Click “New File System”

Select Data Mover: server_2, Click “Next”


Page 138 of 146

Select the Storage Pool radio button. On the next screen select the specific Storage Pool you want to use. Two pools are defined by default; one for Performance and one for Economy.

Click “Next”


Page 139 of 146

Enter the name for the new file system: nfsfs1

Enter the size of the new file system: enter an appropriate value in MB Click “Next”, “Next”, “Next” Click “Close”


Page 140 of 146

Back on the main panel:

Click “NFS Exports”, Click “New”


Page 141 of 146

Select the appropriate file system; i.e., nfsfs1 (/nfsfs1) Enter TCP/IP Addresses for each ESX host that will access this export in the “Root Hosts:” field

Note: this field accepts wild cards and CIDR formatted entries Click “OK”

Celerra Manager will finish the task and present a list of the NFS exports as confirmation.


Page 142 of 146


Click on the “Configuration” tab, Select Storage, Click “Add Storage”

Select the “Network File System” radio button, Click “Next”


Page 143 of 146

In the Properties panel enter the Server: name or IP and the Folder: that server is exporting:

Example: Server: 192.168.1.247

(NOTE: this is a datamover (cge0) interface NOT the control station)

Folder: /nfsfs1 Enter a Name in the Datastore Name field: Use something obvious like csprod1_nfs1, Click “Next”

Review the properties, Click “Finish”

Verify the LUN was added to the ESX host.


Page 144 of 146

Appendix C: Basic VI Commands Credits to: Douglas (Doug) Palovick Homepage: Palovick.com email: [email protected]

What is VI and is it for you?

VI is an extremely powerful text editor (not to be confused with a word processor). Here you will find basic VI commands for use within the VI, or vim, editor.

Ok, now onto some VI basics…

Opening a file/starting vi

• To start vi, all you need to do is type "vi" (minus the quotes, always minus the quotes from here on through)

• Let’s say I wanted to open a file named foo.txt that resides in the same directory I am currently in. I would type "vi foo.txt"

• Now lets say i wanted to open inetd.conf which lives in the /etc directory. I would type "vi /etc/inetd.conf". The directory path to inetd.conf is always the same, no matter what directory you are in.

Quitting VI

• First let’s make sure you are in what is called "command mode". We do this by pressing the Esc key.

• Now we press " : ". You should see a little prompt at the bottom of your page. Now press "q" and hit enter.

• If it will not let you quit, it is because you have made some changes to the file and you must press ":", then "q!", then enter to force vi to quit (warning, this will disregard any changes you have currently made).

• If you would like to save the changes you have made within you vi session press ":", then "wq", now press enter. If vi ever complains that you're trying to make changes to a read-only file, try "w!" or "wq!".

• If you ever have any problems with doing this or other "command mode" commands hit the Esc key once or twice to make sure you are in command mode and try again.


Page 145 of 146

Basic command mode commands for inserting text

• First make sure you are in command mode by pressing the Esc key and then make sure you press ":" before you try any of the commands bellow. Please note that after you enter one of these commands in command mode you will be put into insert mode, which will enable you to enter new text into the file. In order to get back to command mode you will have to press the Esc key again.

c - Add changes to file. This is a commonly used command for editing a file with vi. When this option is invoked, you rewrite over the text you want to change by backing up over it using the del key. Simply re-type over the text you have backed up over. When you use the left arrow key to back up over text you can not rewrite over the text you backed up over, but just insert new text at the point where you stopped backing up. Make sense? If not, no worries, you'll catch on. a - Append text after the cursor. A - Append text at the end of the line. d - delete text i - Insert text before the cursor. I - Insert text at the beginning of the line. R - Allows you to overwrite over text.


Page 146 of 146

Appendix D: Troubleshooting Tips

HTTPS Issues To view whether the HTTPD daemons are enabled at the Control Station and to reenable them if necessary, type: $ ps -e|grep httpd If the HTTPD daemons are not running, restart the Celerra Manager by switching to root and typing: /nas/http/nas_ezadm/etc/script restart

Are all of the Celerra VSA processes running? To view whether the Celerra VSA daemons are enabled at the Control Station and to re-enable them, type: [nasadmin@csprod1 ~]$ ps -e|grep nas | awk ' { print $4 } ' | sort | uniq nas_boxmonitor nas_eventcollec nas_eventlog nas_mcd nas_replicate nas_watchdog The complete list of daemons is shown in the Output. The output list for the server might be different. If the daemons are not running, restart them by typing: /etc/rc.d/init.d/nas stop /etc/rc.d/init.d/nas start

Restart a data mover server_cpu server_2 -reboot -monitor now You can verify when the system is back online by using /nas/sbin/getreason Code 5 indicates the Data Mover or blade is available. You might also see the following codes as the Data Mover or blade restarts:

� 0 (reset) — Data Mover or blade performing a power on self-test (POST) � 1 (DOS booted) — Data Mover or blade booting to DOS � 3 (loaded) — operating system loaded and initializing � 4 (ready) — operating system initialized � 5 (contacted) — Data Mover or blade available

Documents

EMC Celerra VSA and VMware SRM Complete Setup and Configuration Guide Revision 1.0.1doc