SAN Implementation Workshop_StudentGuide

NetApp University

Student Guide

NETAPP UNIVERSITY

SAN Implementation Workshop Student Guide

Course Number: STRSW-ILT-SANIMP-R1 Catalog Number: STRSW-ILT-SANIMP-R1-SG

Revision: 2.0

NetApp University - Do Not Distribute

2 SAN Implementation Workshop: Welcome

© 2009 NetApp. This material is intended for training use only. Not authorized for reproduction purposes.

ATTENTION The information contained in this guide is intended for training use only. This guide contains information and activities that, while beneficial for the purposes of training in a closed, non-production environment, can result in downtime or other severe consequences and therefore are not intended as a reference guide. This guide is not a technical reference and should not, under any circumstances, be used in production environments. To obtain reference materials, please refer to the NetApp product documentation located at http://now.netapp.com/ for product information.

COPYRIGHT © 2009 NetApp. All rights reserved. Printed in the U.S.A. Specifications subject to change without notice.

No part of this book covered by copyright may be reproduced in any form or by any means—graphic, electronic, or mechanical, including photocopying, recording, taping, or storage in an electronic retrieval system—without prior written permission of the copyright owner.

NetApp reserves the right to change any products described herein at any time and without notice. NetApp assumes no responsibility or liability arising from the use of products or materials described herein, except as expressly agreed to in writing by NetApp. The use or purchase of this product or materials does not convey a license under any patent rights, trademark rights, or any other intellectual property rights of NetApp.

The product described in this manual may be protected by one or more U.S. patents, foreign patents, or pending applications.

RESTRICTED RIGHTS LEGEND Use, duplication, or disclosure by the government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.277-7103 (October 1988) and FAR 52-227-19 (June 1987).

TRADEMARK INFORMATION NetApp, the NetApp logo, Go further, faster, Data ONTAP, ApplianceWatch, BareMetal, Center-to-Edge, ContentDirector, gFiler, MultiStore, SecureAdmin, Smart SAN, SnapCache, SnapDrive, SnapMover, Snapshot, vFiler, Web Filer, SpinAV, SpinManager, SpinMirror, SpinShot, FAServer, NearStore, NetCache, WAFL, DataFabric, FilerView, SecureShare, SnapManager, SnapMirror, SnapRestore, SnapVault, Spinnaker Networks, the Spinnaker Networks logo, SpinAccess, SpinCluster, SpinFS, SpinHA, SpinMove, SpinServer, and SpinStor are trademarks or registered trademarks of NetApp, Inc. in the United States and other countries.

Apple is a registered trademark and QuickTime is a trademark of Apple Computer, Inc. in the United States and/or other countries.

Microsoft is a registered trademark and Windows Media is a trademark of Microsoft Corporation in the United States and/or other countries.

RealAudio, RealNetworks, RealPlayer, RealSystem, RealText, and RealVideo are registered trademarks and RealMedia, RealProxy, and SureStream are trademarks of RealNetworks, Inc. in the United States and/or other countries.

All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such.

NetApp is a licensee of the CompactFlash and CF Logo trademarks.




TABLE OF CONTENTS WELCOME ........................................................................................................................................ 1

MODULE 1: SAN REVIEW ............................................................................................................ 1-1

MODULE 2: WINDOWS FC CONNECTIVITY ............................................................................... 2-1

MODULE 3: WINDOWS ISCSI CONNECTIVITY .......................................................................... 3-1

MODULE 4: WINDOWS LUN ACCESS ........................................................................................ 4-1

MODULE 5: VSPHERE OVERVIEW ............................................................................................. 5-1

MODULE 6: VSPHERE ISCSI CONNECTIVITY ........................................................................... 6-1

MODULE 7: VSPHERE FC CONNECTIVITY ................................................................................ 7-1

MODULE 8: VSPHERE LUN ACCESS ......................................................................................... 8-1

MODULE 9: RED HAT OVERVIEW ............................................................................................... 9-1

MODULE 10: RED HAT FC CONNECTIVITY ............................................................................. 10-1

MODULE 11: RED HAT ISCSI CONNECTIVITY ......................................................................... 11-1

MODULE 12: RED HAT LUN ACCESS....................................................................................... 12-1

MODULE 13: LUN PROVISIONING ............................................................................................ 13-1

MODULE 14: SAN MANAGEMENT ............................................................................................ 14-1

MODULE 15: SAN TROUBLESHOOTING .................................................................................. 15-1

APPENDIX 1: FC DETAILS ........................................................................................................ A1-1

APPENDIX 2: INTRODUCTION TO FCOE ................................................................................ A2-1

APPENDIX 3: INTERNET STORAGE NAME SERVICE ............................................................ A3-1

APPENDIX 4: MULTIPLE CONNECTION SESSIONS .............................................................. A4-1

APPENDIX 5: STORAGE MANAGER FOR SANS .................................................................... A5-1

APPENDIX 6: HYPER-V INTRODUCTION ................................................................................ A6-1

APPENDIX 7: SAN TROUBLESHOOTING ON WINDOWS ...................................................... A7-1

APPENDIX 8: NFS DATASTORES ............................................................................................ A8-1

APPENDIX 9: SERVER CONSOLIDATION ............................................................................... A9-1

APPENDIX 10: NPIV TROUBLESHOOTING ........................................................................... A10-1

APPENDIX 11: VMWARE SNAPSHOTS AND NETAPP SNAPSHOT COPIES ..................... A11-1

APPENDIX 12: VIRTUAL STORAGE CONSOLE .................................................................... A12-1

APPENDIX 13: SNAPDRIVE FOR UNIX (LINUX VERSION) .................................................. A13-1




SAN Implementation Workshop

Course Number: STRSW-ILT-SANIMP-R1

SAN IMPLEMENTATION WORKSHOP




© 2009 NetApp. All rights reserved.

Logistics

Introductions Schedule (start time, breaks, lunch, close) Telephones and messages Food and drinks Restrooms

LOGISTICS





Safety

Alarm signal Evacuation route Assembly area Electrical safety

SAFETY





Course Objectives

By the end of this course, you should be able to: Define and describe storage area networks using Fibre

Channel Protocol (FCP) and Internet SCSI (iSCSI) Configure Windows® Server 2008 R2, vSphere™ (ESX

4.0), Red Hat® 5.3, and Data ONTAP® for iSCSI connectivity

Configure Windows Server 2008 R2, vSphere (ESX 4.0), Red Hat 5.3, and Data ONTAP for FC connectivity

Create and access a LUN by way of FCP and iSCSI from Windows Server 2008 R2, vSphere (ESX 4.0), and Red Hat 5.3

COURSE OBJECTIVES





Course Objectives (Cont.)

Install and use SnapDrive® for Windows and SnapDrive for UNIX® (in Appendix 13) to create LUNs, Snapshot LUNs, restore LUNs from a Snapshot copy, and remove LUNs

Size, clone, back up and recover LUNs on Windows Server 2008 R2, vSphere (ESX 4.0), and Red Hat 5.3

Troubleshoot SAN connectivity and performance issues

COURSE OBJECTIVES (CONT.)





Course Agenda

Day 1– Introductions and Overview– Module 1: SAN Review– Module 2: Windows FC Connectivity– Module 3: Windows iSCSI Connectivity– Module 4: Windows LUN Access

Day 2– Module 5: vSphere Overview– Module 6: vSphere iSCSI Connectivity– Module 7: vSphere FC Connectivity– Module 8: vSphere LUN Access

COURSE AGENDA





Course Agenda (Cont.)

Day 3– Module 9: Red Hat Overview– Module 10: Red Hat FC Connectivity– Module 11: Red Hat iSCSI Connectivity– Module 12: Red Hat LUN Access– Module 13: LUN Provisioning– Module 14: Data ONTAP SAN Management– Module 15: Data ONTAP SAN Troubleshooting

NOTE: Additional materials may be found in the appendixes

COURSE AGENDA (CONT.)





Information Sources

NOWTM (NetApp on the Web)– http://NOW.NetApp.com

NetApp University – http://www.netapp.com/us/services/university/

NetApp University Support– http://netappusupport.custhelp.com

INFORMATION SOURCES





Typographic ConventionsConvention Type of Information

Italic Font

Book titles.Words or characters that require special attention.Variable names or placeholders for information you must supply, for example:

An ifstat command looks like this:ifstat -z -a <interface>

The name of the interface for which you want to view statistics is interface.

Monospaced font

Command names, daemon names, and option names.Information displayed on the system console or other computer monitors.The contents of files.

Bold monospaced font

Words or characters you type, for example:Enter the following command:options httpd.enable on

license add <code1> <code2>

TYPOGRAPHIC CONVENTIONS


1-1 SAN Implementation Workshop: SAN Review


SAN Review

Module 1SAN Implementation Workshop

SAN REVIEW





Module Objectives

By the end of this module, you should be able to: Describe the differences between network-

attached storage (NAS) and storage area network (SAN)

List the methods to implement SAN solutions Define logical unit number, initiator and target Describe ports, worldwide names, and

worldwide port names List the steps to implement a SAN

MODULE OBJECTIVES





Assumptions of the Course

This course is unlike other NetApp® courses– It is not designed to teach you the concepts

This class provides the opportunity to implement FC and IP SANs on the following platforms:– Microsoft® Windows® Server 2008 R2– VMware® vSphere™ (also known as ESX 4.0)– Red Hat® Enterprise Linux® 5.3

In this module, we will review the basics of a SAN

ASSUMPTIONS OF THE COURSE





SAN Review

SAN REVIEW





Question

SANNAS

NetApp FAS

NFSCIFSCorporate

LAN

iSCSI

FCoEFC

What does NAS and SAN stand for and what is the difference?

QUESTION Operating systems and applications request data either at the block level or the file level. Network-attached storage (NAS) provides file-level access to data on a storage system. Access is by way of a network, using Data ONTAP® services such as CIFS and NFS. Storage area networks (SANs) provide block-level access to data on a storage system. SAN solutions can be any mixture of iSCSI or FC protocols. When both SAN and NAS storage are present on the same storage system, it is referred to as “unified storage.”





SCSI

SAN uses Small Computer System Interface (SCSI) protocol over a distance

SCSI features:– Block-level access– Efficiency– Lower overhead– Resiliency

SCSI Small Computer System Interface (SCSI) is a set of standards that define commands, protocols, and interfaces used to transmit data. SCSI allows low-level “block” access to data in units of 512-byte blocks. This is highly efficient and has low overhead compared to NAS or “file” level access. SCSI has a high level of resiliency that makes it perfect for an enterprise-level protocol.





SCSI on Host and Controller

Controller

Host

SCSI DriverFile SystemApplication

WAFL®SAN Services

Direct-Attached Storage

Direct-Attached DevicesSCSI Adapter

Fibre Channel

SCSI ON HOST AND CONTROLLER Traditionally, storage is attached to a local machine. SCSI is used for transmitting data between a host and peripheral devices either through SCSI adapters or other adapters that communicate using SCSI commands.





Logical UnitHost


WAFLSAN Services

The virtual disk is a single file on

the server

LUN

The logical unit is accessed by a

Logical Unit Number (LUN)

Controller

Direct-Attached Devices

LOGICAL UNIT





Question

What are the benefits of using a NetApp SAN over direct-attached storage (DAS) device?– Availability Data is generally more reliable than it is with DAS

– Storage utilization Space can be assigned to optimize usage

– Centralized management Provisioning is centralized

– Disaster recovery and backup Centralize backup and recovery

QUESTION SAN provides numerous benefits over the traditional NAS environment:

Availability: As data is moved off of local hosts and onto enterprise storage arrays, data access is generally more reliable.

Storage Utilization: Space can be assigned based upon appropriate need allowing a higher optimization of the storage.

Management: Since all storage is centralized with SAN, management of the storage becomes more efficient and effective.

Disaster Recovery and Backup: Centralizing data allows for easier and more effective backup and recovery strategies.





Terms Review

TERMS REVIEW





QuestionHost


WAFLSAN Services

LUN

Target

Initiator

What is the host called in a SAN?

What is the storage system called in a SAN?

Controller

QUESTION Initiators, including Windows and UNIX®-type hosts, are consumers or clients within a SCSI relationship. Targets, including NetApp controllers and storage arrays, present data as logical units and are the servers within a SCSI relationship.





How may a SAN be implemented?– Fibre Channel (FC) Referred to as FC SAN Uses Fibre Channel Protocol (FCP) to communicate

Uses Fibre Channel over Ethernet (FCoE) to communicate

– Internet Protocol (IP) Referred to as IP SAN Uses Internet SCSI (iSCSI) to communicate

Question

Ethernet IP TCP iSCSI SCSI

Ethernet FCoE FC Frame SCSI

Physical Data FC Frame SCSI

QUESTION LUNs on a NetApp storage system can be accessed through either a Fibre Channel (FC SAN) fabric using the Fibre Channel Protocol (FCP)or an Ethernet network using the Fibre Channel over Ethernet (FCoE) or Internet SCSI (iSCSI) protocols. In all cases, the transport portals (FCP, FCoE or iSCSI) carry encapsulated SCSI commands as the data transport mechanism.

iSCSI is an IETF standard found here: http://www.ietf.org/rfc/rfc3720.txt?number=3720





Question

Target

InitiatorSCSI DriverFile SystemApplication

WAFLSAN Services

Ethernet Port Fibre Channel Port orConverged Network Adapter (CNA)

FC DriveriSCSI DriverTCP/IP Driver

FC DriveriSCSI DriverTCP/IP Driver

IPSAN

FC SAN

LUN

What are the ports called in an IP and FC SAN?

QUESTION Data is communicated over ports. In an IP SAN, the data is communicated by way of Ethernet ports. In an FC SAN, the data is communicated over Fibre Channel ports.





Question


20:00:00:2b:34:26:a6:56

50:0a:09:80:86:f7:c7:86

21:00:00:2b:34:26:a6:56

50:0a:09:81:86:f7:c7:86

Worldwide Node Name (WWNN) Worldwide Port Name (WWPN)

WAFLSAN Services

LUN FC SAN

IPSAN

What are the Nodes and Ports called in an FC SAN?

Target

Initiator

QUESTION In FC SAN, a worldwide node name (WWNN) describes a machine while a worldwide port name (WWPN) describes a physical port attached to that machine.

The FC specification for the naming of nodes and ports on those nodes can be fairly complicated. Each device is given a globally unique WWNN and an associated WWPN for each port on the node. WWNNs and WWPNs are 64-bit addresses made up of 16 hexadecimal digits grouped together in twos with a colon separating each pair (for example, 21:00:00:2b:34:26:a6:54).

The first number in the address defines what the other numbers in the address represent, according to the FC specification. The first number is generally a 1, 2, or 5. In the example of QLogic® initiator host bus adapters (HBAs), the first number is generally a 2. For Emulex® initiator HBAs, the first number is generally a 1. Finally, a NetApp storage system is assigned with a 5.





Question


Local Network Connection

Target Portal Group (TPG)

iqn.1999-04.com.a:host

iqn.1998-02.com.netapp:ss1

Portals Worldwide Node (WWN)

WAFLSAN Services

LUN FC SAN

IPSAN

What are the Nodes and Ports called in an IP SAN?

Target

Initiator

QUESTION In IP SAN, the worldwide node (WWN) describes a machine while the portal describes a physical port. Each iSCSI node must have a node name. There are two possible node name formats.

IQN-TYPE DESIGNATOR

The format of this node name is conventionally “iqn.yyyy-mm.backward_naming_authority: unique_device_name.” This is the most popular node name format and is the default used by a NetApp storage system. The components of the logical name are the following:

Type designator, IQN, followed by a period (.) The date when the naming authority acquired the domain name, followed by a period The name of the naming authority, optionally followed by a colon (:) A unique device name EUI-TYPE DESIGNATOR

The format of this node name is “eui.nnnnnnnnnnnnnnnn.” The components of the logical name are the following:

The type designator itself, “eui,” followed by a period (.)

Sixteen hexadecimal digits

Example: “eui.123456789ABCDEF0”





Question


WAFLSAN Services

LUN FC SAN

Directly connectedConnected through

a switch

IPSAN

How can we connect an initiator to a target?

Target

Initiator

QUESTION





Implementing SAN

IMPLEMENTING SAN





Designing and Planning a SAN Solution

1. Installation preparation– Purpose: Determine compatibility issues that need to be resolved– Tools: System Configuration Guide, FC and iSCSI Configuration

Guide, system configuration forms, Interoperability Matrix Tool2. SAN assessment

– Purpose: Collect all information that will be needed to plan the implementation

– Tools: Interoperability Matrix Tool3. Solution verification

– Purpose: Ensure that solution satisfies customer’s requirements– Tools: Interoperability Matrix Tool

4. Gap analysis– Purpose: Identify any activities that need to be performed to fill gaps– Tools: Interoperability Matrix Tool

5. Assessment phase review and Sign off– Purpose: Review solution proposed and obtain customer sign off

DESIGNING AND PLANNING A SAN SOLUTION The System Configuration Guide can be found here:

http://now.netapp.com/NOW/knowledge/docs/hardware/NetApp/syscfg/

The FC & iSCSI Configuration Guide can be found here:

http://now.netapp.com/NOW/knowledge/docs/san/fcp_iscsi_config/index.shtml





Interoperability Matrix Tool

Found at: http://now.netapp.com/matrix– Requires a NOW™ account for authenticationSelect “Storage Area Network

(SAN)”to configure a SAN solution

Then choose yourconfiguration to see

if it is supportedor better yet...

Don’t purchase hardware until you

validate it is supported

Number of configurations

with the current filter

INTEROPERABILITY MATRIX TOOL The NetApp Interoperability Matrix tool is a Web-based utility. You can use the tool to search for information about configurations for NetApp products that work with third-party products and components qualified by NetApp.

The Interoperability Matrix tool contains both supported and certified NetApp configurations. A configuration consists of components, such as operating systems, host utilities, and switches, that NetApp has identified. NetApp assigns the configuration a name, which is a concatenation of the timestamp and an auto-generated number. For example, a configuration created on 21 November 2008 is assigned a configuration name similar to 20081121-021727786.

NOTE: Supported configurations are those that have been qualified by NetApp. Certified configurations are ones that have been qualified by another company to work with NetApp components. Do not confuse a supported configuration with a configuration that has a status of Supported. If a supported or certified configuration has been reviewed and approved by NetApp, the status of the configuration can be Supported or PVR required, regardless of which company qualified the components. A configuration in PVR required state indicates that the configuration has been evaluated and can be supported for all customers upon approval of the associated PVR.





Interoperability Matrix Tool (Cont.)

All configurationsare in the

SAN Report;Available as spreadsheet

or PDF

INTEROPERABILITY MATRIX TOOL (CONT.) Reports are available for all configurations in the format of a PDF or an Excel® spreadsheet.






Select yourhardware or OS

or software...

...and “Add Components”to add them to your

proposed configuration

INTEROPERABILITY MATRIX TOOL (CONT.) You can search the configurations for NetApp products that work with third-party products and components that are qualified by NetApp by using the configuration search feature.






After the platformwas added,

the remaining list is filtered

Click the trash can to remove the item

from the filter

Click here to showthe two results

INTEROPERABILITY MATRIX TOOL (CONT.)






Supported configuration(s)based upon the filter

Export toExcel and PDF

NOTE: Best practice is to print or save to PDFthe supported configuration of your solutiononce purchased, installed, and configured

INTEROPERABILITY MATRIX TOOL (CONT.) To export the search results of a single storage solution to a Microsoft Excel worksheet, complete the following steps:

1. Click Excel. The Export Settings dialog box appears.

2. Select the appropriate component type based on how you want to group the worksheets from the Worksheet Grouping list.

3. Select the columns to be exported from the Component Types to export list. 4. Click Export.

The search results are grouped based on your requirements and are exported to an Excel sheet.

To export your search results to a PDF, complete the following steps:

1. Click Pdf. The Pdf Export Settings dialog box appears.

2. Select the appropriate option based on which you want to group the search results from Grouping.

3. Select the columns to be exported from Component Types to export. 4. Click Export.

The search results are grouped based on your requirements and are exported to a PDF.





Steps to Implement a SAN

1. Have an initiator discover a target portal2. Create a session between the initiator with

the target (make the bindings persistent between reboots)

3. Create an igroup on the storage system if necessary

4. Create a LUN on the storage system 5. Map the LUN to an igroup on the storage

system 6. Find the LUN on the host7. Prepare the disk for the host OS if necessary

STEPS TO IMPLEMENT A SAN





1. Have an Initiator Discover a Target

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Must tell host where to discover the target by either host name (IP) or by a name service

When ports are active, discovery is

automatic either direct connected or by a name service

Target

Initiator

1. HAVE AN INITIATOR DISCOVER A TARGET The first step in a SAN implementation is discovery of a target. In IP SAN, an administrator must tell the client where to discover a target by either directly connecting by host name (IP) or discovery through a name service (iSNS). See Appendix 3. In FC SAN, if the ports are active, discovery is automatic either directly connected or by a name service. For more information about IP SAN discovery, see Module 3. For more information concerning FC SAN discovery, see Module 2.





1. Have an Initiator Discover a Target (Cont.)

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Single pathMultiple paths

Target

Initiator

1. HAVE AN INITIATOR DISCOVER A TARGET (CONT.)





1. Have an Initiator Discover a Target (Cont.)

Ethernet Fibre Channel

Target

Initiator



Active-activecontroller configuration

FC connects over the IC

iSCSI does not connect over IC

1. HAVE AN INITIATOR DISCOVER A TARGET (CONT.)





2. Create a Session

Ethernet

Ethernet

Fibre Channel

Fibre Channel

A session associates an initiator with a target; persisting the session is possible to ensure availability after reboot

Target

Initiator

2. CREATE A SESSION Sessions associate the initiators with targets. A session may be persisted to ensure availability after a host reboots.





3. Create an igroup

My_IP_igroupiqn.1999-04.com.a:system

OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Place WWN (iqn / eui) in igroups for iSCSI

Place WWPN in igroups for FC

Target

Initiator

3. CREATE AN IGROUP An igroup is a group of one or more initiators that have access to a target. In IP SAN, an administrator references an initiator by WWN. In FC SAN, an administrator references an initiator by WWPN.





4. Create a Logical Unit


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Create a Logical Unit

LUNbLUNa

Target

Initiator

4. CREATE A LOGICAL UNIT The next step in implementing a SAN is to create a logical unit. A logical unit is a logical representation of a physical unit of storage. It is a collection of, or a part of, physical or virtual disks configured as a single disk. When you create a logical unit, it is automatically striped across many physical disks. Data ONTAP manages logical units at the block level, so it cannot interpret the file system or data in a logical unit.





5. Map a Logical Unit to an Igroup


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Map a logical unit to an igroup by assigning a logical unit number

LUNbLUNa

NOTE: This step is also called LUN masking

Target

Initiator

1 2

5. MAP A LOGICAL UNIT TO AN IGROUP The logical unit is mapped to an igroup and referenced by an ID. The logical unit is then referred to as a logical unit number, or LUN.





6. Find the Disk


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

1 2LUNbLUNa

/

dev- disk 1- disk 2- disk 3

LUNa is identified by client OSLUNb is identified by client OS

Target

Initiator

6. FIND THE DISK The LUN is then identified by the client operating system. From the host, LUNs appear as local disks, allowing you to format and store data on them.





6. Find the Disk (Cont.)


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:5621:00:00:2b:34:26:a6:57

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

1 2

If multiple paths exist, the same LUN might appear more

than once

/

dev- disk 1- disk 2 - disk 4 - disk 3 - disk 5

LUNbLUNa

Target

Initiator

6. FIND THE DISK (CONT.) If multiple paths exist, the LUN will appear more than once unless multipathing software is used.





7. Prepare the Disk for the Host OS (Cont.)


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Treat LUNs as a single disk or combine together using a volume manager

1 2

/


Target

Initiator

LUNbLUNa

7. PREPARE THE DISK FOR THE HOST OS (CONT.) LUNs may be used as single disk or combined together using a host-based volume manager.





7. Prepare the Disk for the Host OS


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Label, format, add a file system, and mount the

LUN per the OS

1 2LUNbLUNa

/


mount lunb

mount luna

Target

Initiator

7. PREPARE THE DISK FOR THE HOST OS Finally, the logical unit must be labeled, formatted, a file system added, and finally mounted by the client OS.





LUN Setup Complete


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre ChannelFile System

1 2

/


mount lunb

mount luna

Target

Initiator

LUNbLUNa

LUN SETUP COMPLETE





Evaluation

How did you do? If you are unfamiliar with many of the concepts

here in this this module, please see the SAN Administration on Data ONTAP course for instructional development of the SAN concepts

This course is focused on providing the steps to implement a SAN on the following platforms:– Microsoft Windows Server 2008 R2– VMware vSphere (ESX 4.0)– Red Hat Enterprise Linux 5.3

EVALUATION





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe the differences between network-

attached storage (NAS) and storage area network (SAN)

List the methods to implement SAN solutions Define logical unit number, initiator and target Describe ports, worldwide names, and

worldwide port names List the steps to implement a SAN

MODULE SUMMARY




Exercise

Module 1: SAN ReviewEstimated Time: 30 minutes

EXERCISE Please refer to your Exercise Guide materials for more instructions.


2-1 SAN Implementation Workshop: Windows FC Connectivity


Windows FC Connectivity


WINDOWS FC CONNECTIVITY





Module Objectives

By the end of this module, you should be able to: Describe multiple path implementation with

Fibre Channel (FC) connectivity Describe how to configure FC ports on

Windows® and NetApp® systems Describe commands and utilities to identify the

worldwide node name (WWNN) and worldwide port name (WWPN) on Windows and NetApp systems

Use commands and utilities to examine FC switch activity

MODULE OBJECTIVES





FC Connectivity Configuration

The following are the steps to configure FC SAN:1. Determine the FC topology2. Verify host HBA configuration, drivers,

firmware, cables, and multipathing software3. Configure the switch (if in the topology)4. Configure the target(s) 5. Configure the initiator(s)6. Cable the devices together7. Implement FC zoning (if required)

FC CONNECTIVITY CONFIGURATION





FC Topology

FC TOPOLOGY





FC Connectivity

Type of topologies:1. Direct-attached2. Single fabric3. Dual fabric

NetApp FAS

FC CONNECTIVITY NOTE: Fibre Channel-Arbitrated Loop (FC-AL), private loop and public loop topologies are not discussed in this presentation.





Direct-Attached Topology

Fibre Channel

Also called point-to-point

Windows

DIRECT-ATTACHED TOPOLOGY Initially, Fibre Channel (FC) point-to-point topologies were seen as a replacement for the parallel SCSI bus, to overcome bandwidth and distance limitations. FC at 100 Mbps was superior to SCSI at 10 to 20 Mbps, and as SCSI progressed to 40, 80, then 160 Mbps, FC stayed ahead with 200 Mbps then 400 Mbps. SCSI bandwidth was reaching a ceiling where FC at 200 Mbps was just getting started. FC point-to-point also overcame the severe distance limitations of SCSI, but one limitation remained: it connected one initiator to one target, supporting only the simplest topology. This provides limited connectivity.





Problems with Direct-Attached Topology

Fibre Channel

. . .

Does not scale

No faulttolerance

Windows Windows

PROBLEMS WITH DIRECT-ATTACHED TOPOLOGY Direct-attached topology does not scale and is therefore not appropriate for enterprise environments. There is also no fault tolerance. If the cable or an HBA is defective, then a host will lose all connectivity with its storage as in the example above.





Single-Fabric Topology

Fibre Channel

Inter-SwitchLink (ISL) = single-fabric

Windows

SINGLE-FABRIC TOPOLOGY The switched fabric uses a 24-bit addressing scheme with a 64-bit WWPN and WWNN. This scheme has 12 million possibleaddresses, and the initiator-target pair got a dedicated non-blocking path to ensure full bandwidth.

In this configuration, all devices are connected to FC switches.

Single fabric is a switched fabric topology where the servers are attached to NetApp storage controllers through a single FC fabric. This fabric may consist of multiple FC switches that are connected together.





Problems with Single-Fabric Topology

Fibre Channel

Only one name server per fabric

Not fully redundant

Windows

PROBLEMS WITH SINGLE-FABRIC TOPOLOGY





Dual Fabric Topology

Fibre Channel

Switches not connected = dual fabric

Twoswitchesrequired

Cluster the initiator for complete redundancy

Windows

DUAL FABRIC TOPOLOGY





Topology Summary

To optimize connectivity: Architect clustered hosts Implement a dual fabric design with multiple

paths Configure active-active storage systems

NOTE: The exercise environment has an active-active storage configuration with single hosts and single switch

TOPOLOGY SUMMARY





FC Exercise Environment

Fibre Channel

0c - 0

76

0d - 1 0c - 2 0d - 3

switchport

Storage System 1 Storage System 2

Windows Server 2008 R2

FC EXERCISE ENVIRONMENT





Data ONTAP

DATA ONTAP





High Availability

The exercise environment’s storage systems have been configured as a high-availability (HA) pair

To configure controller failover:system and system2> license add xxxxxx

system and system2> reboot

system> cf enable

HIGH AVAILABILITY





Data ONTAP as an FC Target

Data ONTAP® 6.3 and later has support for FC Data ONTAP features:

– Built-in FC functionality – Simple LUN creation and management

To properly configure Data ONTAP for FC connectivity:– Confirm FC HBA(s) / port(s)– Configure and verify Fibre Channel protocol– Configure the FC Target HBA(s)– Identify the worldwide node name (WWNN)– Identify the worldwide port name (WWPN)

DATA ONTAP AS AN FC TARGET





HBA Confirmation

Confirm the current HBAs:system> sysconfig -a

To identify the type of the on-board FC ports:system> fcadmin config

Local

Adapter Type State Status

------------------------------------------------

0a initiator CONFIGURED online

0b initiator CONFIGURED online

0c initiator CONFIGURED online

0d initiator CONFIGURED online

NOTE: Add-on cards are configured to be either an initiator or target and cannot be changed

HBA CONFIRMATION The fcadmin utility manages the Fibre Channel adapters used by the storage subsystem. Use these commands to show link- and loop-level protocol statistics, list the storage devices connected to the storage system, and configure the personality of embedded adapters.





HBA Confirmation (Cont.)

To change an onboard interface from an initiator to a target:system> fcadmin config -d 0b

system> fcadmin config -t target 0b

system> reboot

To change an onboard interface from a target to an initiator:system> fcadmin config -d 0b

system> fcadmin config -t initiator 0b

system> reboot

Down the interfaceReconfigure the interface as a target

A reboot is required

HBA CONFIRMATION (CONT.)





Configuring FC Services in Data ONTAP

1. Verify Fibre Channel protocol service is running:– fcp status

2. Verify FC is licensed (license the FC services if needed):– license

– license add XXXXXX

3. Start the FC service:– fcp start

CONFIGURING FC SERVICES IN DATA ONTAP





Configuring FC HBA in Data ONTAP

1. List the installed target HBAs:– fcp show adapters

2. Take an HBA offline:– fcp config adapter down

3. Set target HBA speed to the FC switch port’s speed to improve takeover / giveback performance:– fcp config adapter speed [auto|1|2|4]

4. Bring an HBA online:– fcp config adapter up

CONFIGURING FC HBA IN DATA ONTAP





Identify WWNN in Data ONTAP

WWNN uniquely identifies the storage system– The default WWNN is generated by a serial

number in its NVRAM and stored on disk Normally doesn’t need to be changed

To identify the WWNN:system> fcp nodename

Fibre Channel nodename: 50:0a:09:80:86:f7:c7:86

(500a098086f7c786)

To change the WWN:system> fcp nodename [-f] new_wwnn

Available in Data ONTAP 7.3.1.1

and later

IDENTIFY WWNN IN DATA ONTAP





Identify WWPN in Data ONTAP

WWPN uniquely identifies an FC HBA port WWPNs are determined by:

– WWNN– Controller failover mode (cfmode)– Internal port index

To verify the default WWPN:system> fcp portname show

Portname Adapter

-------- -------

50:0a:09:81:86:f7:c7:86 0c

50:0a:09:82:86:f7:c7:86 0d

To change a WWPN:system> fcp portname set adapter

Available in Data ONTAP 7.3.1.1

and later

IDENTIFY WWPN IN DATA ONTAP Within Fibre Channel (FC) SAN, worldwide port names (or WWPNs) uniquely identify each Fibre Channel port. Each 64-bit address is determined by three factors. The first factor is the worldwide node name (WWNN), which is the unique identifier for the NetApp storage system running as an FC target device server. The second factor is the controller failover mode (cfmode) currently set on the NetApp storage system. The third and final factor is that each FC target port has an internal port index range that assists in assigning the WWPNs.





FCP cfmodes Defined

Controller failover mode (cfmode) determines how HBAs of storage systems in an active-active configuration:– Log in to the fabric– Provide access to local and partner LUNs

Both storage systems in the active-active configuration must have the same settings– fcp show cfmode to verify current setting– fcp set cfmode to set the cfmode Requires advanced mode to set cfmodepriv set advanced

FCP CFMODES DEFINED Cfmode only applies to Fibre Channel environments in an active-active NetApp storage controller configuration. The cfmode determines how target ports do the following:

Log in to the fabric Handle local and partner traffic for a cluster Provide access to local and partner LUNs in a cluster In the original release of Data ONTAP 6.3, which included SAN support for Fibre Channel, cfmode standby was the implied default. There was not a setting for cfmode in that release, and it was not called cfmode standby. However, when Data ONTAP 6.5 was released, four cfmodes were introduced. One of these modes was standby. The others were partner, mixed and dual fabric. In Data ONTAP 7.1, a new cfmode called single system image (SSI) became available. SSI is the default cfmode for new installations with Data ONTAP 7.2. The availability of standby, partner, mixed and dual fabric modes is dependent on the storage controller model, Data ONTAP version, and/or the use of 2-Gb or 4-Gb FC ports. With Data ONTAP 7.3, the only available cfmode is single system image.





FCP cfmode Types

Prior to Data ONTAP 7.3, there were five types of cfmodes:cfmode Supported systems Benefits and limitations

partner All systems except the FAS270c, FAS20x0, FAS3040, FAS3070, FAS31x0, FAS60x0 and any FAS system with a 4-Gb or 8-Gb target FC adapter

Supports all host OS types Supports all switches

single_image All systems Supports all host OS types Supports all switches Makes all LUNs available on all target

ports

dual_fabric FAS270c only Supports all host OS types Requires fewer switch ports Does not support all switches;

requires switches that support public loop

standby All systems except the FAS270c, FAS20x0, FAS31x0, FAS6040, FAS6080 and FAS6030 / FAS6070 with a 4-Gb or 8-Gb FC adapter

Requires more switch ports Supports only Windows and Solaris™

hosts

mixed All systems except the FAS270c, FAS20x0, FAS30x0, FAS31x0, FAS6040, FAS6080 and FAS6030 / FAS6070 with a 4-Gb or 8-Gb FC adapter

Supports all operating systems Does not support all switches;

requires switches that support public loop

FCP CFMODE TYPES There are five possible cfmodes on the storage controller. Only one cfmode can be set per each storage controller, and in a cluster situation the cfmode must be the same for both systems.

STANDBY

The standby mode is supported on all systems except the FAS270c. It supports only Windows and Solaris operating systems. In addition, this mode requires additional switch ports.

PARTNER

The partner mode is supported on all systems except the FAS270c and the FAS6000 series. All switches and host operating systems are supported.

DUAL-FABRIC

The dual-fabric mode is only supported on a FAS270c. All host operating systems are supported by this mode. This mode requires a switch that supports a public loop.

MIXED

The mixed mode is supported on all systems except the FAS270c and the FAS6000 series. Mixed mode supports all host operating systems, but requires a switch that supports a public loop.

SINGLE IMAGE

The single image mode is supported on all systems, switches, and host operating systems. This mode makes all LUNs available on all target ports.





FCP cfmodes Types (Cont.)

Data ONTAP 7.3 and later only supports single_image cfmode– Partner and standby are supported only on upgrades of

existing systems that currently support and use these cfmodes modes

– After an upgrade to 7.3+, a storage system pair may only be (re)configured to single_image

single_image cfmode– Each active-active pair has a single WWNN, allowing

both storage systems in the active-active configuration to function as a single Fibre Channel storage system

– All LUNs in an HA are available on all ports in the HA pair

– Multipathing software is required

FCP CFMODES TYPES (CONT.) The single_image cfmode setting is available in Data ONTAP 7.1. This cfmode setting is the default for new installs Data ONTAP 7.2 and later. Upgrades to Data ONTAP 7.2 will retain the cfmode from the previous version. In SSI cfmode, an active-active storage controller configuration has a single WWNN, and both systems in the configuration function as a single Fibre Channel node. Each node in the cluster shares the partner node’s LUN map information.

All LUNs in the cluster are available on all ports in the cluster by default. As a result, more paths to each LUN are stored on the cluster. Any port can provide access to both local and partner LUNs. You can specify the LUNs available on a subset of ports by defining port sets and binding them to an igroup. Any host in the igroup can then access the LUNs only by connecting to the target ports in the port set.





Single Image Example

0c 0c 0d0d

Active-ActiveConfiguration

Controller 1 Controller 2

Host

LUNs LUNs

Solid blue lines are paths to the LUNs being served by Controller 1

Dashed green lines are paths to the LUNs being served by Controller 2

Multipathing software required

Fabric 1 Fabric 2

SINGLE IMAGE EXAMPLE LUNs from both controllers are visible through a single physical (and logical) port.

A single FC port is a primary path for a LUN served by that controller and a secondary path for a LUN on the partner controller.





Single Image Example (Cont.)


Host

LUNs LUNs



MP layer works around the failure

Switch/Fabric 1 will experience a failure


x


Fabric 1 Fabric 2

0c 0c 0d0d

SINGLE IMAGE EXAMPLE (CONT.) Switch 1 experiences a failure. The host multipathing software layer works around that failure to reroute the I/O through the secondary path to the LUN.







Host

LUNs LUNs



Controller 2 takes over all operations MP layer works around the failure

Controller 1 will experience a failure


xActive-ActiveConfiguration

Fabric 1 Fabric 2

0c 0c 0d0d

SINGLE IMAGE EXAMPLE (CONT.) When Controller 1 experiences a failure, Controller 2 takes over all operations. The host multipathing software layer works around the failure by rerouting I/O through the Controller 2 path to the LUN.







Host

LUNs LUNs





Fabric 1 Fabric 2

0c 0c 0d0d

SINGLE IMAGE EXAMPLE (CONT.) Single system image can protect against controller and fabric failure with as little as a single port. However, NetApp recommends a fully redundant configuration if more ports are available. This flexibility allows single system image cfmode to support models like the FAS270c, as well as models like the FAS3000 and FAS6000 series.






Loop ModeLoop Mode



Host

LUNs LUNs




0c 0c 0d0d

SINGLE IMAGE EXAMPLE (CONT.) The FAS270c and FAS3000 active-active controllers may be directly attached to the host when single image cfmode is used. This configuration is not supported with any FAS800 or FAS900 series (standby, partner, or mixed cfmodes).

Single system image does support this configuration. SSI allows ports to alternate between fabric point-to-point login and individual loop. All LUNs are available on a single port in the event that one of the links fails.





fcp config (Single System Image Mode)

system or system2> fcp nodenameFibre Channel nodename: 50:0a:09:80:86:f7:c7:86 (500a098086f7c786)system or system2> fcp show cfmodefcp show cfmode: single_imagesystem> fcp config0c: ONLINE <ADAPTER UP> PTP Fabric

host address 011000portname 50:0a:09:81:96:f7:c7:86 nodename 50:0a:09:80:86:f7:c7:86mediatype auto speed auto

0d: ONLINE <ADAPTER UP> PTP Fabrichost address 011100portname 50:0a:09:82:96:f7:c7:86 nodename 50:0a:09:80:86:f7:c7:86mediatype auto speed auto

system2> fcp config0c: ONLINE <ADAPTER UP> PTP Fabric



FCP CONFIG (SINGLE SYSTEM IMAGE MODE) This is an example of the adapter settings for single_image cfmode. Notice that all node names are identical and that the media type is set to auto. This means that the ports log into the fabric using point-to-point mode. If point-to-point mode fails, then the ports will try loop mode.





Windows

WINDOWS





Windows as an FC Initiator

NetApp has supported Windows as an FC initiator OS since Windows 2000 Server

Windows Server 2008 has many advantages over previous versions– New tools Storage Explorer Storage Manager for

SANs– Multipath I/O (MPIO)– Built-in FC drivers

Windows must be properly configured for FC connectivity

WINDOWS AS AN FC INITIATOR Windows Server 2008 provides many new features that make configuring an FC SAN easier. Storage Explorer provides a one-stop interface for investigating local HBAs as well as the FC switches, if present. Storage Manager for SANs is an additional tool available in Windows Server 2003 R2 and later that allows configuring a SAN environment. Storage Manager for SANs requires the Virtual Disk Service add-in provided by NetApp at the NOW™ site.





Windows FC Design and Installation

1. Verify host operating system releases, required patches, and NetApp Windows Host Utility Kit with Interoperability Matrix– Use System Properties dialog to verify release

2. Install compatible host bus adapters (HBAs)3. Install and configure required HBA drivers and utilities4. Verify an HBA:

– Emulex: Use HBAnyware® – QLogic: Use SANsurfer– All HBA Types: Device Manager Dialog

5. Install compatible NetApp Windows Host Utility Kit– Provides Perl scripts to diagnose and troubleshoot Windows– Example: windows_info provides diagnostic information

WINDOWS FC DESIGN AND INSTALLATION Host utilities contain software tools and documentation that allow you to configure a host in a NetApp SAN environment.

NOTE: Host utilities were formerly called Host Attach and Support Kits. Kits that were released before this naming convention changed are still called Host Attach and Support Kits. The term host utilities will be used in this course, but be aware that NetApp is in the process of transitioning to this name.

Host utilities are available from the Download Software page on the NOW site at:

http://now.netapp.com/NOW/cgi-bin/software





Windows Implementation

After installation, to configure a Windows Emulex or QLogic implementation: Verify the HBA is enabled Identify the WWNN on the host HBA(s) Identify the WWPN on the host HBA(s) Verify connectivity between the initiator(s) and

target

WINDOWS IMPLEMENTATION





Verify that Windows Server 2008 has identified the HBA(s)

Windows/Emulex Implementation

Within Device Manager

Select Storage controllers

Enabled Emulex HBAsDouble-click to investigate

WINDOWS/EMULEX IMPLEMENTATION





Windows/Emulex Implementation (Cont.)

Identify the driver associated with the HBA(s)

Other HBAs will be ata different location

For more information

WINDOWS/EMULEX IMPLEMENTATION (CONT.)






Verify the HBA(s) is connected on Windows Server 2008

Locate the correct server In this example, two Emulex HBAs are

installed







Identify the local WWPN(s) on Windows Server 2008

Select one of the HBAs The WWPN

The WWNN






Discovery

Fibre Channel

Fibre Channel

When ports are active (and properly zoned), discovery is automatic

Target

Initiator

DISCOVERY Within FC SAN, discovery is automatic unless switch zoning prevents it. See Appendix 1 for a discussion about switch zoning.





Data ONTAP Discovery of Initiators

Verify connectivity from the storage system:system> fcp show initiators

Initiators connected on adapter 0c:

Portname Group

-------- -----

10:00:00:00:c9:6b:77:b4

NOTE: For convenience, you may assign an alias to the Windows WWPN

Windows WWPN

DATA ONTAP DISCOVERY OF INITIATORS





WWPN Aliases

In large FC installations, it can be difficult to identify WWPNs because of their cryptic 64-bit name– Example: 10:00:00:00:c9:6b:77:b4

For convenience, WWPN may be assigned a name or “alias” within Data ONTAP

Both target and initiator ports may be aliased

WWPN ALIASES One common problem administrators face in large Fibre installations is determining how to distinguish between WWPNs due to their cryptic 64-bit naming conventions. Now with Data ONTAP 7.3, administrators can rename or “alias” a WWPN with a more convenient name to assist in easy identification. Aliases may be used for both target and initiator ports.





fcp wwpn-alias Command

To alias a WWPN:– Use fcp wwpn-alias set [-f] alias wwpn

– Example:system> fcp wwpn-alias set WIN1-FC

10:00:00:00:c9:6b:77:b4

To remove an alias:– Use fcp wwpn-alias remove {-a alias…|-w wwpn}

– Example: system> fcp wwpn-alias remove -a WIN1-FC WIN2-FC

(removes the aliases)system> fcp wwpn-alias remove -w

10:00:00:00:c9:6b:77:b4

(removes all aliases for a particular WWPN) To show aliases:

– Use fcp wwpn-alias show [-a alias | -w wwpn]

FCP WWPN-ALIAS COMMAND To alias a WWPN, an administrator must use the new fcp wwpn-alias set command. The -f switch can be used to force or reassign an existing alias to a new WWPN. The new WWPN may have multiple aliases, but only one alias can be assigned to a single WWPN.

To remove an alias, an administrator may use either the -a switch to remove one or more particular aliases or the -w switch to remove all aliases for a given WWPN with the new fcp wwpn-alias remove command.

To verify all aliases, an administrator may use the fcp wwpn-alias show command. The administrator can then limit the alias return by requesting to see only a particular alias with the -a switch or a particular WWPN with the -w switch.





Alias Rules

A storage system can have up to 1024 aliases An alias can have the following characters:

A-Z, a-z, 0-9, '_','-','.','{','}' and no spaces Many aliases maybe associated with a single

WWPN, but each alias will be assigned to only one WWPN

Use fcp wwpn-alias help subcommandfor more information on the subcommand

ALIAS RULES The following rules apply to WWPN aliases:

A storage system can have up to 1,024 aliases. An alias can have the following characters: A-Z, a-z, 0-9, '_','-','.','{','}' but no spaces. Many aliases may be associated with a single WWPN, but each alias will be assigned to

only one WWPN. Use the fcp wwpn-alias help <subcommand> function for more information on a particular subcommand.





WWPN Aliases Example

system> fcp wwpn-alias set WIN1-FC 10:00:00:00:c9:6b:77:b4

system> fcp wwpn-alias show

WWPN Alias

---- -----

10:00:00:00:c9:6b:77:b4 WIN1-FC

system> fcp show initiators


Portname Group

-------- -----

10:00:00:00:c9:6b:77:b4

WWPN Alias(es): WIN1-FC

WWPN ALIASES EXAMPLE





Windows Discovery of Targets

Verify connectivity from Windows Server 2008

Select the Brocade®fabric

Storage systems’ WWPNs show up

Windows WWPNs show up

WINDOWS DISCOVERY OF TARGETS





Binding

Binding or mapping in FC SAN is the process of associating an OS device name to a target’s worldwide port name

Persistent binding in FC SAN is the process of ensuring that the same binding occurs even after a host OS reboot

NOTE: Binding is done automatically in most modern OS; therefore, it does not need to be manually configured

BINDING





Window/Emulex Binding

Select

Current Bindings

CurrentSCSI IDs

Do not use persistent

bindingSCSI ID = Bus Number, Target ID

From Device Properties ...

WINDOW/EMULEX BINDING There are no native Windows Server 2003 or 2008 tools for verifying binding of an initiator and one or more targets. Therefore, an administrator must use third-party tools such as HBAnyware from Emulex.

NOTE: Do not use FC-persistent binding within a Windows SAN environment.





Command-Line Interface for Server Core

Command-line interface (CLI) commands for FC management is available for the HBA vendors– Emulex: hbacmd– QLogic: scli

Commands available to:– Verify connectivity– Interrogate the fabric– Manage bindings– Verify configuration– Administrate VPORTs (discussed in module 7)

COMMAND-LINE INTERFACE FOR SERVER CORE





Multipath I/O

MULTIPATH I/O





Multiple Paths to a LUN

Fibre Channel

0c - 0

76

0d - 1 0c - 2 0d - 3

switchport



MULTIPLE PATHS TO A LUN





Multiple Paths

When multiple paths are present to a LUN, the same LUN would appear multiple times– The same LUN would appear a single instance for

each path available

Without MPIO With MPIO

MULTIPLE PATHS





Optimized or Non-optimized Paths

Not all paths are necessarily equal– Optimized or primary or favored = active path

between initiator and target Same latency level

– Non-optimized or secondary or unfavored = inactive path between initiator and target Latency differs between path

OPTIMIZED OR NON-OPTIMIZED PATHS





Multipath Access

Symmetric– All paths are favored

or optimized

Asymmetric– Only certain paths are

favored or optimized

0c 0d

LUN

0c 0d

But how do you determine

optimized or non-optimized

paths?

MULTIPATH ACCESS





Windows Multipath FC Implementation

Storage management software available for Windows:– Native Windows Disk Management– Veritas™ Storage Foundation

Native Disk Management in W2K8 will use ALUA by default with Microsoft native Device Specific Module (DSM); enable AULA on the igroupsystem> igroup set my_igroup alua on

When possible, use the NetApp DSM and not enabled AULA on the igroupsystem> igroup set my_igroup alua off

This course focuses on native Disk Management, NetApp DSM and Emulex HBAs

Pre-defined igroup Default is off

WINDOWS MULTIPATH FC IMPLEMENTATION





Windows Native Multipath I/O

Windows Server 2008 can be configured to support Multipath I/O

Right-click Features and

select Add Feature Multipath I/O

added

No need for a reboot in R2

WINDOWS NATIVE MULTIPATH I/O





Windows Native Multipath I/O (Cont.)

Out of the box, Windows multipath I/O comes with a generic DSM

NetApp recommends installing the NetApp Data ONTAP DSM– Verify with Interoperability Matrix for the

recommended version

WINDOWS NATIVE MULTIPATH I/O (CONT.)





NetApp Data ONTAP DSM 3.3.1

Features:– Supports Windows Server 2008 R2– Supports multiple load-balancing policies– Support for claiming iSCSI LUNs– Coexists with other DSMs– Multiprotocol LUN support (simultaneous iSCSI

and FC paths to the same LUN) Requirements:

– Windows Server 2003 or 2008 or 2008 R2– Data ONTAP 7.2.2+

NETAPP DATA ONTAP DSM 3.3.1 The Data ONTAP DSM for Windows MPIO is a Device Specific Module (DSM) that works with the Microsoft Windows MPIO drivers (mpdev.sys, mpio.sys, and mpspfltr.sys) to manage multiple paths between NetApp and IBM® N series storage systems and Windows host computers. The DSM includes the storage-system-specific intelligence needed to correctly identify paths and to manage path failure and recovery.





Install NetApp DSM

INSTALL NETAPP DSM





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe multiple path implementation with

Fibre Channel (FC) connectivity Describe how to configure FC ports on

Windows and NetApp systems Describe commands and utilities to identify the

worldwide node name (WWNN) and worldwide port name (WWPN) on Windows and NetApp systems

Use commands and utilities to examine FC switch activity

MODULE SUMMARY




Exercise

Module 2: Windows FC ConnectivityEstimated Time: 45 minutes



3-1 SAN Implementation Workshop: Windows iSCSI Connectivity


Windows iSCSI Connectivity


WINDOWS ISCSI CONNECTIVITY





Module Objectives


iSCSI connectivity Configure network ports on Windows® and

NetApp®systems Identify the worldwide node (WWN) on

Windows and NetApp systems Set up and verify multiple path iSCSI

connectivity between Windows and NetApp systems

MODULE OBJECTIVES





Ethernet IP TCP iSCSI SCSI

iSCSI encapsulates SCSI-3 command frames in IP packets

– Uses TCP port 3260– IETF standard documented RFC3720

http://www.ietf.org/rfc/rfc3720.txt?number=3720

The iSCSI standard specifies:– Connection negotiation– Authentication methods– Device discovery

iSCSI Review

ISCSI REVIEW





iSCSI Node Names Review

Symbolic name used to uniquely identify nodes (targets and initiators)

Two naming formats:– IQN (iSCSI Qualified Name): defined by RFC3720, a

DNS-based naming format

Example: iqn.1992-08.com.netapp:sn.101171811– EUI (Extended Unique Identifier): OUI-based scheme

similar to WWN

Example: eui.495f1de83eb8

Type Date Domain Domain OrganizationallyRegistered Name Defined

Type OUI + Organizationally Defined

ISCSI NODE NAMES REVIEW Valid characters for iSCSI node names are:

ASCII dash ('-') , dot ('.'), colon (':') ASCII lower-case characters - ('a' through 'z‘, '0' through '9')

Maximum size is 223 bytes No white space is allowed Upper case characters are converted to lower case IQN iqn.yyyy-mm.backward_naming_authority:device yyyy-mm is the month and year in which the naming authority acquired the domain name backward_naming_authority is the reverse domain name of the entity responsible for

naming this device device is the unique host name for the device Extended Unique Identifier (EUI) – “eui” + “.” + 16 hexadecimal digits NOTE: Older Microsoft iSCSI software initiator allowed an underscore character in the IQN name. Data ONTAP® complies with the iSCSI specification and will not recognize it.





IP Connectivity

IP CONNECTIVITY





IP Connectivity

Type of topologies:1. Direct-attached2. Network

NetApp FAS

IP CONNECTIVITY





Direct-Attached Topology

Also called point-to-point

Windows

iSCSI

DIRECT-ATTACHED TOPOLOGY In direct-attached topologies, servers (or hosts) are directly attached to the NetApp controller using a crossover cable. It is not possible to directly attach to more than one controller in a high-availability configuration.





Problems with Direct-Attached Topology

. . .

Does not scale

No faulttolerance

Windows Windows

iSCSI

PROBLEMS WITH DIRECT-ATTACHED TOPOLOGY Direct-attached topologies do not scale and are therefore not appropriate for enterprise environments. There is also no fault tolerance. If the cable or an adapter is defective, then a host will lose all connectivity with its storage.





Network Topology

Windows

iSCSI

Dual switches provides extra redundancy

Cluster the initiator for complete redundancy

NETWORK TOPOLOGY In a network environment, servers are attached to NetApp controllers through Ethernet switches. This network may consist of multiple Ethernet switches in any configuration.

There are two types of switched environments, dedicated Ethernet and shared Ethernet. In a dedicated Ethernet, there is no extraneous network traffic. The network is totally dedicated to iSCSI and related management traffic. Such a network is typically located in a secure data center. Direct-attached and dedicated Ethernet networks represent approximately 90% of current iSCSI deployments. In a shared Ethernet, the network is shared with other traffic or a corporate Ethernet network. This typically introduces firewalls, routers, and IP security (IPsec) into the Ethernet network.





iSCSI Adapters Types

Because iSCSI implements SAN over IP, administrators have choices on how to connect to an iSCSI network:

ServerProcessing

NIC/HBAProcessing

NIC and iSCSI Soft Initiator

Network Interface

IP

TCP

iSCSI

SCSI

Application

OtherProtocols

TOE and iSCSI Soft Initiator

Network Interface

IP

TCP

iSCSI

SCSI

Application

OtherProtocols

iSCSI Hardware Initiator/HBA

Network Interface

IP

TCP

iSCSI

SCSI

Application

ISCSI ADAPTERS TYPES There are three forms of iSCSI initiators: iSCSI software initiators with standard Ethernet network interface cards (NICs), iSCSI software initiators with Ethernet TCP offload engine (TOE) NICs, and iSCSI hardware initiators (host bus adapters, or HBAs). The iSCSI software initiator over standard Ethernet NIC solution is easy to install and has no extra costs as most modern hosts have one or two onboard gigabit Ethernet NICs standard. The drawback of this solution is that the host CPU must process iSCSI and TCP/IP protocol data and application performance may suffer due to the cost of the iSCSI and TCP/IP load.

The TOE offloads TCP from the host processor, allowing the host to focus on application processing. TOEs do not offload iSCSI processing from the host and therefore require an iSCSI software initiator.

With an iSCSI HBA, all of the functions associated with iSCSI are moved off the host processor(s) and onto a card, similar to an FC HBA or a SCSI HBA. This allows the host processor to work solely on the application, handing all storage-related work to the iSCSI HBA. The iSCSI hardware initiator solution is more expensive and has platform compatibility considerations. In addition, the iSCSI HBA may provide support for SAN boot or the ability to boot from the iSCSI LUN on the NetApp storage system. Each path may also have a different single point of failure, clustering support capabilities, platform support considerations, and integration with NetApp SnapDrive®.

NOTE: The iSCSI Support Matrix on the NOW™ site contains the most up-to-date information.





Topology Summary

To optimize connectivity:– Architect clustered hosts – Implement dual switches with multiple paths– Configure active-active storage systems

NOTE: The exercise environment will implement the high availability (active-active) storage system configuration with several multiple path techniques

TOPOLOGY SUMMARY





IP Exercise Environment

iSCSI

e0a



e0b e0c e0a e0b e0c

e0a are usedfor managementaccess only for

storage systemswithout e0M

IP EXERCISE ENVIRONMENT





Discovery

For an initiator and target to communicate, the initiator must discover the target

Proper configuration of the initiator OS is required for discovery

Discovery is accomplished over TCP port 3260 We will investigate:

– Data ONTAP setup– Windows Server 2003/2008 software initiator

with a standard NIC

DISCOVERY NOTE: Discovery occurs generally over TCP port 3260. Therefore, this port must not be blocked by firewall servers positioned between the initiator and target on a network.





Data ONTAP

DATA ONTAP





Data ONTAP as an iSCSI Target

Data ONTAP 6.4 and later has support for iSCSI Data ONTAP features:

– Built-in iSCSI service– Simple LUN creation and management

Data ONTAP must be properly configured for iSCSI connectivity1. Configure IP interfaces2. Configure iSCSI services3. Configure the iSCSI interfaces4. Identify the worldwide name (WWN)

DATA ONTAP AS AN ISCSI TARGET





Configuring Interfaces

1. List the available interfaces:– ifconfig -a

2. Take an interface offline:– ifconfig interface_name down

3. Configure the interface:– ifconfig interface_name ipaddress

4. Bring an interface online:– ifconfig interface_name up

NOTE:Virtual interfaces (interface groups) may also be configured to be used with the iSCSI service

CONFIGURING INTERFACES





Configuring iSCSI Services in Data ONTAP

1. Verify the iSCSI service is running:– iscsi status

2. Verify iSCSI is licensed (license it if needed):– license

– license add XXXXXX

3. Start the iSCSI service:– iscsi start

CONFIGURING ISCSI SERVICES IN DATA ONTAP





Verify Interfaces

Verify the interface is enabled for iSCSI: – iscsi interface show

– By default, all interfaces are enabled To enable the interface for iSCSI traffic:

– iscsi interface enable interface_name

To disable iSCSI traffic for a particular interface:– iscsi interface disable interface_name

VERIFY INTERFACES





Interface Access List

Administrators may force initiators to access a storage system through certain interfaces– iscsi interface accesslist add initiator_name {-a|interface_name}

– By default, all initiators may use any interface that is enabled for iSCSI traffic

– To display the current access list, use: iscsi interface accesslist show

– To remove an entry from the access list, use: iscsi interface accesslist remove initiator_name {-a|interface_name}

INTERFACE ACCESS LIST The accesslist feature controls initiator access to network interfaces. By default, an initiator does not have an access list and can access the storage system through any network interface. The administrator can create an access list using the add subcommand:

iscsi interface accesslist add initiator {-a | interface...}

This creates an access list for initiator with a list of network interface names.

The specified initiator can only log in through the network interfaces in the access list.

If the specified initiator sends a SendTargets request, it will receive a list of target addresses. These target addresses are associated with only those network interfaces that are included in the access list.

An existing access list can be edited by means of the add and remove subcommands:

iscsi interface accesslist remove <initiator> [-a | <interface>...]

The -a parameter adds or removes all network interfaces. When the last network interface is removed from an access list, the access list itself is removed.





Identifying WWN in Data ONTAP

To identify the WWN:system> iscsi nodename

iSCSI target nodename: iqn.1992-08.com.netapp:system

Remember WWPNs are not used within iSCSI

IDENTIFYING WWN IN DATA ONTAP





Interfaces on the Storage System

Verify interface status:system> ifconfig -a

...e0b: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500ether 00:a0:98:03:28:8f (auto-unknown-down) flowcontrol full

e0c: flags=108042<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500ether 00:a0:98:03:28:8f (auto-unknown-down) flowcontrol full

...

Enable the interface:system> ifconfig e0b 10.254.134.75 up

...[system: netif.linkUp:info]: Ethernet e0b: Link up.system> ifconfig e0c 10.254.134.81 up

...[system: netif.linkUp:info]: Ethernet e0c: Link up.

Ensure that the iSCSI service may use the interface:system> iscsi interface enable e0b e0c

INTERFACES ON THE STORAGE SYSTEM





Windows

WINDOWS





Windows as an iSCSI Initiator

NetApp has supported Windows as an iSCSI initiator OS since Windows 2000 Server – Always check Interoperability Matrix Tool for current

supported Operating Systems Windows Server 2008 has many advantages over

previous versions– New tools such as Storage Explorer and Storage

Manager for SANs Windows Server 2008 R2 has many advantages over

Windows Server 2008– User interface enhancement and redesign– iSCSI boot support for up to 32 paths

Windows must be properly configured for iSCSI connectivity over a standard network interface

WINDOWS AS AN ISCSI INITIATOR





Windows iSCSI Design and Installation

1. Verify host operating system releases, required patches, and NetApp iSCSI Host Utility Kit with the Interoperability Matrix Tool and FC and iSCSI configuration guides– Use System Properties Dialog

2. Identify and verify a network interface is properly configured or install a supported iSCSI HBA or TOE

3. In Windows Server 2008, the software initiator is preinstalled

4. Install compatible NetApp Windows Host Utility Kit– Provides Perl scripts to monitor and diagnose iSCSI on

Windows

WINDOWS ISCSI DESIGN AND INSTALLATION





Windows/NIC Implementation

After installation, to configure a Windowsstandard NIC software initiator implementation:1. Identify the local network interface(s) to use2. Verify iSCSI Initiator driver is enabled and the

service is started3. Identify the WWN for the local Windows host4. Identify which method of discovery to use and

enter the storage system’s portal IP address or iSNS address

5. Configure authentication security if necessary6. Verify discovery & log on to the storage system

WINDOWS/NIC IMPLEMENTATION





Windows/NIC Implementation (Cont.)

1. Identify and configure the local interfaces

WINDOWS/NIC IMPLEMENTATION (CONT.)






2. Verify iSCSI Initiatordriver is enabled and verify iSCSI Initiator service is started

Window Server 2008 R2 version shown. NOTE:In Windows Server 2003, the category is called SCSI and RAID controllers







iSCSI initiator may be configured through:– Storage Explorer– iSCSI Initiator Properties Dialog

Selectand thenconfigure

WINDOWS/NIC IMPLEMENTATION (CONT.) The Microsoft iSCSI software initiator may be configured through either Storage Explorer in Windows Server 2008 or the iSCSI Initiator Properties dialog in Windows Server 2003 or 2008.






3. Identify WWN using iSCSI Initiator Properties

Current WWN

To change the WWN

Select first

WINDOWS/NIC IMPLEMENTATION (CONT.) The local host WWN appears on the Configuration tab of the iSCSI Initiator Properties dialog. It generally does not need to be changed.





WNN Spoofing

iSCSI node names are:– Spoof-able– Sniff-able– Can be attacked

NetApp recommends using authentication methods and other security techniques (discussed later)

WNN SPOOFING





Discovery

Ethernet

Ethernet

Target

Initiator

Discovery is not automatic

DISCOVERY Unlike FC discovery, iSCSI discovery is not automatic.






4. Discovery with iSCSI Initiator Properties Dialog

Click here to discovera target portal

or add iSNSserver to poll

Add storage system’s IP address

To set security

Neverchange the port

WINDOWS/NIC IMPLEMENTATION (CONT.) From the Discovery tab on the iSCSI Initiator Properties dialog, an administration may set the method of discovery of targets. Discovery may be accomplished either through add-a-target’s portal address or by polling an Internet Storage Name Service (iSNS) server. See appendix 3 for a discussion of iSNS.

When adding a target portal address, if authentication is required, an administrator will set this by clicking the Advanced button. Next, iSCSI authentication is discussed in detail.





iSCSI Authentication in Windows

5.To increase security, iSCSI may be configured to require authentication– Authentication methods: CHAP

– Unidirectional - targets will authenticate initiators– Bidirectional - initiators and targets will authenticate

each other

RADIUS– IPsec could also be used increase security– This course will discuss using CHAP

authentication, but will not use it in the exercise

ISCSI AUTHENTICATION IN WINDOWS There are two methods of authenticating systems in iSCSI:

CHALLENGE HANDSHAKE AUTHENTICATION PROTOCOL (CHAP)

CHAP requires a known secret that is shared by both ends of the authentication. There are four basic steps of unidirectional authentication:

After the completion of the link establishment phases, the target sends a “challenge” message to the initiator.

The initiator responds with a one-way hash function of the shared secret. The target checks the response against its own calculation of the expected hash value. At random intervals, the target will send a new challenge to the initiator and repeat Steps

1, 2, and 3. NOTE: In bidirectional authentication the process is also implemented in the reverse.

REMOTE AUTHENTICATION DIAL-IN USER SERVICE (RADIUS)

RADIUS is a networking protocol that uses access servers to provide centralized management of access to large networks.

Proper authentication resists man-in-the-middle attacks as well as other attacks.

IPsec can also be used to increase security.





iSCSI Unidirectional CHAP Authentication

Configure the discoveryto use the CHAP authentication

To configure bidirectional, check here and then...

system> iscsi security add -i iqn.1991-05.com.microsoft:win -s CHAP -n iqn.1991-05.com.microsoft:win-p thisismysecret

Check to configure

ISCSI UNIDIRECTIONAL CHAP AUTHENTICATION To configure unidirectional (inbound) CHAP authentication for the Microsoft Software Initiator, enter the iscsi security add command on the storage system and enter the user name and shared secret as appropriate. By convention, the user name is generally the same as the WWN.

The Microsoft iSCSI Software Initiator requires both the initiator and target CHAP passwords to be at least 12 bytes if IPsec encryption is not being used. The maximum password length is 16 bytes regardless of whether IPsec is used.

NOTE: Data ONTAP provides an iscsi security generate command that creates a random 128-bit key that may, in some cases, be used as the password.





iSCSI Bidirectional CHAP Authentication

Set Windows CHAP secret

On the storage system:

Set CHAP secret from switch -o

system> iscsi security add -i iqn.1991-05.com.microsoft:win-s CHAP -n iqn.1991-05.com.microsoft:win-p thisismysecret-o thisismysecret2-m iqn.1991-05.com.microsoft:win

ISCSI BIDIRECTIONAL CHAP AUTHENTICATION To add bidirectional (inbound and outbound) authentication to the Microsoft software initiator, check the mutual authentication on the Advanced Setting (see previous page) and then add the CHAP secret from switch -o of the iscsi security add command.

NOTE: The user name is the same as the WWN but the shared secret is different. This is because the user name and password combination cannot be the same for inbound and outbound settings on a storage system.






6. Discovered in iSCSI Initiator Properties Dialog

Storage system is discovered

Bothtarget portals discovered

(New in Windows 2008 R2)






Binding

iSCSI binding or logging on is the process of creating a session between an initiator and a target

Persistent binding ensures that an initiator binds to a target after a reboot of the initiator OS

BINDING






Targets in the iSCSI Initiator Properties Dialog


Click here to connect

To change the interface that is used with which

to connect

Best practice:Checkboth







Connection in iSCSI Initiator Properties Dialog

NOTE: Console message appears

[system: iscsi.notice:notice]: ISCSI: New session from initiator iqn.1991-05.com.microsoft:dev05s2.development.netappu.com at IP addr10.254.144.75







Disconnect in iSCSI Initiator Properties Dialog

Select session and click

To disconnect allsession for

particular targetSelect first

WINDOWS/NIC IMPLEMENTATION (CONT.) To disconnect, select the session from the target Properties dialog and click Disconnect.






iSCSI persistent binding

WINDOWS/NIC IMPLEMENTATION (CONT.) While logging into a target, an administrator may optionally set the target as a favorite target that automatically logs in when the local host is booted.





Windows Server 2008 provides fully functional command-line interface (CLI)

Important for:– Windows Server 2008 Server Core– Scripting

Windows Server 2008 Server Core is a minimal server installation option; provides:– Low-maintenance– Limited attack surface– Limited functionality– Command prompt interface

NOTE: In the exercise environment, we will use PowerShell

Command-Line Interface Commands

COMMAND-LINE INTERFACE COMMANDS





iscsicli Command

Start the iscsicliutility:PS C:\Users\Adminstrator> iscsicli

Microsoft iSCSI Initiator Version 6.1 Build 7600

[iqn.1991-05.com.microsoft:win] Enter command o^C to exit

Verify the initiator:> listinitiators

Initiators List:

Root\ISCSIPRT\0000_0

ISCSICLI COMMAND





iscsicli Command (Cont.)

Add target portals:> qaddtargetportal 10.254.144.75

The operation completed successfully.

List target portals:> listtargetportals

Total of 1 portals are persisted:

Address and Socket : 10.254.144.75 3260

Symbolic Name :

Initiator Name :

Port Number : <Any Port>

Security Flags : 0x0

Version : 0

Information Specified: 0x0

Login Flags : 0x0

ISCSICLI COMMAND (CONT.)






List targets:> listtargets

Targets List:

iqn.1992-08.com.netapp:system

Verify discovery method:> targetinfo iqn.1992-08.com.netapp:system

Discovery Mechanisms :

"SendTargets:*10.254.144.75 0003260 Root\ISCSIPRT\0000_0 “







Log in to target (creates first connection):> qlogintarget iqn.1992-08.com.netapp:system

Session Id is 0xfffffa8002936018-0x4000013700000004

Connection Id is 0xfffffa8002936018-0x4

Verify discovery method:> targetinfo iqn.1992-08.com.netapp:system

Discovery Mechanisms :

"SendTargets:*10.254.144.75 0003260 Root\ISCSIPRT\0000_0 “







List session information:> sessionlistTotal of 1 sessionsSession Id : fffffa8002936018-4000013700000004Initiator Node Name : iqn.1991-05.com.microsoft:winTarget Node Name : (null)Target Name : iqn.1992-08.com.netapp:systemISID : 40 00 01 37 00 00TSID : 06 00Number Connections : 1

Connections:Connection Id : fffffa8002936018-4Initiator Portal : 0.0.0.0/3780Target Portal : 10.254.144.75/3260CID : 01 00







Additional commands available:– Session Management LogoutTarget

PersistentLoginTarget

ListPersistentTargets

RemovePersistentTarget

ChapSecret...

– Connection Management AddConnection

RemoveConnection

Other commands available






Multipath Techniques

MULTIPATH TECHNIQUES





Multiple Paths to a LUN

iSCSI

e0a



e0b e0c e0a e0b e0c



MULTIPLE PATHS TO A LUN





Multipath Access

Symmetric– All paths are favored

or optimized

Asymmetric– Only certain paths are

favored or optimized

10G 10G

LUN

10G 1G

LUN

MULTIPATH ACCESS





iSCSI Optimized/Non-optimized Paths

The iSCSI protocol generally doesn’t require AULA the mechanism of determining the optimized path is already present– Always verify when AULA is supported by using

the Interoperability Matrix However, you can set the path priority:

– Enable ALUA:system> igroup set igroup1 alua yes

– Set the path priority:system> iscsi tpgroup alua set e0c optimized

system> iscsi tpgroup alua set e0b non-optimized

igroupassociated with

initiator(s)

ISCSI OPTIMIZED/NON-OPTIMIZED PATHS





Techniques for Multipath iSCSI

At the initiator:– Link aggregation – Multiple Connections per Session (MCS) – Multipathing Technology Layer or

Multipath I/O (MPIO) At the target:

– Link aggregation (interface groups) See Data ONTAP Fundamentals course

– Target Portal Groups (TPGs) that are related to sessions and connections

– Set optimized or non-optimized paths

TECHNIQUES FOR MULTIPATH ISCSI





Link Aggregation

Implemented by 802.3ad IEEE spec

Called “teaming,” “channel bonding,” and “interface groups”

Pros:– Supports all network

protocols Cons:

– Not supported with Microsoft iSCSI software initiator

– Same path often used to avoid out-of-order delivery

Teamingdriver

One TCP connection via two paths

Disk-Class Driver

To storagesystem

SCSI Layer

iSCSI Initiator

TCP/IP

NIC

NIC Driver (GbE)

NIC

LINK AGGREGATION Host-side NIC Teaming (802.3ad) is supported by NetApp on many non-Windows hosts.





Multiple Connections per Session

MCS creates multipathsstarting at the iSCSI session layer

Pros:– Supported by Microsoft

software initiator 2.0+– No dependency on

Ethernet infrastructure Cons:

– Not supported by iSCSI HBAs

– Both initiator and target must support MCS

NICNIC

Disk-Class Driver

NIC Driver NIC Driver

TCP/IP TCP/IP

To storagesystem

SCSI Layer

iSCSI Initiator

MULTIPLE CONNECTIONS PER SESSION Multiple Connections per Session (MCS): MCS creates the multiple paths starting at the iSCSI session layer of the storage stack. Both the iSCSI initiator (host) and the iSCSI target (controller) need to support multi-connection sessions in order to configure sessions with multiple connections. MCS requires a software initiator. MCS should not be confused with the Microsoft Cluster Service or Microsoft Consulting Services.

See Appendix 4 for a discussion of MCS.





Multipathing Technology Layer (MPIO)

“Classic” method of multipathing

Pros:– Achieve multipath with

FC and iSCSI– Supports multiple load-

balancing algorithms– Support software and

hardware initiators Cons:

– Usually requires add-on software (Device Specific Module)

NICNIC

Disk-Class Driver

TCP/IP

NIC Driver

TCP/IP

NIC Driver

To storagesystem

Multipathing

Technology Layer

SCSI Layer

iSCSI Initiator iSCSI Initiator

MULTIPATHING TECHNOLOGY LAYER (MPIO) Multipath Input/Output (MPIO): The "classic" way to do multipathing is to insert a separate multipathing layer into the storage stack. This method is not specific to iSCSI or to any underlying transport, and is the standard way to achieve multipathing access to Fibre Channel and even parallel SCSI targets. There are multiple implementations of this type of multipathing on the various operating systems. The MPIO infrastructure offered by Microsoft is the standard way to do this on Windows Server technologies. With the Microsoft MPIO, each storage vendor supplies a device-specific module for its storage array.

ONTAP DSM: In the past, the ONTAPDSM (previous called NTAP DSM) was bundled with SnapDrive. Beginning with SnapDrive 4.2 and later, there is a separate install for the ONTAP DSM.

Microsoft iSCSI DSM: Microsoft iSCSI DSM is supported in active-passive and active-active modes. It requires a software initiator.

VERITAS™ DSM: Veritas DSM is supported beginning with Windows Server 2003.





Target Portal Groups

Target Portal Group (TPG) One or more interfaces assigned to a TPG

– Interfaces may be physical or virtual By default, one interface per TPG One iSCSI session per TPG When an initiator discovers a target, a session

will be established with the TPG

TARGET PORTAL GROUPS A target portal group is a set of one or more storage system network interfaces that can be used for an iSCSI session between an initiator and a target. A target portal group is identified by a name and a numeric tag.

For iSCSI sessions that use multiple connections, all of the connections must use interfaces in the same target portal group. Each interface belongs to one and only one target portal group. Interfaces can be physical interfaces or logical interfaces (VLANs and vifs).

Starting with Data ONTAP 7.1, you can explicitly create target portal groups and assign tag values.

Because a session can use interfaces in only one target portal group, you may want to put all of your interfaces in one large group. However, some initiators are also limited to one session with a given target portal group. To support MPIO, you need to have one session per path, and therefore more than one target portal group.

If you do not plan to use multi-connection iSCSI sessions, you do not need to create target portal groups.

If you do plan to use multi-connection sessions, create a target portal group that contains all of the interfaces you want to use for one iSCSI session. However, note that you cannot combine iSCSI hardware-accelerated interfaces with standard iSCSI storage system interfaces in the same target portal group.





Connections Versus Sessions

Four sessions (4 TPG) with One connection (1 interface) each

CONN 68/3

CONN 68/4

CONN 68/2

CONN 68/1SESSION 68 One session (1 TPG) with

Four connections (4 interface)

SESSION 64 CONN 64/1




Microsoft’s Multiple Connections per Session (MCS)

Microsoft’s Multipath I/O (MPIO

CONNECTIONS VERSUS SESSIONS The diagram provides two examples illustrating sessions and connections, and how they relate to each other.

The top example shows four sessions each with one connection. Microsoft MPIO would use a session and connection configuration like this. The MPIO software manages how the sessions are used to move data between the host and the target LUNs.

The bottom example shows four connections within a single session. Both examples have four connections. The difference in the examples is in the number of sessions.





Target Portal Groups (Cont.)

List Target portal groups:system> iscsi tpgroup show

TPGTag Name Member Interfaces

1000 e0a_default e0a

1001 e0b_default e0b

1002 e0c_default e0c

1003 e0d_default e0d

No need to change the default unless supporting MCS (see Appendix 4)

TARGET PORTAL GROUPS (CONT.)





Multipath I/O

MULTIPATH I/O





Windows Multipath iSCSI Implementation

Windows supports two techniques of iSCSI multipathing:– Device Specific Module (DSM) for Multipath IO – MCS (see Appendix 4)

Windows requires an initiator that can be either:– An iSCSI HBA initiator– A software initiator Microsoft software initiator is generally used

– Windows Server 2003 - must install software initiator– Windows Server 2008 - software initiator preinstalled

Storage management software available for Windows:– Native Windows Disk Management– Veritas Storage Foundation

This course will focus on a MPIO native Disk Management with the Microsoft software initiator implementation

WINDOWS MULTIPATH ISCSI IMPLEMENTATION





iSCSI MPIO on Windows

To enable iSCSI MPIO on Windows Server:1. Install and/or enable NICs on initiator and

target2. Verify IP connectivity3. Add Multipath I/O feature4. Install NetApp DSM5. Allow Microsoft software initiator to discover

the target portals6. Create two sessions (different target portal

groups) with the target7. Verify multiple paths

ISCSI MPIO ON WINDOWS





Add Multipath I/O Feature

No need for a reboot in R2

ADD MULTIPATH I/O FEATURE





Windows Native Multipath I/O (Cont.)

Out of the box, native multipath I/O comes with a generic Device Specific Module (DSM)

NetApp recommends installing the NetApp Data ONTAP DSM– Verify with Interoperability Matrix for

recommended version

WINDOWS NATIVE MULTIPATH I/O (CONT.)





NetApp Data ONTAP DSM 3.3.1

Features:– Supports Windows Server 2008 R2– Supports multiple load-balancing policies– Support for claiming iSCSI LUNs– Coexists with other DSMs– Multiprotocol LUN support (simultaneous iSCSI

and FC paths to the same LUN) Requirements:

– Windows Server 2003 or 2008 or 2008 R2– Data ONTAP 7.2.2+

NETAPP DATA ONTAP DSM 3.3.1





First Session Created

Initiator has now connected with the first target portal

Target Portal Group ID

First session

Click to add another session

system> iscsi tpgroup showTPGTag Name Member

Interfaces1000 e0a_default e0a1001 e0b_default e0b1002 e0c_default e0c1003 e0d_default e0d

FIRST SESSION CREATED





Connect to the Second Portal

Connect to the second portal

Different initiator IP and target portal IP than first session

This functionality is available because multiple target portal groups were identified during discovery

CONNECT TO THE SECOND PORTAL





Second Session Created

Initiator has now connected with the both target portals

Target portal group ID

Second session

system> iscsi tpgroup showTPGTag Name Member

Interfaces1000 e0a_default e0a1001 e0b_default e0b1002 e0c_default e0c1003 e0d_default e0d

SECOND SESSION CREATED





Module Summary

MODULE SUMMARY





Module Summary


iSCSI connectivity Configure network ports on Windows and

NetApp systems Identify the worldwide node (WWN) on

Windows and NetApp systems Set up and verify multiple path iSCSI

connectivity between Windows and NetApp systems

MODULE SUMMARY




Exercise

Module 3: Windows iSCSI ConnectivityEstimated Time: 30 minutes



4-1 SAN Implementation Workshop: Windows LUN Access


Windows LUN Access


WINDOWS LUN ACCESS





Module Objectives

By the end of this module, you should be able to: Describe the steps to allow a Windows®Server

2008 R2 initiator to access a LUN on a storage system

MODULE OBJECTIVES





LUN Access Overview

LUN ACCESS OVERVIEW





LUN Access

To connect an initiator to a target’s LUN:1. Create an igroup if necessary2. Create the LUN3. Map the LUN to the igroup4. Find the LUN on the initiator 5. Prepare the LUN as a new disk on the

initiator

LUN ACCESS





1. Create an igroup


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Place WWN (IQN or eui) in igroups for iSCSI

Place WWPN in igroups for FC

Target

Initiator

1. CREATE AN IGROUP If necessary, create an igroup to provide access to a LUN.

Initiator groups (igroups) are tables of host identifiers (FC WWPNs or iSCSI WWNs) that are used to control access to LUNs. Typically, you want all of the host's host bus adapters (HBAs) or software initiators to have access to a LUN. If you are using multipathing software or have clustered hosts, each HBA or software initiator of each clustered host needs redundant paths to the same LUN.

You can create igroups that specify which initiators have access to the LUNs either before or after you create LUNs, but you must create igroups before you can map a LUN to an igroup.

Initiator groups can have multiple initiators, and multiple igroups can have the same initiator. However, you cannot map a LUN to multiple igroups that have the same initiator.

NOTE: An initiator cannot be a member of igroups of differing operating systems types (ostypes). Also, a given igroup can be used for FC or iSCSI, but not both.





Steps to Create an igroup

1. Optionally, verify initiators connectivity:– fcp show initiators

– iscsi initiators show

2. Create the igroup and place the initiators into the igroup:– igroup create {-i|-f} -t ostype

igroup_name[node, node…]

i = iSCSI igroupf = FC igroupostype = solaris, windows, hpux, aix, linux, netware, vmwarenode

– iSCSI type has worldwide node (WWN - IQN or eui)– FC type has worldwide port name (WWPN - may be aliased)

3. Verify the igroup:– igroup show

STEPS TO CREATE AN IGROUP Use the igroup create command to configure an igroup on a storage system. Note that you add nodes to the igroup and, therefore, the optional step of listing the currently connected initiators is provided in the first step.





Data ONTAP Configuration

Add WWPNs to igroup:system> igroup create -f -t windows iWIN_fcp

system> igroup add iWIN_fcp WIN1-FC WIN2-FC

Verify igroup:system> igroup show -v

iWIN_fcp (FCP) (ostype: windows):

10:00:00:00:c9:6b:77:b3 (logged in on: 0d, 0c)


10:00:00:00:c9:6b:77:b4 (logged in on: 0d, 0c)


NOTE:Connectedusing paths displayed

DATA ONTAP CONFIGURATION





2. Create a Logical Unit

LUNs may exists in any volumes or qtreeswith no NAS data

Ethernet

Ethernet

Fibre Channel

Fibre Channel

LUNbLUNaLUNa exists in vol1 in aggr1 /vol/vol1/LUNa.lun

LUNb exists in vol2 in aggr1 /vol/vol2/LUNb.lun

Target

Initiator

2. CREATE A LOGICAL UNIT LUNs may exist in any volume or qtree.When creating traditional or flexible volumes that contain LUNs, follow these guidelines:

Do not create any LUNs in the system’s root volume. Data ONTAP® uses this volume to administer the storage system. The default root volume is /vol/vol0.

Ensure that no other files or directories exist in a volume that contains a LUN. If this is not possible and you are storing LUNs and files in the same volume, use a separate qtree to contain the LUNs.

If multiple hosts share the same volume, create a qtree on the volume to store all LUNs for the same host. This is a recommended best practice that simplifies LUN administration and tracking.

Ensure that the volume option create_ucode is set to on (vol options <volname> create_ucode on). Data ONTAP requires that the path of a volume or qtree containing a LUN be in the Unicode format. This option is off by default when you create a volume. It is important to enable this option for volumes that will contain LUNs.

To simplify management, use naming conventions for LUNs and volumes that reflect their ownership or the way that they are used.





Steps to Create a LUN

Create the aggregate for the LUN:system> aggr create aggr_SAN 7

Create the volume for the LUN:system> vol create vol_SAN0 aggr_SAN 10g

Set Snapshot™ policy for the volume: (more on this in Module 13)

system> snap reserve vol_SAN0 0

system> vol options vol_SAN0 nosnap on

Optional: Create a qtree for the LUN:system> qtree create /vol/vol_SAN0/qtSAN0

STEPS TO CREATE A LUN





Steps to Create a LUN (Cont.)

1. Create a LUN– lun create -s size -t ostype lun_pathsize = in bytes by default

– Use m for megabytes– Use g for gigabytesNOTE: LUN sizing is discussed in detail in Module 13

ostype = solaris, windows, hpux, aix, linux, netware, and vmware, windows_gpt, windows_2008

lun_path

– LUN path begins with /vol/{VolumeName}/[qtreeName]– Last portion of path is the LUN Name– Example: /vol/vol_SAN0/qtSAN0/lun0

2. Verify the LUN– lun show

STEPS TO CREATE A LUN (CONT.) Use the lun create command to create a LUN.

The host ostype indicates the type of operating system running on the host that accesses the LUN, which also determines the following:

Geometry used to access data on the LUN Minimum LUN sizes Layout of data for multiprotocol access





3. Map a Logical Unit to an igroup


OS Type: Windows

My_FC_igroup21:00:00:2b:34:26:a6:56

OS Type: Windows

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Map a Logical Unit to an igroup by assigning a Logical Unit Number

1 2LUNbLUNa

NOTE: This step is also called LUN masking

Target

Initiator

3. MAP A LOGICAL UNIT TO AN IGROUP When you map the LUN to the igroup, you grant the initiators in the igroup access to the LUN. If you do not map a LUN, the LUN is not accessible to any hosts. Data ONTAP maintains a separate LUN map for each igroup to support a large number of hosts and to enforce access control.





Steps to Mask the LUN

1. Map a LUN to an igroup:– lun map lun_path igroup_name [lun_id]

lun_path = path name of a LUN igroup_name = name of an initiator group lun_id = unique identification number that the initiator

uses when the LUN is mapped to it– If not entered, automatically assigned

Example:lun map /vol/vol1/qtree1/luna My_IP_igroup 1

2. Verify the LUN mapping:– lun show –m

STEPS TO MASK THE LUN Use the lun map command to map an igroup to a LUN.

You map a LUN to an igroup by specifying the following attributes:

LUN NAME

Specify the path name of the LUN to be mapped.

INITIATOR GROUP

Specify the name of the igroup that contains the hosts that will access the LUN.

LUN ID

Assign a number for the LUN ID, or accept the default LUN ID. Typically, the default LUN ID begins with 0 and increases in increments of one as each additional LUN is created. The host associates the LUN ID with the location and path name of the LUN. The range of valid LUN ID numbers depends on the host. For detailed information, see the documentation provided with your host utilities kit.





LUN Creation Using Wizards

LUN CREATION USING WIZARDS





LUN Creation Using Wizards

You can ease the creation of LUN and igroupsusing wizards

Administrators can use:– lun setup command– SnapDrive® (discussed later in this module and

with the Linux® material)– NetApp® System Manager (discussed later in

this course)– Provisioning Manager

LUN CREATION USING WIZARDS





lun setup Command

LUN creation, optionally igroup creation, and igroup and LUN mapping may be accomplished with a single command: lun setup

– A wizard-like command that prompts the user for relevant information

– The result of the command is a newly created LUN mapped to a new or existing igroup

LUN SETUP COMMAND The lun setup command prompts you through the process of creating a LUN, creating an igroup, and mapping the LUN to an igroup. The volume where the LUN will reside must be created before running lun setup.





Creating a LUN with lun setup

system> lun setup

This setup will take you through the steps needed to create LUNs and to make them accessible by initiators. You can type ^C (Control-C)at any time to abort the setup and no unconfirmed changes will be made to the system.

Do you want to create a LUN? [y]: y

Multiprotocol type of LUN (solaris/windows/hpux/aix/linux/netware/vmware/windows_gpt/windows_2008)[linux]: windows

A LUN path must be absolute. A LUN can only reside in a volume or qtree root. For example, to create a LUN with name "lun0" in the qtree root /vol/vol1/q0, specify the path as "/vol/vol1/q0/lun0".

Enter LUN path: /vol/SAN/lun1

CREATING A LUN WITH LUN SETUP Multiprotocol type of LUN: Specify the OS that will be accessing the LUN.

LUN path: Specify the name of the LUN and where it will be located. This is referred to as the long_lun_path (example: /vol/SAN/lun1).





Creating a LUN with lun setup (Cont.)

A LUN can be created with or without space reservations being enabled.

Space reservation guarantees that data writes to that LUN will never fail.

Do you want the LUN to be space reserved? [y]: y

Size for a LUN is specified in bytes. You can use single-character multiplier suffixes: b(sectors), k(KB), m(MB), g(GB) or t(TB).

Enter LUN size: 1g

You can add a comment string to describe the contents of the LUN.

Please type a string (without quotes), or hit ENTER if you don't want to supply a comment.

Enter comment string: Windows LUN

CREATING A LUN WITH LUN SETUP (CONT.) Space reservations: Specify whether you want the LUN created with space reservations enabled. By default, space reservations are enabled.

LUN size: Specify the size of the LUN.

Comment: Create a comment or a brief description about the LUN.






The LUN will be accessible to an initiator group. You can use an existing group name, or supply a new name to create a new initiator group. Enter '?' to see existing initiator group names.

Name of initiator group []: ?

No existing initiator groups.

Name of initiator group []: iWIN_fcp

Type of initiator group iWIN_fcp (FCP/iSCSI) [FCP]: FCP

CREATING A LUN WITH LUN SETUP (CONT.) Initiator group name: Create or specify an igroup.

Type of initiator group: If you entered a new igroup name, specify which protocol will be used by the hosts in the igroup.






A Fibre Channel Protocol (FCP) initiator group is a collection of initiator port names. Each port name (WWPN) is 16 hexadecimal digits, separated (only) by optional colon (:) characters. You can separate port names by commas. Enter '?' to display a list of connected initiators. Hit ENTER when you are done adding port names to this group.

Enter comma separated portnames: ?


Portname Group

10:00:00:00:c9:2d:9f:76

10:00:00:00:c9:2d:a0:63

CREATING A LUN WITH LUN SETUP (CONT.) FCP: Entering a question mark (?) for portname will display all the initiators connected by way of FCP. This includes initiators on any hosts accessed using FCP.

iSCSI: The (?) will not display iSCSI initiators that are connected. To see the iSCSI initiators connected, run the command iscsi show initiator from the storage system.






Enter comma separated portnames: 10:00:00:00:c9:2d:9f:76

Enter comma separated portnames: <CR>

The initiator group has an associated OS type. The following are currently supported: solaris, windows, hpux, aix, linux, netware or vmware.

OS type of initiator group "iWIN_fcp" [windows]: windows

The LUN will be accessible to all the initiators in the initiator group. Enter '?' to display LUNs already in use by one or more initiators in group "iWIN_fcp".

LUN ID at which initiator group "iWIN_fcp" sees "/vol/SAN/lun1" [0]: 1

CREATING A LUN WITH LUN SETUP (CONT.) Enter port names: Enter the port names for initiators you want to include in your initiator group.

Select an OS type for the initiator group: OS type governs the finer details of SCSI protocol interaction with the initiators, including cluster failover behavior.

LUN ID: LUN ID allows LUN to igroup mapping. The lun setup command will use the next available ID by default.






LUN Path : /vol/SAN/lun1

OS Type : windows

Size : 1.0g (1077511680)

Comment : Windows LUN

Initiator Group : iWIN_fcp

Initiator Group Type : FCP

Initiator Group Members : 10:00:00:00:c9:2d:9f:76

Mapped to LUN-ID : 1

Do you want to accept this configuration? [y]: y

Do you want to create another LUN? [n]: n

CREATING A LUN WITH LUN SETUP (CONT.) Verify the configuration: Verify the configuration you have entered, and then accept or reject the configuration by typing y (yes) or n (no). Select n when asked if you want to create another LUN in order to exit lun setup.





Windows Setup

WINDOWS SETUP





Windows Steps

To connect an initiator to a target’s LUN:1. Create an igroup2. Create the LUN3. Map the LUN to the igroup4. Find the LUN on the initiator5. Prepare the LUN as a new disk on the

initiator

WINDOWS STEPS





4. Find the LUN on Windows

Using the Disk Management Tool, “Rescan Disks”

Right-click and choose “Rescan Disks”

Select DiskManagement

4. FIND THE LUN ON WINDOWS On a Windows host, you must partition and format any new LUN. To perform these tasks on Windows, use the Disk Management tool. First, access Computer Management by right-clicking My Computer and selecting Manage. Select Disk Management.

NOTE: The Disk Management tool will see and treat the LUN as though it is a local disk.

Within Computer Management, expand Storage and double-click Disk Management. In order for Disk Management to discover the new LUNs (virtual disks), select Action > Rescan Disks. From the Action menu, you can see tasks that may be performed on the new disk(s).





The LUN appears

4. Find the LUN on Windows (Cont.)

The LUN is offline

4. FIND THE LUN ON WINDOWS (CONT.)





Make the LUN online

5. Preparing the LUN for Windows

NOTE: The LUN is not initialized

Right-click,select Online

5. PREPARING THE LUN FOR WINDOWS The disk must be brought online before we can use it. To bring the disk online, right-click “Disk #” and select Online.





5. Preparing the LUN for Windows (Cont.)

Initialize the LUN in Windows Server 2008

Right-click,select

Initialize

NOTE: MBR is the default

5. PREPARING THE LUN FOR WINDOWS (CONT.) The disk must be initialized before it is formatted and partitioned. To initialize a disk, right-click “Disk #” and select Initialize. The administrator may select either a Master Boot Record (MBR) or a GUID Partition Table (GPT).





Partition Styles

Master Boot Record (MBR)– Traditional style– Uses partition table on the

first sector of disk– Supports up to 2 TB unless

dynamic disks are used– Two types:

Primary: used to format and mount directly

Extended: used to create logical drives (format and mount logical drives)

GUID Partition Table (GPT) – Supported in all versions of

Windows Server 2008 (default for 64-bit versions)

– Partition data is stored in redundant primary and backup tables

– Uses CRC32 checksum to verify partition data between tables

– Supports up to 18 EB and up to 128 partitions per disk

– Can’t be removable disks

PARTITION STYLES To convert from MBR to GPT, use convert gpt.

To convert from GPT to MBR, use convert mbr.






The LUN is now ready for provisioning Windows administrators may use either:

– Disk Management’s New Simple Volume Wizard in Windows Server 2008

– Disk Management’s Create Partition Wizard in Windows Server 2003

– Share and Storage Management Available on Windows Server 2008 or later One location for administrating shares and storage We will examine this tool in this module

– Storage Manager for SANs (see appendix) Available on Windows Server 2003 R2 or later Requires Data ONTAP VDS Hardware Provider,

downloadable at the NOW™ site

5. PREPARING THE LUN FOR WINDOWS (CONT.) The LUN is now ready for provisioning. Administrators may use one of several possible tools to configure the LUN.

DISK MANAGEMENT

Disk Management is the traditional method for provisioning LUNs. In Windows Server 2003, the Create Partition Wizard is available while in Windows Server 2008, the same wizard has been renamed as the New Simple Volume Wizard.

SHARE AND STORAGE MANAGEMENT

The Share and Storage Management tool is only available in Windows Server 2008 and later. Later in this presentation, we will review the Provision Storage Wizard within the new Share and Storage Management tool.

STORAGE MANAGER FOR SANS

Storage Manager for SANs helps you create and manage LUNs on Fibre Channel and iSCSI disk drive subsystems that support Virtual Disk Service (VDS) in your storage area network (SAN). See Appendix 5 for a discussion of using Storage Manager for SANs.





Basic or Dynamic Disks

Windows allows administrators to treat a LUN as single disk or combine it with other dynamic disks using a volume manager

Basic disk– Are like other disks– Used to manage a

LUN as a single disk

Dynamic disk types– Spanned– Striped– Mirrored– RAID-5

NOTE: We will first examine the basic disk configuration and thenlook at a spanned dynamic disk configuration

BASIC OR DYNAMIC DISKS






Windows Server 2008 has Disk Management’s New Simple Volume Wizard

Right-click,select

New Simple Volume

The wizard will launch

5. PREPARING THE LUN FOR WINDOWS (CONT.) In Windows Server 2003, to partition the disk, select Action > All Tasks > Create Partition or right-click the unallocated box and select Create Partition.

In Windows Server 2008, to partition the disk, select Action > All Tasks > New Simple Volume or right-click the unallocated box and select New Simple Volume.

After the disk is initialized, partitioned, and formatted, you should test access to the disk by navigating to it and creating a file.

NOTE: The presentation displays the Windows Server 2008 version of the wizard. There are only cosmetic changes between the Windows Server 2003 and Windows Server 2008 version.






New Simple Volume Wizard (Cont.)

Specify the method to mount

Specify the volume size

5. PREPARING THE LUN FOR WINDOWS (CONT.)





New Simple Volume Wizard (Cont.)


Verify the configurationand finish

Specify the format







Alternatively, a LUN may be formatted through the Share and Storage Management for Windows 2008

5. PREPARING THE LUN FOR WINDOWS (CONT.) The Share and Storage Management Tool is new in Windows Server 2008. This one tool allows administrators to provision storage and share it to network users through a single interface.






Navigate to the Volumes tab to investigate the current volume configuration

Click here

Select Provision Storageto configure the LUN

5. PREPARING THE LUN FOR WINDOWS (CONT.) The Provision Storage wizard is an alternative to the New Simple Volume Wizard of Windows Server 2008 or the Create Partition Wizard of Windows Server 2003.






Because our LUN is identified as a disk, select here

Then click Next







Select the appropriate disk and then select Next







Determine the size of your volume and

select Next







Determine how to mount the new

volume and select Next







Determine whether to format the volume and then select Next







Review, and if correct, select

Create







NOTE: You may want to share out the new

volume

5. PREPARING THE LUN FOR WINDOWS (CONT.) NOTE: The Provision Storage Wizard can launch the Share Storage Wizard after completing.





The new volume is now available for use







4. Find the LUN with CLI

Start DiskPart utility:PS C:\Users\Adminstrator> diskpart

Rescan for disks:DISKPART> rescan

List current disks visible:DISKPART> list disk

Disk ### Status Size Free DynGpt

-------- ------------- ------- ------- --- ---

Disk 0 Online 68 GB 0 B

Disk 1 Online 2055 MB 1088 KB *

Disk 2 Offline 2055 MB 2055 MB

The discovered LUN is offline

4. FIND THE LUN WITH CLI





5. Preparing the LUN with CLI

Select the offline disk (LUN)DISKPART> select disk 2

Online the disk DISKPART> online disk

List details on selected diskDISKPART> detail diskNETAPP LUN Multi-Path

Disk DeviceDisk ID: 00000000Type : iSCSIStatus : OnlinePath : 0Target : 0LUN ID : 1Location Path : UNAVAILABLE

Current Read-only State : YesRead-only : YesBoot Disk : NoPagefile Disk : NoHibernation File Disk :NoCrashdump Disk : NoClustered Disk : NoThere are no volumes.

NOTE: The LUN defaultsto read-only

5. PREPARING THE LUN WITH CLI





5. Preparing the LUN with CLI (Cont.)

Make the disk writableDISKPART> attributes disk clear readonly

Create a primary partitionDISKPART> create partition primary

List partitionsDISKPART> list partition

Partition ### Type Size Offset

------------- ---------------- ------- -------

Partition 1 Primary 2054 MB 64 KB

Identify the partition number

5. PREPARING THE LUN WITH CLI (CONT.)






Select partition 1DISKPART> select partition 1

List the details on selected partitionDISKPART> detail partition

Partition 1

Type : 06

Hidden: No

Active: No

Offset in Bytes: 65536

Volume ### Ltr Label Fs Type Size Status Info

--------------------------------------------------------

* Volume 4 Partition 2054 MB Healthy Offline

Volume already created







Mount disk DISKPART> assign mount=“C:\Data\CLI”

Format partitionDISKPART> format fs=NTFS label=“My CLI Volume” quick

List volumesDISKPART> list volume

Volume ### Ltr Label Fs Type Size Status Info

-----------------------------------------------------------------

Volume 0 D DVD-ROM 0 B No Media

Volume 1 Partition 100 MB Healthy Offline

Volume 2 C NTFS Partition 68 GB Healthy Boot

Volume 3 E NTFS Partition 2022 MB Healthy

* Volume 4 C:\Data\CLI\ NTFS Partition 2054 MB Healthy

Exit DiskPartDISKPART> exit

Pre-created folder






Dynamic Disks

In this example, two 5-GB LUNs were created

LUNs broughtonline

Right-click,select

Initialize

DYNAMIC DISKS





Dynamic Disks (Cont.)

Convert LUNs to dynamic disks

LUNs initialized

Right-click,select

Convert toDynamic Disk...

DYNAMIC DISKS (CONT.)






Start the New Spanned Volume Wizard

LUNs are dynamic

Right-click,select New SpannedVolume

Wizard....







New Spanned Volume Wizard (Cont.)Added disk 2 to spanned

volume







New Spanned Volume Wizard (Cont.)

Selected disk 2

Used only 2 GB of the total 5 GB

Total size of spanned volume








Format volume

Specify the mount point








Review the settings

Click to create the spanned volume







New Spanned Volume Created

Volume Eis a

spannedvolume






Windows Stack

Format LUN

Disk 1 Disk 2File System Added

E:Mounts

SCSI\DISK&VEN_NETAPP&PROD_LUN\4&61E00BC&0&000001SCSI Devices Path

F:

Port 3,4 Bus 0, Target 0, LUN1 Port 3,4 Bus 0, Target 0, LUN2

SCSI\DISK&VEN_NETAPP&PROD_LUN\4&29BC0C71&0&000100

LUN 1

LUN 1

LUN 2

NTFS

LUN 2

NTFS

Spanned E:

WINDOWS STACK





File Services Role

After the LUN is available, the File Service role may be added to Windows Server 2008

Roles addedthrough

Server Manager

FILE SERVICES ROLE The File Services server role in the Windows Server 2008 operating system provides technologies that help manage storage, enable file replication, manage shared folders, ensure fast file searching, and enable access for UNIX® client computers. See Microsoft®’s TechNet Web site for more details:

http://technet.microsoft.com/en-us/library/cc730983(WS.10).aspx.




Exercise

Module 4: Windows LUN AccessTasks 1-6

Estimated Time: 45 minutes

EXERCISE Please refer to your Exercise Guide for more instruction.





SnapDrive for Windows

SNAPDRIVE FOR WINDOWS





SnapDrive software provides:– Simple storage provisioning of LUNs (virtual

disks)– Consistent data Snapshot copies– Automation for backups and recoveries

SnapDrive is available for:– Solaris™– Windows – AIX®

This course investigates SnapDrive 6.1 for Windows

SnapDrive

– HP-UX®

– Linux

SNAPDRIVE






SNAPDRIVE FOR WINDOWS





SnapDrive 6.1 for Windows

Features:– Supports Windows Server 2008 R2– Runs as a Windows service– Communicates through RPC, HTTP, or HTTPS RPC requires CIFS to be set up

– Integrates with Performance Manager– Supports VMware® Guest OS features such as

VMotion™ Two Interfaces:

– Command-line interface (CLI) sdcli.exe

– Graphical user interface (GUI) Integrates into Microsoft Management Console (MMC)

SNAPDRIVE 6.1 FOR WINDOWS SnapDrive for Windows software integrates with the Windows volume manager so that storage systems can serve as storage devices for application data in Windows Server 2003 and 2008 environments.

SnapDrive manages LUNs on a storage system, making this storage available as local disks on Windows hosts. This allows Windows hosts to interact with the LUNs just as if they belonged to a direct-attached disk array.

SnapDrive for Windows provides the following additional features:

It enables online storage configuration, LUN expansion, and streamlined management. It integrates Snapshot technology, which creates point-in-time images of data stored on

LUNs. It works in conjunction with SnapMirror® software to facilitate disaster recovery from

asynchronously mirrored destination volumes. When used with Data ONTAP 7.1 or later, it allows for fractional reserve monitoring and

rapid LUN restoration.





Install SnapDrive 6.1 for Windows

Requires .NET Framework 3.5 SP1 Requires a valid license key or a controller with

SnapDrive licensed Requires IP-to-hostname resolution Default ports on host:

– RPC 808– HTTP 4094– HTTPS 4095

Default ports on Data ONTAP:– RPC 445– HTTP 80– HTTP 443

INSTALL SNAPDRIVE 6.1 FOR WINDOWS When installing SnapDrive 6.0 for Windows, several hotfixes are required. See the documentation for details. .NET Framework 3.0 is also required. During configuration, an administrator chooses a default method to communicate between SnapDrive and a storage system. The default port for RPC is 808, 4094 for HTTP, and 4095 for HTTPS.





RPC Configuration

SnapDrive service runs under a local administrator’s account

The same administrator account must be configured as an administrator on the storage system

To communicate with a storage system, you can:– Add a domain user to the local user

account

– Or use pass-through authenticationsystem> useradmin user add SDService

-g Administrators

system> useradmin domainuser add Development\SDService -g Administrators

RPC CONFIGURATION NOTE: For HTTP or HTTPS configuration, you do not have to have the same user as the SnapDrive service.






Integrates into MMC

SNAPDRIVE FOR WINDOWS The SnapDrive for Windows interface is an MMC snap-in that appears under the Storage node by default.





Storage System Management

Configure the storage systems to manage

Select

Select

1

2

STORAGE SYSTEM MANAGEMENT To manage a storage system from SnapDrive, Select Storage System Management and then add the appropriate storage system.





SnapDrive Services

Local SnapDrive Server

SNAPDRIVE SERVICES





LUN Creation with SnapDrive for Windows

Select Disk and then Create Disk

Select

Select

1

2

LUN CREATION WITH SNAPDRIVE FOR WINDOWS LUNs managed by SnapDrive are located under Disks. Select Create Disk to start the Create Disk Wizard.





Create Disk Wizard

Select a predefined storage system

CREATE DISK WIZARD





Create Disk Wizard (Cont.)

CREATE DISK WIZARD (CONT.)












CREATE DISK WIZARD (CONT.) NOTE: This wizard page will only appear if properties need to be altered on the storage system’s volume.


















Success

CREATE DISK WIZARD (CONT.) To view the LUNs (virtual disks) created with SnapDrive, expand SnapDrive, the local SnapDrive services, and Disk. The LUN appears with a series of identifiers.

In the example, the Disk Identification is listed as LUN [4,0,0,0](T:\). The number 4 refers to the SCSI port that is being used to maintain a connection to the LUN. The first 0 refers to the bus number. The second 0 refers to the target ID, and the third 0 refers to the LUN number assigned by either the storage system or SnapDrive. Finally, the letter T is the local drive letter from which the disk may be accessed.

NOTE: This information is very valuable when troubleshooting FC and iSCSI-based SANs.





Snapshot Creation with SnapDrive

Created a file on the virtual disk

SNAPSHOT CREATION WITH SNAPDRIVE





Snapshot Creation with SnapDrive (Cont.)

Select

SNAPSHOT CREATION WITH SNAPDRIVE (CONT.)





Snapshot Creation with SnapDrive (Cont.)

SNAPSHOT CREATION WITH SNAPDRIVE (CONT.)





Snapshot Restore with SnapDrive

Delete a file on the virtual disk

The test fileis deleted

SNAPSHOT RESTORE WITH SNAPDRIVE





Snapshot Restore with SnapDrive (Cont.)

Select theSnapshot

Click Restore Disk

1

2

SNAPSHOT RESTORE WITH SNAPDRIVE (CONT.)






Performs a single file Snapshot restore SnapRestore® must be licensed






Verify the file is restored on the virtual disk


The test file is back






LUN Removal with SnapDrive for Windows

To remove,right-click and select Delete Disk

LUN REMOVAL WITH SNAPDRIVE FOR WINDOWS




Exercise

Module 4: Windows LUN AccessTasks 7-10Estimated Time: 40 minutes






Multipath I/O in Windows

MULTIPATH I/O IN WINDOWS





Windows MPIO

LUNa

Device (LUN) arrives

Adapter/PCI busdiscovers/detects

PnP/Disk Manager Port Driver

DSM modules

interrogation

Microsoft DSMclaims device

NetApp DSMclaims device

MSFT DSM claims

device?

Is thepath

known?

Expose pseudo deviceof real device (with MPIO) Don’t expose device

Yes

No

Yes NoCreate pseudo

deviceGroup under pseudo

node/device

WINDOWS MPIO





Verify Paths

The LUN should appear in the DSM Interface

Optionally, set load-balancing policy

Right-click and pick from menuto select a load-balancing policy

VERIFY PATHS





Load-Balancing Policies

Least Queue Depth– High performance– Default Policy

Optimizes distribution of I/O load

LOAD-BALANCING POLICIES





Load-Balancing Policies (Cont.)

Least Weighted Paths– Active-Passive policy

Determines best path and assigns it as the active path

LOAD-BALANCING POLICIES (CONT.)






Auto Assign







Failover







Round Robin







Round Robin with Subset






Multiple Path MPIO Configuration

MULTIPLE PATH MPIO CONFIGURATION





MPIO Panel

Within Control Panel, the MPIO panel can be used to verify MPIO services

NetApp LUNmanaged by

Windows’ MPIO framework

MPIO PANEL





MPIO Panel (Cont.)

Registered DSMs: 2================+--------------------------------|-------------------|----|----|---|DSM Name | Version |PRP ||--------------------------------|-------------------|----|----|----|--||Microsoft DSM |006.0001.07600.16385|0||Data ONTAP DSM |003.0003.25090.093+--------------------------------|-------------------|----|----|----|--...Data ONTAP DSM==============MPIO Disk14: 04 Paths, Least Queue Depth,

SN: C4n3J4ROg/Mi Supported Load Balance Policies: FOO

Path ID State SCSI Address-----------------------------------------------------------0000000003000101 Active/Optimized

Adapter: Emulex LightPulse HBA - Storport MiniController: C1677CB400000000 ...

NOTE: Redacted output

Using NetApp DSM

MPIO PANEL (CONT.)




Exercise

Module 4: Windows LUN AccessTask 11-12 (optional Task 13)Estimated Time: 15 minutes






Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe the steps to allow a Windows® Server

2008 R2 initiator to access a LUN on a storage system

MODULE SUMMARY


5-1 SAN Implementation Workshop: vSphere Overview


vSphere Overview


VSPHERE OVERVIEW





Module Objectives

By the end of this module, you should be able to: Describe virtualization and how it can be used

to promote server efficiency Explain methods of mapping NetApp® storage

to vSphere™ datastores List the interfaces to administrate vSphere

MODULE OBJECTIVES





Virtualization Overview

Before virtualization:– Single OS image per machine– Software and hardware tightly

coupled– Running multiple applications on

same machine often creates conflict

– Underutilized resources– Inflexible and costly infrastructure

After virtualization:– Hardware-independence of

operating system and applications

– Virtual machines can be provisioned to any system

– Can manage OS and application as a single unit by encapsulating them into virtual machines

x86 Architecture

Operating System

Application

x86 Architecture

VMware Virtualization Layer

Application

Operating System

Application

Operating System

CPU Memory NIC Disk CPU Memory NIC Disk

CPU Memory NIC HBA DiskCPU Memory NIC HBA Disk

VIRTUALIZATION OVERVIEW Virtualization is described this way in VMware® materials: “Virtualization is an abstraction layer that decouples the physical hardware from the operating system to deliver greater IT resource utilization and flexibility.”

Virtualization allows multiple virtual machines with heterogeneous operating systems to run in isolation, side-by-side on the same physical machine.

Each virtual machine has its own set of virtual hardware (for example, RAM, CPU, NIC, and so on) upon which an operating system and applications are loaded.

VMware virtualizes servers; NetApp virtualizes storage.

There are other server virtualization applications, such as Microsoft® Virtual Server, or Xen in Red Hat® Enterprise Linux® 5.0.





Hosted architecture:– Installs and runs as an

application– Relies on host OS for device

support and physical resource management

Bare-metal (hypervisor) architecture:– Lean virtualization-centric

kernel– Service console for agents

and helper applications

Hosted and Bare-Metal Strategies

x86 Architecture

Host Operating System

Virtualization Layer

Application

Guest Operating SystemApplication

VMware Products:VMware Server, Workstation

VMware Products:VMware ESX

CPU Memory NIC HBA Disk

x86 Architecture


App

OS

Clustering Software

Service Console

App

OS

App

OS

App

OS


HOSTED AND BARE-METAL STRATEGIES Hosted: The VMware server or workstation software runs as an application on top of a hosting OS (typically on top of Windows®).

Bare-metal: The VMware ESX Server product is an OS in itself based on Linux. ESX runs directly on x86 hardware (no hosting OS).





Virtual Machine File System

(VMFS)

x86 Architecture



Server Consolidation

VM VM

Application

Operating System

CPU Memory NIC Disk

Application

Operating System

CPU Memory NIC Disk

.vmdk .vmdk

LocalDatastore

For an enterprisesolution,

cluster ESX and attach

NetApp storage

SERVER CONSOLIDATION With vSphere and NetApp storage, administrators may transform their data centers by converting physical machines-to-virtual machines (P2V).

vSphere stores virtual machine (VM) data in datastores. Datastores can be shared by more than one VM as shown in this diagram.

VMs can access their storage by way of:

Virtual disks (VMDKs) stored on the VMware File System (VMFS) accessed through FCP or iSCSI

VMDKs stored on NFS Raw device mappings (RDMs) by way of FC or iSCSI





vSphere or ESX 4.0

VMware vSphere is the fourth generation of the VMware’s ESX server

New in vSphere:– Native NIC iSCSI multipathing– New Pluggable Storage Architecture– VMDirectPath to assign a PCI adapter directly to

a virtual machine– Growing virtual disks and VMware vStorage

VMFS volumes while they are live NOTE: This course will use ESX 4.0 and

vSphere interchangeably

VSPHERE OR ESX 4.0 VMDirectPath I/O allows a guest operating system on a virtual machine to directly access physical PCI and PCIe devices connected to a host.





NetApp and vSphere

NETAPP AND VSPHERE





NetApp FAS Array

ESX Cluster

VMFS Datastore

Datastore

Flexible Volume

FC / FCoE / iSCSI

VMFS

VM1VDisk0

VM2VDisk0

VM3VDisk0

VM4VDisk0

1.vmx1.vmdk

2.vmx2.vmdk

3.vmx3.vmdk

4.vmx4.vmdk

LUN

VMFS DATASTORE The VMware Virtual Machine File System (VMFS) is a high-performance clustered file system that provides datastores, which are shared storage pools. VMFS datastores can be configured with LUNs accessed by Fibre Channel, iSCSI, or Fibre Channel over Ethernet. VMFS allows traditional LUNs to be accessed simultaneously by every ESX Server in a cluster.

VMFS provides the VMware administrator with a fair amount of independence from the storage administrator. By deploying shared datastores, the VMware administrator is free to provision storage to virtual machines as needed. In this design, most data management operations are performed exclusively through VMware vCenter Server.

This storage design can be challenging in the area of performance monitoring and scaling. Because shared datastores serve the aggregated I/O demands of multiple VMs, this architecture doesn’t natively allow a storage array to identify the I/O load generated by an individual VM. This issue can be exacerbated by spanning VMFS volumes across multiple LUNs.





NetApp FAS ArrayFlexible Volume

ESX Cluster

NAS Datastore

Datastore

NFS

VM1VDisk0

VM2VDisk0

VM3VDisk0

VM4VDisk0

1.vmx1.vmdk

2.vmx2.vmdk

3.vmx3.vmdk

4.vmx4.vmdk

NAS DATASTORE In addition to VMFS, vSphere allows a customer to leverage enterprise-class NFS servers in order to provide datastores with concurrent access by all of the nodes in an ESX cluster. This method of access is very similar to that with VMFS. NFS provides high performance, the lowest per-port storage costs (as compared to Fibre Channel solutions), and some advanced data management capabilities.

Deploying VMware with NetApp NFS datastores is the easiest means to integrate VMware virtualization technologies directly with the NetApp WAFL® (Write Anywhere File Layout) file system, our advanced data management and storage virtualization engine.

See appendix 8 for more information about NAS datastores.






ESX Cluster

Raw Device Mapping (RDM) Datastore

Datastore

LUN LUN LUN

FC / FCoE / iSCSI

.vmx/.vmdk files could be with the LUN or separate

VM1VDisk0

VM2VDisk0

VM3VDisk0

VMFS

1.vmx

1.vmdk

2.vmx

2.vmdk

*.vmx

*.vmdk

Open Read Write

RAW DEVICE MAPPING (RDM) DATASTORE ESX allows for virtual machines to have direct access to LUNs for specific use cases such as P2V clustering or storage vendor management tools. This type of access is referred to as a raw device mapping and can be configured with Fibre Channel, iSCSI, and Fibre Channel over Ethernet. In this design, ESX acts as a connection proxy between the VM and the storage array.

Unlike VMFS and NFS, RDMs are not used to provide shared datastores.

RDMs are an enabling technology for solutions such as virtual machine and physical-to-virtual-machine host-based clustering, such as with Microsoft Cluster Server (MSCS). RDMs provide traditional LUN access to a host. Therefore, they can achieve high individual disk I/O performance, and they can be easily monitored for disk performance by a storage array.





Datastore Comparison Table

Capability / Feature FC/FCoE iSCSI NFSFormat VMFS or RDM VMFS or RDM NetApp WAFLMaximum number of datastores or LUNs 256 256 64

Maximum datastore size 64 TB 64 TB 16 TB*Maximum LUN / NAS file system size 2 TB 2 TB 16 TB*

Recommended VMDKs per LUN / NAS file system 16 16 250

Optimal queue depth per LUN / file system 64 64 N/A

Available link speeds 4 and 8 Gb FC / 10 GbE 1 and 10 GbE 1 and 10 GbE

* Data ONTAP 8.0 supports up to 100 TB per aggregate

DATASTORE COMPARISON TABLE Differentiating what is available with each type of datastore and storage protocol can require considering many points. The following table compares the features available with each storage option.





vSphere Interfaces

VSPHERE INTERFACES





vSphere Graphical Interfaces

vCenter Server• Requires additional

license• Centralized

management of multiple ESX hosts

• Automatic detection of changes

• Runs as a Windows Service

vSphere Client• No additional license

required• Used to manage

single ESX host• No automatic

detection of changes• Installs to local

machine

vSphere Web• No additional license

required• Perform basic VM

management and configuration

• No automatic detection of changes

• http://esx_host/ui

VSPHERE GRAPHICAL INTERFACES





vSphere Client

DefaultDatastore

Selected

VSPHERE CLIENT





vSphere Command-Line Interface (CLI)

Service Console

• Requires shell user account access

• Root account disabled

CLI Package

• Runs in: • Windows Shell• PowerShell• Linux

vMA

• A pre-built VM hosted in ESX that contains the CLI Package

In the exercise, you will change this

VSPHERE COMMAND-LINE INTERFACE (CLI)





CLI Examples

If a CLI Package or vMA is used, commands must have connection information:esxcli --server esx_ip --username user

--password passwd ...

If the user logs in to the Service Console with the appropriate user account, no additional connection information is neededesxcli ...

Commands might differ between interfacesCLI Packages/vMA Service Console Description

vicfg-vmknic esxcfg-vmknic NIC managementvicfg-vswitch esxcfg-vswitch Virtual switch managementesxcli esxcli Storage plug-in management

CLI EXAMPLES





User Accounts Within vSphere

Right-click to add new

user

Selected

Strong passwords required

USER ACCOUNTS WITHIN VSPHERE





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe virtualization and how it can be used

to promote server efficiency Explain methods of mapping NetApp storage

to vSphere datastores List the interfaces to administrate vSphere

MODULE SUMMARY




Exercise

Module 5: vSphere OverviewTime Estimate: 30 minutes



6-1 SAN Implementation Workshop: vSphere iSCSI Connectivity


vSphere iSCSI Connectivity


VSPHERE ISCSI CONNECTIVITY





Module Objectives


iSCSI connectivity for vSphere™ and NetApp®

systems Configure network ports on vSphere systems Identify the worldwide node (WWN) on

vSphere systems Set up and verify multiple path iSCSI

connectivity between vSphere and NetApp systems

MODULE OBJECTIVES





IP Connectivity

IP CONNECTIVITY






iSCSI SAN Storage Virtualization

ESX ClusterVM1SCSI Controller

VM2SCSI Controller

HBA

iSCSI software initiatorVirtualization Layer

NICHBA

LAN

NIC

LANiSCSI

VDisk0 VDisk0

VMFS

1.vmx1.vmdk

2.vmx2.vmdk

LUN

ISCSI SAN STORAGE VIRTUALIZATION





vSphere Network Modeling

PhysicalEthernetAdapterDefinitions

vSphereEthernetModeling

VSPHERE NETWORK MODELING





vSphere Network Modeling (Cont.)

vSwitches model physical switches, NICs associated with vSwitch

Service Console communicates over vmnic0 with vswif0 IP address by defaultPort Groups define bandwidth limitation

and VLAN tagging policies for each member NIC

VMkernels support iSCSI or NFS stacks using PluggableStorage Architecture (PSA)

vmnics associate with an IP range

VSPHERE NETWORK MODELING (CONT.) Switch virtualization (vSwitch)

– Associating Physical NICs with virtual switch – Designate a subnet to the switch – Use separate switches to isolate VMs

Port virtualization (VMkernel) – Creates a port between VM(s) and network – Associate a specific switch’s NIC(s) to a particular IP address – NICs can be active, failover, or inactive – Multiple active NICs use teaming (IEEE x.x) – Has pluggable storage architecture (PSA)





VMkernel’s PSA

VMkernel

Pluggable Storage Architecture (PSA)

Native Multipathing Plug-in (NMP)

Third-Party Multiple

MultipathingPlug-ins (MPP)

Storage ArrayType Plug-in (SATP)

Path SelectionPlug-in (PSP)

VMKERNEL’S PSA vSphere has a new architecture that allows third-party multiple multipathing plug-ins (MPP) to take complete control of the path failover and load balancing operations for a specific storage devices. Alternatively,, Native Multipathing Plug-in (NMP) supports all storage arrays on the VMware storage hardware compatibility list (HCL) and provide default path selection algorithms for both failover (by way of the Storage Array Type Plug-in or SATP) and load balancing operations (by way of Path Selection Plug-in or PSP).





ESX Network Design for IP Storage

The goal is to design a network that:– Is redundant across physical switches– Uses multiple physical paths simultaneously– Can scale to additional physical interfaces

Two high-level options– Traditional Ethernet: two storage subnets,

multiple storage and host IPs– Cross-stack EtherChannel: one storage subnet,

multiple storage IPs, and IP load balancing

ESX NETWORK DESIGN FOR IP STORAGE When designing an ESX network for IP storage, the goal is a network that is redundant across physical switches and that can use multiple physical paths simultaneously. You also want a network that can scale to additional physical interfaces. There are two high-level ESX network designs that will meet this goal—one with and the other without cross-stack EtherChannel.

The design that uses Traditional Ethernet uses two storage subnets, with multiple storage and multiple host IPs.

The cross-stack EtherChannel design uses one storage subnet, with multiple storage IPs, and IP load balancing.





Traditional Ethernet

Multiple storage and ESX IPs are required ESX requires two VMkernel ports, each on a different subnet Storage node requires IP on each subnet

TRADITIONAL ETHERNET By contrast, the traditional IP storage design without cross-stack EtherChannel uses multiple storage IPs and multiple ESX IPs. There must be two VMkernel ports on the ESX server, each on a different subnet. In addition, the storage node needs an IP address on each subnet. This design uses single-mode vifs between the storage controllers and the Ethernet infrastructure.





Traditional Ethernet (Cont.)

VMkernel port NIC teaming properties for two VMkernel ports Connections are manually balanced by selecting a different active

adapter for each VMkernel port

TRADITIONAL ETHERNET (CONT.) When using the traditional IP storage design without cross-stack EtherChannel, open the NIC Teaming tab on each VMkernel port Properties screen and select the Link status only network failover option. Manually balance connections by first selecting Override vSwitch Failover Order and by selecting a different Active Adapter for each VMkernel port.





Traditional Ethernet (Cont.)

Distribute datastores across storage IP addresses

TRADITIONAL ETHERNET (CONT.) This slide shows IP storage and cross-stack EtherChannel with datastores distributed across storage IP addresses.





Subnet Failover on Link Failure

On link failure, the affected subnet or connection moves to the other NIC, sharing the link

SUBNET FAILOVER ON LINK FAILURE On link failure, the affected subnet or connection moves to the other NIC and shares the link.





Cross-Stack EtherChannel

Multiple storage IPs are required ESX host requires one VMkernel port

CROSS-STACK ETHERCHANNEL This slide shows IP storage and cross-stack EtherChannel. Notice that multiple storage IP addresses are required but the ESX host needs only one VMkernel port. Multi-mode vifs provide link redundancy on the storage side.





Cross-Stack EtherChannel (Cont.)

VMkernel port NIC Teaming properties for cross-stack EtherChannel

IP load balancing balances connections across links

CROSS-STACK ETHERCHANNEL (CONT.) When using the cross-stack EtherChannel design, open the NIC Teaming tab on the VMkernel port Properties screen and select Route based IP hash load balancing and Link status only network failover. The ESX Server will then IP load balance connections across the available links.






iSCSI

e0a


ESX 4.0

e0b e0c e0a e0b e0c








Data ONTAP

DATA ONTAP





Review from Day 1

iSCSI service licensed and started:system2> license add XXXXXX

system2> iscsi start

The following interfaces configured for iSCSI:system2> iscsi interface showInterface e0a disabledInterface e0b enabledInterface e0c enabledInterface e0d disabled

Identify the WWN on NetApp storage:system2> iscsinodenameiSCSI target nodename: iqn.1992-08.com.netapp:sn.101201757

NOTE: No TPGroupchanges

Using system 2 for iSCSI

REVIEW FROM DAY 1





vSphere (ESX 4.0)

VSPHERE (ESX 4.0)





ESX as an iSCSI Initiator

NetApp has supported ESX as an iSCSI initiator OS since ESX 3.0

ESX 4.0 has many advantages over previous versions– New implementation of iSCSI stack– True multiple path over standard NICs – Pluggable Storage Architecture (PSA) for

multipath I/O support ESX 4.0 must be properly configured for iSCSI

connectivity over a standard network interface

ESX AS AN ISCSI INITIATOR





ESX 4.0 iSCSI Design and Installation

1. Verify ESX releases required patches, and ESX Host Utilities Kit (HUK) with Interoperability Matrix– Interoperability Matrix can be found on the

NOW™ site2. Install compatible ESX HUK from NetApp

– Provides Perl scripts to monitor and diagnose iSCSI on vSphere

– Example: esx_info provides information about the iSCSI configuration

3. Configure standard network interface(s) or install supported iSCSI HBA(s)

ESX 4.0 ISCSI DESIGN AND INSTALLATION





ESX/NIC iSCSI Implementation

After installation, to configure an ESXstandard NIC software initiator implementation:1. Configure the virtual infrastructure:

– Identify the local network adapter(s) and switch(es) to use– Configure VMkernel(s)– Configure jumbo frames if desired

2. Enable Software iSCSI Client and associate VMkernel(s) to initiator

3. Identify the WWN (IQN or eui) for the local ESX host4. Identify which method of discovery to use and enter

the storage system’s portal IP address 5. Configure authentication security if necessary6. Verify discovery and log on to the storage system

ESX/NIC ISCSI IMPLEMENTATION





Configure Virtual Infrastructure

vSwitch Considerations– NetApp recommends separating IP-based storage traffic

from public IP network traffic by implementing: Separate physical network segments, or VLAN segments

– It is recommended to not allow routing of data between networks

– Normally, administrators would: Create a new vSwitch to isolate VMs Add two or more adapters to the second switch Configure a service console port for the second switch

Exercise environment– Only have two NICs configured– Will not configure a second vSwitch, use the vSwitch0

CONFIGURE VIRTUAL INFRASTRUCTURE






1. Identify the local network adapter(s)

Currentlytwo adapters

One vSwitch with only one NIC

Need to add the second adapter to the vSwitch






Configure Virtual Infrastructure (Cont.)

1. Add NIC to vSwitch

Add an NIC tovSwitch After wizard,

NIC added

CONFIGURE VIRTUAL INFRASTRUCTURE (CONT.)






1. First VMkernel creation

NOTE: VMkernelcreation wizard is

abridged in this slide







1. Second VMkernel creation

NOTE: VMkernelcreation wizard is abridged in this slide

New VMkernel







VMkernel 1 VMkernel 2

1:1 Mapping of adaptersVMkernels






Jumbo Frame Support

ESX can be configured to use jumbo frame packets up to 9000 bytes– Network must support jumbo frames end-to-end– Enabled at vSwitch:# vicfg-vswitch -m MTU vSwitch

– Enabled at VMkernel:# esxcfg-vmknic -a -I ip -n netmask -m MTU port_group_name

Data ONTAP® supports jumbo frame– Configure on the interface that uses iSCSI:system2> ifconfig e0b mtusize 9000

JUMBO FRAME SUPPORT





Software iSCSI Client

2. Enable the Software iSCSI Client

Select

Check to enable# esxcfg-swiscsi --enableor

SOFTWARE ISCSI CLIENT





Associate VMkernels with Initiator

Log in to Service Console or use CLI Packages to issue the following commands:# esxcli swiscsi nic add -n vmk0 -d vmhba33

# esxcli swiscsi nic add -n vmk1 -d vmhba33

Verify:# esxcli swiscsi nic list -d vmhba33vmk0pNic name: vmnic0ipv4 address:10.254.135.203ipv4 net mask:255.255.252.0ipv6 addresses:mac address:00:21:5e:6f:2c:a0mtu: 1500toe: falsetso: truetcp checksum: falsevlan: true

VMkernel name

Initiator name

link connected: trueethernet speed: 1000packets received: 531701packets sent: 8719NIC driver: bnx2driver version: 1.6.9firmware version: 4.4.1 ipms 1.6.0

vmk1 ...

ASSOCIATE VMKERNELS WITH INITIATOR





ESX Worldwide Node Name (WNN)

3. Identify ESX WNN

ESX WNN

ESX WORLDWIDE NODE NAME (WNN)





Discovery

Ethernet

Ethernet


Target

Initiator

DISCOVERY





Discovery Methods

Dynamic (Send Targets)– Obtains a list of accessible

targets from the NetApp storage system

Static– Can only access particular

target portals at NetApp storage system designated by ESX Administrator

4. Configure discovery method

DISCOVERY METHODS





Dynamic Discovery Configure

To set security

Storagesystem’s iSCSI-enabledinterface address

DYNAMIC DISCOVERY CONFIGURE





Authentication Security Method

5. To increase security, iSCSI may be configured to require authentication

– Authentication method: CHAP– Unidirectional - targets will authenticate

initiators– Bidirectional - initiators and targets will

authenticate each other– This course will discuss using CHAP


AUTHENTICATION SECURITY METHOD





ESX Unidirectional CHAP Configuration

system2> iscsi security add -i iqn.1998-01.com.vmware:esx -s CHAP -n iqn.1998-01.com.vmware:esx-p thisismysecret

To configure bidirectional, check here and then...

ESX UNIDIRECTIONAL CHAP CONFIGURATION





ESX Bidirectional CHAP Configuration

system2> iscsi security add -i iqn.1998-01.com.vmware:esx-s CHAP -n iqn.1998-01.com.vmware:esx-p thisismysecret-m iqn.1992-08.com.netapp:ss-o thisismysecret2

ESX BIDIRECTIONAL CHAP CONFIGURATION





Limit an Interface from Discovery

Restricting connection over specific interfaces

Select interfaceto remove

Click to remove

orsystem2> iscsi interface disable e0a

orsystem2> iscsi interface accesslist…

LIMIT AN INTERFACE FROM DISCOVERY





ESX Discovery from CLI

Verify iSCSI discovery from CLI:# vmkiscsi-tool -D vmhba33===Discovery Properties for Adapter vmhba33===iSnsDiscoverySettable : 0...

staticDiscoverySettable : 0staticDiscoveryEnabled : 1sendTargetsDiscoverySettable : 0sendTargetsDiscoveryEnabled : 1slpDiscoverySettable : 0DISCOVERY ADDRESS : 10.254.133.239STATIC DISCOVERY TARGET

NAME : iqn.1992-08.com.netapp:sn.101201757ADDRESS : 10.254.133.239:3260

STATIC DISCOVERY TARGETNAME : iqn.1992-08.com.netapp:sn.101201757ADDRESS : 10.254.133.240:3260

ESX DISCOVERY FROM CLI





ESX Binding

A session occurs automatically:# vmkiscsi-tool -T vmhba33NAME: iqn.1992-08.com.netapp:sn.101201757ALIAS:DISCOVERY METHOD FLAGS: 0SEND TARGETS DISCOVERY SETTABLE: 0SEND TARGETS DISCOVERY ENABLED: 0Portal 0: 10.254.133.239:3260Portal 1: 10.254.133.240:3260

To view the session on the storage system:system2> iscsi session show Session 25Initiator InformationInitiator Name: iqn.1998-01.com.vmware:esxISID: 00:02:3d:00:00:03

ESX BINDING





Module Summary

MODULE SUMMARY





Module Summary


iSCSI connectivity for vSphere and NetApp systems

Configure network ports on vSphere systems Identify the worldwide node (WWN) on

vSphere systems Set up and verify multiple path iSCSI

connectivity between vSphere and NetApp systems

MODULE SUMMARY




Exercise

Module 6: vSphereiSCSI ConnectivityEstimated Time: 30 minutes



7-1 SAN Implementation Workshop: vSphere FC Connectivity


vSphere FC Connectivity


VSPHERE FC CONNECTIVITY





Module Objectives


Fibre Channel (FC) connectivity for vSphere™ and NetApp® systems

Configure FC ports on vSphere systems Identify the worldwide node name (WWNN)

and worldwide port name (WWPN) on vSpheresystems

Set up and verify multiple path FC connectivity between vSphere and NetApp systems

MODULE OBJECTIVES





FC Topology

FC TOPOLOGY






FC SAN Storage Virtualization

ESX ClusterVMSCSI Controller

VMSCSI Controller

HBA

CNAHBA

Fibre

CNA

LAN FCoEFC

Virtualization Layer

VDisk0 VDisk0

VMFS

1.vmx1.vmdk

2.vmx2.vmdk

LUN

FC SAN STORAGE VIRTUALIZATION





N-Port ID Virtualization (NPIV)

FCP

ESX

VM

VMHBA

VDisk0

VDisk0

VPORT

VPORT

NetApp FAS Array

LUN

LUN

HBA

When the VM is powered on, a VPORT is created

Each VPORT appears as a physical HBA to the fabric -

gets its own WWNN and WWPN

WWPNs must be zoned properlyand masked in igroups

LUNs must be RDM

HBAs and switchmust be NPIV

compatible

N-PORT ID VIRTUALIZATION (NPIV) N-Port ID Virtualization (NPIV) allows HBAs to create VPORTs that can be assigned a separate WWNN and WWPN from the fabric. These VPORTs can then be associated a VM allowing unique access to VM’s raw device mapping (RDM)LUN.

NPIV requires the following:

Either an Emulex HBA with NPIV-capable firmware or a QLogic4-Gb HBA. Brocade® switches with Fabric OS 5.1.0or later. Cisco® switches with SAN-OS 3.0(1) McData® switches with E/OS 8.0 Data ONTAP 7.2 or later ESX 3.5 or later Always check the Interoperability Matrix for the current supported configurations.





N-Port ID Virtualization (Cont.)

Reasons for NPIV:– VM-level chargebacks: I/O throughput tracked

per VM enabling application or user-level chargebacks

– Bi-directional association of VMs with storage: Trace VM to RDM and RDM back to VM

– VM migration: VMware VMotion™ supports preservation of VPORT ID when a VM is moved to a new ESX server

– HBA upgrades: Physical adapters can be replaced or upgraded with minimal change to SAN configuration

N-PORT ID VIRTUALIZATION (CONT.) NPIV allows storage administrators tighter control over their storage. Specifically, NPIV allows:

VM-level chargebacks which allows I/O for a specific VM to be tracked using the virtual WWPN.

Bi-directional association of VMs with storage which allow SAN administrators to trace from VM to LUN for troubleshooting connectivity issues.

VM migration which is possible because VMware VMotion supports preservation of the VPORT ID when a VM is moved to a new ESX Server.

HBA upgrades which can be upgraded or replaced with only minimal impact to the SAN configurations such as zoning.






Fibre Channel

0c - 0

98

0d - 1 0c - 2 0d - 3

switchport


ESX 4.0 Server






Data ONTAP

DATA ONTAP





Review from Day 1

FC service was licensed and started:system and system2> license add XXXXXX

system and system2> fcp start

The following interfaces were configured as an FC target:system and system2> fcadmin config

Local


------------------------------------------------



0c target CONFIGURED online

0d target CONFIGURED online

REVIEW FROM DAY 1





Review from Day 1 (Cont.)

Configure the dual storage systems as an active-active configuration– License cluster and reboot:

system and system2> license add XXXXXX


– Verify cfmode: system> fcp show cfmode

fcp show cfmode: single_image

– Enable the cluster:system> cf enable

– Verify the active-active relationship:system> cf status

Cluster enabled, system2 is up.

REVIEW FROM DAY 1 (CONT.)





system or system2> fcp nodenameFibre Channel nodename: 50:0a:09:80:86:f7:c7:86 (500a098086f7c786)

system> fcp config0c: ONLINE <ADAPTER UP> PTP Fabric







Verify configuration






vSphere (ESX 4.0)

VSPHERE (ESX 4.0)





ESX as an FC Initiator

NetApp has supported ESX as an FC initiator OS since ESX 3.0

ESX 4.0 has many advantages over previous versions– NPIV support– Updated popular HBA drivers– Pluggable Storage Architecture (PSA) for

multipath I/O support ESX must be properly configured for FC

connectivity

ESX AS AN FC INITIATOR





ESX 4.0 FC Design and Installation

1. Verify ESX releases required patches, and VMware Host Utilities Kit (HUK) with Interoperability Matrix– Interoperability Matrix can be found on the NOW™ site

2. Install compatible host bus adapters (HBAs)3. Install and configure required HBA drivers and utilities4. Verify an HBA

– Emulex: Use HBAnyware®

– QLogic: Use SANsurfer– All: Use lspci command

5. Install compatible VMware HUK from NetApp:– Provides Perl scripts to monitor and diagnose FC on

vSphere– Example: esx_info provides information about the FC

configuration such as the adapters file

ESX 4.0 FC DESIGN AND INSTALLATION





vSphere/Emulex Implementation

After installation, to configure a vSphere/Emulex implementation: Verify the HBA is enabled Identify the WWNN on the host Identify the WWPN on the host Verify connectivity between the initiator and

target

VSPHERE/EMULEX IMPLEMENTATION





Verify ESX 4.0 has identified the HBA(s)

vSphere/Emulex Implementation (Cont.)

Selected

Selected

HBAs

VSPHERE/EMULEX IMPLEMENTATION (CONT.)





Verify ESX 4.0 has identified the HBA(s) (Cont.)# cd /opt/netapp/santools

# ./esx_info fc

...

# cd /tmp/netapp/netapp/esx_info

# cat adaptersadapter name: vmhba3WWPN: 10000000c958299aWWNN: 20000000c958299adriver name: lpfc820model: 111-00308model description: NetApp 111-00308 4Gb 2-port

PCI-X2 Fibre Channel Adapter...adapter name: vmhba4

...


Same as the output of ./sanlun fcp show adapter -v






Verify ESX 4.0 has identified the HBA(s) (Cont.)# lspci | grep -i Fibre

14:01.0 Fibre Channel: Emulex Corporation LP11000 4Gb Fibre Channel Host Adapter (rev 01)

14:01.1 Fibre Channel: Emulex Corporation LP11000 4Gb Fibre Channel Host Adapter (rev 01)

Identify the driver associated with the HBA(s)# vmkload_mod -l | grep -i "lpfc“

lpfc820 0x418030089000 0x72000 0x417ff0e9dca0 0xd000 33 Yes


lpfc is the Emulex’s driver name

Specific driver name; driver is

loaded







Verify the HBA(s) configuration:# cd /opt/netapp/santools

# ./config_hba --query

lpfc820 enabled=1 options=‘lpfc_devloss_tmo=120‘

or# esxcfg-module -g lpfc820

lpfc820 enabled=1 options=‘lpfc_devloss_tmo=120‘

Configure the HBA(s):# ./config_hba --configure

Specific loaded driver name

Done during HUK install; only revisit when LUNs are

present






Identify the local WWNN/WWPN(s) on ESX 4.0


Selected

Selected

HBAs

WWNN WWPNfor selected HBA






Identify the local WWNN on ESX 4.0:# esxcfg-info | grep -i “Node Number”

|--World Wide Node Number......0x20000000c958299a

|--World Wide Node Number......0x20000000c958299b

Identify the local WWNP(s) on ESX 4.0:# esxcfg-info | grep -i “Port Number”

|--World Wide Port Number......0x10000000c958299a

|--World Wide Port Number......0x10000000c958299b

WWNN and WWPNS also available from:# cd /opt/netapp/santools

# ./sanlun fcp show adapter -v


Look for thisexactly

Look for thisexactly






Discovery

Fibre Channel

Fibre Channel

When ports are active, discovery

is automatic

Target

Initiator

DISCOVERY






Verify connectivity from the storage system:system2> fcp show initiators


Portname Group

-------- -----

10:00:00:00:c9:58:29:9b

10:00:00:00:c9:58:29:9a

Initiators connected on adapter 0d:

Portname Group

-------- -----

10:00:00:00:c9:58:29:9b

10:00:00:00:c9:58:29:9a

NOTE: For convenience, you may assign an alias to the ESX WWPN

ESX WWPNs






vSphere Discovery of Targets

Verify connectivity from ESX 4.0 using HBAnyware# cd /usr/sbin/hbanyware

# ./hbacmd allnodeinfo 10:00:00:00:c9:58:29:9a

All Node Info for 10:00:00:00:c9:58:29:9a

Node Type : WWPN

FCP ID : 10000

SCSI Bus Number: 0

SCSI Target Num: 0

Node WWN : 50:0A:09:80:86:88:37:5D

Port WWN : 50:0A:09:81:96:88:37:5D

OS Device Name : /proc/scsi/lpfc820/40,0

...

Provide a localWWPN

Storage System 1 0c port

All four ports (0c, 0d) within high-availability pair are visible

VSPHERE DISCOVERY OF TARGETS Node Type : WWPN

FCP ID : 10200

SCSI Bus Number: 0

SCSI Target Num: 2

Node WWN : 50:0A:09:80:86:88:37:5D

Port WWN : 50:0A:09:81:86:88:37:5D


Node Type : WWPN

FCP ID : 10300

SCSI Bus Number: 0

SCSI Target Num: 3

Node WWN : 50:0A:09:80:86:88:37:5D

Port WWN : 50:0A:09:82:86:88:37:5D






Module Summary

MODULE SUMMARY





Module Summary


Fibre Channel (FC) connectivity for vSphere™ and NetApp® systems

Configure FC ports on vSphere systems Identify the worldwide node name (WWNN)

and worldwide port name (WWPN) on vSpheresystems

Set up and verify multiple path FC connectivity between vSphere and NetApp systems

MODULE SUMMARY




Exercise

Module 7: vSphere FC ConnectivityEstimated Time: 60 minutes



8-1 SAN Implementation Workshop: vSphere LUN Access


vSphere LUN Access


VSPHERE LUN ACCESS





Module Objectives

By the end of this module, you should be able to: Describe the steps to allow a vSphere™

initiator to access a LUN on a storage system as a VMFS datastore

Describe the steps to allow a vSphere initiator to create a VM with a raw device mapping (RDM) disk from a storage system’s LUN

MODULE OBJECTIVES





LUN Access Review

LUN ACCESS REVIEW





LUN Access


initiator

LUN ACCESS





LUN Setup

My_IP_igroupiqn.1998-01.com.vmware:esx

OS Type: vmware

My_FC_igroup10:00:00:00:c9:58:29:9a

OS Type: vmware

Ethernet

Ethernet

Fibre Channel


1 2

Target

Initiator

LUNbLUNa

LUN SETUP





Data ONTAP Setup

DATA ONTAP SETUP





Data ONTAP SAN Configuration

Create storage container– Create an aggregate– Create a volume– Ensure automatic

Snapshot copies are disabled

– Create a qtree(optional)

In this exercise, we will use NetApp®

System Manager to perform these tasks

Add HA pair

Selectedand clickto create

an aggregate Then createvolume and

qtree

DATA ONTAP SAN CONFIGURATION NetApp System Manager provides comprehensive management and capability of one or more arrays through a simple, easy to use, intuitive GUI.

NetApp System Manager is a Windows® MMC 3.0 application that supports discovery, set up, FC, iSCSI, CIFS, NFS, deduplication, provisioning, thin provisioning, Snapshot™ technology and configuration management of multiple NetApp storage systems from a single pane of glass.

To learn more, go to the NOW™ site.





Create an igroup

Selectedfirst

Selected

Then add the ESX WWPNs

CREATE AN IGROUP





Create a LUN

Selectedfirst

Selected

For a VMFS datastore,the type will be VMware

For an RDM datastore,the type will be specific

to the OS installedAt the end of Wizard, the LUN will be

created and mapped appropriately

CREATE A LUN





LUN Decisions for ESX

When creating a LUN, keep the following in mind:– One LUN per VMFS datastore– You might want fewer, larger LUNs: More flexibility to create VMs without storage changes More flexibility for resizing virtual disks Fewer VMFS datastores to manage

– You might want more, smaller LUNs: More efficient use of storage More flexibility in multipathing policy Microsoft® Cluster requires each cluster disk on its

own LUN

LUN DECISIONS FOR ESX





vSphere Setup

VSPHERE SETUP





vSphere Steps


initiator

NOTE: Step 5 differs depending on whether you are creating a VMFS or RDM datastore

VSPHERE STEPS





4. Find the LUN on vSphere (iSCSI)

Rescan if LUNdoesn’t appear

Identifier

Storage adapter Channel # Target # LUN #

Selected

Selected

4. FIND THE LUN ON VSPHERE (ISCSI)





4. Find the LUN on vSphere (FC)

Selected

The LUN

To configure LUN discovery

Configure the new LUN with optimal NetApp/ESX settings:# ./config_hba --configure

Fine tunewith the

Host Utilities Kit

4. FIND THE LUN ON VSPHERE (FC)





Configuring LUN Discovery

Selected

Limits scans on target to

Logical Units with numbers

0 - 255

When set to1, looks for

nonsequentialLogical Unitnumbering

CONFIGURING LUN DISCOVERY





4. Find the LUN on vSphere (CLI)

With the Host Utilities Kit:# cd /opt/netapp/santools# ./sanlun lun showcontroller: lun-pathname device filename adapter protocol lun size lun state vmkdisk name

system2: /vol/vol_VMFS_fcp/qt_Store1/lun1 /dev/sdcvmhba3 FCP 40.0g (42953867264) GOOD naa.60a980004335432d576f525844457678 Identifier

4. FIND THE LUN ON VSPHERE (CLI)





4. Find the LUN on vSphere (CLI)

With ESX:# esxcli nmp device list...naa.60a980004335432d576f525844457678

Device Display Name: NETAPP Fibre Channel Disk (naa.60a980004335432d576f525844457678)

Storage Array Type: VMW_SATP_ALUAStorage Array Type Device Config:

{implicit_support=on;explicit_support=off;explicit_allow=on;alua_followover=on;{TPG_id=2,TPG_state=AO}

{TPG_id=3,TPG_state=ANO}}Path Selection Policy: VMW_PSP_MRUPath Selection Policy Device Config:

Current Path=vmhba4:C0:T2:L0Working Paths: vmhba4:C0:T2:L0

....

NMP = Native Multipathing Plug-in

4. FIND THE LUN ON VSPHERE (CLI)





Manage Paths

MANAGE PATHS





Manage Paths on the Service Console

With ESX:

– PSP types: VMW_PSP_RR = Round robin VM_PSP_Fixed = Fixed VM_PSP_MRU = Most Recently Used

– SATP types supported for NetApp arrays: VMW_SATP_DEFAULT_AA = Symmetric access VM_SATP_ALUA = Asymmetric access

# esxcli nmp satp setdefault -psp <PSP> -satp <satp>....

SATP = Storage Array Type Plug-in

MANAGE PATHS ON THE SERVICE CONSOLE





Creating a VMFS Datastore

5. Preparing the LUN for VMFS

5. PREPARING THE LUN FOR VMFS





5. Preparing the LUN for VMFS (Cont.)

5. PREPARING THE LUN FOR VMFS (CONT.)






Give the datastorea name












Create a VM in a VMFS Datastore

Give a nameto the VM

CREATE A VM IN A VMFS DATASTORE





Create a VM in a VMFS Datastore (Cont.)

Choose the VMFS datastore

CREATE A VM IN A VMFS DATASTORE (CONT.)



































Using Datastores in vSphere

You can use datastores to create secondary hard disks for existing VMs

RDMs canalso be used if available

Secondarydisk will

show up in guest OS

USING DATASTORES IN VSPHERE





Multiple Disk Prioritization

Change the priority of multiple disks to maximize throughput of thrashed disks

Possible choices:• Low• Normal• High• Custom

MULTIPLE DISK PRIORITIZATION





Physical Files Investigation

Browse the datastore to investigate files

PHYSICAL FILES INVESTIGATION





x86/x64 Architecture



How VMs Access Data on a SAN

VM

Application

Operating System

CPU Memory NIC Disk

Guest OS reads or writes SCSI disk

Device drivers communicate with virtual SCSI controllers

VMkernel

Virtual SCSI controller forwardscommand to VMkernel

VMkernel: Locates the .vmdk in VMFS Maps request to block on .vmdk Sends I/O to iSCSI initiator or

FC HBA

Software initiator or HBA send I/O toNetApp storage

HOW VMS ACCESS DATA ON A SAN





Raw Device Mapping (RDM) Devices

RAW DEVICE MAPPING (RDM) DEVICES





Create a VM with an RDM

Create a LUN witha type associated with

the guest OS andthen start VM creation wizard

Make sureyou choose

Custom

CREATE A VM WITH AN RDM





Create a VM with an RDM (Cont.)

CREATE A VM WITH AN RDM (CONT.)












Check hereto set NPIV

settings (if using NPIV)

The propertiesdialog appears






Setting NPIV Settings

Later it will have

the WWNN(s) and WWPN(s)

Select to generate new WWNs

NOTE: Add these WWPN(s) to igroups

SETTING NPIV SETTINGS





NPIV Observed

Before boot up of NPIV-supported VM:switch> switchshow...Area Port Media Speed State Proto=====================================0 0 id N2 Online F-Port 50:0a:09:81:96:88:37:5d1 1 id N2 Online F-Port 50:0a:09:82:96:88:37:5d2 2 id N2 Online F-Port 50:0a:09:81:86:88:37:5d3 3 id N2 Online F-Port 50:0a:09:82:86:88:37:5d

...8 8 id N4 Online F-Port 10:00:00:00:c9:58:29:9a9 9 id N4 Online F-Port 10:00:00:00:c9:58:29:9b

switch> portcfgshowPorts of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15...NPIV capability ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON...

Only one connection (non-NPIV)

NPIV turned on

NPIV OBSERVED





NPIV Observed (Cont.)

After boot up of NPIV-supported VM:switch> switchshow...Area Port Media Speed State Proto=====================================0 0 id N2 Online F-Port 50:0a:09:81:96:88:37:5d1 1 id N2 Online F-Port 50:0a:09:82:96:88:37:5d2 2 id N2 Online F-Port 50:0a:09:81:86:88:37:5d3 3 id N2 Online F-Port 50:0a:09:82:86:88:37:5d

...8 8 id N4 Online F-Port 2 NPIV public9 9 id N4 Online F-Port 2 NPIV public

switch> portshow 8...portWwn: 20:08:00:05:1e:02:99:c4portWwn of device(s) connected:

27:ee:00:0c:29:00:05:a710:00:00:00:c9:58:29:9a

Two connections (NPIV)

NPIV’s WWPNs show up


27:ee:00:0c:29:00:06:a710:00:00:00:c9:58:29:9a

NPIV OBSERVED (CONT.)






After boot up of NPIV-supported VM (Cont.):switch> portloginshow 8Type PID World Wide Name credit df_sz cos=====================================================fe 010801 27:ee:00:0c:29:00:05:a7 16 2048 c scr=3fe 010800 10:00:00:00:c9:58:29:9a 16 2048 c scr=3ff 010801 27:ee:00:0c:29:00:05:a7 12 2048 c d_id=FFFFFCff 010800 10:00:00:00:c9:58:29:9a 12 2048 c d_id=FFFFFC

switch> portloginshow 9Type PID World Wide Name credit df_sz cos=====================================================fe 010901 27:ee:00:0c:29:00:06:a7 16 2048 c scr=3fe 010900 10:00:00:00:c9:58:29:9b 16 2048 c scr=3ff 010901 27:ee:00:0c:29:00:06:a7 12 2048 c d_id=FFFFFCff 010900 10:00:00:00:c9:58:29:9b 12 2048 c d_id=FFFFFC






Physical Files Investigation

Browse the datastore to investigate files

.vmdkis pointerto LUN

PHYSICAL FILES INVESTIGATION





Alignment Issues

ALIGNMENT ISSUES





VMFS Datastore and Alignment Problems

NetApp FAS Array

ESX Cluster

FC / FCoE / iSCSI

VM1 VM2 VM3 VM4

1.vmx1.vmdk

2.vmx2.vmdk

3.vmx3.vmdk

4.vmx4.vmdk

VMFS LUNs are setto vmware type

However, vmdk willbe of other OS types

This can lead to problems...depending on the OS

NOTE:RDM and other

LUNs don’t sufferfrom this problem

if the LUN typeattribute is set

correctly

VMFS DATASTORE AND ALIGNMENT PROBLEMS





Partition Alignment and Misalignment

Failure to align the VM’s file system’s partition to the physical storage array can cause negative impact on performance leading to multiple I/O operations for a single I/O request from the VM

0 1 2 3 4 5 6 7 8 9 10

4-KB WAFL®Block512 byte LBA

Un-aligned write, creates partial writes in WAFL

PARTITION ALIGNMENT AND MISALIGNMENT





Identifying Alignment Partition on VM

In the VM, use msinfo32.exe to identify the current partition starting offset

Windows Server 2003 has an offset

of 32256

IDENTIFYING ALIGNMENT PARTITION ON VM





MBRscan Tool

Within ESX Host Utilities Kit 5.1, use MBRscanto identify current alignment:

# cd /opt/netapp/santools# ./mbrscan --allBuilding file list......--------------------/vmfs/volumes/4a96738c-187213b0-a38d-00215e6f2ca0/Dev05s2/Dev05s2-flat.vmdk p1 (NTFS) lba:2048 offset:1048576 aligned:Yes/vmfs/volumes/4a96738c-187213b0-a38d-00215e6f2ca0/Dev05s2/Dev05s2-flat.vmdk p2 (NTFS) lba:206848 offset:105906176 aligned:Yes--------------------/vmfs/volumes/4a96738c-187213b0-a38d-00215e6f2ca0/Win2003 FC VMFS/Win2003 FC VMFS-flat.vmdk p1 (NTFS) lba:63 offset:32256 aligned:No MBRscan confirms

Windows Server 2003 has an offset of 32,256

Not aligned withWAFL blocks

Windows Server 2008 R2 (64-bit) aligned correctly from install

MBRSCAN TOOL





MBRalign Tool

Within ESX Host Utilities Kit 5.1, use MBRalignto correct alignment on existing VMs:

# cd /opt/netapp/santools# ./mbralign "/vmfs/volumes/4a96738c-187213b0-a38d-00215e6f2ca0/Win2003 FC VMFS/Win2003 FC VMFS-flat.vmdk"Part Type old LBA New Start LBA New End LBA Length in KBP1 07 63 64 16755796 8377866

NOTICE:This tool does not check for the existence of Virtual Machine snapshots or linked clones.The use of this tool on a vmdk file that has a snapshot or linked clone associated with it can result in unrecoverable data loss and/or data corruption.Are you sure that no snapshots/linked clones exist for this vmdk? (y/n)y

MBRALIGN TOOL





MBRalign Tool (Cont.)

Within ESX Host Utilities Kit 5.1, use MBRalignto correct alignment on existing VMs:

Creating a backup of /vmfs/volumes/4a96738c-187213b0-a38d-00215e6f2ca0/Win2003 FC VMFS/Win2003 FC VMFS.vmdkCreating a backup of /vmfs/volumes/4a96738c-187213b0-a38d-00215e6f2ca0/Win2003 FC VMFS/Win2003 FC VMFS-flat.vmdk

Creating a copy the Master Boot RecordWorking on partition P1 (3): Starting to migrate blocks from 32256 to 32768.12801 read ops in 4 sec. 98.88% read (24.87 mB/s). 98.88% written (24.87 mB/s)

Working on space not in any partition: Starting to migrate blocks.100.00 percent complete. 100.00 percent written. . Making adjustments to /vmfs/volumes/4a96738c-187213b0-a38d-00215e6f2ca0/Win2003 FC VMFS/Win2003 FC VMFS-flat.vmdk.Adjusting the descriptor file.

Change alignment from 32256 to 32768

MBRALIGN TOOL (CONT.)





Verify Alignment Changes

Within ESX Host Utilities Kit 5.1, use MBRscanto verify alignment:

# cd /opt/netapp/santools# ./mbrscan --allBuilding file list...--------------------Failed to open /vmfs/volumes/4a76cd5b-ed77e188-1d1d-00215e6f2ca2/esxconsole-4a76cd30-32f4-2f68-f710-00215e6f2ca0/esxconsole-flat.vmdk - [Device or resource busy]...--------------------/vmfs/volumes/4a96738c-187213b0-a38d-00215e6f2ca0/Win2003 FC VMFS/Win2003 FC VMFS-flat.vmdk p1 (NTFS) lba:64 offset:32768 aligned:Yes-------------------- MBRscan confirms

Windows Server 2003now has 32768Aligned with

WAFL blocks

VERIFY ALIGNMENT CHANGES





Verify Alignment Change in VM

In the VM, use msinfo32.exe to verify the new partition starting offset

Windows Server 2003 confirms the

new offset

VERIFY ALIGNMENT CHANGE IN VM





Properly Aligned Partitions for New VMs

If you have new Windows Server 2003 VMs, you can pre-create the partition table to be properly aligned before installation:1. Boot the VM with Microsoft’s Windows Preinstall

Environment (WinPE) CD2. Select Start -> Run and enter DISKPART3. Enter Select Disk04. Enter Create Partition Primary Align=325. Reboot the VM with WinPE CD6. Install the operating system as normal

NOTE: Similar steps can be taken with other operating systems

PROPERLY ALIGNED PARTITIONS FOR NEW VMS





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe the steps to allow a vSphere initiator to

access a LUN on a storage system as a VMFS datastore

Describe the steps to allow a vSphere initiator to create a VM with a raw device mapping (RDM) disk from a storage system’s LUN

MODULE SUMMARY




Exercise

Module 8: vSphere LUN AccessEstimated Time: 60 minutes



9-1 SAN Implementation Workshop: Red Hat Overview


Red Hat Overview


RED HAT OVERVIEW





Module Overview

In this module, we will cover the following: Describe Red Hat® Enterprise Linux®

Explain how NetApp® storage is ideal for LUNs managed by Red Hat Enterprise Linux

MODULE OVERVIEW





Linux

Linux is a free UNIX®-type operating system originally created by Linus Torvalds

Linux is:– Open source– Licensed under GNU GPL– Has a kernel developed by the Linux kernel

mailing list

LINUX





Red Hat and Linux

Red Hat Enterprise Linux 5 (RHEL 5)– Supports Linux kernel 2.6.18– Server editions: Red Hat Enterprise Linux Advanced Platform

– For mission-critical and enterprise computer systems

Red Hat Enterprise Linux – For supported network servers– Limited to two CPUs

RHEL 5: – Update 3: Latest stable build during the building

of this course– Update 4: Introduces hypervisor technology

RED HAT AND LINUX





Red Hat and Storage

RHEL 5 supports:– SAN FC SAN

– FC protocol by way of Emulex and QLogic HBAs IP SAN

– iSCSI by way of a built-in software initiator and standard NICs

– Logical Volume Manager (LVM) for creating and managing pools of virtual storage

– DM Multipath for managing multiple paths to a LUN

RED HAT AND STORAGE





NetApp and RHEL 5

NetApp provides an integrated solution that enables storage, delivery and management of data and content to achieve your business needs

Together Red Hat and NetApp provide an excellent platform for applications such as:– Oracle®

– Virtualization with Red Hat Enterprise Linux 5.4

NETAPP AND RHEL 5





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe Red Hat Enterprise Linux Explain how NetApp storage is ideal for LUNs

managed by Red Hat Enterprise Linux

MODULE SUMMARY





Exercise

Due to hardware constraints, your Windows®

Server 2008 R2 (W2K8R2) machine has been reimaged to Red Hat Enterprise Linux (RHEL) 5.3

The WWPNs of the W2K8R2 are now the WWPNs of RHEL 5.3

To prepare for this day, this exercise asks you:– To offline all FC-attached LUNs that were

associated with the W2K8R2 FC igroups– To delete the W2K8R2 FC igroups

EXERCISE




Exercise

Module 9: Red Hat OverviewEstimated Time: 10 minutes



10-1 SAN Implementation Workshop: Red Hat FC Connectivity


Red Hat FC Connectivity


RED HAT FC CONNECTIVITY





Module Objectives


Fibre Channel (FC) connectivity for Red Hat® and NetApp® systems

Configure FC ports on Red Hat systems Identify the worldwide node name (WWNN)

and worldwide port name (WWPN) on Red Hat systems

Set up and verify multiple path FC connectivity between Red Hat and NetApp systems

MODULE OBJECTIVES





FC Topology

FC TOPOLOGY






Fibre Channel

0c - 0

76

0d - 1 0c - 2 0d - 3

switchport


Red Hat 5.3






Data ONTAP

DATA ONTAP





Review from Day 1

FC protocol service was licensed and started:system and system2> license add XXXXXX

system and system2> fcp start

The following interfaces were configured as an FC target:system and system2> fcadmin config

Local


------------------------------------------------



0c target CONFIGURED online

0d target CONFIGURED online

REVIEW FROM DAY 1






Configure the dual storage systems as an active-active configuration– License cluster and reboot:

system and system2> license add XXXXXX


– Verify cfmode: system> fcp show cfmode

fcp show cfmode: single_image

– Enable the cluster:system> cf enable

– Verify the active-active relationship:system> cf status

Cluster enabled, system2 is up.






system or system2> fcp nodenameFibre Channel nodename: 50:0a:09:80:86:f7:c7:86 (500a098086f7c786)

system> fcp config0c: ONLINE <ADAPTER UP> PTP Fabric







Verify configuration






Red Hat

RED HAT





Red Hat as an FC Initiator

NetApp has supported Red Hat as an FC initiator OS since Red Hat 3

Red Hat 5.3 has many advantages over previous versions:– Updated FC and iSCSI drivers– Better scalability

Red Hat must be properly configured for FC connectivity

RED HAT AS AN FC INITIATOR





Red Hat 5.3 Design and Installation

1. Verify host operating system releases, required patches, and NetApp Linux Host Utility Kit with Interoperability Matrix– Use /etc/redhat-release and uname -a to verify Red

Hat version– Interoperability Matrix can be found on the NOW™ site

2. Install compatible host bus adapters (HBAs)3. Install and configure required HBA drivers and utilities

if needed 4. Verify an HBA:

– All HBA Types: lspci– Emulex: Use /usr/sbin/lpfc/lputil or HBAnyware®

– QLogic: Use /usr/bin/scli or SANsurfer

RED HAT 5.3 DESIGN AND INSTALLATION





Red Hat/Emulex Implementation

After installation, to configure a Red Hat/Emulex implementation:1. Identify the correct HBA driver2. Verify that Red Hat has correctly identified

the HBA and loaded the driver3. Identify the WWNN and WWPN on the host4. Verify connectivity between the initiator and

target

RED HAT/EMULEX IMPLEMENTATION





Red Hat/Emulex Implementation (Cont.)

Verify that Red Hat has identified the HBA(s)# cd /sys/class/scsi_host

# ls

host0 host1 host2

Identify the driver associated with the HBA(s)# cd host1

# ls

... fwrev

... modeldesc

... modelname

... npiv_info

... portnum

... serialnum

... lpfc_drvr_version

Emulex HBA(s) or in this case on HBA with two ports

NetApp part number of NetApp sold HBAs

# cat lpfc_drvr_versionEmulex LightPulseFibre Channel SCSI driver 8.2.0.33.3p

Current installed driver

# cat fwrev2.72A2 (B3F2.72A2), sli-3

RED HAT/EMULEX IMPLEMENTATION (CONT.)






Interoperability Matrix requires an update to driver based upon existing firmware:# tar zxf lpfc_2.6_driver_kit-8.2.0.39-1.tar.gz

# cd lpfc_2.6_driver_kit-8.2.0.39-1

# ./lpfc-install ...

# reboot

The driver can be found here:# ls/lib/modules/2.6.18-128.el5/kernel/drivers/scsi/lpfc

lpfc.ko

To verify the driver when loaded:# modprobe -c | greplpfc

To load the driver (if needed):# modprobe -v lpfc

The kernel build number of Linux®







Install compatible Linux Host Utility Kit (HUK)– Provides Perl scripts to configure and tune Red Hat– Example: sanlun application to manage LUNs from Red

Hat– HUK requires packages to be installed: libnl.so = libnl-1.0-0.10.pre5.5.x86_64.rpm libnl.so = libnl-1.0-0.10.pre5.5.i386.rpm HBAnyware = elxlinuxapps-4.0a31-8.2.0.39-1-1.tar

Use 32-bit

version

Use 64-bit

version

NOTE: You must install these libraries in this order







Install HBAnyware:# tar xvf elxlinuxapps-4.0a31-8.2.0.39-1-1.tar# cd elxlinuxapps-4.0a31-8.2.0.39-1-1# ./install

Select desired mode of operation for HBAnyware1 Local Mode : HBA's on this Platform can be managed by

HBAnyware clients on this Platform Only.2 Managed Mode: HBA's on this Platform can be managed by

local or remote HBAnyware clients.3 Remote Mode : Same as '2' plus HBAnyware clients on this

Platform can manage local and remote HBA's.

...







Install HUK:# tar zxf netapp_linux_host_utilities_5_0.tar.gz

# cd netapp_linux_host_utilities_5_0

# ./install

# cd /opt/netapp/santools

# ./san_version

NetApp Linux Host Utilities version 5.0

Identify WWPN of HBA(s):# cd /opt/netapp/santools

# sanlunfcp show adapters

host1 WWPN:10000000c96b77b4

host2 WWPN:10000000c96b77b3

or# cat /sys/class/fc_host/host1/port_name

0x10000000c96b77b4

Same as /sys/class/scsi_host






Discovery and Session Creation

Fibre Channel

Fibre Channel

When ports are active (and properly zoned), discovery is automatic

Target

Initiator

DISCOVERY AND SESSION CREATION






Verify connectivity from the storage system:system> fcp show initiators


Portname Group

-------- -----

10:00:00:00:c9:6b:77:b3

10:00:00:00:c9:6b:77:b4

NOTE: For convenience, alias the Red Hat WWPNs

Red Hat WWPNs






Module Summary

MODULE SUMMARY





Module Summary


Fibre Channel (FC) connectivity for Red Hat and NetApp systems

Configure FC ports on Red Hat systems Identify the worldwide node name (WWNN)

and worldwide port name (WWPN) on Red Hat systems

Set up and verify multiple path FC connectivity between Red Hat and NetApp systems

MODULE SUMMARY




Exercise

Module 10: Red Hat FC ConnectivityEstimated Time: 30 minutes



11-1 SAN Implementation Workshop: Red Hat iSCSI Connectivity


Red Hat iSCSI Connectivity


RED HAT ISCSI CONNECTIVITY





Module Objectives


iSCSI connectivity for Red Hat® and NetApp®

systems Configure network ports on Red Hat systems Identify the worldwide node (WWN) on Red

Hat systems Set up and verify multiple path IP connectivity

between Red Hat and NetApp systems

MODULE OBJECTIVES





IP Connectivity

IP CONNECTIVITY






iSCSI

e0a


RHEL

e0b e0c e0a e0b e0c








Data ONTAP

DATA ONTAP





Review from Day 1

iSCSI service licensed and started:system> license add XXXXXX

system> iscsi start

The following interfaces configured for iSCSI:system> iscsi interface show

Interface e0a disabledInterface e0b enabledInterface e0c enabledInterface e0d disabled

Identify the WWN on NetApp storage:system> iscsi nodename

iSCSI target nodename: iqn.1992-08.com.netapp:system

NOTE: No TPGroupchanges

Using system 1 for iSCSI

REVIEW FROM DAY 1





Red Hat

RED HAT





Red Hat as an iSCSI Initiator

NetApp has supported Red Hat as an iSCSI Initiator OS since Red Hat 4

Red Hat 5.3 has many advantages over previous versions with:– Packages providing iSCSI device drivers and

utilities– iSCSI software initiator for standard network

interfaces Red Hat must be configured properly for iSCSI

connectivity over a standard network interface

RED HAT AS AN ISCSI INITIATOR





Red Hat 5.3 Design and Installation

1. Verify host operating system releases, required patches, and NetApp Linux® Host Utility Kit with Interoperability Matrix:– Use /etc/redhat-release and uname -a to verify

Red Hat version– Interoperability Matrix can be found on the NOW™ site

2. Install iSCSI software packages and patches:# rpm -ivh iscsi-initiator-utils-6.2.0.868-0.18.el5.x86_64.rpm

3. Identify and verify a network interface is properly configured

4. Install compatible Linux Host Utility Kit– Provides Perl scripts to configure and tune Red Hat for iSCSI– Example: sanlun application to manage LUNs from Red Hat

RED HAT 5.3 DESIGN AND INSTALLATION





Red Hat/NIC Implementation

After installation, to configure a Red Hat standardNIC software initiator implementation:1. Identify the local network interface(s) to use2. Verify the iSCSI service and WWN for host3. Configure authentication security if necessary4. Identify which method of discovery to use and

enter the storage system’s portal IP address 5. Verify discovery and sessions with the target

RED HAT/NIC IMPLEMENTATION





Red Hat/NIC Implementation (Cont.)

1. Using the software initiator, Red Hat supports iSCSI over standard network interface– Investigate and configure interfaces:# ifconfig -a

eth0

Link encap:Ethernet HWaddr 00:21:5E:6F:18:C4

inet addr:10.254.132.63 Bcast:10.254.135.25

Mask:255.255.252.0

inet6 addr: fe80::221:5eff:fe6f:18c4/64

Scope:Link UP BROADCAST RUNNING MULTICAST

MTU:1500 Metric:1

...

eth1

Link encap:Ethernet HWaddr 00:21:5E:6F:18:C6

BROADCAST MULTICAST MTU:1500 Metric:1 ...

Unconfigured adapter; configure adapter with ifconfig and

add to rc daemon

RED HAT/NIC IMPLEMENTATION (CONT.)






2. View the iSCSI Service and initiator node name– Start iSCSI service:# service iscsi start

– Verify status of iSCSI service:# service iscsi status

iscsid (pid 5236 5235) is running...

– Identify the initiator’s node name:# cat /etc/iscsi/initiatorname.iscsi

InitiatorName=iqn.1994-05.com.redhat:rhel

Red Hat WWN






Discovery

Ethernet

Ethernet


Target

Initiator

DISCOVERY





iSCSI Authentication in Red Hat 5

3. To increase security, iSCSI may be configured to require authentication– Authentication methods: CHAP session authentication


each other CHAP discovery (send targets) authentication


each other– This course will discuss using CHAP session


ISCSI AUTHENTICATION IN RED HAT 5





iSCSI Unidirectional CHAP Authentication

Configure initiator’s user name and password:# vi /etc/iscsi/iscsid.conf...#node.session.auth.username = username#node.session.auth.password = thisismysecret

To configure CHAP, enable CHAP authentication:# vi /etc/iscsi/iscsid.conf...#node.session.auth.authmethod = CHAP

# service iscsi restart

On the storage system, register the CHAP secret:system> iscsi security add

-i iqn.1994-05.com.redhat:rhel -s CHAP -p thisismysecret-n username

Same user name and password

Uncomment

ISCSI UNIDIRECTIONAL CHAP AUTHENTICATION





iSCSI Bidirectional CHAP Authentication

Configure unidirectional CHAP and then configure the reverse direction:

system> iscsi security add -i iqn.1994-05.com.redhat:rhel -s CHAP -p thisismysecret-n username-o thisismysecret2-m username2

On Red Hat, register the storage system’s CHAP secret:# vi /etc/iscsi/iscsid.conf...#node.session.auth.username = username2#node.session.auth.password = thisismysecret2

On Red Hat, restart the iSCSI service:# service iscsi restart

User name and passwordof inbound and outbound

cannot be the same

Don’t forgot to uncomment

ISCSI BIDIRECTIONAL CHAP AUTHENTICATION





Red Hat/NIC Implementation

4. Discovery is possible through either:– Static discovery iSCSI targets added manually

– Send-targets discovery IP address of the target is added Initiator communicates to target over port 3260

– Internet Storage Name Service (iSNS) Centralized management of discovery and

configuration of iSCSI networksThis course will focus on the send-target discovery method

RED HAT/NIC IMPLEMENTATION






Set up iSCSI interface (for multiple paths):# iscsiadm -m iface -I iface0 --op=new

# iscsiadm -m iface -I iface0 --op=update -n iface.net_ifacename -v eth0

# iscsiadm -m iface -I iface1 --op=new

# iscsiadm -m iface -I iface1 --op=update -n iface.net_ifacename -v eth1

Verify iSCSI interfaces:# iscsiadm -m ifaceiface0 tcp,default,eth0iface1 tcp,default,eth1# ls /var/lib/iscsi/ifacesiface0 iface1

Red Hat’s eth0 and eth1 interface name

Use vi or iscsiadm to configure







Set up send targets discovery with interfaces:# iscsiadm -m discovery -t sendtargets -p 10.254.133.239 -I iface0 -I iface1

10.254.133.239:3260,1001 iqn.1992-08.com.netapp:system




IP address of iSCSI-enabled interface on the storage system Target portals discovered

by each Red Hat interface







5. Explore the iSCSI targets discovered:# iscsiadm -m node --op=show#BEGIN RECORD 2.0-868node.name = iqn.1992-08.com.netapp:systemnode.tpgt = 1002node.startup = automaticiface.hwaddress = 00:21:5E:6F:18:C6iface.iscsi_ifacename = iface1iface.net_ifacename = defaultiface.transport_name = tcpnode.discovery_address = 10.254.133.239 ...#BEGIN RECORD 2.0-868node.name = iqn.1992-08.com.netapp:systemnode.tpgt = 1002node.startup = automaticiface.hwaddress = 00:21:5E:6F:18:C4iface.iscsi_ifacename = iface0...







Observe discovered targets:# iscsiadm -m node











Create a session with discovered targets:# iscsiadm -m node -l

Logging in to [iface: iface1, target: iqn.1992-08.com.netapp:system, portal: 10.254.133.240,3260]




...






Red Hat iSCSI Sessions

View the current sessions:

View the sessions on the storage system:

# iscsiadm -m sessiontcp: [1] 10.254.133.240:3260,1002 iqn.1992-08.com.netapp:systemtcp: [2] 10.254.133.240:3260,1002 iqn.1992-08.com.netapp:systemtcp: [3] 10.254.133.239:3260,1001 iqn.1992-08.com.netapp:systemtcp: [4] 10.254.133.239:3260,1001 iqn.1992-08.com.netapp:system

system> iscsi session show Session 30Initiator InformationInitiator Name: iqn.1994-05.com.redhat:rhelISID: 00:02:3d:01:00:00Initiator Alias: dev05s2.development.netappu.com

Session 31Initiator Information...

RED HAT ISCSI SESSIONS





Module Summary

MODULE SUMMARY





Module Summary


iSCSI connectivity for Red Hat and NetApp systems

Configure network ports on Red Hat systems Identify the worldwide node (WWN) on Red

Hat systems Set up and verify multiple path IP connectivity

between Red Hat and NetApp systems

MODULE SUMMARY




Exercise

Module 11: Red Hat iSCSI ConnectivityEstimated Time: 30 minutes



12-1 SAN Implementation Workshop: Red Hat LUN Access


Red Hat LUN Access


RED HAT LUN ACCESS





Module Objectives

By the end of this module, you should be able to: Describe the steps to allow a Red Hat® initiator

to access a LUN on a storage system

MODULE OBJECTIVES





LUN Access Review

LUN ACCESS REVIEW





LUN Access


initiator

LUN ACCESS





LUN Access Review

My_IP_igroupiqn.1999-05.com.redhat:rhelOS Type: linux

My_FC_igroup10:00:00:00:c9:6b:77:b4OS Type: linux

Ethernet

Ethernet

Fibre Channel


1 2

Target

Initiator

LUNbLUNa

LUN ACCESS REVIEW





Data ONTAP Setup

DATA ONTAP SETUP





Data ONTAP SAN Configuration

Create storage container:– Create an aggregate– Create a volume– Ensure automatic

Snapshot copies are disabled

– Create a qtree(optional)

In this exercise, you can use either thecommand-line interface or NetApp®

System Manager

system> aggr create ...system> vol create ...system> vol options...no_snapsystem> qtree create...

DATA ONTAP SAN CONFIGURATION





Create an igroup

You can create an igroup (LUN masking) using:– Command-line interface (CLI)– NetApp System Manager– Provisioning Manager– Host Utilities Kit# cd /opt/netapp/santools

# sanlunfcp show adapter -c

Enter this controller command to create an initiator group for this system:

igroup create -f -t linux"dev05s2.development.netappu.com" 10000000c96b77b4 10000000c96b77b3

CREATE AN IGROUP





Create a LUN

You can create a LUN using:– CLI– NetApp System Manager– Provisioning Manager– SnapDrive® (see appendix)

CREATE A LUN





Red Hat Setup

RED HAT SETUP





Red Hat Steps


initiator

RED HAT STEPS





4. Find the LUN / 5. Prepare the LUN

Rescan HBA # cd /usr/sbin/lpfc

# ./lun_scan

Use fdisk -l command# fdisk -l...

Disk /dev/sdb: 2147 MB, 2147483648 bytes

67 heads, 62 sectors/track, 1009 cylinders

Units = cylinders of 4154 * 512 = 2126848 bytes

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 2147 MB, 2147483648 bytes

67 heads, 62 sectors/track, 1009 cylinders

Units = cylinders of 4154 * 512 = 2126848 bytes

Disk /dev/sdc doesn't contain a valid partition table...

The LUN shows up 8 times... sdb - sdi;

therefore use multipath

4. FIND THE LUN / 5. PREPARE THE LUN





Multipath I/O

MULTIPATH I/O





Multiple I/O Configuration

Red Hat may be configured for multipath I/O (MPIO) using:– DM-Multipath: Allows native support Provides redundancy Improves performance

– Veritas™ Storage Foundation: Heterogeneous storage management software

In this module, we will focus on DM-Multipath

MULTIPLE I/O CONFIGURATION





Device Identifiers

RHEL will see a total of eight devices without MPIO in your exercise environment– /dev/sdb... /dev/sdi

Under DM-Multipath’s control, these devices may be seen:– /dev/mapper/mpathn Used when creating

logical volumes– /dev/mpath/mpathn Convenience list under one

directory, never use– /dev/dm-n Internal use, never use

0 1 2 3 4 5 6 7 8 9

Storage System 1

0 1

Storage System 2

2 3

RHEL

6 7

Whenuser_friendly_names

option set to yes

DEVICE IDENTIFIERS





DM-Multipath Configuration Overview

1. Install/verify device-mapper-multipathrpm

2. Edit the multipath.conf file:– Comment out the default blacklist– Change any of the existing defaults as needed– Save the configuration file

3. Start the multipath daemons4. Create multipath device with the multipath

command

DM-MULTIPATH CONFIGURATION OVERVIEW





DM-Multipath Configuration

1. Install/verify device-mapper-multipath:– In Red Hat 5.3, DM-Multipath is installed by

default– Verify rpms installation:# rpm -q device-mapper

device-mapper-1.02.28-2.el5

device-mapper-1.02.28-2.el5

# rpm -q device-mapper-multipath

device-mapper-multipath-0.4.7-23.el5

DM-MULTIPATH CONFIGURATION





DM-Multipath Configuration (Cont.)

2. Edit the multipath.conf file – Comment out the default blacklist:#blacklist {

# devnode "*"

#}

– Verify that user_friendly_names is set to true:defaults {

user_friendly_names yes

}

– Disable local drive from using DM-Multipath:blacklist {

devnode "sd[a]$“

}

DM-MULTIPATH CONFIGURATION (CONT.)





2. Edit the multipath.conf file (Cont.)– Set up NetApp device:devices { device {

vendor "NetApp"product "LUN"path_grouping_policy group_by_priogetuid_callout "/sbin/scsi_id -g -u -s /block/%n"prio_callout “/sbin/santools/mpath_prio_ontap /dev/%n"features "1 queue_if_no_path"path_checker readsector0failback immediate}}

– path_grouping _policy values: failover = 1 path per priority group multibus = all valid paths in 1 priority group group_by_serial = 1 priority group per serial number group_by_prio = 1 priority group per priority value group_by_node_name = 1 priority group per WWNN








3. Configure the multipath daemons– Start up: # modprobe dm-multipath

# service multipathd start

Starting multipathd daemon:

– Reload:# service multipathd reload

– Stop:# service multipathd stop

– Restart:# service multipathd restart

– Check the status:# service multipathd status

Load the driverStart the service

Start and stops the service







4. Create multipath device with the multipathcommand:# multipath -v2create: mpath0 (360a980004335432d576f526c526d6d6a)NETAPP,LUN

[size=2.0G][features=1 queue_if_no_path][hwhandler=0][n/a]

\_ round-robin 0 [prio=16][undef]

\_ 1:0:2:0 sdd 8:48 [undef][ready]

\_ 1:0:3:0 sde 8:64 [undef][ready]

\_ 2:0:2:0 sdh 8:112 [undef][ready]

\_ 2:0:3:0 sdi 8:128 [undef][ready]


\_ 1:0:0:0 sdb 8:16 [undef][ready]

\_ 1:0:1:0 sdc 8:32 [undef][ready]

\_ 2:0:0:0 sdf 8:80 [undef][ready]

\_ 2:0:1:0 sdg 8:96 [undef][ready]

The LUN is visible






Single Disk or Manage by Volume Manager

My_IP_igroupiqn.1999-05.com.redhat:rhelOS Type: linux

My_FC_igroup10:00:00:00:c9:6b:77:b4OS Type: linux

Ethernet

Ethernet

Fibre Channel

Fibre Channel

Treat LUNs as a single disk or combine together using a volume manager

1 2

/


mount lunb

mount luna

Target

Initiator

LUNbLUNa

SINGLE DISK OR MANAGE BY VOLUME MANAGER LUNs may be used as single disk or combined together using a host-based volume manager.





Single Disk Configuration

Format the multipath device:# fdisk /dev/mapper/mpath0

Command (m for help): n

Command action

e extended

p primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-261, default 1): 1

Last cylinder or +size or +sizeM or +sizeK(1-261, default 261): 261

Command (m for help): t

Hex code (type L to list codes): 83

Command (m for help): w

SINGLE DISK CONFIGURATION





Single Disk Configuration (Cont.)

Create a file system in the multipath device:# mkfs -t ext3 /dev/mapper/mpath0

Mount the multipath device:# mkdir /mnt/lunx

# mount /dev/mapper/mpath0 /mnt/lunx

Test access:# cd /mnt/lunx

# touch foo

SINGLE DISK CONFIGURATION (CONT.)





Volume Managers

Red Hat 5.3 supports the following volume managers:– Logical Volume Manager (LVM) 2 Native support Flexible capacity New ASCII metadata format

– Veritas This module and accompanying exercise will

focus on LVM2

VOLUME MANAGERS





LVM2 Architecture

PhysicalVolume

PhysicalVolume

Volume Group

Logical Volume

Logical Volume

LUN 1 LUN 2

LVM2 ARCHITECTURE To create a LVM2 logical volume, the physical volumes and/or LUNs are combined into a volume group (VG). This creates a pool of disk space out of which LVM2 logical volumes (LV) can be allocated. This is very much like how NetApp can assign physical disks to an aggregate (the LVM2’s volume group) and then can create flexible volumes (the LVM2’s logical volume) out of the resources of the aggregate.





LVM2 Configuration Overview

1. Initialize partitions for a LVM2 volume group:– This step labels the partitions appropriately

2. Create a volume group3. Create a logical volume from the resources in

a volume group4. Create a file system with the logical volume5. Mount the logical volume:

– Create a mount point– Mount the logical volume

LVM2 CONFIGURATION OVERVIEW





LVM2 Configuration

1. Initialize partitions for a LVM2 volume group# multipath -v1 -l

mpath0 mpath2

# dd if=/dev/zero of=/dev/mapper/mpath0 bs=512 count=1

# lvm

lvm> pvcreate/dev/mapper/mpath0 /dev/mapper/mpath2

lvm> pvs

PV VG FmtAttrPSizePfree

/dev/dm-2 lvm2 -- 2.00G 2.00G

/dev/dm-3 lvm2 -- 2.00G 2.00G

/dev/sda2 VolGroup00 lvm2 a- 68.12G 0

Identify the LUNs to add to the volume group

Create physical partitions

View physical partitions

Not requiredif new LUN; lvm2 requiresno partitiontable

LVM2 CONFIGURATION





LVM2 Configuration (Cont.)

2. Create a volume group:lvm> vgcreate vgGroup1 /dev/mapper/mpath0

/dev/mapper/mpath2

lvm> vgsVG #PV #LV #SN Attr VSize VFreeVolGroup00 1 2 0 wz--n- 68.12G 0vgGroup1 2 0 0 wz--n- 3.99G 3.99G

lvm> vgdisplay vgGroup1--- Volume group ---VG Name vgGroup1System IDFormat lvm2Metadata Areas 2Metadata Sequence No 1VG Access read/writeVG Status resizable....

LVM2 CONFIGURATION (CONT.)






3. Create a logical volume from the resources in a volume group:

lvm> lvcreate -L 3G -n lvVolA vgGroup1

lvm> lvsLV VG Attr Lsize......lvVolA vgGroup1 -wi-a- 3.00G

lvm> lvdisplay -v /dev/vgGroup1/lvVolAUsing logical volume(s) on command line--- Logical volume ---LV Name /dev/vgGroup1/lvVolAVG Name vgGroup1LV UUID ViZevW-AxTx-UUKe-ASig-SDOG-J3FN-feRDf7LV Write Access read/writeLV Status available...







4. Create a file system with the logical volume:– EXT3 file system:

# mkfs -t ext3 /dev/vgGroup1/lvVolA

– GFS file system: # gfs_mkfs -plock_nolock -j 1 /dev/vgGroup1/lvVolA

NOTE: The exercise environment will use the EXT3 file system







5. Mount the logical volume– Create a mount point:

# mkdir /mnt/lvVolAfs

# mount /dev/mapper/vgGroup1-lvVolA /mnt/lvVolAfs

– Or use /etc/fstab– Test access:

# cd /mnt/lvVolAfs

# touch foo






Red Hat Stack

/mnt/lvVolAfsMounts

LUN 1 LUN 2

/dev/sdb - /dev/sdiSCSI Devices Path

/dev/sbj - /dev/sdm

LUN 1

LUN 2

Logical VolumeFile System Added EXT3

/dev/mapper/mpath0DM-Multipath Devices Path /dev/mapper/mpath2

LUN 1

LUN 2

LVM2

Volume Group

LUN 1 LUN 2

Logical Volume

/dev/mapper/mpath0LUN 1

Single Volume

EXT3

/mnt/lunx

RED HAT STACK





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe the steps to allow a Red Hat initiator

to access a LUN on a storage system

MODULE SUMMARY




Exercise

Module 12: Red Hat LUN AccessEstimated Time: 60 minutes



13-1 SAN Implementation Workshop: LUN Provisioning


LUN Provisioning


LUN PROVISIONING





Module Objectives

By the end of this module, you should be able to: Describe how and when a LUN consumes

space from its containing volume Discuss backup guarantees through

Snapshot™ reserve Discuss the overwrite guarantee for space-

reserved LUNs Analyze the default LUN configuration and two

thin provisioning configurations

MODULE OBJECTIVES





Space-Reserved LUNs

SPACE-RESERVED LUNS





Space Reservation

LUNs are either: – Space reserved, that is, full provisioning (default)

The LUN size is reserved or “taken” out of the volume when the LUN is created

May provide overwrite protection depending upon fractional reserve setting - discussed later

Only allowed on a FlexVol® volume with volume or file volume space guarantee

– Nonspace reserved, that is, thin provisioning Data is “taken” out of the volume when written to the LUN by the

host client LUN reservation set by:

– lun create -o noreserve...

– lun set reservation {enable|disable}

– During lun setup command:... Do you want the LUN to be space reserved? [y]:

SPACE RESERVATION





Protection for LUNs

When creating a LUN, two objectives become apparent:– How to create backups for a LUN– How to ensure that an application can continue

to write to a LUN To analyze these objectives, this presentation

will investigate three concepts:– Snapshot copies– Snapshot reserve– Fractional reserve

PROTECTION FOR LUNS





Backup Guarantee

BACKUP GUARANTEE





Snapshot Copies

Snapshot copies allow quick and efficient backups of volumes that contain a LUN

However, when taking Snapshot copies, Data ONTAP®cannot guarantee that no application is writing to the LUN– Results: possible inconsistent Snapshot copies

To guarantee consistent Snapshot copies:– Shut down applications using the LUN– Unmount the file system– Use APIs to activate hot backup modes SnapDrive®

SnapManager ®

SNAPSHOT COPIES





Snapshot Reserve

Snapshot reserve defines a percentage of the volume that is reserved for Snapshot copies– Set at the volume level

– Volumes used in SAN environments should have the Snapshot reserve set to zero

system> snap reserve

Volume vol_SAN1: current snapshot reserve is 20% or 2097152 k-bytes.

20%

Volume 1 Space Reservation

Snapshot Reserve

SNAPSHOT RESERVE





Verify Space Available for a LUN

lun maxsize returns the maximum possible size of a LUN on a given volume or qtree with or without Snapshot reservesystem> lun maxsize /vol/vol_SAN1

Space available for a LUN of type: solaris, aix, hpux, linux, netware, vmware, openvms or image

Without snapshot reserve: 5.9g (6356467712)

With snapshot reserve: 2.0g (2134900736)

With complete snapshot reserve: 2.0g (2134900736)

Space available for a LUN of type: windows, windows_gpt or windows_2008

Without snapshot reserve: 5.9g (6349916160)

With snapshot reserve: 2.0g (2130347520)

With complete snapshot reserve: 2.0g (2130347520)

VERIFY SPACE AVAILABLE FOR A LUN





Overwrite Guarantee

OVERWRITE GUARANTEE





Overwrite Protection

Overwrites– When a Snapshot copy is first taken, the data

blocks are owned by the active file system and the Snapshot copy

– Only as the data is overwritten in a LUN are the data blocks exclusively owned by the Snapshot copy

Fractional reserve– Provides extra protection for ensuring that a host

can write to a LUN that has Snapshot copies– Independent of Snapshot reserve

OVERWRITE PROTECTION





Fractional Reserve Defined

Fractional reserve– A volume attribute vol options vol_name fractional_reserve [%]

– Fractional reserve can be between 0% to 100% Defaults to 100% Less than 100% allows thin provisioning Snapshot space of

LUNs

– A multiplier for space-reserved LUNs– Space is reserved when a Snapshot copy is taken Equaling to the LUN’s Snapshot size, up to the total overwrite

protection size Total overwrite protection = LUN size x fractional reserve

– The overwrite protection space is used only after all other space in the volume

FRACTIONAL RESERVE DEFINED





Space Reservation Enabled Fractional

Reserve 100%

vol1 1 GB

10-GB volume is created – 80% Active File System (AFS)

and 20% Snapshot Reserve(SSR) df -r report:

Fractional Reserve Example

FRACTIONAL RESERVE EXAMPLE





Create a space-reserved LUN:– lun create –s 2g–t windows /vol/vol1/LUN1

df -r reports:

Fractional Reserve Example (Cont.)


Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR

FRACTIONAL RESERVE EXAMPLE (CONT.)





Write 100% to the LUN

df -r reports:



Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR






First Snapshot copy is created– snap create vol1 snap1

df -r reports:



Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR

Actual LUN usage is still 4






Overwrite the entire LUN

df -r reports:



Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR







Second Snapshot copy is created– snap create vol1 snap2

df -r reports:



Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR







Overwrite the entire LUN again

df -r reports:




Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR






Third Snapshot copy is created– snap create vol1 snap3

df -r reports:



Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR








df -r reports:



Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR







Fourth Snapshot copy is created– snap create vol1 snap4

df -r reports:


Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR









df -r reports:


Actual LUN usage is 2


Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR






Fifth Snapshot copy fails– Cannot guarantee overwrite space

= Snapshot size x fractional reserve df -r reports:



Reserve 100%

vol1 1 GB

Full Reserved

LUN1 (2GB) SSR






Analysis

ANALYSIS





Snapshot and Fractional Reserve

Snapshot reserve set asides space in a volume for backups– Snapshot reserve may expand into the active file

system Fractional reserve is used in calculating the

amount of space in a volume set aside to ensure overwrites– If a LUN is completely filled and a Snapshot copy is

taken, then with a fractional reserve of 100%, there is enough space guaranteed to completely overwrite the LUN and still preserve the old data through the Snapshot copy

SNAPSHOT AND FRACTIONAL RESERVE





Snapshot and Fractional Reserve (Cont.)

Fractional reserve may not be an efficient use of space

Example 1: Fully Provisioned

Even with fractional reserve set to 100%, the system still ran out of space

Fractional reserve only delayed the inevitable

Conclusion: Better Snapshot copies management is needed

SNAPSHOT AND FRACTIONAL RESERVE (CONT.)





Snapshot and Fractional Reserve (Cont.)

Fractional reserve may not be an efficient use of space

Example 2: Fully Provisioned

Administrators have to plan a larger volume size to provide for the guaranteed overwrite reserve

And the LUN may never need the overwrite reserve

Conclusion:Thin Provisioning

SNAPSHOT AND FRACTIONAL RESERVE (CONT.)





Snapshot / Fractional Reserve: Conclusion

Use fractional reserve if you need guaranteed overwrites– To minimize space usage, you may consider disabling

fractional reserve However, Snapshot copies can fill up a volume if not

managed properly– Could prevent writes to a LUN if no space available

To better manage Snapshot copies, administrators may:– Delete Snapshot copies (automatically) – Expand the size of the volume that contains the LUN

SNAPSHOT / FRACTIONAL RESERVE: CONCLUSION





Snapshot Automatic Delete

Snapshot autodelete determines when (if) Snapshot copies will be automatically deleted– Set at volume level

snap autodelete vol[on|off|show|reset]

If autodelete is enabled, then options:snap autodelete vol options option val

OptionsValuecommitment try, disrupttrigger volume, snap_reserve, space_reservetarget_free_space 1-100delete_order oldest_first, newest_firstdefer_delete scheduled, user_created, prefix, noneprefix <string>

SNAPSHOT AUTOMATIC DELETE





Volume Autosize

Might want to grow the volume vol autosize determines if a volume should grow

when nearly full Both snapshot autodelete and vol autosize

use the value wafl_reclaim_threshold– Data ONTAP 7.1 - 7.2.3: 98%– Data 7.2.4 and later (threshold depends on volume

size):

Variable Name VOL size Valuewafl_reclaim_threshold_t: tiny vols < 20G threshold = 85%wafl_reclaim_threshold_s: small vols from 20G to < 100G threshold = 90%wafl_reclaim_threshold_m: medium vols from 100G to < 500G threshold = 92%wafl_reclaim_threshold_l: large vols from 500G to < 1T threshold = 95% wafl_reclaim_threshold_ xl: xlarge vols from 1T up threshold = 98%

VOLUME AUTOSIZE This value when changed from the defaults is not persistent; it will change back to the default values after reboot. So if you want to change this value (for example 90% for tiny volumes of less than 20G) and have it persist after reboots then you should add the following line to EACH /etc/rcfile on BOTH controllers: priv set –q diag;

setflagwafl_reclaim_threshold_t90;

priv set;





Volume Autosize (Cont.)

Configuration– Set at volume level– Possible values: ON

– Increment size (default 5% of original size)– Maximum size (default 120% of original size)

OFF– vol autosize vol_name [-m size[k|m|g|t]]

[-i size[k|m|g|t]] [on|off|reset]

VOLUME AUTOSIZE (CONT.) NOTE: Volume autosize can only be run a maximum of 10 times on any particular volume. If you set the incremental size too small, you will not be able to expand it as much as you may like. For that reason, you generally should use the -m and -i switch when configuring the volume autosize feature to set the incremental size and the maximum size to something larger than the defaults. ALSO NOTE: The volume can grow only to a maximum size that is 10 times the original volume size.





Administrator’s Choice

Administrators may choose which procedure to employ first: – snapshot auto delete

– vol autosize

– Use the volume option: try_first

Possible values:– snap_delete

– volume_grow (default)

Example:– vol options vol_name try_first snap_delete

ADMINISTRATOR’S CHOICE





Configurations

CONFIGURATIONS





Thin Provisioning Configuration #1

Easy to manage Sacrifices Snapshot copies before LUNs Doesn’t use shared space of aggregate until auto_grow is used

Configuration Default ValueGuarantee volumeLUN reservation onFractional Reserve 0%Snapshot Reserve 20%Auto delete snap_reserveAuto grow ontry_first snap_delete

THIN PROVISIONING CONFIGURATION #1





Thin Provisioning Configuration #2

Uses shared space of aggregate Deletes Snapshot copies when maximum size

reached Volumes are independent of each other

Configuration Default ValueGuarantee volumeLUN reservation onFractional Reserve 0%Snapshot Reserve 0%Auto delete volumeAuto grow ontry_first auto_grow

THIN PROVISIONING CONFIGURATION #2





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe how and when a LUN consumes

space from its containing volume Discuss backup guarantees through Snapshot

reserve Discuss the overwrite guarantee for space-

reserved LUNs Analyze the default LUN configuration and two

thin provisioning configurations

MODULE SUMMARY




Exercise

Module 13: LUN ProvisioningEstimated Time: 45 minutes



14-1 SAN Implementation Workshop: SAN Management


SAN Management


SAN MANAGEMENT





Module Objectives

By the end of this module, you should be able to: Perform administrative tasks on FC target ports Perform administrative tasks on LUNs Perform administrative tasks on igroups

MODULE OBJECTIVES





Port Management

PORT MANAGEMENT





Port Management

Storage administrators may need to perform the following tasks while managing FC target ports on a storage system: Enable and disable the FC target port

(previously discussed) Assign onboard FC ports to be targets

(previously discussed) Designate an alias for FC initiator or target

ports (previously discussed) Configure FC attributes to FC target ports

(previous discussed) Configure the WWPN on an FC target port

PORT MANAGEMENT





Port Assignment CLI Command

With Data ONTAP® 7.3 and later, WWPNs may be directly assigned by administrators

An assigned WWPN may only be chosen from a set of valid WWPNs– Use fcp subcommand: portname

In active-active environments, you can only set the WWPNs for locally managed ports and not the partner’s ports

PORT ASSIGNMENT CLI COMMAND





Steps to Assign a WWPN on an FC port

1. Use fcp portname show to display all assigned WWPNs

2. Use fcp config adapter down to bring adapter offline

3. Use fcp portname set adapter wwpn to assign an unused WWPN to a port– Use fcp portname swap adapter1 adapter2

to swap WWPNs between ports 4. Use fcp config adapter up to bring the

adapter online again5. Validate the new configuration

STEPS TO ASSIGN A WWPN ON AN FC PORT





Example: fcp portname show

Use fcp portname show -v to display used and unused WWPNs:– Example:

system> fcp portname show -v

Portname Adapter

-------- -------

50:0a:09:81:96:f7:c7:86 0c

50:0a:09:82:96:f7:c7:86 0d

50:0a:09:83:96:f7:c7:86 unused

50:0a:09:84:96:f7:c7:86 unused

50:0a:09:85:96:f7:c7:86 unused

50:0a:09:86:96:f7:c7:86 unused

50:0a:09:87:96:f7:c7:86 unused

50:0a:09:88:96:f7:c7:86 unused

...

EXAMPLE: FCP PORTNAME SHOW





Setting FC Target Port’s WWPNs

Adapter must be offline before changing WWPN :– fcp config adapter down

Use fcp portname set to configure the port– Example:

system> fcp portname set 0d 50:0a:09:83:96:f7:c7:86

WARNING! Changing WWPN on an adapter may cause

initiator(s) fail to reconnect to this adapter.

Do you want to continue (y/n)? y

Change will take effect when adapter is onlined.

SETTING FC TARGET PORT’S WWPNS





Swapping FC Target Port’s WWPNs

WWPNs of two ports may be swapped– Adapters must be offline to perform this action– Use fcp portname swap command– Example:

system> fcp portname swap 0c 0d

WARNING! Changing WWPNs on adapters may cause

initiator(s) fail to reconnect to these adapters.

Do you want to continue (y/n)? y

Changes will take effect when adapters are onlined.

Bring the adapter online after configuration: – fcp config adapter up

SWAPPING FC TARGET PORT’S WWPNS





LUN Management

LUN MANAGEMENT





LUN Management

Storage administrators often need to perform the following tasks while managing LUNs on a storage system: Disable or enable a LUN Add a comment to a LUN Rename a LUN Resize a LUN Clone a LUN Remove a LUN

LUN MANAGEMENT





LUN Offline/Online

To control availability of LUN, administrators may: Take a LUN offline

– lun offline lun_path

Example:system> lun offline /vol/vol_SAN1/lun0

Bring a LUN online– lun online lun_path

Example:system> lun online /vol/vol_SAN1/lun0

LUN OFFLINE/ONLINE





Add a Comment to a LUN

A LUN description is usually to identify the purpose of a LUN To add a comment:

– lun comment lun_path [comment]

Example:system> lun comment /vol/vol_SAN1/lun0 “For Payroll”

NOTE: If you are using spaces within the comment, place the comment within quotes

To view a LUN comment:– lun comment lun_path command– lun show -v

ADD A COMMENT TO A LUN





Rename a LUN

To rename a LUN: lun move old_lun_path new_lun_path

– Example:system> lun move /vol/vol_SAN1/lun0

/vol/vol_SAN1/newlun

NOTE: Both paths must be in the same volume

RENAME A LUN





Resize a LUN

Administrators may increase or decrease the size of a LUNNOTE: It is usually more reliable to create a new LUN andcopy the data to it than resize an existing LUN

A host file system supports only limited usage of this feature– Windows® supports resizing only on basic disks– Veritas™ does not support resizing on 3.5 or earlier

A LUN can only grow 10x its original size To resize a LUN:

– Take the LUN offline– lun resize [-f] lun_pathnew_size

RESIZE A LUN





Clone a LUN

A LUN clone is a point-in-time, writeable copy of a LUN in a Snapshot™ copy

The LUN clone shares space with a LUN in its backing Snapshot copy

Initial LUN clone’s content exists in the backing Snapshot copy while changed data is written to the active file system

LUN clone may be used for testing

CLONE A LUN To clone a LUN, complete the following steps:

1. Begin the clone operation, enter the following command:

lun clone start lun_clone_path

2. To display the progress of the clone operation,enter the following command:

lun clone status lun_clone_path

3. To display all the clones,enter either of the following commands:

lun clone show [lun-path] lun show -c

4. If you need to stop the clone process,enter the following the command:

lun clone stop lun-path





Snapshot File System

Clone a LUN (Cont.)

Unallocated block in LUN

Allocated block in sameLUN (same inode) at given layer

Block visible from lowerlayer (shared) LUN

system> snap create /vol/vol_SAN1 s0

Create a Snapshot copy of the volume that

contains the LUN

Active File System

CLONE A LUN (CONT.)





LUN Clone with Backing




LUN Attribute Pointerto Snapshot “Backup”

Clone a LUN (Cont.)

system> lun clone create /vol/vol_SAN1/lun0clone –b /vol/vol_SAN1/lun0 s0

Allocated block in LUN atgiven layer with backing

These blocks are different

Create a LUN clone with s0 Snapshot copy as its backing


Active File System

CLONE A LUN (CONT.)





Clone a LUN (Cont.)

Snapshot copies that are used as a backing Snapshot copy for a LUN clone are in a busy state and cannot be deleted:– Use lun snap usage to identify which LUN

clones are using a particular Snapshot copy– Example:

system> lun snap usage vol_SAN1 s0

Active:

LUN: /vol/vol_SAN1/lun0clone

Backed By: /vol/vol_SAN1/.snapshot/s0/lun0

NOTE: You must split or destroy the LUN clone todelete the Snapshot copy

Use lun show -v to display all LUNs and LUN clones with their backing Snapshot copies

CLONE A LUN (CONT.)





LUN Clone with Backing




LUN Attribute Pointerto Snapshot “Backup”

Clone a LUN (Cont.)

Allocated block in LUN atgiven layer with backing

These blocks are different

system> lun clone split start /vol/vol_SAN1/lun0clone

Start the split


Active File System

CLONE A LUN (CONT.)





Remove a LUN

The process to remove a LUN depends on the initiator OS

This course will investigate:– Windows Server 2003/2008/2008 R2– vSphere™– Red Hat® Enterprise Linux® 5

Regardless of the initiator OS, the first step is to stop all applications that are using the LUN as storage

REMOVE A LUN





Remove a LUN: Windows Server

Take the LUN offline

Right-click,select offline

REMOVE A LUN: WINDOWS SERVER





Remove a LUN: Windows Server(Cont.)

In Windows Server 2008, you can’t take the disk offline, so use Device Manager to “uninstall” the device

Right-click andselect uninstall

REMOVE A LUN: WINDOWS SERVER(CONT.)





Remove a LUN: Windows Server (Cont.)

Regardless of whether the LUN was accessed through FC or iSCSI, the following steps are used to destroy it:– Take the LUN offline:

system> lun offline /vol/vol_SAN2/lun3

– Unmap the LUN from igroup:system> lun unmap /vol/vol_SAN2/lun3 iWin_fcp

– Destroy the LUN offline:system> lun destroy /vol/vol_SAN2/lun3

FC example






Remove a LUN: Windows Server(Cont.)

Rescan and the LUN should disappear

Right-click, select

Rescan Disks






Removing VM(s) using the LUN

Remove a LUN: vSphere

Right-click, select

remove

REMOVE A LUN: VSPHERE





Removing a LUN destroys data

Remove a LUN: vSphere (Cont.)

Right-click, select Delete

To keep the data, take the LUN offline on the storage system

REMOVE A LUN: VSPHERE (CONT.)





Rescan and the LUN should disappear


LUN is gone







Regardless of whether the LUN was accessed through FC or iSCSI, the following steps are used to destroy it: Take the LUN offline:system> lun offline /vol/lun2/lun2

Unmap the LUN from igroup:system> lun unmap /vol/lun2/lun2 win-1617b81a

Destroy the LUN offline:system> lun destroy /vol/lun2/lun2

iSCSI example






Remove a LUN: Linux

Unmount the logical volume# umount /dev/mapper/vgGroup1-lvVolA

Remove the mount point# rmdir /mnt/lvVolA

Inactivate the logical volume# lgchange -an lvVolA

Remove the logical volume# lvremove /dev/vgGroup1/lvVolA

REMOVE A LUN: LINUX





Remove a LUN: Linux (Cont.)

Inactivate the volume group:# vgchange -an vgGroup1

Either remove a physical volume (LUN):# pvs -0+pv_used

PV VG Fmt Attr Psize Pfree Used

/dev/dm-2 vgGroup1 lvm2 a- 2.00G 2.00G 0

/dev/dm-3 vgGroup1 lvm2 a- 2.00G 2.00G 0

/dev/sda2 VolGroup00 lvm2 a- 68.12G 0 68.12G

# vgreduce vgGroup1 /dev/dm-3

# vgchange -ay vgGroup1

Or destroy the volume group all together:# vgremove vgGroup1

REMOVE A LUN: LINUX (CONT.)






Check the SCSI device definitions:# multipath -d

...

create: mpath2 (360a980004335432d576f52726b663350) NETAPP,LUN

[size=2.0G][features=1 queue_if_no_path][hwhandler=0][n/a]


\_ 3:0:0:0 sdj 8:144 [undef][ready]

\_ 4:0:0:0 sdk 8:160 [undef][ready]

\_ 5:0:0:0 sdl 8:176 [undef][ready]

\_ 6:0:0:0 sdm 8:192 [undef][ready]

One LUN;record SCSI device IDs







List the current multipath definitions:# ls -l /dev/mapper...

brw-rw---- 1 root disk 253, 2 Sep 13 13:53 mpath0


brw-rw---- 1 root disk 253, 5 Sep 14 08:31 mpath3...

Remove the LUN’s multipath definition:# multipath -f mpath2

# ls -l /dev/mapper...


brw-rw---- 1 root disk 253, 5 Sep 14 08:31 mpath3...

LUN definition

gone






List the current SCSI device definitions:

...

# ls -l /sys/bus/scsi/drivers/sd...lrwxrwxrwx 1 root root 0 Sep 14 09:47 3:0:0:0 -> ../../../../devices/platform/host3/session1/target3:0:0/3:0:0:0lrwxrwxrwx 1 root root 0 Sep 14 09:47 3:0:0:1 -> ../../../../devices/platform/host3/session1/target3:0:0/3:0:0:1lrwxrwxrwx 1 root root 0 Sep 14 09:47 4:0:0:0 -> ../../../../devices/platform/host4/session2/target4:0:0/4:0:0:0lrwxrwxrwx 1 root root 0 Sep 14 09:47 4:0:0:1 -> ../../../../devices/platform/host4/session2/target4:0:0/4:0:0:1lrwxrwxrwx 1 root root 0 Sep 14 09:47 5:0:0:0 -> ../../../../devices/platform/host5/session3/target5:0:0/5:0:0:0lrwxrwxrwx 1 root root 0 Sep 14 09:47 5:0:0:1 -> ../../../../devices/platform/host5/session3/target5:0:0/5:0:0:1lrwxrwxrwx 1 root root 0 Sep 14 09:47 6:0:0:0 -> ../../../../devices/platform/host6/session4/target6:0:0/6:0:0:0lrwxrwxrwx 1 root root 0 Sep 14 09:47 6:0:0:1 -> ../../../../devices/platform/host6/session4/target6:0:0/6:0:0:1...


One LUN







Take the LUN offline:system> lun offline /vol/lun2/lun2

Unmap the LUN from igroup:system> lun unmap /vol/lun2/lun2 lunix-igroup

Destroy the LUN offline:system> lun destroy /vol/lun2/lun2







Remove the SCSI definitions:

Verify:# ls -l /sys/bus/scsi/drivers/sd...lrwxrwxrwx 1 root root 0 Sep 14 09:47 3:0:0:1 -> ../../../../devices/platform/host3/session1/target3:0:0/3:0:0:1lrwxrwxrwx 1 root root 0 Sep 14 09:47 4:0:0:1 -> ../../../../devices/platform/host4/session2/target4:0:0/4:0:0:1lrwxrwxrwx 1 root root 0 Sep 14 09:47 5:0:0:1 -> ../../../../devices/platform/host5/session3/target5:0:0/5:0:0:1lrwxrwxrwx 1 root root 0 Sep 14 09:47 6:0:0:1 -> ../../../../devices/platform/host6/session4/target6:0:0/6:0:0:1...

# echo 1 > 3:0:0:0/delete# echo 1 > 4:0:0:0/delete# echo 1 > 5:0:0:0/delete# echo 1 > 6:0:0:0/delete

Local devices are

gone






igroup Management

IGROUP MANAGEMENT





igroup Management

Storage administrators often need to perform the following tasks while managing igroups on a storage system: Map and unmap LUNs and igroups Define port sets for an igroup

IGROUP MANAGEMENT





Map or Unmap igroups and LUNs

To unmap an igroup and LUN:– Take a LUN offline– lun unmap lun_path igroup LUN_ID

Example:system> lun offline /vol/vol_SAN1/lun0

system> lun unmap /vol/vol_SAN1/lun0 iUnix_fcp 0

To map an igroup and LUN: – lun map lun_path igroup LUN_ID

Example:system> lun map /vol/vol_SAN1/lun0 iUnix_fcp 0

MAP OR UNMAP IGROUPS AND LUNS





Port Sets

Port sets are a collection of ports associated with an igroup If an igroup is not associated with a port set,

then an initiator that belongs to that igroup can see its target LUNs on all ports

If an igroup is associated with a port set, then an initiator that belongs to that igroup can see its target LUNs on the ports belonging only to the port set

PORT SETS





Port Set Administration

To create a port set:system> portset create -f myportset

– NOTE: -f means FC – Only FC port sets are currently supported

Add port to a port set:system> portset add myportset 0csystem> portset showmyportset (FCP):

ports:system 0csystem2 0c

To associate a port set to an igroup:system> igroup create -f -t solaris -a myportset newigroup

– NOTE: The port set cannot be empty for an igroupto bind to it

PORT SET ADMINISTRATION





Port Set Administration (Cont.)

To bind an igroup and a port set (without creating the igroup):system> igroup bind newigroup myportset

To disassociate an igroup and a port set:system> igroup unbind newigroup

To destroy a port set:system> portset destroy myportset

PORT SET ADMINISTRATION (CONT.)





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Perform administrative tasks on FC target ports Perform administrative tasks on LUNs Perform administrative tasks on igroups

MODULE SUMMARY




ExerciseModule 14: SAN ManagementEstimated Time: 30 minutes



15-1 SAN Implementation Workshop: SAN Troubleshooting


SAN Troubleshooting


SAN TROUBLESHOOTING





Module Objectives

By the end of this module, you should be able to: Describe how to diagnose a problem within a

SAN environment Review diagnostic tools and techniques

available for:– Data ONTAP®

MODULE OBJECTIVES





SAN Troubleshooting

SAN TROUBLESHOOTING





Origin of Problems

Initiator Target

Direct-Attached Storage

Fabric/Network

SCSI over FC

SCSI over TCP/IP(iSCSI)

FCDriver

SCSIDriver

FileSystem

Application

SCSI Adapters

Windows or UNIXFC HBAs

Data ONTAP

LUN

Ethernet NICs

SCSI over TCP/IP(iSCSI)

iSCSI HBAs

Storage System

FC or SATA Attached

TCP/IP

iSCSIDriver

iSCSIDriver

TCP/IP

SAN ServicesFCDriver

WAFL®

RAID

ORIGIN OF PROBLEMS





The SAN Troubleshooting Process

Is there a problem with the fabric or network? Is there a problem with one of your switches? Is there a problem with the target? Is there a problem with the host?

THE SAN TROUBLESHOOTING PROCESS





Diagnose the Problem

Understand the SAN environment– What is the fabric or network configuration? Are you using Fibre Channel protocol or iSCSI? What kind of zoning or VLANs are in use (if any)? What is the switch type? Has any new software been installed on the host or target?

– What is the storage system configuration? What is the storage system model? What version of Data ONTAP is running? Is FC or iSCSI licensed and started? Is the logical unit number (LUN) created and mapped to the

correct igroup? Is the system cabled in an active-active storage controller

configuration?

DIAGNOSE THE PROBLEM





Diagnose the Problem (Cont.)

Understand the SAN environment– What host(s) are involved and what is their

configuration? What is the OS version? Are necessary patches

installed? Is this an iSCSI or FC solution? If iSCSI, do you

have an HBA or software initiator? What applications are running (SQL Server™,

Oracle®, Exchange, and so on)? What host-side multipathing and/or clustering

options are being used? Is SnapDrive®being used?

DIAGNOSE THE PROBLEM (CONT.)





Determine the Nature of the Problem

Can the host and target communicate? Is the problem performance related? Are LUNs visible to host? Do you receive error messages? Did the host panic? Did the storage system

(target) panic? When does the problem occur?

– After reboot (of host or target)?– During heavy load? – Does the problem occur when SnapMirror® is

started?

DETERMINE THE NATURE OF THE PROBLEM





Fabric and Network Troubleshooting

What is the fabric or network protocol (FC or iSCSI)?

What type of fabric and network is being used?– Direct Attached (FC or iSCSI) – Issues may be

related to cabling, port hardware– Single fabric (FC) - Issues may be related to

switches, zoning– Dual fabric (FC) - Issues may be related to

switches, zoning– Network attached (dedicated or shared

Ethernet) (iSCSI) – Issues may be related to VLANs, a firewall, router tables

FABRIC AND NETWORK TROUBLESHOOTING





FC Fabric Checklist

Do you have path connectivity? Are the switches on? Do they have unique

domain IDs? If multiple switches are present, do they have

identical fabric parameters? Are zones set up correctly? Are switches running correct firmware

versions? Are the ports configured correctly? Are hard and soft zones configured correctly? Is the physical cabling correct?

FC FABRIC CHECKLIST





iSCSI Storage Network Checklist

Do you have path connectivity? Are duplex and jumbo frame Ethernet

parameters configured correctly? Are you using a firewall? Are VLANs configured correctly? Are hosts and controllers cabled correctly? Is

port hardware functioning correctly? Was the port state reconfigured by an

administrator (shared Ethernet)? Are clients configured correctly to find the DNS

and iSNS servers?

ISCSI STORAGE NETWORK CHECKLIST





Data ONTAP

DATA ONTAP





Data ONTAP Troubleshooting

To troubleshoot a SAN environment in Data ONTAP: Verify SAN configuration on a storage system Display general usage statistics Analyze FC statistics Analyze iSCSI statistics Verify read and write performance

DATA ONTAP TROUBLESHOOTING





Verify LUN Environmentsystem> lun config_check -vChecking igroup ostype & fcp cfmode compatibility======================================================

No Problems Found

Checking local and partner cfmode======================================================

No Problems Found

Checking for down fcp interfaces======================================================The following FCP HBAs appear to be down

0c OFFLINED BY USER/SYSTEM

Checking initiators with mixed/incompatible settings======================================================

No Problems Found

Potential Problem

VERIFY LUN ENVIRONMENT





Verify LUN Environment (Cont.) Checking igroup ALUA settings======================================================

No Problems Found

Checking for nodename conflicts======================================================

No Problems Found

Checking for initiator group and lun map conflicts======================================================

No Problems Found

Checking for igroup ALUA conflicts======================================================

No Problems Found

VERIFY LUN ENVIRONMENT (CONT.)





Usage Statisticssystem> uptime12:57pm up 4:42 0 NFS ops, 0 CIFS ops, 0 HTTP ops, 70 FCP ops,

425 iSCSI ops

system> availtimeService statistics as of Thu Apr 24 12:57:20 PDT 2008System (UP). First recorded (15393111) on Mon Oct 29 09:05:29 PDT 2007

P 16, 1441184, 1439532, Fri Dec 14 06:44:46 PST 2007U 3, 542, 206, Wed Nov 14 14:28:39 PST 2007PF 1, 36368, 36368, Sat Dec 15 02:36:13 PST 2007

...FCP (UP). First recorded (11421816) on Fri Dec 14 07:13:44 PST 2007

P 12, 14325, 13548, Mon Apr 7 18:48:06 PDT 2008PF 1, 36375, 36375, Sat Dec 15 02:36:20 PST 2007

iSCSI (UP). First recorded (13485623) on Tue Nov 20 09:56:57 PST 2007P 12, 5394, 3078, Thu Apr 17 15:43:29 PDT 2008PF 1, 36375, 36375, Sat Dec 15 02:36:20 PST 2007

USAGE STATISTICS





Display FC Protocol Statisticssystem> fcp statsStatistics for adapter 0cRead Ops: 0Write Ops: 0Other Ops: 25KBytes In: 0KBytes Out: 0Adapter Resets: 1Frame Overruns: 0Frame Underruns: 0Initiators Connected: 3Link Breaks: 0LIP Resets: 0SCSI Requests Dropped: 0Interrupts: 66Spurious Interrupts: 0Total Logins: 11Total Logouts: 8CRC Errors: 0Adapter Qfulls: 0Queue Depth: 0

Statistics for adapter 0dRead Ops: 0Write Ops: 0Other Ops: 30KBytes In: 0KBytes Out: 0Adapter Resets: 0Frame Overruns: 0Frame Underruns: 0Initiators Connected: 4Link Breaks: 0LIP Resets: 0SCSI Requests Dropped: 0Interrupts: 88Spurious Interrupts: 0Total Logins: 8Total Logouts: 4CRC Errors: 0Adapter Qfulls: 0Queue Depth: 0

DISPLAY FC PROTOCOL STATISTICS





Display FC Protocol Statistics (Cont.)Statistics for adapter vticRead Ops: 0Write Ops: 0Other Ops: 6KBytes In: 1KBytes Out: 0out_of_vtic_cmdblks: 0out_of_vtic_msgs: 0out_of_vtic_resp_msgs: 0out_of_bulk_msgs: 0out_of_bulk_buffers: 0out_of_r2t_buffers: 0

...

Zero out the statistics to get a clear readingsystem> fcp stats -z

DISPLAY FC PROTOCOL STATISTICS (CONT.)





Display iSCSI Statisticssystem> iscsi statsiSCSI PDUs ReceivedSCSI-Cmd: 226 | Nop-Out: 8 | SCSI TaskMgtCmd: 0LoginReq: 620 | LogoutReq: 21 | Text Req: 5DataOut: 0 | SNACK: 0 | Unknown: 0Total: 880

iSCSI PDUs TransmittedSCSI-Rsp: 89 | Nop-In: 8 | SCSI TaskMgtRsp: 0LoginRsp: 620 | LogoutRsp: 21 | TextRsp: 5Data_In: 218 | R2T: 0 | Asyncmsg: 2Reject: 0Total: 963

DISPLAY ISCSI STATISTICS





Display iSCSI Statistics (Cont.)iSCSI CDBsDataIn Blocks: 218 | DataOut Blocks: 0Error Status: 8 | Success Status: 218Total CDBs: 226

iSCSI ERRORSFailed Logins: 0 | Failed TaskMgt: 0Failed Logouts: 0 | Failed TextCmd: 0Protocol: 0Digest: 0PDU discards (outside CmdSN window): 0PDU discards (invalid header): 0Total: 0

Zero out the statistics to get a clear readingsystem> iscsi stats -z

DISPLAY ISCSI STATISTICS (CONT.)





Read and Write Performance of a LUN

To improve read/write performance of a LUN, it is important to have the LUN (a single file) sequentially within the aggregate:system> reallocate on

Reallocation scans are on

system> reallocate start /vol/vol_SAN1/lun0

NOTE: This will create a normal reallocation scan that runs every 24 hours

To disable the scans:system> reallocate stop /vol/vol_SAN1/lun0

READ AND WRITE PERFORMANCE OF A LUN





nSANity

nSANity is an extensible diagnostic data collection framework

Download from the NOW™ site (Windows and Linux version)

Supports:– Data ONTAP 7G, GX, and 8.0 7-Mode– Switches Cisco®

Brocade®

QLogic– Hosts Windows® Server 2003 and 2008 VMware® 3.0.1, 3.5, and 4.0 Linux® with any 2.6 kernel-based distributions Solaris™

More supported platforms coming soon

NSANITY





nSANity (Cont.)

With nSANity, you can interrogate your Windows, Linux, and storage systemC:\nsanity-win32> nsanity.exe windows://localhost

Using internal command definitions

Starting collection @ 2009-09-08 17:30:49

Beginning data collection for windows @ localhost

Running select * from Win32_OperatingSystem

Running select * from Win32_QuickFixEngineering

Running select * from Win32_LogicalDisk

Running select * from Win32_LogicalDiskRootDirectory

Running select * from Win32_LogicalDiskToPartition

...

Collected data saved to file, 20090909003049_nsanity.xml.gz

NSANITY (CONT.)





nSANity (Cont.)

You can then pass this file off to your NetApp®

Professional Services representative to read the resulting xml output

NSANITY (CONT.)





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to: Describe how to diagnose a problem within a

SAN environment Review diagnostic tools and techniques

available for:– Data ONTAP®

MODULE SUMMARY





Additional Resources

Education– Instructor-led training Design and Implement VMware Solutions on NetApp

Storage Design and Implement MS Hyper-V Solutions on NetApp

Storage Design and Implement Virtualization Solutions on NetApp

Storage– Online courses SnapDrive for Windows Overview and Install SnapDrive for Windows Operate and Troubleshoot

Web sites– NOW (NetApp on the Web) Site

http://NOW.NetApp.com– NetApp

http://www.netapp.com

ADDITIONAL RESOURCES




Thank you!

Please fill out an evaluation.

THANK YOU!


A1-1 SAN Implementation Workshop: FC Details


FC Details

Appendix 1SAN Implementation Workshop

FC DETAILS





Module Objectives

Bythe end of this module, you should be able to:Describe FC details important to configure and troubleshoot FC topologies

MODULE OBJECTIVES





Fabric Cabling

FABRIC CABLING





Cabling: Types

Multimode fiber– Typically orange or aqua

blue in color – Cable type is always

printed on the cable– 50 or 62.5µm core – 50/125 = 50µm core and

125µm cladding

Singlemode fiber– Not directly supported by

NetApp– Typically yellow in color– 9µm core

CABLING: TYPES





Cabling: Illustration

Multimode will have a 50 µm or 62.5 µm coreSinglemode will have an 8.3/9 µm core

Cable Jacket: The outermost layer of the fiber cable

Strengthening fibers: The strengthening fibers that help protect the core against damage during installation or from being crushed

Coating: This layer of thicker plastic surrounds the cladding and helps protect the fiber coreCladding: The layer that protects the core and causes the necessary reflection to allow light to travel through the fiber-core segmentCore: The physical component that transports the optical data signal, made up of a continuous strand of glass. The core's diameter is measured in microns

CABLING: ILLUSTRATION





Cabling: Distances

Multimode optical cable distances:– All wavelengths below are 850 nm

Media Type Speed Distance Channel Attenuation----------------------------------------------------------------------------------50 µm 1 Gb 2-500m 3.2562.5 µm 1 Gb 2-300m 2.5550 µm 2 Gb 2-300m 2.5562.5 µm 2 Gb 2-150m 2.0350 µm 4 Gb 2-150m62.5 µm 4 Gb 2-70m

CABLING: DISTANCES





Cabling: Distances

Note: Distances assume perfect (no) joints, patch panels, etc. YMMV…

CABLING: DISTANCES





Cabling: Cable Attenuation

Do not mix 50 µm and 62.5 µm– This commonly occurs with extension cablesPatch panels and fiber extension cables significantly reduce total cable lengthENSURE that you have accounted for every segment of the cable run If a patch panel is in a run, you MUST determine channel attenuation– Cable distance formulas are available on the

NOWTMsite

CABLING: CABLE ATTENUATION Channel attenuation is the maximum amount of signal loss, in decibels (dB), supported by the interface specification over the maximum stated cable and data rate parameters. The loss value is based upon a direct point-to-point connection that has a single source and single destination connection and continuous fiber, no patch or splice connections.

For 50 and 62.5 µm at 850 nm:

Cable attenuation: 3.5 dB/km Connector pair loss: 0.75 dB Splice loss: 0.3 dB





Most commonly seen:

– Lucent Connector (LC)

– Siemens Connector (SC)

– Straight Tip (ST)

– Multi-fiber Push-On (MPO)

– Small Form Factor Pluggable (SFP)

Cabling: Connectors

CABLING: CONNECTORS





FC Addressing

FC ADDRESSING





Nodes Attached to a Fabric

Nodes (ports) link to a fabric– Send out a FLOGI

(fabric login)– Get back a Fiber

Channel ID (unique)– Register with fabric as an

initiator or target– Initiators can request

targets which are visible (zoning)

Initiators then can PLOGI (login into target port)– Creates a session

HostInitiator

HostInitiator

HostInitiator

JBODTarget

SystemTarget

Switch1Port Port Port

Port Port PortPort Port

NODES ATTACHED TO A FABRIC Nodes (ports) link to a fabric

Send out a FLOGI Get back a Fabric Channel ID (unique ID) Register with Fabric as Initiator or Target Initiator can say give me targets allowed to see (zoning)

Then Initiators can then PLOGI (login in to target Port)

Create a session





Port Types in Switched Configuration

HostN_Port

HostN_Port

HostN_Port

HostN_Port

NL_PortSystemN_Port

JBODNL_Port

Switch1F_Port F_Port F_Port

F_Port U_Port FL_PortU_Port E_Port Switch2

U_Port U_Port F_Port

U_Port U_Port FL_PortE_Port U_PortISL

PORT TYPES IN SWITCHED CONFIGURATION Device port types:

N_Port: Node port, a fabric device directly attached (Host or RAID) NL_Port: Node loop port, a device attached to a loop (just a bunch of disks, also called

JBOD) - FAS270 (Virtual port) Switch port types:

E_Port: An expansion port that is used for Inter-Switch Links ISL: An Inter-Switch Link is a link that connects switches together utilizing E_Ports F_Port: A fabric port to which an N_Port (node) attaches FL_Port: A fabric loop port is a port to which an NL_Port (loop device) attaches G_Port: A generic port that is in a transitional state either to become an E_Port or F_Port U_Port: A universal port that is waiting to become some other port

An N_Port connects to an F_Port

An NL_Port connects to an FL_Port

Switch port types are typically auto-configured

E ports carry frames between switches for configuration and fabric management. They serve as a conduit between switches for frames destined to remote N ports and NL ports. E ports support class 2, class 3, and class F service.




There are several less common switch port types:

QLogic and Cisco:

TL_Port - In translative loop port (TL port) mode, an interface functions as a translative loop port. It may be connected to one or more private loop devices (NL ports). TL mode allows nonfabric-aware devices to communicate with fabric devices.

Cisco-specific:

TE_Port: In trunking E port (TE port) mode, an interface functions as a trunking expansion port. It may be connected to another TE port to create an extended ISL (EISL) between two switches.

Brocade-specific:

EX_Port - An E_Port from a router to an edge fabric; the router terminates EX_Ports preventing fabric merges

L_Port - A loop port that is only displayed in switchshow output VE_Port - A virtual E_Port that terminates at the switch and does not propagate fabric

services or routing topology information from one edge fabric to the other VEX_Port - A virtual E_Port that terminates at the switch and does not propagate fabric

services or routing topology information from one edge fabric to the other, when an FCIP connection is involved





Identifiers

HostN_Port

HostN_Port

HostN_Port

NL_PortSystemN_Port


F_Port U_Port FL_PortU_Port E_Port

Each switch has a unique Domain ID

Each port has an associated Area ID

If the port is associated with multiple nodes (such as a NL_Port), the Port Number is something other than 00

IDENTIFIERS





Fibre Channel ID

Each device in the fabric is assigned a 24-bit address during FLOGIThe format is XXYYZZ, where:– XX = Domain ID– YY = Area ID – ZZ = Port Number (AL_PA)

Area ID(8 bits)

Domain ID (8 bits)

Port Number(8 bits)

24-bit Address

FIBRE CHANNEL ID The node address, also known as arbitrated loop physical address (ALPA) identifies a device on a Fibre-Channel Arbitrated Loop (FC-AL). For example, it could identify a particular disk within an FC-AL JBOD. For point-to-point connections between the switch and the edge device, the node address (ZZ) is set to 0x00.

The domain ID usually represents the domain ID assigned to the FC switch where the FC node connects.

The area ID usually represents the port number on the FC switch where the FC node connects (for example, if a host is connected to port 77, area ID = 4D).

Cisco switches use a different method for determining the area ID.

The domain ID and the area ID parts of the FC ID are assigned during the fabric login (FLOGI) process by the FC login server fabric service. The login server fabric service is discussed further as part of the fabric services section.





Domain ID

A unique number between 1 and 239 that identifies the switch (or virtual switch) in a fabricUnless configured statically, the principal switch manages domain ID addressingRecommendation is to use static domain IDs– Useful when hard zoning– Needed for persistent FC IDsBrocade:– Use configure to set the Domain ID and other

parameters

DOMAIN ID





Switch Roles

Principal switch:– Default behavior is that the switch with the

lowest WWN will become the principal switch– Principal switch is defined during a fabric

change eventA new switch cannot become the principal switch when it joins a stable fabric

– Recommendation is to designate a core switch as principal

SWITCH ROLES





Principal Switch

Principal switch functions– Assigns domain IDs to subordinate switches– Merges zoning information– Exchanges FSPF routing tables– Exchanges SNS information– In Brocade fabrics, it handles time

synchronization for all the switches in the fabric

PRINCIPAL SWITCH If not using static domain IDs, preferred IDs, or contiguous assignment (Cisco) when a switch attempts to join the fabric, it will send a random domain ID to the principal switch for approval. The principal switch will either accept the ID (if it is not in use) or deny the ID (if it is in use).





Fibre Channel Login

Types of Fibre Channel Login– Fabric login– Port login– Process login

FIBRE CHANNEL LOGIN





Fabric Login

HostN_Port

HostN_Port

HostN_Port

JBODNL_Port

DiskN_Port



FLOGI: Fabric login is a process by which a node makes a logical connection to the fabric domain

HostN_Port

JBODNL_Port

Switch2U_Port U_Port F_Port

U_Port U_Port FL_PortE_Port U_Port

FABRIC LOGIN After a fabric-capable Fibre Channel device is attached to a fabric switch, it will carry out a fabric login, or FLOGI

Similar to port login, FLOGI is an extended link service command that sets up a session between two participants. With FLOGI, a session is created between an N_Port or NL_Port and the switch.

An N_Port will send a FLOGI frame that contains its node name, its N_Port Name, and service parameters to a well-known address of 0xFFFFFE (login server).





Fabric Services: Login Service

Handles FLOGIA port that needs to connect to the fabric must log in with this serverThe node sends a FLOGI frame with the source ID (S_ID) filed filled in only for its AL_PA value The Login Service then sends a response with the destination ID (D_ID) field filled in with the device's AL_PA and the newly assigned domain and area values

FABRIC SERVICES: LOGIN SERVICE AL_PA: Arbitrated loop physical address also known as node address. Used with Fibre Channel-Arbitrated Loop (FC-AL) devices, otherwise AL_PA=0.





FC ID in Data ONTAP

Before FLOGIsystem> fcp show adapter -v 0c

Slot: 0c

Description: Fibre Channel Target Adapter 0c (Dual-channel, QLogic 2322 (2362) rev. 3)

Status: ONLINE

Host Port Address: 0x000000...

After FLOGIsystem> fcp show adapter -v 0c

Slot: 0c

Description: Fibre Channel Target Adapter 0c (Dual-channel, QLogic 2322 (2362) rev. 3)

Status: ONLINE

Host Port Address: 0x010000...

No Fabric ID

Fabric ID assigned

FC ID IN DATA ONTAP





Port Login

HostN_Port

HostN_Port

HostN_Port

JBODNL_Port

DiskN_Port



PLOGI: Port login is a port-to-port login process where initiators establish sessions with targets

HostN_Port

JBODNL_Port



PORT LOGIN Port login, or PLOGI, is used to establish a session between two N_Ports (devices)

Required before any upper-level commands or operations can be performed. Two N_Ports swap service parameters and make themselves known to each other.

In a point-to-point topology, PLOGI is the process by which an FC initiator port establishes a connection with an FC target port. In a switched topology, PLOGI is the process by which an FC initiator port or FC target port establishes a connection with the switch.





Fabric Services: Name Service

Simple name server is also known as SNSThe name server is a database used to store information about devices attached to a fabricThe name server gets information from a device through the PLOGI frame at initialization and through subsequent registration frames Maps the 24-bit fabric address and the 64-bit WWNHosts query the name server for all attached devices and their capabilities

FABRIC SERVICES: NAME SERVICE





Process Login

HostN_Port

HostN_Port

HostN_Port

JBODNL_Port

DiskN_Port



PRLI: Process login is used to set up the environment between related processes on an originating N_Port and a responding N_Port

HostN_Port

JBODNL_Port



PROCESS LOGIN A group of related processes is collectively known as an image pair. Use of process login is optional from the perspective of the Fibre Channel FC-2 layer, but may be required by a specific upper-level protocol, as in the case of SCSI-FCP mapping.





Fabric Addressing: Example

RAIDca0200

RAID120300

120100 120200 ca0100

JBOD12040f

Domain ID: 18 Domain ID: 202ISL

AL_PA: 15

FABRIC ADDRESSING: EXAMPLE Domain ID: 18 = 1216

Domain ID: 202 = ca16

AP_PA for JBOD: 15 = 0F16

AP_PA for P2P nodes: 0





FC Services

FC SERVICES





Fabric Services

All switches implement a set of services that are distributed across all the devicesTypes of Fabric Services– Login services - 0xFFFFFE– Name services - 0xFFFFFC– Registered State Change Notification (RSCN) -

0xFFFFFD

FABRIC SERVICES Switches distribute fabric information among themselves through Class F service frames. Switches exchange information in their servers so that all individual switch servers contain the same information. This creates a singular fabric entity and makes it appear that there is only one of each type of server.

Like nodes, fabric services have addresses, but the address of a fabric service is a fixed value called a well-known address. Well-known addresses are reserved by the FC standard.





Fabric Services: RSCN

Provides state change notification service to registered nodesDevices that use this service are generally hosts that want to keep track of a number of storage targetsA device registers for state change notification by transmitting a state change registration frame (SCR) to the well-known addressWhen there is a change in fabric topology, the switch controller transmits a registered state change notification frame to the device to let it know that there has been a changeIt is up to the device to query the name server to assess the state of the fabric at this time

FABRIC SERVICES: RSCN





FC Routing Mechanisms

FC ROUTING MECHANISMS





Routing Mechanisms

Complex fabric can be made of interconnected switches and director

ISL

ROUTING MECHANISMS





Routing Mechanism: FSPF

Fabric shortest path first (FSPF) is the standard path selection protocol in FC

ISL

ROUTING MECHANISM: FSPF FSPF keeps tracks of links on all switches and associates a cost with each link Cost = proportional to the number of hops FSPF computes paths from a switch to all other switches in the fabric by adding the cost

of all links traversed by the path FSPF chooses the path that minimizes the cost





Routing Mechanism: Trunking

Trunking is a feature of switches that enables traffic to be distributed across available ISLs while still preserving in-order delivery

ROUTING MECHANISM: TRUNKING





ISLs Without Trunking

ISLs provide for connection between switchesAny switch in the fabric can have one or more links to another switch in the fabric ISLs are allocated in a round-robin fashion to share the load on the system (S_IDs are assigned a default path through a specific ISL)Adding an ISL between switches is dynamic and can be done while the switch is active

ISLS WITHOUT TRUNKING Adding a new Inter-Switch Link (ISL) will result in a routing re-computation and new allocation of ISL links between source and destination ports. Similarly, removing a link will result in fabric shortest path first (FSPF) routing re-computation across the fabric and possible fabric reconfiguration.

Adding ISLs will cause routing traffic and zoning data to be updated across the fabric through a spanning tree. The total number of ISLs is not as relevant as the amount of configuration changing, as each change will result in a recalculation of routes in the fabric. When numerous fabric reconfigurations occur (removing or adding links, rebooting a switch, and so on), the load on the switches’ CPUs will be increased and some fabric events may time out while waiting on CPU response. This occurs only during fabric reconfiguration activities and does affect not frame traffic per se, only tasks that require use of the CPU. No CPU intervention is required for normal frame routing; this is all done by switch hardware.

No more than eight ISLs between any two switches are supported. More than eight ports can be used on a switch for ISL traffic as long as no more than eight go to a single adjacent switch.

NOTE: A spanning tree connects all switches from the so called principal switch to all subordinate switches. This tree spans in a way such that each switch (or leaf of the tree) is connected to other switches, even if there is more than one ISL between them―that is to say, there are no loops.





ISLs Without Trunking (Cont.)

The switch guarantees in-order delivery. However, if one Fibre Channel device loads up its assigned ISL highly and for lengthy periods of time, a second device assigned to this same ISL may not get all of its data through in a timely manner.

4 parallel ISLs

load diminished

Director Director

ISLS WITHOUT TRUNKING (CONT.)





ISLs With Trunking

Combines two to eight ISLs into a single logical connection between switches

load diminished

Director Director

Director Director

Director Director

ISL Trunking

no load impaired

ISLS WITH TRUNKING ISL trunking:

- Load is balanced among the ISLs in the trunk

- Minimizes congestion

- Is typically auto-configured

ISL trunking is an optional license for Brocade switches.





Cisco PortChannels

Trunking is a commonly used term in the storage industry However, the Cisco SAN-OS software and switches in the Cisco MDS 9000 Family implement trunking and PortChanneling as follows:– PortChanneling enables several physical links to be

combined into one aggregated logical link– Trunking enables a link transmitting frames in the

EISL format to carry (trunk) multiple VSAN traffic – When trunking is operational on an E port, that E

port becomes a TE portA TE port is specific to switches in the Cisco MDS 9000 FamilyAn industry standard E port can link to other vendor switches and is referred to as a non trunking interface (see Figure 14-2 and Figure 14-3)

CISCO PORTCHANNELS





Fabric Events

FABRIC EVENTS





Fabric Connectivity: FSPF

Fabric shortest path first– Standard path selection protocol used by Fibre

Channel fabrics– FSPF keeps track of the links on all switches in

the fabric and associates a cost with each link– Enabled by default on all switches– Automatically calculates the best path between

any two switches in a fabric

FABRIC CONNECTIVITY: FSPF FSPF dynamically compute routes throughout a fabric by establishing the shortest and quickest path between any two switches.

Select an alternative path in the event of the failure of a given path. FSPF supports multiple paths and automatically computes an alternative path around a failed link.

It provides a preferred route when two equal paths are available. The preferred route is chosen by different algorithms by different vendors.

FSPF uses a topology database to keep track of the state of the links on all switches in the fabric and associates a cost with each link.

Each time a new switch enters the fabric, a link state record (LSR) is sent to the neighboring switches, and then flooded throughout the fabric.

Link costs are based on speed and hop count:

1 Gb = 1,000 2 Gb = 500

Links can be given a static value and a reference for doing so is provided here.

http://www.redbooks.ibm.com/abstracts/tips0034.html?Open





Fabric Events: RSCN

Registered state change notification (RSCN)– Fibre Channel service that informs hosts about

changes in the fabric– Hosts receive this information by registering with

the fabric controller (0xFFFFFD)– It is up to the nodes to query the name server

again to obtain the new information– The details of the changed information are not

delivered by the switch in the RSCN that was sent to the nodes

FABRIC EVENTS: RSCN





Fabric Events: RSCN (Cont.)

An RSCN is typically disruptive, I/O momentarily pauses– Significantly impacts tape devices and hosts

that require streaming read or writeThe impact of an RSCN can be mitigated through zoning or RSCN suppression (each switch vendor implements it differently)– Configure RSCN suppression for initiators only

FABRIC EVENTS: RSCN (CONT.)





Fabric Events: RSCN (Cont.)

An RSCN is generated when:– Disks join or leave the fabric– A name server registration change occurs

A host joins or leaves the fabric– A new zone is activatedIf each host is in a separate zone, RSCN suppression is not required

FABRIC EVENTS: RSCN (CONT.)





Brocade FC Switches: Licensing

Many features are optional– Advanced zoning – ISL trunking– Advanced performance monitoring– Extended fabric: Allows higher port buffer

credits for long-distance connections– Fabric watch: SAN health monitor– Full fabric

VL2 switches only support 2 switches in a fabricVL4 switches only support 4 switches in a fabric

BROCADE FC SWITCHES: LICENSING





Brocade FC Switches: Licensing (Cont.)

Read the product bulletins for each switch– http://web.netapp.com/engineering/projects/releases/sled

gehammer/brocade/brocade_project_page.htmOn newer model switches, do not assume all ports are licensed– 200e ships by default with 8 SFPs and only 8 of the 16

ports licensedCan be upgraded to 12 or 16 ports (license)The 200e is a no-fabric switch by default

– 4900 ships by default with only 32 of its 64 ports licensedAlways check customer’s equipment for a “full fabric” license through licenseShow

BROCADE FC SWITCHES: LICENSING (CONT.)





Brocade FC Switches: Licensing (Cont.)

NetApp packages the following licenses:– Restricted fabric 3200/3800/3250/3850/200E

Web tools Advanced zoning

– Full fabric 3200/3800/3250/3850/200E/3900/4100/4900/48000

Web tools Advanced zoning Fabric watch

– All switches may add the extra cost optional licenses of Advanced Performance Monitoring and the Distance bundle that includes licenses for extended fabrics, remote switch, and trunking

BROCADE FC SWITCHES: LICENSING (CONT.)





FC Zoning

FC ZONING





Zoning

A zone is a list of Fibre Channel ports or nodes that can communicate with each other in a Fibre Channel fabricA zone is essentially a filter that limits the information the name server returns to the initiator during a query

ZONING Devices can be members of more than one zone. This enables the creation of zones that contain some, but not all, of the same members. This type of configuration is referred to as overlapping zones. In this case, a concept of “most permissible access” is employed. As long as at least one zone contains both the initiator and target, the target is made available to the initiator, even though other zones containing the initiator do not contain the target device.





Zoning: Reasons for Use

Provides control over which nodes in a fabric can view and access which resourcesIncreases network securityLimits data loss or corruptionReduces or eliminates cross-talk between initiator HBAsLimits available paths

ZONING: REASONS FOR USE Some hosts require that the number of available paths between each host and target pair be limited. For example, HP® PV-Links can use only eight paths; additional paths can expose the LUN to unregulated access and possibly to data corruption.

Further, reducing the number of paths provides the following two benefits:

Simplifies administration of attached disks on a UNIX® host by making the output of host platform tools (such as iostat, sanlun, and ioscan/lsdev) more concise and easier to read.

Eliminates the additional CFO time that occurs with single-path active-passive setups. For example, consider a 32-path setup where the multipathing software uses a single-path active-passive path selection algorithm, as required by DotHillSANpath on AIX® hosts clustered with high-availability cluster multiprocessing (HACMP). If a cluster failover occurs, the host must attempt and fail all 16 primary paths consecutively before reaching the failed storage system’s partner through a secondary path. This can take much longer than the regular failover time. Reducing the number of paths to eight or less can prevent this increased downtime.

With fewer paths to monitor, zoning results in a limited number of registered state change notifications (RSCNs) to a given group of devices on a storage area network (SAN), thereby improving SAN availability and performance.

By allowing data access to be restricted, zoning enhances security of the cluster.





Zoning: Reasons for Use (Cont.)

Reduces fabric service trafficIncreases overall stabilitySeparate tape-based SAN traffic from disk-based SAN trafficSeparate NetApp® SAN traffic from other vendors SAN traffic (recommended)

ZONING: REASONS FOR USE (CONT.) Whenever a change occurs in the name server, such as a device addition to, or removal from, the fabric, the fabric OS generates a State Change Notification (SCN). In the absence of zoning, an SCN is sent to all devices in the fabric, with each device querying the name server to determine how the membership of the fabric has changed. This process occurs even if the fabric change does not affect the device being notified. Especially in large fabrics, this can result in a significant amount of fabric service traffic (although typically for only a short time).

For instance, if a new initiator joins the fabric, there is little reason to notify all the other initiators of the change because initiators do not usually communicate with each other. An equivalent situation exists with targets. Because targets generally do not communicate with each other, they have little use in being notified about the addition or removal of another target.

Although all devices are supposed to handle SCN traffic without affecting normal operation, this is not always the case. Thus, the overall stability of the fabric increases after zoning is implemented: the fabric can restrict SCN delivery to only those devices in zones which contain the added device. The same holds true when a device is removed from the fabric. The list of available devices is restricted to just those of interest to the initiator, rather than all devices in the fabric.





Zone Configuration

To configure a zone:– Configure default zoning mode– Identify the fabric type of zone to create– Create the zone– Add fabric objects to the zone– Create zone configuration– Identify how to enforce zoningThis appendix will focus on Brocade’s Fabric OS 6.2

ZONE CONFIGURATION





Default Zoning Mode

Default zoning mode determines what occurs if no zoning implemented or if no effective zone configurationChoices:– All access– No access To set the default zoning modeswitch> defzone --noaccess [--allaccess]

To view the default zoning modeswitch> defzone --show

DEFAULT ZONING MODE





Zone Types

Different vendors have different types of zonesBrocade with Fabric OS 6.2:– Regular zones– Broadcast zones– Traffic isolation zones– QoS zones– Redirection zonesThis module will focus on regular zones

ZONE TYPES Regular zone

Broadcast zones restricts boardcast well-known address (FFFFFF) packets to only those devices that are members of the broadcast zone. A broadcast zone does not allow access within its members in any way. If you want to allow or restrict access between any devices, you must create regular zones for that purpose.

Traffic isolation zones is an implementation of a traffic isolation routing features which allows restricting of ISL communication between switches by using failover or load-balancing techniques.

Quality of Service (QoS) zones allows administrators to assign high or low priority to prioritize traffic.

Frame Redirection zones provides a means to redirect traffic flow between a host and a target to virtualization and encryption applications so that those applications can perform without having to reconfigure the host and target.

Please see the Fabric OS Administrator’s Guide for your version of Brocade’s Fabric OS.





Best Practices of Zoning

Always zone using the highest Fabric OS-level switchZone using the core switch versus an edge switchZone using an enterprise-class platform rather than a switchNetApp recommends creating a separate zone for with a 1:1 mapping between an initiator port and a target port– If this is not possible, create zones from each

initiator port and all target ports

BEST PRACTICES OF ZONING





Zoning: Naming Conventions

If working on an existing SAN, use the established naming conventionNaming convention goals– Simple– Meaningful – Consistent

Use A-Z, a-z, 0-9, _Zoning Implementation Strategies for Brocade SAN Fabrics– Search for Zoning_Imp_WP_00.pdf at

www.brocade.com

ZONING: NAMING CONVENTIONS Zoning Implementation Strategies for Brocade SAN Fabrics:

http://www.brocade.com/san/pdf/whitepapers/Zoning_Imp_WP_00.pdf





Zoning: Naming Conventions (Cont.)

Alias prefixes– SRV: Server– STO: Storage– TPE: Tape– VRA: Virtualization appliance

Examples:– SRV_MAILSERVER_SLT5: Server with hostname

“mailserver” in PCI slot 5– TPA_LTO9_SNG: Tape with LTO drive number 9, single-

attached– STO_DSK3456_5C: Storage unit with serial ID 3456 on

the fifth card in port C– STO_FAS3050c_C1_1a: Controller 1 of a FAS3050c

connected to target port 1a

ZONING: NAMING CONVENTIONS (CONT.)






For dual fabrics, append a fabric identifier– SRV_MAILSERVER_SLT5_fabA– SRV_MAILSERVER_SLT5_fabric_A







Zones– Zone name should identify the host or cluster– Use the ZNE_ prefix– Include the fabric name if applicable– Zone for a host

ZNE_MAILSERVER_SLT5 (single fabric)ZNE_MAILSERVER_SLT5_fabric_A

– Zone for a cluster:ZNE_MSCS_EXCH_fabric_A (would contain)– SRV_MSCS_EXCH1_fabric_A– SRV_MSCS_EXCH2_fabric_A– STO_x







Zone sets / configurations– Use PROD_ prefix for the primary fabric

configurationPROD_A or PROD_FABRIC_APROD_B or PROD_FABRIC_B

– For temporary configurations use descriptive names

BACKUP_ARECOVERY_2TEST_18jun02







Valid characters: a-z, A-Z, 0-9, $-^_Alias names cannot use a hyphen (-), use underscore (_)NetApp Zoning Central– Search for Zoning.html at

http://eng-web.rtp.netapp.com

ZONING: NAMING CONVENTIONS (CONT.) NetApp Zoning Central

Browse to http://eng-web.rtp.netapp.com/techpubs/docs/zoning/zoning.html





Zone Creation

To create a regular zoneswitch> zonecreate “zonename”,

“member[;member...]”

switch> cfgsave

– zonename is the logical name of the zone– member is a single member or list of members:

A domain,port pairDevice WWNNDevice WWPNZone alias

To add members to the zoneswitch> zoneadd “zonename”,


switch> cfgsave

A collection of device WWNN or WWPN created using aliCreatecommand

Save the configuration after any modifications

ZONE CREATION





Zoning: Zone Sets

A zone set consists of one or more zonesA zone set can be activated or deactivated as a single entity across all switches in the fabricOnly one zone set can be activated at any timeA zone can be a member of more than one zone setZone set = Brocade “configuration”

ZONING: ZONE SETS





Zoning: Zone Sets (Cont.)

Zone Set: ZNE_Host1_fab_a

STO_RAID1_p1

SRV_Host1_1a SRV_Host2_1a STO_FAS3050a_0c

STO_RAID2_p1

Switch

ZNE_Host2_fab_aZNE_Host1_fab_a

Zone Set: NEW_fab_a

STO_RAID1_p1

SRV_Host1_1a SRV_Host2_1a STO_FAS3050a_0c

STO_RAID2_p1

Switch

ZNE_Host2_fab_a_new

ZNE_Host1_fab_a_new

ZONING: ZONE SETS (CONT.)





Zone Configuration

To create a zone configurationswitch> cfgcreate “cfgname”,


switch> cfgsave

– cfgname is the logical name of the zone– member is a zone name or list of zone names

Enable and disable a zone configurationswitch> cfgenable “cfgname”

switch> cfgdisable “cfgname”

Viewed active configurationswitch> cfgactvshow

You can also add and remove members and delete zone configuration

ZONE CONFIGURATION





Zone Enforcement

Zones have a list of fabric objectsZone enforcement can be done with:– Software-enforced– Hardware-enforced

ZONE ENFORCEMENT





Software-Enforced Zoning

Called “soft zoning”, “fabric-based zoning” or “session-based zoning”Prevents hosts from discovering unauthorized target devicesEnsures that the name server does not return any information to an unauthorized initiator in response to a name server queryDoes not prohibit access to the device... just limits access to name service query

SOFTWARE-ENFORCED ZONING





Hardware-Enforced Zoning

Called “hard zoning” or “ASIC-enforced zoning”Prevents host from discovering and communicating to unauthorized target devicesEach frame is checked by hardware (ASIC) before the packet is delivered

HARDWARE-ENFORCED ZONING





Zone Enforcement Identification

Brocade’s Fabric OS uses hardware-enforced zoning (for each zone) whenever the fabric membership or zone configuration changes– NOTE: connecting a device specified by WWN

to a port specified in a domain,port zone results in loss of the hardware enforcement in both zones

To identify the current zone enforcement mechanism:switch> portzoneshow

ZONE ENFORCEMENT IDENTIFICATION





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Describe FC details important to configure and troubleshoot FC topologies

MODULE SUMMARY


A2-1 SAN Implementation Workshop: Introduction to FCoE


Introduction to FCoE


INTRODUCTION TO FCOE





Module Objectives

By the end of this module, you should be able to:Distinguish the differences between Fibre Channel (FC), Fibre Channel over Ethernet (FCoE) and Internet SCSI (iSCSI) protocolsDescribe path implementation with FCoE connectivityDescribe how to configure FCoE ports on Windows and NetApp systems

MODULE OBJECTIVES





Protocols Stacks

OSI iSCSISCSI

iSCSI

TCP

IP

MAC

Physical

FCoEFC-4 Protocol

FC-3 Service

FC-2 Frame

FCoE Mapping

MAC

Physical

FC6 Presentation

5 Session

4 Transport

3 Network

2 Data Link

1 Physical

FC-4 Protocol

FC-3 Service

FC-2 Frame

FC-1 Data

FC-0 Physical

PROTOCOLS STACKS





FCoE Connectivity Configuration

The following are the steps to configure a FCoE SAN:1. Determine the FC Topology2. Cable the nodes3. Configure the target(s) 4. Configure the initiator(s)5. Verify Connectivity

FCOE CONNECTIVITY CONFIGURATION





FCoE Topology

FCOE TOPOLOGY





FCoE: Past Topologies

FCTarget

FC Initiator

FC Fabric

Before FCoE, Ethernet and FC were discreet networks

Web

Ethernet

FCOE: PAST TOPOLOGIES





FCoE: Current Topologies

FCoEEnabledSwitch

FCTargetFC Fabric

Web

FCoE Initiator

With FCoEand a FCoE-enabled switch, initiators can communicate with FC and FCoE networks;Still two discreet networks

Ethernet

FC Initiator

FCOE: CURRENT TOPOLOGIES





FCoE: Future Topologies

FCoEEnabledSwitch

FCTargetFC Fabric

Web

FCoE Initiator

Ethernet

FC Initiator

FCoE Initiator

Lossless Ethernet cloud (1Gb or 10Gb)

Add FCoE initiators in the Ethernet cloud

FCOE: FUTURE TOPOLOGIES





FCoE: Future Topologies (Cont.)

FCoEEnabledSwitch

FCTargetFC Fabric

Web

FCoE Initiator

Ethernet

FC Initiator

FCoE Initiator

Lossless Ethernet cloud (1Gb or 10Gb)

Add FCoE targets in the Ethernet cloud

FCoETarget

FCOE: FUTURE TOPOLOGIES (CONT.)





FCoEEnabledSwitch FC Fabric

FCoE:Traffic flow

Blade Servers or Racks of Servers w/ CNA or S/W FCoE

Cisco Nexus5020 (available)Brocade (soon)

4GB or 8GBFC environment

FCoEInitiator FC

TargetMAC FC MAC FCFC FC

FC Fabric

FCOE:TRAFFIC FLOW





Current FC + FCoE = SINGLE SAN

Stateless, just encapsulates and decapsulates

Consolidation without management issues

Single namespace,single management space

FCoEEnabledSwitch

FCoE Initiator

FCTarget

FC Initiator

FC Fabric

FCoETarget

Web

Standard FC Session

CURRENT FC + FCOE = SINGLE SAN





Future FC + FCoE = SINGLE SAN

Stateless, just encapsulates and decapsulates

Standard FC Session

Consolidation without management issues

Single namespace,single management space

FCoEEnabledSwitch

FCoE Initiator

FCTarget

FC Initiator

FC Fabric

FCoETarget

Web

Ethernet

FUTURE FC + FCOE = SINGLE SAN





Network Topology

Storage System 1

Windows

iSCSI

. . .Windows

RequiresLossless Ethernet

ConvergedNetwork Adapters (CNAs)

FCoE 10-GbE switch

NETWORK TOPOLOGY Requires lossless Ethernet

Either physical channel pause (non-consolidated) or Priority based Flow Control (PFC 802.1Qbb)

Requires mini-Jumbo Frames (~2.5k)

there is no segmentation of FC frames





FCoE Adapters Types

Since FCoE implements SAN over Ethernet, administrators have choices on how to connect to an FCoE network:

ServerProcessing

NIC/HBAProcessing

10 Gb NIC & FCoE Soft Initiator

Ethernet

Ethernet Driver

Net Device

SCSI

Application

FCoE Layer

FCoE Hardware Initiator/CNA

Ethernet

Ethernet Driver

Net Device

FCoE Layer

SCSI

Application

FCOE ADAPTERS TYPES For Data ONTAP 7.3.2, the product requirements for this HBA are:

Support product quality of FCoE target function. The level of feature and quality are expected to be comparable to 8G FC target HBA.

Support demo quality of NIC function to be used by iSCSI software target. Although we have implemented the driver for NIC function in 7.3.2, it is disabled by default. That means an user can not use its NIC function. To enable the NIC function, it needs to go through PVR process.





Switch Configuration

SWITCH CONFIGURATION





Switch Configuration Overview

Configuring a Cisco Nexus 5020 for FCoE– 3 step process for loading and enabling FCoE

Load the license– Install license <licensefile>

Enable the FCoE feature– Via Gui or “feature fcoe”

Reboot the switch– Reload

SWITCH CONFIGURATION OVERVIEW





FCoE Switch: Cisco Nexus 5020

FCOE SWITCH: CISCO NEXUS 5020





Switch Configuration StepsHow to configure a port to accept FCoE– Enter configuration mode

# conf t– Create Virtual interface group (VIG)

(config)# int vig 4– Bind a physical interface to the VIG

(config-if)# bind int eth 1/4– Create a virtual FC port on the VIG then enable the port

(config-if)# int vfc 4/1(config-if)# no shut

– Create a virtual ethernet port on the VIG then enable the port(config-if)# int veth 4/1(config-if)# no shut

– Exit the VIG configuration mode(config-if)# exit

– Force enable FCoE on the physical ethernet port(config)# int eth 1/4(config-if)# fcoe mode on

SWITCH CONFIGURATION STEPS





Switch Configuration Example-----------------------------------------------------------------------------------------------------------------Interface Vsan Admin Admin Status SFP Oper Oper Port

Mode Trunk Mode Speed ChannelMode (Gbps)

-----------------------------------------------------------------------------------------------------------------fc2/1 1 auto on trunking swl TE 4 --fc2/2 1 auto on sfpAbsent -- -- -------------------------------------------------------------------------------------------------------------------Interface Status IP Address Speed MTU Port

Channel-----------------------------------------------------------------------------------------------------------------Ethernet1/27 sfpIsAbsen -- -- 1500 --Ethernet1/28 notConnect -- 10000 1500 --Ethernet1/34 up -- 10000 1500 -------------------------------------------------------------------------------------------------------------------Interface Status IP Address Speed MTU Port

Channel-----------------------------------------------------------------------------------------------------------------vethernet17/1 init -- -- 1500 --vethernet19/1 holdDown -- -- 1500 --vethernet34/1 up -- 10000 1500 -------------------------------------------------------------------------------------------------------------------Interface Vsan Admin Admin Status SFP Oper Oper Port

Mode Trunk Mode Speed ChannelMode (Gbps)

-----------------------------------------------------------------------------------------------------------------vfc17/1 1 F -- down -- -- --vfc18/1 1 F -- Init -- -- --vfc34/1 1 F -- up -- F auto --

SWITCH CONFIGURATION EXAMPLE





Target Configuration

TARGET CONFIGURATION





Data ONTAP as an FCoE Target

Data ONTAP 7.3.1 and later has support for FCoE – Data ONTAP 7.3.1 supported the following

configuration:

NetApp FAS/V-Series with QLogic Target Card(X1138A-R6)

Cisco Nexus 5020VMware ESX 3.5U3, RHEL 5.3, Windows 2003/2008; QLogic, Emulex CNA

DATA ONTAP AS AN FCOE TARGET Here are the 3 key project deliverables:

1. The host initiators are being qualified by the SAN Engineering/QA teams. These are 3rd party HBA's referred too as CNAs (Converged Network Adapters). The 2 initial vendors are Qlogic (partno QLE8042) and Emulex (partno 21000 and 21002). Both CNAs are 2 port and both support a maximum of 4G FC bandwidth (next generation cards will support a maximum of 10G FC bandwidth). Both CNA's autonegotiate to 10GbE. Both CNA's use SFP+ pluggable modules. Both support copper and optical SFP+ pluggables. The NetApp qualifications are not tied to any new Host Utilities or software release. The IMT will be updated with a new FCoE protocol once the qualifications complete. The initial OS's to support FCoE are Windows, ESX and Linux.

2. The Cisco Nexus 5020 switch. This switch supports 40 10GbE ports capable of FCoE, and through an optional plug in component, 8 native FC ports. The NPI page for this switch is located HERE. The 5010 is planned for early 2009.

3. NetApp FCoE target expansion card (internal code name is Mercury). This new adapter is tied to ONTAP 7.3.1. The internal partno is X1138A-R6. This is essentially the same Qlogic CNA initiator card. Sales and support for this target HBA is purely via PVR. This is a gen1 card from Qlogic so there are future cards on the horizon to reduce cost and power consumption.





First Generation CNA functional operation

PCIe

FCoEEngine

Existing hostNIC driver

Existing host/target FC driver

All MAC, FCoE and Ethernet magic is handled here –making it transparent to the existing host FC S/W.

Regular Ethernet traffic flows through to NIC

10Gb Ethernet port

FCNIC

FIRST GENERATION CNA FUNCTIONAL OPERATION





Data ONTAP as an FCoE Target - Phase 2

NetApp introduces Phase 2 of the FCoE Support in Data ONTAP 7.3.2– Data ONTAP 7.3.2 supported the following

target configuration:

QLogic QLE8152 Dual Port 10-GbE Ethernet to PCIe CNA(X1109A-R6)

Brocade® BR 1020 Dual Port10 Gbps FCoE/CEE CNA(X1113A-R6)

DATA ONTAP AS AN FCOE TARGET - PHASE 2





2nd Generation CNA functional operation

PCIe

FCoEEngine

Existing hostNIC driver

Existing host/target FC driver

All MAC, FCoE and Ethernet magic is handled here –making it transparent to the existing host FC S/W.

Regular Ethernet traffic flows through to NIC

10Gb Ethernet port

NEW Integrated Component –containing all 3 functions

FCNIC

2ND GENERATION CNA FUNCTIONAL OPERATION





Configuration Examples – Target

system> fcp show adaptersSlot: 4aDescription: Fibre Channel Target Adapter 4a (Dual-channel, QLogic 2432 (8432) rev. 3)Adapter Type: LocalStatus: ONLINEFC Nodename: 50:0a:09:80:87:d9:2b:7d (500a098087d92b7d)FC Portname: 50:0a:09:83:87:d9:2b:7d (500a098387d92b7d)Standby: No

Slot: 1aDescription: Fibre Channel Target Adapter 1a (Dual-channel, QLogic 2532 (2562) rev. 2)Adapter Type: LocalStatus: ONLINEFC Nodename: 50:0a:09:80:87:d9:2b:7d (500a098087d92b7d)FC Portname: 50:0a:09:85:87:d9:2b:7d (500a098587d92b7d)Standby: No

CONFIGURATION EXAMPLES – TARGET





Configuration Examples - Target (Cont.)

system> fcp show adapters -v 4aSlot: 4aDescription: Fibre Channel Target Adapter 4a (Dual-channel, QLogic 2432 (8432) rev. 3)Status: OFFLINED BY USER/SYSTEMHost Port Address: 0x000000Firmware Rev: 4.5.2PCI Bus Width: 64-bitPCI Clock Speed: 33 MHzFC Nodename: 50:0a:09:80:87:d9:2b:7d (500a098087d92b7d)FC Portname: 50:0a:09:83:87:d9:2b:7d (500a098387d92b7d)Cacheline Size: 16FC Packet Size: 2048SRAM Parity: YesExternal GBIC: NoData Link Rate: 0 GBitAdapter Type: LocalFabric Established: NoConnection Established: LoopMediatype: autoPartner Adapter: NoneStandby: NoTarget Port ID: 0x3

CONFIGURATION EXAMPLES - TARGET (CONT.)





Initiator Configuration

INITIATOR CONFIGURATION





Windows Initiator

Windows– Versions

2003 x32 and x64 2008 x32 and x64, including HyperV

– DriversBoth QLogic and Emulex FCoE CNA drivers are available for download at the vendors websiteNOTE: FCoE CNAs can not be installed in the same box as FC HBAs.

– Host UtilitiesWHU 5.0 supports the FCoE CNAs Correct SNIA API is included with the driversBoth FCoE cards use the same HBA timeout setting as their FC HBA counterpart.

WINDOWS INITIATOR QLogic and Emulex 4Gb HBA’s represent over $10M in revenue FY2008 50 – 50 split between QLogic and Emulex





Configuration Examples - Host

CONFIGURATION EXAMPLES - HOST





VMware Initiator SupportVMware– Versions

ESX 3.5u2 ESXi 3.5u2ESX 3.5u3ESXi 3.5u3

– DriversESX 3.5 Update 2 - Both QLogic and Emulex FCoE CNA drivers are available for download at http://www.vmware.comESX 3.5 Update 3 –Should be inbox.NOTE: Just like with Windows, FCoE CNAs can not be installed in the same box as FC HBAs.

– Host UtilitiesEHU 5.0 supports the FCoE CNAs QLogic SNIA API still needs updating. (the EHU 5.0 ships v4.00build12) See BURT#312983 for more information.Correct Emulex SNIA API is included in EHU 5.0Both FCoE cards use the same HBA timeout setting as their FC HBA counterpart. (This info will be in EHU 5.0 TOI)

VMWARE INITIATOR SUPPORT QLogic and Emulex 4Gb HBA’s represent over $10M in revenue FY2008 50 – 50 split between QLogic and Emulex





Linux Initiator Support

Linux– Versions

RHEL 5u2 x32 and x64 – Drivers

Both QLogic and Emulex FCoE CNA drivers are available for download at the vendors websiteNOTE: FCoE CNAs cannot be installed in the same box as FC HBAs.

– Host UtilitiesHU 4.1.2 Both FCoE cards use the same HBA timeout setting as their FC HBA counterpart.

LINUX INITIATOR SUPPORT QLogic and Emulex 4Gb HBA’s represent over $10M in revenue FY2008 50 – 50 split between QLogic and Emulex





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Distinguish the differences between Fibre Channel (FC), Fibre Channel over Ethernet (FCoE) and Internet SCSI (iSCSI) protocolsDescribe path implementation with FCoEconnectivityDescribe how to configure FCoE ports on Windows and NetApp systems

MODULE SUMMARY


A3-1 SAN Implementation Workshop: Internet Storage Name Service


Internet Storage Name Service


INTERNET STORAGE NAME SERVICE





Module Objectives

By the end of this module, you should be able to:Describe Internet Storage Name Service (iSNS) on Windows Server 2008 R2Configure a storage system to use an iSNS

MODULE OBJECTIVES





Internet Storage Name Service Overview

INTERNET STORAGE NAME SERVICE OVERVIEW





Internet Storage Name Service (iSNS)

iSNS provides similar services as found in Fibre Channel networks iSNS Server– Repository of iSCSI nodes

InitiatorsTargetsManagement

– Dynamic repository = ping nodes to determine whether still present on fabric

Windows Server 2008 R2– Integrates iSNS with Active Directory

INTERNET STORAGE NAME SERVICE (ISNS)





Discovery Domains

iSNS organizes initiators and targets into Discovery Domains (DD)– DD are like zones in Fibre Channel fabrics– DD partition resources in your SAN

Default DD DD-W2KR2

DISCOVERY DOMAINS





Discovery Domains can then be added to Discovery Domains Set (DDS)– DDS group DD(s) – DDS are then enabled or disabled (activating or

inactivating the DD(s) within the DDS)

Discovery Domains Sets

DD-W2KR2DD-W2KR2DD-W2KR2

DDS-W2KR2

DISCOVERY DOMAINS SETS





Windows Server 2008 R2’s iSNS

WINDOWS SERVER 2008 R2’S ISNS





Install iSNS Feature

Add the iSNSFeature and

complete the install

INSTALL ISNS FEATURE





iSNS Interfaces

Found under Start menu

But you can also useStorage Explorer

ISNS INTERFACES





Create a Discovery Domain

CREATE A DISCOVERY DOMAIN





Discovery Domain

The new discovery domain

DISCOVERY DOMAIN





Create a Discovery Domain Set

The new initiator in the discovery domain

CREATE A DISCOVERY DOMAIN SET





Adding DD to a DDS

The new discovery domain set is disabled

Drag the discovery domain to

the discoverydomain set

ADDING DD TO A DDS





Adding DD to a DDS (Cont.)

Discovery Domain added to Discovery Domain Set

Enable to Discovery

Domain Set byclicking here

ADDING DD TO A DDS (CONT.)





Configuration Complete

Disabled thedefault

Discovery Domain Set

Node register with default Discovery Domain if present

CONFIGURATION COMPLETE





iSNS on Data ONTAP

Data ONTAP has supported Data ONTAP since Data ONTAP 6.4Data ONTAP 7.3.1.1 supports:– iSNS communicating over IPv4 and IPv6To configure iSNS:– Configure the iSNS service– Start the iSNS service– Verify iSNS service registered successfully with

the iSNS server

ISNS ON DATA ONTAP





Configuring iSNS Service

Configure the iSNS Servicesystem> iscsi isns config 10.254.132.63

Start the iSNS Servicesystem> iscsi isns start

Verify the iSNS Servicesystem> iscsi isns show

iSNS Entity id: NOT CONFIGURED

iSNS Server ip-addr: 10.254.132.63

iSNS Status: Enabled

system> iscsi isns show

iSNS Entity id: isns:00000003

iSNS Server ip-addr: 10.254.132.63

iSNS Status: Enabled

IP address of iSNS server (W2k8 R2)

Still hasn’t registered...

wait for a moment

CONFIGURING ISNS SERVICE





Configuration Complete

The storage system

successfullyregistered with

the iSNSserver

If node registered with default Discovery Domain, drag and drop it into your customized DD

CONFIGURATION COMPLETE






iSCSI Initiator Properties Dialog - Discovery

Add iSNSserver to poll







iSCSI Initiator Properties Dialog - Discovered







Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Describe Internet Storage Name Service (iSNS) on Windows Server 2008 R2Configure a storage system to use an iSNS

MODULE SUMMARY


A4-1 SAN Implementation Workshop: Multiple Connection Sessions


Multiple Connection Sessions


MULTIPLE CONNECTION SESSIONS





Module Objectives

By the end of this module, you should be able to:Describe multiple connection sessions (MCS) in Windows Server 2008 R2

MODULE OBJECTIVES





Connections versus Sessions

Four Sessions ( 4 TPG) with One Connection (1 interface) each

CONN 68/3

CONN 68/4

CONN 68/2

CONN 68/1SESSION 68 One Session (1 TPG) with

Four Connections (4 interface)





Microsoft’s Multiple Connections per Session (MCS)

Microsoft’s Multipath I/O (MPIO

CONNECTIONS VERSUS SESSIONS





MCS

MCS





iSCSI MCS on Windows

To enable iSCSI MCS on Windows:1. Install and enable NICs on target2. Verify IP Connectivity3. Create a new TPG on the storage system and

add necessary interfaces 4. Set the Microsoft software initiator to discover

both interfaces with the new TPG5. Connect to the TPG6. Create more than one connection with the

Microsoft software initiator7. Verify multiple paths

ISCSI MCS ON WINDOWS





Create a New Target Portal Group

To create a TPGsystem> iscsi tpgroup create mytp

To assign an interface to a TPGsystem> iscsi tpgroup add -f mytp e0b e0c

Verify TPGsystem> iscsi tpgroup showTPGTag Name Member Interfaces

1 mytp e0b, e0c1000 e0a_default e0a1001 e0b_default (none)1002 e0c_default (none)1003 e0d_default e0d

Force it because the interfaces already belong to another TPG

Place more than one interface into the TPG to allow failure of a connections without

disrupting the iSCSI session

CREATE A NEW TARGET PORTAL GROUP





Discovery

Discovery the storage system

DISCOVERY





Discovery (Cont.)

Discovery the storage system

DISCOVERY (CONT.)





Connect to the Target Portal Group

Log on to the TPG

Choose one of the portals(i.e. interfaces) that youwant to be the first connection

CONNECT TO THE TARGET PORTAL GROUP





Configuring MCS

One Session

Error occurs if Log on pressed twice

One Connection

CONFIGURING MCS





Configuring MCS (Cont.)

Verify current sessionssystem> iscsi session showSession 58Initiator InformationInitiator Name: iqn.1991-05.com.microsoft:windowsmachine

ISID: 40:00:01:37:00:00

Verify current connectionssystem> iscsi connection show -vNo new connections

Session connectionsConnection 58/1:State: Full_Feature_PhaseRemote Endpoint: 10.254.132.63:50055Local Endpoint: 10.254.144.75:3260Local Interface: e0b

Only one session

Only one connection

CONFIGURING MCS (CONT.)






Click Connections

One Connection







Set the load balance policy and click Apply







Add another connection Select different source IPand target portal






Verify Paths

New connection shows up

Two connections

One Session

VERIFY PATHS





Verify Paths (Cont.)

Verify current sessionssystem> iscsi session showSession 58Initiator InformationInitiator Name: iqn.1991-05.com.microsoft:windowsmachine

ISID: 40:00:01:37:00:00

Verify current connectionssystem> iscsi connection show -v...Session connectionsConnection 58/1:State: Full_Feature_PhaseRemote Endpoint: 10.254.132.63:50055Local Endpoint: 10.254.144.75:3260Local Interface: e0b

Connection 58/2:State: Full_Feature_PhaseRemote Endpoint: 10.254.132.64:50061Local Endpoint: 10.254.144.81:3260Local Interface: e0c

Only one session

Two connections

VERIFY PATHS (CONT.)





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Describe multiple connection sessions (MCS) in Windows Server 2008 R2

MODULE SUMMARY


A5-1 SAN Implementation Workshop: Storage Manager for SANs


Storage Manager for SANs







Module Objectives

By the end of this module, you should be able to:Explore the Windows Server 2008 Server R2’s Storage Manager for SANs tool to create a LUN

MODULE OBJECTIVES





Storage Manager for SANs






Storage Manager for SAN

Windows Server 2003 R2 introduced a new tool for SAN administrators to manager LUN on enterprise storage systemsWindows Server 2008 R2 also has the Storage Manager for SAN (SMfS) tool available to SAN administratorsTo use SMfS:– Install the SMfS feature– Install Data ONTAP VDS Hardware Provider– Configure LUNs with SMfS

STORAGE MANAGER FOR SAN





Install iSNS Feature

Add the SMfSFeature and complete the install

INSTALL ISNS FEATURE





Install VDS Hardware Provider

NetApp provides at the NOW site a VDS Hardware Provider that communicates with NetApp storage systems– Works with FCP configuration using Data

ONTAP 7.03 and later– Works with iSCSI configuration using Data

ONTAP 7.1 and later– Reboot after install

INSTALL VDS HARDWARE PROVIDER





Configure VDS Hardware Provider

Launch the VDS Hardware Provider from Control Panel

Add the storage system to the provider

CONFIGURE VDS HARDWARE PROVIDER





Storage Manager for SAN

STORAGE MANAGER FOR SAN





Manage Server Connections

MANAGE SERVER CONNECTIONS





Create a LUN

CREATE A LUN





Create a LUN (Cont.)

CREATE A LUN (CONT.)
























Wizard creates:1. Volume2. A LUN with new

igroup(s)






Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Explore the Windows Server 2008 Server R2’s Storage Manager for SANs tool to create a LUN

MODULE SUMMARY


A6-1 SAN Implementation Workshop: Hyper-V Introduction


Hyper-V Introduction


HYPER-V INTRODUCTION





Module Objectives

Bythe end of this module, you should be able to:Introduce Microsoft’s Hyper-VImplement Hyper-V on Microsoft Windows Server 2008 R2Configure a LUN to Hyper-V to host VMs

MODULE OBJECTIVES





Hyper-V

HYPER-V





What is Hyper-V?

Hyper-V (previously code-named Viridian) is Microsoft's new server virtualization technology– Hypervisor Model

Does not run on top of a host OS– Loads at boot time

Creates a layer of virtualization between the server hardware and the operating systems it hosts

– Optional feature of Windows Server 2008Will be available also as a stand-alone product called Microsoft Hyper-V Server

– Free FeatureIncluded with Windows Server 2008 license

WHAT IS HYPER-V?





Windows Server 2008 R2 Role

Selected

WINDOWS SERVER 2008 R2 ROLE





Windows Server 2008 R2 Role (Cont.)

WINDOWS SERVER 2008 R2 ROLE (CONT.)






Selected interfaces tobe used with VMs


















Hyper-V Administration

HYPER-V ADMINISTRATION





How is Hyper-V Different from VMware ESX

Integrated Hypervisor– Drivers run within

hypervisor– Requires ESX virtual

device drivers

Micro-kernel Hypervisor– Drivers run within guest

and parent OS– Requires VSP (modified)

parent OS drivers– Requires VSC (modified)

guest OS drivers

Physical Hardware

Microsoft Hypervisor

(Parent OS)

VM 1(Child OS)

VM 2(Child OS)

Virtualization Stack

DriversDriversDrivers DriversDriversDrivers DriversDriversDrivers

Microsoft Hyper-V

Physical Hardware

VMware Hypervisor

VMware(Console OS)

VM 1(Guest OS)

VM 2(Guest OS)

DriversDriversDrivers

VMware ESX Server

HOW IS HYPER-V DIFFERENT FROM VMWARE ESX





Shared Devices

Microsoft Hyper-V

Requires special drivers– Installed as part of integration

components

VMware ESX Server

Does not require specialized drivers in the Guest OS

Parent OS Guest VMs

How the I/Os are Processed

Hypervisor

VM0

I/O Service

Device Drivers Guest OS & Apps

VMn

Guest OS & Apps

Shared Devices

Console OS(Optional) Guest VMs

Hypervisor

VMn

Guest OS & Apps

VM0

Guest OS & Apps

I/O Service

Device Drivers

HOW THE I/OS ARE PROCESSED





Provisioning Storage for Hyper-V

Create a VM using NetApp storage

Create a new VM

ClickNext

PROVISIONING STORAGE FOR HYPER-V





Provisioning Storage for Hyper-V (Cont.)

Create a VM using NetApp storage (Cont.)

Place the VM’s configurationfile on a NetAppLUN for betterprotection

NetApp LUN that is formatted and mounted

PROVISIONING STORAGE FOR HYPER-V (CONT.)














Create VirtualHard Disk (VHD) file on NetApp LUN













Provisioning Storage for Hyper-V


New VM using a VHD on a NetApp LUN

PROVISIONING STORAGE FOR HYPER-V





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Introduce Microsoft’s Hyper-VImplement Hyper-V on Microsoft Windows Server 2008 R2Configure a LUN to Hyper-V to host VMs

MODULE SUMMARY


A7-1 SAN Implementation Workshop: SAN Troubleshooting on Windows


SAN Troubleshooting on Windows


SAN TROUBLESHOOTING ON WINDOWS





Module Objectives

By the end of this module, you should be able to:Review diagnostic tools and techniques available for WindowsDefine and configure queue depths on Windows

MODULE OBJECTIVES





Windows SAN Troubleshooting

WINDOWS SAN TROUBLESHOOTING





Windows Troubleshooting

To troubleshoot a SAN environment in Windows:View Fibre Channel statisticsVerify partition within WindowsUse the scripts provided in the Host Utilities Kit (HUK)SnapDrive collection utility to capture diagnostic information (if available)

WINDOWS TROUBLESHOOTING





View FC Statistics

To verify FC statistics in Windows Server 2008:

Select the HBA, right-click and select Properties

Select toview statistics

VIEW FC STATISTICS





View FC Statistics (Cont.)

To verify FC statistics in Windows Server 2008:

Port statistics

VIEW FC STATISTICS (CONT.)





View FC Statistics (Cont.)

To verify FC statistics in Windows Server 2003:Emulex exampleQLogic example

VIEW FC STATISTICS (CONT.)





Verify Partition Configuration

To verify partition configuration from Windows:C:\> wmic partition get BlockSize, StartingOffset, Name, Index, Type

BlockSize Index Name StartingOffset Type

512 0 Disk #1, Partition #0 33571840 GPT: Basic Data

512 0 Disk #2, Partition #0 65536 Installable File System

512 0 Disk #0, Partition #0 32256 Unknown

512 1 Disk #0, Partition #1 32901120 Installable File System

VERIFY PARTITION CONFIGURATION





Host Utilities Kit Scripts

Windows Host Utilities Kit provides the following scripts:– brocade_info.exe - collects information about Brocade switches

installed– cisco_info.exe - collects information about Cisco switches installed – fcconfig.exe - used by the installation program to set HBA timeout

values– fcconfig.ini - used by the fcconfig.exe program– filer_info.exe - collects information about the storage system– hba_info.exe - collects information about the host bus adapters

(HBAs) installed on the host machine– mcdata_info.exe - collects information about McDATA switches

installed– qlogic_info.exe - collects information about QLogic switches installed– san_version.exe - displays the version of Host Utilities– windows_info.exe - collects configuration information about your OS

HOST UTILITIES KIT SCRIPTS





Windows SnapDrive Data Collection Utility

What is SnapDrvDc?– SnapDrvDc.exe is a Windows utility that gathers

Windows host and storage system information– Downloadable at the NOW site– Data is helpful for troubleshooting– rsh must be enabled to use SnapDrvDc.exe

WINDOWS SNAPDRIVE DATA COLLECTION UTILITY





Click hereto start data collection

Add controller’s DNSname here, and click Add

Tools - SnapDrvDc Utility

Information is gathered for the host and all storage system you have added

1

2

3

TOOLS - SNAPDRVDC UTILITY





Sample of Files Collected

The files are placed in a directory named NetApp_SNAPDRV_DC_XX-XX-XXThe folder will be created wherever you place the SnapDrvDc.exe file

SAMPLE OF FILES COLLECTED





View the Log

The following is a sample of the data collection successes and failures:SnapDrive Data collection tool file version:2.0.0.336

=======================================================

Successfully dumped the application eventlog

Successfully dumped the system eventlog

Successfully opened registry key = SYSTEM\CurrentControlSet\Services\Eventlog\Application\SnapDrive

Successfully read registry key information = EventMessageFile

Successfully opened registry key = SYSTEM\CurrentControlSet\Services\SWSvc

Successfully read registry key information = DependOnGroup

Successfully opened registry key = SYSTEM\CurrentControlSet\Services\SWSvc

...

VIEW THE LOG





Host Bus Adapters

HOST BUS ADAPTERS





HBA API

Application Programming Interface for management of Fibre Channel Host Bus AdaptersControlled by SNIA (Storage Networking Industry Association)Platform independentVendor independentInteroperable

HBA API The HBA API is a Storage Networking Industry Association (SNIA) specification that provides a vendor neutral interface for managing aspects of HBAs. SNIA has since turned over the specification to T11.

For more information see: http://www.snia.org/tech_activities/hba_api/ and ftp://ftp.t11.org/t11/docs/02-149v0.pdf.





Command Queues – Device to HBA

Host HBA Queue

Disk 1:

Device Queues

Disk 2

Disk 3

Disk N

Command Queue – the number of outstanding commands allowed per LUN and per targetDevices: Command queue limit imposed by OS and sometimes initiatorInitiators: Limit the command queue per targetTarget: Command queue for all initiators and LUNs

COMMAND QUEUES – DEVICE TO HBA Initiators are configured to limit the number of commands it sends to a target and LUN. Limiting the number of outstanding command is also called a throttle. The optimal throttle is value is one that does not allow the host to over utilize target queues.





Queues – Transmission Example

Queue Depth = 1

Queue Depth = 10

QUEUES – TRANSMISSION EXAMPLE Little’s Law describes the relationship between throughput and latency. It measures throughput as work in progress divided by response time. This has implications for queues, the number of outstanding commands (per target or per LUN). As response time increases, the queue depth must become larger in order to get the same number of I/Os. To translate Little’s Law to storage I/O terminology, the throughput is recognized as the number of I/O’s completed per measurement period (e.g., second). The maximum needed queue depth then, is equal to the number of I/Os per second (throughput) multiplied the response time.

The example at the top shows a queue depth of 1. The example at the bottom demonstrates a queue depth of 10.

The response time (latency) is the same regardless of the queue depth. Although the throughput is much better when more I/Os are in flight simultaneously. Maximum performance is achieved when throughput is greatest.





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Review diagnostic tools and techniques available for WindowsDefine and configure queue depths on Windows

MODULE SUMMARY


A8-1 SAN Implementation Workshop: NFS Datastores


NFS Datastores


NFS DATASTORES





Module Objectives

By the end of this module, you should be able to:Connect a NetApp volume as a vSphere’s NFS Datastores

MODULE OBJECTIVES





NFS Datastores

NFS DATASTORES






ESX Cluster

NAS Datastore

Datastore

NFS

NIC

LAN

Flexible Volume

1.vmx1.vmdk

2.vmx2.vmdk

3.vmx3.vmdk

4.vmx4.vmdk

VM1VDisk0

VM2VDisk0

VM3VDisk0

VM4VDisk0

NIC

NAS DATASTORE





Data ONTAP NFS Configuration

License NFSsystem2> license add xxxxxxx

Create volume for an existing aggregatesystem2> vol create nfs_store aggr1 20g

Turn access updates offsystem2> vol optons nfs_store

no_atime_update on

Ensuring volumes security style to UNIXsystem2> qtree security /vol/nfs_store unix

DATA ONTAP NFS CONFIGURATION





Data ONTAP NFS Configuration (Cont.)

Verify exports:system2> exportfs /vol/nfs_store -sec=sys,rw,nosuid

Verify connectivity:system2> ifconfig e0a

e0a: flags=948043 <UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500 inet 10.254.144.91 netmask 0xfffffc00 broadcast 10.254.147.255 ether 00:a0:98:09:f2:42 (auto-1000t-fd-up) flowcontrol full

New volumes automatically get exported (default):system2> options nfs.export.auto-updatenfs.export.auto-update on

DATA ONTAP NFS CONFIGURATION (CONT.)





Data ONTAP NFS Configuration (Cont.)

Alternatively, you can use System Manager to create a NFS datastore:

DATA ONTAP NFS CONFIGURATION (CONT.)






Configure Virtual Infrastructure with standard NICs :1. Configure the virtual infrastructure

– Identify the local network interface(s) to use– Configure switch– Configure VMkernel(s)– Configure jumbo frames if desired

2. Verify NFS connection to the storage system






vSphere NFS Configuration

Verify number of NFS datastores

Verify Max of 64

VSPHERE NFS CONFIGURATION





vSphere NFS Configuration (Cont.)

Verify TCP/IP Heap Size settings:

Verify 30

Verify 120

VSPHERE NFS CONFIGURATION (CONT.)






Verify NFS Heatbeat settings:

Verify 12

Verify 10







Create the datastore

Select NetworkFile System







Pathto export







NFSDatastore

Now, Create a VM in the datastore






Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Connect a NetApp volume as a vSphere’s NFS Datastores

MODULE SUMMARY


A9-1 SAN Implementation Workshop: Server Consolidation


Server Consolidation


SERVER CONSOLIDATION





Module Objectives

By the end of this module, you should be able to:Migrate servers from unreliable storage to reliable NetApp storage

MODULE OBJECTIVES





Serious Datacenter Problem

You can setup new servers using SAN booting techniques to reliable NetApp storageHowever, many pre-existing servers are out there in most existing datacenters running on unreliable storage

With NetApp and VMware, you can change this

SERIOUS DATACENTER PROBLEM





vCenter

VCENTER





vCenter

vCenter provides a enterprise management interface for vSpheres in your data center

VCENTER





vCenter Features

vCenter provides additional ways to report storage with the Storage Views

VCENTER FEATURES





vCenter Features (Cont.)

vCenter provides additional ways to report storage with the Storage Views (Cont.)

VCENTER FEATURES (CONT.)





vCenter Plugins

Additional features are available through vCenter’s plugins

VCENTER PLUGINS





Guided Consolidation

GUIDED CONSOLIDATION





vCenter’s Guided Consolidation

Guided Consolidation is a plugin to vCenterthat allow small enterprise to find servers by domain, IP, host name or a range of IPs and convert them from physical-to-virtual (P2V)

VCENTER’S GUIDED CONSOLIDATION





Install vCenter Converter

After install, make the plug-in available to vCenter

INSTALL VCENTER CONVERTER





Verify Configuration

Everything is installedand running properly

VERIFY CONFIGURATION





Analysis Phase Start

ANALYSIS PHASE START





Analysis Phase

Analysis phasetakes at least 24 hours

to gather information aboutservers you plan to consolidate

ANALYSIS PHASE





Analysis Phase Complete

Analysis phaseis complete;

ready for consolidationto NetApp storage

Or right-click and manually consolidate

the server

ANALYSIS PHASE COMPLETE





Plan Consolidation Wizard

This only showsone server, but

the wizard supportmany servers

Wizard automaticallychooses best

NetApp datastoreor use manual method

to choose

PLAN CONSOLIDATION WIZARD





Consolidation Phase

Consolidationin progress

CONSOLIDATION PHASE





Consolidation Complete

Your important servers now

running on NetAppreliable storage

CONSOLIDATION COMPLETE





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Migrate servers from unreliable storage to reliable NetApp storage

MODULE SUMMARY


A10-1 SAN Implementation Workshop: NPIV Troubleshooting


NPIV Troubleshooting


NPIV TROUBLESHOOTING





Module Objectives

By the end of this module, you should be able to:Convert a VMFS .vmdk to a RDM .vmdkDescribe the VPORT creation flow and troubleshoot NPIV

MODULE OBJECTIVES





Converting a VMFS Disk Image to RDM

CONVERTING A VMFS DISK IMAGE TO RDM





VMFS Disk Image to RDM

NPIV requires RDM-based VMsIf you have existing VMFS-based disk images, you can convert the VMFS image to an RDM:

# vmkfstools -i <from_disk> <to_disk> -d <rdm:|rdmp:> device

– <from_disk> = Name of existing VMFS .vmdk– <to_disk> = Name of new RDM .vmdk– <rdm:|rdmp:> = Disk type to map via VMFS– <device> = Raw device name of SAN-disk disk

Example:# vmkfstools -i

/vmfs/volumes/storage1/rhel5/rhel5.vmdk /vmfs/volumes/storage1/rhel5-rdm.vmdk -d rdm:/vmfs/devices/disks/vmhba4:0:0:0

VMFS DISK IMAGE TO RDM





NPIV Troubleshooting

NPIV TROUBLESHOOTING





Setting NPIV Settings

Later it will have

the WWNN(s) and WWPN(s)

Verify generated new WWNs

SETTING NPIV SETTINGS





LUN Masking Verification

Verify WWPN(s) added to igroupssystem2> igroup show iESX_fcp

iESX_fcp (FCP) (ostype: vmware):

10:00:00:00:c9:58:29:9a (logged in on: vtic, 0d, 0c)

WWPN Alias(es): ESX1-FC

10:00:00:00:c9:58:29:9b (logged in on: vtic, 0c, 0d)

WWPN Alias(es): ESX2-FC

27:ee:00:0c:29:00:05:a7 (logged in on: 0c, vtic, 0d)

27:ee:00:0c:29:00:06:a7 (logged in on: 0d, vtic, 0c)

VPORT WWPNs

LUN MASKING VERIFICATION





VPORT Creation Flow

NPIV option

on?

Fabric PLOGI

VM off

VMkernel requestVPORT creation

Driver sends FDISC to fabric

SCSI targetreports LUNs

Operationunsuccessful

RDM mounted viaphysical HBA port

RDM mounted viaNPIV port

VM power on complete

Power on VM

NoHBA / firmware

doesn’t support NPIV

Switchdoesn’t support NPIV

VPORT CREATION FLOW





Verify Supported HBA(s)

From the Service Console:# cd /proc/scsi/lpfc820

# ls

4 5

# cat 4

Emulex LightPulse Fibre Channel SCSI driver 8.2.0.30.49vmw

NetApp 111-00308 4Gb 2-port PCI-X2 Fibre Channel Adapter on PCI bus 14 device 08 irq 145 port 0

BoardNum: 0

Firmware Version: 2.72A2 (B3F2.72A2)

Portname: 10:00:00:00:c9:58:29:9a

Nodename: 20:00:00:00:c9:58:29:9a

SLI Rev: 3

NPIV Supported: VPIs max 100 VPIs used 1

RPIs max 512 RPIs used 18

Example using Emulex

Port 0

Confirmed 1 VPORT created

Verify Supported Firmware

VERIFY SUPPORTED HBA(S)





Verify Supported HBA(s) (Cont.)

From the Service Console (Cont.):Vport List:

Vport DID 0x10801, vpi 1, state 0x20

Portname: 27:ee:00:0c:29:00:05:a7 Nodename: 27:ee:00:0c:29:00:04:a7

Link Up - Ready:

PortID 0x10800

Fabric

Current speed 4G

Physical Port Discovered Nodes: Count 4

t00 DID 010000 State 06 WWPN 50:0a:09:81:96:88:37:5d WWNN 50:0a:09:80:86:88:37:5d




ConfirmedVPORT usage

VERIFY SUPPORTED HBA(S) (CONT.)





Verify Brocade Support for NPIV

Verify NPIV-supported on the Brocade:switch> portcfgshowPorts of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15...NPIV capability ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON...

NPIV turned on

VERIFY BROCADE SUPPORT FOR NPIV





NPIV Observed

After boot up of NPIV-supported VM:switch> switchshow...Area Port Media Speed State Proto=====================================0 0 id N2 Online F-Port 50:0a:09:81:96:88:37:5d1 1 id N2 Online F-Port 50:0a:09:82:96:88:37:5d2 2 id N2 Online F-Port 50:0a:09:81:86:88:37:5d3 3 id N2 Online F-Port 50:0a:09:82:86:88:37:5d

...8 8 id N4 Online F-Port 2 NPIV public9 9 id N4 Online F-Port 2 NPIV public


27:ee:00:0c:29:00:05:a710:00:00:00:c9:58:29:9a

Two connections (NPIV)

NPIV’s WWPNs show up


27:ee:00:0c:29:00:06:a710:00:00:00:c9:58:29:9a

NPIV OBSERVED






After boot up of NPIV-supported VM (Cont.):switch> portloginshow 8Type PID World Wide Name credit df_sz cos=====================================================fe 010801 27:ee:00:0c:29:00:05:a7 16 2048 c scr=3fe 010800 10:00:00:00:c9:58:29:9a 16 2048 c scr=3ff 010801 27:ee:00:0c:29:00:05:a7 12 2048 c d_id=FFFFFCff 010800 10:00:00:00:c9:58:29:9a 12 2048 c d_id=FFFFFC

switch> portloginshow 9Type PID World Wide Name credit df_sz cos=====================================================fe 010901 27:ee:00:0c:29:00:06:a7 16 2048 c scr=3fe 010900 10:00:00:00:c9:58:29:9b 16 2048 c scr=3ff 010901 27:ee:00:0c:29:00:06:a7 12 2048 c d_id=FFFFFCff 010900 10:00:00:00:c9:58:29:9b 12 2048 c d_id=FFFFFC






Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Convert a VMFS .vmdk to a RDM .vmdkDescribe the VPORT creation flow and troubleshoot NPIV

MODULE SUMMARY


A11-1 SAN Implementation Workshop: VMware Snapshots and NetApp Snapshot Copies


VMware Snapshots and NetApp Snapshot Copies


VMWARE SNAPSHOTS AND NETAPP SNAPSHOT COPIES





Module Objectives

By the end of this module, you should be able to:Distinguish between VMware snapshots and NetApp Snapshot™ copies

MODULE OBJECTIVES





Snapshot Technologies

SNAPSHOT TECHNOLOGIES





VMware Snapshots

ESX provides snapshot functionality– Managed in vCenter– Only supported on VMDKs and Virtual RDMs– Limited to 32 snapshots– VMs can remain online while taking snapshots– VMDKs are locked and disk IO is written to log files– VM memory is also snapped and logged– More VMs and/or snapshots degrades performance

Up to 30% performance penalty– VMware recommends storage vendor snapshots for

performance and scalability– VMware recommends storage vendor snapshots for VM

backup and recoveryTypical use case– Upgrading and patching guest OS in VM– Enablement of Storage vMotion, vRDMs, and linked clones

VMWARE SNAPSHOTS Performance degradations are per VMware's documentation.





NetApp Snapshot Copies

NetApp Snapshot copies can be easily integrated– Maximum of 255 per volume– Supports all datastore types– VMs can remain online while taking a Snapshot– Operation is scripted (see TR3428)

Single script can coordinate Snapshot of entire datacenter– For crash consistency we call VMware snapshots to

prepare the VM for a NetApp Snapshot– No performance degradation– Required for consistent FlexClone®, LUN clone,

SnapMirror®, SnapVault® operations– Group data in FlexVol® volumes by Snapshot policies

Typical use case– Regular backup and recovery operations of VM virtual

disks

NETAPP SNAPSHOT COPIES The VMs need to be quiesced (set in hot backup mode) before NetApp Snapshot copies are taken off the storage that provisions the VMware datastores used by each VM. These operations can be scripted as shown in “TR3428 – Netapp and VMware ESX 3.0 Storage Best Practices”. This is very similar in principle to the way a database needs to be quiesced before its storage is snapped.

NetApp Snapshot copies can be taken off of storage provisioning NFS, RDM (physical and virtual), and VMFS datastores, that is all VMware datastore types.





NetApp Snapshot Caveats

VMDKs on VMFS– Snapshot is at FlexVol level– Entire VMFS is backed up and restored– VMDK restore is a copy out operation from Snapshot to

production datastoreVMDKs on NFS– Single File SnapRestore® for VMDK recoveryRDMs– SnapRestore or LUN clone for RDM recovery– Can be connected by any server (virtual or physical)

Easy single file recovery

NETAPP SNAPSHOT CAVEATS VMDKs on VMFs is LUN clone or Single File SnapRestore (LUN clone recommended)

VMDK on NFS is Single File SnapRestore or FlexClone (Single File SnapRestore preferred)

RDMs Single File SnapRestore, LUN clone, or FlexClone work

Single file recovery from Snapshot copies is easy with RDMs, with VMDKs they cannot be release so in practice recovery is through a proxy server.





SnapMirror and SnapVault

To ensure that VMs are crash consistent is critical– To ensure that a VM will not require a checkdisk during

boot and avoid possible data loss, take consistent Snapshot copies, then replicate them

Data layout is critical– Group datacenter datastores by Snapshot policy

Each datacenter should have its own volumeMultiple volumes per datacenter for multiple replication schedulesSeparate swap, page file, user and sys temp from OS datastores

RDM DR recovery require re-creating map files

SNAPMIRROR AND SNAPVAULT





VMware Clones

VM clones duplicate:– VM configuration file (.vmx file)– VM Virtual Disk(s) (.vmdk file)– In VirtualCenter RDMs clone to VMDKs by defaultStrengths– Can be completed within VirtualCenter– Is used to easily deploy new VMsAreas of concern– Consumes storage (no thin provisioning*)– Cloning takes a long time to completeTypical use case– Permanent VM deployment from template

VMWARE CLONES *Thin provisioning can be accomplished at the command line.

VMware clones are good for permanent VM deployment from a template.





NetApp Clones

Permanent VM Deployment (from template)– NFS through Single File SnapRestore

Clones internally versus over Ethernet from ESX– RDM through LUN cloneTemporary VM Deployment– For test, dev, training, demos, and so on– NFS through FlexClone– VMFS and RDM by way of LUN clone or FlexCloneTypical use case– Fast temporary deployment of VM for training, demos

NETAPP CLONES NetApp clones are good for fast temporary deployment of VMs for training, test, dev, demos, and so on. For VMs with either RDMs or NFS VMDKs, disconnect disk first and complete VM template.





VMware Thin Provisioning

VMware thin provisioning is available on VMDKs– Does not RDMs– NFS VMDKs are always thin provisioned– No way to thin provision within VirtualCenter

You must script– Thin-provisioned VMDKs

Have performance degradation– Trade-off capacity for performance

Can be converted to zero-thick VMDK

VMWARE THIN PROVISIONING VMDKs cannot be converted to thin-provisioned format.

Zero thin: space in the VMDK file is allocated on demand.

Zero thick: empty space is pre-allocated in the VMDK file.





NetApp Thin Provisioning

NetApp Thin Provisioning is available on VMFS, NFS, and RDMs– No performance degradation– VMFS

Reduce overhead when using multiple datastoresCan be combined with thin-provisioned VMDKs for maximum savings

– NFSReduces FlexVol size

– RDMsReduces FlexVol sizeReduce overhead of unused LUN capacity

NETAPP THIN PROVISIONING





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Distinguish between VMware snapshots and NetApp Snapshot™ copies

MODULE SUMMARY


A12-1 SAN Implementation Workshop: Virtual Storage Console


Virtual Storage Console


VIRTUAL STORAGE CONSOLE





Module Objectives

By the end of this module, you should be able to:Describe the NetApp Virtual Storage Console and how it assist in administrating virtual machines using NetApp storage

MODULE OBJECTIVES






VIRTUAL STORAGE CONSOLE






Virtual Storage Console (VSC)– Is a plug-in to VMware vCenter– Provides a Storage Management pane to show

discovered NetApp controllers and ESX hosts along with information related to each

– Provides a drop down menu per ESX host for setting NetApp recommended HBA/CNA Adapter Settings, MPIO Settings and/or NFS settings

VIRTUAL STORAGE CONSOLE Netapp Virtual Storage Console (VSC) is a vSphere NetApp plug-in to provide similar support as the ESX Host Utilities. VSC supports the following:

Provides a Storage Management pane to show discovered NetApp controllers and ESX hosts along with information related to each (for example, IP information, Data ONTAP version, status, capacity, usage, protocols, adapter and MPIO status settings).

Provides a drop down menu per ESX host for setting NetApp recommended HBA/CNA Adapter settings, MPIO settings and/or NFS settings.





Virtual Storage Console (Cont.)

Virtual Storage Console (VSC) – Provides a Storage Details pane to show

controller datastore info in addition to detailed Lun and volume info, deduplication settings and capacity readings (datastore, lun, volume).

– Provides a Data Collection pane to allow for Controller, ESX host and switch info script data collection. VSC also provides limited integration with nSANity off the toolchest. VSC 1.0 will not package nSANity.

– NFS storage details panel

VIRTUAL STORAGE CONSOLE (CONT.) VSC also supports the following:

Provides a Storage Details pane to show controller datastore information in addition to detailed LUN and volume info, deduplication settings and capacity readings for datastores, LUNs, and volumes.

Provides a Data Collection pane to allow for Controller, ESX host and switch info script data collection. VSC also provides limited integration with nSANity off the toolchest. VSC 1.0 will not package nSANity.

NFS storage details panel.





VSC 1.0 Requirements

VSC 1.0 – Supports ESX 4.0 and ESXi 4.0– Offers limited support for ESX 3.5 and ESXi 3.5– Requires vCenter Server 4.0– Supports Data ONTAP 7.3.1.1 and later for all

SAN and NAS functions

VSC 1.0 REQUIREMENTS





VSC: Integrated Storage ViewThe VSC tab

SAN and NASsupport Automatic

discovery

VSC: INTEGRATED STORAGE VIEW





VSC: Virtual Storage Infrastructure

ESX hostconfigurationStorage

systemconfiguration

VSC: VIRTUAL STORAGE INFRASTRUCTURE





VSC: Configuration Optimization

Optimizehost settings

VSC: CONFIGURATION OPTIMIZATION





VSC: Controller Status and Capacity

Deduplication savings

Status

VSC: CONTROLLER STATUS AND CAPACITY





VSC: Support Tools

Data collectionon support tools

VSC: SUPPORT TOOLS





VSC Takeaways

Integrated Virtual Host and Storage Management– See storage “back” and host “in” views of ESX

datastores (SAN and NAS)Storage Status and Capacity Information– Know how much your datastore is using + how

much is availableSimplified configuration for improved RAS– Provides automated tools for host settings and

configuration optimization

VSC TAKEAWAYS





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Describe the NetApp Virtual Storage Console and how it assist in administrating virtual machines using NetApp storage

MODULE SUMMARY


A13-1 SAN Implementation Workshop: SnapDrive for UNIX (Linux Version)


SnapDrive for UNIX(Linux version)


SNAPDRIVE FOR UNIX(LINUX VERSION)





Module Objectives

By the end of this module, you should be able to:Describe the steps to configure SnapDrive for UNIX on Linux

MODULE OBJECTIVES





SnapDrive for UNIX

SNAPDRIVE FOR UNIX





SnapDrive 4.1 for UNIX

Features:– Runs as a daemon service– Supports Red Hat DM-Multipath and Veritas

Dynamic Multi-Pathing– Intelligent provisioning and managing storage – Creates and restores Snapshot copies– Volume-based single file Snapshot restore– Integrates with SnapManager software through a

Web service– NOTE: SnapDrive does not support FC drivers and

OpeniSCSICommand-line interface:– # snapdrive ...

SNAPDRIVE 4.1 FOR UNIX





Installation of SnapDrive on Linux

Install the application# rpm -U -v netapp.snapdrive.linux_4_1_1.rpm

Preparing packages for installation...

netapp.snapdrive-4.1.1-1

Starting snapdrive daemon: Successfully started daemon

Explore the application# cd /opt/NetApp/snapdrive

# ls

bin docs snapdrive.conf snapdrived.rcscriptdiag SDU4_1_1_notice.txt snapdrived_cronsnapdrived_var.rcscript

# cd bin

# ls

snapdrive snapdrived

The daemonMain file

Configuration file

Daemon starts automatically

INSTALLATION OF SNAPDRIVE ON LINUX





Installation of SnapDrive on Linux (Cont.)

SnapDrive requires two support libraries:– sg3_utils-libs-1.25-4.el5.i386.rpm

# rpm -U -vh sg3_utils-libs-1.25-4.el5.i386.rpm

– sg3_utils-1.25-4.el5.i386.rpm# rpm -U -vh sg3_utils-1.25-4.el5.i386.rpm

INSTALLATION OF SNAPDRIVE ON LINUX (CONT.)





SnapDrive Daemon

To use SnapDrive 4.1 for Linux, start the daemon:# cd /opt/NetApp/snapdrive

# snapdrived start

Successfully started daemon

To stop the daemon:# snapdrived stop

Successfully stopped daemon

Status of the daemon:# snapdrived status

Snapdrive Daemon Version : 4.1.1 (Change 942392 Built Fri Jul 17 04:56:45 PDT 2009)

Snapdrive Daemon start time : Sun Sep 27 12:01:55 2009

Total Commands Executed : 5

Job Status: No command in execution

User must be logged in as root user to execute daemon commands

SNAPDRIVE DAEMON





Set Up SnapDrive Access

By default, SnapDrive 4.1 uses SSL– Configure the storage system for SSL

system> secureadmin setup ssl

Configure SSL with RHEL# openssl genrsa 1024 > host.key

# chmod 400 host.key

# openssl req -new -x509 -nodes -sha1 -days 365 -key host.key > host.cert...

# openssl x509 -noout -fingerprint -text < host.cert > host.info

# cat host.cert host.key > /opt/NetApp/snapdrive/snapdrive.pem

# rm host.key

# chmod 400 /opt/NetApp/snapdrive/snapdrive.pem

Follow the wizard

Follow the wizard

SET UP SNAPDRIVE ACCESS





Set Up SnapDrive Access (Cont.)

Verify snapdrive.conf values# cat snapdrive.conf

...

#use-https-to-sdu-daemon=on

#contact-https-port-sdu-daemon=4095

#sdu-daemon-certificate-path=/opt/NetApp/snapdrive/snapdrive.pem # location of https server certificate ...

Uncomment, if you are using a secure connection

SET UP SNAPDRIVE ACCESS (CONT.)






Verify snapdrive.conf values# cat snapdrive.conf

...

#PATH="/sbin:/usr/sbin:/bin:/usr/bin:/opt/NTAP/SANToolkit/bin:/opt/sanlun/bin" # toolset search path

...

Verify that OS can properly resolve host name – Requires both forward and reverse resolution– For example:

DNS# vi /etc/hosts

Verify the path to the Host Utilities Kit is correct







Verify current permissions to a storage system:# snapdrive config access list storage_system_1_IP

ALL ACCESS

Commands allowed:

snap createsnap restoresnap deletesnap renamestorage createstorage resizesnap connectstorage connectstorage deletesnap disconnectstorage disconnect







Set the user name and password for access:# snapdrive config set root storage_system_1_IP

Password:

Verify Password:

Using root user for educationalpurposes only






Creating a LUN with SnapDrive

To create a LUN# snapdrive storage create -lun system:/vol/SANvol/lun-lunsize 100m

To create a LUN, provide LUN access, add a file system, and mount the virtual disk# snapdrive storage create -fs /mnt/lun -lunsystem:/vol/SANvol/lun -lunsize 100m -nolvm -fstype ext3 -igroup my_ig

# cd /mnt/lun

# touch foo

# ls

foo lost+found

igroupcreated earlier

CREATING A LUN WITH SNAPDRIVE





LUN Management

Configure Linux environment variable# export LVM_SUPPRESS_FD_WARNINGS=1

# snapdrived stop

# snapdrived start

List LUNs available# snapdrive storage list -all

Host devices and file systems:

raw device:/dev/sdj1 mount point:/mnt/lun(persistent) fstype ext3

device ... size proto state clone lun path backing snapshot

------ ... ---- ----- ----- ----- -------- ----------------

/dev/sdj 100m fcp online No system:/vol/SANvol/lun -

Restart the SnapDrivedaemon

LUN MANAGEMENT





Snapshot Creation with SnapDrive

To create a Snapshot of a mounted LUN:# snapdrive snap create -fs /mnt/lun -snapname my_snap

Starting snap create /mnt/lun

WARNING: DO NOT CONTROL-C!

If snap create is interrupted, incomplete snapdrivegenerated data may remain on the filer volume(s)which may interfere with other snap operations.

Successfully created snapshot my_snap on system:/vol/SANvol

snapshot my_snap contains:

file system: /mnt/lun

SNAPSHOT CREATION WITH SNAPDRIVE





List Snapshot Copies with SnapDrive

List the Snapshot copies for a mounted LUN:# snapdrive snap list -v -fs /mnt/lun

snap name host date snapped

--------------------------------------------------------------

system:/vol/SANvol:my_snap rhel Oct 22 16:23 /mnt/lun

host OS: Linux 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008

snapshot name: my_snap

file system: type: ext3 mountpoint: /mnt/lun

lun path dev paths

--------------------------------------

system:/vol/SANvol/lun /dev/sdj

LIST SNAPSHOT COPIES WITH SNAPDRIVE





Snapshot Restore with SnapDrive

To restore a mounted LUN:# pwd

/mnt/lun

# ls

foo lost+found

# rm foo

# ls

lost+found

# cd ..

# pwd

/mnt

# snapdrive snap restore -fs /mnt/lun -snapname my_snap

...

# cd lun

# ls

foo lost+found

Verify that you are not in the mountpoint

Requires snaprestorelicense on the storage system

Remove a file

The file is restored

SNAPSHOT RESTORE WITH SNAPDRIVE





Removing a LUN with SnapDrive

To remove a mounted LUN:# pwd

/mnt

# ls

lun otherlun

# snapdrive storage delete -fs /mnt/lun

delete file system /mnt/lun

- fs /mnt/lun ... deleted

- LUN system:/vol/SANvol/lun ... deleted

# ls

otherlun

Verify that you are not in the mountpoint

REMOVING A LUN WITH SNAPDRIVE





Module Summary

MODULE SUMMARY





Module Summary

In this module, you should have learned to:Describe the steps to configure SnapDrive for UNIX on Linux

MODULE SUMMARY


Documents

SAN Implementation Workshop_StudentGuide