SAN Features ONTAP 9 - NetApp | NetApp Hands-On … · • Move a volume containing a LUN, and observe iSCSI traffic and MPIO changes. • Demonstrate SFMoD on a LUN, and observe

SAN Features ONTAP 9

October 2017 | SL10333 Version 1.3.2

SAN Features ONTAP 92 © 2017 NetApp, Inc. All rights reserved. NetApp Proprietary

TABLE OF CONTENTS

1 Introduction...................................................................................................................................... 4

1.1 Lab Objectives........................................................................................................................... 4

1.2 Prerequisites.............................................................................................................................. 4

2 Lab Environment............................................................................................................................. 5

2.1 Lab Diagram...............................................................................................................................5

3 Lab Activities................................................................................................................................... 6

3.1 Preparing the Windows Server 2012 R2 Host........................................................................ 6

3.1.1 Login..................................................................................................................................................................... 6

3.1.2 View LUN Configuration....................................................................................................................................... 7

3.1.3 Start iSCSI traffic to LUN..................................................................................................................................... 8

3.2 DataMotion for Volumes......................................................................................................... 10

3.2.1 Preparation..........................................................................................................................................................10

3.2.2 Start DataMotion for Volumes............................................................................................................................ 11

3.3 DataMotion for LUN.................................................................................................................14

3.3.1 Preparation..........................................................................................................................................................14

3.3.2 Start DataMotion for LUN...................................................................................................................................14

3.4 Selective LUN Mapping...........................................................................................................16

3.4.1 Preparation..........................................................................................................................................................16

3.4.2 Add Paths........................................................................................................................................................... 16

3.4.3 DataMotion for Volumes..................................................................................................................................... 18

3.4.4 Remove Paths.................................................................................................................................................... 20

3.5 Connecting to and Preparing the RHEL Host...................................................................... 21

3.5.1 Login................................................................................................................................................................... 21

3.5.2 View LUN configuration...................................................................................................................................... 23

3.5.3 Start iSCSI traffic to LUN................................................................................................................................... 24

3.6 DataMotion for Volumes......................................................................................................... 25


3.6.1 Preparation..........................................................................................................................................................25

3.6.2 Start DataMotion for Volumes............................................................................................................................ 25

3.7 DataMotion for LUN.................................................................................................................27

3.7.1 Preparation..........................................................................................................................................................27

3.7.2 Start DataMotion for LUN...................................................................................................................................27

3.8 Selective LUN Mapping...........................................................................................................29

3.8.1 Preparation..........................................................................................................................................................29

3.8.2 Add paths............................................................................................................................................................29

3.8.3 DataMotion for Volumes..................................................................................................................................... 31

3.8.4 Remove paths.....................................................................................................................................................33

4 Lab Limitations.............................................................................................................................. 35

5 Software..........................................................................................................................................36

6 Version History.............................................................................................................................. 37


1 IntroductionThe purpose of this lab is to introduce SAN features in ONTAP® 9.1. SAN features introduced in this labinclude DataMotion for LUNs and Selective LUN mapping. Also covered is MPIO path management in WindowsServer 2012 R2, and Red Hat Enterprise Linux release 6.5 while using these new SAN features. The lab alsodemonstrates the differences between DataMotion for LUNs and DataMotion for Volumes, and how they workwith Selective LUN mapping.

1.1 Lab Objectives• This lab has the following goals and objectives:• Introduce and demonstrate SAN features in ONTAP 9.1.

• Demonstrate these features on Windows Server 2012 R2.• Demonstrate these features on Red Hat Enterprise Linux Server release 6.5.

• Demonstrate and manage MPIO during data mobility operations.

1.2 PrerequisitesWe recommend that you have a basic understanding of the following concepts before you start this lab:

• ONTAP, Windows Server, and Red Hat Enterprise Linux.• SAN storage, such as the initiator-target concept.


2 Lab EnvironmentThis lab uses Jumphost to demonstrate ONTAP storage features on a Windows host. RHEL1 is used todemonstrate the features on a Linux host. The storage system is cluster1, running ONTAP, and is composedof 4 nodes (2 HA-pairs). RHEL1 and cluster1 are accessed using SSH through PuTTY sessions, and theyauthenticate using SSH keys that are already established.

Both Jumphost and RHEL1 already have 8 iSCSI sessions established, 2 sessions per node for 4 nodes.Sessions on each particular node are separated by a different subnet, to follow the dual-subnet design forresilience and redundancy. Jumphost and RHEL1 have 2 NICs each, in order to reach both subnets.

Hostname Description

Jumphost.demo.netapp.comWindows Server 2012 R2

Password: Netapp1!

RHEL1.demo.netapp.comRed Hat Enterprise Linux Server release 6.5

Password not needed, using SSH key exchange with Jumphost.

cluster1.demo.netapp.comONTAP

Password not needed, using SSH key exchange with Jumphost.

2.1 Lab DiagramThe following illustration identifies the components of this lab.

Figure 2-1:


3 Lab ActivitiesThis lab is designed to accomplish the following four major tasks:

• Prepare the host, and understand the initial configuration of the iSCSI SAN.• Move a volume containing a LUN, and observe iSCSI traffic and MPIO changes.• Demonstrate SFMoD on a LUN, and observe iSCSI traffic and MPIO changes. Contrast with

DataMotion for Volumes.• Demonstrate SLM by moving a volume containing a LUN while maintaining direct paths to the LUN.

This involves expanding/contracting the LUN masking on the cluster.

Use the following lab activities to accomplish those tasks:

• Preparing the Windows Server 2012 R2 Host on page 6• DataMotion for Volumes on page 10• DataMotion for LUN on page 14• Selective LUN Mapping on page 16• Connecting to and Preparing the RHEL Host on page 21• DataMotion for Volumes on page 25• DataMotion for LUN on page 27• DataMotion for LUN on page 27

3.1 Preparing the Windows Server 2012 R2 HostThis section begins the preparation of the Windows Server 2012 R2 host.

3.1.1 Login

Perform the following activities to log in to the Windows Server 2012 R2:

1. Log in to Jumphost which is a Windows Server 2012 R2 server.2. Double-click Computer Management on the desktop. You may need to maximize the window.

Figure 3-1:


3.1.2 View LUN Configuration

1. In the left pane, under the Storage section, click Data ONTAP(R) DSM Management.

Observe the LUN’s MPIO configuration. There are 4 paths. Even though you have 8 iSCSI sessions,there are only 4 paths to lun1 because of Selective LUN Mapping, which is also covered in this guide.The paths to cluster1-01 are direct paths, and labeled “Active/Optimized”. The paths to cluster1-02,which are used during an HA event, are indirect paths and labeled “Active/Non-Optimized”.

Figure 3-2:

Now, you will observe the LUN mapping from the clustered Data ONTAP view.2. Right-click the PuTTY icon on the taskbar and click cluster1 under Recent Sessions.

Figure 3-3:

3. Issue the set adv command, and when prompted to continue, enter y.

Using username "admin".Authenticating with public key "rsa-key-20140813"`cluster1::> set advWarning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel.Do you want to continue? {y|n}: ycluster1::*>


4. Issue the lun mapping show -vserver svm_win -fields reporting-nodes command.

cluster1::*> lun mapping show -vserver svm_win -fields reporting-nodesvserver path igroup reporting-nodes------- ----------------- --------- -----------------------svm_win /vol/vol_win/lun1 win_iscsi cluster1-01,cluster1-02svm_win /vol/vol_win/lun2 win_iscsi cluster1-01,cluster1-022 entries were displayed.

5. Observe the LUN mapping. The nodes cluster1-01 and cluster1-02 are reporting nodes; they reportand permit connections to the LUNs in the list. Minimize the “Computer Management” window, and thePuTTY session window.

Figure 3-4:

3.1.3 Start iSCSI traffic to LUN

Now, you will generate iSCSI traffic to lun1.

1. Double-click lun1.icf on the desktop. Wait a few seconds for Iometer to initialize.

Figure 3-5:

2. Click the Results Display tab in the Iometer window. This will permit you to see the current traffic levels.3. Make sure to choose Last Update.


4. Click the green flag button near the top of the Iometer window. This will begin the traffic to lun1.

Figure 3-6:

5. Click Save so that the Iometer traffic begins.

Figure 3-7:

6. Iometer has now started a 4KiB block, 50% Read, 0% Random workload on lun1.


Figure 3-8:

3.2 DataMotion for VolumesNetApp DataMotion allows you to nondiruptively migrate volumes containing LUNs between aggregates in thesame cluster. This lab activity will demonstrate ONTAP data mobility by moving a volume containing a LUN that isaccess through iSCSI nondisruptively.

3.2.1 Preparation

Perform the following task to prepare the desktop of the Jumphost with Windows into each component usedduring this lab activity in order to view the operation.

1. Restore the cluster1 PuTTY window by clicking the PuTTY icon on the taskbar.

Figure 3-9:

2. Create two more cluster1 PuTTY sessions. You will use them to monitor iSCSI traffic on cluster1-01 andcluster1-02.

3. Arrange and resize the windows to prepare to monitor the environment as the vol move occurs.


Figure 3-10:

4. In one of the PuTTY session windows, run the run -node cluster1-01 -command sysstat -i command,and in another PuTTY session window run the run -node cluster1-02 -command sysstat -i command.This allows you to monitor the iSCSI traffic on cluster1-01 and cluster1-02 as the vol move occurs.

Node cluster1-01:

cluster1::> run -node cluster1-01 -command sysstat -i CPU NFS CIFS iSCSI Net kB/s Disk kB/s iSCSI kB/s Cache in out read write in out age 42% 0 0 851 1821 2015 21 21 1650 1835 21s 44% 0 0 829 1817 1912 1484 5277 1653 1745 21s 58% 0 0 782 1777 1751 231 17754 1617 1587 21s

Node cluster1-02:


3.2.2 Start DataMotion for Volumes

The following activities start the move of the volume to a new destination aggregate. During the move you will seetraffic continuing to pass over the LUN and following the move you will see the affect on the preferred paths.


Figure 3-11:

1. To start the vol move to cluster1-02, issue the vol move start -vserver svm_win -volume vol_win -destination-aggregate aggr2 command in the third PuTTY session window.

cluster1::> vol move start -vserver svm_win -volume vol_win -destination-aggregate aggr2[Job 825] Job is queued: Move "vol_win" in Vserver "svm_win" to aggregate "aggr2". Use the "volume move show -vserver svm_win -volume vol_win" command to view the status of this operation.

2. To show the status of the vol move, issue the vol move show -vserver svm_win -volume vol_wincommand.

cluster1::> vol move show -vserver svm_win -volume vol_win Vserver Name: svm_win Volume Name: vol_win Actual Completion Time: - Bytes Remaining: 771.9MB Destination Aggregate: aggr2 Detailed Status: Transferring data: 955.6MB sent.Estimated Time of Completion: Tue Sep 02 15:31:25 2014 Managing Node: cluster1-01 Percentage Complete: 54% Move Phase: replicatingEstimated Remaining Duration: 00:00:11 Replication Throughput: 68.26MB/s Duration of Move: 00:00:16 Source Aggregate: aggr1 Start Time of Move: Tue Sep 02 15:30:58 2014 Move State: healthycluster1::> vol move show -vserver svm_win -volume vol_win Vserver Name: svm_win Volume Name: vol_win Actual Completion Time: Tue Sep 02 15:31:41 2014 Bytes Remaining: - Destination Aggregate: aggr2 Detailed Status: SuccessfulEstimated Time of Completion: - Managing Node: cluster1-01 Percentage Complete: 100% Move Phase: completedEstimated Remaining Duration: - Replication Throughput: 71.20MB/s Duration of Move: 00:00:43 Source Aggregate: aggr1 Start Time of Move: Tue Sep 02 15:30:58 2014 Move State: done

3. Observe the status of Iometer and iSCSI sysstat traffic on cluster1-01 and cluster1-02 as the volume ismoved, and cutover occurs.


Figure 3-12:

Note: Since there is a constant level of IO on the volume, cutover may be deferred if ONTAP isunable to quiesce the volume; usually cutover is successful after the first deferral.

cluster1::> vol move show -vserver svm_win -volume vol_win Vserver Name: svm_win Volume Name: vol_win Actual Completion Time: - Bytes Remaining: 12.65MB Destination Aggregate: aggr2 Detailed Status: Waiting to Cutover. (1.61GB Sent)::Reason: Preparing source volume for cutover: Volume quiesce failed because there are outstanding file system requests on the volume (Volume can't be quiesced as it did not drain in time.) Volume move job at decision pointEstimated Time of Completion: Tue Sep 02 16:14:10 2014 Managing Node: cluster1-01 Percentage Complete: 98% Move Phase: cutover_soft_deferredEstimated Remaining Duration: 00:00:45 Replication Throughput: 82.27MB/s Duration of Move: 00:01:04 Source Aggregate: aggr1 Start Time of Move: Tue Sep 02 16:12:21 2014 Move State: healthy

4. Observe the path changes in Data ONTAP(R) DSM.

Note: You may have to click Refresh in the right pane.


5. Verify the volume now resides on aggr2.

cluster1::> vol show -vserver svm_winVserver Volume Aggregate State Type Size Available Used%--------- ------------ ------------ ---------- ---- ---------- ---------- -----svm_win svm_win_root aggr1 online RW 20MB 18.88MB 5%svm_win vol_win aggr2 online RW 10GB 7.91GB 20%2 entries were displayed.

You have just completed a DataMotion for Volume operation in an iSCSI SAN environment. Notice howthis feature enables non-disruptive data mobility within the cluster, even with load on the data, whilemaintaining direct (Active/Optimized) SAN paths. Also, with the value of Data ONTAP DSM, you caneasily see path changes and LUN location in the context of your SVM and cluster.

3.3 DataMotion for LUNThis section demonstrates a new SAN feature ONTAP 9.1. Single File Move on Demand (SFMoD) allows you tomove and copy LUNs between volumes.

3.3.1 Preparation

1. Previously, you moved a volume from one aggregate to another, now you are going to move a lun fromone volume to a new volume and see the affect on the live data traffic and path..

2. To create the destination volume dest_vol on aggr1, issue the vol create -vserver svm_win -volumedest_vol -aggregate aggr1 -size 10GB -state online command.

3. Enter y if prompted.

cluster1::> vol create -vserver svm_win -volume dest_vol -aggregate aggr1 -size 10GB -state online[Job 860] Job succeeded: Successful

3.3.2 Start DataMotion for LUN

1. Initiate the DataMotion for LUN by running the lun move start -vserver svm_win -destination-path /vol/dest_vol/lun1 -source-path /vol/vol_win/lun1 command.

cluster1::> lun move start -vserver svm_win -destination-path /vol/dest_vol/lun1 -source-path /vol/vol_win/lun1Following LUN moves have been started:"svm_win:/vol/vol_win/lun1" move to "svm_win:/vol/dest_vol/lun1"


2. To show the status of the LUN move, run the lun move show -vserver svm_win or, for a moredetailed output, lun move show -vserver svm_win –instance command.

cluster1::> lun move show -vserver svm_winVserver Destination Path Status Progress--------- ------------------------------- --------------- --------svm_win /vol/dest_vol/lun1 Data 100%cluster1::> lun move show -vserver svm_win -instance Vserver Name: svm_win Destination Path: /vol/dest_vol/lun1 Source Path: /vol/vol_win/lun1 Is Destination Promoted Late: falseMaximum Transfer Rate (per sec): 0B LUN Move Status: Complete LUN Move Progress (%): 100% Elapsed Time: 0h0m52s Cutover Time: 0h0m0s Is Snapshot Fenced: false Is Destination Ready: true Last Failure Reason: -

3. Observe the status of Iometer & iSCSI sysstat traffic on cluster1-01 & cluster1-02 as the LUN is moved.Notice how cutover occurs immediately with DataMotion for LUN, whereas DataMotion for Volumescutover occurs after movement is complete.

4. Observe the path changes in Data ONTAP(R) DSM. The node cluster1-01 now holds the direct paths

marked Active/Optimized.



5. Verify the LUN now resides in dest_vol on aggr1.

cluster1::> lun show -vserver svm_winVserver Path State Mapped Type Size--------- ------------------------------- ------- -------- -------- --------svm_win /vol/dest_vol/lun1 online mapped windows 1.00GBsvm_win /vol/vol_win/lun2 online mapped windows 2.01GB2 entries were displayed.

You have just completed a DataMotion for LUN operation in an iSCSI SAN environment. DataMotion forLUN is a new SAN feature in the next major release of clustered Data ONTAP. Notice how this featureenables non-disruptive mobility of sub-volume data within the cluster, even with load on the data, whilemaintaining direct (Active/Optimized) SAN paths. Also, with the value of Data ONTAP DSM, you caneasily see path changes and LUN location in the context of your SVM and cluster.

3.4 Selective LUN MappingThis section demonstrates a new SAN feature in ONTAP 9.1. Selective LUN Mapping (SLM) enables LUNmasking at the node level. SLM is enabled by default on all created LUNs. It can be used with, or without portsets.

3.4.1 Preparation

1. The node cluster1-01 is currently hosting lun1, this is shown by issuing the lun show -vserversvm_win -path /vol/dest_vol/lun1 -fields node command.

cluster1::> lun show -vserver svm_win -path /vol/dest_vol/lun1 -fields nodevserver path node------- ------------------ -----------svm_win /vol/dest_vol/lun1 cluster1-01

2. SLM maps the LUN to the hosting node’s HA-pair, so in this case, nodes cluster1-01 and cluster1-02present the LUN to its mapped igroup. To show this, enter advanced mode (if not already in it) and issuethe lun mapping show -vserver svm_win -fields reporting-nodes command.

cluster1::> set advWarning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel.Do you want to continue? {y|n}: ycluster1::*> lun mapping show -vserver svm_win -fields reporting-nodesvserver path igroup reporting-nodes------- ------------------ --------- -----------------------svm_win /vol/dest_vol/lun1 win_iscsi cluster1-01,cluster1-02svm_win /vol/vol_win/lun2 win_iscsi cluster1-01,cluster1-022 entries were displayed.

3.4.2 Add Paths

1. You will now prepare the cluster for dest_vol to be moved to aggr3, which is owned by cluster1-03. Sincecluster1-03 belongs to the cluster1-03/cluster1-04 HA-pair, if you were to move dest_vol to aggr3, thenlun1 would still be mapped to the cluster1–01/cluster1–02 HA-pair, and all paths would become indirect.

To avoid this, bring the destination HA-pair into the current SLM configuration, add the new paths toWindows, move the volume, and then remove the source HA-pair paths once the move is complete. Thisway, you maintain at least 2 direct paths.

2. To add the destination HA-pair to the SLM configuration for lun1, issue the lun mapping add-reporting-nodes -vserver svm_win -path /vol/dest_vol/lun1 -igroup win_iscsi -

destination-aggregate aggr3 command.

cluster1::*> lun mapping add-reporting-nodes -vserver svm_win -path /vol/dest_vol/lun1 -igroup win_iscsi -destination-aggregate aggr3


3. To verify the new SLM configuration for lun1, issue the lun mapping show -vserver svm_win -fields reporting-nodes command.

cluster1::*> lun mapping show -vserver svm_win -path /vol/dest_vol/lun1 -fields reporting-nodesvserver path igroup reporting-nodes------- ------------------ --------- -----------------------------------------------svm_win /vol/dest_vol/lun1 win_iscsi cluster1-01,cluster1-02,cluster1-03,cluster1-04

4. Click Disk Management in the left pane of the Computer Management window.

Figure 3-13:

5. Click Action near the top of the window, then click Rescan Disks. This will initiate Windows to scan itsSCSI bus, and will pick up the new paths from the destination HA-pair.

Figure 3-14:

6. In the left pane, under the “Storage” section, click Data ONTAP(R) DSM Management. Observe thepath changes in Data ONTAP(R) DSM. There should now be 8 paths shown, 2 paths per node for 4nodes. Now the LUN will have direct paths on its destination HA-pair.



Figure 3-15:

7. To observe iSCSI traffic as the vol move occurs, click inside the PuTTY session running sysstat oncluster1-02 and press Ctrl-D to move back to the cluster shell, then issue the run -node cluster1-03 -command sysstat –i command.

Note: If you are using a MAC you will press command+D.

Figure 3-16:

3.4.3 DataMotion for Volumes

Now that Windows has paths to all the nodes, you can move the volume to aggr3 and Data ONTAP(R) DSM willhandle the Active/Optimized path changes so you can maintain direct paths to the LUN.

1. Issue the vol move start -vserver svm_win -volume dest_vol -destination-aggregate aggr3command.

cluster1::*> vol move start -vserver svm_win -volume dest_vol -destination-aggregate aggr3


[Job 868] Job is queued: Move "dest_vol" in Vserver "svm_win" to aggregate "aggr3". Use the "volume move show -vserver svm_win -volume dest_vol" command to view the status of this operation.

2. To show the status of the vol move, issue the vol move show -vserver svm_win -volume dest_volcommand.

cluster1::*> vol move show -vserver svm_win -volume dest_vol Vserver Name: svm_win Volume Name: dest_vol Actual Completion Time: - Bytes Remaining: 898.8MB Specified Action For Cutover: retry_on_failure Specified Cutover Time Window: 45 Destination Aggregate: aggr3 Destination Node: cluster1-03 Detailed Status: Transferring data: 463.1MB sent. Estimated Time of Completion: Wed Sep 03 16:39:41 2014 Job ID: 868 Managing Node: cluster1-01 Percentage Complete: 33% Move Phase: replicating Prior Issues Encountered: - Estimated Remaining Duration: 00:00:15 Replication Throughput: 57.89MB/s Duration of Move: 00:00:10 Source Aggregate: aggr1 Source Node: cluster1-01 Start Time of Move: Wed Sep 03 16:39:16 2014 Move State: healthyMove Initiated by Auto Balance Aggregate: false

3. Observe the status of Iometer and iSCSI sysstat traffic on cluster1-01 and cluster1-03 as the volume ismoved, and cutover occurs.

Note: Since there is a constant level of I/O on the volume, cutover may be deferred if ONTAP isunable to quiesce the volume; usually cutover is successful after the first deferral.

Figure 3-17:

4. Observe the path changes in Data ONTAP(R) DSM; cluster1-03 should now own both of the direct,Active/Optimized paths.

Note: You may have to click Refresh in the right pane for the changes to be visible.


Figure 3-18:


cluster1::*> vol show -vserver svm_winVserver Volume Aggregate State Type Size Available Used%--------- ------------ ------------ ---------- ---- ---------- ---------- -----svm_win dest_vol aggr3 online RW 10GB 8.19GB 18%svm_win svm_win_root aggr1 online RW 20MB 18.88MB 5%svm_win vol_win aggr2 online RW 10GB 8.50GB 14%3 entries were displayed.

3.4.4 Remove Paths

1. Now you will remove the previous HA-pair from the lun1 masking. Issue the lun mapping remove-reporting-nodes -vserver svm_win -path /vol/dest_vol/lun1 -igroup win_iscsi -remote-

nodes true command.

cluster1::> lun mapping remove-reporting-nodes -vserver svm_win -path /vol/dest_vol/lun1 -igroup win_iscsi -remote-nodes true

2. Verify the reporting nodes for lun1, enter advanced mode (if not already in it) and issue the lun mappingshow -vserver svm_win -fields reporting-nodes command.

cluster1::> set advWarning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel.Do you want to continue? {y|n}: ycluster1::*> lun mapping show -vserver svm_win -fields reporting-nodesvserver path igroup reporting-nodes------- ------------------ --------- -----------------------svm_win /vol/dest_vol/lun1 win_iscsi cluster1-03,cluster1-04svm_win /vol/vol_win/lun2 win_iscsi cluster1-01,cluster1-022 entries were displayed.

3. Observe the path changes in Data ONTAP(R) DSM.



Figure 3-19:

You have just completed a DataMotion for Volume operation while utilizing Selective LUN Mappingin an iSCSI SAN environment. Notice how Selective LUN Mapping enables greater, more granularcontrol of SAN paths to data across the cluster. This enables you to choose which controllers presentparticular LUNs on their SAN LIFs, and also helps you maintain direct (Active/Optimized) paths to datawhile engaging in non-disruptive data mobility operations within the cluster. Also, with the value of DataONTAP DSM, you can easily see path changes and LUN location in the context of your SVM and cluster.

Please note that using Selective LUN Mapping in conjunction with DataMotion for LUN is also supported.

3.5 Connecting to and Preparing the RHEL HostThis section illustrates how to connect to and prepare the RHEL host.

Note: If you are starting this section right after completing the Windows section, then close all currentlyopen windows and begin at Step 2 below.

3.5.1 Login

1. From the Jumphost (the Windows Server 2012 R2) close any open windows to ensure you have enoughspace on your desktop.

2. Right-click the PuTTY icon on the taskbar, and click rhel1 under Recent Sessions.

Note: Once the PuTTY session terminal window opens, feel free to adjust the size of the windowto your preference, as it will be the main RHEL session window where commands will be run.


Figure 3-20:

3. Open another rhel1 PuTTY session. You will use this secondary session to initiate storage workloads.4. Right-click the PuTTY icon on the taskbar and click cluster1 under “Recent Sessions”.

Figure 3-21:

5. Repeat the previous step twice, so that you have a total of three cluster1 PuTTY session windows. Youwill use two of these windows to monitor iSCSI traffic, and the other to initiate cluster commands.


Figure 3-22:

3.5.2 View LUN configuration

1. In the main rhel1 session window, issue the df –h command. This shows the currently mounted devices.Notice the partition /dev/mapper/mpathbp1 is mounted at /lun1, and is 1GB in size. That is the LUN youwill work with.

[root@RHEL1 ~]# df -hFilesystem Size Used Avail Use% Mounted on/dev/mapper/vg_rhel1-lv_root 12G 4.9G 6.0G 45% /tmpfs 1.9G 76K 1.9G 1% /dev/shm/dev/sda1 485M 40M 421M 9% /boot/dev/mapper/mpathap1 2.0G 1.1G 853M 56% /lun2/dev/mapper/mpathbp1 1007M 518M 438M 55% /lun1

2. Next, issue the multipath –ll command (the letter "l" twice). Observe the paths for mpathb. Noticethe top two paths have “status=active”. These are the direct paths; the other two paths labeled“status=enabled” are the indirect paths.

[root@RHEL1 ~]# multipath -llmpathb (3600a0980774f6a344d5d463251667a63) dm-2 NETAPP,LUN C-Modesize=1.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 6:0:0:1 sdc 8:32 active ready running| `- 4:0:0:1 sdb 8:16 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 7:0:0:1 sdf 8:80 active ready running `- 10:0:0:1 sdg 8:96 active ready runningmpatha (3600a0980774f6a344d5d463251667a64) dm-3 NETAPP,LUN C-Modesize=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 6:0:0:2 sde 8:64 active ready running| `- 4:0:0:2 sdd 8:48 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 10:0:0:2 sdi 8:128 active ready running `- 7:0:0:2 sdh 8:112 active ready running

3. To view the current iSCSI sessions, issue the iscsiadm -m session command. There should be 8sessions, 2 sessions per node for 4 nodes. Even though you have 8 iSCSI sessions, there are only 4paths to lun1 because of Selective LUN Mapping.

[root@RHEL1 ~]# iscsiadm -m sessiontcp: [1] 192.168.2.32:3260,1039 iqn.1992-08.com.netapp:sn.deff0b1123dc11e4be370050569922da:vs.6tcp: [2] 192.168.1.33:3260,1040 iqn.1992-08.com.netapp:sn.deff0b1123dc11e4be370050569922da:vs.6


tcp: [3] 192.168.1.32:3260,1038 iqn.1992-08.com.netapp:sn.deff0b1123dc11e4be370050569922da:vs.6tcp: [4] 192.168.2.33:3260,1041 iqn.1992-08.com.netapp:sn.deff0b1123dc11e4be370050569922da:vs.6tcp: [5] 192.168.1.34:3260,1042 iqn.1992-08.com.netapp:sn.deff0b1123dc11e4be370050569922da:vs.6tcp: [6] 192.168.1.31:3260,1036 iqn.1992-08.com.netapp:sn.deff0b1123dc11e4be370050569922da:vs.6tcp: [7] 192.168.2.31:3260,1037 iqn.1992-08.com.netapp:sn.deff0b1123dc11e4be370050569922da:vs.6tcp: [8] 192.168.2.34:3260,1043 iqn.1992-08.com.netapp:sn.deff0b1123dc11e4be370050569922da:vs.6

4. Issue the snapdrive storage show -devices command. This command is provided by NetApp’sSnapDrive for UNIX application. Observe the LUN path on the SVM.

[root@RHEL1 ~]# snapdrive storage show -devicesConnected LUNs and devices: device filename adapter path size proto state clone lun path backing snapshot---------------- ------- ---- ---- ----- ----- ----- -------- ----------------/dev/mapper/mpathb - P 1g iscsi online No svm_linux:/vol/vol_linux/lun1 -/dev/mapper/mpatha - P 2g iscsi online No svm_linux:/vol/vol_linux/lun2 -

5. Issue the sanlun lun show command. This command is provided by NetApp’s Linux Host Utilities Kit.This provides additional useful LUN information.

[root@RHEL1 ~]# sanlun lun showcontroller(7mode)/ device host lunvserver(Cmode) lun-pathname filename adapter protocol size mode-------------------------------------------------------------------------------------------svm_linux /vol/vol_linux/lun2 /dev/sdd host4 iSCSI 2g Csvm_linux /vol/vol_linux/lun2 /dev/sdh host7 iSCSI 2g Csvm_linux /vol/vol_linux/lun1 /dev/sdg host10 iSCSI 1g Csvm_linux /vol/vol_linux/lun1 /dev/sdf host7 iSCSI 1g Csvm_linux /vol/vol_linux/lun2 /dev/sdi host10 iSCSI 2g Csvm_linux /vol/vol_linux/lun2 /dev/sde host6 iSCSI 2g Csvm_linux /vol/vol_linux/lun1 /dev/sdc host6 iSCSI 1g Csvm_linux /vol/vol_linux/lun1 /dev/sdb host4 iSCSI 1g C

6. In the primary cluster1 PuTTY session window, issue the set adv command, and enter y when promptedto continue.

Using username "admin".Authenticating with public key "rsa-key-20140813"cluster1::> set advWarning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel.Do you want to continue? {y|n}: ycluster1::*>

7. Issue the lun mapping show -vserver svm_linux -fields reporting-nodes command. Observe thecurrent LUN mapping.

cluster1::> lun mapping show -vserver svm_linux -fields reporting-nodesvserver path igroup reporting-nodes--------- ------------------- ----------- -----------------------svm_linux /vol/vol_linux/lun1 linux_iscsi cluster1-03,cluster1-04svm_linux /vol/vol_linux/lun2 linux_iscsi cluster1-03,cluster1-042 entries were displayed.

3.5.3 Start iSCSI traffic to LUN

1. In the secondary rhel1 session window, you will start I/O to lun1 using SIO, a NetApp IO utility. Issue thesio 50 0 4k 500m 10000 2 /lun1/500M.randomFile command.

[root@RHEL1 ~]# sio 50 0 4k 500m 10000 2 /lun1/500M.randomFileVersion: 3.00


3.6 DataMotion for VolumesThis section demonstrates ONTAP data mobility by moving a volume containing a LUN being accessed throughiSCSI.

3.6.1 Preparation

1. To observe iSCSI traffic on cluster1–03 and cluster1–04 as the vol move occurs, use two of the cluster1PuTTY session windows. In one window, issue the run -node cluster1-03 -command sysstat -icommand , and in another PuTTY session window, issue the run -node cluster1-04 -command sysstat –i command.

Node cluster1-03


Node cluster1-04

cluster1::> run -node cluster1-04 -command sysstat -i CPU NFS CIFS iSCSI Net kB/s Disk kB/s iSCSI kB/s Cache in out read write in out age 9% 0 0 0 4 4 0 5 0 0 16s 14% 0 0 0 6 5 27 16 0 0 16s

7% 0 0 0 5 10 5 0 0 0 16s

3.6.2 Start DataMotion for Volumes

1. To start the vol move to cluster1–04, issue the vol move start -vserver svm_linux -volume vol_linux -destination-aggregate aggr4 command in the primary cluster1 PuTTY session window.

cluster1::> vol move start -vserver svm_linux -volume vol_linux -destination-aggregate aggr4[Job 942] Job is queued: Move "vol_linux" in Vserver "svm_linux" to aggregate "aggr4". Use the "volume move show -vserver svm_linux -volume vol_linux" command to view the status of this operation.


2. To show the status of the vol move, issue the vol move show -vserver svm_linux -volume vol_linuxcommand.

cluster1::> volume move show -vserver svm_linux -volume vol_linux Vserver Name: svm_linux Volume Name: vol_linux Actual Completion Time: - Bytes Remaining: 1.51GB Destination Aggregate: aggr4 Detailed Status: Transferring data: 460.2MB sent.Estimated Time of Completion: Fri Sep 05 18:50:45 2014 Managing Node: cluster1-03 Percentage Complete: 22% Move Phase: replicatingEstimated Remaining Duration: 00:00:29 Replication Throughput: 53.04MB/s Duration of Move: 00:00:12 Source Aggregate: aggr3 Start Time of Move: Fri Sep 05 18:50:04 2014 Move State: healthycluster1::> volume move show -vserver svm_linux -volume vol_linux Vserver Name: svm_linux Volume Name: vol_linux Actual Completion Time: Fri Sep 05 18:53:24 2014 Bytes Remaining: - Destination Aggregate: aggr4 Detailed Status: SuccessfulEstimated Time of Completion: - Managing Node: cluster1-03 Percentage Complete: 100% Move Phase: completedEstimated Remaining Duration: - Replication Throughput: 29.96MB/s Duration of Move: 00:03:20 Source Aggregate: aggr3 Start Time of Move: Fri Sep 05 18:50:04 2014 Move State: done

3. Observe the status of the iSCSI sysstat traffic on cluster1–03 and cluster1–04 as the volume is movedand cutover occurs.

Note: Since there is a constant high level of IO on the volume, cutover may be deferred ifONTAP is unable to quiesce the volume, usually cutover is successful after the first deferral.

cluster1::> volume move show -vserver svm_linux -volume vol_linux Vserver Name: svm_linux Volume Name: vol_linux Actual Completion Time: - Bytes Remaining: 687.9MB Destination Aggregate: aggr4 Detailed Status: Waiting to Cutover. (1.86GB Sent)::Reason: Preparing source volume for cutover: Volume quiesce failed because there are outstanding file system requests on the volume (Volume can't be quiesced as it did not drain in time.) Volume move job at decision pointEstimated Time of Completion: Fri Sep 05 18:52:55 2014 Managing Node: cluster1-03 Percentage Complete: 72% Move Phase: cutover_soft_deferredEstimated Remaining Duration: 00:00:45 Replication Throughput: 29.96MB/s Duration of Move: 00:02:06 Source Aggregate: aggr3 Start Time of Move: Fri Sep 05 18:50:04 2014 Move State: healthy

4. Observe the path changes by running the multipath -ll command in the primary rhel1 PuTTY sessionwindow. Notice how, under mpathb, sdf and sdg are now “status=active”, and sdc and sdb are now“status=enabled” while before it was opposite. This is because the volume is now on aggr4, whichis owned by cluster1–04. Therefore, the paths to cluster1–04 are now direct paths, and the paths tocluster1–03 are now indirect paths.

[root@RHEL1 ~]# multipath -llmpathb (3600a0980774f6a344d5d463251667a63) dm-2 NETAPP,LUN C-Modesize=1.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active


| |- 7:0:0:1 sdf 8:80 active ready running| `- 10:0:0:1 sdg 8:96 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 6:0:0:1 sdc 8:32 active ready running `- 4:0:0:1 sdb 8:16 active ready runningmpatha (3600a0980774f6a344d5d463251667a64) dm-3 NETAPP,LUN C-Modesize=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 7:0:0:2 sdh 8:112 active ready running| `- 10:0:0:2 sdi 8:128 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 4:0:0:2 sdd 8:48 active ready running `- 6:0:0:2 sde 8:64 active ready running


cluster1::> vol show -vserver svm_linuxVserver Volume Aggregate State Type Size Available Used%--------- ------------ ------------ ---------- ---- ---------- ---------- -----svm_linux svm_linux_root aggr3 online RW 20MB 18.88MB 5%svm_linux vol_linux aggr4 online RW 10GB 7.62GB 23%2 entries were displayed.

You have just completed a DataMotion for Volume operation in an iSCSI SAN environment. Notice howthis feature enables non-disruptive data mobility within the cluster, even with load on the data, whilemaintaining direct (Active/Optimized) SAN paths.

3.7 DataMotion for LUNThis section demonstrates a new SAN feature in the ONTAP 9.1. Single File Move on Demand (SFMoD) allowsyou to move and copy LUNs between volumes.

3.7.1 Preparation

Now that you’ve moved vol_linux to cluster1–04 and aggr4, this activity demonstrates DataMotion for LUN, whereyou move lun1 from vol_linux to a new volume on cluster1–03, and observe iSCSI traffic and path changes.

1. To create the destination volume dest_vol on aggr3, issue the vol create -vserver svm_linux -volumedest_vol -aggregate aggr3 -size 10GB -state online command.

2. Enter y if prompted to continue.

cluster1::> vol create -vserver svm_linux -volume dest_vol -aggregate aggr3 -size 10GB -state online[Job 943] Job succeeded: Successful

3.7.2 Start DataMotion for LUN

1. Initiate the DataMotion for LUN by issuing the lun move start -vserver svm_linux -destination-path /vol/dest_vol/lun1 -source-path /vol/vol_linux/lun1 command.

cluster1::> lun move start -vserver svm_linux -destination-path /vol/dest_vol/lun1 -source-path /vol/vol_linux/lun1Following LUN moves have been started:"svm_linux:/vol/vol_linux/lun1" move to "svm_linux:/vol/dest_vol/lun1"

2. To show the status of the LUN move, issue the lun move show -vserver svm_linux command. Or, for amore detailed output, issue the lun move show -vserver svm_linux –instance command.

cluster1::> lun move show -vserver svm_linuxVserver Destination Path Status Progress--------- ------------------------------- --------------- --------svm_linux /vol/dest_vol/lun1 Data 34%cluster1::> lun move show -vserver svm_linux -instance Vserver Name: svm_linux Destination Path: /vol/dest_vol/lun1 Source Path: /vol/vol_linux/lun1


Is Destination Promoted Late: falseMaximum Transfer Rate (per sec): 0B LUN Move Status: Data LUN Move Progress (%): 78% Elapsed Time: 0h0m12s Cutover Time: 0h0m0s Is Snapshot Fenced: false Is Destination Ready: true Last Failure Reason: -

3. Observe the status of iSCSI sysstat traffic on cluster1–03 and cluster1–04 as the LUN is moved. Noticehow cutover occurs immediately with DataMotion for LUN, whereas vol move cutover occurs aftermovement is complete.

Node cluster1-03:

CPU NFS CIFS iSCSI Net kB/s Disk kB/s iSCSI kB/s Cache in out read write in out age 8% 0 0 3 12 11 21 21 0 0 2 7% 0 0 0 5 3 0 0 0 0 2 14% 0 0 0 14 13 16 16 0 0 2 6% 0 0 0 11 11 5 5 0 0 2 6% 0 0 0 19 17 27 16 0 0 2 12% 0 0 0 13 11 8 723 0 0 2 6% 0 0 0 5 4 16 16 0 0 2 34% 0 0 7 51 60 743 9326 0 1 2 50% 0 0 0 74169 1921 1074 43078 0 0 2 82% 0 0 0 122692 3398 3299 93979 0 0 2 75% 0 0 0 37758 1299 458 110822 0 0 2 75% 0 0 80 88095 2489 991 123566 415 0 2 86% 0 0 845 130610 3436 1274 121662 4637 0 2 83% 0 0 1704 105350 2963 565 115920 8367 0 2 88% 0 0 3273 31807 1404 606 109707 14736 0 2 86% 0 0 2511 38768 1401 1426 108224 18020 0 2 82% 0 0 2946 50431 1748 610 78244 23669 0 2 70% 0 0 444 28254 759 295 50430 12767 0 2 78% 0 0 1834 43857 1216 670 54331 36546 0 2 83% 0 0 3825 32001 1057 938 50797 30252 1 2

Node cluster1-04:

CPU NFS CIFS iSCSI Net kB/s Disk kB/s iSCSI kB/s Cache in out read write in out age 55% 0 0 3625 36309 94298 3047 0 31481 0 >60 47% 0 0 3209 24881 30530 4786 21 22226 0 >60 51% 0 0 1997 14680 101059 4657 0 11062 0 >60 63% 0 0 2103 15997 129604 1500 16 11584 0 >60 43% 0 0 1829 11586 73248 5312 7670 8827 0 >60 32% 0 0 3328 16669 16827 90 4027 14736 0 >60 30% 0 0 2236 18842 18944 0 0 17027 0 >60 29% 0 0 2387 20776 20889 16 16 18804 0 >60 64% 0 0 895 10428 10462 16 5 9480 0 >60 18% 0 0 389 15465 15453 5 16 14272 0 >60 6% 0 0 0 19 17 21 0 0 0 >60 12% 0 0 0 4 3 1385 5289 0 0 >60 5% 0 0 0 5 4 37 37 0 0 >60 5% 0 0 3 4 3 16 16 0 0 >60 13% 0 0 0 4 3 0 0 0 0 >60 7% 0 0 0 6 4 32 21 0 0 >60 12% 0 0 0 7 5 0 0 0 0 >60 11% 0 0 3 5 4 27 16 0 0 >60 13% 0 0 0 13 6 850 1867 0 0 >60 25% 0 0 0 7 7 175 96 0 0 >60

4. Observe the path changes by issuing the multipath -ll command in the primary rhel1 PuTTY sessionwindow. Notice how, under mpathb, sdc and sdb are now “status=active”, and sdf and sdg are now“status=enabled” where before it was the opposite. This is because lun1 is now on dest_vol, whichresides on aggr3 which is owned by cluster1–03. Therefore the paths to cluster1–03 are now directpaths, and the paths to cluster1–04 are now indirect paths.

[root@RHEL1 ~]# multipath -llmpathb (3600a0980774f6a344d5d463251667a63) dm-2 NETAPP,LUN C-Modesize=1.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 6:0:0:1 sdc 8:32 active ready running


| `- 4:0:0:1 sdb 8:16 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 7:0:0:1 sdf 8:80 active ready running `- 10:0:0:1 sdg 8:96 active ready runningmpatha (3600a0980774f6a344d5d463251667a64) dm-3 NETAPP,LUN C-Modesize=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 7:0:0:2 sdh 8:112 active ready running| `- 10:0:0:2 sdi 8:128 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 4:0:0:2 sdd 8:48 active ready running `- 6:0:0:2 sde 8:64 active ready running

5. Verify that lun1 now resides in dest_vol on aggr3.

cluster1::> lun show -vserver svm_linuxVserver Path State Mapped Type Size--------- ------------------------------- ------- -------- -------- --------svm_linux /vol/dest_vol/lun1 online mapped linux 1GBsvm_linux /vol/vol_linux/lun2 online mapped linux 2GB2 entries were displayed.

You have just completed a DataMotion for LUN operation in an iSCSI SAN environment. Notice how thisfeature enables non-disruptive mobility of sub-volume data within the cluster, even with load on the data,while maintaining direct (Active/Optimized) SAN paths.

3.8 Selective LUN MappingThis section demonstrates another new SAN feature in ONTAP 9.1. Selective LUN Mapping (SLM) enables LUNmasking at the node level. SLM is enabled by default on all created LUNs. It can be used with, or without portsets.

3.8.1 Preparation

1. cluster1–03 is currently hosting lun1. You can view this by issuing the lun show -vserver svm_linux -path /vol/dest_vol/lun1 -fields node command.

cluster1::> lun show -vserver svm_linux -path /vol/dest_vol/lun1 -fields nodevserver path node--------- ------------------ -----------svm_linux /vol/dest_vol/lun1 cluster1-03

2. SLM maps the LUN to the hosting node’s HA-pair, so in this case, cluster1–03 and cluster1–04 presentthe LUN to its mapped igroup. To show this, enter advanced mode (if not already in it) and issue the lunmapping show -vserver svm_linux -fields reporting-nodes command.

cluster1::> set advWarning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel.Do you want to continue? {y|n}: ycluster1::*> lun mapping show -vserver svm_linux -fields reporting-nodesvserver path igroup reporting-nodes--------- ------------------ ----------- -----------------------svm_linux /vol/dest_vol/lun1 linux_iscsi cluster1-03,cluster1-04svm_linux /vol/vol_linux/lun2 linux_iscsi cluster1-03,cluster1-042 entries were displayed.

3.8.2 Add paths

You will now prepare the cluster for dest_vol to be moved to aggr1, which is owned by cluster1–01. Sincecluster1–01 belongs to the cluster1–01/cluster1–02 HA pair, if you were to move dest_vol to aggr1, then lun1would still be mapped to the cluster1–03/cluster1–04 HA pair, and all paths would become indirect.

To avoid this, you will bring the destination HA-pair into the current SLM configuration, add the new paths toRHEL, move the volume, and then remove the source HA-pair paths once the move is complete. This way, youmaintain at least 2 direct paths.


1. To add the destination HA pair to the SLM configuration for lun1, issue the lun mapping add-reporting-nodes -vserver svm_linux -path /vol/dest_vol/lun1 -igroup linux_iscsi -destination-aggregate

aggr1 command.

cluster1::*> lun mapping add-reporting-nodes -vserver svm_linux -path /vol/dest_vol/lun1 -igroup linux_iscsi -destination-aggregate aggr1

2. To verify the new SLM configuration for lun1, issue the lun mapping show -vserver svm_linux -fieldsreporting-nodes command.

cluster1::*> lun mapping show -vserver svm_linux -path /vol/dest_vol/lun1 -fields reporting-nodesvserver path igroup reporting-nodes--------- ------------------ ----------- -----------------------------------------------svm_linux /vol/dest_vol/lun1 linux_iscsi cluster1-01,cluster1-02,cluster1-03,cluster1-04

3. In the primary RHEL terminal, issue the rescan-scsi-bus.sh command. This will initiate RHEL to scan itsSCSI bus to pick up the new paths from the destination HA pair.

[root@RHEL1 ~]# rescan-scsi-bus.shHost adapter 0 (ata_piix) found.Host adapter 1 (ata_piix) found.Host adapter 10 (iscsi_tcp) found.Host adapter 2 (mptsas) found.Host adapter 3 (iscsi_tcp) found.Host adapter 4 (iscsi_tcp) found.Host adapter 5 (iscsi_tcp) found.Host adapter 6 (iscsi_tcp) found.Host adapter 7 (iscsi_tcp) found.Host adapter 8 (iscsi_tcp) found.Host adapter 9 (iscsi_tcp) found.Scanning SCSI subsystem for new devicesScanning host 0 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNsScanning for device 0 0 0 0 ...OLD: Host: scsi0 Channel: 00 Id: 00 Lun: 00 Vendor: NECVMWar Model: VMware IDE CDR00 Rev: 1.00 Type: CD-ROM ANSI SCSI revision: 05Scanning host 1 channels 0 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNsScanning host 2 channels 0 for SCSI target IDs 0 1 2 3 4 5 6 7, all LUNsScanning for device 2 0 0 0 ...OLD: Host: scsi2 Channel: 00 Id: 00 Lun: 00 Vendor: VMware Model: Virtual disk Rev: 1.0 Type: Direct-Access ANSI SCSI revision: 02Scanning host 3 for all SCSI target IDs, all LUNsScanning for device 3 0 0 1 ...OLD: Host: scsi3 Channel: 00 Id: 00 Lun: 01 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning host 4 for all SCSI target IDs, all LUNsScanning for device 4 0 0 1 ...OLD: Host: scsi4 Channel: 00 Id: 00 Lun: 01 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning for device 4 0 0 2 ...OLD: Host: scsi4 Channel: 00 Id: 00 Lun: 02 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning host 5 for all SCSI target IDs, all LUNsScanning for device 5 0 0 1 ...OLD: Host: scsi5 Channel: 00 Id: 00 Lun: 01 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning host 6 for all SCSI target IDs, all LUNsScanning for device 6 0 0 1 ...OLD: Host: scsi6 Channel: 00 Id: 00 Lun: 01 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning for device 6 0 0 2 ...OLD: Host: scsi6 Channel: 00 Id: 00 Lun: 02 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning host 7 for all SCSI target IDs, all LUNsScanning for device 7 0 0 1 ...OLD: Host: scsi7 Channel: 00 Id: 00 Lun: 01 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning for device 7 0 0 2 ...


OLD: Host: scsi7 Channel: 00 Id: 00 Lun: 02 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning host 8 for all SCSI target IDs, all LUNsScanning for device 8 0 0 1 ...OLD: Host: scsi8 Channel: 00 Id: 00 Lun: 01 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning host 9 for all SCSI target IDs, all LUNsScanning for device 9 0 0 1 ...OLD: Host: scsi9 Channel: 00 Id: 00 Lun: 01 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning host 10 for all SCSI target IDs, all LUNsScanning for device 10 0 0 1 ...OLD: Host: scsi10 Channel: 00 Id: 00 Lun: 01 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 05Scanning for device 10 0 0 2 ...OLD: Host: scsi10 Channel: 00 Id: 00 Lun: 02 Vendor: NETAPP Model: LUN C-Mode Rev: 8300 Type: Direct-Access ANSI SCSI revision: 050 new device(s) found.0 device(s) removed.

4. Observe the path changes by issuing the multipath –ll command. There should now be 8 paths shownunder mpathb, with 2 direct paths and 6 indirect paths. The 2 direct paths are the paths to cluster1–03,and the 6 indirect paths are the paths to cluster1–01,cluster1–02, and cluster1–04 (2 paths per node).

[root@RHEL1 ~]# multipath -llmpathb (3600a0980774f6a344d5d463251667a63) dm-2 NETAPP,LUN C-Modesize=1.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 6:0:0:1 sdc 8:32 active ready running| `- 4:0:0:1 sdb 8:16 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 7:0:0:1 sdf 8:80 active ready running |- 10:0:0:1 sdg 8:96 active ready running |- 3:0:0:1 sdj 8:144 active ready running |- 5:0:0:1 sdk 8:160 active ready running |- 8:0:0:1 sdl 8:176 active ready running `- 9:0:0:1 sdm 8:192 active ready runningmpatha (3600a0980774f6a344d5d463251667a64) dm-3 NETAPP,LUN C-Modesize=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 7:0:0:2 sdh 8:112 active ready running| `- 10:0:0:2 sdi 8:128 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 4:0:0:2 sdd 8:48 active ready running `- 6:0:0:2 sde 8:64 active ready running

5. To observe iSCSI traffic as the vol move occurs, click inside the PuTTY session running sysstat oncluster1–04, and click Ctrl-D to move back to the cluster shell. Then issue the run -node cluster1-01 -command sysstat –i command.

Note: If you are using a MAC you will press command+D.

9% 0 0 0 4 4 11 0 0 0 >606% 0 0 3 6 5 5 16 0 0 >60cluster1::> run -node cluster1-01 -command sysstat -i CPU NFS CIFS iSCSI Net kB/s Disk kB/s iSCSI kB/s Cache in out read write in out age 2% 0 0 0 7 9 27 16 0 0 >60 3% 0 0 0 4 5 0 0 0 0 >60 25% 0 0 0 4 8 32 21 0 0 >60

3.8.3 DataMotion for Volumes

1. Now that RHEL has paths to all the nodes, you can move the volume to aggr1 and RHEL’s native MPIOwill handle the direct/indirect path changes so you can maintain direct paths to the LUN. Issue the volmove start -vserver svm_linux -volume dest_vol -destination-aggregate aggr1 command.

cluster1::*> vol move start -vserver svm_linux -volume dest_vol -destination-aggregate aggr1


[Job 1010] Job is queued: Move "dest_vol" in Vserver "svm_linux" to aggregate "aggr1". Use the "volume move show -vserver svm_linux -volume dest_vol" command to view the status of this operation.

2. To show the status of the vol move, issue the vol move show -vserver svm_linux -volume dest_volcommand.

cluster1::*> vol move show -vserver svm_linux -volume dest_vol Vserver Name: svm_linux Volume Name: dest_vol Actual Completion Time: - Bytes Remaining: 1.17GB Specified Action For Cutover: retry_on_failure Specified Cutover Time Window: 45 Destination Aggregate: aggr1 Destination Node: cluster1-01 Detailed Status: Transferring data: 169.4MB sent. Estimated Time of Completion: Mon Sep 08 14:03:27 2014 Job ID: 1010 Managing Node: cluster1-03 Percentage Complete: 12% Move Phase: replicating Prior Issues Encountered: - Estimated Remaining Duration: 00:01:03 Replication Throughput: 18.82MB/s Duration of Move: 00:00:13 Source Aggregate: aggr3 Source Node: cluster1-03 Start Time of Move: Mon Sep 08 14:02:11 2014 Move State: healthyMove Initiated by Auto Balance Aggregate: falsecluster1::*> vol move show -vserver svm_linux -volume dest_vol Vserver Name: svm_linux Volume Name: dest_vol Actual Completion Time: - Bytes Remaining: 653.1MB Specified Action For Cutover: retry_on_failure Specified Cutover Time Window: 45 Destination Aggregate: aggr1 Destination Node: cluster1-01 Detailed Status: Waiting to Cutover. (1.28GB Sent)::Reason: Preparing source volume for cutover: Volume quiesce failed because there are outstanding file system requests on the volume (Volume can't be quiesced as it did not drain in time.) Volume move job at decision point Estimated Time of Completion: Mon Sep 08 14:05:26 2014 Job ID: 1010 Managing Node: cluster1-03 Percentage Complete: 66% Move Phase: cutover_soft_deferred Prior Issues Encountered: 9/8/2014 14:03:24 : Preparing source volume for cutover: Volume quiesce failed because there are outstanding file system requests on the volume (Volume can't be quiesced as it did not drain in time.) Estimated Remaining Duration: 00:00:45 Replication Throughput: 22.33MB/s Duration of Move: 00:02:30 Source Aggregate: aggr3 Source Node: cluster1-03 Start Time of Move: Mon Sep 08 14:02:11 2014 Move State: healthyMove Initiated by Auto Balance Aggregate: false

3. Observe the status of the iSCSI sysstat traffic on cluster1–03 and cluster1–01 as the volume is movedand cutover occurs.

Note: Since there is a constant high level of I/O on the volume, cutover may be deferred ifONTAP is unable to quiesce the volume, usually cutover is successful after the first deferral.

cluster1::*> vol move show -vserver svm_linux -volume dest_vol Vserver Name: svm_linux Volume Name: dest_vol Actual Completion Time: Mon Sep 08 14:05:59 2014 Bytes Remaining: - Specified Action For Cutover: retry_on_failure Specified Cutover Time Window: 45 Destination Aggregate: aggr1 Destination Node: cluster1-01 Detailed Status: Successful


Estimated Time of Completion: - Job ID: 1010 Managing Node: cluster1-03 Percentage Complete: 100% Move Phase: completed Prior Issues Encountered: 9/8/2014 14:03:24 : Preparing source volume for cutover: Volume quiesce failed because there are outstanding file system requests on the volume (Volume can't be quiesced as it did not drain in time.) Estimated Remaining Duration: - Replication Throughput: 23.23MB/s Duration of Move: 00:03:48 Source Aggregate: aggr3 Source Node: cluster1-03 Start Time of Move: Mon Sep 08 14:02:11 2014 Move State: doneMove Initiated by Auto Balance Aggregate: false

4. Observe the path changes by issuing the multipath -ll command in the primary rhel1 PuTTY sessionwindow. There should still be 8 paths shown under mpathb, with 2 direct paths and 6 indirect paths.However, the direct paths have changed. Before, sdc/sdb were direct paths, but now sdm/sdl are directpaths, because dest_vol (which owns lun1) is now hosted by cluster1–01 on aggr1.

[root@RHEL1 ~]# multipath -llmpathb (3600a0980774f6a344d5d463251667a63) dm-2 NETAPP,LUN C-Modesize=1.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 9:0:0:1 sdm 8:192 active ready running| `- 8:0:0:1 sdl 8:176 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 4:0:0:1 sdb 8:16 active ready running |- 6:0:0:1 sdc 8:32 active ready running |- 7:0:0:1 sdf 8:80 active ready running |- 10:0:0:1 sdg 8:96 active ready running |- 3:0:0:1 sdj 8:144 active ready running `- 5:0:0:1 sdk 8:160 active ready runningmpatha (3600a0980774f6a344d5d463251667a64) dm-3 NETAPP,LUN C-Modesize=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 7:0:0:2 sdh 8:112 active ready running| `- 10:0:0:2 sdi 8:128 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 4:0:0:2 sdd 8:48 active ready running `- 6:0:0:2 sde 8:64 active ready running


cluster1::*> vol show -vserver svm_linuxVserver Volume Aggregate State Type Size Available Used%--------- ------------ ------------ ---------- ---- ---------- ---------- -----svm_linux dest_vol aggr1 online RW 10GB 7.99GB 20%svm_linux svm_linux_root aggr3 online RW 20MB 18.88MB 5%svm_linux vol_linux aggr4 online RW 10GB 8.42GB 15%3 entries were displayed.

3.8.4 Remove paths

1. To remove the previous HA pair from the lun1 masking, issue the lun mapping remove-reporting-nodes -vserver svm_linux -path /vol/dest_vol/lun1 -igroup linux_iscsi -remote-nodes true command.

cluster1::> lun mapping remove-reporting-nodes -vserver svm_linux -path /vol/dest_vol/lun1 -igroup linux_iscsi -remote-nodes true

2. To verify the reporting nodes for lun1, enter advanced mode (if not already in it) and issue the lunmapping show -vserver svm_linux -fields reporting-nodes command.

cluster1::> set advWarning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel.Do you want to continue? {y|n}: ycluster1::*> lun mapping show -vserver svm_linux -path /vol/dest_vol/lun1 -igroup linux_iscsi -fields reporting-nodesvserver path igroup reporting-nodes


--------- ------------------ ----------- -----------------------svm_linux /vol/dest_vol/lun1 linux_iscsi cluster1-01,cluster1-02

3. Observe the path changes by issuing the multipath -ll command in the primary rhel1 PuTTY sessionwindow. There should still be 8 paths shown under mpathb, with 2 direct paths, 2 indirect paths, and 4“faulty” paths. The 4 “faulty” paths are a result of removing the cluster1–03/cluster1–04 HA pair from theSLM configuration for lun1.

[root@RHEL1 ~]# multipath -llSep 08 14:32:44 | sdb: couldn't get target port groupSep 08 14:32:44 | sdc: couldn't get target port groupSep 08 14:32:44 | sdf: couldn't get target port groupSep 08 14:32:44 | sdg: couldn't get target port groupmpathb (3600a0980774f6a344d5d463251667a63) dm-2 NETAPP,LUN C-Modesize=1.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 9:0:0:1 sdm 8:192 active ready running| `- 8:0:0:1 sdl 8:176 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 4:0:0:1 sdb 8:16 failed faulty running |- 6:0:0:1 sdc 8:32 failed faulty running |- 7:0:0:1 sdf 8:80 failed faulty running |- 10:0:0:1 sdg 8:96 failed faulty running |- 3:0:0:1 sdj 8:144 active ready running `- 5:0:0:1 sdk 8:160 active ready runningmpatha (3600a0980774f6a344d5d463251667a64) dm-3 NETAPP,LUN C-Modesize=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 7:0:0:2 sdh 8:112 active ready running| `- 10:0:0:2 sdi 8:128 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 4:0:0:2 sdd 8:48 active ready running `- 6:0:0:2 sde 8:64 active ready running

4. To remove these stale paths, use the same utility script, and request a scan on existing block devices.

[root@RHEL1 ~]# /usr/bin/rescan-scsi-bus.sh -r

5. Observe the path changes by issuing the multipath -ll command in the primary rhel1 PuTTY sessionwindow. Notice how sdb, sdc, sdf, and sdg have been removed from the multipath configuration. Thereshould now be 4 paths total, 2 direct and 2 indirect. The direct paths belong to cluster1–01, and theindirect paths belong to cluster1–02.

[root@RHEL1 ~]# multipath -llmpathb (3600a0980774f6a344d5d463251667a63) dm-2 NETAPP,LUN C-Modesize=1.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 8:0:0:1 sdl 8:176 active ready running| `- 9:0:0:1 sdm 8:192 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 3:0:0:1 sdj 8:144 active ready running `- 5:0:0:1 sdk 8:160 active ready runningmpatha (3600a0980774f6a344d5d463251667a64) dm-3 NETAPP,LUN C-Modesize=2.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 alua' wp=rw|-+- policy='round-robin 0' prio=50 status=active| |- 10:0:0:2 sdi 8:128 active ready running| `- 7:0:0:2 sdh 8:112 active ready running`-+- policy='round-robin 0' prio=10 status=enabled |- 6:0:0:2 sde 8:64 active ready running `- 4:0:0:2 sdd 8:48 active ready running

You have just completed a DataMotion for Volume operation while utilizing Selective LUN Mapping inan iSCSI SAN environment. Notice how Selective LUN Mapping enables greater, more granular controlof SAN paths to data across the cluster. This enables you to choose which controllers present particularLUNs on their SAN LIFs, and also helps you maintain direct (Active/Optimized) paths to data whileengaging in non-disruptive data mobility operations within the cluster.

Note: Selective LUN Mapping in conjunction with DataMotion for LUN is also supported.


4 Lab Limitations• Fibre Channel is not supported.• Performance is limited.


5 Software• ONTAP® 9.1• Data ONTAP® DSM• SnapDrive• System Manager• PuTTY• Iometer


6 Version History

Version Date Document Version History

Version 1.0.0 October 2014 Initial Release

Version 1.1.0 October 2015 Corrected linux/windows section swap

Version 1.2.0 December 2015 Updated components

Version 1.3.0 August 2017 Updated components

Version 1.3.1 September 2017 Updated lab diagram

Version 1.3.2 October 2017 Minor typo edits

Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exactproduct and feature versions described in this document are supported for your specific environment.The NetApp IMT defines the product components and versions that can be used to constructconfigurations that are supported by NetApp. Specific results depend on each customer's installation inaccordance with published specifications.

NetApp provides no representations or warranties regarding the accuracy, reliability, or serviceability of anyinformation or recommendations provided in this publication, or with respect to any results that may be obtainedby the use of the information or observance of any recommendations provided herein. The information in thisdocument is distributed AS IS, and the use of this information or the implementation of any recommendations ortechniques herein is a customer’s responsibility and depends on the customer’s ability to evaluate and integratethem into the customer’s operational environment. This document and the information contained herein may beused solely in connection with the NetApp products discussed in this document.

Go further, faster®

© 2017NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consentof NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, Data ONTAP®,ONTAP®, OnCommand®, SANtricity®, FlexPod®, SnapCenter®, and SolidFire® are trademarks or registeredtrademarks of NetApp, Inc. in the United States and/or other countries. All other brands or products are trademarks orregistered trademarks of their respective holders and should be treated as such.

http://support.netapp.com/matrix/mtx/login.do

Documents

SAN Features ONTAP 9 - NetApp | NetApp Hands-On … · • Move a volume containing a LUN, and observe iSCSI traffic and MPIO changes. • Demonstrate SFMoD on a LUN, and observe