Veritas Volume Manager 1

Sun Microsystems, Inc.UBRM05-104

500 Eldorado Blvd.Broomfield, CO 80021

U.S.A.

StudentGuide WithInstructorNotes

VERITAS Volume Manager4.0 Administration

ES-310

Revision D

December 19, 2003 10:15 am

Please

Recycle

Copyright 2004 Sun Microsystems Inc. 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved.

This product or document is protected by copyright and distributed under licenses restricting its use, copying, distribution, anddecompilation. No part of this product or document may be reproduced in any form by any means without prior written authorization ofSun and its licensors, if any.

Third-party software, including font technology, is copyrighted and licensed from Sun suppliers.

Sun, Sun Microsystems, the Sun logo, Solaris, OpenBoot, Ultra, Sun Blade, Sun StorEdge, Solstice DiskSuite, RSM, SunPlex, Sun Fire, Java,Sun BluePrints, Sun Enterprise, SunOS, and SunSolve are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. andother countries.

All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. in the U.S. andother countries. Products bearing SPARC trademarks are based upon an architecture developed by Sun Microsystems, Inc.

UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd.

Adobe is a registered trademark of Adobe Systems, Incorporated. PostScript is a trademark or a registered trademark of Adobe Systems,Incorporated, which may be registered in certain jurisdictions.

ORACLE is a registered trademark of Oracle Corporation.

The OPEN LOOK and Sun Graphical User Interface was developed by Sun Microsystems, Inc. for its users and licensees. Sun acknowledgesthe pioneering efforts of Xerox in researching and developing the concept of visual or graphical user interfaces for the computer industry.Sun holds a non-exclusive license from Xerox to the Xerox Graphical User Interface, which license also covers Sun’s licensees whoimplement OPEN LOOK GUIs and otherwise comply with Sun’s written license agreements.

U.S. Government approval might be required when exporting the product.

RESTRICTED RIGHTS: Use, duplication, or disclosure by the U.S. Government is subject to restrictions of FAR 52.227-14(g)(2)(6/87) andFAR 52.227-19(6/87), or DFAR 252.227-7015 (b)(6/95) and DFAR 227.7202-3(a).

DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS, ANDWARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSEOR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BELEGALLY INVALID.

THIS MANUAL IS DESIGNED TO SUPPORT AN INSTRUCTOR-LED TRAINING (ILT) COURSE AND IS INTENDED TO BEUSED FOR REFERENCE PURPOSES IN CONJUNCTION WITH THE ILT COURSE. THE MANUAL IS NOT A STANDALONETRAINING TOOL. USE OF THE MANUAL FOR SELF-STUDY WITHOUT CLASS ATTENDANCE IS NOT RECOMMENDED.

Export Control Classification Number (ECCN): 19 December 2001

Please

Recycle

Copyright 2004 Sun Microsystems Inc. 4150 Network Circle, Santa Clara, California 95054, Etats-Unis. Tous droits réservés.

Ce produit ou document est protégé par un copyright et distribué avec des licences qui en restreignent l’utilisation, la copie, la distribution,et la décompilation. Aucune partie de ce produit ou document ne peut être reproduite sous aucune forme, par quelque moyen que ce soit,sans l’autorisation préalable et écrite de Sun et de ses bailleurs de licence, s’il y en a.

Le logiciel détenu par des tiers, et qui comprend la technologie relative aux polices de caractères, est protégé par un copyright et licenciépar des fournisseurs de Sun.

Sun, Sun Microsystems, le logo Sun, Solaris, OpenBoot, Ultra, Sun Blade, Sun StorEdge, Solstice DiskSuite, RSM, SunPlex, Sun Fire, Java,Sun BluePrints, Sun Enterprise, SunOS, et SunSolve sont des marques de fabrique ou des marques déposées de Sun Microsystems, Inc. auxEtats-Unis et dans d’autres pays.

Toutes les marques SPARC sont utilisées sous licence sont des marques de fabrique ou des marques déposées de SPARC International, Inc.aux Etats-Unis et dans d’autres pays. Les produits portant les marques SPARC sont basés sur une architecture développée par SunMicrosystems, Inc.

UNIX est une marques déposée aux Etats-Unis et dans d’autres pays et licenciée exclusivement par X/Open Company, Ltd.

Adobe est une marque enregistree de Adobe Systems, Incorporated. PostScript est une marque de fabrique d‘Adobe Systems, Incorporated,laquelle pourrait être déposée dans certaines juridictions aux Etats-Unis et dans d’autres pays.

ORACLE est une marque déposée registre de Oracle Corporation.

L’interfaces d’utilisation graphique OPEN LOOK et Sun™ a été développée par Sun Microsystems, Inc. pour ses utilisateurs et licenciés.Sun reconnaît les efforts de pionniers de Xerox pour larecherche et le développement du concept des interfaces d’utilisation visuelle ougraphique pour l’industrie de l’informatique. Sun détient une licence non exclusive de Xerox sur l’interface d’utilisation graphique Xerox,cette licence couvrant également les licenciés de Sun qui mettent en place l’interface d’utilisation graphique OPEN LOOK et qui en outrese conforment aux licences écrites de Sun.

L’accord du gouvernement américain est requis avant l’exportation du produit.

LA DOCUMENTATION EST FOURNIE “EN L’ETAT” ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIESEXPRESSES OU TACITES SONT FORMELLEMENT EXCLUES, DANS LA MESURE AUTORISEE PAR LA LOI APPLICABLE, YCOMPRIS NOTAMMENT TOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNEUTILISATION PARTICULIERE OU A L’ABSENCE DE CONTREFAÇON.

CE MANUEL DE RÉFÉRENCE DOIT ÊTRE UTILISÉ DANS LE CADRE D’UN COURS DE FORMATION DIRIGÉ PAR UNINSTRUCTEUR (ILT). IL NE S’AGIT PAS D’UN OUTIL DE FORMATION INDÉPENDANT. NOUS VOUS DÉCONSEILLONS DEL’UTILISER DANS LE CADRE D’UNE AUTO-FORMATION.

ivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Table of Contents

About This Course .............................................................Preface-xivCourse Goals........................................................................ Preface-xivCourse Map............................................................................Preface-xvTopics Not Covered.............................................................Preface-xviHow Prepared Are You?....................................................Preface-xviiIntroductions ..................................................................... Preface-xviiiHow to Use Course Materials ............................................Preface-xixConventions ........................................................................... Preface-xx

Icons ............................................................................... Preface-xxTypographical Conventions ..................................... Preface-xxi

Notes to the Instructor........................................................Preface-xxii

Sun Storage Concepts .....................................................................1-1Objectives ........................................................................................... 1-1Disk Storage Administration Introduction .................................... 1-2

VxVM Software Installation .................................................... 1-2VxVM Initialization .................................................................. 1-2RAID Volume Design.............................................................. 1-3RAID Volume Creation............................................................ 1-3RAID Volume Administration................................................ 1-4

Interfaces for Sun Storage Devices .................................................. 1-5SCSI Overview........................................................................... 1-5SCSI Interface Implementation .............................................. 1-6SCSI Interface Standards.......................................................... 1-7SCSI Priority.............................................................................. 1-9SCSI Phases and the Move to Fibre Channel ........................ 1-9Fibre Channel Technology....................................................... 1-9Fibre Channel-Arbitrated Loop ........................................... 1-10Advantages of FC-AL............................................................. 1-10Fibre Channel Compared to SCSI........................................ 1-11

RAID Technology ............................................................................ 1-12Host-Based RAID (Software RAID Technology)................ 1-12Controller-Based RAID (Hardware RAID Technology) ... 1-13

v VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Disk Storage Concepts..................................................................... 1-14Hot Swapping.......................................................................... 1-14Storage Area Networking ..................................................... 1-16Multihost Storage Access...................................................... 1-21Multipath Storage Access ..................................................... 1-23

Storage Configuration Identification ............................................ 1-28Conducting Physical Inventory ............................................ 1-28Displaying Storage Configurations ...................................... 1-28Identifying Controller Addressing...................................... 1-30Identifying Device Path Components.................................. 1-31Identifying DMP Devices...................................................... 1-34

Storage Array Firmware ................................................................. 1-35Fibre Channel HBA Cards ..................................................... 1-35Verifying Fibre Channel HBA Firmware ........................... 1-36Verifying SPARCstorage Array 100 Firmware.................. 1-37Verifying Sun StorEdge A5x00 Array Firmware............... 1-38Verifying Sun StorEdge T3 Array Firmware ..................... 1-39Verifying Sun StorEdge A5x00 Disk Drive Firmware ...... 1-40Firmware Upgrade Best Practices........................................ 1-41

Exercise: Recording Your Storage Configuration ....................... 1-42Preparation............................................................................... 1-42Task 1 – Reviewing Sun Storage Features ........................... 1-42Task 2 – Identifying Host Adapter Configurations ........... 1-44Task 3 – Identifying Storage Array Configurations.......... 1-45Task 4 – Verifying Storage Interface Firmware

Revisions ............................................................................... 1-46Task 5 – Verifying Array Disk Drive Firmware

Revisions ............................................................................... 1-47Exercise Summary............................................................................ 1-48

Managing Data ................................................................................. 2-1Objectives ........................................................................................... 2-1Virtual Disk Management ................................................................ 2-2

Availability................................................................................. 2-2Performance............................................................................... 2-2Scalability .................................................................................. 2-3Maintainability .......................................................................... 2-3

RAID Technology Introduction ....................................................... 2-4Supported RAID Standards..................................................... 2-4RAID Terminology .................................................................. 2-5

RAID Level Common Features ........................................................ 2-6Concatenation – RAID 0........................................................... 2-6Striping – RAID 0 ...................................................................... 2-8Mirroring – RAID 1................................................................ 2-10Mirrored Stripe – RAID 0+1 ................................................. 2-12Mirrored Concatenation – RAID 0+1 .................................. 2-14Striped Mirror – RAID 1+0 ................................................... 2-15

viCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Concatenated Mirror – RAID 1+0........................................ 2-17Striping With Distributed Parity – RAID 5 ........................ 2-19

Exercise: Optimizing System Configurations .............................. 2-21Preparation............................................................................... 2-21Task 1 – Reviewing Software RAID Features .................... 2-22Task 2 – Identifying Availability and Performance

Cabling.................................................................................. 2-25Task 3 – Optimizing RAID-0 Volumes ............................... 2-26Task 4 – Optimizing RAID-1 Volumes ............................... 2-27Task 5 – Optimizing RAID-5 Volumes ............................... 2-28Task 6 – Optimizing RAID 0+1 Volumes ........................... 2-29Task 7 – Optimizing RAID 1+0 Volumes ........................... 2-30Task 8 – Identifying Effective Storage Utilization............. 2-31Task 9 – Selecting Disk Drives for Use................................. 2-32

Exercise Summary............................................................................ 2-33

VERITAS Volume Manager Installation ..........................................3-1Objectives ........................................................................................... 3-1Installation Planning.......................................................................... 3-2

System Downtime..................................................................... 3-2Storage Configuration Assessment ........................................ 3-2Upgrade Resources ................................................................... 3-3Licensing .................................................................................... 3-3Current System Checkpoint .................................................... 3-3Backups....................................................................................... 3-3Testing the New Configuration .............................................. 3-3

Researching Software Patches.......................................................... 3-4Researching Current Patch Information................................ 3-4Installing Patches ..................................................................... 3-6Vendor Software Patches ......................................................... 3-6

Installing VxVM Software ................................................................ 3-7Software Distribution Overview............................................. 3-7Software Package Installation ................................................ 3-9Software Installation User Interaction ................................ 3-10

Initializing VxVM Using the vxinstall Utility ......................... 3-13The vxinstall Utility Dialogue .......................................... 3-13Licensing Requirements......................................................... 3-14Verifying Licensed Features ................................................. 3-15

VxVM Post-Installation Environment........................................... 3-16VxVM System Files ................................................................. 3-16System Startup Messages...................................................... 3-18System Startup Processes...................................................... 3-19System and User Executable Files ....................................... 3-21Verifying Initial Disk Drive Status ....................................... 3-23

Preparing for VxVM Disk Drive Management............................ 3-24Disk Drive Initialization Process .......................................... 3-24Disk Drive Encapsulation Process........................................ 3-27

vii VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Protecting Storage Devices From Usage............................. 3-29Global Exclusion .................................................................... 3-32

Installing the VEA............................................................................ 3-35VEA Software Initialization.................................................. 3-36VEA Client Software Startup ............................................... 3-37Host Connection Window .................................................... 3-38Resolving Low-Bandwidth Access Problems .................... 3-39

Using Basic VEA Features .............................................................. 3-40Main Window Functional Areas........................................... 3-40Resizing Display Panes .......................................................... 3-45Modifying Preferences .......................................................... 3-46Customizing the Grid Display ............................................. 3-47Examining VEA Command Logs ......................................... 3-48Using the VEA Search Tool .................................................. 3-49

Decoding VxVM Error Messages .................................................. 3-50Exercise: Configuring VxVM.......................................................... 3-51

Preparation............................................................................... 3-51Task 1 – Reviewing Key Lecture Points.............................. 3-52Task 2 – Installing the VxVM Software .............................. 3-55Task 3 – Verifying the VxVM System Files ........................ 3-57Task 4 – Evaluating the Storage Configuration................. 3-58Task 5 – Installing the VEA Client Software ...................... 3-59Task 6 – Starting the VEA Client Software......................... 3-60Task 7 – Customizing the VEA GUI Appearance ............. 3-61Task 8 – Navigating the VxVM Technical Manuals........... 3-61Task 9 – Using the VxVM Error Numbering System........ 3-63


VERITAS Volume Manager Basic Operations ............................... 4-1Objectives ........................................................................................... 4-1VxVM Disk Group Functions........................................................... 4-2

Primary Functions of a Disk Group ....................................... 4-2VxVM Disk Drives .................................................................... 4-3Standard VxVM Disk Groups ................................................ 4-4Shared VxVM Disk Groups .................................................... 4-5Cross-Platform Data Sharing Disk Groups .......................... 4-6

VxVM Disk Group Operations ........................................................ 4-7Verifying Disk Group Status ............................................................ 4-8

Using the vxdisk Command to Verify Disk GroupStatus........................................................................................ 4-8

Using the vxdg Command to Verify Disk Group Status..... 4-8Administering Disk Groups Using the vxdiskadm Utility.......... 4-9

Functional Overview ............................................................. 4-10Creating a New Disk Group................................................. 4-11Removing a Disk Drive From a Disk Group...................... 4-12

viiiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Administering Disk Groups Using Command-LinePrograms ........................................................................................... 4-13

Using the vxdiskunsetup Command................................. 4-13Initializing Disk Drives ......................................................... 4-14Using the vxdg Command..................................................... 4-14Adding and Removing Disk Drives.................................... 4-16Importing and Deporting Disk Groups .............................. 4-17Destroying a Disk Group...................................................... 4-19Renaming VxVM Disk Drives............................................... 4-19

Administering Disk Groups Using the VEA GUI ....................... 4-20Creating a New Disk Group.................................................. 4-20Adding and Removing Disk Drives.................................... 4-22Deporting Disk Groups......................................................... 4-24Importing Disk Groups......................................................... 4-25Destroying a Disk Group...................................................... 4-26Renaming VxVM Disk Drives.............................................. 4-27Displaying VEA Object Properties ...................................... 4-28

Exercise: Performing VxVM Basic Operations ............................ 4-29Preparation.............................................................................. 4-30Task 1 – Reviewing Key Lecture Points.............................. 4-31Task 2 – Verifying the VxVM Environment....................... 4-32Task 3 – Verifying the Initial Disk Drive Status ................ 4-33Task 4 – Setting the Default Disk Drive Format ................ 4-34Task 5 – Initializing Disk Drives.......................................... 4-35Task 6 – Creating New Disk Groups.................................... 4-36Task 7 – Viewing Command Logs....................................... 4-38Task 8 – Importing and Deporting Disk Groups................ 4-38Task 9 – Destroying a Disk Group....................................... 4-40Task 10 – Renaming Disk Drives .......................................... 4-40Task 11 – Using the vxdiskadm Utility to Perform Basic

Operations (Optional) ......................................................... 4-41Task 12 – Verifying Ending Lab Status............................... 4-42


VERITAS Volume Manager Volume Operations ............................5-1Objectives ........................................................................................... 5-1Interpreting Volume Structure Listings.......................................... 5-2

Subdisks...................................................................................... 5-2Plexes .......................................................................................... 5-3Volumes...................................................................................... 5-4Volume Structure Examples................................................... 5-5

Volume Planning ............................................................................... 5-6Volume Distribution................................................................. 5-6Volume Naming Conventions ............................................... 5-8Space Allocation Planning ....................................................... 5-8Selecting Volume Types......................................................... 5-11

ix VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Creating Volumes Using the VEA GUI ........................................ 5-13Disk Selection Method............................................................ 5-13Using the Disk Selection Form............................................. 5-14Using the Volume Attributes Form..................................... 5-15Using the Create File System Form ..................................... 5-16

Creating Volumes Using the vxassist Command.................... 5-17The vxassist Command ...................................................... 5-17Specifying Volume Size.......................................................... 5-17Using vxassist Command Options .................................. 5-18

Modifying Volume Access Attributes........................................... 5-20Verifying Volume Ownership............................................... 5-20Modifying Volume Ownership and Permissions............... 5-20

Adding a UFS File System to Existing Volumes ......................... 5-21Using the VEA GUI to Add a File System........................... 5-21Adding a File System From the Command Line............... 5-23Enabling the Solaris OS UFS Logging Feature ................... 5-24

Administering Volume Logs.......................................................... 5-25Using DRLs .............................................................................. 5-25Using RAID-5 Logs................................................................ 5-26Planning Log Placement ....................................................... 5-27Adding a Volume Log From the VEA GUI......................... 5-28Adding a Volume Log From the Command Line ............. 5-29Removing Volume Logs Using the VEA GUI.................... 5-30Removing Volume Logs From the Command Line.......... 5-31

Using the VEA GUI to Analyze Volume Structures ................... 5-32Displaying Volume Layout Details ...................................... 5-32Viewing Disk Volume Mapping and Performance........... 5-33

Exercise: Creating Volumes and File Systems ............................. 5-34Preparation............................................................................... 5-34Task 1 – Reviewing Key Lecture Points.............................. 5-35Task 2 – Creating a Volume.................................................. 5-37Task 3 – Adding a Volume Mirror ...................................... 5-39Task 4 – Adding a File System to a Volume........................ 5-41Task 5 – Adding a DRL ......................................................... 5-43Task 6 – Resizing a Volume and File System..................... 5-45Task 7 – Creating a RAID-5 Volume ................................... 5-47Task 8 – Analyzing Volumes Using the VEA GUI............ 5-49Task 9 – Verifying Ending Lab Status................................. 5-54


VERITAS Volume Manager Advanced Operations ....................... 6-1Objectives ........................................................................................... 6-1Boot Disk Encapsulation and Mirroring......................................... 6-2

Optimizing the Boot Disk Hardware Configuration ........... 6-2Boot Disk Encapsulation Prerequisites ................................. 6-3Encapsulating the System Boot Disk..................................... 6-4

xCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Mirroring the System Boot Disk ............................................ 6-5Verifying the Completed Boot Disk Configuration ............ 6-6

Creating a Best Practice Boot Disk Configuration......................... 6-8Establishing Boot Disk Configuration Policies .................. 6-10Modifying an Existing Boot Disk Configuration............... 6-11

Administering Hot Devices............................................................ 6-13Selecting Hot Device Operational Mode ............................. 6-13Evaluating Hot-Device Configurations .............................. 6-16Administering Hot Devices Using Command-Line

Programs .............................................................................. 6-17Administering Hot Devices Using the VEA GUI.............. 6-18Controlling Relocation Recovery Time............................... 6-19

Evacuating a Disk Drive ................................................................. 6-20Identifying Evacuation Conflicts .......................................... 6-20Preparing for Evacuation....................................................... 6-20Identifying Suitable Evacuation Disk Drives..................... 6-21Evacuation Using the vxevac Command .......................... 6-22Evacuation Using the VEA GUI........................................... 6-23Evacuation Using the vxdiskadm Utility ........................... 6-24

Moving Disk Drives Without Preserving Data ........................... 6-25Moving a Disk Drive Using the Command Line................ 6-25Moving a Disk Drive Using the VEA GUI ......................... 6-26

Moving Populated Disk Drives to a New Disk Group............... 6-27Evaluating Disk Drive Involvement .................................... 6-27Saving the Configuration....................................................... 6-28Moving the Disk Drives to a New Disk Group .................. 6-29Reloading the Volume Configuration.................................. 6-30

Backing Up and Restoring Disk Group Configurations ............ 6-31Automatic Configuration Backup ........................................ 6-31Manual Configuration Backup and Restore ...................... 6-33

Importing Disk Groups After a System Crash ............................ 6-36Importing the rootdg Disk Group After a Crash ............. 6-37

Volume Snapshot Operations ........................................................ 6-38Snapshot Process..................................................................... 6-38Using the VEA GUI to Create a Snapshot .......................... 6-39Using the Command Line to Create a Snapshot ............... 6-43

Online Volume Relayout ................................................................ 6-44Volume Relayout Prerequisites............................................. 6-44Volume Relayout Using the Command Line..................... 6-45Volume Relayout Using the VEA GUI................................ 6-47

Creating Layered Volumes............................................................. 6-49Layered Volume Disk Requirements ................................... 6-49Evaluating Available Disk Space......................................... 6-50Creating Layered Volumes From the Command Line ..... 6-51Creating Layered Volumes Using the VEA GUI............... 6-52Identifying Layered Volume Subcomponents................... 6-53

xi VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Basic Intelligent Storage Provisioning Administration.............. 6-54Primary ISP Components ...................................................... 6-54Using Storage Pool Set Templates ....................................... 6-56Using Storage Pool Templates ............................................. 6-58Using Application Volume Templates ............................... 6-60Creating Application Volumes Using the vxvoladm

Command............................................................................. 6-62Creating Application Volumes Using the VEA GUI ........ 6-63Interpreting Application Volume Configurations ............ 6-65

Replacing Failed Disk Drives ......................................................... 6-66Failure Behavior ...................................................................... 6-66Evaluating Failure Severity ................................................... 6-67General Disk Drive Replacement Process .......................... 6-70

Exercise: Performing Advanced Operations................................ 6-72Preparation............................................................................... 6-72Task 1 – Reviewing Key Lecture Points.............................. 6-73Task 2 – Encapsulating the System Boot Disk ................... 6-76Task 3 – Mirroring the System Boot Disk............................ 6-78Task 4 – Performing an Online Volume Relayout............. 6-80Task 5 – Evacuating a Disk Drive ........................................ 6-82Task 6 – Moving a Populated Volume................................. 6-82Task 7 – Performing a Snapshot Backup ............................ 6-84Task 8 – Creating a Layered Volume .................................. 6-85Task 9 – Replacing a Failed Disk Drive .............................. 6-86Task 10 – Using Intelligent Storage Provisioning .............. 6-88Task 11 – Configuring a Best Practice Boot Disk............... 6-91


VERITAS File System Basic Operations........................................ 7-1Objectives ........................................................................................... 7-1Basic VxFS Features ........................................................................... 7-2

Extent-Based Space Allocation................................................ 7-2File System Intent Logging..................................................... 7-3

Installing the VxFS Software ............................................................ 7-4Creating VxFS File Systems.............................................................. 7-5Extended VxFS Mount Options ....................................................... 7-6

Intent Log Behavior .................................................................. 7-6Error Handling Behavior ......................................................... 7-7Other VxFS Mount Options..................................................... 7-7

Online File System Administration................................................. 7-8Online Defragmentation .......................................................... 7-8Online Resizing ......................................................................... 7-8Online Backup and Restore ..................................................... 7-8

xiiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Exercise: Configuring VxFS.............................................................. 7-9Preparation................................................................................. 7-9Task 1 – Reviewing Key Lecture Points.............................. 7-10Task 2 – Installing the VxFS Software................................. 7-11Task 3 – Creating a VxFS File System ................................. 7-13Task 4 – Resizing a VxFS File System .................................. 7-14Task 5 – Defragmenting a VxFS File System....................... 7-15Task 6 – Backing Up and Restoring a VxFS File System ... 7-16Task 7 – Using VxFS Extended Mount Options ................. 7-17


VERITAS Volume Manager Performance Management ................8-1Objectives ........................................................................................... 8-1Performance Improvement Techniques ......................................... 8-2

Data Assignment Strategies..................................................... 8-2Volume Structure Strategies.................................................... 8-4Read Policy Strategies ............................................................. 8-6Hardware Configuration Strategies ....................................... 8-7

Using Performance Analysis Tools ................................................. 8-8Gathering Volume Performance Statistics ............................ 8-8Gathering Application Performance Statistics................... 8-11

RAID-5 Write Performance ............................................................ 8-12Read-Modify-Write Operations ............................................ 8-12Reconstruct-Write Operations............................................... 8-13Full-Stripe Write Operations ................................................. 8-14

Exercise: Demonstrating Performance Differences..................... 8-15Preparation............................................................................... 8-15Task 1 – Reviewing Key Lecture Points.............................. 8-16Task 2 – Performing a RAID-5 Write Performance Test .. 8-18Task 3 – Performing a Striped Volume Write Performance

Test ........................................................................................ 8-21Exercise Summary............................................................................ 8-22

Preface-xivCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Preface

About This Course

Course Goals

Upon completion of this course, you should be able to:

● Install and initialize VERITAS Volume Manager (VxVM) software

● Define VxVM objects

● Describe public and private regions

● Start and customize the Volume Manager Storage AdministratorVERITAS Enterprise Administrator (VEA) graphical user interface(GUI)

● Perform operations using the command-line interface

● Perform disk and volume operations

● Create redundant array of independent disk (RAID) volumes

● Set up dirty-region logs (DRLs)

● Perform common file system operations using the VEA GUI

● Create new disk groups, remove disks from group, move disksbetween disk groups, and deport and import disk groups betweenservers

● Simulate disk failure and complete a disk recovery

● Create and mange hot-spare pools

● Manage and disable the hot-relocation feature

● Perform basic performance analysis

Course Map

Preface-xv VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Course Map

The following course map enables you to see what you haveaccomplished and where you are going in reference to the course goals.

Overview

Sun StorageConcepts

Managing Data

VERITAS Volume Manager Introduction

VERITAS VolumeManager

Installation

VERITAS VolumeManager Basic

Operations

VERITAS Volume Manager Operations

VERITASVolume

ManagerVolume

Operations

VERITASVolume Manager

AdvancedOperations

VERITASFile System

BasicOperations

VERITAS File System Operations

VERITASVolume Manager

PerformanceManagement

Topics Not Covered

About This Course Preface-xviCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Topics Not Covered

This course does not cover the following topics. Many of these topics arecovered in other courses offered by Sun Educational Services:

● Solaris™ Operating System (Solaris OS) administration – Covered inSA-239: Intermediate System Administration for the Solaris™ 9 OperatingEnvironment and SA-299: Advanced System Administration for theSolaris™ 9 Operating Environment

● Storage system maintenance – Covered in SM-250: Sun™ SoftwareRAID Storage Systems Maintenance

Refer to the Sun Educational Services catalog for specific information andregistration.

How Prepared Are You?

Preface-xvii VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

How Prepared Are You?

To be sure you are prepared to take this course, can you answer yes to thefollowing questions?

● Can you perform the following Solaris 9 OS administrationoperations?

● Configure environmental variables

● Control system run levels

● Use basic OpenBoot™ commands

● Can you physically configure the following Sun systems andperipherals?

● Desktop or server-class systems

● Sun storage arrays

● Can you use the following UNIX® administrative commands?

● chmod, chown, and chgrp

● format

● man

● mkdir

● mkfs /newfs

● mount

● patchadd

● pkgadd and pkgrm

● vi

Introductions

About This Course Preface-xviiiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Introductions

Now that you have been introduced to the course, introduce yourself tothe other students and the instructor, addressing the following items:

● Name

● Company affiliation

● Title, function, and job responsibility

● Experience related to topics presented in this course

● Reasons for enrolling in this course

● Expectations for this course

How to Use Course Materials

Preface-xix VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

How to Use Course Materials

To enable you to succeed in this course, these course materials employ alearning module that is composed of the following components:

● Goals – You should be able to accomplish the course goals afterfinishing this course and meeting all of its objectives.

● Objectives – You should be able to accomplish the objectives aftercompleting a portion of instructional content. Objectives supportgoals and can support other higher-level objectives.

● Lecture – The instructor presents information specific to theobjectives of the module. This information helps you learn theknowledge and skills necessary to succeed with the activities.

● Activities – The activities take on various forms, such as exercises,self-checks, discussions, and demonstrations. Activities are used tofacilitate mastery of an objective.

● Visual aids – The instructor might use several visual aids to convey aconcept, such as a process, in a visual form. Visual aids commonlycontain graphics, animation, and video.

Conventions

About This Course Preface-xxCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Conventions

The following conventions are used in this course to represent varioustraining elements and alternative learning resources.

Icons

?!

Discussion – Indicates a small-group or class discussion on the currenttopic is recommended at this time.

Note – Indicates additional information that can help students but is notcrucial to their understanding of the concept being described. Studentsshould be able to understand the concept or complete the task withoutthis information. Examples of notational information include keywordshortcuts and minor system adjustments.

Caution – Indicates that there is a risk of personal injury from anonelectrical hazard, or risk of irreversible damage to data, software, orthe operating system. A caution indicates that the possibility of a hazard(as opposed to certainty) might happen, depending on the action of theuser.

Conventions

Preface-xxi VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Typographical Conventions

Courier is used for the names of commands, files, directories,programming code, and on-screen computer output; for example:

Use ls -al to list all files.system% You have mail .

Courier bold is used for characters and numbers that you type; forexample:

To list the files in this directory, type:# ls

Courier italics is used for variables and command-line placeholdersthat are replaced with a real name or value; for example:

To delete a file, use the rm filename command.

Courier italics bold is used to represent variables whose values areto be entered by the student as part of an activity; for example:

Type chmod a+rwx filename to grant read, write, and executerights for filename to world, group, and users.

Palatino italics is used for book titles, new words or terms, or words thatyou want to emphasize; for example:

Read Chapter 6 in the User’s Guide.These are called class options.

Notes to the Instructor

About This Course Preface-xxiiCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Notes to the Instructor

For classroom setup suggestions, refer to the ES310_revC_setup.txt file.

The ES310_revC_setup.txt file contains specific setup instructions about:

● General equipment configuration for this course

● Patch download instructions

● VxVM requirements and download instructions

● Supported hardware

● Required lab files

1-1Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Module 1

Sun Storage Concepts

Objectives

Upon completion of this module, you should be able to:

● Describe the major disk storage administration tasks

● Describe Sun storage interface types

● Describe available RAID technologies including:

● Host-based RAID technology

● Controller-based RAID technology

● Describe disk storage concepts that are common to many storageinstallations including:

● Hot swapping

● Storage area networking

● Multihost access

● Multipath access

● Identify storage configurations including:

● Conducting physical inventory

● Displaying storage configurations

● Identifying controller addresses

● Decoding logical device paths

● Verify storage array firmware revisions

Disk Storage Administration Introduction

1-2 VERITAS Volume Manager 4.0 AdministrationCopyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D


Installing, configuring, and managing a VxVM installation requires bothsoftware and hardware knowledge.

VxVM Software Installation

Installing the VxVM software is essentially the same as installing anySolaris OS application. You can use either the standard command-line orGUI installation tools.

VxVM Initialization

When you install VxVM, at least one disk drive must be brought underVxVM control using the vxinstall utility. You can either encapsulate adisk, which preserves existing data on the disk, or you can initialize a disk,which effectively destroys existing data.

Required Hardware Knowledge

The VxVM installation process is always the same, regardless of thesystem platform or storage technology used. However, you must be ableto identify storage array device addresses and differentiate them fromother types of disk storage addresses.

If you are not familiar with the device address strategy in your particularinstallation, you might accidentally initialize the wrong disk drives. Thiserror could destroy valuable data, including the operating system.


Sun Storage Concepts 1-3Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

RAID Volume Design

Generally, RAID volume structures are designed with one or more of thefollowing goals in mind:

● Cost savings

● Performance

● Availability

● Maintainability

In most cases, compromises are made when choosing among cost savings,performance, availability, and maintainability.


You must have a thorough understanding of interface types, addressingschemes, and internal hardware structure to achieve design goals. It ispossible to design virtual volume structures without this backgroundknowledge, but the result might perform poorly and might not have thereliability that is required for your application.

RAID Volume Creation

You can create RAID volume structures using the VxVM GUI orcommand-line programs. Command-line programs are sometimes a betterchoice to use, especially when volume creation must be automated usingscript files.

You can configure the GUI to display command-line equivalents for eachoperation.


Even though you might not be responsible for the design of your VxVMvolume structures, you must still be familiar with most aspects of yourparticular storage devices.



RAID Volume Administration

In larger installations, the most common VxVM administrative task isidentifying and replacing failed disk drives. At the most basic level, thistask involves the use of a single VxVM utility, the vxdiskadm utility. Forcertain storage platforms, you must use another utility, luxadm , duringthe disk replacement process.

Most VxVM administrative tasks require analyzing error messages usingVxVM utilities, such as vxprint and vxdisk , along with some basicSolaris OS commands.


Administering RAID volumes requires a number of hardware-relatedskills including:

● Decoding device error messages

● Relating device addresses to physical devices

● Following hardware removal procedures that are appropriate foreach particular disk storage technology

Interfaces for Sun Storage Devices



Sun storage devices are accessed through one of two basic interface types:

● Small Computer System Interface (SCSI)

● Fiber-optic

Each of the basic interface types has two or more variations, which haveevolved over a period of several years. The interfaces have improved inthe following areas:

● Data transfer speed

● Data transfer latency

● Interface cable lengths

SCSI Overview

SCSI was initially implemented in the 1980s as a way of making theinterface between the host computer system and the disks independent ofthe computer manufacturer. Prior to the introduction of SCSI, all thecomputer manufacturers had their own way of connecting the hostcomputer system to the disk drives.

SCSI introduced the idea of intelligent disk drives where the hostcomputer system requested the transfer of a block of data from the disk.The host system had no need to know the underlying disk geometry. Itissued a request to the disk for the transfer of a block of data. The shift ofintelligence from the host computer system to the disk allowed the samedisk to be used by different manufacturers, which ultimately led tocheaper, faster, and larger disk drives.

The connection between the host system was by the SCSI bus, for which aset of standards was agreed upon. The speed and data capacity of theSCSI bus has been increased to allow for the higher demands of today’sservers. One of the earliest problems faced with SCSI was the differingcable lengths from the host system to the disk drives themselves. For theSCSI bus to reliably operate over differing cable lengths, two electricalconnections methods were defined: Single-ended (for short connectionlengths) and differential (for connection over longer cables).



SCSI Interface Implementation

Both single-ended SCSI and differential SCSI are currently used with Sunstorage products.

Single-Ended SCSI

As shown in Figure 1-1, each bit is transmitted using one signal,referenced to 0 volts. This allows cable lengths up to six meters withstandard SCSI-1 devices.

Figure 1-1 Single-Ended SCSI

Differential SCSI

As shown in Figure 1-2, the data bits are sent using two equal andopposite voltages. These allow the signal to travel farther withoutdegradation. Differential SCSI allows cable lengths up to 25 meters.

Figure 1-2 Differential SCSI

inv add



SCSI Interface Standards

A partial list of the SCSI-1, SCSI-2, and SCSI-3 standards currently usedby Sun are shown in Table 1-1. Other SCSI standards are used by differentmanufacturers.

Table 1-1 Sun SCSI Standards

Name(Type)

ClockSpeed

DataWidth

Data Rate(CommandRate)

NumberofDevices

SCSI(SCSI-1)

5 MHz 8 bits 5 Mbytes/sec 8

Fast SCSI(SCSI-2)


Wide SCSI(SCSI-2)


Fast/WideSCSI(SCSI-2)


UltraDifferentialWide SCSI(SCSI-3)




SCSI-3 Standards

Currently, only recent Sun systems such as the Ultra™ 60 workstation,Ultra 80 workstation, and the Sun Blade™ 1000 workstations, have SCSI-3compatible interfaces. The related specifications are:

● Ultra 60 workstation – 68-pin connector, 40-Mbyte/sec Ultra SCSI(SCSI-3), two channels (synchronous)

● Ultra 80 workstation – 68-pin connector, 40-Mbyte/sec Ultra SCSI(SCSI-3), two channels (synchronous)

● Sun Blade 1000 workstation – One single-ended 68-pin SCSIconnector (16-bit Ultra/Fast/Wide, 40-Mbyte/sec), internal disks areFibre Channel

Table 1-2 shows some of the current SCSI-3 standards.

Table 1-2 SCSI-3 Standards

Type Speed WidthSingle-EndedLength

HighVoltageDifferentialLength

LowVoltageDifferentialLength

Targets

SCSI 3U 20 MB/sec 8 bits 1.5 m 25 m – 8

SCSI 3 U 20 MB/sec 8 bits 3 m – – 4

SCSI 3 FW 20 MB/sec 16 bits 6 m 25 m – 16

SCSI 3 UW 40 MB/sec 16 bits – 25 m – 16

SCSI 3 UW 40 MB/sec 16 bits 1.5 m – – 8

SCSI 3 UW 40 MB/sec 16 bits 3 m – – 4

SCSI 3 U2 40 MB/sec 8 bits – 12 m 12 m 8

SCSI 3 U2 80 MB/sec 16 bits – 25 m 25 m 2

SCSI 3 U2W 80 MB/sec 16 bits – 12 m 12 m 16

SCSI 3 U2W 80 MB/sec 16 bits – 25 m 25 m 2



SCSI Priority

The bus arbitration mechanism for SCSI uses the SCSI target ID todetermine priority. Narrow SCSI has target addresses 0–7. Target 7 hashighest priority (usually the ID of the controller), and target 8 has thelowest. Performance can be affected through injudicious use of SCSI targetaddresses.

Wide SCSI uses target addresses 0–15 in the following priority: 7, 6, 5, 4, 3,2, 1, 0, 15, 14, 13, 12, 11, 10, 9, 8. Address 7 is the highest priority.

SCSI Phases and the Move to Fibre Channel

Although it appears as if SCSI has improved performance eight-fold, thisis an exaggeration. SCSI has six phases, one of which is data transfer to orfrom the disk. Other phases include sense (status), command transfer, andbus arbitration. SCSI supports backward compatibility, so a narrow, slowdisk can run off a fast-wide controller. This capability is achieved byhaving the administration of SCSI run at 5 megahertz (MHz) on narrowcabling. Each device is allowed to negotiate upwards. That is, to be fast,ultra fast, wide, and so on. The data transfer phase benefits from theimproved clock speed, but command transfer does not. It takes about5 milliseconds (ms) on each form of SCSI.

The last 10 years have witnessed a 10,000-fold increase in computerperformance. At the same time, requirements are increasing for morerobust, highly available, disaster-tolerant computing resources. Computerresources continue to be pushed to their limits.

Fibre Channel Technology

Fibre Channel technology is the answer to the growing problems ofSCSI-based peripherals. Fibre Channel is a high-performance serialinterconnect standard designed for bidirectional, point-to-pointcommunications between servers, storage systems, workstations,switches, and hubs. It offers a variety of benefits over other link-levelprotocols, including efficiency, high performance, scalability, simplicity,ease of use and installation, and support for popular high-level protocols.Fibre Channel also offers a higher level of security and reliability throughelectrical isolation. There is no electromagnetic (EM) influence.



Fibre Channel-Arbitrated Loop

An important enhancement to Fibre Channel technology has been thedevelopment of Fibre Channel-Arbitrated Loop (FC-AL), which wasdeveloped specifically to meet the needs of storage interconnects. Using aloop topology, FC-AL can support basic configurations and sophisticatedarrangements of hubs, switches, servers, and storage systems.Furthermore, by using SCSI protocols over the much faster, more robustFibre Channel link, FC-AL provides higher levels of performance withoutrequiring expensive and complex changes to existing device drivers andfirmware.

Advantages of FC-AL

The FC-AL development effort is part of the American National StandardsInstitute/International Organization for Standardization (ANSI/ISO)accredited SCSI-3 standard. This standard helps to prevent the creation ofnon-conforming, incompatible implementations. Virtually all majorsystem vendors are implementing FC-AL, as are all major disk drive andstorage system vendors.

FC-AL operates on both fiber-optic cable and copper wire, and it can beused for more than just disk input/output (I/O). The Fibre Channelspecification supports high-speed system and network interconnectsusing a wide variety of popular protocols, including:

● SCSI

● Internet Protocol (IP)

● Adaptation Layer for Computer Data (AAL5) (ATM)

● Fibre Channel Link Encapsulation (FC-LE)

● Institute of Electrical and Electronics Engineers specification for datalink layer transmission (IEEE 802.2)



Fibre Channel Compared to SCSI

As shown in Table 1-3, Fibre Channel technology has many advantagescompared to SCSI.

Table 1-3 Fibre Channel Compared to SCSI

Fibre Channel SCSI

Full duplex operation Half-duplex operation

100 Mbytes/sec 40 Mbytes/sec

Performance not affected bydisk position.

Performance can be negativelyaffected by the physical diskposition.

Multipath disk access issupported.

A disk can connect to only oneSCSI channel.

Up to four hosts can connect toan appropriate array.

A maximum of two hosts canconnect to a single SCSIchannel.

Serial data transmission Parallel data transmission

Suitable for use in a SAN. Itallows the use of switches andhubs.

Unsuitable for use in a SAN. Itallows no use of switches andhubs.

Can connect 30 km (25 MHz)/10 km (100 MHz) maximumdistances.

Maximum distance is 25 meterswith a differential SCSI.

RAID Technology


RAID Technology

RAID virtual data structures can be created and managed by softwareapplications, or they can be a resident-hardware function of some storagedevices.

Host-Based RAID (Software RAID Technology)

VxVM is a good example of software RAID technology. All Sun storagearrays can be used with VxVM. As shown in Figure 1-3, user applicationsaccess a virtual structure through a single path that is composed of threeseparate disk drives.

Figure 1-3 Host-Based RAID Technology

A typical virtual volume path name would be similar to the following:

/dev/vx/dsk/dga/volume-01

Although the physical paths to the three disk drives in Figure 1-3 stillexist, they are not accessed directly by users or applications. Only thevirtual volume paths are referenced by users.

Software that runs on the host system creates and manages the virtualsoftware.

Storage Array

VM Software

User or ApplicationAccess

T1 T2 T3

Controller c4

1-Gbytephysicaldisks

3-Gbytevirtual volume

RAID Technology


Controller-Based RAID (Hardware RAID Technology)

Controller-based RAID solutions use firmware that runs internally onstorage array logic boards to maintain virtual structures that arecomposed of one or more physical disk drives. Sun hardware RAIDmodels include: Sun StorEdge™ A1000, Sun StorEdge A3000, SunStorEdge A3500, Sun StorEdge A3500FC, and Sun StorEdge T3 arrays.

As shown in Figure 1-4, RAID configuration software (Sun StorEdgeRAID Manager) running on the host system configures virtual structuresin the array controller board. After initial configuration, the controllerboard firmware manages the virtual structures.

Figure 1-4 Controller-Based RAID Technology

Note – The Sun StorEdge T3 array RAID structures are configured usingeither the Sun StorEdge Component Manager software or resident storagearray operating system commands.

A typical hardware RAID device appears to be the same as any logicaldevice path, such as /dev/dsk/c0t5d0s0 . Applications do not recognizethe underlying RAID structures.

Hardware RAID solutions can have better performance than host-basedRAID solutions for some types of RAID structures. Hardware RAIDoverhead calculations are performed at very high speeds by thecontroller-resident hardware, instead of on the host system as inhost-based RAID. Hardware-based RAID-5 write performance can bemuch better than host-based RAID 5.

Host System Storage Array

Host BusAdapter

RAID ConfigurationSoftware

RAID Firmware

UserAccess

Disk

DiskDiskDisk

Disk

DiskDiskDisk

Disk Storage Concepts



This section describes the following disk storage concepts that arecommon to many storage installations:

● Hot swapping

● Storage area networking

● Multihost access

● Multipath access

Hot Swapping

Most Sun storage arrays are engineered so that a failed disk drive can bereplaced without interrupting customer applications. Thedisk-replacement process includes one or more software operations thatcan vary with each disk storage platform.

General VxVM Disk-Replacement Procedure

In its basic form, the process to replace a failed disk drive that is underVxVM control is as follows:

1. Use the VxVM vxdiskadm utility to logically remove the disk.

2. Use the VxVM vxdiskadm utility to logically install the new disk.

Caution – Never hot-swap a disk that is under control of VxVM. Alwaysuse the vxdiskadm utility or command-line equivalents.



Alternative Disk-Replacement Procedure

The VxVM disk replacement process is more complex for some storagearrays, such as the Sun StorEdge A5x00 array. The Sun StorEdge A5x00array procedure is as follows:

1. Use the VxVM vxdiskadm utility options 4 and 11 to logicallyremove the disk and place it offline.

2. Use the luxadm utility’s remove_device command.

3. Use the luxadm utility’s insert_device command.

4. Run the vxdctl enable command to read in the new configuration.

5. Use the VxVM vxdiskadm utility option 5 to logically install the newdisk.

. Note – You must be familiar with the disk-replacement process for yourparticular disk storage devices.



Storage Area Networking

The current Sun storage area network (SAN) implementation is composedof the following hardware and software:

● Supported arrays are Sun StorEdge A3500, Sun StorEdge, A5200, SunStorEdge T3, and Sun StorEdge T3+ arrays

● Sun Fibre Channel switches (8 and 16 ports)

● Management software (fabric and switch management)

SANs are usually based on the Fibre Channel data communicationsstandard.

Basic SAN Configuration

Traditionally, data is made available through Network File System (NFS)mounts over Ethernet user networks. This configuration is referred to asnetwork-attached storage (NAS). User networks can be overloaded by heavydata traffic. A SAN is a Fibre Channel network that off-loads data trafficfrom the user network. As shown in Figure 1-5, a SAN is a high-performance network composed of servers, storage devices, andinterconnect devices, such as switches, hubs, cables, and converters.

Figure 1-5 SAN Components

A SAN can enable gigabit-speed data transfer with high-availability (nosingle-point-of-failure). Redundant SAN switches can be configured tohave automatic failover capability. Data is automatically re-routedthrough a redundant switch.

Switches

Sun StorEdge T3 Array Partner Pair

Host Adapter

Host Adapter

TL

TL

Fiber-optic cables



SAN Definitions

The SAN fabric. is composed of several elements. Without a basicunderstanding, the terminology can be confusing or meaningless.

SAN Fabric

Networks that use Fibre Channel switches are referred to as fabrics. Theterm fabric characterizes a network of multiple switches as opposed to anetwork with a single switch. Each connection in a fabric can use the full100-megabytes per second (Mbyte/sec) Fibre Channel bandwidth.

Switches in a fabric use a routing technique known as cut-throughswitching. Cut-through switching refers to the process by which the FibreChannel switch can route the incoming data frame almost immediately byreading its link-level destination ID (D_ID). It does not need to read theentire frame prior to transmitting it to its target.

Fibre Channel Frames

The data elements referred to as data packets in Transmission ControlProtocol/Internet Protocol (TCP/IP) are referred to as frames in FibreChannel. The basic Fibre Channel frame structure contains a 24-byteheader followed by up to 2112 bytes of data. A key portion of the 24-byteheader is a 3-byte link-level D_ID that defines the port address to whichthe data frame must be transmitted.

Fibre Channel Devices

There are two types of Fibre Channel devices, public and private.

A Fibre Channel private loop is a traditional storage configuration, suchas one or more Sun StorEdge A5200 arrays, connected to a host systemeither singly, in a daisy-chain configuration, or through a Fibre Channelhub. The devices are available on the loop only to the physicallyconnected host.

Private devices do not have full Fibre Channel addressing capability. Theyhave only the Arbitrated Loop Physical Address (ALPA) portion of theFibre Channel physical address. These devices exist only on loops, and,unless the Switch offers extra support, these devices cannot communicateoutside their own loop.



Public devices have full Fibre Channel addressing capability and,therefore, can communicate with any other public device on the fabric.They can be connected directly to the switch (one device per port) orarranged in an arbitrated loop with up to 127 nodes in the loop and theloop connected to the switch.

Switch Port Functions

You can configure the Fibre Channel switch ports to function in severalways using switch management software. The primary reason fordifferent port functionality is to allow selective access between FibreChannel devices and host systems. You should only use the followingport configurations:

● Fabric port (F_Port) – A fabric port connects a Fibre Channel switchto a fabric-aware node port (or N_Port) on an end-device.

● Segmented loop port (SL_Port) – Segmented loop ports providesupport for private arbitrated loops on a Fibre Channel switch. Allsegmented loop ports in the same SL zone behave as one privatearbitrated loop (and so they share the same ALPA space).

● Translated loop port (TL_Port) – Translated loop ports providesupport for public and private loop devices on a Fibre Channelswitch. Translated loop ports translate between private and publicaddresses, allowing public devices and private devices tocommunicate with one another.

● Trunk port (T_Port) – A trunk port connects a Fibre Channel switchto another Fibre Channel switch (this is known as cascading).

Zones

Zoning refers to the deliberate segregation of SAN resources from otherSAN resources. Zones essentially create sub-networks that providedifferent levels of connectivity or addressability between different hostsand devices on the network. Routing tables control the access of hosts todevices. You can individually configure device ports to be accessible orinaccessible to other specific ports.



Sample SAN Configuration

The example in Figure 1-6 shows how a multihost SAN configurationmight be arranged to support both private and public use of storageresources.

Figure 1-6 Sample SAN Multihost Configuration

Note – To take advantage of the multipath connections from each hostsystem, you must use a multipath product, such as VERITAS DMP or theSun StorEdge Traffic Manager software.

Sun StorEdge L180 or Sun StorEdge L700 FC Tape Library

Sun StorEdgeA5200 Array

1

3

5

7

2

4

6

8

Switch 0

Zone

Zone

Zone

1

3

5

7

2

4

6

8

Switch 1

Zone

Zone

Zone

HostAdapter

HostAdapter

HostAdapter

HostAdapter

Sun Enterprise 420 Server

Sun Enterprise 3500 Server

Sun StorEdge T3Array Partner Pairs

TM

TM

TM

TM



SAN Device Addressing

Traditionally, the Solaris OS has named storage devices based on thecontroller, target ID, and logical unit number of the device. When largeSANs with multiple paths to large storage arrays are constructed, thisnaming convention could lead to thousands of targets per controller andstorage identified more than once by different controllers and targets. Thenew method incorporates the World Wide Name (WWN) of the deviceinto the device name used by the host. The new names provide the benefitof uniquely identifying storage devices to the host. When usingSun StorEdge Traffic Manager software, a device with multipleconnections to a host is known to that host by one name.

Old device path:

/devices/pci@f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@3,0

New device path:

/devices/pci@f,4000/pci@4/SUNW,qlc@4/fp@0,0/ssd@w50020f200000225,0

Old symbolic device name:

/dev/dsk/c4t3d0s2

New symbolic designation:

/dev/dsk/c4t50020f200000225d0s2

The number of storage devices that can be attached to a host can grow tothe thousands with the advent of SANs with native fabric connectivity.Probing all these devices at boot time and creating device nodes canincrease the boot time greatly. In addition, a host might not need access toall of the storage devices it can access. The Sun StorEdge Network FCSwitch-16, Version 3.0, no longer creates device nodes for every storagedevice attached. Instead, the administrator creates device nodes ondemand by using the cfgadm utility. The device nodes, once created, arepersistent across reboots.



Multihost Storage Access

With the advent of technology, such as the Sun StorEdge A5200 array, asmany as four different hosts can be connected to the same storage device.Multihost connections are also possible on some versions of the SCSI.

Multi-Initiated SCSI

Sun StorEdge MultiPack storage devices support physical SCSI interfaceconnections from two different host systems. The SCSI interface on eachof the systems must have a different initiator identifier (ID) setting, whichis a system firmware configuration known as the scsi-initiator-id .

As shown in Figure 1-7, you must change the scsi-initiator-id on oneof the host systems to eliminate the addressing conflict between the twohost systems.

Figure 1-7 Multi-Initiated SCSI Configuration

The SCSI initiator values are changed using complex system firmwarecommands. The process of changing these values varies with systemhardware platforms.

Do not change the external SCSI bus, scsi-initiator-id , globally, change it at the interface card level.Read the documentation carefully. The procedures are hardware platform-specific.

Host System A

Internal SCSI Bus

Internal SCSI Bus Internal SCSI Bus

Internal SCSI Bus

Host System B

scsi-initiator-id= 7 scsi-initiator-id= 7

scsi-initiator-id= 7scsi-initiator-id=6

SCSICard

SCSICard

In

t9 t12t10 t13

t11 t14

Out



Multihost Fiber-Optic Interface

Two different fiber-optic interface storage arrays support multiple hostconnections. The SPARCstorage® Array 100 unit allows up to two hostsystems to connect to a single storage array. The Sun StorEdge A5000, SunStorEdge 5100, and Sun StorEdge 5200 (Sun StorEdge A5x00) arrays allowup to four host system connections. The serial optical channel (SOC)connections are shown in Figure 1-8.

Figure 1-8 Multihost Fiber-Optic Configurations

Host 0SOC+host adapter

Sun StorEdgeA5x00 StorageArray

Host 1InterfaceBoard A

InterfaceBoard B

Port A

Port B

Host 2

Host 1

Host 2

Host 3

SOChost adapter

SPARCstorageArray 100

Interfaceboard



Multipath Storage Access

Multipathing is defined as dual connections to a storage array from asingle host system. Multipathing can be used to provide redundant accesspaths in case of hardware failures and, in some cases, to perform loadbalancing between the two access paths. There are several differenthardware and software schemes that can be used to implement multipathconfigurations.

Redundant Dual Active Controller Driver

Some Sun storage devices allow dual connections to a storage array froma single host system. As shown in Figure 1-9, one host adapter can beconfigured as a backup if the primary access path fails.

Figure 1-9 Redundant Dual Active Controller Driver

The Redundant Dual Active Controller (RDAC) driver is a specialpurpose driver that manages dual-interface connections. This driver isavailable with some of the Sun storage arrays, which include the SunStorEdge A3500 and Sun StorEdge A3500FC array models.

Applications directly interface with the RDAC driver and are unaware ofinterface failure. If one of the dual-controller paths fails, the RDAC driverautomatically directs I/O to the functioning path.

Host SystemStorage Array

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Ultra SCSICard

Ultra SCSICard

Controller

ControllerC2

C1

RAIDConfigurationSoftware

RDACDriver



Note – The Sun StorEdge A3500FC array uses a Fibre Channel interfaceinstead of the SCSI interface used on the other RDAC-controlled storagearrays.

Alternate Path Driver for the Solaris™ OS

The Alternate Path (AP) software for the Solaris OS works with DynamicReconfiguration (DR) to provide the ability to move all I/O off a systemboard before removal for upgrade or repair. AP is not applicable to allarchitectures.

As shown in Figure 1-10, the AP software package contains user-levelapplications and kernel device drivers to allow the construction ofmeta-I/O devices from multiple controllers to a single physical device(disk or network). This function allows I/O on active disk and networkI/O adapters to be seamlessly redirected to a previously specified andconfigured I/O device. Failed I/O operations to a device areautomatically retried on the alternate path.

Figure 1-10 Alternate Path Hardware Configuration

Board #1

System Interconnect

Dual-Ported Storage Device

Board #2



Dynamic Multipathing Driver

The Dynamic Multipathing (DMP) driver is unique to the VxVM product.It is used only with fiber-optic interface storage arrays. As shown inFigure 1-11, the DMP driver can access the same storage array throughmore than one path. The DMP driver automatically manages multiplepaths to the storage array. Depending on the storage array model, thepaths are either used for load-balancing in a primary mode of operationor in a backup mode of operation in the event of a failure.

Figure 1-11 Dynamic Multipathing Driver

Specific paths can be enabled and disabled with the VxVM vxdmpadmcommand.

Note – During a VxVM installation, you must take special steps to ensurethat the DMP feature is compatible with AP, SAN, and the Sun StorEdgeTraffic Manager software.

Host SystemStorage Array

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

Driv

e

HBACard

HBACard

Interface

InterfaceC2

C1

DMPDriver

HBA=Host Bus Adapter



Sun StorEdge™ Traffic Manager Software

The Sun StorEdge Traffic Manager software is a new architecture forrepresenting and managing Fibre Channel devices that are accessiblethrough multiple host controller interfaces (HCI).

Sun StorEdge Traffic Manager software provides automatic load balancingand automatic failover to an alternate controller. A typical Sun StorEdgeTraffic Manager software configuration is shown in Figure 1-12.

Figure 1-12 Sun StorEdge Traffic Manager Software Configuration

Sun StorEdge Traffic Manager is the official name for the MPxIO product.

The minimum software and hardware requirements for Sun StorEdgeTraffic Manager software are:

● Solaris 8 OE Update 4

● VxVM 3.2

● Solstice DiskSuite™ 4.2.1 software

● Sun StorEdge A5200 and Sun StorEdge T3 arrays

● Peripheral Component Interconnect (PCI) single- and dual-FibreChannel network adapters

Board #1

System Interconnect

Dual-Ported Storage Device

Board #2



Multipath Software Summary

The RDAC, AP, DMP, and Sun StorEdge Traffic Manager software cancoexist in some configurations, but you might need to selectivelyconfigure each interface for use by only one of the applications. Table 1-4compares some of the multipathing software features.

Table 1-4 Multipath Software Comparison

Functionality Sun StorEdgeTraffic Manager

AlternatePathing DMP RDAC

Vendor Sun Sun VERITASSoftwareCorporation

LSI Logic/Symbios

Maximumpaths

Two Two Four Two

Active/active Yes No Yes No

Active/passive Yes Yes Yes Yes

Automaticfailover

Yes Yes Yes Yes

Supporteddevices

Sun StorEdgeA5200 array, SunStorEdge T3array

SPARCstorageArray 100, SunStorEdge A5x00array, Sun StorEdgeT3 array

Any VxVMsupporteddisk

LSI Logicdevicesonly

Host busadapter type

Fibre Channelonly

Fibre Channel only SCSI andFibre Channel

SCSI andFibreChannel

Comments Cannot runwith DMPat this time

Cannot run withDMP, and notsupported onSun Fire™ servers

VxVM-manageddisks only

LSI Logic/Symbioscontrollersonly

Naming Special names/dev/scsi_vhci

Special names/dev/ap

Specialnames/dev/vx/dmp

Nativenames

Environment Solaris 8 OE,Update 4 orhigher

Solaris 2.5.1 orhigher

Multipleoperatingsystems andlevels

Multipleoperatingsystemsand levels

Storage Configuration Identification



To be an effective administrator, you must be able to identify storageconfigurations and decode logical device addresses.

Conducting Physical Inventory

The first step in determining your storage configuration is to examine andrecord the number and types of storage arrays. You can trace cables froma system to attached storage devices.

It is helpful to create a configuration diagram showing all attachedstorage devices and any unique identification information associated withthem, such as world wide numbers, IP addresses, or names.

Displaying Storage Configurations

The format utility and the luxadm utility are commonly used to displayall available storage devices.

Using the luxadm Command to Display Storage

Use the luxadm probe option as follows to locate several types of Sunstorage arrays including Sun StorEdge T3 array logical unit numbers(LUNs). In the following example, two SENA type arrays were foundalong with two single-LUN T3 storage arrays. The luxadm command doesnot identify the exact model of storage.

# luxadm probeFound Enclosure(s):SENA Name:AA Node WWN:5080020000034ed8 Logical Path:/dev/es/ses0 Logical Path:/dev/es/ses1Node WWN:50020f200000c193 Device Type:Disk device Logical Path:/dev/rdsk/c2t1d0s2Node WWN:50020f200000c367 Device Type:Disk device Logical Path:/dev/rdsk/c3t1d0s2



Using the format Command to Display Storage

The Solaris OS format utility is a reliable program for gathering basicstorage configuration information. This utility reports all storage devices,regardless of type or model. The following sample output shows aconfiguration consisting of two internal disk drives and a single SunStorEdge T3B array that is divided into six LUNs.

# formatSearching for disks...done

AVAILABLE DISK SELECTIONS: 0. c0t0d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135> /pci@1f,4000/scsi@3/sd@0,0 1. c0t1d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135> /pci@1f,4000/scsi@3/sd@1,0 2. c2t1d0 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,0 3. c2t1d1 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,1 4. c2t1d2 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,2 5. c2t1d3 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,3 6. c2t1d4 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,4 7. c2t1d5 <SUN-T300-0201 cyl 35113 alt 2 hd 14 sec 128> /pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,5Specify disk (enter its number):



Identifying Controller Addressing

When the Solaris OS software is first configured, all disk storage interfacesare assigned a unique controller number. As shown in Figure 1-13,controller numbers are a single point of attachment to a single storagedevice or hub.

Figure 1-13 Controller Address Numbering

Using VxVM Commands to Display Controller Addresses

After the VxVM software is installed and licensed, you use the vxdmpadmcommand to display the basic controller configuration. The following isan example.

# vxdmpadm listctlr allCTLR-NAME ENCLR-TYPE STATE ENCLR-NAME=====================================================c0 Disk ENABLED Diskc2 SENA ENABLED SENA0c3 SENA ENABLED SENA1c4 T3 ENABLED T30c5 T3 ENABLED T31

c0

Host System

InternalSCSI c1

c2

c3

UDWS

FC-AL

FC-AL

Sun StorEdgeD1000 Array

Sun StorEdgeA5x00 Array



FC-AL Hub

0 1 2 3 4 5 6



Identifying Device Path Components

To configure and maintain storage devices, you must understand theterminology that describes and locates storage interface boards, storagearrays, and disk drives within a storage array.

Each disk drive in a non-hardware RAID Sun storage array is identifiedby two unique paths:

● Physical device path (under the /devices directory)

● Logical device path (under the /dev/dsk and /dev/rdsk directories)

System drivers and applications use the device paths to access specificdisk drives.

Physical Device Path Components

Physical paths describe the addresses of physical hardware componentsthat connect to a particular device. A typical physical device path is:

# ls -l /dev/dsk/c0t0d0s2lrwxrwxrwx 1 root root 41 Oct 21 21:01 /dev/dsk/c0t0d0s2 ->../../devices/pci@1f,4000/scsi@3/sd@0,0:c

# ls -l /dev/dsk/c2t1d0s2lrwxrwxrwx 1 root root 74 Oct 21 21:01 /dev/dsk/c2t1d0s2 ->../../devices/pci@6,4000/pci@2/SUNW,qlc@5/fp@0,0/ssd@w50020f230000c193,0:c

# ls -l /dev/dsk/c3t1d0s2lrwxrwxrwx 1 root root 74 Sep 24 22:46 /dev/dsk/c2t1d0s2 ->../../devices/pci@6,4000/pci@3/SUNW,qlc@4/fp@0,0/ssd@w21000020370c055a,0:c

Physical device paths describe internal hardware paths in a manner thatvaries depending on system type, interface type, and storage device type.The paths show the location and type of interface cards and also the typeof software drivers used.

Use the man pages to research physical path components as follows:

# man pci# man qlc# man ssd



Logical Device Path Components

Logical device paths to disk drives are found under the /dev/dskdirectory for block devices and under the /dev/rdsk directory for rawdevices.

A logical device path is a more basic representation of a physical path. Atypical logical device path is:

/dev/dsk/c2t3d0s7

Each logical device is linked to a physical device path:

# ls -l /dev/dsk/c1t3d0s2lrwxrwxrwx 1 root root 90 Dec 2 1998/dev/dsk/c1t3d0s2 ->../../devices/iounit@f,e1200000/sbi@0,0/SUNW,soc@3,0/SUNW,pln@a0000800,201cad7e/ssd@3,0:c

The following path, /dev/dsk/c0t3d0s2 , is an example of a typicallogical device path. Standard logical device paths are composed of fourcomponents:

● Controller address (c0)

● Target address (t3 )

● Device address (d0)

● Slice address (s2)



Typical Storage Device Addresses

The number of devices associated with each target depends on the type ofstorage device. The relationship between target and device numbers forsoftware RAID Sun storage is as follows:

● SPARCstorage Array 100:

● Thirty disks

● Six targets, t0–t5

● Five devices (d0–d4 ) associated with each target

● SPARCstorage® RSM™ array:

● Two selectable target ranges

● Seven disks

● Seven targets, t0 –t6 or t8 –t14

● A single device (d0) associated with each target

● Sun StorEdge D1000 array:

● Two selectable target ranges

● Eight disks, t0 –t3 and t8 –t11

● Twelve disks, t0 –t5 and t8 –t13


● Sun StorEdge A5x00 array:

● Four selectable target ranges

● Fourteen disks, targets, t0 –t6 and t16 –t22

● Twenty-two disks, targets t0 –t10 and t16 –t26


● Sun StorEdge MultiPack array:

● Two selectable target ranges for a six-disk model

● Six disks, targets, t1 –t6 or t9 –t14

● Twelve disks, targets t2 –t5 and t8 –t15


● Sun StorEdge MultiPack-FC array:

● Fifteen selectable target ranges

● Six disks, targets, t8 –t13




Identifying DMP Devices

DMP connections can be identified using the format utility as follows:

AVAILABLE DISK SELECTIONS:0. c0t0d0 <SUN2.1G cyl 2733 alt 2 hd 19 sec 80>/sbus@3,0/SUNW,fas@3,8800000/sd@0,0

1. c2t33d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>/sbus@3,0/SUNW,socal@0,0/sf@0,0/ssd@w22000020370c0de8,02. c3t33d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>/sbus@3,0/SUNW,socal@0,0/sf@1,0/ssd@w21000020370c0de8,0

Notice that the device paths for devices 1 and 2 have the same disk driveidentifier, 20370c0de8 . Because the controller numbers are different,devices 1 and 2 are connected to two different controller interfaces in thesame system.

Storage Array Firmware



Several Fibre Channel hardware components have resident firmware thatcan be at various revision levels. You must verify the minimum levels forthis firmware before starting an installation. If necessary, you need toupgrade the firmware to an acceptable revision level. The related FibreChannel components are:

● Fibre Channel HBA cards

● Storage array controller/interface boards

● Storage array disk drives

Fibre Channel HBA Cards

There are several generations of Fibre Channel HBA cards including:

● FC/S (also known as FC25/S or SOC cards)

The FC/S HBAs are SBus-based cards that were used with the olderSPARCstorage arrays and SBus systems.

● FC100/S (also known as SOC+ cards)

The FC100/S HBAs were faster versions of the FC/S cards and werealso used with older SBus-based systems.

● FC100/P and FC100/2P (ISP 2100 or ISP 2200 cards)

The FC100/P HBAs are PCI bus cards. The FC100/P with ISP 2100chips are single channel only. The FC100 cards with ISP 2200 chipscan be single channel (/P) or dual channel (/2P). There are differentfirmware files for the ISP 2100 and ISP 2200 versions.

● Compact PCI (cPCI) Dual Fibre Channel Adapter

Note – The numbers ISP 2100 and ISP 2200 are model numbers ofintegrated circuit chips on the interface cards.



Verifying Fibre Channel HBA Firmware

Fibre Channel HBA card firmware revisions can be checked with theluxadm utility, which is a standard Solaris OS command that resides in the/usr/sbin directory.

There are different luxadm command options for each generation of FibreChannel HBA cards. However, the most current version of luxadm has asingle option (fcode_download ) that can be used to verify and upgradefirmware on most Fibre Channel HBA cards. An example of the commandoutput follows.

# /usr/sbin/luxadm fcode_download -p

Found Path to 0 FC/S Cards Complete

Found Path to 0 FC100/S Cards Complete

Found Path to 2 FC100/P, ISP2200 Devices

Opening Device: /devices/pci@6,4000/SUNW,ifp@2:devctl Detected FCode Version: FC100/P FC-AL Host AdapterDriver: 1.9 00/03/10

Opening Device: /devices/pci@6,4000/SUNW,ifp@3:devctl Detected FCode Version: FC100/P FC-AL Host AdapterDriver: 1.9 00/03/10 Complete

The firmware revision in the preceding example is 1.9.

Caution – The same luxadm command option also downloads newfirmware into the HBA cards.



Verifying SPARCstorage Array 100 Firmware

You use the luxadm display command to determine SPARCstorageArray 100 controller board firmware revisions. You use the controlleraddress to identify the array you want to examine. An example follows.

# /usr/sbin/luxadm display c3

SPARCstorage Array 110 Configuration (luxadm version: 1.42 SunOS 5.6)Controller path:/devices/sbus@6,0/SUNW,soc@d,10000/SUNW,pln@a0000000,78ccf9:ctlr DEVICE STATUS TRAY 1 TRAY 2 TRAY 3slot1 RESERVED Drive: 2,0 Drive: 4,02 Drive: 0,1 Drive: 2,1 Drive: 4,13 Drive: 0,2 Drive: 2,2 Drive: 4,24 Drive: 0,3 Drive: 2,3 Drive: 4,35 Drive: 0,4 Drive: 2,4 Drive: 4,46 Drive: 1,0 Drive: 3,0 Drive: 5,07 Drive: 1,1 Drive: 3,1 Drive: 5,18 Drive: 1,2 Drive: 3,2 Drive: 5,29 Drive: 1,3 Drive: 3,3 Drive: 5,310 Drive: 1,4 Drive: 3,4 Drive: 5,4

CONTROLLER STATUSVendor: SUNProduct ID: SSA110Product Rev: 1.0Firmware Rev: 3.6Serial Num: 00000078CCF9Accumulate Performance Statistics: Enabled



Verifying Sun StorEdge A5x00 Array Firmware

You can use the luxadm command to display the firmware revision ofSun StorEdge A5x00 array interface boards. An example follows.

# luxadm display AASENA DISK STATUS

SLOT FRONT DISKS (Node WWN) REAR DISKS (Node WWN)0 Not Installed On (O.K.) 200000203704f9fc1 On (O.K.) 20000020370c055a Not Installed2 Not Installed Not Installed3 On (O.K.) 20000020370d5c28 On (O.K.) 20000020370d54a94 Not Installed On (O.K.) 20000020370d528a5 Not Installed Not Installed6 On (O.K.) 20000020370d5676 On (O.K.) 20000020370c0b8a

SUBSYSTEM STATUSFW Revision:1.09 Box ID:0 Node WWN:5080020000034ed8 EnclosureName:AAPower Supplies (0,2 in front, 1 in rear) 0 O.K.(rev.-02) 1 O.K.(rev.-02) 2 Not InstalledFans (0 in front, 1 in rear) 0 O.K.(rev.-05) 1 O.K.(rev.-00)ESI Interface board(IB) (A top, B bottom) A: O.K.(rev.-04) GBIC module (1 on left, 0 on right in IB) 0 O.K.(mod.-05) 1 Not Installed B: O.K.(rev.-04) GBIC module (1 on left, 0 on right in IB) 0 Not Installed 1 Not InstalledDisk backplane (0 in front, 1 in rear) Front Backplane: O.K.(rev.-04) Temperature sensors (on front backplane) 0:33ºC 1:34ºC 2:36ºC 3:36ºC 4:34ºC 5:33ºC 6:36ºC (All temperatures are NORMAL.) Rear Backplane: O.K.(rev.-04) Temperature sensors (on rear backplane) 0:36ºC 1:36ºC 2:37ºC 3:34ºC 4:33ºC 5:36ºC 6:37ºC (All temperatures are NORMAL.)Interconnect assembly O.K.(rev.-02)Loop configuration Loop A is configured as a single loop. Loop B is configured as a single loop.



Verifying Sun StorEdge T3 Array Firmware

The Sun StorEdge T3 arrays have internal configuration and controlsoftware (pSOS) that you use to verify firmware revisions. You mustperform a telnet command to a particular array, log in as user root , anduse the ver (version) command to verify the current firmware revision.

An example of a typical session follows.

$ telnet t3Trying 129.150.47.115...Connected to purple15.Escape character is ’̂ ]’.

pSOSystem (129.150.47.115)

Login: rootPassword:T300 Release 1.00 1999/12/15 16:55:46 (129.150.47.115)

t3:/:<1> ver

T300 Release 1.14 1999/12/15 16:55:46 (129.150.47.115)

Note – In the preceding example, the firmware version is 1.14.



Verifying Sun StorEdge A5x00 Disk Drive Firmware

You can use the luxadm command to verify the firmware revision of thedisk drives, as shown in the following example. The firmware isupgraded using a special program that is contained in the related diskfirmware patch.

# luxadm disp AA,f1DEVICE PROPERTIES for disk: AA,f1 Status(Port A): O.K. Vendor: SEAGATE Product ID: ST19171FCSUN9.0G WWN(Node): 20000020370d336f WWN(Port_A): 21000020370d336f Revision: 117E Serial Num: 9812R69750 Unformatted Capacity: 8637.338 MByte Read Cache: Enabled Minimum prefetch: 0x0 Maximum prefetch: 0xffff Location: In slot 1 in the Front of the enclosure named: AAA Path(s): /dev/rdsk/c2t1d0s2 /devices/pci@6,4000/SUNW,ifp@3/ssd@w21000020370d336f,0:c,raw

In the preceding example, the disk drive firmware revision is 117E.

Note – You can also use the format utility’s inquiry command option toverify firmware revisions in selected disk drives.



Firmware Upgrade Best Practices

Updating storage array firmware can be a complex task. There are strictdependency rules and procedures that must be followed or permanentdamage to interface cards can result. Check with your Sun fieldrepresentative before attempting any firmware updates. The most currentversions of firmware are available through software patches.

You should read all firmware-related patch README notes carefully. TheREADME notes frequently have specific warnings and procedurerequirements that can help prevent extended system downtime.

In some cases, permanent HBA damage can result if you try to upgradefirmware from an old version to a new version. Review the patchREADME notes for instructions informing you to first perform anupgrade to an intermediate revision level.

You should always idle your system when upgrading storage-relatedfirmware. Usually, the firmware patch README notes will instruct you toplace all systems attached to the storage in single-user mode.

Ask for help if you are not sure how to proceed.

Exercise: Recording Your Storage Configuration



In this exercise, you complete the following tasks:

● Review Sun storage features

● Identify host adapter configurations

● Identify storage array configurations

● Verify storage interface firmware revisions

● Verify array disk drive firmware revisions

Preparation

Ask your instructor to identify the system and storage that is assigned foryour use during this exercise.

If you want to simplify the task of documenting the training system configuration, you can precede this labwith a short lecture describing your particular system configuration.

Task 1 – Reviewing Sun Storage Features

Answer the following questions concerning the general features of Sunstorage devices and tools.

1. List at least four major administration tasks associated with VxVM.

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

The answers are installation, initialization, volume design, volume creation, and volume administration.

2. Which of the following are not related to multihost storage access?

a. Multi-initiated SCSI

b. Dynamic Multipathing

c. Multihost fiber-optic

d. Multi-level sequencing

The answers are b and d.



3. List four software applications that can be used to configuremultipath storage.

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

The answers are Redundant Dual Active Controller Driver (RDAC), Solaris Alternate Path Driver (AP),VERITAS Dynamic Multipathing driver (DMP), and Sun StorEdge Traffic Manager software.

4. List two applications that are used to administer controller-basedRAID storage.

_____________________________________________________________

_____________________________________________________________

The answers are Sun StorEdge RAID Manager, pSOS, or Sun StorEdge Component Manager.

5. Which of the following tools assist with swapping out a failed diskdrive?

a. vxdiskunsetup

b. vxdiskadm

c. vxdisk

d. vxdctl

e. luxadm

The answers are b, d, and e.

6. Which of the following apply to traditional SCSI interfaces?

a. They are SAN compatible.

b. They transfer data in parallel.

c. You can attach up to four hosts to a single storage unit.

d. Maximum data transfer speed is currently 40-Mbytes/sec.


7. What is the main advantage of differential SCSI interfaces?

_____________________________________________________________

The answer is longer maximum cable lengths.



8. What is the primary reason for using SAN?

_____________________________________________________________

The answer is to off-load data traffic from user networks.

9. What SAN feature isolates data resource from general access?

a. Access control lists (ACL)

b. Partitioning

c. Zoning

d. Segmentation

e. Isolation protocol

The correct answer is c.

10. What is the primary SAN data transfer mechanism?

a. Packets

b. Bitmaps

c. Groups

d. Frames

The correct answer is d.

Task 2 – Identifying Host Adapter Configurations

Perform the following steps to determine the controller numbers of theHBA cards in your classroom system.

1. Record the model number of your classroom system. Determine thisby visual inspection and the uname -a command.

_____________________________________________________________

2. Record the controller numbers of each HBA on your classroomsystem. Either inspect the /dev/dsk directory contents or use theformat utility to examine your system.

_____________________________________________________________

_____________________________________________________________



Task 3 – Identifying Storage Array Configurations

Each of the following sections is dedicated to a particular storage arraymodel and describes the most useful methods of gathering more specificstorage array information.

Before proceeding with this task, record the type of storage arraysattached to your classroom system and how many of each type there are.Determine this by visual inspection.

Type of Storage Arrays Number of Each Type

_______________ _____

_______________ _____

_______________ _____

SPARCstorage Array 100 and Sun StorEdge A5x00 Array

Use the luxadm utility to determine very specific information about bothof these storage devices. The most commonly used commands are:

# luxadm probe# luxadm display controller_number# luxadm display enclosure_name# luxadm display logical_path# luxadm display enclosure_name , disk_location# luxadm fcode_download -p

For each storage array, record the WWN, the enclosure name(Sun StorEdge A5x00 array only), the number of disks present, and thecontroller number.

WWN Enclosure Number of Controller(12 or 16 digits) Name Disks Installed Number

_______________ _____ _____ _____

_______________ _____ _____ _____

_______________ _____ _____ _____

Note – For the Sun StorEdge A5x00 array, the WWN of the enclosure isnot used in the physical path. You must use luxadm display commandto determine the WWN of the Sun StorEdge A5x00 units.



Sun StorEdge D1000 Array

The luxadm utility does not recognize the Sun StorEdge D1000 array.Specific array information must be gathered using other tools such as theformat utility and visual identification.

Use the format utility to determine the controller number of each SunStorEdge D1000 HBA card and the number of disks in each storage unit.

Controller Number Array Type Number of Disks

_______ __________ _________

_______ __________ _________

_______ __________ _________

_______ __________ _________

Task 4 – Verifying Storage Interface FirmwareRevisions

Use the luxadm utility to determine the firmware revision levels of yourclassroom system storage interface components.

1. Record the firmware revision of each HBA card.

Controller Number Host Adapter Firmware Revision

_______ ______________________

_______ ______________________

_______ ______________________

_______ ______________________

2. Record the firmware revision of each SPARCstorage Array 100controller board or Sun StorEdge A5x00 interface board.

Controller Number Array Controller/IB FirmwareRevision

_______ ______________________

_______ ______________________

_______ ______________________

_______ ______________________



Task 5 – Verifying Array Disk Drive Firmware Revisions

Use the luxadm utility or the format utility to determine the firmwarerevisions of your classroom storage array disk drives.

Record the firmware revision of at least one Sun StorEdge A5x00 diskdrive if you have this array model.

Controller Number Disk Drive Firmware Revision

_______ ______________________

_______ ______________________

_______ ______________________

_______ ______________________

Exercise Summary


Exercise Summary

?!

Discussion – Take a few minutes to discuss what experiences, issues, ordiscoveries you had during the lab exercises.

Manage the discussion here based on the time allowed for this module, which was given in the “About ThisCourse” module. If you find you do not have time to spend on discussion, then just highlight the key conceptsstudents should have learned from the lab exercise.

● Experiences

Ask students what their overall experiences with this exercise have been. You might want to go over anytrouble spots or especially confusing areas at this time.

● Interpretations

Ask students to interpret what they observed during any aspects of this exercise.

● Conclusions

Have students articulate any conclusions they reached as a result of this exercise experience.

● Applications

Explore with students how they might apply what they learned in this exercise to situations at their workplace.


Module 2

Managing Data

Objectives


● List the advantages of using virtual disk management

● Describe standard RAID terminology

● List the common features of each supported RAID level including:

● Concatenation – RAID 0

● Striping – RAID 0

● Mirroring – RAID 1

● Mirrored Stripe – RAID 0+1

● Mirrored Concatenation – RAID 0+1

● Striped Mirror – RAID 1+0

● Concatenated Mirror – RAID 1+0

● Striping with distributed parity – RAID 5

● Describe the optimum hardware configuration for each supportedRAID level

Virtual Disk Management



Servers today must meet high standards in the areas of availability,performance, scalability, and maintainability.

Availability

VxVM provides availability improvements in this area in the followingways:

● Preventing failed disk drives from making data unavailable

The probability of a single disk drive failure increases with thenumber of disk drives on a system. Data redundancy techniquesprevent failed disk drives from making data unavailable.

● Allowing file systems to grow while they are in use

Allowing file systems to grow while they are in use reduces thesystem downtime and eases the system administration burden.

● Allowing multiple-host configurations

In a dual-host configuration, one host can take over disk drivemanagement for another failed host. This configuration prevents afailed host from making data unavailable.

Performance

Many applications today require high data throughput levels. The VxVMproducts can assist in this area by more efficiently balancing the I/O loadacross disk drives.


Managing Data 2-3Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Scalability

Traditionally, file system size has been limited to the size of a single diskdrive. Using VxVM techniques, you can create file systems that consist ofmany disk drives. The fact that there are multiple disk drives istransparent to all applications. The size limit of file systems is increased tothe UNIX limit of 1 terabyte (Tbyte).

Note – The size of a single file is limited to approximately 800 gigabytes(Gbytes) because of the space required for file system overhead.

Maintainability

Administering large installations can be much easier with the assistanceof well-designed tools. VxVM has both graphical and command-line toolsto assist administrators.

VxVM’s graphical administration tool is called VERITAS EnterpriseAdministrator (VEA). VEA offers complete management assistance for allfunctions.

VxVM also has a number of command-line programs and utilities that areuseful and are preferred by many administrators. They can also be used inshell programs to perform virtually all administration tasks.

RAID Technology Introduction



The RAID concept was introduced at the University of California atBerkeley in 1987 by David Patterson, Garth Gibson, and Randy Katz.Their goal was to show that RAID could be made to achieve performancecomparable to or higher than the available performance on single, largeexpensive disk drives of the day.

During the development phase of the project, it was determined that itwas necessary to provide redundancy to avoid data loss due to frequentdisk drive failures. This aspect of the project became of great importanceto the future of RAID.

Supported RAID Standards

Many RAID levels are technologically possible but are not commonlyused. The complete list of RAID levels includes:

● RAID 0 – Striping or concatenation

● RAID 1 – Mirroring

● RAID 0+1 – Mirroring plus concatenation or striping

● RAID 1+0 – Concatenation or striping plus mirroring

● RAID 2 – Hamming code correction

● RAID 3 – Striping with dedicated parity

● RAID 4 – Independent reads and writes

● RAID 5 – Striping with distributed parity

● RAID 6 – RAID 5 with a second parity calculation

Note – RAID levels 2, 3, 4, and 6 are not available with VxVM. They arenot commonly implemented in commercial applications. RAID 0+1 andRAID 1+0 are not true RAID levels but are abstractions composed of morethan one RAID level.

RAID 6 is a proprietary controller-based RAID scheme owned by a single manufacturer. It is similar toRAID 5, but has additional parity that is written to disk. It has a higher tolerance to disk drive failures thanRAID 5 structures.



RAID Terminology

In the explanation of RAID levels in the following pages, a number oftechnical terms are used to describe RAID structure components. Thefollowing are some of the definitions:

● Stripe unit refers to a sequential group of data blocks on a single diskdrive. The stripe unit size is configurable.

● The terms disk drive and column are synonymous in RAIDdiscussions.

● Stripe width is the stripe unit size times the number of columns.

● Transfer rate and I/O per second (IOPS) are performance metrics:

● Transfer rate is the speed (measured in Mbytes per second) atwhich a system can move data through its controller. In RAIDsystems, read and write transfer rates can vary considerably.High transfer rates are particularly valuable for applicationsthat must move large amounts of data quickly, such asdocument imaging, data mining, or digital video applications.

● IOPS is a measure of the ability of a storage system to handlemultiple, independent I/O requests in a certain period of time.RAID systems with high transfer rates do not always have goodIOPS performance. Database and transaction processingsystems are examples of applications that typically require highI/O rate performance.

RAID Level Common Features



The section reviews the common features of the following supportedRAID levels:

● Concatenation – RAID 0

● Striping – RAID 0

● Mirroring – RAID 1

● Mirrored Stripe – RAID 0+1

● Mirrored Concatenation – RAID 0+1

● Striped Mirror – RAID 1+0

● Concatenated Mirror – RAID 1+0

● Striping with distributed parity – RAID 5

Concatenation – RAID 0

The primary reason for using concatenation is to create a virtual diskdrive that is larger than one physical disk drive. Concatenation obtainsmore storage capacity by logically combining portions of two or morephysical disk drives. Concatenation also enables you to grow a virtualdisk drive by concatenating additional physical disk drive space to it. Thistechnique does not restrict the mix of different size drives and memberdrives can be of any size. Therefore, no storage space is lost.

The example in Figure 2-1 on page 2-7 shows the concatenation of threephysical disk drives. Each portion of the concatenation is managed byVxVM and is called a subdisk. A subdisk is the basic unit that VxVM usesto assemble and control all data storage areas.



The array management software is responsible for taking the threephysical disk drives and combining them into one virtual disk drive. Thisdisk drive is presented to the application as a contiguous storage area.

Figure 2-1 RAID-0 Concatenated Structure

The term block represents a disk drive block or sector (512 bytes) of data.

Concatenation is the default vxassist configuration.

Advantages

The following advantages can be gained by using concatenated structures:

● Concatenation can improve performance for random I/O becausethe data is spread over multiple disk drives.

Ask students to discuss the impact of concatenation on performance and cost.

● One hundred percent of the disk drive capacity is available for userdata.

PhysicalDisk 1

PhysicalDisk 2

Array ManagementSoftware

PhysicalDisk 3

Block 2001

Block 3000

Block 1

Virtual Disk

Block 1001

Block 2000

Block 1

Block 1000

Block 3000



Limitations

The limitations of concatenated structures include:

● If you use only concatenation, there is no redundancy. Concatenatedvolumes can be mirrored to achieve redundancy.

● Concatenation is less reliable because the loss of one disk driveresults in the loss of data on all disk drives.

● When the disk drives are full, the data is spread throughout all themembers. However, when the disk drives are not full, the last diskdrives are unused, thereby lowering the use of all the drives.

Ask students to discuss the impact of concatenation on failure and recovery.

Striping – RAID 0

The primary reason for using striping is to improve IOPS performance.The performance increase comes from accessing the data in parallel.Parallel access increases I/O throughput because all disk drives in thevirtual device are busy most of the time servicing I/O requests.

The array management software is responsible for making the array looklike a single virtual disk drive. Striping takes portions of multiple physicaldisk drives and combines them into one virtual disk drive that ispresented to the application.

As shown in Figure 2-2 on page 2-9, the I/O stream is divided intosegments called stripe units (SUs), which are mapped across two or morephysical disk drives, forming one logical storage unit. The stripe units areinterleaved so that the combined space is made alternately from eachslice, and is, in effect, shuffled like a deck of cards. The stripe units areanalogous to the lanes of a freeway.



There is no data protection in this scheme and, because of the way thatstriping is implemented, the loss of one disk drive results in loss of dataon all striped disk drives. Therefore, while this implementation improvesperformance, it degrades reliability.

Figure 2-2 RAID 0 Striped Structure

The layout specification to use for vxassist is layout=stripe .

Advantages

The following advantages can be gained by using a striped structure:

● Performance is improved for large sequential I/O requests and forrandom I/O. You can optimize stripe unit size for sequential orrandom access.

● One hundred percent of the disk drive capacity is available for data.

Ask students to discuss the impact of striping on performance and cost.

Limitations

Some of the limitations of striped structures are:

● There is no redundancy.

● Striping is less reliable, because the loss of one disk drive results inthe loss of data on all striped disk drives.

Ask students to discuss the impact of striping on failure and recovery.

PhysicalDisk 1

PhysicalDisk 2


PhysicalDisk 3

SU 1SU 2SU 3SU 4SU 5SU 6

SU 1 SU 4

SU 2 SU 5

SU 3 SU 6 Virtual Disk

SU = Stripe Unit



Mirroring – RAID 1

The primary reason for using mirroring is to provide a high level ofavailability or reliability.

Mirroring provides data redundancy by recording data multiple times onindependent spindles. The mirrored disk drives appear as one virtual diskdrive to the application. In the event of a physical disk drive failure, themirror on the failed disk drive becomes unavailable, but the systemcontinues to operate using the unaffected mirrors.

The array management software takes duplicate copies of the data locatedon multiple physical disk drives and presents one virtual disk drive to theapplication, as shown in Figure 2-3.

Figure 2-3 RAID-1 Mirror Structure

In VxVM, the mirror is seen as a single logical address, block 0 to n blocksin length. Because of this, VxVM does not concern itself with the format ofeach individual mirror. One side of the mirror can be striped, and theother side of the mirror can be concatenated. This implementation can beused if there is a lack of enough physical disk drives or for performancereasons. VxVM can guarantee consistent data across both sides of themirror, no matter what the format, because it writes to a given logicalblock address.

The layout specification to use for vxassist is layout=mirror .


Block 1

Block 2

Block 3

Block 4

Block 1

Block 2

Block 3

Block 4

Block 1

Block 2

Block 3

Block 4

Virtual Disk



Advantages

The following advantages can be gained by using a RAID-1 mirroredstructure:

● There is a fully redundant copy of the data on one or more diskdrives.

If the mirror resides in a storage array that is attached to a differentinterface board, a high level of availability can be achieved.

● All drives can be used for reads to improve performance.

Mirroring improves read performance only in a multiuser ormultitasking situation where more than one disk drive member cansatisfy read requests. Conversely, if there is only a single threadreading from the volume, performance is not improved.

● You can set up three-way mirroring, but there is a performancepenalty.

Write performance can suffer up to 44 percent with a three-waymirror.

Ask students to discuss the impact of mirroring on performance and cost.

Ask students to discuss the impact of mirroring on failure and recovery.

Limitations

Some of the limitations of using a RAID-1 mirrored structure are:

● Mirroring uses twice as many disk drives, which essentially doublesthe cost per Mbyte of storage space.

● Mirroring degrades write performance by about 15 percent. This issubstantially less than the typical RAID-5 write penalty (which candegrade as much as 70 percent).



Mirrored Stripe – RAID 0+1

The primary reason for using mirrored stripe volumes is to gain theperformance offered by striping and the availability offered by mirroring.

Mirrored stripe configurations are a relatively high-cost installation, butmany customers consider it a worthwhile investment.

As shown in Figure 2-4, two drives are first striped and then mirrored.The reliability is as high as with mirroring. Because the technique ofstriping is also used, the performance is much better than when using justmirroring.

Figure 2-4 Mirrored Stripe RAID Structure

The layout specification to use for vxassist is layout=mirror-stripe .

ArrayManagement

Software

SU 1SU 2SU 3SU 4SU 5SU 6SU 7SU 8

ArrayManagement

Software


SU 2SU 4SU 6SU 8

SU 1SU 3SU 5SU 7

SU 2SU 4SU 6SU 8

SU 1SU 3SU 5SU 7 Array

ManagementSoftware


Virtual Disk

Striping

Mirroring

Striping

Virtual DisksPhysical Disk



Advantages

The primary advantage of mirrored stripe volumes is spreading dataacross multiple disk drives (improved I/O per second) while providingadded redundancy of the data.

Limitations

Some of the limitations of using a Mirrored Stripe – RAID 0+1 are:

● Mirrored stripe volumes suffer the high cost of mirroring, requiringtwice the disk drive space of non-redundant volumes.

● The loss of a single disk drive disables a mirror and results in theloss of redundancy.



Mirrored Concatenation – RAID 0+1

The primary reason for using mirrored concatenation volumes is to gainthe availability offered by mirroring while maximizing storage utilization.

Mirrored concatenation volumes can be a relatively high-cost installation,but many customers consider it a worthwhile investment.

As shown in Figure 2-5, two drives are first concatenated and thenmirrored for increased reliability. Because the technique of concatenationis used, varied storage segments of dissimilar size can be combined tomaximize storage utilization.

Figure 2-5 Mirrored Concatenation RAID Structure

The layout specification to use for vxassist is layout=mirror-concat .

Advantages

The primary advantage of mirrored concatenations is increased reliabilitycombined with maximized storage utilization.

Limitations

Mirrored concatenated volumes suffer the high cost of mirroring,requiring twice the disk drive space of non-redundant volumes.

Array Management

Software

Array Management

Software

Array Management

Software

Block 0

Block 999

Blocks 0-499

Blocks 500-999

VolumeBlocks 0-499

Blocks 500-999

Concatenation

Concatenation

Mirroring



Striped Mirror – RAID 1+0

Striped mirror volumes have the performance and reliability advantagesof mirrored stripe volumes, but can tolerate a higher percentage of diskdrive failures without data loss.

Striped mirror volumes also have a quicker recovery time after a diskdrive failure because only a single stripe must be resynchronized insteadof an entire mirror. As a best practice, use striped mirror volumes for largevolumes where failure recovery time and performance are issues.

As shown in Figure 2-6, the concept of striped mirrors is fundamentallydifferent from mirrored stripes. In a striped mirror configuration, eachstripe is mirrored separately. Additional internal structures calledsub-volumes (SVs) are used to implement RAID 1+0 volumes.

Figure 2-6 Striped Mirror RAID Structure

Note – VxVM refers to the striped mirror configuration as a layeredvolume or as a Pro volume structure.

The layout specification to use for vxassist is layout=stripe-mirror .

ArrayManagement

Software

SU 1SU 2

ArrayManagement

SoftwareSU 1

SV 2

SV 1

SV 2

SV 2

ArrayManagement

SoftwareSU 2

Virtual Disk

Mirroring

Striping

Mirroring

Virtual DisksPhysical DiskSub-Volumes



Advantages

The following advantages can be gained by using a Striped Mirror –RAID 1+0 structure:

● Because each stripe is mirrored separately, a larger number of diskdrive failures can be tolerated without disabling the volume.Availability increases exponentially with disk drive (stripecomponent) count.

● Striped mirror configurations have the same performance benefits ofmirrored stripe volumes.

Limitations

Some of the limitations of using a Striped Mirror – RAID 1+0 are:

● Striped mirror volumes suffer the high cost of mirroring, requiringtwice the disk drive space of non-redundant volumes.

● A mirror cannot be detached and used to perform an online backup.With a RAID 0+1 volume, you can detach a mirror and back it up totape.

Note – Do not detach a volume mirror to perform online data backups.The active data is not protected by redundancy during the backupoperation.



Concatenated Mirror – RAID 1+0

Concatenated mirror volumes generally have a much quicker recoverytime after a disk drive failure because only a portion of the concatenationmust be resynchronized (instead of an entire mirror).

As a best practice, use concatenated mirror volumes for large volumeswhere failure recovery time is an issue.

As shown in Figure 2-6, the concept of a concatenated mirror volume isfundamentally different from a mirrored concatenation volume. In aconcatenated mirror configuration, each part of a concatenation ismirrored separately.

Figure 2-7 Concatenated Mirror RAID Structure

Note – VxVM refers to the concatenated mirror configuration as a layeredvolume and as a Pro volume structure.

The layout specification to use for vxassist is layout=concat-mirror .

Block 1

ArrayManagement

Software

Virtual Disk

Block 1

Block 3000

Block 1000

Block 1001

Block 2000

Block 2001

Block 3000

Block 1

Block 1000

Block 1001

Block 2000

Block 2001

Block 3000

Mirror

Mirror

Mirror



Advantages

Because each portion of the concatenation is mirrored separately, recoverytime is much faster.

Limitations

Some of the limitations of using a Concatenated Mirror – RAID 1+0 are:

● Concatenated mirror volumes suffer the high cost of mirroring,requiring twice the disk drive space of non-redundant volumes.

● A mirror cannot be detached and used to perform an online backup.With a mirrored concatenated volume, you can detach a mirror andback it up to tape.

Note – Do not detach a volume mirror when performing online databackups. The active data is not protected by redundancy during thebackup operation.



Striping With Distributed Parity – RAID 5

RAID-5 configurations can be an attractive choice for read-intensiveapplications that require increased data protection.

Three of the RAID levels introduced by the Berkeley Group have beenreferred to as parity RAID because they use a common data protectionmechanism. RAID 3, 4, and 5 all use the concept of bit-by-bit parity toprotect against data loss.

RAID 3, 4, and 5 all implement the Boolean Exclusive OR (XOR) function tocompute parity. Parity is implemented bit-by-bit to corresponding stripeunits of member drives, and the result is written to a corresponding paritydisk drive found in RAID 3 and 4. As shown in Figure 2-8, the parity (P)is distributed throughout all the member drives in RAID 5.

Figure 2-8 RAID-5 Structure

Additional features include:

● Independent access is available to individual drives.

● Data and parity are both striped across spindles.

● Reads per second can reach the disk drive rate multiplied by thenumber of disk drives (minus the parity disk) in a column.

Ask students to discuss the impact of RAID 5 on performance, cost, failure, and recovery.

The layout specifications to use for vxassist is layout=raid5 (or raid5nolog), logging is the default.

Disk 3

Disk 2


Disk 1

Disk 4

SU 1SU 2SU 3SU 4SU 5SU 6SU 7SU 8SU 9SU 10SU 11SU 12

Virtual Disk

SU 1 SU 5 SU 9 P(10-12)

SU 2 SU 6 P(7-9) SU 10

SU 3 P(4-6) SU 7 SU 11

P(1-3) SU 4 SU 8 SU 12 SU = Stripe Unit



Advantages

Some advantages of RAID-5 volumes are:

● Parity protects against single disk drive failure.

● RAID 5 requires only one additional drive beyond those used fordata.

● Read performance can be similar to a RAID 0 striped volume.

Limitations

Some limitations of RAID-5 volumes are:

● A minimum of three disk drives are required to implement RAID 5in VxVM.

● RAID 5 cannot be mirrored using VxVM. Redundancy is providedthrough the parity information.

● Write-intensive performance is poor.

Overall random I/O performance is dependent on the percentage ofwrites. If more than 20 percent of the I/O operations are writes,consider an alternative RAID option, such as RAID 1+0.

● There can be severe performance degradation with a failed diskdrive in a write-intensive environment.

Point out that this is the reason for hardware-based RAID, such as that used in the Sun StorEdge A3500 andSun StorEdge A1000 arrays.

Exercise: Optimizing System Configurations




● Review software RAID features

● Identify availability and performance cabling

● Optimize RAID-0 volumes



● Optimize RAID-0+1 volumes

● Optimize RAID-1+0 volumes

● Identify effective storage utilization

● Select disk drives for use

Preparation

The first several tasks in this exercise are group discussions aboutoptimizing hardware configurations to suit particular volume structures.

The last task involves identifying and recording your VxVM serverconfiguration and selecting six disk drives for use during the remainder ofthis course.

You should work on VxVM servers in small groups of two or three. Eachgroup has six disk drives with which to work. Ideally, three of the diskdrives are on one array, and three are on a different array.

You must examine your classroom setup and determine which disk drivesyou are going to use.

Caution – More than one group might be working on the same server. It isessential that each group is using their own disk drives and does notaccidentally reconfigure disk drives that are being used by another group.



Task 1 – Reviewing Software RAID Features

Answer the following questions concerning the general features ofsoftware RAID administration.

1. List four major areas of concern when using software RAIDproducts.

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

The answers are availability, performance, scalability, and maintainability.

2. Which of the following RAID levels are not supported by VxVM?

a. RAID 0

b. RAID 1

c. RAID 2

d. RAID 3

e. RAID 4

f. RAID 5

g. RAID 6

The answers are c, d, e, and g.

3. Which of the following are characteristics of a RAID-0 concatenatedstructure?

a. High tolerance to disk drive failures

b. Maximized disk space utilization

c. Enhanced write performance

d. Good random read performance

e. Data storage space is contiguous

The answers are b, d, and e.



4. Which of the following are characteristics of a RAID-0 stripedstructure?


b. Maximized disk space availability

c. Enhanced I/O performance

d. Good random read performance


The answers are b, c, and d.

5. Which of the following are characteristics of a RAID-1 mirroredstructure?

a. Enhanced tolerance to disk drive failures


c. Enhanced read performance

d. Enhanced write performance


The answers are a and c.

6. Which of the following are characteristics of a RAID-0+1 mirroredstripe structure?




d. Poor write performance



7. What is the size of disk drive block?

_____________________________________________________________

The answer is 512 bytes



8. Which of the following are characteristics of a RAID-1+0 stripedmirror structure?




d. Poor write performance



9. Which of the following are characteristics of a RAID-5 structure?




d. Potentially poor write performance


The answers are a and d.

10. What is another term for striping?

_____________________________________________________________

The answer is interleaving.

11. Which of the following RAID structures has the highest tolerance todisk drive failures?

a. RAID 0+1

b. RAID 5

c. RAID 1

d. RAID 1+0

e. RAID 0

The answer is d.



Task 2 – Identifying Availability and PerformanceCabling

For increased performance, availability, or both, you can distribute virtualvolume components across system boards, host bus adapters (HBAs), andstorage array targets, as shown in Figure 2-9.

Figure 2-9 Availability and Performance Cabling

?!

Discussion – Consider the following:

What are the general advantages of the physical configuration shown inFigure 2-9?

Multiple boards, HBAs, and arrays reduce the possibility of a catastrophic failure that disables an entire site.

Volume mirrors can be distributed for higher availability.

Volume stripes can be distributed for higher performance.

Distribution across array targets can increase general performance.

System Board

Array

Host System

t1

t2

t3

Array

t1

t2

t3

HBA c3

System Board HBA c4



Task 3 – Optimizing RAID-0 Volumes

Each of the following volume structures represent some advantage. Theadvantage might be cost reduction, simplicity, availability, orperformance.

Availability, performance, or both, can be improved for some RAIDstructures by optimizing the physical system configuration. For someRAID structures, such as the basic concatenation or stripe shown inFigure 2-10, you can improve performance but not availability.

Figure 2-10 RAID-0 Volume Structures

?!


1. What hardware configurations can improve the availability orperformance of either structure shown in Figure 2-10?

2. What is the ideal hardware configuration assuming there are nohardware budget limitations?

Hint – Assume each subdisk or stripe is a different disk drive: losing one disk drive disables the volume.

Good – One system board, one HBA, and one array is as good as the concatenation gets. The stripeperformance can be improved a lot.

Better – One system board, four HBAs, and four arrays provide the best stripe performance.

Best – Four system boards, four HBAs, and four arrays provide a slight performance gain.

Subdisk 1 Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

Subdisk 2

Subdisk 3

Subdisk 4

RAID 0 (Concatenated)

RAID 0 (Striped)

Volume Volume




RAID-1 volumes consist of two mirrors, as shown in Figure 2-11. Each ofthe mirrors can be a concatenation, but that does not affect the optimizedhardware configuration.


?!


1. What hardware configurations can improve the availability orperformance of the structure shown in Figure 2-11?


Hint – Assume each subdisk is a different disk drive: losing two disk drives can disable the volume.

Good – One system board, two HBAs, and two arrays provide near maximum availability, but not muchpossibility of performance increase.

Better – Two system boards, two HBAs, and two arrays provide a slight availability increase, but noperformance gains.

Best – There is not much else to do except use a SunPlex™ platform configuration.

Subdisk 1

Subdisk 2

Subdisk 3

Subdisk 4

Subdisk 1

Subdisk 2

Subdisk 3

Subdisk 4

Mirror Mirror

RAID 1 (Mirrored)Volume




RAID-5 volumes are constructed in a fashion similar to a RAID-0 stripedvolume, but they have the added complication of distributed parity. Atypical RAID-5 volume structure is shown in Figure 2-12.


?!




Hint – Assume each stripe or log is a different disk drive: losing two disk drives can disable the volume.

Good – One system board, one HBA, and one array: a single HBA failure disables the volume.

Better – Four system boards, four HBAs, and four arrays provide some availability increase and betterperformance.

Best – There is no best. It is generally all or nothing with RAID-5 structures.

Volume

Dataor

Parity

Dataor

Parity

Dataor

Parity

Dataor

Parity

Subdisk 1 Subdisk 2 Subdisk 3 Subdisk 4

RAID 5



Task 6 – Optimizing RAID 0+1 Volumes

As shown in Figure 2-13, RAID 0+1 volumes combine mirroring andstriping with the mirror components above the stripes. The primaryshortcoming is that the loss of any stripe disables a mirror.

Figure 2-13 RAID 0+1 Volume Structure

?!




Hint – Assume each stripe is a different disk drive: losing two disk drives can disable the volume.

Good – Two system boards, two HBAs, and two arrays provide some availability and a performance gain fromstriping.

Better – Two system boards, two HBAs, two hubs, and eight arrays increase availability and improveperformance.

Best – Eight system boards, eight HBAs, and eight arrays provide the best availability and performance.


MirrorMirror

Volume




Task 7 – Optimizing RAID 1+0 Volumes

As shown in Figure 2-14, RAID 1+0 volumes combine striping andmirroring with the stripe components above the mirrors. If the hardwareis properly configured, a RAID 1+0 volume can tolerate a higherpercentage of hardware failures without disabling the volume.

Figure 2-14 RAID 1+0 Volume Structure

?!


What hardware configurations can improve the availability orperformance of the structure shown in Figure 2-14?

Hint – Assume each mirror is a different disk drive: you can lose up to four disk drives without disabling thevolume.

Good – Two system boards, two HBAs, and two arrays provide availability and some performance gains fromstriping. Primary mirrors are on one array, secondaries are on the other.

Better – Two system boards, two HBAs, two hubs, and eight arrays increase availability and improveperformance.

Best – Eight system boards, eight HBAs, and eight arrays provide the best availability and performance.

StripeUnit

StripeUnit

StripeUnit

StripeUnit

MirrorSubdisk

MirrorSubdisk

Volume

MirrorSubdisk

MirrorSubdisk

MirrorSubdisk

MirrorSubdisk

MirrorSubdisk

MirrorSubdisk



Task 8 – Identifying Effective Storage Utilization

The storage configuration shown in Figure 2-15 is sufficient tosimultaneously configure an example of every supported RAID structure.

Figure 2-15 Storage Utilization Study

1. Circle the disk drives in Figure 2-15 that you would use to build thefollowing RAID structures:

● A three-disk RAID-0 striped volume

● A two-disk RAID-0 concatenated volume

● A two-disk RAID-1 mirrored volume

● A four-disk RAID-5 volume (no log)

● A four-disk RAID-0+1 mirror-stripe volume

● A four-disk RAID-1+0 stripe-mirror volume

2. If all the disk drives in Figure 2-15 are 9 Gbytes in size, what is theapproximate data storage available for each of the structures?

Assume you are using entire disk drives. There is one disk drive left over for a spare.

t1

t2

t3

t4

t5

t32

t33

t34

t35

t36

HBA c3

HBA c4

HBA c5

HBA c6

ArrayArrayArrayArray

t64

t65

t66

t67

t68

t96

t97

t98

t99

t100



Task 9 – Selecting Disk Drives for Use

Each workgroup must select six disk drives for use during the rest of thiscourse. The ideal configuration of disk drives is:

● Three disk drives on one storage array

● Three more disk drives on a different storage array

You create and destroy disk groups and several different volumestructures, so it is important that the disk drives you select are not beingused by another group.

Use the following commands to select and record the logical addresses ofthe six disk drives that your group chooses:

● format

● luxadm probe

● luxadm display

Record the logical paths to the six disk drives for your workgroup in theform c2t3d4 .

Disk: _______________ Disk: _______________

Disk: _______________ Disk: _______________

Disk: _______________ Disk: _______________

Caution – If there are other workgroups using the same VxVM server,you must check with them to ensure that you are not using some of theirdisk drives.

Exercise Summary


Exercise Summary

?!



● Experiences


● Interpretations


● Conclusions


● Applications



Module 3

VERITAS Volume Manager Installation

Objectives


● List the key elements of pre-installation planning

● Research VxVM software patch requirements

● Install the VxVM software

● Initialize the VxVM software

● Verify the post-installation environment

● Prepare for virtual disk drive management

● Install the VEA client software

● Use the basic VEA features

● Use the VxVM error numbering system

Installation Planning



VxVM installations vary in size from small desktop systems to largeservers with Tbytes of data storage. Regardless of the system size, theinstallation should be carefully planned in advance.

System Downtime

During a new installation or an upgrade, some system downtime isalways required. Usually you should schedule system downtime so that itoccurs during off-peak system-usage time. Thorough pre-installationplanning usually minimizes the system downtime.

Storage Configuration Assessment

You must decide which disks should be managed by VxVM. As youfurther configure VxVM, you must determine the logical organization orgrouping of the disk drives. For example, 10 of the disk drives should beallocated for accounting applications, and 20 disk drives should beallocated for marketing applications.

You have the option of not placing certain disk drives under VxVMcontrol. This option is useful if you have applications that are currentlyusing file systems or partitions, and you do not want to update theapplications’ references to these file systems or partitions.

In contrast, you might want to put your system disk under VxVM controlso that it can be mirrored.

Existing data on disk drives can be encapsulated. When data isencapsulated, the partition is added to VxVM’s control, leaving thepartition intact and maintaining the integrity of the data. This action canbe done at installation or at a later time.

You might also need to plan for new disk storage devices. In addition, youmight need to add more memory and larger backup tape systems tocompensate for the increased storage load.


VERITAS Volume Manager Installation 3-3Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Upgrade Resources

Some of the most frustrating installation issues can be discovering thatyou are missing a CD-ROM, discovering that you do not have the neededpatches, or discovering that you have misplaced the installationdocuments. Having all the required CD-ROMs, patches on theappropriate media, and documentation minimizes your frustration. Notonly should you have the documentation (for example, release notes andinstallation procedures), but you should read it. Reading the installationdocumentation is the only way to ensure that you have all of the requiredpatches.

Licensing

VxVM uses license keys to control access. If you have aSPARCstorage Array 100 or a Sun StorEdge A5x00 array attached to yoursystem, VxVM automatically configures a basic-use license. You can alsoconfigure non-array drives connected to the same host. Other storagearrays might require manual license installations.

Current System Checkpoint

When installing a new and complex application, such as VxVM, you mustalways be prepared to return your system to its original state.

Backups

Not only must you have backups, but you must verify them. If there is ahardware failure or not enough space to facilitate the upgrade, you mustbe able to recover or back out the software. Perform a complete backupimmediately prior to the installation process.

Testing the New Configuration

If this is a new installation, test it prior to going into production mode. Itwould be ideal to test all the components, including the storagesubsystem. During this time, any issues related to patches and firmwarecan be resolved.

Researching Software Patches



Before installing the VxVM software, you should research any patchesthat might be required. Your Sun field representative can assist you.

Ensure that you read all of the README notes in all of the patches.

Storage-related patches can require careful attention and knowledgebecause they frequently must be installed in groups and in an exactsequence.

Array firmware patches usually install new software drivers that aresometimes designed to work only with a small range of array firmwarerevisions.

Caution – If the mismatch between the system software drivers and thearray-resident firmware is too great, the storage arrays can becomeunavailable. Correcting the problem can be difficult and might requireSun support and hours of downtime.

Researching Current Patch Information

Current patch information can be obtained from the external Sun Web siteat http://patchpro.sun.com . From the Web site, you can use thePatchPro Interactive tool to display a customized patch listing that reflectsyour system configuration. You can configure a PatchPro query to includethe following information:

● Solaris OS release

● Platform type (system model)

● Disk arrays and disk drives

● Tape libraries and tape drives

● Switches and HBA models

● SAN products

● Software versions (common applications, such as VxVM)



Currently, the information is organized and retrieved in separatecategories, such as Enterprise Systems Products and Network StorageProducts.

Figure 3-1 shows a typical SunSolveSM PatchPro Network StorageProducts listing for the following configuration:

● Solaris 9 12/02 Operating System

● Sun Fire™ V480 server

● Sun StorEdge A5200 arrays

● Sun StorEdge T3B arrays

● X6799A (FC100/2P) Fibre Channel interface cards

Figure 3-1 SunSolve Program PatchPro Information

The configuration used to produce the preceding output purposely uses some out-of-date components.A configuration using all of the most recent components produces little or no output.

At the time of writing, PatchPro has still not been updated to reflect the current VxVM version.

Additional patches might be required or recommended by the applicationvendor.



Installing Patches

The following is a typical patch installation scenario for the configurationshown in Figure 3-1 on page 3-5:

1. Pay close attention to the PatchPro listing Legend section and thePatch Fixes column.

The order of the patches can be critical. Firmware patches must beinstalled with care. You must carefully study all firmware patchREADME notes before taking any action, especially in the followingareas:

● Keyword and Synopsis section

● Patches Required With This Patch section

● WARNING and Patch Installation Instructions sections

2. Examine the /var/sadm/patch directory to check for patches thatwere installed after the operating system installation.

You can also use the patchadd -p command, but it displays manyscreens of patches that are incorporated into the currently installedoperating system.

3. Verify all firmware levels before attempting to install firmwarepatches.

Verifying firmware levels varies according to system and storagetypes. Older products are checked using the luxadm command.Newer products, such as the Sun StorEdge T3 array, require you touse array-resident firmware programs to verify revision levels.

Vendor Software Patches

Patches that are distributed on vendor CD-ROMs or listed in vendordocumentation are usually out of date. Do your research before startingpatch installation.

Installing VxVM Software



With every release of VxVM, there is an installation guide thataccompanies the software. This guide is a comprehensive document thatdefines the various scenarios for installing and upgrading currentVERITAS software products, as well as the Solaris OS.

Software Distribution Overview

The VxVM software distribution CD-ROM contains the followingtop-level directories:

● pkgs

The pkgs directory contains all the software files and documentation.

● scripts

The scripts directory contains script files to assist you inperforming VxVM upgrades. These scripts (upgrade_start andupgrade_finish ) are referenced in the upgrade procedures in theVxVM documentation.

There are other scripts (fixsetup , fixstartup , fixmountroot , andfixunroot ) that can be used to attempt recovery from difficult bootfailure situations. The use of these scripts is not documented, andthey should not be used without express instructions from technicalsupport personnel.

● win32

The win32 directory contains a single disk operating system (DOS)executable, SETUP.EXE, for installing the VEA client software onMicrosoft Windows systems.

● patches

The patches directory contains any patches that were critical at thetime of the product release.

● release_notes

The release_notes directory contains several small PortableDocument Format (PDF) and Adobe® PostScript release documents.



● support

The support directory contains a group of support tools used togather configuration information. Use these tools only under thedirection of technical support personnel. Sun technical supportpersonnel use a different information gathering tool called Explorer.

● veritas_enabled

The veritas_enabled directory contains many library files tosupport a wide range of Sun and third-party storage arrays.

Software Package Summary

The VxVM software distribution CD-ROM contains the software packageslisted in Table 3-1. Installation should be performed in the order shown.

The VERITAS File System (VxFS) package, VRTSfspro , is required even ifyou are not going to install the VxFS software. There are several otherVxFS packages on the distribution CD-ROM that are not discussed in thismodule.

Table 3-1 VxVM Software Packages

Package Title

VRTSvlic VERITAS licensing utilities

VRTSvxvm VxVM software

VRTSvmdoc VxVM user documentation

VRTSvmman VxVM manual pages

VRTSob VEA Service

VRTSobgui VEA

VRTSalloc Intelligent Storage Provisioning

VRTSddlpr Device Discovery Layer

VRTSvmpro VERITAS Volume Manager Management ServicesProvider

VRTSfspro VERITAS File System Management Services Provider



Software Package Installation

You should use the pkgadd command to install the VxVM softwarepackages. To avoid package dependency messages, install the packages inthe order shown in the following example.

1. Install the license package and the basic VxVM packages.

# pkgadd -d . VRTSvlic VRTSvxvm VRTSvmdoc VRTSvmman

2. Install the VEA software packages and reference the customadministration file, VRTSobadmin.

# pkgadd -a ../scripts/VRTSobadmin -d . VRTSobVRTSobgui

3. Install the remaining VxVM software packages.

# pkgadd -d . VRTSalloc VRTSddlpr VRTSvmpro VRTSfspro

Note – There is an additional localization package, VRTSmuob, that addssome localized VEA information in French, Japanese, or Chinese. TheVRTSmuobpackage can be added at any time if needed.

The example does not show the installation of the VxFS software packages.



Software Installation User Interaction

Throughout the VxVM software installation, you are asked questions, andsome warning messages appear. The following sections summarizerequired interactions by software package.

The VRTSvlic Package Installation Questions

This package contains scripts which will be executedwith super-user permission during the process ofinstalling this package. Do you want to continue withthe installation of <VRTSvlic> [y,n,?] y

The VRTSvxvmPackage Installation Questions

Install for which version of Solaris? [9, 8, 7](default: 9): 9

This package contains scripts which will be executedwith super-user permission during the process ofinstalling this package. Do you want to continue withthe installation of <VRTSvxvm> [y,n,?] y

The VRTSvmdocPackage Installation Questions (None)

The VRTSvmdocpackage does not prompt for any user-input duringits installation.

The VRTSvmmanPackage Installation Questions

The following files are already installed on the systemand are being used by another package:

/opt/VRTS <attribute change only>/opt/VRTS/man <attribute change only>

Do you want to install these conflicting files[y,n,?,q] y



The VRTSobPackage Installation Questions

The selected base directory </opt/VRTSob> must existbefore installation is attempted.

Do you want this directory created now [y,n,?,q] y

The VRTSobgui Package Installation Questions

The VRTSobgui package does not prompt for any user-input duringits installation.

The VRTSalloc Package Installation Questions

This package contains scripts which will be executedwith super-user permission during the process ofinstalling this package. Do you want to continue withthe installation of <VRTSalloc> [y,n,?] y

The VRTSddlpr Package Installation Questions

This package contains scripts which will be executedwith super-user permission during the process ofinstalling this package. Do you want to continue withthe installation of <VRTSddlpr> [y,n,?] y

The VRTSvmproPackage Installation Questions

This package contains scripts which will be executedwith super-user permission during the process ofinstalling this package. Do you want to continue withthe installation of <VRTSvmpro> [y,n,?] y

The VRTSfspro Package Installation Questions

The following files are already installed on the systemand are being used by another package:

* /opt/VRTSob <attribute change only>* - conflict with a file which does not belong to anypackage.



Do you want to install these conflicting files[y,n,?,q] y

This package contains scripts which will be executedwith super-user permission during the process ofinstalling this package. Do you want to continue withthe installation of <VRTSfspro> [y,n,?] y

At this point, the VxVM software is installed but not operational. If youreboot the system, you see at least two VxVM error messages similar tothe following:

VxVM NOTICE V-5-2-3347 Volume Manager not startedVxVM NOTICE V-5-2-3365 VxVM not startedVxVM Provider initialization warning: Configurationdaemon is not accessible

Only the vxsvc daemon is running. VxVM must be initialized using thevxinstall utility before it can start successfully.

Initializing VxVM Using the vxinstall Utility



The VxVM software cannot start correctly until it has been initializedusing the vxinstall utility. In previous VxVM versions, a disk groupnamed rootdg containing at least one disk drive was a mandatoryrequirement for VxVM operation. The VERITAS Volume Manager 4.0release has eliminated that requirement.

The vxinstall Utility Dialogue

As of this release, the vxinstall utility no longer calls the vxdiskadmutility to configure an initial disk group. As a result of the vxinstallchanges and the removal of the rootdg disk group requirement, theVxVM initialization process is now greatly simplified. The following is anexample of the entire dialogue.

# vxinstallVxVM uses license keys to control access. If you have a SPARCstorageArray (SSA) controller or a Sun Enterprise Network Array (SENA)controller attached to your system, then VxVM will grant you a limiteduse license automatically. The SSA and/or SENA license grants youunrestricted use of disks attached to an SSA or SENA controller, butdisallows striping, RAID-5, and DMP on non-SSA and non-SENA disks. Ifyou are not running an SSA or SENA controller, then you must obtain alicense key to operate.

Licensing information: System host ID: 80960386 Host type: SUNW,Ultra-4 SPARCstorage Array or Sun Enterprise Network Array: found

Do you want to use enclosure based names for all disks ?[y,n,q,?] (default: n) n

Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added diskarray DISKS, datype = Disk

Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added diskarray 5080020000034ed8, datype = SENASep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added diskarray 5080020000029e70, datype = SENASep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added diskarray 60020f200000c3670000000000000000, datype = T3



Sep 3 11:38:17 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-34 added diskarray 60020f200000c1930000000000000000, datype = T3Sep 3 11:38:18 ns-east-104 vxdmp: WARNING: VxVM vxdmp V-5-0-336Unlicensed array S/N 60020f200000c3670000000000000000 installedSep 3 11:38:18 ns-east-104 vxdmp: WARNING: VxVM vxdmp V-5-0-336Unlicensed array S/N 60020f200000c1930000000000000000 installed

Do you want to setup a system wide default disk group?[y,n,q,?] (default: y) n

The installation is successfully completed.

Licensing Requirements

The configuration used in the following example features twoSun StorEdge T3B arrays. According to the restrictions outlined in thevxinstall output, an additional license must be installed.

If there are no automatically licensed storage arrays attached, thevxinstall utility prompts you for a license key.

SPARCstorage Array or Sun Enterprise Network Array: Noarrays found. No valid licenses found.

VM lite/full license(s) is needed to proceed withvxinstall. See VERTIAS Volume Manager Installation Guidefor more details on obtaining and installing the licensekeys.

Are you prepared to enter a license key [y,n,q] (default:y) y

Enter your license key : 8ZYE-XJ4R-UPQ3-DOBL-PPO3-PNT6-PRVP

Note – You can also use the vxlicinst utility to manually install a licensekey at any time.



Verifying Licensed Features

After the system is initialized using the vxinstall command, you verifythe licensed features using the vxdctl license command. Following areexamples of several licensed configurations.

● Example 1 – The following is a system with only Sun StorEdgeA5200 arrays attached and an automatically generated license.

# vxdctl licenseLicensed features: Mirroring Root Mirroring Concatenation Disk-spanning DMP (multipath enabled) Striping (restricted to Sun Enterprise Network Array) RAID-5 (restricted to Sun Enterprise Network Array)

● Example 2 – The following is a system with only Sun StorEdgeA5200 arrays attached and with an additional manually installedlicense.

# vxdctl licenseAll features are available: Mirroring Root Mirroring Concatenation Disk-spanning Striping RAID-5 VxSmartSync DMP (multipath enabled)

● Example 3 – The following is a system with only Sun StorEdge T3Barrays attached and with a manually installed license.

# vxdctl licenseAll features are available: Mirroring Root Mirroring Concatenation Disk-spanning Striping RAID-5 VxSmartSync DMP (multipath enabled)

VxVM Post-Installation Environment



After the VxVM software is installed and initialized, you must be familiarwith the general environment to be an effective administrator.

VxVM System Files

During VxVM installation, the following changes are made to the/etc/system file, and VxVM startup files are added to several of the/etc run-level directories.

Changes to the /etc/system File

Entries are appended to the end of the /etc/system file to force-loadvarious VxVM drivers, as shown in the example:

* vxvm_START (do not remove)forceload: drv/vxdmpforceload: drv/vxioforceload: drv/vxspec* vxvm_END (do not remove)

Run-Level File Additions

A number of script files are added to the /etc run-level directories tocontrol the VxVM software when the system is booted or shut down.

● /etc/rc0.d/K50isisd

This script file stops the ISIS service daemon (vxsvc ) associated withthe VEA graphical interface when transitioning to run-level 0 duringa system shutdown.

● /etc/rcS.d/K29vxvm-vsshutdown

This script file sends a shutdown message to VEA clients

● /etc/rcS.d/K50isisd

This script file stops the ISIS service daemon (vxsvc ) associated withthe VEA graphical interface when transitioning to the single-userrun-level.



● /etc/rcS.d/S25vxvm-sysboot

This script file runs early in the boot sequence to configure the / and/usr volumes. This file also contains configurable debuggingparameters.

● /etc/rcS.d/S35vxvm-startup1

This script file runs after the / and /usr volumes are available. Italso makes other volumes available that are needed by the Solaris OSearly in the Solaris OS boot sequence, such as swap and /var .

● /etc/rcS.d/S85vxvm-startup2

This script file starts I/O daemons, rebuilds the /dev/vx/dsk and/dev/vx/rdsk directories, imports all disk groups, and starts allvolumes that were not started earlier in the boot sequence.

● /etc/rcS.d/S86vxvm-reconfig

This script file contains commands to execute the fsck utility on theroot partition before anything else on the system executes.

● /etc/rc2.d/S50isisd

This script file starts the ISIS service daemon (vxsvc ) associated withthe VEA graphical interface during the system boot sequence.

● /rc2.d/S94vxnm-vxnetd

This script file starts the vxnetd daemon if the VVR software optionis installed and licensed.

● /etc/rc2.d/S95vxvm-recover

This script file attaches and resynchronizes plexes and starts severalVxVM watch daemons, including: vxrelocd , vxcached , andvxconfigbackupd . You can also modify this file to change thedefault VxVM disk drive failure response from hot relocation to hotsparing.

● /etc/rc2.d/S96vradmind

This script file starts the vradmind daemon if the VERITAS VolumeReplicator (VVR) software option is installed and licensed.

● /etc/rc2.d/S96vxrsyncd

This script file starts the vxrsyncd daemon if VVR is installed andlicensed.



System Startup Messages

When the Solaris OS is booted and the VxVM startup files execute, severalimportant boot messages appear. In the following example, the keymessages are in bold type.

Rebooting with command: bootBoot device: disk:a File and args:SunOS Release 5.9 Version Generic_112233-03 32-bitCopyright 1983-2002 Sun Microsystems, Inc. All rights reserved.Use is subject to license terms.VxVM sysboot INFO V-5-2-3245 Starting restore daemon...VxVM sysboot INFO V-5-2-3242 starting in boot mode...

NOTICE: VxVM vxdmp V-5-0-34added disk array DISKS, datype = DiskNOTICE: VxVM vxdmp V-5-0-34added disk array 5080020000034ed8, datype = SENANOTICE: VxVM vxdmp V-5-0-34added disk array 5080020000029e70, datype = SENANOTICE: VxVM vxdmp V-5-0-34added disk array 60020f200000c3670000000000000000, datype = T3NOTICE: VxVM vxdmp V-5-0-34added disk array 60020f200000c1930000000000000000, datype = T3

Hostname: ns-east-104VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup...The system is coming up. Please wait.NIS domain name is Ecd.East.Sun.COMstarting rpc services: rpcbind keyserv ypbind done.Setting netmask of lo0 to 255.0.0.0Setting netmask of hme0 to 255.255.255.0Setting default IPv4 interface for multicast: add net 224.0/4: gatewayns-east-104syslog service starting.volume management starting.The system is ready.

ns-east-104 console login:



System Startup Processes

The following is an example of typical VxVM processes to help youunderstand how they are started during a system boot operation.

# ps -e |grep vxPID TTY TIME CMD

13 ? 0:04 vxconfig (vxconfigd)296 ? 0:05 vxsvc

473 ? 0:00 vxrelocd507 ? 0:00 vxrelocd

495 ? 0:00 vxnotify509 ? 0:00 vxnotify513 ? 0:00 vxnotify

474 ? 0:00 vxcached512 ? 0:00 vxcached

475 ? 0:00 vxconfig (vxconfigbackupd)494 ? 0:00 vxconfig (vxconfigbackupd)

The vxconfigd Daemon

The volume configuration daemon (vxconfigd ) is started by the/etc/rcS.d/S25vxvm-sysboot script early in the boot process. Thisdaemon must be running for the VxVM software to function.

The vxsvc Daemon

The vxsvc daemon is started by the /etc/rc2.d/S50isisd script. Thevxsvc daemon services requests from the VEA graphical interface.

The vxrelocd or vxsparecheck Daemons

One of these daemons is started by the S95vxvm-recover script duringthe boot process. The vxrelocd daemons monitor for VxVM failureevents and relocate failed subdisks. By default, the/etc/rc2.d/S95vxvm-recover script file starts the newer vxrelocdprocesses, which provide hot relocation. You can edit the file so that itstarts the older vxsparecheck process that provides only hot sparing.



The vxnotify Daemons

These daemons are started by either the vxrelocd or the vxsparecheckdaemons. The vxnotify daemons display events related to disk andconfiguration changes, as managed by the VxVM configuration daemon,vxconfigd .

The vxcached Daemons

The vxcached daemons manage cache volumes associated withspace-optimized snapshots. Cache space is added when needed.

The vxconfigbackupd Daemons

The vxconfigbackupd daemons are started by theetc/rc2.d/S95vxvm-recover script. These daemons record any VxVMdisk group configuration changes in the /etc/vx/cbr/bk directory.

The vxrecover Daemon

This daemon can be run by the /etc/rcS.d/S35vxvm-startup1 ,/etc/rcS.d/S85vxvm-startup2 , or the /etc/rc2.d/S95vxvm-recoverscript files during a system boot, depending on the need for volumerepair. The daemon terminates when the repair process is completed.

Volume-repair needs differ depending on several variables, such as asystem crash. The repair typically involves attaching plexes andresynchronizing mirrors.



System and User Executable Files

VxVM uses many different script and binary program files to performmanagement and administration functions. Many programs call otherprograms. Many of the script files and programs should not be directlyrun by a user.

VxVM Software in the /opt Directory

The following directories are present in the /opt directory after astandard VxVM software installation:

● /opt/VRTS

This directory contains the VxVM man pages.

● /opt/VRTSalloc

This directory contains the VxVM storage allocation software knownas Intelligent Storage Provisioning.

● /opt/VRTSddlpr

This directory contains software to enable and disable the devicediscovery layer feature associated with the VEA application.

● /opt/VRTSfspro

This directory contains VxFS registry information used by the vxsvcdaemon.

● /opt/VRTSob

This directory contains the VEA client and server software.

● /opt/VRTSlic

This directory contains VxVM license installation and verificationprograms

● /opt/VRTSvmpro

This directory contains VxVM registry information used by thevxsvc daemon.

● /opt/VRTSvxms

This directory contains library files for a DMP plug-in.

● /opt/VRTSvxvm

This directory contains the VERITAS technical manuals.



VxVM Software in the /usr/sbin Directory

Although there are many VxVM program files in the /usr/sbin directory,only a few are commonly used. They include vxassist , vxstat , vxinfo ,vxprint , vxtask , vxinstall , vxdg , vxdisk , and vxdiskadm .

# ls /usr/sbin/vx*vxadm vxdiskpr vxpool vxstatvxassist vxdmpadm vxprint vxtaskvxcache vxedit vxrecover vxtemplatevxclust vxexport vxrecover.wrap vxtracevxcmdlog vxibc vxrelayout vxtranslogvxconfigd vxinfo vxrlink vxtunevxdco vxinstall vxrsync vxusertemplatevxdctl vxiod vxrvg vxvolvxddladm vxmake vxscriptlog vxvoladmvxdg vxmemstat vxsd vxvoladmtaskvxdisk vxmend vxsnap vxvsetvxdiskadd vxnetd vxspvxdiskadm vxnotify vxspcshowvxdiskconfig vxplex vxstart_vvr

VxVM Software in the /etc/vx/bin Directory

The script and program files in the /etc/vx/bin directory are called byhigher-level user commands and are not commonly used directly.

# ls /etc/vx/binegettxt vxckdiskrm vxedvtoc vxr5vrfystrtovoff vxclustadm vxeeprom vxreattachugettxt vxclustipc vxencap vxrelocdvsshutdown vxcntrllist vxevac vxresizevxa5kchk vxconfigbackup vxldiskcmd vxrootvxapslice vxconfigbackupd vxmirror vxrootmirvxbadcxcld vxconfigrestore vxmksdpart vxslicervxbaddxcld vxconvarrayinfo vxnewdmname vxsparevxbootsetup vxcxcld vxparms vxsparecheckvxcached vxdarestore vxpartadd vxsplitlinesvxcap-part vxdevlist vxpartinfo vxswapctlvxcap-vol vxdevpromnm vxpartrm vxtaginfovxcdsconvert vxdisksetup vxpartrmall vxunrelocvxcheckda vxdiskunsetup vxprtvtoc vxunrootvxchksundev vxdxcld vxr5check



Verifying Initial Disk Drive Status

After a fresh VxVM installation, you should immediately assess thecurrent disk storage status. You should carefully research any existing filesystems before initiating encapsulation or initialization operations.

The vxdisk utility shows the current VxVM status of all disks drivesattached to the system.

# vxdisk listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:none - - online invalidc0t1d0s2 auto:none - - online invalidc2t1d0s2 auto:none - - online invalidc2t3d0s2 auto:none - - online invalidc2t5d0s2 auto:none - - online invalidc2t16d0s2 auto:none testdg01 testdg onlinec2t18d0s2 auto:none - - online invalidc2t20d0s2 auto:none - - online invalidc2t22d0s2 auto:none - - online invalidc3t32d0s2 auto:none - - online invalidc3t33d0s2 auto:sliced - - onlinec3t35d0s2 auto:none - - online invalidc3t37d0s2 auto:none - - online invalidc3t50d0s2 auto:none - - online invalidc3t52d0s2 auto:none - - online invalidc4t1d0s2 auto:none - - online invalidc5t1d0s2 auto:none - - online invalid

Disks that show a status of online invalid are not under VxVM control.Disks that show a status of online have been initialized, but are notassigned to a disk group. When online disks are added to a disk group,they are assigned a name which appears in the DISK column. By default,the disk name is derived from the name of the disk group.

What is not evident is that slice 7 of the disk c2t16d0 is mounted with afile system. You must plan for all existing data before proceeding withdisk drive encapsulation or initialization.

Preparing for VxVM Disk Drive Management



Disk drives can be placed under VxVM software control in twofundamentally different ways. If a disk drive has existing file systems,VxVM will prompt you to encapsulate the disk, thereby preserving anyexisting data. An alternate method of placing disks under VxVM controlis to initialize the disk drive, which destroys existing data.

You configure special files to protect specific storage devices frominadvertent use by VxVM command-line programs.

Disk Drive Initialization Process

When any disk drive is initialized by VxVM, the disk drive isrepartitioned into a standard VxVM configuration. The initialized diskdrive has only two slices, slices 3 and 4. One slice is very small and storesVxVM configuration records. The other slice is the rest of the disk drive.

When a disk drive is initialized by VxVM, all existing data is lost.

Note – When you use VxVM software such as the vxdiskadm utility tomanage disk drives, the software takes extensive steps to detect anyexisting data structures.

VxVM Initialized Disk Format

As shown in Figure 3-2 on page 3-25, a physical disk drive that has beeninitialized by VxVM is divided into two sections called the private regionand the public region:

● The private region is used for configuration information.

● The public region is used for data storage.



By default, VxVM uses partitions 3 and 4 for the private and publicregions.

Figure 3-2 VxVM Physical Disk Drive Layout

VxVM requires a single cylinder for the private region. On larger drives,one cylinder can store more than an Mbyte.

The public region is configured to be the rest of the physical disk drive.The volume table of contents (VTOC) listing for a freshly initializedVxVM disk drive is shown in the following example. Some output isomitted for clarity.

# prtvtoc /dev/dsk/c2t22d0s2......* First Sector Last* Partition Tag Flags Sector Count Sector

2 5 01 0 17682084 17682083 3 15 01 0 3591 3590 4 14 01 3591 17678493 17682083

Private Region

Public Region

VxVM User Data Storage

VxVM configuration andmanagement information



Private Region Disk Header

The disk header is a block stored in the private region of a disk drive thatdefines the following import properties of the disk drive:

● Current host ownership of the disk drive

When a disk drive is part of a disk group that is in active use by aparticular host, the disk drive is stamped with that host’s host ID(host name). If another VxVM system attempts to access the diskdrive, its VxVM daemons detect that the disk drive has anonmatching host ID (host name) and disallows access until the firstsystem releases the disk drive.

● Disk identifier

A 64-byte unique identifier is assigned to a physical disk drive whenits private region is initialized.

Private Region Configuration Database

The configuration database (sometimes called configdb ) containsinformation about the configuration of a particular disk group. By default,VxVM keeps four copies of the configuration database per disk group toavoid any possibility of losing the disk group information. Each copy ofthe configuration database contains the following information:

● dgname – The disk group name is assigned by the administrator.

● dgid – The disk group ID is a 64-byte universally unique identifierthat is assigned by VxVM to a disk group when the disk group iscreated. This identifier is in addition to the disk group name.

● Records – VxVM virtual structure information for the disk group.

Kernel Log

The kernel log is kept in the private region on the disk drive and iswritten by the VxVM kernel. The kernel log contains records describingcertain types of actions, such as transaction commits, plex detachesresulting from I/O failures, dirty-region log failures, the first write to avolume, and volume close information. The kernel log is used after acrash or clean reboot to recover the state of the disk group just prior to thecrash or reboot.



Disk Drive Encapsulation Process

When any disk drive is encapsulated by VxVM, existing file systems arepreserved and a small portion of the disk drive is used for VxVMoverhead. If the disk drive is the system boot disk, VxVM is aware of thisdistinction and takes special care to preserve all data.

Generally, you do not encapsulate disk drives with existing data unlessyou want to increase availability or performance of the data through theuse of software RAID structures.

When a disk with existing data structures (such as a mounted file system)is encapsulated, VxVM analyzes the disk structure and takes measures topreserve all existing data and the disk partition map found on block zero.

Disk Drive Before Encapsulation

The following shows the partition map and mount information of aSolaris OS disk drive with a mounted file system. For clarity, some outputis omitted.

# prtvtoc /dev/rdsk/c2t16d0s2First Sector Last Mount

Partition Tag Flags Sector Count Sector Directory

2 5 01 0 17682084 176820837 0 00 0 2100735 2100734 /Test

# more /etc/vfstab |grep Test/dev/dsk/c2t16d0s7/dev/rdsk/c2t16d0s7 /Test ufs 2 yes -



Disk Drive After Encapsulation

After encapsulation, using a VxVM utility such as vxdiskadm , the disk’spartition map and mount information are modified as shown in thefollowing example.

# prtvtoc /dev/rdsk/c2t16d0s2First Sector Last Mount

Partition Tag Flags Sector Count Sector Directory

2 5 01 0 17682084 176820833 14 01 0 17682084 176820834 15 01 17674902 7182 17682083

# more /etc/vfstab |grep Test/dev/vx/dsk/testdg/Test /dev/vx/rdsk/testdg/Test/Test ufs 2 yes -#NOTE: volume Test (/Test) encapsulated partition c2t16d0s7



Protecting Storage Devices From Usage

Before initializing or encapsulating storage devices, you can automaticallyexclude storage devices from VxVM use using two different methods.Depending on which method you use, the effects range from limitedexclusion to total exclusion. The exclusion files prevent the vxdiskadmutility and the VEA GUI from using the excluded devices.

Determining the VERITAS Storage Types

Although the format utility displays all known storage devices, you mustuse the vxdmpadm command as shown to display the storage types andenclosure names for use in device exclusion.

# vxdmpadm listctlr allCTLR-NAME ENCLR-TYPE STATE ENCLR-NAME=====================================================c0 Disk ENABLED Diskc2 SENA ENABLED SENA0c3 SENA ENABLED SENA1c4 T3 ENABLED T30c5 T3 ENABLED T31

# vxdmpadm listenclosure allENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE=========================================================================Disk Disk DISKS CONNECTED DiskSENA0 SENA 5080020000034ed8 CONNECTED A/ASENA1 SENA 5080020000029e70 CONNECTED A/AT30 T3 60020f200000c3670000 CONNECTED -T31 T3 60020f200000c1930000 CONNECTED -

The controller names (CTLR-NAME) and enclosure names (ENCLR-NAME) areused in the appropriate exclude files.

The disk array type field (datype ) displayed during the system bootprocess equates to the ENCLR_TYPEfield displayed in the output of thevxdmpadm command.



Limited Exclusion

You can exclude enclosure, controllers, or devices from use by thevxinstall and vxdiskadm utilities by manually modifying three specialfiles in the /etc/vx directory. The following describes the three files andan explanation of their intent.

● enclr.exclude

Each line of the enclr.exclude file specifies an enclosure to beexcluded, for example, sena1 .

● cntrls.exclude

Each line of the cntrls.exclude file specifies the address of acontroller to exclude, for example, c2 .

● disks.exclude

Each line of the disks.exclude file specifies a disk drive to exclude,for example, c0t3d0 .

Caution – The three manual exclusion files do not prevent other VxVMcommands from seeing and operating on the storage devices. You can stillsee and perform operations on all the devices using VxVM commands,such as vxdg , vxdisk , vxdisksetup , and vxassist .

The manual exclusion files are used to protect specific storage devicesfrom being initialized or encapsulated after an initial software installation.

The exclusion files are also useful to protect specific SAN storage devices.You can remove or rename the manual exclusion files after you completethe initialization or encapsulation process.



Manual Exclusion File Formats

The following example shows the format using each of the differentmanual exclusion files.

# more /etc/vx/enclr.excludeSENA0

# more /etc/vx/cntrls.excludec4

# more /etc/vx/disks.excludec0t0d0c0t1d0

In the previous example, a Sun StorEdge A5000 array (SENA0) is excluded,a Sun StorEdge T3 array connected to controller c4 is excluded, and twointernal system disk drives are excluded.

If you try to initialize all attached storage using the vxdiskadm utility, yousee exclusion messages similar to the following:

Select disk devices to add:[<pattern-list>,all,list,q,?]all

VxVM INFO V-5-2-428This disk that you specified has been excluded by the/etc/vx/cntrls.exclude file:

/dev/vx/rdmp/c4t1d0s2

These disks that you specified have been excluded by the/etc/vx/disks.exclude file:

/dev/vx/rdmp/c0t0d0s2 /dev/vx/rdmp/c0t1d0s2

These disks that you specified have been excluded by the/etc/vx/enclr.exclude file:

/dev/vx/rdmp/SENA0_0s2 /dev/vx/rdmp/SENA0_1s2/dev/vx/rdmp/SENA0_2s2 /dev/vx/rdmp/SENA0_3s2/dev/vx/rdmp/SENA0_4s2 /dev/vx/rdmp/SENA0_5s2/dev/vx/rdmp/SENA0_6s2



Global Exclusion

There are two additional files, /etc/vx/vxvm.exclude and/etc/vx/vxdmp.exclude , that should not be manually edited. They aremodified indirectly using the vxdiskadm utility option 17, Preventmultipathing/Suppress devices from VxVM’s view .

Storage that is suppressed using the vxdiskadm utility, is removed fromthe view of all VxVM software. This feature is intended for complexconfigurations that require the coexistence of AP, DMP, and SAN devices.

The following is an abbreviated example of excluding a controller fromVxVM’s view using the vxdiskadm utility.

# vxdisk listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:none - - online invalidc0t1d0s2 auto:none - - online invalidc2t1d0s2 auto:none - - online invalidc2t3d0s2 auto:none - - online invalidc2t5d0s2 auto:none - - online invalidc2t16d0s2 auto:none - - online invalidc2t18d0s2 auto:none - - online invalidc2t20d0s2 auto:none - - online invalidc2t22d0s2 auto:none - - online invalidc3t32d0s2 auto:none - - online invalidc3t33d0s2 auto:none - - online invalidc3t35d0s2 auto:none - - online invalidc3t37d0s2 auto:none - - online invalidc3t50d0s2 auto:none - - online invalidc3t52d0s2 auto:none - - online invalidc4t1d0s2 auto:none - - online invalidc5t1d0s2 auto:none - - online invalid

# vxdiskadm........Select an operation to perform: 17

Exclude DevicesMenu: VolumeManager/Disk/ExcludeDevices VxVM INFO V-5-2-1239



This operation might lead to some devices being suppressedfrom VxVM’s view or prevent them from being multipathed byvxdmp (This operation can be reversed using the vxdiskadmcommand).

Do you want to continue ? [y,n,q,?] (default: y) y

Volume Manager Device OperationsMenu: VolumeManager/Disk/ExcludeDevices

1 Suppress all paths through a controller from VxVM’s view2 Suppress a path from VxVM’s view3 Suppress disks from VxVM’s view by specifying a VID:PID4 Suppress all but one paths to a disk5 Prevent multipathing of all disks on a controller by VxVM6 Prevent multipathing of a disk by VxVM7 Prevent multipathing of disks by specifying a VID:PID8 List currently suppressed/non-multipathed devices........Select an operation to perform: 1

Exclude controllers from VxVMMenu: VolumeManager/Disk/ExcludeDevices/CTLR-VXVM Use this operation to exclude all paths through acontroller from VxVM.

Enter a controller name [<ctlr-name>,all,list,list-exclude,q,?] c4

All paths to the following enclosures through controller c4will be hidden from VxVM as a result of this operation: T30

Continue operation? [y,n,q,?] (default: y) y

Sep 11 14:21:57 ns-east-104 vxdmp: NOTICE: VxVM vxdmp V-5-0-110 disabled controller/pci@6,4000/pci@4/SUNW,qlc@4/fp@0,0 connected to disk array60020f200000c3670000000000000000 VxVM NOTICE V-5-2-1323The controller c4 will be disabled. The entries for pathson controller c4 will still be visible through VxVMutilities in a disabled state till a reboot. After a rebootthese entries will not be seen.....



....# reboot........# vxdisk listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:none - - online invalidc0t1d0s2 auto:none - - online invalidc2t1d0s2 auto:none - - online invalidc2t3d0s2 auto:none - - online invalidc2t5d0s2 auto:none - - online invalidc2t16d0s2 auto:none - - online invalidc2t18d0s2 auto:none - - online invalidc2t20d0s2 auto:none - - online invalidc2t22d0s2 auto:none - - online invalidc3t32d0s2 auto:none - - online invalidc3t33d0s2 auto:none - - online invalidc3t35d0s2 auto:none - - online invalidc3t37d0s2 auto:none - - online invalidc3t50d0s2 auto:none - - online invalidc3t52d0s2 auto:none - - online invalidc5t1d0s2 auto:none - - online invalid

# more /etc/vx/vxvm.excludeexclude_all 0paths#controllersc4 /pci@6,4000/pci@4/SUNW,qlc@4/fp@0,0#product#pathgroups#

Note – The format utility still sees the c4 controller and can use itnormally.

Installing the VEA


Installing the VEA

The VEA software is composed of server and client software. The serversoftware must reside on the VxVM server. The client software can beinstalled on the VxVM server, but it can also be installed on one or moreremote administration systems.

The VEA software is a graphical VxVM administrative interface.

As shown in Figure 3-3, you can use the VEA client software to:

● Run the client software remotely on an administration system

● Run the client software on the VxVM server and display the VEAGUI locally

● Run the client software on the VxVM server and display the VEAGUI remotely

Figure 3-3 VEA Server/Client Relationship

To remotely display an application, you set the local DISPLAY variable topoint to a remote system: setenv DISPLAY 129.148.152.93:0.0 .

Local DisplayMonitor

VM Server

Network

VEA ClientSoftware

VEA ClientSoftware

VEA ServerSoftware

Remote System

Disk

Disk Groups

Volumes

Installing the VEA


VEA Software Initialization

The VEA GUI is a Java™ technology application that can be run locally onthe VxVM server or remotely on any networked system. The VEA clientsoftware can run in any Java runtime environment.

VEA Software Package Installation

The VRTSob and VRTSobgui packages are both installed on the VxVMserver, which includes the VEA client interface and the VEA serversoftware. You can start the client software on the server. However, theVEA client software package, VRTSobgui , is more commonly installed ona remote administration workstation.

VEA Server Software Startup

The VEA server software daemon, vxsvc , is automatically started at boottime by the /etc/rc2.d/S50isisd script.

You can manually stop and start the VEA server software on the VxVMserver using the /etc/init.d/isisd stop (or start ) command options.

Table 3-2 shows the options you can use to control the/opt/VRTSob/bin/vxsvc program directly:

Table 3-2 The /opt/VRTSob/bin/vxsvc Program Options

Option Function

-k Shuts down the vxsvc daemon.

-m Returns the current VEA service state.

-v Prints the version of the VEA service.

-r Specifies the registry file to be used.

-n Disables client connection authentication.

-f Starts the vxsvc daemon in the foreground for debugging.

Installing the VEA


VEA Client Software Startup

You start the VEA client software by manually running the/opt/VRTSvmsa/bin/vmsa script file. The VEA client software can bestarted and displayed on the server, started on the server and remotelydisplayed on another system, or loaded and started on a remote system.

Start the VEA client software by running the /opt/VRTSob/bin/veascript file:

# /opt/VRTSob/bin/vea &

You can also use the startup options shown in Table 3-3 as needed.

Table 3-3 Startup Options

Option Function

-v Prints the version of the VEA client software.

-d Starts the client software in debug mode.

-s Specifies the fully qualified class name for the skin.

-c Specifies the absolute path of the configuration file for thecurrent skin.

-p Specifies the frequency (in minutes) for the VEA GUI tocheck the status of the VEA server processes.

-cp Specifies the user’s class path (for libraries) to be appendedto the application’s class path.

-DVAR Specifies the environment variable to be set pointing to theJava software directory. This option works only onMicrosoft Windows.

Installing the VEA


Host Connection Window

When the client software starts, a Connection window, as shown inFigure 3-4, is displayed. Enter the name of the VxVM server and theappropriate authentication information.

Figure 3-4 VEA Host Connection Window

If you enable the Remember password feature, the next time you connectyou select the hostname from the pull-down menu. The Username andPassword fields are automatically configured.

Installing the VEA


Resolving Low-Bandwidth Access Problems

If you need to access a VxVM server using a low-bandwidth connection,such as a dial-up modem, you must take special measures to prevent fatalconnection errors.

Low-Bandwidth Download Failures

During the initial VEA client-to-server connection cycle, approximately7 Mbytes of VEA plug-ins are downloaded from the VxVM server into theclient $HOME/.VRTSob directory. In a low-bandwidth environment, thedownload process commonly fails approximately 15 minutes into theprocess with cryptic and misleading errors such as File System Full andJava exception errors.

Low-Bandwidth Corrective Measures

Perform the following steps to permanently eliminate low-bandwidthdownload failures:

1. The first time you start the VEA client software, increase the hostdiscovery frequency from the default value of 3 minutes toapproximately 45 minutes.

% vea -p 45 &

2. In the VEA, on the Tools menu, perform the following steps:

a. Select Preferences, and then select General.

b. Disable the Delete temporary files on exit feature.

After the first-time download, the plug-in download filesremain in the $HOME/.VRTSob directory and do not need to bedownloaded again.

3. Initiate the connection to the VxVM server.

The first connection might take up to 20 minutes to download the plug-infiles. Subsequent connections take less than 2 minutes. Administrativeoperations response times are acceptable even through a 56-Kbytemodem.

If the connection fails while performing administrative tasks, reconnectusing the toolbar Connect icon.

Using Basic VEA Features



Before using the VEA GUI for administrative duties, you must be familiarwith the VEA components and options.

Main Window Functional Areas

The VEA GUI has distinct functional areas, as shown in Figure 3-5.

Figure 3-5 VEA GUI

Note – The Actions menu entries change according to the type of objectsbeing displayed in the grid area. Some of the toolbar icons’ functions alsochange as different objects are displayed in the grid area.

Menu bar Tool bar

Object tree Grid area Message area



Menu Bar Functions

The menu bar in the VEA GUI has the functions shown in Figure 3-6. TheActions menu entries change according to the type of objects currentlydisplayed in the grid area.

The tear off menu opens a separate window relating to the tabs in thecurrent grid area display. The tear off feature is useful when analyzingmultiple aspects of a grid area display.

Figure 3-6 Menu Bar Functions



Toolbar Buttons

The toolbar, shown in Figure 3-7, provides direct access to general VEAfunctions. Some of the toolbar selections change according to the type ofobjects being displayed in the grid area.

Figure 3-7 Toolbar Icons

All the toolbar functions are available elsewhere in menus, but the toolbaroffers a convenient way to access commonly used functions.

The toolbar buttons perform the following functions:

● Connect to a VEA server

● Disconnect from the current VEA server

● Create a new volume

● Create a new disk group

● Search for virtual objects by criteria



Object Tree Pane

The object tree window, shown in Figure 3-8, has an icon for every type ofVEA object that is referenced during VxVM administration. The objectsare arranged in a hierarchy starting with VxVM servers at the top.

You can expand small nodes on the object tree branches to displaydetailed information about the node’s subject.

Figure 3-8 Object Tree Expansion



Grid Area Pane

When you select an object tree icon with the first mouse button, expandedconfiguration information about that object appears in the grid area.

The grid area display, shown in Figure 3-9, results from selectingEnclosure in the object tree.

Figure 3-9 Grid Area



Resizing Display Panes

The object tree and message areas, shown in Figure 3-10, have two typesof pane resizing controls: arrows for fixed size changes and a slider forvariable size changes.

Figure 3-10 Pane Resizing Controls

In the previous example, the object tree pane has been widened using itsresizing bar, and the message area has been fully collapsed using itsresizing arrow so that it is no longer visible.

Pane resizing arrows Pane resizing bar



Modifying Preferences

The Preferences window, accessed by selecting Preferences from the Toolsmenu, contains three tabs.

● Appearance tab (shown in Figure 3-11) – Used to modify the generallook and feel of the VEA GUI.

Figure 3-11 Preferences Window – Appearance Tab

● General tab – Used to modify settings so that any downloadedplug-in files are retained locally when you exit VEA. This can save 45minutes of download time if you are running the VEA clientsoftware from a dial-up modem.

● Volume Manager General tab – Present only when you are connectedto a VxVM server and modifies several minor features.



Customizing the Grid Display

Using drag-and-drop and the display tabs, you can modify the type ofdata displayed in the grid area, and you can customize the location of thedata status columns.

Rearranging Grid Display Status Columns

You can temporarily rearrange display columns in the grid usingdrag-and-drop on the column headers. The effect is temporary.

Using the Grid Display Data Tabs

Each type of grid display has data tabs associated with it. You can use thetabs to display different information related to the current grid display.

By default, when a disk group is displayed in the grid area, the Disks tabis active. In the example shown in Figure 3-12, the Volumes tab is selectedso that all volumes associated with the sdgA disk group are displayed.

Figure 3-12 Grid Display Data Tabs



Examining VEA Command Logs

The VxVM server logs commands resulting from VEA client softwareoperations in the /var/vx/isis/command.log file. The log file is a usefultool for learning VxVM command-line program syntax.

Examples of typical command log entries:

Description: Add Disk Date: Tue Dec 11 20:09:15 2003 Command: /usr/sbin/vxdg init sdgA sdgA00=c3t32d0s2sdgA01=c3t33d0s2 sdgA02=c3t35d0s2 sdgA03=c3t37d0s2 Output: Exit Code: 0

Description: Rename Disk Date: Tue Dec 11 20:26:15 2003 Command: /usr/sbin/vxedit -g sdgA -v rename sdgA01sdisk01 Output: Exit Code: 0

Description: Create Volume Date: Wed Dec 12 11:51:41 2003

Command: /usr/sbin/vxassist -g sdgA -b make newvol_011638400s layout=nostripe Output: Exit Code: 0

Note – An exit code of 0 means the command executed successfully. Seethe vxintro man page for a list of standard exit codes.



Using the VEA Search Tool

You can use the VEA Search form, as shown in Figure 3-13, to compileconfiguration summaries for supported VxVM objecting includingcontrollers, enclosures, disks, disk groups, volumes, and file systems.

In the example shown in Figure 3-13, a search was made for all volumesat least 1 Gbyte in size.

Figure 3-13 VEA Search Form

To use the search tool, perform the following steps:

1. Use the pull-down lists and type-in fields to define the criteria to usein your search.

2. Click Add to add the criteria to the criteria list.

3. Repeat the previous two steps for each criteria to use.

4. Click the Search Now button to perform the search and display theresults.

Decoding VxVM Error Messages


Decoding VxVM Error Messages

The VxVM 4.0 software error message format is organized differently thanin previous VxVM releases. All VxVM 4.0 error messages contain uniqueerror numbers that are listed in the VERITAS Volume Manager 4.0Troubleshooting Guide.

The new error message numbers are grouped into two sections as follows:

● Errors V-5-0-2 through error V-5-0-386

● Errors V-5-1-90 through error V-5-1-5929

Error message are composed of the following 5 fields:

● Product name

● Product component

● Severity level

● Error message number

● Message text

An example of a typical error message follows.

# vxdg deport dgSPVxVM vxdg ERROR V-5-1-584 Disk group dgSP: Some volumes inthe disk group are in use

Most of the error listings in the VERITAS Volume Manager 4.0Troubleshooting Guide contain a description of the error and, if appropriate,suggested administrative actions.

Exercise: Configuring VxVM




● Review key lecture points

● Install the VxVM software

● Verify the VxVM system files

● Evaluate the storage configuration

● Install the VEA client software

● Start the VEA client software

● Customize the VEA client appearance

● Navigate the VxVM technical manuals

● Use the VxVM error numbering system

Preparation

If you are installing the VxVM software on a central server, yourinstructor must perform the installation as a demonstration.

1. Ask your instructor for the location of the VxVM software.

VxVM location: ______________________________

2. If the lab system does not have certain Sun storage array modelsattached, you are asked for a license string during the VxVMVRTSvlic package installation. Ask your instructor for ademonstration license string.

Demo license: _____________________________

The Adobe Acrobat Reader program must be available to the students to examine the PDF versions of theVERITAS documents.

Discuss the classroom configuration and the process students should follow until everyone is clear about howto proceed.

Divide students into groups depending on how many systems you have available for VxVM installation.

You can install and initialize the VxVM software yourself while students watch. It is up to you to determinewhich method works best in your classroom configuration.



Task 1 – Reviewing Key Lecture Points

Answer the following questions about the major areas of concern duringthe VxVM installation and initialization process.

1. Which of the following are not part of planning a VxVM installation?

a. Estimating system downtime

b. Backing up your system

c. Running a system performance check

d. Verifying correct boot disk configuration

e. Determining licensing requirements

The answer is c.

2. Which of the following are useful for researching patchrequirements?

a. The patch README notes

b. The prtvtoc command

c. The patchadd command

d. The PatchPro tool

e. The /var/adm directory contents

The answers are a, c, and d.

3. Which of the following VxVM packages should be installed first?

a. VRTSvmman

b. VRTSob

c. VRTSvxvm

d. VRTSvlic

e. VRTSvmdoc

The answer is d.



4. What is the correct order for the following installation steps?

a. Install storage firmware patches.

b. Install VxVM software patches.

c. Encapsulate the system boot disk.

d. Install required operating system patches.

e. Install VxVM software packages.

The correct order is d, a, e, b, and c.

5. Which slices does VxVM use when it initializes a disk drive?

a. Slices 6 and 7

b. Slices 2 and 4

c. Slices 3 and 4

d. Slices 0 and 1

e. Slices 4 and 7

The answer is c.

6. What is the default amount of space that VxVM requires for a diskdrive’s private region?

a. 4800 sectors

b. 2048 sectors

c. 1648 sectors

d. 4096 sectors

e. 1024 sectors

The answer is b.

7. When should you use VxVM disk drive encapsulation instead ofdisk drive initialization?

__________________________________________________________________________________________________________________________

The answer is when you want to preserve existing data on the disk drive.



8. What is the primary purpose of the enlr.exclude ,cntrls.exclude , and disks.exclude files?

a. To make storage devices invisible to all VxVM software

b. To make storage devices invisible to the AP software

c. To make storage devices invisible to the vxinstall andvxdiskadm utilities

d. To make storage devices invisible to the DMP software

The answer is c.

9. Which of the following are not VxVM system processes?

a. vxnotify

b. vxdiskadm

c. vxconfigd

d. vxrecover

e. vxdctl

The answers are b and e.

10. Which directory contains the VxVM technical manuals?

a. /opt/VRTSdocs

b. /etc/vx/docs

c. /opt/VRTSvxvm/docs

d. /usr/vx/docs

The answer is c.



Task 2 – Installing the VxVM Software

To install the VxVM software, complete the following steps:

1. Log in as user root on the system attached to the storage arrays.

2. Change to the VxVM installation package location.

3. Install the license package and the basic VxVM packages.

# pkgadd -d . VRTSvlic VRTSvxvm VRTSvmdoc VRTSvmman

4. Install the VEA software packages and reference the customadministration file, VRTSobadmin.

# pkgadd -a ../scripts/VRTSobadmin -d . VRTSob VRTSobgui

5. Install the remaining VxVM software packages.

# pkgadd -d . VRTSalloc VRTSddlpr VRTSvmpro VRTSfspro

Note – There is an additional localization package, VRTSmuob, that addssome localized VEA information in French, Japanese, or German. TheVRTSmuobpackage can be added at any time if needed.



6. During the installation, answer yes to all questions unless you feelthere are serious problems.

If there are problems, ask your instructor for advice.

7. After the package installation is completed, log out and log back inagain.

The installation process usually alters search-path values.

8. Use the vxinstall utility to initialize VxVM and answer thefollowing key questions:

Do you want to use enclosure based names for all disks?[y,n,q,?] (default: n) nDo you want to setup a system wide default disk group?[y,n,q,?] (default: y) n

Note – If your system does not have Sun SPARCstorage 100 orSun StorEdge A5x00 arrays attached, you are prompted for a license key.

9. Verify that the vxprint status command completes without errors.

# vxprint

10. Verify that the following VxVM command and man page locationsare configured in your environment. Edit the .profile or .cshrcfiles as necessary.

# echo $PATH/usr/sbin:/usr/bin:/opt/VRTS/bin:/opt/VRTSalloc/bin:/opt/VRTSvlic/bin:/opt/VRTSvmpro/bin:/etc/vx/bin

# echo $MANPATH/usr/man:/opt/VRTS/man:/opt/VRTSvlic/man

The correct syntax for .profile file entries in the Bourne shellenvironment is:

PATH=$PATH:/opt/VRTS/bin:/opt/VRTSalloc/bin:/opt/VRTSvlic/bin:/opt/VRTSvmpro/bin /etc/vx/bin

MANPATH=/usr/man:/opt/VRTS/man:/opt/VRTSlic/manexport PATH MANPATH

The correct syntax for .cshrc file entries in the C shell environment is:

set path = ($path /opt/VRTS/bin /opt/VRTSalloc/bin/opt/VRTSvlic/bin /opt/VRTSvmpro/bin /etc/vx/bin)

setenv MANPATH ’/usr/man:/opt/VRTS/man:/opt/VRTSlic/man’



Task 3 – Verifying the VxVM System Files

Perform the following steps to verify the post-installation files:

1. Verify that the following directories are present in the /opt directory.

# ls /optSUNWits VRTS VRTSddlpr VRTSob VRTSvmpro VRTSvxvmSUNWrtvc VRTSalloc VRTSfspro VRTSvlic VRTSvxms

2. Verify that the following software is present in the /usr/sbindirectory.

# cd /usr/sbin# ls vx*vxadm vxdiskpr vxpool vxstatvxassist vxdmpadm vxprint vxtaskvxcache vxedit vxrecover vxtemplatevxclust vxexport vxrecover.wrap vxtracevxcmdlog vxibc vxrelayout vxtranslogvxconfigd vxinfo vxrlink vxtunevxdco vxinstall vxrsync vxusertemplatevxdctl vxiod vxrvg vxvolvxddladm vxmake vxscriptlog vxvoladmvxdg vxmemstat vxsd vxvoladmtaskvxdisk vxmend vxsnap vxvsetvxdiskadd vxnetd vxspvxdiskadm vxnotify vxspcshowvxdiskconfig vxplex vxstart_vvr

3. Verify that the following software is present in the /etc/vx/bindirectory.

# ls /etc/vx/binegettxt vxckdiskrm vxedvtoc vxr5vrfystrtovoff vxclustadm vxeeprom vxreattachugettxt vxclustipc vxencap vxrelocdvsshutdown vxcntrllist vxevac vxresizevxa5kchk vxconfigbackup vxldiskcmd vxrootvxapslice vxconfigbackupd vxmirror vxrootmirvxbadcxcld vxconfigrestore vxmksdpart vxslicervxbaddxcld vxconvarrayinfo vxnewdmname vxsparevxbootsetup vxcxcld vxparms vxsparecheckvxcached vxdarestore vxpartadd vxsplitlinesvxcap-part vxdevlist vxpartinfo vxswapctlvxcap-vol vxdevpromnm vxpartrm vxtaginfovxcdsconvert vxdisksetup vxpartrmall vxunrelocvxcheckda vxdiskunsetup vxprtvtoc vxunrootvxchksundev vxdxcld vxr5check



Task 4 – Evaluating the Storage Configuration

Perform the following steps to evaluate your storage configuration:

1. Type the vxdmpadm listctlr all command and the formatcommand to evaluate your storage configuration. Use the Control-Dkeyboard sequence to exit from the format utility.

2. Record the controller name, enclosure name, and number of devicesassociated with each enclosure.

Controller Number Enclosure Name Number of Devices



Task 5 – Installing the VEA Client Software

Use the following steps to install the VEA client software on youradministration system:

1. On your administration workstation, change to the location of theVRTSobgui software package.

Your instructor should have given you the location of the VxVMsoftware in a previous exercise.

2. Install the VEA client software on your administration system:

# pkgadd -d . VRTSobgui

Processing package instance <VRTSobgui> from </tmp>

VERITAS Enterprise Administrator

The selected base directory </opt/VRTSob> must exist beforeinstallation is attempted.

Do you want this directory created now [y,n,?,q] yUsing </opt/VRTSob> as the package base directory.## Processing package information.## Processing system information.## Verifying disk space requirements.## Checking for conflicts with packages already installed.## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-userpermission during the process of installing this package.

Do you want to continue with the installation of <VRTSobgui> [y,n,?] y



Task 6 – Starting the VEA Client Software

To start the VEA client software, complete the following steps:

1. Start the VEA client software on your administration system:

# vea &

2. Perform the following steps in the Session Initialization window, asshown in Figure 3-14:

a. Type the VxVM server host name.

b. Type the user name root .

c. Type the password.

d. Ensure that the Remember password box is checked.

Figure 3-14 VEA Client Initialization

The VEA GUI initializes after a short delay.



Task 7 – Customizing the VEA GUI Appearance

Perform the following steps to customize the appearance of the VEA GUI:

1. Click the left mouse button on the Tools menu, and then selectPreferences from the pop-up menu.

2. In the Preferences window, click the Appearance tab.

3. Click Items and select Toolbar.

4. Click Icons and Text.

5. Click OK.

Task 8 – Navigating the VxVM Technical Manuals

There are several technical manuals installed in the /opt/VRTSvxvm/docsdirectory. Perform the following steps to examine the content of themanuals:

1. Change to the VxVM manual directory.

# cd /opt/VRTSvxvm/docs# lspitc_ag.pdf vxvm_hwnotes.pdf vxvm_tshoot.pdfvxvm_ag.pdf vxvm_ig.pdf vxvm_ug.pdfvxvm_cdsag.pdf vxvm_ispag.pdf

2. View the contents of the PDF manuals using the Adobe AcrobatReader (acroread ) program

The hypertext links and search features of the Adobe Acrobat Readerare useful when you are looking for specific information in themanuals. Use the Control-F sequence to enable the Adobe AcrobatFind window.



3. Use the Adobe Acrobat Find feature to answer the followingquestions for each of the indicated VxVM technical manuals.

a. Answer the following questions for the vxvm_hwnotes.pdfVxVM manual:

1. Which Sun storage array models are supported for bootdisk mirroring?

_____________________________________________________

_____________________________________________________

_____________________________________________________

2. What is the mirroring configuration requirement when youupgrade T3B firmware?

_____________________________________________________

_____________________________________________________

_____________________________________________________

b. Answer the following questions for the vxvm_ig.pdf VxVMmanual:

1. Is the Solaris OS 2.6 release supported by the currentVxVM software? _____________________________________

2. When is the installvm script used?

_____________________________________________________

_____________________________________________________

_____________________________________________________



Task 9 – Using the VxVM Error Numbering System

Perform the following steps to research the use of the VERITAS uniqueerror numbering system.

Use the Adobe Acrobat Reader (acroread ) program to review Chapter 5,Error Messages, of the VERITAS Volume Manager Troubleshooting Guide.

Answer the following questions:

1. What are the five fields in the error message format?

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

2. What are the six severity levels?

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

3. What is the suggested action for error number V-5-0-145 ?

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

Exercise Summary


Exercise Summary

?!



● Experiences


● Interpretations


● Conclusions


● Applications



Module 4

VERITAS Volume Manager BasicOperations

Objectives


● Describe the function of VxVM disk groups

● List disk group administrative operations including:

● Initialize disk drives for VxVM use

● Create disk groups

● Add and remove disk drives for a disk group

● Import and deport disk groups

● Destroy a disk group

● Rename VxVM disk drives

● Administer disk groups using the vxdiskadm utility

● Administer disk groups using command-line programs

● Administer disk groups using the VEA GUI

VxVM Disk Group Functions



A disk group is a collection of VxVM disk drives that share a commonconfiguration. Volume structures are created within a disk group and areconfined to the disk drives that are associated with that disk group.

Primary Functions of a Disk Group

Disk groups have two primary functions:

● Assist administrative management

● Provide increased data availability

Easier Administration

Disk groups enable you to group disk drives into logical collections foradministrative convenience. You can group them according to departmentor application. For example, you can create separate disk groups for sales,finance, and development.

Increased Data Availability

You can move a disk group and its components as a unit from one hostmachine to another. This feature provides higher availability of the data inthe following ways:

● If one system fails, another system running VxVM can import thefailed system’s disk group and provide access to it.

● The first system deports the disk group.

● Deporting a disk group disables access to that disk group by the firstsystem. Another host can then import the disk group and startaccessing all the disk drives in the disk group.

● The second system imports the disk group and starts accessing it.

● A host can only import disk groups with unique names. Therefore,all disk groups on all systems should be given unique names, withthe exception of the rootdg disk group.


VERITAS Volume Manager Basic Operations 4-3Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

VxVM Disk Drives

There are two phases to bringing a physical disk drive under VxVMcontrol. Sometimes both operations are done in one step and you areunaware that the process is more complex.

When you bring a disk drive under VxVM control, you can:

● Add it to a new or existing disk group

● Add it to the free-disk pool

The easiest operation is to add a disk drive to the free-disk pool. Thevxdisksetup command repartitions the disk drive into VxVM format,and then a blank header is written to the disk drive.

If you add a disk drive to a disk group, the disk drive is assigned aunique media name and it is associated with a disk group object. Thisinformation is then written into the blank header on the disk drive.

Disk Drive Media Names

Unless you intervene, the default media names that are assigned to diskdrives are based on either the disk group name or the logical device pathto the disk drive.

Default disk group-based media names are similar to dgX01 or DGa04. Thedefault device path-based media names assigned by some command-lineprograms are similar to c3t0d16s2 or c5t4d0s2 . Device path-basedmedia names can lead to confusing status and configuration listings.



Standard VxVM Disk Groups

A disk group is a collection of VxVM disk drives that share a commonconfiguration. Typically, the disk group contains volumes that are allrelated in some way, such as file system volumes that belong to aparticular department or database volumes that are all tables for a singledatabase.

Each disk group is owned by a single host system. The current ownershipis written into all configuration records. Many of the disk drives in thedisk group have a copy of the configuration record.

A disk group and all its components can be moved as a unit from onehost system to another. Usually, both host systems are connected to thesame dual-ported storage arrays.

As shown in Figure 4-1, even though a second host is attached to the samestorage array, access is allowed only to the current owner of the diskgroup. A disk group can be deported from one host and imported by adifferent host, but this is generally used as an emergency solution to acatastrophic host system failure. When a disk group is imported by adifferent host, the name of the new host is written into the disk-basedVxVM configuration records.

Figure 4-1 VxVM Disk Group Ownership

X Host 2Host 1

The disk groupis owned by

Host 1

Volume Volume

Disk Group

A

Storage Array

A



Shared VxVM Disk Groups

VxVM has an optional shared disk group feature that is licensed. The shareddisk group feature allows two or more host systems simultaneous access tothe same disk group. The host systems are referred to as nodes in theSunPlex system environment. Shared disk groups are used in theSun Cluster software environment to support the ORACLE® 9i RealApplication Clusters (RAC) database application. Multiple ORACLE 9iRAC hosts can access a single database image. As shown in Figure 4-2,shared disk groups are owned by a cluster_name and not by the name ofa single host.

When a shared disk group is imported by any of the attached nodes, thename of the Sun Cluster software cluster (cluster_name ) is written intothe disk-resident VxVM configuration records, and the disk group isautomatically accessible by all of the attached nodes.

Figure 4-2 Shared VxVM Disk Group Ownership

Note – To prevent data corruption, all write activity must be coordinatedbetween the nodes. The ORACLE 9i RAC database uses a global lock-management scheme to accomplish this coordination.

Host 1 Host 2

The disk groupis owned by acluster_name

Volume Volume

Disk Group

A0 A1

Sun StorEdge A5200 Array

B1B0



Cross-Platform Data Sharing Disk Groups

A new VxVM disk group format called Cross-Platform Data Sharing(CDS) allows data residing on multi-host storage arrays to be utilized bydifferent operating systems.

The CDS feature is not licensed by Sun but, by default, disk drives areinitialized in the CDS format.

The new cdsdisk partition map allocates all disk space to slice 7 asshown in the following example.

Part Tag Flag Cylinders Size Blocks 0 unassigned wm 0 0 (0/0/0) 0 1 unassigned wm 0 0 (0/0/0) 0 2 backup wu 0 - 4923 8.43GB (4924/0/0) 17682084 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 - wu 0 - 4923 8.43GB (4924/0/0) 17682084

Overriding the CDS Format

Technically, the cdsdisk format does not interfere with standard VxVMoperation. However, if you are not comfortable using the cdsdisk formatat your site, you can disable it so that the vxdiskadm utility uses thesliced format when initializing disk drives.

Before initializing or encapsulating any disk drives for VxVM use, use thevxdiskadm utility option 22, Change/Display the default disklayouts , to modify the default disk format and private region size. Thechanges are stored in the /etc/default/vxdisk file. Use the followingfile format:

# more /etc/default/vxdiskformat=slicedprivlen=2048

VxVM Disk Group Operations


VxVM Disk Group Operations

Basic VxVM administrative functions are those that affect disk drives anddisk groups. The VxVM disk operations are performed using a variety ofprograms including command-line and graphical.

The following list briefly summarizes each type of disk-related operation.

● Placing disk drives under VxVM control

● Removing disk drives from VxVM control

● Adding a disk drive to a disk group

● Removing disk drives from a disk group

● Renaming disk drives

● Creating disk groups

● Destroying disk groups

● Importing disk groups

● Deporting disk groups

The most commonly used tools for basic operations include:

● The vxdisksetup and vxdiskunsetup commands

● The vxdg command

● The vxedit command

● The vxdiskadm utility

● The VEA GUI

Verifying Disk Group Status


Verifying Disk Group Status

The most commonly used disk group status commands are vxdisk andvxdg . Command-line status is usually more efficient than using the VEAGUI.

Using the vxdisk Command to Verify Disk GroupStatus

Use the following vxdisk command to verify the status of the vxdiskadmchanges. Disks that show a status of online invalid are not underVxVM control. Disks that show a status of online have been initializedbut are not assigned to a disk group. When online disks are added to adisk group, they are assigned a name which appears in the DISK column.By default, the disk name is derived from the name of the disk group.

# vxdisk listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:none - - online invalidc0t1d0s2 auto:none - - online invalidc2t1d0s2 auto:cdsdisk a5kdg01 sdga onlinec2t3d0s2 auto:sliced a5kdg02 sdga onlinec2t5d0s2 auto:simple a5kdg03 sdga onlinec2t16d0s2 auto:sliced newDG01 newDG onlinec2t18d0s2 auto:none - - online invalidc2t20d0s2 auto:none - - online invalidc2t22d0s2 auto:none - - online invalidc3t32d0s2 auto:cdsdisk pdga01 pdga onlinec3t33d0s2 auto:cdsdisk pdga02 pdga onlinec3t35d0s2 auto:cdsdisk pdga03 pdga onlinec3t37d0s2 auto:sliced newDG02 newDG onlinec3t50d0s2 auto:none - - online invalidc3t52d0s2 auto:sliced - - online

Using the vxdg Command to Verify Disk Group Status

Use the following vxdg command to verify the current status and uniqueID of all active disk groups.

# vxdg listNAME STATE IDsdga enabled 1064619733.28.ns-east-104pdga enabled 1065123027.40.ns-east-104

Administering Disk Groups Using the vxdiskadm Utility



The vxdiskadm utility is a Bourne shell program that providesmenu-based user operations. The following example shows that when thevxdiskadm utility is started, the following menu operations are available:

# vxdiskadmVolume Manager Support OperationsMenu: VolumeManager/Disk

1 Add or initialize one or more disks 2 Encapsulate one or more disks 3 Remove a disk 4 Remove a disk for replacement 5 Replace a failed or removed disk 6 Mirror volumes on a disk 7 Move volumes from a disk 8 Enable access to (import) a disk group 9 Remove access to (deport) a disk group10 Enable (online) a disk device11 Disable (offline) a disk device12 Mark a disk as a spare for a disk group13 Turn off the spare flag on a disk14 Unrelocate subdisks back to a disk15 Exclude a disk from hot-relocation use16 Make a disk available for hot-relocation use17 Prevent multipathing/Suppress devices from VxVM’s view18 Allow multipathing/Unsuppress devices from VxVM’s view19 List currently suppressed/non-multipathed devices20 Change the disk naming scheme21 Get the newly connected/zoned disks in VxVM view22 Change/Display the default disk layouts23 Mark a disk as allocator-reserved for a disk group24 Turn off the allocator-reserved flag on a disklist List disk information

? Display help about menu?? Display help about the menuing systemq Exit from menus

Select an operation to perform: 1



Functional Overview

The title displayed when the vxdiskadm utility starts up is VolumeManager Support Operations . The vxdiskadm utility performs a widerange of support functions, but also offers assistance in performing anumber of common administrative tasks.

The basic disk-related vxdiskadm administrative functions presented inthis module are:

● Option 1 Add or initialize one or more disks

You use this operation to add one or more disk drives to a diskgroup.

You can add the selected disk drives to an existing disk group or to anew disk group that is created as a part of the operation.

The disk drives are assigned a default media name based on thename of the intended disk group. You can override the default diskdrive name if needed.

The selected disk drives can also be initialized without adding themto a disk group by entering a disk group name of none .

● Option 2 Encapsulate one or more disks

You use this operation to bring disks under VxVM control whilepreserving existing data.

You can also use this operation if you intend to mirror the data forincreased reliability or stripe the data for increased performance.

● Option 3 Remove a disk

You use this operation to remove a disk drive from a disk group andreturn it to a pool for reuse. The disk remains in a VxVM format butits media name is cleared.

● Option 8 Enable access to a disk group

You use this option to place a disk group online (imports) that waspreviously deported (taken offline).

● Option 9 Remove access to a disk group

You use this option to take a disk group offline (deports). None ofthe disk group volume structures are available while it is deported.



Creating a New Disk Group

Before creating a new disk group, you must know the logical address of atleast one disk drive you want to be in the disk group. You should alsoselect media names for any disk drives you intend to add to the new diskgroup.

You use vxdiskadm Option 1, shown in the following example, to create anew disk group and add one or more disk drives.


Select disk devices to add: [<pattern-list>,all,list,q,?] c2t5d0Which disk group [<group>,none,list,q,?] (default: rootdg) newDGCreate a new group named newDG? [y,n,q,?] (default: y) yUse a default disk name for the disk? [y,n,q,?] (default: y) yAdd disk as a spare disk for newDG? [y,n,q,?] (default: n) y

Enter the desired format[cdsdisk,sliced,simple,q,?] (default: sliced) sliced

Enter desired private region length[<privlen>,q,?] (default: 2048) 2048

VxVM NOTICE V-5-2-120Creating a new disk group named newDG containing the diskdevice c2t16d0 with the name newDG01.VxVM INFO V-5-2-305 Setting spare flag for disk newDG01in disk group newDG.

Add or initialize other disks? [y,n,q,?] (default: n) n

Note – For clarity, many informational messages are omitted from theprevious example.



Removing a Disk Drive From a Disk Group

You use vxdiskadm Option 3 to remove a disk drive from a disk group, asshown in the following example. The disk drive is placed in the free-diskpool and is still under VxVM control.


Remove a diskMenu: VolumeManager/Disk/RemoveDisk

Enter disk name [<disk>,list,q,?] newDG03VxVM NOTICE V-5-2-284 Requested operation is to remove disk newDG03from group newDG.

Continue with operation? [y,n,q,?] (default: y) yVxVM INFO V-5-2-268 Removal of disk newDG03 is complete.

Remove another disk? [y,n,q,?] (default: n) n

Administering Disk Groups Using Command-Line Programs


Administering Disk Groups Using Command-LinePrograms

Although there are many VxVM command-line programs, only a few arenecessary for the most commonly performed operations. This sectiondescribes the basic uses of the following command-line programs:

● vxdiskunsetup

● vxdisksetup

● vxdg

Note – Only selected options for each command are described in thismodule. When appropriate, other options are described in later modules.

Using the vxdiskunsetup Command

Caution – Be careful when using the vxdiskunsetup command. You candestroy existing data on disk drives.

During the VxVM software installation and initialization, you might seeerror messages, such as:

VxVM:vxconfigd: WARNING: Disk c3t35d0 in group hanfs:Disk device not foundVxVM:vxconfigd: WARNING: Disk c2t18d0 in group hadbms:Disk device not found

These errors can indicate that there are disk drives that still contain VxVMconfiguration records from a previous installation. You can clear thesedisk drives and return them to an uninitialized state by using thevxdiskunsetup command, as shown in the following example:

# /etc/vx/bin/vxdiskunsetup -C c2t3d0

The vxdiskunsetup command will not clear a disk drive if the VxVMconfiguration records indicate it is imported by some other host. The -Coption forces the de-partitioning of the disk drive in such a case. The diskdrives are returned to standard Solaris OS partitioning.



Initializing Disk Drives

The vxdisksetup command initializes disk drives for VxVM use, butdoes not add them to a disk group. The disk drives remain in a free poolfor future VxVM use. A typical vxdisksetup command is:

# vxdisksetup -i c2t0d0

Without any options, the vxdisksetup command only repartitions thespecified disk drive into the VxVM partition format. With the -i option,VxVM configuration records are written in the private region.

Note – Usually, the vxdisksetup command is not used directly. It iscalled by other VxVM programs.

Using the vxdg Command

Although the vxdg command has many options, only a few are presentedin this module. You have already seen how the vxdiskadm utility createsnew disk groups and adds disk drives to a disk group. You can also usethe vxdg command to perform those functions. The vxdg commandoptions related to basic disk-group administration are:

● vxdg init

● vxdg adddisk

● vxdg rmdisk

● vxdg import

● vxdg deport

● vxdg destroy

Note – You must initialize a disk drive before it can be added to a new orexisting disk group using the vxdg command.



Using the vxdg Command to Create a New Disk Group

To create a new disk group using the vxdg command, you must furnishthe disk drive logical access name (accessname ) of at least one disk driveto be added to the disk group. The VxVM accessname is essentially thelogical path to the disk drive in the form: c3t4d0 . You should also specifya media name for the disk drive. If you do not specify a media name, itdefaults to the accessname . The following shows a typical session toinitialize a new disk drive and add it to a new disk group.

# vxdisk listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:none - - online invalidc0t1d0s2 auto:none - - online invalidc2t1d0s2 auto:none - - online invalidc2t3d0s2 auto:none - - online invalidc2t5d0s2 auto:none - - online invalidc2t16d0s2 auto:none - - online invalidc2t18d0s2 auto:none - - online invalidc2t20d0s2 auto:none - - online invalidc2t22d0s2 auto:none - - online invalidc3t32d0s2 auto:none - - online invalidc3t33d0s2 auto:none - - online invalidc3t35d0s2 auto:none - - online invalidc3t37d0s2 auto:none - - online invalidc3t50d0s2 auto:none - - online invalidc3t52d0s2 auto:none - - online invalid

# vxdisksetup -i c2t1d0# vxdisksetup -i c2t3d0

# vxdg init newDG ndg-01=c2t1d0 ndg-02=c2t3d0

# vxdisk -g newDG listDEVICE TYPE DISK GROUP STATUSc2t1d0s2 auto:sliced ndg-01 newDG online sparec2t3d0s2 auto:sliced ndg-02 newDG online spare

# vxdg listNAME STATE IDnewDG enabled 1065465185.43.ns-east-104



Adding and Removing Disk Drives

The following shows an example of initializing two disk drives, addingthem to an existing disk group, and then removing one of them.

# vxdisksetup -i c2t5d0# vxdisksetup -i c2t16d0

# vxdg -g newDG adddisk ndg-03=c2t5d0 ndg-04=c2t16d0

# vxdisk listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:none - - online invalidc0t1d0s2 auto:none - - online invalidc2t1d0s2 auto:sliced ndg-01 newDG onlinec2t3d0s2 auto:sliced ndg-02 newDG onlinec2t5d0s2 auto:sliced ndg-03 newDG onlinec2t16d0s2 auto:sliced ndg-04 newDG onlinec2t18d0s2 auto:none - - online invalidc2t20d0s2 auto:none - - online invalidc2t22d0s2 auto:none - - online invalidc3t32d0s2 auto:none - - online invalidc3t33d0s2 auto:none - - online invalidc3t35d0s2 auto:none - - online invalidc3t37d0s2 auto:none - - online invalidc3t50d0s2 auto:none - - online invalidc3t52d0s2 auto:none - - online invalid

# vxdg -g newDG rmdisk ndg-04

The removed disk drive is still initialized and is available for future use. Itis in the free-disk pool and shows a status of auto:sliced and onlinewith no DISK or GROUPentry.



Importing and Deporting Disk Groups

At times it is useful to deport a disk group. This function makes the diskgroup unavailable and invisible to most commands. If the disk groupresides on dual-ported storage arrays, a different host can import the diskgroup. This action is useful if a host system fails. A different host systemthat is running VxVM can import the disk group and make the dataavailable again to users. The following example shows the process ofdeportng a disk group.

# vxdg listNAME STATE IDsdga enabled 1064619733.28.ns-east-104newDG enabled 1065465185.43.ns-east-104pdga enabled 1065123027.40.ns-east-104

# vxdg deport newDG


When a disk group is deported, the host ID stored on all disk drives in thedisk group is cleared (unless a new host ID is specified with -h ).Therefore, the disk group is not reimported automatically when thesystem is rebooted.

The disk group can be deported with the host ID unchanged or you canchange the host ID to another system during the deport operation.

Use the vxdg import command option to import a disk group again. Forexample:

# vxdg import newDG



Identifying Deported Disk Groups

If you forget the name of a deported disk group, you can use the vxdiskcommand to identify currently deported disk groups. The followingexample shows the names of currently deported disk groups enclosed inparenthesis.

# vxdisk -o alldgs listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:none - - online invalidc0t1d0s2 auto:none - - online invalidc2t1d0s2 auto:cdsdisk a5kdg01 sdga online nohotusec2t3d0s2 auto:sliced a5kdg02 sdga online nohotusec2t5d0s2 auto:simple a5kdg03 sdga online nohotusec2t16d0s2 auto:sliced - (newDG) onlinec2t18d0s2 auto:sliced - - onlinec2t20d0s2 auto:sliced - - onlinec2t22d0s2 auto:none - - online invalidc3t32d0s2 auto:cdsdisk pdga01 pdga onlinec3t33d0s2 auto:cdsdisk pdga02 pdga onlinec3t35d0s2 auto:cdsdisk pdga03 pdga onlinec3t37d0s2 auto:sliced - (newDG) onlinec3t50d0s2 auto:none - - online invalidc3t52d0s2 auto:sliced - - online

You can also use the vxdiskadm utility option 8, Enable access to(import) a disk group , to identify deported disk groups. Thefollowing is an excerpt of the resulting output.

Select disk group to import [<group>,list,q,?] (default:list) list

GROUP newDG (id: 1065465185.43.ns-east-104) DEVICES: c2t16d0 c3t37d0

Note – The VEA interface also displays deported disk groups.



Destroying a Disk Group

Before destroying a disk group, you must be absolutely sure that none ofthe data on the disk drives is important. A disk group cannot bedestroyed if any volumes in the disk group are in use. You can destroyonly imported disk groups. The following is an example of using the vxdgcommand to destroy a disk group.

# vxdg listNAME STATE IDsdga enabled 1064619733.28.ns-east-104newDG enabled 1065465185.43.ns-east-104pdga enabled 1065123027.40.ns-east-104

# vxdg destroy newDG


Renaming VxVM Disk Drives

When disk drives are added to a disk group, the disk drives are givenstandard VxVM media names, such as dga01 , dga02 , and dga03 . Youmight want to rename the disk drives in a group so that there is noconfusion about their intended use.

You can use the following vxedit command to rename VxVM diskdrives:

# vxedit -g sdga rename a5kdg01 sdga01# vxedit -g sdga rename a5kdg02 sdga02# vxedit -g sdga rename a5kdg03 sdga03

Note – Volume components such as plexes and subdisks are namedaccording to the VxVM disk media name. If you rename a disk, the plexand subdisk names do not change accordingly. It is best to rename disksbefore creating volume structures on them.

Administering Disk Groups Using the VEA GUI



The following section describe a series of tasks using the VEA GUI. Onlykey points and input forms are presented.

Creating a New Disk Group

You can use the VEA GUI to create a new disk group. Complete thefollowing steps:

1. If appropriate, you can pre-select disk drives in the grid area bysimultaneously pressing the Control key and mouse button.

2. Select the New Disk Group entry from the Actions menu.

3. Complete the New Disk Group Form, as shown in Figure 4-3.

Disk names default to variations of the disk group name, such asnewDG01or newDG02. Default disk names are usually acceptable.

Figure 4-3 New Disk Group Wizard



4. Click Next to continue.

As shown in Figure 4-4, VxVM informs you of potentially dangeroussituations it detects. Click on Yes to continue or No to abort.

Figure 4-4 New Disk Group Warning

5. Complete the Organization window, as shown in Figure 4-5, byselecting a disk group organization principle of None.

Figure 4-5 New Disk Group Organization



Adding and Removing Disk Drives

To add disks to, or remove disk drives from, a disk group, complete thefollowing steps:

1. Select the disk group in the object tree, and then click the thirdmouse button.

2. Select Add Disk to Disk Group from the pop-up menu, as shown inFigure 4-6.

Figure 4-6 Disk Groups Disk Add Menu Item



3. Complete the Add Disk wizard form, as shown in Figure 4-7.

Figure 4-7 Add Disk to Disk Group Wizard



Deporting Disk Groups

To deport a disk group, complete the following steps:


2. Select Deport from the pop-up menu, as shown in Figure 4-8.

Figure 4-8 Disk Groups Deport Menu Item

3. Complete the Deport Disk Group form, as shown in Figure 4-9.

Usually, you do not modify the disk group name or host systemownership during a deport operation.

Figure 4-9 Deport Disk Group Form



Importing Disk Groups

To import a disk group, complete the following steps:

1. Display Disk Groups in the grid area, and then select the deporteddisk group.

2. Click the third mouse button on the deported disk group and selectImport from the pop-up menu, as shown in Figure 4-10.

Figure 4-10 Import Disk Group Menu Item

3. Complete the Import Disk Group form, as shown in Figure 4-11.

Typically, the Force option is necessary only after a system crash.

Figure 4-11 Import Disk Group Form



Destroying a Disk Group

A disk group must be imported before it can be destroyed. To destroy adisk group, complete the following steps:


2. Select Destroy Disk Group from the pop-up menu, as shown inFigure 4-12.

Figure 4-12 Disk Group Destroy Menu

If there are mounted volumes associated with the disk group, thedisk group destroy operation fails, as shown in Figure 4-13.

Figure 4-13 Destroy Disk Group Failure



Renaming VxVM Disk Drives

To rename VxVM disk drives, complete the following steps:

1. To change the media name of a VxVM disk drive, complete thefollowing steps:

a. Display the disk group disk drives in the grid area.

b. Select the disk drive to be renamed, as shown in Figure 4-14.

Figure 4-14 Disk Drive Media Name Listing

2. Click the third mouse button on the highlighted disk, and then selectRename from the pop-up menu.

3. Complete the Rename Disk form, as shown in Figure 4-15.

Figure 4-15 Rename Disk Form



Displaying VEA Object Properties

Most VxVM objects have associated properties that detail namingschemes, construction details, and current status.

To display the properties of a VxVM object, complete the following steps:

1. Click on the object and the third mouse button.

2. Select Properties from the object’s pop-up menu.

The Properties of a disk group is shown in Figure 4-16. ObjectProperties windows can have several tabs depending on thecomplexity of the object.

Figure 4-16 Disk Group Object Properties Form

Exercise: Performing VxVM Basic Operations





● Verify the VxVM environment

● Verify the initial disk drive status

● Set the default disk drive format

● Initialize disk drives

● Create new disk groups

● View command logs

● Import and deport disk groups

● Destroy a disk group

● Rename disk drives

● Use the vxdiskadm utility to perform basic operations (optional)

● Verify ending lab status



Preparation

If the tasks in this exercise are performed by small groups using diskdrives residing on a central VxVM server, each group must take care tonot interfere with another group’s storage resources and structures.

Ask your instructor to provide two unique code letters for yourworkgroup (A and B, C and D, E and F, and so on).

Workgroup code letters: dg ___ dg ___

Copy the logical paths to six disk drives for your work group from theinformation in Module 2, ‘‘Managing Data” in ‘‘Task 9 – Selecting DiskDrives for Use’’ on page 2-32.

Disk: _______________ Disk: _______________

Disk: _______________ Disk: _______________

Disk: _______________ Disk: _______________

Assign each workgroup’s two code letters (A/B, C/D, E/F) so they can create two unique disk group names,such as dgA and dgB, or dgE and dgF. A workgroup consists of two or more students working with six diskdrives from one keyboard on an administration workstation.

An antiquated, but useful, management tool is to have each workgroup write the logical paths of theirassigned disk drives on a 3 by 5 card and tape the card to their display monitor.

Many of the tasks are performed twice. The first time using the VEA GUIand the second time using command-line programs. For most tasks, youmust destroy the structures before creating them again. Destroying anddeleting structures is part of regular VxVM administrative tasks.




Answer the following questions about the major areas of concern duringthe VxVM installation and initialization process.

1. What is the primary use for a shared disk group?

a. Support for load balancing

b. Support for multi-host storage access

c. Support for parallel databases

d. Support for multipath storage access

The answer is c.

2. Which two answers best describe the VxVM disk-drive initializationprocess?

a. Preserves existing file systems

b. Places the disk drive in a free disk pool

c. Adds the disk drive to a disk group

d. Destroys existing file systems


3. Which of the following commands is not used to perform disk groupadministration?

a. vxdisk

b. vxassist

c. vxdg

d. vxdiskadm

e. vxdisksetup

The answer is b.

4. Which of the following commands is used to rename disk drives?

a. vxdg

b. vxassist

c. vxdisk

d. vxedit

The answer is d.



Task 2 – Verifying the VxVM Environment

Complete the following steps to verify the environment on the VxVMadministration system and the VxVM server:

1. Log in to the VxVM server as user root and enter the env shellcommand.

2. Verify that the following environment exists on the server:

PATH=/usr/sbin:/usr/bin:/opt/VRTS/bin:/opt/VRTSalloc/bin:/opt/VRTSvlic/bin:/opt/VRTSvmpro/bin:/etc/vx/bin

MANPATH=/usr/man:/opt/VRTS/man:/opt/VRTSlic/man

3. If you are working from a remote administration system, log out ofthe VxVM server.

4. On the remote administration system, complete the following steps:

a. Type the env shell command.

b. Verify that the /opt/VRTS/bin and /opt/VRTSob/bindirectories are part of the PATHvariable.

c. Verify that the /opt/VRTS/man directory is part of the MANPATHvariable.



Task 3 – Verifying the Initial Disk Drive Status

To verify the initial status of your assigned disk drives, complete thefollowing steps:

1. Log in as user root to the VxVM server.

2. Verify the status of your assigned disk drives.

# vxdisk list

3. Check the output carefully and verify that each disk drive assignedto your workgroup shows a status of online invalid .

If any of your assigned disk drives show a status other than onlineinvalid , check with your instructor. If appropriate, use thevxdiskunsetup command, as shown in the following example, toreturn all of your assigned disk drives to an online invalid status.

# /etc/vx/bin/vxdiskunsetup -C c0t22d0

Note – You must substitute the VxVM accessname of your disk drives.

Do not proceed until all your assigned disk drives are uninitialized.



Task 4 – Setting the Default Disk Drive Format

Set the default VxVM disk initialization format on the VxVM server bycompleting the following steps:

1. Log in to the VxVM server as user root .

2. Complete the following steps:

a. Start the vxdiskadm utility.

b. Select option 22, Change/Display the Default Disk Layouts.

# vxdiskadm

3. Select option 1 from the Volume Manager Preferences menu.

4. Select option 1 from the Disk Initialization Preferences menu andtype sliced for the desired format.

5. Select option 2 from the Disk Initialization Preferences menu andtype 2048 for the desired private region length.

6. Type q or quit at least twice to exit the vxdiskadm utility.

7. Verify the default disk initialization format values are set correctly.

# more /etc/default/vxdiskformat=slicedprivlen=2048

Caution – Values entered in the /etc/default/vxdisk file are honoredonly by command-line programs such as the vxdiskadm and vxassistutilities. The VEA GUI does not reference the /etc/default/vxdisk file.



Task 5 – Initializing Disk Drives

Perform the following procedures to initialize your assigned disk drivesfor use by VxVM. There are two procedures: one for using the VEA GUIand one for using command-line programs.

Using the VEA GUI to Initialize Disk Drives

To initialize three of your assigned disk drives using the VEA GUI,complete the following steps:

1. Expand the VxVM server node in the object tree and click Disks.

2. Select three of your assigned disk drives in the grid area bysimultaneously pressing the Control key while using the mousebutton.

3. Click the third mouse button on one of your highlighted disk drives,and select Initialize Disk in the pop-up menu.

4. In the Initialize Disk form, click Yes To All .

After a short delay, three of your assigned disk drives should appearin the grid area with a status of Free .


a. Use the vxdiskunsetup command to return each of yourassigned disk drives to an uninitialized state.

b. Replace the accessname variable in the following commandwith the address of your disk drives (for example, c4t3d0 ).

# vxdiskunsetup -C accessname

6. Verify that the status of your assigned disk drives is once againonline invalid .

# vxdisk list



Using Command-Line Programs to Initialize Disk Drives

To initialize your remaining three disk drives using the vxdisksetupcommand-line program, complete the following steps:

1. Use the vxdisksetup command to initialize each of your assigneddisk drives, replacing the accessname with the address of your diskdrives (for example, c4t3d0 ).

# vxdisksetup -i accessname

2. Verify the status of your assigned disk drives is online .

# vxdisk list

Task 6 – Creating New Disk Groups

In this section, you create a disk group containing three of your assigneddisk drives. Name the disk group according to your first assignedworkgroup letter. For example, if your assigned work group letters are Aand B, then this first disk group should be named dgA.

Using the VEA GUI to Create a New Disk Group

To create a new disk group using the VEA GUI, complete the followingsteps:

1. Click New Group in the Toolbar.

The initial New Disk Group Wizard form appears.

2. Click the Do not show this page next time box, and then clickNext.

The disk selection form appears.

3. Complete the disk selection form as follows:

a. Type the name of your new disk group.

b. In the Available disk column, select three of your assigned diskdrives, and then click Add.

c. Do not enter disk names.

d. Click Next.

The organization principle form appears.



4. Complete the organization principle form as follows:

a. Select the None organization principle.

b. Click Next.

The summary form appears.

5. Click Finish on the summary form to create the new disk group.

6. Verify that your new disk group appears in the VEA object tree.

Using the vxdg Command to Create a Disk Group

Use the vxdg command to create a second disk group that contains yourthree remaining assigned disk drives. Name the disk group according toyour second workgroup letter. For example, if your work group letters areA and B, then this second disk group should be named dgB.

Complete the following steps:

1. Use the vxdg init command to create a second new disk group andadd one of your remaining assigned disk drives.

# vxdg init dgname medianame=accessname

For example:

vxdg init dgB dgB-01=c2t3d0

2. Add the two remaining assigned disk drives to the new disk drivegroup.

# vxdg -g dgname adddisk medianame=accessname

For example:

vxdg -g dgB adddisk dgB-02=c2t5d0 dgb-03=c2t6d0

3. Use the vxprint command to verify the status of your new diskgroup.

4. Use the vxdg rmdisk command to remove one of the disk drivesfrom your new disk group.

# vxdg -g dgname rmdisk medianame

For example: vxdg -g dgB rmdisk dgB-03

5. Use the vxdg adddisk command to add the same disk drive backinto your disk group.



Task 7 – Viewing Command Logs

The command log records the command-line programs used toaccomplish VEA tasks. It is a good learning tool.

To view the command log file, complete the following steps:


2. View the command log file.

# cd /var/vx/isis# lsalertlog state vea_portalcommand.log swap vxisis.locksavedqueries tasklog vxisis.log# more ./command.log

Task 8 – Importing and Deporting Disk Groups

In this task, you deport and import one or both of your disk groups.

Using the VEA GUI to Import and Deport Disk Groups

To import and deport disk groups using the VEA GUI, complete thefollowing steps:

1. Display disk groups in the grid area.

2. Click the name of one of your disk groups and select Deport DiskGroup from its pop-up menu.

The Deport Disk Group form appears.

Note – You can rename a disk group during a deport operation. You canalso assign ownership to a different host. You might do this if you neededto take the current host down for maintenance and wanted a differenthost system to manage the disk group for a while. You might also renamethe disk group if the second host already had a disk group with the samename.

3. Click OK.




a. Click Browse and highlight the disk group you want to deport.

b. Click OK.

The status of your disk group should change to Deported in thegrid area display.

5. Click your deported disk group in the grid area and select Importfrom its pop-up menu.

Using Command-Line Programs to Import and Deport DiskGroups

To import and deport disk groups using command-line programs,complete the following steps:

1. Log in to the VxVM server and use the vxdg deport command todeport either of your disk groups.

# vxdg deport dgname

For example: vxdg deport dgA .

2. Use the vxdisk command to identify the deported disk group.

# vxdisk -o alldgs list

3. Use the vxdg import command to import your disk group again.

# vxdg import dgname

For example: vxdg import dgA .



Task 9 – Destroying a Disk Group

In this task, you destroy one of your disk groups.

Using the VEA GUI to Destroy a Disk Group

To destroy a disk group using the VEA GUI, complete the following steps:

1. Display Disk Groups in the grid area.

2. Click one of your disk groups and select Destroy Disk Group fromits pop-up menu.

3. Click New Group in the Toolbar and recreate the disk group youdestroyed.

Using Command-Line Programs to Destroy a Disk Group

To destroy one of your disk groups using command line programs,complete the following steps:

1. Use the vxdg command to destroy one of your disk groups.

# vxdg destroy dgname

For example: vxdg destroy dgA .

2. Recreate the destroyed disk group using the tool of your choice.

Task 10 – Renaming Disk Drives

In this task, you rename one or more of your assigned disk drives.

Using the VEA GUI to Rename Disk Drives

To rename disk drives using the VEA GUI, complete the following steps:

1. Display the disk drives from one of your disk groups in the gridarea.

2. Click one of the disk drives and select Rename Disk from its pop-upmenu.

3. Enter a unique media name for your disk drive.



Using Command-Line Programs to Rename Disk Drives

To rename disk drives using command-line programs, complete thefollowing step:

Using the vxedit command, restore the original media name of the diskdrive you modified in ‘‘Using the VEA GUI to Rename Disk Drives’’ onpage 4-40.

# vxedit rename oldname newname

For example: vxedit rename zx12 dgB-03 .

Task 11 – Using the vxdiskadm Utility to Perform BasicOperations (Optional)

Ask your instructor if there is enough time to perform this optional task.

To recreate your disk groups using the vxdiskadm utility features,complete the following steps:

1. Use the vxdg command to destroy both of your disk groups.

# vxdg destroy dgname

2. Use the vxdiskadm utility option 1 to recreate both of your assigneddisk groups.

3. Use the vxdiskadm utility option 3 to remove a disk drive from eachof your disk groups.

4. Use the vxdiskadm utility option 1 to add the previously removeddisk drives into your disk groups.

5. Use the vxdiskadm utility option 9 to deport one or both of yourdisk groups.

6. Use the vxdiskadm utility option 8 to import one or both of yourassigned disk groups.



Task 12 – Verifying Ending Lab Status

To verify your system configuration, complete the following steps:

1. Use the vxdisk list command to ensure that both of your diskgroups are still complete and meet the following guidelines:

● You have two disk groups name dgX and dgY.

● Each disk group contains three disk drives.

● None of your assigned disk drives are designated as hot spares.

Note – Substitute your workgroup codes for the X and Y in dgX and dgY.

2. Use the vxprint command to verify that there are no subdiskstructures in either of your disk groups.

Exercise Summary


Exercise Summary

?!



● Experiences


● Interpretations


● Conclusions


● Applications



Module 5

VERITAS Volume Manager VolumeOperations

Objectives


● Interpret volume structure listings

● Describe volume planning activities

● Create volumes using the VEA GUI

● Create volumes using the vxassist command

● Modify volume access attributes

● Add file systems to existing volumes

● Add and remove volume logs

● Use the VEA GUI to analyze volume structures

Interpreting Volume Structure Listings



VERITAS VxVM volume structures are composed of three primarycomponents: subdisks, plexes, and a volume name. The components areassigned names based on default conventions or can be manually chosen.

Subdisks

A subdisk is a set of contiguous disk blocks. A subdisk must resideentirely on a single physical disk drive. The public region of a disk drivein a disk group can be divided into one or more subdisks. The subdiskscannot overlap or share the same portions of a public region.

The smallest possible subdisk is a single sector (512 bytes), and the largestsubdisk is the entire VxVM public region.

By default, subdisks are named based on the VxVM media name of thedisk drive on which they reside. This relationship is shown in Figure 5-1.

Figure 5-1 Subdisk Naming Conventions

Ask why spanning storage arrays might be a good idea. One answer is striping for performance or mirroringfor availability.

disk01-01

disk01-02

disk01

Physical DiskSubdisks

Disk Group (DGa)

Physical Diskdisk02-01

disk02-02

disk02

VxVM Disk

VxVM Disk

c3t12d0

c4t33d0

disk02-03


VERITAS Volume Manager Volume Operations 5-3Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Plexes

The VxVM application uses subdisks to build virtual objects called plexes.A plex consists of one or more subdisks located on one or more physicaldisk drives. Figure 5-2 shows the relationship of subdisks to plexes in adisk group named DGa.

Figure 5-2 Plex Configurations

The data to be stored on the subdisks of a plex can be organized by usingany of the following methods:

● Concatenation

● Striping

● Striping with parity (RAID 5)

Note – A plex can contain a maximum of 4096 subdisks.

disk01-01

disk01-02

disk01

Physical Disk

Disk Group (DGa)


disk02-02

disk02

VxVM Disk

VxVM Disk

disk01-01

disk01-02

vol01-01

Plex

disk02-01

disk02-02

vol01-02

Plex

c3t12d0

c4t33d0

disk02-03



Volumes

A volume consists of one or more plexes. By definition, a volume withtwo plexes is mirrored. Figure 5-3 shows the relationship of plexes in amirrored volume in a disk group named DGa.

Figure 5-3 Mirrored Volume Structure

You should understand the following important points about volumes:

● Volumes can have more than two mirrors.

Three mirrors is usually as many as most customers ever have in critical applications.

● RAID-5 volumes cannot be mirrored.

● A plex can also be a logging structure that is not used for datastorage.

disk01-01

disk01-02

disk01

Physical Disk

Disk Group (DGa)


disk02-02

disk02

VxVM Disk

VxVM Disk

disk01-01

disk01-02

vol01-01

Plex

Volume

disk02-01

disk02-02

vol01-02

vol01

Plex

c3t12d0

c4t33d0

disk02-03



Volume Structure Examples

The following vxprint output shows an example of two volumes thatbelong to the same disk group. The vol_01 volume is a simpleconcatenation and the vol_02 volume is mirrored across two controllers.The listing indicates the following details:

● Both volumes are 2 Gbytes in size (4194304 blocks/sectors)

● vol_01 is composed of a single plex and associated subdisk

● vol_02 is composed of two plexes each with an associated subdisk

# vxprint -g newDGTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg newDG newDG - - - -

dm newDG01 c2t16d0s2 - 17679776 - -dm newDG02 c2t18d0s2 - 17679776 - -dm newDG03 c2t20d0s2 - 17679776 - -dm newDG04 c3t32d0s2 - 17679776 - -dm newDG05 c3t33d0s2 - 17679776 - -dm newDG06 c3t35d0s2 - 17679776 - -

v vol_01 fsgen ENABLED 4194304 - ACTIVEpl vol_01-01 vol_01 ENABLED 4194304 - ACTIVEsd newDG01-01 vol_01-01 ENABLED 4194304 0 -

v vol_02 fsgen ENABLED 4194304 - SYNCpl vol_02-01 vol_02 ENABLED 4194304 - ACTIVEsd newDG04-01 vol_02-01 ENABLED 4194304 0 -pl vol_02-02 vol_02 ENABLED 4194304 - ACTIVEsd newDG01-02 vol_02-02 ENABLED 4194304 0 -

The following vxdg output shows the amount of available disk space. TheOFFSET column represents the amount of space currently used and theLENGTH column represents the amount of free space.

# vxdg -g newDG freeDISK DEVICE TAG OFFSET LENGTH FLAGSnewDG01 c2t16d0s2 c2t16d0 8388608 9291168 -newDG02 c2t18d0s2 c2t18d0 0 17679776 -newDG03 c2t20d0s2 c2t20d0 0 17679776 -newDG04 c3t32d0s2 c3t32d0 4194304 13485472 -newDG05 c3t33d0s2 c3t33d0 0 17679776 -newDG06 c3t35d0s2 c3t35d0 0 17679776 -

Volume Planning


Volume Planning

Creating volume structures is easy to do. It is also easy to make mistakesunless you understand each aspect of the volume creation process.

Volume Distribution

A common mistake is to place all the disk drives in a single disk group.The configuration records for a disk group cannot contain information formore than 4096 objects. Each volume, plex, subdisk, and disk drive isconsidered to be an object and requires 256 bytes of space in the privateregion. The default private region length is 2048 blocks.

Another reason for organizing disk drives into separate disk groups isthat you might want to deport a disk group and import it to anotherconnected host system. This action can be part of a disaster recovery planor a load balancing measure.

You can design a disk group so that it is better for particular tasks. Eachdisk group shown in Figure 5-4 has three disk drives, and each disk driveis in a different storage array.

Figure 5-4 Striped Volume Disk Groups

Host System

Array Array Array

c2c1c0

d1 d2 d3

d4 d5 d6

d7 d8 d9

DGa

DGb

DGc

Volume Planning


Disk groups organized in this manner are good for creating stripedvolume types (such as RAID 5) and for mirrored volumes. The mostimportant feature is that each disk drive in the disk group is in a separateenclosure and on a different controller.

Note – Exercise care with disk groups that span storage arrays. You mustbe sure that the loss of an entire array does not disrupt both mirrors in avolume or more than one column in a RAID-5 volume.

Another disk group structure, such as the one shown in Figure 5-5, wouldbe better used with simple concatenated volumes.

Figure 5-5 Concatenated Volume Disk Groups

If the volumes are large, static, read-only structures that need only aperiodic backup to tape, they do not need any higher level of reliability oravailability.

Host System

Array Array Array

c2c1c0

d1 d2 d3

d4 d5 d6

d7 d8 d9

DGa

DGb

DGc

Volume Planning


Volume Naming Conventions

Within a disk group, the VxVM software requires a unique name for eachnew volume that is created. The volume names are usually systematic,such as vol01 , vol02 , vol03 , and so on. Naming conventions can be usedto reflect volume attributes, such as:

● The volume structure

● Which department uses them

● With which database they are associated

● Special purposes within a workgroup

Although naming conventions might not seem to be of much importance,they can help establish priorities during emergency situations, such asmajor power outages.

Space Allocation Planning

Although choosing a general size for a volume is frequently dictated bythe application, administrators frequently want to use as much space as ispractical on a set of disk drives. There are several ways to allocate spacefor a volume. Among them are:

● Let the VxVM software automatically find the space.

● Limit the search for space to selected disk drives in a group.

● Research available space with command-line programs.

Note – The VEA GUI New Volume Wizard also has limited space researchcapabilities during new volume creation.

Automatic Space Allocation

If you do not specify disk resources when creating volumes, the VxVMsoftware automatically finds portions of unused disk space and assemblesthem into a volume. This action can lead to a disorganized structure andcreate poor performance for some volume types.

Volume Planning


Restricted Space Allocation

Rather than letting VxVM find space anywhere within a disk group, it isfrequently better to direct VxVM to use a particular disk drive for a newvolume.

The disk group shown in Figure 5-6 can be used in several different waysdepending on the type of volume structures you require:

● A RAID-5 volume might use disk drives d1, d2, and d3

● A concatenated volume might use disk drives d1, d4, and d7.

● A mirrored and concatenated volume might use disk drives d1, d4,and d7 for one mirror and disk drives d3, d6, and d9 for a secondmirror.

Figure 5-6 Selecting Disk Drives for a Volume

Host System

Array Array Array

c2c1c0

d1 d2 d3

d4 d5 d6

d7 d8 d9

DGa

Volume Planning


Researched Space Allocation

Analyzing free disk space before creating a volume is sometimesbeneficial. Look for patterns of free space that fit your needs. Thefollowing examples show how to research free space in a disk group.

The vxdg command gathers a rough estimate of available disk space. Thefollowing is an example of using vxdg on a 9-Gbyte disk drive.

# vxdg -g newDG freeDISK DEVICE TAG OFFSET LENGTH FLAGSndg-01 c2t1d0s2 c2t1d0 0 17674902 -ndg-02 c2t3d0s2 c2t3d0 0 17674902 -ndg-03 c3t32d0s2 c3t32d0 0 17674902 -ndg-04 c3t33d0s2 c3t33d0 0 17674902 -

After creating a 6-Gbyte mirrored volume using the ndg-01 and ndg-03disks, the following disk space is available:

# vxdg -g newDG freeDISK DEVICE TAG OFFSET LENGTH FLAGSndg-01 c2t1d0s2 c2t1d0 12586455 5088447 -ndg-02 c2t3d0s2 c2t3d0 0 17674902 -ndg-03 c3t32d0s2 c3t32d0 12586455 5088447 -ndg-04 c3t33d0s2 c3t33d0 0 17674902 -

After rough planning, you use the vxassist maxsize command toresearch more detailed information. The following examples show how touse the vxassist command:

# vxassist -g newDG maxsize \layout=nomirror,nostripe ndg-01 ndg-02 ndg-03 ndg-04Maximum volume size: 45524992 (22229Mb)

This is a basic concatenation that uses almost all of the available space, that is, 22 Gbytes.

# vxassist -g newDG maxsize \layout=raid5,nolog ndg-01 ndg-02 ndg-03 ndg-04Maximum volume size: 20353024 (9938Mb)

RAID-5 volume column size is limited by the size of the smallest available column, which is 5,088,447blocks in these examples. Additionally, approximately one disk drive’s worth of space is lost to parity.

The vxassist maxsize command is not usually needed unless you haveespecially limited disk drive space and need to maximize its use. It is agood practice to leave a small amount of space for log placement.

Volume Planning


Selecting Volume Types

The VxVM application supports several types of volumes. Each volumetype has a specific layout specifier. The following sections briefly describeeach volume type and its associated layout specification.

Concatenated

The advantages of concatenated volumes are efficient use of storage spaceand easier hardware requirements.

Layout specification: concat

Striped

The advantages of striped volumes are improved read and writeperformance.

Layout specification: stripe

Mirrored Concatenation

The advantage of a mirrored concatenation volume is improvedavailability through data redundancy.

Layout specification: mirror-concat

Mirrored Stripe

The advantages of a mirrored stripe volume are improved availability andincreased read and write performance.

Layout specification: mirror-stripe

Volume Planning


Concatenated Mirror (Layered Volume Type)

The advantages of concatenated mirror volumes are improved availabilityand faster recovery time. This is because only a portion of a mirror mustbe recovered.

Layout specification: concat-mirror

Striped Mirror (Layered Volume Type)

The advantages of striped mirror volumes are improved performance,faster recovery times, and high disk drive failure tolerance.

Layout specification: stripe-mirror

RAID 5

The advantage of RAID-5 volumes are somewhat improved availabilitywith limited negative impact on disk space utilization.

Layout specification: raid5 (or raid5nolog)

Creating Volumes Using the VEA GUI



A New Volume Wizard is initiated in the VEA GUI using either of thefollowing methods:

● The menu bar’s Actions menu New Volume item

● The toolbar’s New Volume button

Disk Selection Method

The initial New Volume Wizard form prompts you to select the diskselection method to be used. The default is to let VxVM decide whichdisks to use. However, when creating complex volumes, it is a bestpractice to enable manual disk selection, as shown in Figure 5-7.

Figure 5-7 New Volume Wizard Disk Selection Method



Using the Disk Selection Form

As shown in Figure 5-8, the New Volume Wizard disk selection formassists you in locating and selecting suitable disk drives.

Figure 5-8 New Volume Wizard Disk Selection

The Mirror Across and Stripe Across check boxes let you choose how youwant stripes or mirrors distributed across your storage configuration. TheMirror Across Tray applies to specific storage array models that haveseparate disk drive trays in a single array. Unless you later specify astriped or mirrored volume structure, these features do not perform anyuseful function.

The Ordered check box is an advanced function that uses the specifiedstorage first to create concatenation, then form columns, and finally tocreate mirrors. Ordered allocation is an advanced subject presented laterin this course.



Using the Volume Attributes Form

The New Volume Wizard volume attributes form, shown in Figure 5-9,assists you in defining the name, size, and type of a volume structure.

Figure 5-9 New Volume Wizard Attributes

Consider the following points when configuring new volume attributes:

● Name volumes according to their purpose.

● Use Maxsize after you have configured everything else. The Maxsizefunction calculates space based on the volume layout specification.The space available varies widely depending on the target volumeconfiguration.

● Use the Columns and Stripe unit size functions when you arecreating volumes that have columns such as striped or RAID-5volumes.

● Note that the Mirrored check box is automatically enabled whenConcatenated Mirrored or Striped Mirrored structures are selected.Manually selecting Concatenated, and then selecting Mirrored,creates a different structure than selecting Concatenated Mirrored.



Using the Create File System Form

As shown in Figure 5-10, the New Volume Wizard File System formprompts for file system type (UNIX file system [UFS] or VxFS) and mountoptions. VEA automatically creates the mount point, modifies the/etc/vfstab file, and initializes and mounts the new file system.

The New File System Details button enables newfs and mkfs option entry.The Mount File System Details button allows volume ownership andprotection entry.

Figure 5-10 New Volume Wizard File System Form

Note – The VEA server logs the commands that perform all functions inthe /var/vx/isis/command.log file. The log file is a useful learning tool.

Creating Volumes Using the vxassist Command



The vxassist program performs a wide range of volume-related taskswithout the complexity of lower-level programs, such as vxvol . Thevxassist program also protects against many mistakes that can be madewhen using lower-level command-line programs.

The vxassist Command

The vxassist command has many options. Most of the options usedefault values if values are not explicitly entered.

The most basic form of the vxassist command, which creates a volume,is:

# vxassist make vol02 50m

The problem with this format is that is assumes the following:

● A default disk group, if one is configured

● The default volume type is a basic concatenation

● Any disk drives that have available space might be used

If you do not specify options, the vxassist command probably cannotgive you what you need. It can also create a poorly performing volume orcontribute to poor performance of other volumes.

Specifying Volume Size

Volume sizes can be specified using the following units:

● b (blocks)

● g (gigabytes)

● k (kilobytes)

● m(megabytes)

● s (sectors)



Using vxassist Command Options

If you furnish even a few options with the vxassist command, theoutcome is more clearly defined. A typical command using limitedoptions is:

# vxassist -g dg2 make newvol 2g \layout=raid5,nolog disk01 disk02 disk03

This form of the vxassist command is more explicit and guarantees thatthe following are true:

● The disk group that is used is dg2 .

● The name of the volume is newvol .

● The size of the volume is 2 Gbytes.

● This is a RAID-5 volume without a log and with three columns.

● All disk space comes from disk01 , disk02 , and disk03 .

Although there are many vxassist command options, only a few arecommonly used. Some of them require careful study. Always read themanual (man) pages and related documentation before attempting to usemost of these options.

The vxintro and vxassist man pages contain useful information that isdifficult to find elsewhere.

Default Option Values

Pay close attention to the default values of command options. Whenspecifying volume layouts, ensure that you understand the default valueor configuration for a particular volume type. The following examplesshow some layout specifications for RAID-5 volume types:

layout=raid5 (creates a RAID-5 log by default)

layout=raid5log

layout=raid5,nolog

layout=noraid5log

The first two variations are equivalent and create the same volumestructure. The last two are also equivalent.



Examples of vxassist Command Options

The following examples show each type of supported volume structure.The layered volumes require a larger minimum number of disk drives toimplement.

# vxassist -g dg3 make newvol 20m \layout=concat disk01

# vxassist -g dg3 make newvol 20m \layout=stripe disk01 disk02

# vxassist -g dg3 make newvol 20m \layout=mirror-concat disk01 disk02

# vxassist -g dg3 make newvol 20m \layout=mirror-strip disk01 disk02 disk03 disk04

# vxassist -g dg3 make newvol 20m \layout=concat-mirror disk01 disk02 disk03 disk04

# vxassist -g dg3 make newvol 20m \layout=stripe-mirror disk01 disk02 disk03 disk04

# vxassist -g dg3 make newvol 20m \layout=raid5,nolog disk01 disk02 disk03

The vxassist command can frequently determine the best way to use thespecified disk drives (media names) in a volume structure.

Modifying Volume Access Attributes


Modifying Volume Access Attributes

When new volumes are created in a disk group, they are given a set ofdefault access attributes that include:

● Owner

● Group

● Mode

The owner, group, and mode are usually those of the root user. For somevolumes, especially raw volumes that are used by a database, the volumeownership must be modified.

Caution – Do not use the chown, chgrp , or chmod command to set rawvolume attributes. This is because the attributes revert to their originalvalues after each system reboot. Change raw volume attributes usingVxVM commands.

Verifying Volume Ownership

The ownership and permissions of raw volumes can be checked likeordinary system files. You use the ls command to examine the rawvolume files in the /dev/vx/rdsk /dg_name directories.

# ls -l /dev/vx/rdsk/newDG/testvolcrw------- 1 root root 199,73000 Feb3 20:37 /dev/vx/rdsk/newDG/testvol

Modifying Volume Ownership and Permissions

To run applications, such as ORACLE 9i RAC, it might be necessary tochange read/write permissions and ownership of the raw volumes. Youuse the vxedit command to change the raw volume’s permissions orownership.

# vxedit -g tpcs set user=oracle group=dba mode=660 acct06

Note – You can also use the VEA GUI New Volume Wizard to set volumeownership and permissions during initial volume creation. Or, later, youcan use the Actions menu File System entry to create a file system andconfigure mount information for an existing volume.

Adding a UFS File System to Existing Volumes



Adding a UFS file system to a volume is easy to do. You can fill out a formusing the VEA GUI, or you can create a file system from the commandline using standard Solaris OS commands. Both methods are described inthis section.

Note – VxVM also supports the VxFS file system type, but the VxFSfeatures are licensed separately.

Using the VEA GUI to Add a File System

To add a new file system to an existing volume using the VEA GUI,highlight the target volume in the grid area and select File System NewFile System from the Actions menu. The resulting Create File SystemForm is identical to the New Volume Wizard Create File System Form.

New File System Details Form

Using the New File System Details Form, shown in Figure 5-11, you canmake fundamental file system changes.

Figure 5-11 New File System Details Form

You enter a comma-separated list of mkfs file system options in the Extraoptions text field. Consult the mkfs and mkfs_ufs man pages for moredetailed option information.



Mount Details Form

You can enter valid file system mount options, as shown in Figure 5-12.

Figure 5-12 Mount Details Form



Adding a File System From the Command Line

When a new file system is initialized from the command line, you canadjust certain file system parameters to make more efficient use ofavailable space. They are:

● Minimum file system free space

● Number of bytes per inode

File System Free Space

By default, the newfs utility calculates the minimum free space basedon partition size (64 Mbytes ÷ partition size × 100), rounded down to thenearest integer. The default value is limited to between 1 percent and10 percent.

When the file system is full, the free space can only be accessed by userroot . It can act as an emergency overflow.

# newfs -m 10 /dev/vx/rdsk/newDG/vol_01

In very large file systems, you can safely set the minimum free space to asmaller percentage.

Number of Bytes per Inode

The newfs utility calculates the number of bytes per inode based on filesystem size. By default, the newfs utility calculates the number of inodesas follows:

● 2048 bytes per inode for 0-1 Gbyte file system size

● 4096 bytes per inode for 1-2 Gbytes file system size

● 6144 bytes per inode for 2-3 Gbytes file system size

● 8192 bytes per inode for file system larger than 3 Gbytes

If you intend to create a large file system that will contain a small numberof very large files, you might be able to decrease the total number ofinodes, for example:

# newfs -i 10240 /dev/vx/rdsk/newDG/vol01



Enabling the Solaris OS UFS Logging Feature

UFS logging is a standard feature of the Solaris 8 OS.

If the logging option is specified for a file system, then logging is enabledfor the duration of the mounted file system. Logging is the process ofstoring transactions (changes that make up a complete UFS operation) in alog before the transactions are applied to the file system. After atransaction is stored, the transaction can be applied to the file system. Thisprocess prevents file systems from becoming inconsistent, thereforeeliminating the need to run the fsck command. Because the fsckcommand can be bypassed, logging reduces the time required to reboot asystem if it crashes or after an unclean halt. The default behavior is nologging.

The log is allocated from free blocks on the file system, and it is sized atapproximately 1 Mbyte per 1 Gbyte of file system, up to a maximum of64 Mbytes. Logging can be enabled on any UFS, including root (/ ). Thelog created by UFS logging is continually flushed as it fills up. The log istotally flushed when the file system is unmounted or when thelockfs -f command is used.

Example of File System Commands

The following commands summarize the types of operations that must beperformed to manually create a UFS logging file system on an existingVxVM volume.

# newfs /dev/vx/rdsk/newDG/testvol

# mkdir /Test

# vi /etc/vfstab/dev/vx/dsk/newDG/testvol /dev/vx/rdsk/newDG/testvol /Testufs 1 yes logging

# mount /Test

Note – The default file system behavior is nologging .

Administering Volume Logs



Both mirrored volumes and RAID-5 volumes can have logs. However, thelogs for RAID 5 perform an entirely different function than the mirroredvolume log, which is called a DRL.

Using DRLs

A DRL is a VxVM log file that tracks data changes made to mirroredvolumes. The DRL speeds recovery time when a failed mirror must besynchronized with a surviving mirror.

A DRL is a small, special-purpose plex attached to a mirrored volumewhich has the following features:

● It is a log that keeps track of the regions within volumes that havechanged as a result of write operations to a plex. It does this bymaintaining a bitmap and storing this information in a log subdisk.

● After a system failure, only the regions marked as changed (dirty) inthe DRL are recovered.

The following example shows a mirrored volume with a DRL. Notice thatthe log subdisk does not reside on either of the mirror disk drives.

# vxprint -g newDG mirvolTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv mirvol fsgen ENABLED 4096 - ACTIVEpl mirvol-01 mirvol ENABLED 7182 - ACTIVEsd ndg-01-01 mirvol-01 ENABLED 7182 0 -pl mirvol-02 mirvol ENABLED 7182 - ACTIVEsd ndg-03-01 mirvol-02 ENABLED 7182 0 -pl mirvol-03 mirvol ENABLED LOGONLY - ACTIVEsd ndg-02-03 mirvol-03 ENABLED 33 LOG -

DRLs are very small compared to the volume data storage.



Using RAID-5 Logs

RAID-5 logs help prevent data corruption in case a system crashes duringa write operation. Without logging, if a system fails during a writeoperation, there is no way to tell if the data and parity were both writtento the disk drives. This could result in corrupted data.

When RAID-5 logging is used, a copy of the data and parity are written tothe RAID-5 log before being written to the disk drives.

RAID-5 logging is optional, but RAID-5 logs are created by default. Youshould always run a system with RAID-5 logs to ensure data integrity.

The following example shows a RAID-5 volume with a log. Notice thatthe log subdisk does not reside on either of the stripe disk drives.

# vxprint -g newDG raidvolTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv raidvol raid5 ENABLED 4096 - ACTIVEpl raidvol-01 raidvol ENABLED 7168 - ACTIVEsd ndg-02-02 raidvol-01 ENABLED 3591 0 -sd ndg-01-03 raidvol-01 ENABLED 3591 0 -sd ndg-03-03 raidvol-01 ENABLED 3591 0 -pl raidvol-02 raidvol ENABLED 3591 - LOGsd ndg-04-02 raidvol-02 ENABLED 3591 0 -

The size of RAID-5 logs is automatically set by VxVM. It is dependent onthe stripe width of the volume. The larger the stripe width (not volume),the larger the RAID-5 log. The RAID-5 log is intended to hold severalfull-stripe writes simultaneously.

The default log size for a RAID-5 volume is four times the full stripewidth (the stripe unit size × the number of stripe columns).



Planning Log Placement

Although logs can be very beneficial to volume recovery after a systemcrash, if they are not properly managed they can create I/O bottlenecksthat negatively impact system performance.

You should plan for both RAID-5 logs and DRLs in advance.

You should take special care with RAID-5 log placement because the datawritten to all RAID-5 stripe units is also written to the log.

As shown in Figure 5-13, leaving a small amount of free space at the endof all disk drives ensures that you always have alternate locations for logplacement or relocation.

Figure 5-13 Log Space Allocation

If possible, a log should not reside on the same disk drive as its relatedvolume.

Note – Log sizes are calculated automatically by the VxVM software.

Vol02_log Log Space Log Space Log Space

Log Space Log Space Log Space Vol01_log

Volume 01

Volume 02



Adding a Volume Log From the VEA GUI

Usually, DRL and RAID-5 logs are added when a new volume is created.However, in some cases it might be preferable to run performance testsbefore deciding on the placement of logs.

To add a DRL or RAID-5 log to an existing volume, click the volume inthe grid area. Next, click the Actions menu and select Log Add from itspop-up menu.

As shown in Figure 5-14, you can either let VxVM automatically assign asuitable log disk or you can enable manual disk assignment.

Figure 5-14 VEA Add Log Form



Adding a Volume Log From the Command Line

You can use the vxassist command to add or remove a log from avolume. Both DRL and RAID-5 logs can be added this way. By default,when adding a log, the vxassist command attempts to determine thecorrect type of log to add and computes an appropriate size for the log.Adding a DRL log to a mirrored volume creates a single log subdisk anda new plex to contain that subdisk. The new plex is then attached to thevolume.

Adding a DRL

To prevent I/O bottlenecks, a DRL should not reside on a disk drive usedby its related volume. It is best to specify the disk drive (media name)where the DRL should be placed. If the DRL location is not specified, thevxassist command assesses the available disk space and decides whereto place the log. The following example shows the addition of a DRL to amirrored volume.

# vxassist addlog mirvol ndg-02# vxprint -g newDG mirvolTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv mirvol fsgen ENABLED 4096 - ACTIVEpl mirvol-01 mirvol ENABLED 7182 - ACTIVEsd ndg-01-01 mirvol-01 ENABLED 7182 0 -pl mirvol-02 mirvol ENABLED 7182 - ACTIVEsd ndg-03-01 mirvol-02 ENABLED 7182 0 -pl mirvol-03 mirvol ENABLED LOGONLY - ACTIVEsd ndg-02-03 mirvol-03 ENABLED 33 LOG -

Adding a RAID-5 Log

The process for adding a RAID-5 log is the same as for a adding a DRL.The following example shows the addition of a log to a RAID-5 volume.

# vxassist addlog raidvol ndg-04# vxprint -g newDG raidvolTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv raidvol raid5 ENABLED 4096 - ACTIVEpl raidvol-01 raidvol ENABLED 7168 - ACTIVEsd ndg-02-02 raidvol-01 ENABLED 3591 0 -sd ndg-01-03 raidvol-01 ENABLED 3591 0 -sd ndg-03-03 raidvol-01 ENABLED 3591 0 -pl raidvol-02 raidvol ENABLED 3591 - LOGsd ndg-04-02 raidvol-02 ENABLED 3591 0 -



Removing Volume Logs Using the VEA GUI

To remove a volume log using the VEA GUI, click the volume in the gridarea. Next, click the Actions menu and select Log Remove from its pop-upmenu. Volumes can have more than one log. The Remove Log form,shown in Figure 5-15,allows you to remove logs either by name or by diskdrive.

Figure 5-15 VEA Remove Log Form

The Quantity/Disk removal method provides two options. You can enterthe quantity of logs to be removed, or you can specify one or more diskdrives on which to preserve log copies.



Removing Volume Logs From the Command Line

The vxassist command format to remove volume logs is the sameregardless of the volume type. The following example shows the removalof logs from both a mirrored volume and a RAID-5 volume.

# vxassist remove log mirvol# vxassist remove log raidvol

Using the VEA GUI to Analyze Volume Structures



You use the VEA GUI to visually examine volume structure, diskallocation, and basic performance.

Displaying Volume Layout Details

To display volume layout details, click on a volume in the grid area andselect Layout View from the pop-up menu. As shown in Figure 5-16, usethe right mouse button in the Volume Layout window to select a volumecomponent and examine its properties.

Figure 5-16 Volume Layout Window



Viewing Disk Volume Mapping and Performance

To view volume-to-disk mapping and performance, click on a disk groupin the VEA GUI and select Volume Disk from the pop-up menu.

The Volume Disk Map, shown in Figure 5-17, displays a map of eachvolume and its associated disk drives.

You can also enable crude performance monitoring by clicking on theView menu and selecting Collect Statistics from the pop-up menu. Thedisk icons change color to indicate basic levels of I/O activity.

Figure 5-17 Volume Disk Map

Exercise: Creating Volumes and File Systems





● Create a volume

● Add a volume mirror

● Add a file system to a volume

● Add a dirty-region log

● Resize a volume and file system

● Create a RAID-5 volume

● Analyze volumes using the VEA GUI

● Verify ending lab status

Preparation

The purpose of this lab is to have you create and destroy VxVM objectsusing both the VEA GUI and command-line programs. Each method hasadvantages.

Ask your instructor for the following information:

● A set of unique volume names for your workgroup:

Concat/mirror volume name: __________________________________

RAID-5 volume name: ________________________________________

● A unique mount point name for your workgroup:

Mount point name: ___________________________________________

Your workgroup’s assigned disk drives should be configured as follows:

● You have two disk groups name dgX and dgY.

● Each disk group contains three disk drives.

● None of your assigned disk drives are designated as hot spares.

Note – Substitute your workgroup codes for the X and Y in dgX and dgY.




Answer the following questions about volume planning, volume access,and volume logging:

1. Why should you avoid automatic volume naming?

a. Automatic names are limited to 32 variations.

b. Automatic names cannot be used by some databases.

c. Automatic names make administration difficult.

d. Automatic names can be very long.

The answer is c.

2. Which of the following commands are useful for space allocationresearch?

a. vxassist maxsize

b. vxdisksetup -i

c. vxdisk list

d. vxdg free

e. vxdiskadm

The answers are a and d.

3. Which of the following volume types has the fastest recovery time?

a. Striped volumes

b. Layered volumes

c. Mirrored volumes

The answer is b.

4. Which of the following volume types has the highest disk drivefailure tolerance?

a. Concatenated mirror

b. Mirrored concatenation

c. RAID 5

d. Striped mirror

The answer is d.



5. Which of the following commands modifies volume accessattributes?

a. vxdisk define

b. chmod -R

c. vxedit set

d. ln -s

The answer is c.

6. What is the purpose of dirty region logging?

a. To increase volume write performance

b. To prevent file system corruption

c. To speed up volume mirror resynchronization

d. To control file system access

e. To prevent internal volume corruption

The answer is c.

7. What is the purpose of RAID-5 logging?

a. To increase volume write performance

b. To prevent file system corruption

c. To speed up volume mirror resynchronization

d. To control file system access

e. To prevent internal volume corruption

The answer is e.



Task 2 – Creating a Volume

Perform both variations of this task in the order they occur.

Using the VEA GUI to Create a Volume

To create a volume using the VEA GUI, complete the following steps:

1. Display your first disk group (dgX) in the grid area.

2. In the toolbar, select New Volume.

3. Configure the New Volume Wizard as follows:

● Manually select disks to use for the volume.

● Select only one of the disks for use in the new volume.

● Enter your assigned concatenated volume name.

● Select the Concatenated layout.

● Select Maxsize.

● After the Maxsize calculation has completed, type 200 in theSize window and select MB from its pull-down menu.

● Select No file system.

● Review the final configuration parameters and click Finish.

Note – The Maxsize feature can be useful when you are trying tomaximize the size of a new volume and when you have limited disk drivespace.

4. Check the status of the new volume by using the vxprint command.

5. Verify that your new volume has a single plex with one subdisk andthat the volume and plex are ENABLEDand ACTIVE.

6. Display the new volume in the grid area.

7. Click the new volume in the grid area and select Properties from itspop-up menu.

8. Examine the volume’s properties and click Cancel when you arefinished.



Using the Command Line to Create a Volume

To create a volume using the command line, use the man pages, asnecessary, to complete the following steps:

1. Open a window and use the rlogin or telnet command to log intothe VxVM server as user root .

2. Stop the volume you created in the previous procedure.

# vxvol -g disk_group stop volume_name

3. Recursively remove the volume.

# vxedit -g disk_group -rf rm volume_name

What is the purpose of the vxedit -f option?

_____________________________________________________________

4. Re-create the 200-Mbyte concatenated volume again by using thevxassist command. You must specify the following items:

● The disk group the volume should be in (-g disk_group )

● The make option

● The name and size of the volume (volname 200m)

● The volume layout (layout=concat )

● The disk drive (media name) you want to use

5. Record the command you used to create the volume.

_____________________________________________________________

Note – It is not necessary to use the layout= option for a concatenatedvolume. The default layout is a concatenation. Any other layout requires aspecific layout option. For example, layout=mirror or layout=raid5 .

6. Use the vxprint command to verify the volume status is showingENABLEDand ACTIVE.

Note – If there is any problem with your new volume, consult with yourinstructor.



Task 3 – Adding a Volume Mirror

Perform both variations of this task in the order that they occur.

Using the VEA GUI to Add a Mirror

To add a mirror to an existing volume, complete the following steps:

1. Display your new volume in the grid area and click the volume.

2. Click the Actions menu and select Mirror Add from its pop-upmenu, as shown in Figure 5-18.

Figure 5-18 Add Mirror Menu



3. Leave the first Add Mirror form configured with its default valueswhich should include the following:

● Number of mirrors to add: 1

● Layout=Concatenated

● Let Volume Manager decide which disks to use

4. On the VxVM server, check the status of the mirrorresynchronization by using the vxtask list command.

5. On the VxVM server, verify the state of your new mirror by usingthe vxprint command.

You should now see two plexes in your volume. Large mirrors take awhile to synchronize.

Until the resynchronization is complete, the related plex is in aTEMPRMSDstate. Consult the vxinfo man page for volume statedefinitions.

Note – Remember that you can review the command-line operations inthe /var/vx/isis/command.log file on the VxVM server.

Using the Command Line to Add a Volume Mirror

To remove a mirror from a volume, and then create a new mirror,complete the following steps:

1. Ensure that both plexes (mirrors) in the volume you added are fullysynchronized and show a status of ENABLEDand ACTIVE.

2. Use the vxassist command to remove one of the mirrors from yourvolume.

For example:

# vxassist -g dgX remove mirror vol_01

3. Use the vxprint command to verify that your volume now has asingle plex and subdisk.

4. Use the vxassist command to re-create the mirrored volume.

5. Use the alloc= parameter to specify the disk media name on whichthe new mirror is to be created.

For example:

# vxassist -g dgX mirror vol_01 alloc=dgX03



6. After the vxassist returns, use the vxprint command to verify thevolume has two plexes and its status is ENABLEDand ACTIVE.

Consult with your instructor if you are having problems.

Note – You can also move a volume mirror to a different disk drive if it ispoorly placed and is causing a performance problem.

Task 4 – Adding a File System to a Volume


Using the VEA GUI to Add a File System

To add a file system using the VEA GUI, complete the following steps:

1. Click your mirrored volume in the grid area, and select File SystemNew File System from the pop-up menu.

2. Configure the New File System form as follows:

● Ensure that the File system type is ufs.

● Leave the Allocation at its default value (1024).

● Enter your assigned mount name.

● Select Create mount point.

● Deselect Read only and Honor setuid.

● Select Add to file system table and mount at boot.

● Set the fsck pass number to 2.

● Examine the New file System Details menu

● Examine the Mount File System Details menu.

● Click OK.

3. On the VxVM server, verify that the following are true:

● The mount point is present in the root directory.

● Your file system is mounted.

● The mount entry is in the /etc/vfstab file.

● The df -kl output seems appropriate for the volume size.



4. Create some test data in the volume’s file system.

# mkfile 2m / mountpoint /myfile

Using the Command Line to Add a File System

Review the command-line operations performed by the VEA software inthe previous section. Complete the following step:

Review the VEA command-line operations recorded in the log file on theVxVM server.

# tail -45 /var/vx/isis/command.log

The most recent commands are appended to the end of the file. Not all thedetails are logged in the commandfile, such as the edits to the/etc/vfstab file.

The following is a summary of using the command line to add a filesystem to an existing volume.

# mkfs -F ufs /dev/vx/rdsk/dgX/xvol-01 409600

# mkdir /Junk

# vi /etc/vfstab/dev/vx/dsk/dgX/xvol-01 /dev/vx/rdsk/dgX/xvol-01 /Junk ufs 1 yes logging

# mount -F ufs -o logging /dev/vx/dsk/ dgX/xvol-01 /Junk

Note – The file system vfstab and mount options enable the UFS loggingfeature. UFS logging is not necessary, but it offers additional file systemprotection and is part of the Solaris OS.



Task 5 – Adding a DRL


Using the VEA GUI to Add a DRL

To add a DRL using the VEA GUI, complete the following steps:

1. Verify there is a disk drive available for the DRL within the samedisk group that does not contain either plex of the mirrored volume.

2. Display your volume in the grid area and click it with the thirdmouse button.

3. Select Log Add in the volume pop-up menu.

4. In the Add Log window, complete the following steps:

a. Click Manual disk assignment.

b. Select a disk drive that is not part of the mirrored volume.

c. Click OK.

5. Return to the command line on the VxVM server and use thevxprint command to verify the following:

● The mirrored volume now has a log plex

● The log is not on the same disk drive as either of the volumemirrors

Note – Look at the subdisk entries to determine log placement.

6. In the VEA GUI, complete the following steps:

a. Click the volume in the grid area.

b. Select Properties from the pop-up menu.

c. Examine both the General and File System tabs.



Using the Command Line to Add a DRL

The following example shows the command sequence to remove and adda DRL to a volume. Practice removing and adding a DRL from yourvolume using the command line.

The disk media name you specify should be on a different disk drive thanthe disk drives used by the volume mirrors.

# vxassist remove log volume_name

# vxassist addlog volume_name media_name



Task 6 – Resizing a Volume and File System

If a volume has a file system, you can increase the size of both the volumeand the file system by using either the vxresize command or the VEAGUI.

Note – There are other methods of increasing a volume’s size and a filesystem’s size. However, the VEA GUI and vxresize command reliablyincrease the size of both the volume and its related file system at the sametime.

The volume to be resized should be of the type fsgen . The vxprintcommand displays the volume type.


Using the Command Line to Resize File Systems

To resize file systems using the command line, complete the followingsteps:

1. Add 2 Mbytes to the size of your mirrored volume and file systemby using the following command:

# vxresize -F ufs -g disk_group volume_name +2m

Note – You can also express the +2mas a new volume length without theplus sign. There are also -s and -x options that ensure the requested sizevalue is appropriate. You can also specify disk media names (for example,disk01 , disk02 ) that you want to be used for the new space.

2. Examine the new volume and file system to ensure that the changeshave taken place.

Large changes can take a long time.

Note – You cannot shrink a volume with a file system unless the filesystem is of VxFS type. Read the vxresize man page for a completedescription of restrictions.



Using the VEA GUI to Resize File Systems

To resize file systems using the VEA GUI, complete the following steps:

1. In the grid area, complete the following steps:

a. Display your volume.

b. Click your volume with the third mouse button.

c. Select Resize Volume in the pop-up menu.

2. Configure the Resize Volume form, shown in Figure 5-19, as follows:

● Enter 2 in the Add By window and select MB from its pull-down menu.

● Let VxVM decide which disks to use for the additional space.

● Click OK.

Figure 5-19 Resize File System Form

3. After the task has completed, verify the results.



Task 7 – Creating a RAID-5 Volume

In this task you create a three-column RAID-5 volume with a separate logdisk (four disk drives total) in your second disk group.


1. On the VxVM server, complete the following steps:

a. Unmount the mirrored volume file system.

b. Stop the volume.

c. Remove its mount entry from the /etc/vfstab file.

2. Destroy the mirrored volume disk group (dgX).

# vxdg destroy disk_group

3. Click the remaining disk group with the third mouse button andselect Add Disk to Disk Group from its pop-up menu.

4. Using the Add Disk Wizard, add the three disk drives from thedestroyed disk group to the remaining disk group.

5. On the VxVM server, calculate the available disk space using four ofthe disk drives in a RAID-5 volume with a log.

For example:

# vxassist -g dgB maxsize layout=raid5log dgB01 dgB02 dgB03 dgB04Maximum volume size: 35348480 (17260Mb)

Caution – Do not create a maximum-size volume.

6. Display the disk drives in your disk group in the grid area.

7. Select four of the disk drives by simultaneously pressing the Controlkey while using the left mouse button.

8. Select the New Volume button in the toolbar.

9. Configure the New Volume form as follows:

● Enter your assigned RAID-5 volume name.

● Enter 200 in the Size field.

● Select MB from the Size pull-down menu.

● Choose RAID 5 in the Layout area. The Number of Columnsfield should automatically be set to 3 with logging is enabled.

● Leave the default Stripe Unit Size at 32.

● Click Next.



● Click Create a File System, and complete the following steps:

● Select a FS Type of ufs .

● Enter your assigned mount point.

● Click Add to File System Table, Mount at Boot and set the fsckpass to 2.

10. Click Next in the Create File System form.

11. Click Finish in the Summary form after you verify that theconfiguration is correct.

12. On the VxVM server, check the status of the new RAID-5 volume byusing the vxprint command.

# vxprint -g dgB raidvolTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv raidvol raid5 ENABLED 409600 - ACTIVEpl raidvol-01 raidvol ENABLED 416512 - ACTIVEsd dgB01-01 raidvol-01 ENABLED 208278 0 -sd dgB04-01 raidvol-01 ENABLED 208278 0 -sd dgB05-01 raidvol-01 ENABLED 208278 0 -pl raidvol-02 raidvol ENABLED 3591 - LOGsd dgB06-01 raidvol-02 ENABLED 3591 0 -

Note – You can add the log later to better control its placement. The diskdrives are not necessarily used in the order you selected them.

13. On the VxVM server, examine the volume creation commands.


14. In the VEA GUI, click the volume with the third mouse button andselect Properties from its pop-up menu.

15. Examine both the General and File System tabs.

16. Cancel the Volume Properties window.

17. Edit the /etc/vfstab file, and comment out or delete the obsoletemount entry from the earlier mirrored volume file system mount.

You must remove or comment out obsolete mount entries becausethey cause errors at boot time.



Task 8 – Analyzing Volumes Using the VEA GUI

The following sections detail different VEA GUI features that are used toanalyze the structure and performance of VxVM volumes.

Displaying Volume Layout Details

To display volume layout details, complete the following steps:

1. Display your volumes in the grid area and complete the followingsteps:

a. Click your RAID-5 volume with the third mouse button.

b. Select Layout View from the pop-up menu.

2. As shown in Figure 5-20, use the third mouse button to highlight avolume component and examine its properties.

Figure 5-20 Volume Layout Window



3. Examine the pop-up menus for each of the different components.

4. Close the Volume Layout window.

Viewing Disk Volume Mapping and Performance

To view volume-to-disk mapping and performance, complete thefollowing steps:

1. In the grid area, click your disk group with the third mouse button,and select Volume/Disk from the pop-up menu.

The Volume/Disk map, shown in Figure 5-21, assists in volumeplanning and disk utilization assesment.

Figure 5-21 Volume/Disk Map

2. Click the View menu with the third mouse button and select DataGathering Options from the pull-down menu.



3. Perform the following steps:

a. Modify the Refresh Interval to 5 seconds and select WriteRequests from the pull-down menu, as shown in Figure 5-22.

b. Click OK.

Figure 5-22 Data Gathering Options



4. Enable performance monitoring by performing the following steps:

a. Click the View menu with the third mouse button.

b. Select Collect Statistics from the pop-up menu, as shown inFigure 5-23.

Figure 5-23 Enabling Performance Monitoring

When performance monitoring is enabled, the relative I/Oactivity of each volume disk drive is indicated by color changesin the small disk icons shown in Figure 5-23.

Red indicates the highest level of I/O activity or slowestperformance. This information can be used to identify basicperformance bottlenecks in disk group volumes.



5. Use the mkfile command to create some test data into your filesystem volume.

6. Observe the activity levels in the Disk/Volume performance display

For example: mkfile 20m /Test/file1

Figure 5-24 Data Activity Indicators



Task 9 – Verifying Ending Lab Status

Type the vxprint command and verify your current VxVM configurationhas the following features:

● One disk group containing six disks

● One RAID-5 volume with a log, mounted

● Two free disk drives left in the disk group

Exercise Summary


Exercise Summary

?!



● Experiences


● Interpretations


● Conclusions


● Applications



Module 6

VERITAS Volume Manager AdvancedOperations

Objectives


● Encapsulate and mirror the system boot disk

● Configure a best practice boot disk

● Administer hot spares and hot relocation

● Evacuate all subdisks from a disk drive

● Move a disk drive without preserving data

● Move a populated disk drive to a new disk group

● Backup and restore a disk group configuration

● Describe how to import a disk group after a system crash

● Perform a volume snapshot backup

● Perform an online volume relayout

● Create VxVM layered volumes

● Perform basic Intelligent Storage Provisioning administration

● Replace a failed disk drive

Boot Disk Encapsulation and Mirroring



When you install the VxVM software on a system, you can place yoursystem boot disk under VxVM control. Later, you can mirror the boot diskto increase availability. You should optimize your boot disk configurationbefore starting the encapsulation process.

Optimizing the Boot Disk Hardware Configuration

Although there are many possible boot disk configuration variations, thissection focuses on the preferred boot disk hardware configuration, whichis shown in Figure 6-1.

Figure 6-1 Boot Disk Hardware Configuration

The ideal boot disk hardware configuration has the following features:

● The boot disk and mirror are on separate interfaces.

● The boot disk and mirror are not in a storage array.

● Only the boot disk and mirror are in the rootdg disk group.

SCSIc0

SCSIc1

SOC

Storage Array

disk group

disk group

c2

rootdg

newdg

rootvol-01 rootvol-02


VERITAS Volume Manager Advanced Operations 6-3Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

Boot Disk Encapsulation Prerequisites

For the boot disk encapsulation process to succeed, the followingprerequisites must be met:

● The boot disk must have at least two unused slices.

● The boot disk must not have any slices in use other than slice 2 andthe following:

● root

● swap

● var

● opt

● usr

There must be at least 2048 sectors at the beginning or end of the bootdisk that are not assigned to any partition. These sectors are needed forthe private region. If necessary, VxVM takes the space from the end of theswap partition, but this can create a difficult-to-manage boot diskconfiguration.

The following is an ideal boot disk partition map, before encapsulation,that has five unconfigured partitions and five cylinders of unassigneddisk space (3875-3879) at the end of the disk.

# format -d c0t0d0......Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 3399 3.50GB (3400/0/0) 7344000 1 swap wu 3400 - 3874 500.98MB (475/0/0) 1026000 2 backup wm 0 - 3879 4.00GB (3880/0/0) 8380800 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

Note – If your boot disk does not have any free cylinders, you boot thesystem from CD-ROM in single-user mode. You use the format utility tomodify the swap partition size and relabel the disk.



Encapsulating the System Boot Disk

It is best to use the vxdiskadm utility option 2, Encapsulate one ormore disks , to encapsulate the system boot disk. The following exampleshows pertinent excerpts from a typical dialog.

# vxdiskadm


Select disk devices to encapsulate:[<pattern-list>,all,list,q,?] c0t0d0

Which disk group [<group>,list,q,?] rootdgThere is no active disk group named rootdg.Create a new group named rootdg? [y,n,q,?] (default: y) yUse a default disk name for the disk? [y,n,q,?] (default: y) y

A new disk group rootdg will be created and the disk device c0t0d0 willbe encapsulated and added to the disk group with the disk name rootdg01


The c0t0d0 disk has been configured for encapsulation.The first stage of encapsulation has completed successfully. You shouldnow reboot your system at the earliest possible opportunity. Theencapsulation will require two or three reboots which will happenautomatically after the next reboot. To reboot execute the command:

shutdown -g0 -y -i6

This will update the /etc/vfstab file so that volume devices are used tomount the file systems on this disk device. You will need to update anyother references such as backup scripts, databases,or manually createdswap devices.



Mirroring the System Boot Disk

You use the vxdiskadm utility to add a mirror disk to the rootdg diskgroup and to mirror the boot disk. The following examples showpertinent excerpts from the vxdiskadm dialog.

Adding a Second Disk to the rootdg Disk Group

You use the vxdiskadm utility option 1, Add or initialize one ormore disks , to add a second disk drive to the rootdg disk group. Thefollowing example shows a completed disk group that is ready formirroring.

# vxdisk -g rootdg listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:sliced rootdg01 rootdg onlinec0t1d0s2 auto:sliced rootdg02 rootdg online nohotuse

Mirroring the Boot Disk

You use the vxdiskadm utility option 6, Mirror volumes on a disk , tomirror the encapsulated boot disk. The following example showspertinent excerpts from the vxdiskadm dialog.

# vxdiskadmSelect an operation to perform: 6Mirror volumes on a disk

Enter disk name [<disk>,list,q,?] rootdg01Enter destination disk [<disk>,list,q,?] (default: any) rootdg02

The requested operation is to mirror all volumes on disk rootdg01 in disk group rootdg onto available disk space on disk rootdg02. VxVM NOTICE V-5-2-229 NOTE: This operation can take a long time tocomplete.

Continue with operation? [y,n,q,?] (default: y) yVxVM vxmirror INFO V-5-2-22 Mirror volume rootvol ...VxVM vxmirror INFO V-5-2-22 Mirror volume swapvol ... VxVM INFO V-5-2-674 Mirroring of disk rootdg01 is complete.

Mirror volumes on another disk? [y,n,q,?] (default: n) n



Verifying the Completed Boot Disk Configuration

The following examples show an encapsulated and mirrored boot disk. Asecond boot device alias is automatically configured when the boot disk ismirrored.

Verifying the VxVM Boot Device Aliases

When a boot disk is encapsulated, the VxVM software automaticallycreates boot device aliases for use when booting from the encapsulatedboot disk root partition or the boot disk mirror, for example:

# eeprom nvramrcnvramrc=devalias vx-rootdg01 /pci@1f,4000/scsi@3/disk@0,0:adevalias vx-rootdg02 /pci@1f,4000/scsi@3/disk@1,0:a

You cannot boot the system using the device aliases until the OpenBootuse-nvramrc? variable is set to true . After the variable is enabled, youcan boot from the primary or mirror boot device aliases, for example:

# eeprom “use-nvramrc?”=true# init 0ok boot vx-rootdg01# init 0ok boot vx-rootdg02

Verifying the VxVM Reserved Disk Group Variables

VxVM has two reserved variables named defaultdg and bootdg . Unlessspecial action is taken, both of the variables are set to a value of nodg .When you encapsulate the system boot disk and place it in the rootdgdisk group, the bootdg variable is automatically updated.

The following example shows the VxVM reserved variables:

# vxdg defaultdgnodg

# vxdg bootdgrootdg



Verifying the rootdg Disk Group Structures

The following is an example of a typical encapsulated and mirrored bootdisk volume structure.

# vxprint -g rootdgTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg rootdg rootdg - - - -

dm rootdg01 c0t0d0s2 - 17674902 - -dm rootdg02 c0t1d0s2 - 17678493 - NOHOTUSE

v rootvol root ENABLED 17139843 - ACTIVEpl rootvol-01 rootvol ENABLED 17139843 - ACTIVEsd rootdg01-B0 rootvol-01 ENABLED 1 0 -sd rootdg01-02 rootvol-01 ENABLED 17139842 1 -pl rootvol-02 rootvol ENABLED 17139843 - ACTIVEsd rootdg02-01 rootvol-02 ENABLED 17139843 0 -

v swapvol swap ENABLED 531468 - ACTIVEpl swapvol-01 swapvol ENABLED 531468 - ACTIVEsd rootdg01-01 swapvol-01 ENABLED 531468 0 -pl swapvol-02 swapvol ENABLED 531468 - ACTIVEsd rootdg02-02 swapvol-02 ENABLED 531468 0 -

Verifying the Encapsulated Boot Disk Partitioning

The following is the boot disk partition map after the encapsulationprocess is completed. The public region (slice 3) is mapped to the entiredisk. The private region (slice 4) is mapped to the last cylinder. The rootand swap data are in the same location as before the encapsulation.

# format -d c0t1d0partition> pCurrent partition table (original):Total disk cylinders available: 4924 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 4772 8.17GB (4773/0/0) 17139843 1 swap wu 4773 - 4920 259.51MB (148/0/0) 531468 2 backup wm 0 - 4923 8.43GB (4924/0/0) 17682084 3 - wu 0 - 4923 8.43GB (4924/0/0) 17682084 4 - wu 4922 - 4923 3.51MB (2/0/0) 7182 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

Creating a Best Practice Boot Disk Configuration



If you encapsulate and mirror your boot disk, you should considerrefining the configuration to improve emergency recovery andserviceability.

If there are no free cylinders for use as a private region duringencapsulation, VxVM takes them from the swap space and places theprivate region (slice 4) in the middle of the public region address space(slice 3). The private region is protected by a subdisk overlay namedrootdg01Priv . Also, block 0 on the boot disk is protected with a subdiskoverlay named rootdg01-B0 . The following example shows anencapsulated boot disk structure:

# format -d c0t0d0...Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 3614 3.72GB (3615/0/0) 7808400 1 swap wu 3618 - 3879 276.33MB (262/0/0) 565920 2 backup wm 0 - 3879 4.00GB (3880/0/0) 8380800 3 - wu 0 - 3879 4.00GB (3880/0/0) 8380800 4 - wu 3615 - 3617 3.16MB (3/0/0) 6480 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

# vxprintDisk group: rootdg

TY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg rootdg rootdg - - - -

dm rootdg01 c0t0d0s2 - 8380799 - -

sd rootdg01Priv - ENABLED 6479 - -

v rootvol root ENABLED 7808400 - ACTIVEpl rootvol-01 rootvol ENABLED 7808400 - ACTIVEsd rootdg01-B0 rootvol-01 ENABLED 1 0 -sd rootdg01-02 rootvol-01 ENABLED 7808399 1 -

v swapvol swap ENABLED 565920 - ACTIVEpl swapvol-01 swapvol ENABLED 565920 - ACTIVEsd rootdg01-01 swapvol-01 ENABLED 565920 0 -



When the boot disk is mirrored, the structure of the boot disk and themirror disk are not identical. This can be confusing and can add difficultyto service and recovery situations.

The partition maps of a wort-case boot disk and mirror disk configurationare organized in a different manner, for example:

# format -d c0t0d0...Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 3614 3.72GB (3615/0/0) 7808400 1 swap wu 3618 - 3879 276.33MB (262/0/0) 565920 2 backup wm 0 - 3879 4.00GB (3880/0/0) 8380800 3 - wu 0 - 3879 4.00GB (3880/0/0) 8380800 4 - wu 3615 - 3617 3.16MB (3/0/0) 6480 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

# format -d c0t1d0...Part Tag Flag Cylinders Size Blocks 0 root wm 3 - 3617 3.72GB (3615/0/0) 7808400 1 swap wu 3618 - 3879 276.33MB (262/0/0) 565920 2 backup wu 0 - 3879 4.00GB (3880/0/0) 8380800 3 - wu 3 - 3879 3.99GB (3877/0/0) 8374320 4 - wu 0 - 2 3.16MB (3/0/0) 6480 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0

The following differences exist between the two disk drives:

● The VxVM private region (partition 4) is in the middle of theprimary boot disk (c0t0d0 ) and is at the beginning of the mirror disk(c0t1d0 ).

● The root and swap partitions on the primary boot disk still remain inthe same physical location, aligned with the original partitionboundaries. They are offset on the mirror disk.

● The boot disk is still directly bootable (ok boot disk ). The mirrordisk must be booted using the nvramrc device alias vx-rootdg02 .



Establishing Boot Disk Configuration Policies

Optimized boot disk configurations can make recovery faster and safer.Administrators can adopt recovery procedures that work for any systemunder their control.

If possible, adhere to the following configuration policies:

● Document each system’s configuration and keep hard copies of theconfigurations easily available. Do not rely on having electroniccopies available.

● Keep the /usr and /opt directories on the root partition if possible.

The content of these directories is relatively stable and keeping themon the root partition simplifies the configuration.

Additionally, the /usr directory contains the VxVM recoverysoftware. Placing the root and /usr software on the same partitionreduces the possibility of having a bootable root file system, butlosing access to the VxVM recovery software residing on a separate/usr partition.

● If possible, also leave the /var directory on the root partition.

● Add an additional disk drive to the rootdg disk group as a hotspare.

Caution – Ensure that the mirror disk is initialized in the VxVM slicedformat and not the cdsdisk format. The cdsdisk format must not beused in the rootdg disk group.

Consult the Sun BluePrints™ document, Towards a Reference Configuration for VxVM Managed Boot Disks,for a more detailed explanation.



Modifying an Existing Boot Disk Configuration

The following section assumes that all disk drives and volumecomponents follow the default VxVM naming conventions.

After using the vxdiskadm utility to create a standard encapsulated bootdisk and mirror configuration, perform the following steps to reorganizethe disks into a best practice configuration.

1. Un-mirror the primary boot disk by removing the mirror plexes fromthe mirror disk. Use the Bourne shell.

# vxassist -g rootdg remove mirror rootvol !rootdg02# vxassist -g rootdg remove mirror swapvol !rootdg02

2. To ensure the correct mirror placement, manually mirror therootvol and swapvol volumes again in the order shown.

# vxrootmir rootdg02# vxassist -g rootdg mirror swapvol rootdg02

3. Disassociate all primary boot disk plexes and recursively removethem.

# vxplex -g rootdg -v rootvol dis rootvol-01# vxplex -g rootdg -v swapvol dis swapvol-01# vxedit -g rootdg -r rm rootdg01Priv# vxedit -g rootdg -r rm rootvol-01# vxedit -g rootdg -r rm swapvol-01

4. Verify that only the mirror disk plexes remain.



v rootvol root ENABLED 7808400 - ACTIVEpl rootvol-02 rootvol ENABLED 7808400 - ACTIVEsd rootdg02-01 rootvol-02 ENABLED 7808400 0 -




5. Remove the rootdg01 disk from the rootdg disk group andcomplete the following steps:

a. Reinitialize the rootdg01 disk.

b. Add the rootdg01 disk back into the rootdg disk group.

# vxdg -g rootdg rmdisk rootdg01# vxdisksetup -i c0t0d0 format=simple# vxdg -g rootdg adddisk rootdg01=c0t0d0

6. Use the vxdiskadm option 6, Mirror volume on a disk , to mirrorrootvol and swapvol on the rootdg02 disk drive back to the newlyinitialized rootdg01 disk drive.

# vxdiskadm...At the prompt below, supply the name of the diskcontaining the volumes to be mirrored.

Enter disk name [<disk>,list,q,?] rootdg02

Enter destination disk [<disk>,list,q,?] (default: any)rootdg01

7. Modify the boot-device parameter and enable nvramrc usage.

# eeprom boot-device="rootdisk rootmirror"# eeprom "use-nvramrc?"=true

8. Use the prtvtoc command and the format utility to verify that theprimary and mirror boot disk drives have exactly the same partitionmaps.

9. Boot the system from each of the available system devices.

ok boot disk0ok boot disk1 .ok boot vx-rootdg01ok boot vx-rootdg02

You can now replace a defective boot disk drive in the same manner asany other VxVM disk drive, and then resynchronize the mirrors.

Administering Hot Devices



Depending on how the /etc/rc2.d/S95vxvm-recover file is configured,either the older hot-spare daemon, vxsparecheck , starts at boot time orthe newer hot-relocation daemon, vxrelocd , starts at boot time.

The functionality of the two daemons is different. By default, hotrelocation is enabled. Either mode of operation requires that the failedobject is redundant. Effectively, the hot-device feature makes a copy of asurviving object mirror.

Selecting Hot Device Operational Mode

You can configure either of two hot device modes of operation. Mostadministrators prefer the newer hot-relocation mode.

Hot-Spare Functionality (Legacy Mode of Operation)

In early versions of VxVM, the hot-spare daemon, vxsparecheck ,detected and reacted to total disk media failures. It did this by moving allredundant objects on the failed disk drive to a pre-designated spare diskdrive in the disk group.

Hot-sparing is an older mode of operation, but can still be enabled, ifnecessary, by editing the /etc/rc2.d/S95vxvm-recover file andcommenting out the vxrelocd root & line, and then uncommenting the#vxsparecheck root & line.

Hot-Relocation Functionality

The hot-relocation daemon, vxrelocd , detects and reacts to partial diskmedia failures. It does this by copying the affected subdisk mirror to freespace on a different disk drive in the group.

Free space can be found on disk drives that have been designated as hotspares. If there are no designated hot spares, VxVM uses available freespace on any disk drive in the disk group that does not have thenohotuse flag set.

Hot relocation can also be performed for subdisks that are part of aRAID-5 volume.



Hot relocation is enabled by default and goes into effect, without systemadministrator intervention, when a failure occurs.

As shown in Figure 6-2, when a subdisk failure is detected, the contents ofthe subdisk are reconstructed on the designated hot spare. The volumecontinues to function with its original full redundancy.

Figure 6-2 Subdisk Relocation

The hot-relocation daemon, vxrelocd , detects and reacts to the followingtypes of failures:

● Disk drive failure

This is first detected as an I/O failure from a VxVM object. VxVMattempts to correct the error. If the error cannot be corrected, VxVMtries to access configuration information in the private region of thedisk drive. If it cannot access the private regions, it considers thedisk drive to have failed.

All plexes associated with the failed disk drive are eventuallydetached. The output of vxprint shows the plex in an IOFAIL state.This state is a hard failure that typically requires replacement of thedisk drive.

● Plex failure

This is detected as an uncorrectable I/O error in the plex. Formirrored volumes, the plex is detached.

Typically, this type of failure is caused by a block read or write errorthat cannot be recovered with a series of retry operations.

It is a good idea to periodically monitor for these types of errors.They might show up as correctable errors for a while and thenbecome hard errors. Sometimes, recoverable read/write errors arespurious and not seen again.

Private

PrimarySubdisk

MirrorSubdisk

Private

Volume

Private

Copy New

Hotspare

Subdisk



● RAID-5 subdisk failure

This is detected as an uncorrectable I/O error in one of theRAID-5 subdisks. The subdisk is detached. The failure is caused bythe same situation as plex failures: unrecoverable block read/writeerrors.

Hot relocation is not possible if any of the following are true:

● Subdisks do not belong to a mirrored or RAID-5 volume.

● Not enough spare disk drive space is available.

● The only available space for relocation is on a disk drive thatcontains any portion of the surviving mirror or RAID-5 volume.

● A mirrored volume has a DRL subdisk as part of its data plex:subdisks belonging to that plex cannot be relocated.

● The failure is a log plex: a new log plex is created and there is norelocation.



Evaluating Hot-Device Configurations

You can verify the hot-relocation and hot-spare status of disk drives fromthe command line. The following example shows command-lineverification:

# vxdg -g dgY nohotuseDISK DEVICE TAG OFFSET LENGTH FLAGSdgY04 c2t1d0s2 c2t1d0 0 17674902 n

# vxdg -g dgY spareDISK DEVICE TAG OFFSET LENGTH FLAGSdgY05 c2t3d0s2 c2t3d0 0 17674902 s

You can also verify and modify disk drive hot-device status in the VEAgrid area, as shown in Figure 6-3.

Figure 6-3 VEA Hot-Device Flags



Administering Hot Devices Using Command-LinePrograms

You use the vxedit command-line program to administer hot devices.

Designating Hot-Spare Disk Drives

A disk designated as a spare is used only for hot relocation. The vxassistutility will not allocate a subdisk on that disk unless forced to bycommand-line arguments. You designate disk drive as spares using thevxedit command, and you verify the spare status of the disk drives withthe vxdisk command. The following example shows the command-lineprocess:

# vxedit -g dgY set spare=on dgY05# vxdisk -g dgY listDEVICE TYPE DISK GROUP STATUSc2t1d0s2 sliced dgY04 dgY onlinec2t3d0s2 sliced dgY05 dgY online sparec2t5d0s2 sliced dgY01 dgY onlinec3t32d0s2 sliced dgY06 dgY onlinec3t33d0s2 sliced dgY02 dgY onlinec3t35d0s2 sliced dgY03 dgY online

Note – If a disk drive is marked as a hot spare, the vxassist utility doesnot create a subdisk on that disk drive unless the disk drive is specificallydesignated in command-line arguments.

Blocking Hot Relocation

If hot relocation is enabled (the default), you can use any disk drive withfree space during the relocation of a failed subdisk if there is no hot-sparespace available. If you do not want a disk drive to be used for hotrelocation, you can mark it for no hot use as follows:

# vxedit -g dgY set nohotuse=on dgY04# vxdisk -g dgY listDEVICE TYPE DISK GROUP STATUSc2t1d0s2 sliced dgY04 dgY online nohotusec2t3d0s2 sliced dgY05 dgY online sparec2t5d0s2 sliced dgY01 dgY onlinec3t32d0s2 sliced dgY06 dgY onlinec3t33d0s2 sliced dgY02 dgY onlinec3t35d0s2 sliced dgY03 dgY online



Administering Hot Devices Using the VEA GUI

Display a disk group’s disk drives in the grid area, click a disk drive withthe third mouse button, and then select Set Disk Usage from its pop-upmenu. Set or reset Spare, No hot use, Reserved, or Reserved for Allocator,as shown in Figure 6-4.

The Spare and No hot use boxes are mutually exclusive.

Figure 6-4 VEA Hot-Device Administration

The Reserve flag, if set, prevents automatic space allocation by utilities,such as the vxassist program, unless the disk drive is specified on thecommand line.

The Reserved for Allocator flag, if set, reserves a disk for exclusive use byISP utilities such as the vxpool and vxvoladm commands.



Controlling Relocation Recovery Time

You can reduce the impact of recovery on system performance byinstructing the vxrelocd daemon to increase the delay between therecovery of each region of a volume. Use the vxrelocd daemon asfollows:

# vxrelocd -o slow=500 &

The value of slow is passed on to the vxrecover command. The defaultvalue is 250 milliseconds.

Monitoring Errors

By default, the vxrelocd daemon sends email notification of errors to theserver root account. You can modify the account name in the vxrelocdroot & line in the etc/rc2.d/S95vxvm-recover file.

You can also examine system error logs for evidence of disk driveproblems, but the email notification to root is usually sufficient.

Evacuating a Disk Drive



The volume structures on a disk drive that is starting to experiencerecoverable data errors can be evacuated to a different disk drive beforethe disk drive fails entirely. Evacuation can reduce the risk of data loss.

You can also use evacuation to reduce or eliminate performancebottlenecks that have been identified.

Evacuation can only be performed on disk drives within the same group.

Identifying Evacuation Conflicts

Before you proceed with disk drive evacuation, carefully investigate theconfiguration of both the failing disk drive and the new disk drive.

You should verify that the evacuation process is not going to create any ofthe following conflicts:

● Both volume mirrors are on the same physical disk drive.

● More than one stripe column of a striped or RAID-5 volume is on thesame disk drive.

Preparing for Evacuation

Before starting the evacuation process, you must:

● Find out with what volume the failing plex or subdisk is associated,and determine the name of the disk drives that are associated with it.

● Find out what disk group is associated with the failing disk drive.

● Determine if any other volumes are associated with the failing diskdrive.

● Find a new disk drive with enough free space to perform theevacuation.

● Check for any volume conflicts associated with the new disk drive.



Identifying Suitable Evacuation Disk Drives

The following example shows how to prepare for disk drive evacuation ifyou are notified of a correctable read error on a subdisk named dgX02-01in disk group dgX.

# vxprint -g dgXTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg dgX dgX - - - -

dm dgX01 c2t16d0s2 - 17674896 - -dm dgX02 c2t18d0s2 - 17674896 - -dm dgX03 c2t1d0s2 - 17674896 - -dm dgX04 c2t3d0s2 - 17674896 - NOHOTUSEdm dgX05 c2t5d0s2 - 17674896 - -dm dgX06 c2t20d0s2 - 17674896 - SPARE

v r5demo raid5 ENABLED 61440 - ACTIVEpl r5demo-01 r5demo ENABLED 61440 - ACTIVEsd dgX03-01 r5demo-01 ENABLED 15360 0 -sd dgX04-01 r5demo-01 ENABLED 15360 0 -sd dgX05-01 r5demo-01 ENABLED 15360 0 -sd dgX01-01 r5demo-01 ENABLED 15360 0 -sd dgX02-01 r5demo-01 ENABLED 15360 0 -

v stdemo fsgen ENABLED 61440 - ACTIVEpl stdemo-01 stdemo ENABLED 61440 - ACTIVEsd dgX03-02 stdemo-01 ENABLED 12288 0 -sd dgX04-02 stdemo-01 ENABLED 12288 0 -sd dgX05-02 stdemo-01 ENABLED 12288 0 -sd dgX01-02 stdemo-01 ENABLED 12288 0 -sd dgX02-02 stdemo-01 ENABLED 12288 0 -

An analysis of the preceding vxprint output indicates the following:

● The failing subdisk, dgX02-01 , is on disk drive dgX02.

● The dgX02-02 subdisk in another volume is also on dgX02.

● Disk dgX06 appears to be unused, is marked as a spare, and is asuitable candidate for evacuating disk dgX02.

For this scenario, evacuate the disk dgX02 to disk dgX06. After theevacuation has completed, replace disk dgX02. You might mark the newdgX02 disk drive as the spare and unmark dgX06. You could also migratethe stripe data from dgX06 back to the new dgX02 disk drive.



Evacuation Using the vxevac Command

The vxevac command moves subdisks from the specified disk drive to anew disk drive. If a destination disk drive is not specified, the vxevaccommand uses any available disk drive. The following example shows atypical vxevac procedure.

# vxevac -g dgX dgX02 dgX06# vxprint -g dgXTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg dgX dgX - - - -

dm dgX01 c2t16d0s2 - 17674896 - -dm dgX02 c2t18d0s2 - 17674896 - -dm dgX03 c2t1d0s2 - 17674896 - -dm dgX04 c2t3d0s2 - 17674896 - NOHOTUSEdm dgX05 c2t5d0s2 - 17674896 - -dm dgX06 c2t20d0s2 - 17674896 - SPARE





Evacuation Using the VEA GUI

To perform an evacuation using the VEA GUI, complete the followingsteps:

1. Select the disk drive that contains the objects and data to be moved.

2. Select Evacuate Disk from the disk drive’s pop-up menu.

3. Configure the Evacuate Disk form, as shown in Figure 6-5.

Figure 6-5 VEA Evacuate Disk Form



Evacuation Using the vxdiskadm Utility

To perform an evacuation using the vxdiskadm utility, complete thefollowing steps:

1. Start the vxdiskadm utility, and select Option 7, Move volumesfrom a disk .

2. Type the media name of the disk drive to be evacuated.

3. Type the media name of the destination disk drive.

The following example shows a summary of the process.

# vxdiskadmSelect an operation to perform: 7Move volumes from a diskEnter disk name [<disk>,list,q,?] dgX06Enter disk name [<disk>,list,q,?] dgX02VxVM NOTICE V-5-2-283Requested operation is to move all volumes from disk dgX06 in group dgX.

NOTE: This operation can take a long time to complete.

Continue with operation? [y,n,q,?] (default: y) yVxVM vxevac INFO V-5-2-24 Move volume r5demo ...VxVM vxevac INFO V-5-2-24 Move volume stdemo ... VxVM INFO V-5-2-188 Evacuation of disk dgX06 is complete.

Move volumes from another disk? [y,n,q,?] (default: n) n

Moving Disk Drives Without Preserving Data



You might want to move a VxVM disk drive to a different disk groupbecause the destination disk group needs the additional disk space. Aslong as the disk drive does not contain any functional data that you needto preserve, the process is fairly easy.

Moving a Disk Drive Using the Command Line

If the disk drive you want to move contains an active volume and you donot care if the data is lost, complete the following steps to move the diskdrive to a different disk group:

1. Use the vxprint command to verify that there are no unexpectedvolumes, or portions of volumes, associated with the disk drive youwant to move.

2. Disable any applications related to the volume.

3. Unmount any file systems related to the volume.

4. Stop the volume.

You stop volumes using the vxvol command as follows:

# vxvol -g old_dg stop vol-02

5. Delete the volume configuration.

Recursively delete all objects in a volume as follows:

# vxedit -g old_dg -rf rm vol-02

6. Remove the disk drive from the disk group.

You use the vxdg command to remove a disk drive from a diskgroup as follows:

# vxdg -g old_dg rmdisk olddg-12

Even after the vxdg rmdisk operation, the disk drive is stillinitialized for VxVM use. The vxdiskunsetup command completelyremoves a disk drive from VxVM control.

7. Add the disk drive to a different disk group.

You use the vxdg command to add the disk drive to a different diskgroup as follows:

# vxdg -g new_dg adddisk newdg-02=c1t3d0



Moving a Disk Drive Using the VEA GUI

Moving a disk drive to a new disk group is easy using the VEA GUI. Youuse the following process to reduce errors:

1. In the grid area, click the third mouse button on the disk group andselect Disk/Volume Map from the pop-up menu.

2. Examine the Volume to Disk Mapping display, shown in Figure 6-6,and verify that there are no unexpected volume structures associatedwith the disk drive you want to move.

Figure 6-6 Analyzing Volume to Disk Relationships

3. Use the pop-up menus available on the volume name and disk drivename to perform any of the following actions:

● Stop any volumes on the disk drive that you want to remove.

● Delete the volume or volumes.

● Remove the disk from the disk group.

The disk drive is returned to the free disk pool and can now beadded to a different disk group.

Moving Populated Disk Drives to a New Disk Group



Moving populated VxVM disk drives to a new or different disk group is atechnique you might use occasionally. One reason to use this technique isif you have mistakenly created all of your volumes in a single disk group.To resolve a performance problem, you would move some of the volumesand their related disk drives to a different disk group.

Caution – Do not perform this operation on a production system withoutfirst backing up the data on all associated volumes. If this process fails,there is no way to recover data without backup tapes.

In this section, a volume called mirvol is moved from a disk groupnamed old_dg to a new disk group named new_dg.

The process of moving populated disks is easier if you have VERITAS FastResync. Currently, Sun does notsell, license, or support this option. VERITAS FastResync enables the vxdg move/split/join options.

Evaluating Disk Drive Involvement

Before you take any action, you must determine which physical diskdrives are part of your target volume. You must also ensure that the diskdrives are not being used by other volumes.

Determining Disk Drive Involvement From the Command Line

Use the vxprint command as follows to determine the volume to diskdrive mapping:

# vxprint -g old_dgTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg old_dg old_dg - - - -

dm olddg-01 c2t16d0s2 - 17674902 - -dm olddg-02 c3t37d0s2 - 17674902 - -dm olddg-03 c3t52d0s2 - 17674902 - -

v mirvol fsgen ENABLED 204800 - ACTIVEpl mirvol-01 mirvol ENABLED 208278 - ACTIVEsd olddg-01-01 mirvol-01 ENABLED 208278 0 -pl mirvol-02 mirvol ENABLED 208278 - ACTIVEsd olddg-02-01 mirvol-02 ENABLED 208278 0 -



In the preceding example, the volume mirvol contains two plexes with asingle subdisk associated with each plex. The volume is associated withtwo disk drives, olddg-01 and olddg-02 .

Note – The VEA GUI’s Disk/Volume Map display can be very helpfulwhen you are trying to determine volume involvement with specific diskdrives.

Saving the Configuration

Use the vxprint command to save the volume configuration of themirvol volume in a file named save_mirvol .

# vxprint -hmQqrL -g old_dg mirvol > \/var/tmp/save_mirvol

Caution – If you are saving layered volumes that have sub-volumes (suchas striped mirror structures), you must add the r and L options to thevxprint command. If you fail to do this, the saved configurationinformation is incomplete.

The vxprint -m option provides detailed configuration information in aformat that can be used later by the vxmake utility. Table 6-1 shows thevxprint command options used in this example.

Table 6-1 The vxprint Command Options

Command Function

-h Lists complete hierarchies.

-m Displays information in a format that can be used asinput to the vxmake utility.

-Q Suppresses the disk group header that separates eachdisk group.

-q Suppresses headers (in addition to disk group header).

-r Displays related records of a volume containing sub-volumes.

-L Affects record grouping when used with the -r option.

-g Specifies the disk group.



Moving the Disk Drives to a New Disk Group

Moving the disk drives to a new disk group requires several steps thatyou have seen earlier in this course. They are:

1. Unmount appropriate file systems and stop any processes that areaccessing the mirvol volume directly.

# umount /Data

2. Stop the volume.

# vxvol -g old_dg stop mirvol

3. Remove the definitions of the structures (volume, plexes, andsubdisks) from the configuration database.

# vxedit -g old_dg -rf rm mirvol

The vxedit command removes the definitions of the volume,plexes, and subdisks from the configuration database for the old diskgroup, old_dg .

Note – Removing the definitions does not affect the data: it only removesselected records from the configuration database. The -r optionrecursively removes the volume and all associated plexes and subdisks.

4. Remove the disk drives from the original disk group.

# vxdg -g old_dg rmdisk olddg-01 olddg-02

5. If the new disk group, new_dg , does not exist, initialize it using oneof the disk drives to be moved (disk old_dg-01 , in this example).

# vxdg init new_dg olddg- 01=c2t16d0

Caution – It is critical that all of the disk drives retain their original medianames when they are added to the new disk group.

6. Add the remaining disk drives to the new disk group.

# vxdg -g new_dg adddisk olddg-02=c3t37d0

7. Verify that the disk drives have been added to the new disk group.

# vxdisk list | grep new_dgc2t16d0s2 sliced olddg-01 new_dg onlinec3t37d0s2 sliced olddg-02 new_dg onlinec3t52d0s2 sliced newdg-01 new_dg online



Reloading the Volume Configuration

To reload the volume configuration, complete the following steps:

1. Use the vxmake command to reload the saved configuration for thevolume vol01 .

# vxmake -g dgY -d /var/tmp/save_mirvol

Earlier the volume configuration was saved in the save_mirvolfile. The -d option specifies the description file to use for buildingsubdisks, plexes, and volumes.

Caution – If the disk drives do not have their original media names, theconfiguration reload fails.

2. Check the volume status. The volume and plexes should be in aDISABLEDstate.

# vxprint -g new_dg mirvolTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv mirvol fsgen DISABLED 204800 - EMPTYpl mirvol-01 mirvol DISABLED 208278 - EMPTYsd olddg-01-01 mirvol-01 ENABLED 208278 0 -pl mirvol-02 mirvol DISABLED 208278 - EMPTYsd olddg-02-01 mirvol-02 ENABLED 208278 0 -

3. Use the vxvol command to bring the volumes back online.

# vxvol -g new_dg init active mirvol

Note – An alternative to this procedure is to create a new volume inanother disk group, and either dump a backup tape onto it or perform adirect copy from the old volume.

4. Edit the /etc/vfstab mount information for the volume’s filesystem and change the disk group in the logical paths.

# vi /etc/vfstab/dev/vx/dsk/new_dg/mirvol /dev/vx/rdsk/new_dg/mirvol /Data ufs 1 yes -

5. Mount and test the file system.

# mount /Data# ls /Datalost+found

Backing Up and Restoring Disk Group Configurations



The VxVM software automatically saves disk group configurationbackups. You can also manually backup disk group configurations.

Automatic Configuration Backup

VxVM keeps updated copies of the configuration status both in memoryand written on one or more storage disk drives. This status is updatedany time there is a change in the VxVM configuration or availability dueto errors or administrative changes. A backup copy of the configuration isautomatically updated on the system boot disk when administrativechanges are made to the configuration.

The primary elements involved are:

● The vxio software driver

● The vxconfigd configuration daemon

● The vxconfigbackupd daemon

Figure 6-7 shows how the kernel configuration table is checked by thevxio driver before the driver attempts to access a virtual structure. Thedisk-resident copies do not need to be examined.

Figure 6-7 Configuration Monitoring Components

KernelConfiguration

Table

ConsultBeforeAccess

DeviceAccessError

vxio

vxconfigd

Storage Array

Update

Update

Data

Driver

configdb

AdministrativeModifications

vxconfigbackupd

/etc/vx/cbr/bk



Initial Volume Configuration

When VxVM starts, the vxconfigd daemon imports disk groups thatbelong to the VxVM server.

When disk groups are imported, the kernel configuration table is createdby the vxconfigd daemon, which reads the disk-resident configdbrecords.

Status Configuration Changes

When the vxio driver is notified of a hard-device error, it disables thevolume it was trying to access and updates the VxVM kernelconfiguration table. The vxio driver also signals the vxconfigd daemonthat a configuration change has taken place.

Administrative Configuration Changes

System administrator changes to the configuration, such as creating a newdisk group, are automatically recorded on the system boot disk by thevxconfigbackupd daemon. A backup directory for each disk group ismaintained in the /etc/vx/cbr/bk directory. The following exampleshows backup files on a system with two disk groups named pridg andsecdg .

# cd /etc/vx/cbr/bk# lspridg.1064606899.21.ns-east-115secdg.1064865853.29.ns-east-115## cd pridg.1064606899.21.ns-east-115# ls1064606899.21.ns-east-115.binconfig1064606899.21.ns-east-115.cfgrec1064606899.21.ns-east-115.dginfo1064606899.21.ns-east-115.diskinfo

Note – If all array-resident configuration copies are lost, thevxconfigrestore command can be used to rebuild the originalstructures. The data on the structures is not preserved.



Manual Configuration Backup and Restore

The vxconfigbackupd daemons are started by theetc/rc2.d/S95vxvm-recover script and record any VxVM disk groupconfiguration changes in the /etc/vx/cbr/bk directory.

You use the vxconfigbackup command to backup the current disk groupconfiguration information. You restore a disk group configuration usingthe vxconfigrestore command.

If there is damage to the disk group configuration records that are storedin the private regions of one or more disk drives in a disk group, the diskgroup import operation might fail. The vxconfigrestore command isused to automatically correct the damaged configuration records or torecreate a disk group from the beginning.

When restoring or repairing damaged disk group records, you must meetthe following criteria:

● Failed disk drives must be replaced prior to using thevxconfigrestore command.

● Replacement disk drives must be initialized for VxVM use prior tousing the vxconfigrestore command.

● All disk drives must have the same physical configuration andlogical addresses as when the configuration backup was performed.

Caution – The vxconfigbackup command does not preserve data. Filesystem and raw volume data must be preserved separately using backupcommands or applications. Volumes with redundant data, such asmirrored or RAID-5 structure, can recover from the loss of a mirror or aRAID-5 stripe. Non-redundant volume types must be reinitialized for filesystem use and the data recovered from backup media.



Using the vxconfigbackup Command

The vxconfigbackup command is used to backup one or more diskgroups. If the name of a disk group is not specified with the command, alldisk groups are backed up.

You use the -l option to specify a backup file location other than thedefault location in /etc/vx/cbr/bk . The following example shows theuse of the vxconfigbackup command.

# vxdg listNAME STATE IDrootdg enabled 1066871088.21.ns-east-104dgX enabled 1066748899.279.ns-east-104

# vxconfigbackup -l /etc/vx/cbr/mybackup dgXStart backing up diskgroup dgX to/etc/vx/cbr/mybackup/dgX.1066748899.279.ns-east-104 ...VxVM NOTICE V-5-2-3100 Backup complete for diskgroup: dgX

# ls /etc/vx/cbr/mybackupdgX.1066748899.279.ns-east-104

# ls /etc/vx/cbr/mybackup/dgX.1066748899.279.ns-east-1041066748899.279.ns-east-104.binconfig1066748899.279.ns-east-104.cfgrec1066748899.279.ns-east-104.dginfo1066748899.279.ns-east-104.diskinfo



Using the vxconfigrestore Command

When you use the vxconfigrestore command to restore a disk groupand its associated structures, the process is performed in two stages:precommit and commit.

# mount |grep R5demo/R5demo on /dev/vx/dsk/dgX/r5demo

# ls -l /R5demototal 28736-rw------T 1 root other 1048576 Oct 27 11:37 file1-rw------T 1 root other 3145728 Oct 27 11:37 file2-rw------T 1 root other 10485760 Oct 27 19:40 file3drwx------ 2 root root 8192 Oct 27 11:35 lost+found

# umount /R5demo# vxvol -g dgX stop r5demo# vxvol -g dgX stop stdemo# vxdg deport dgX

# vxdisksetup c2t3d0 (reinitialize the disk)

# vxdg import dgXVxVM vxdg ERROR V-5-1-587 Disk group dgX: import failed: Disk for diskgroup not found

# vxconfigrestore -p -l /etc/vx/cbr/mybackup dgX (precommit stage)Installing volume manager disk header for c2t18d0s2 ...Installing volume manager disk header for c2t1d0s2 ...Installing volume manager disk header for c2t20d0s2 ...Installing volume manager disk header for c2t16d0s2 ...Installing volume manager disk header for c2t3d0s2 ...Installing volume manager disk header for c2t5d0s2 ...-dgX’s diskgroup configuration is restoredDiskgroup can be accessed in read only and can be examined usingvxprint in this state.

# vxconfigrestore -l /etc/vx/cbr/mybackup -c dgX (commit stage)Committing configuration restoration for diskgroup dgX ....dgX’s diskgroup configuration restoration is committed.

Importing Disk Groups After a System Crash



Depending on the state of a disk group, there are several variations of theimport operation that might be useful. The basic import operation is notcomplicated and can be done using either the vxdg command, thevxdiskadm utility, or the VEA GUI.

If a disk group must be imported after a system crash, the process can bemore difficult. Following are some of the possible variations.

● Performing a typical import of a clean disk group:

# vxdg import disk_group_name

● Importing a disk group to another system after a crash:

# vxdg -C import disk_group_name

The -C option is necessary to clear the old host IDs that were left onthe disk drives after the crash.

# vxdg -fC import disk_group_name

Caution – The -f option forces an import in the event that all the diskdrives are not usable. This option can be dangerous on dual-hostedstorage arrays because the disk group might also be imported to anotherhost. A disk group that is imported to two host systems can becomecorrupted.

# vxrecover -g disk_group_name -sb

You execute this command after a crash to start the volumes andperform a recovery process. This is done automatically during areboot.

● Importing a disk group with a duplicate name:

# vxdg -t -n new_disk_group import disk_group_name

The -t option makes the new disk group name temporary.

Note – When a disk group is imported, the volumes can be in a disabledstate. You can use the vxvol start command to start the volumes.



Importing the rootdg Disk Group After a Crash

After a crash it might be necessary to import the rootdg disk group toanother system to perform repair operations. This process is a little morecomplicated because you cannot have two rootdg disk groups on asystem. This procedure works only if the boot disk can be easily attachedto a different system.

Importing a second rootdg disk group requires the use of multipleoptions to:

● Assign a new temporary disk group name to rootdg .

● Clear the original hostid ownership.

● Use the unique rootdg group identifier:

# vxdg -tC -n new_disk_group import group_id

The difficult part is that you must know the unique rootdg groupidentifier. This value must be known in advance. You can determine therootdg group identifier with the vxdisk command as follows:

# vxdisk -s listDisk: c0t2d0s2type: slicedflags: online ready private autoconfig autoimportimporteddiskid: 791000525.1055.boulderdgname: rootdgdgid: 791000499.1025.boulderhostid: boulder

The disk group is renamed by the importing host.

Note – The vxdisk -s list command lists information for all attacheddisk drives. Disk drives that belong to a cleanly deported disk group havea blank hostid entry.

Volume Snapshot Operations



When you need to back up the data on a volume, such as a file systemvolume, you can use the VxVM snapshot function to create a copy of thevolume. You can then back up the new copy to tape without disruptingservice to the original volume.

Snapshot Process

You must satisfy the following prerequisites before the snapshot processcan be started:

● You must know the name of the volume to be backed up.

● You must provide a name for the new temporary snapshot volume.

● You can specify a specific disk drive to use for the snapshot copy.

● You must have sufficient unused disk space for the snapshot.

The snapshot process is a two-step process. First, a snapshot mirror of avolume is created. If no disk drives are specified for use by the snapshotoperation, VxVM finds free disk space anywhere it can. The resultingsnapshot mirror can be poorly constructed. Because the snapshot mirror isonly a temporary structure, this might not be a problem. However, itcould temporarily interfere with the performance of other volumes.

When the snapshot mirror is fully synchronized, its state changes toSNAPDONE. The snapshot mirror continues to maintain synchronizationwith the parent volume until it is detached.

The final operation detaches the temporary snapshot mirror and attachesit to a regular volume with a name of your choosing.

You can then perform standard backup procedures on the snapshotvolume.



Using the VEA GUI to Create a Snapshot

You can use the snapshot feature from the VEA GUI to create a snapshot.Complete the following steps:


a. Click on the volume with the third mouse button.

b. Select Snap, and then Snap Start from its pop-up menu.

2. Complete the Snap Start Volume form, as shown in Figure 6-8.

It is usually safe to let VxVM decide which disks to use for theSnapshot mirror.

The process of creating and synchronizing a new mirror with theexisting volume data can take some time depending on the volumesize.

Figure 6-8 VEA Snap Start Volume Form




a. Click the volume name with the third mouse button.

b. Select Snap, and then Snap Shot from its pop-up menu.

c. Complete the Snap Shot Volume form as shown in Figure 6-9.

The new mirror is detached and a separate volume is createdfrom it based on a default naming scheme. You can change thedefault snapshot name if needed.

Figure 6-9 VEA Snap Shot Volume Form



4. Use the vxprint command to verify the status of the new snapshotvolume.

In the following case, the snapshot mirror/volume is namedSNAP-stdemo . The parent volume is named stdemo .

# vxprint -g dgXTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg dgX dgX - - - -

dm dgX01 c2t16d0s2 - 17674896 - -dm dgX02 c2t18d0s2 - 17674896 - -dm dgX03 c2t1d0s2 - 17674896 - -dm dgX04 c2t3d0s2 - 17674896 - NOHOTUSEdm dgX05 c2t5d0s2 - 17674896 - -dm dgX06 c2t20d0s2 - 17674896 - -dm dgX07 c3t32d0s2 - 17674896 - -dm dgX08 c3t33d0s2 - 17674896 - -dm dgX09 c3t35d0s2 - 17674896 - -dm dgX10 c3t37d0s2 - 17674896 - -dm dgX11 c3t50d0s2 - 17674896 - -dm dgX12 c3t52d0s2 - 17674896 - -

v SNAP-stdemo fsgen ENABLED 61440 - ACTIVEpl stdemo-02 SNAP-stdemo ENABLED 61440 - ACTIVEsd dgX06-01 stdemo-02 ENABLED 12288 0 -sd dgX07-01 stdemo-02 ENABLED 12288 0 -sd dgX08-01 stdemo-02 ENABLED 12288 0 -sd dgX09-01 stdemo-02 ENABLED 12288 0 -sd dgX10-01 stdemo-02 ENABLED 12288 0 -





5. Create a temporary mount point, and complete the following steps:

a. Mount the new snapshot volume.

b. Back it up to tape.

# mkdir /Temp# mount /dev/vx/dsk/dgX/SNAP-stdemo /Temp# tar tvf /dev/rmt/0 /Temp

6. Unmount and delete the snapshot volume.

# umount /Temp# vxedit -g dgX -rf rm SNAP-stdemo

The VEA GUI volume snapshot menu has additional functions, SnapBack and Snap Clear.

As shown in Figure 6-10, the Snap Back function rejoins the snapshotmirror to the original volume. It also resynchronizes the data ineither direction between the original volume and the snapshotmirror.

Figure 6-10 VEA Snap Back Volume Form

The Snap Clear function disassociates the detached snapshot mirrorfrom the original volume. The Snap Back feature cannot be usedafter the Snap Clear operation has completed.



Using the Command Line to Create a Snapshot

Using the command line to make snapshots is useful for performingbackups from script files. The following example shows thecommand-line snapshot process.

1. Create a snapshot of the stdemo volume and verify it has completed.

# vxassist -g dgX -b snapstart stdemo# vxprint -g dgX stdemoTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv stdemo fsgen ENABLED 61440 - ACTIVEpl stdemo-01 stdemo ENABLED 61440 - ACTIVEsd dgX03-02 stdemo-01 ENABLED 12288 0 -sd dgX04-02 stdemo-01 ENABLED 12288 0 -sd dgX05-02 stdemo-01 ENABLED 12288 0 -sd dgX01-02 stdemo-01 ENABLED 12288 0 -sd dgX02-02 stdemo-01 ENABLED 12288 0 -pl stdemo-02 stdemo ENABLED 61440 - SNAPDONEsd dgX06-01 stdemo-02 ENABLED 12288 0 -sd dgX07-01 stdemo-02 ENABLED 12288 0 -sd dgX08-01 stdemo-02 ENABLED 12288 0 -sd dgX09-01 stdemo-02 ENABLED 12288 0 -sd dgX10-01 stdemo-02 ENABLED 12288 0 -

2. Use the vxassist snapshot option to detach the temporarysnapshot mirror (plex) and associate it with a new volume namedSNAP-stdemo .

# vxassist -g dgX snapshot stdemo-02 SNAP-stdemo

3. Process the volume with the fsck utility to check its integrity, andcomplete the following steps:

a. Mount the volume.

b. Perform a backup of the snapshot volume.

c. Unmount and delete the volume.

# fsck -y /dev/vx/rdsk/dgX/SNAP-stdemo

# mkdir /Temp

# mount /dev/vx/dsk/dgX/SNAP-stdemo /Temp

# tar cvf /dev/rmt/0 /Temp

# umount /Temp

# vxedit -g dgX -rf rm SNAP-stdemo

Online Volume Relayout



Online volume relayout is a tool that you can use to correct configurationmistakes or to enhance the performance or reliability of a volume.

An important feature of online volume relayout is that a volume and itsfile system can remain available during the relayout operation.

The relayout feature can be used to perform many operations, such as:

● Adding more stripe columns to a RAID-5 volume

● Changing the stripe unit size of a volume

● Changing the type of volume from RAID 5 to mirrored orconcatenated

Note – Read the “Changing a VxVM Volume Layout” section of theVERITAS Enterprise Administrator User’s Guide carefully before attemptinga relayout of a production volume.

Volume Relayout Prerequisites

You must research the following information before starting an onlinevolume relayout process:

● Identify the new volume layout.

This includes concatenated, striped, RAID 5, concatenated mirror,striped mirror, additional columns, and new stripe widths.

● Research additional permanent disk space that might be needed bythe new volume layout.

● Research the temporary disk space that might be needed during thevolume layout.

Note – You can also use the vxassist relayout command to accomplishonline volume relayout. Unless you explicity allocate storage, thevxassist command automatically determines where to get thepermanent and temporary disk space that might be needed.



Volume Relayout Using the Command Line

The following examples show the basic process to add an extra column toan existing RAID-5 volume name r5demo . There is enough additional diskspace on the current disk drives being used by the RAID-5 volume for anytemporary space needs. For performance reasons, you should specifywhich disk drive to use for the new column.

# vxprint -g dgX r5demo stdemoTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv r5demo raid5 ENABLED 61440 - ACTIVEpl r5demo-01 r5demo ENABLED 61440 - ACTIVEsd dgX03-01 r5demo-01 ENABLED 15360 0 -sd dgX04-01 r5demo-01 ENABLED 15360 0 -sd dgX05-01 r5demo-01 ENABLED 15360 0 -sd dgX01-01 r5demo-01 ENABLED 15360 0 -sd dgX02-01 r5demo-01 ENABLED 15360 0 -


# vxdg -g dgX freeDISK DEVICE TAG OFFSET LENGTH FLAGSdgX01 c2t16d0s2 c2t16d0 27648 17647248 -dgX02 c2t18d0s2 c2t18d0 27648 17647248 -dgX03 c2t1d0s2 c2t1d0 27648 17647248 -dgX04 c2t3d0s2 c2t3d0 27648 17647248 ndgX05 c2t5d0s2 c2t5d0 27648 17647248 -dgX06 c2t20d0s2 c2t20d0 0 17674896 -

The RAID 5 and RAID 0 stripes are using a total of 27,648 blocks on eachdisk drive or about 14 Mbytes. All disk drives in the disk group havemore than enough space left for use during the relayout operations.

The following example adds an additional column to the RAID-5 volumeand specifies that the new column is to be on the disk dgX06.

# vxassist -g dgX relayout r5demo layout=raid5 alloc="dgX06” ncol=6



Completed RAID 5 Volume Relayout

After the completion of the previous volume relayout example, theRAID-5 volume now has the following configuration.

# vxprint -g dgX r5demoTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv r5demo raid5 ENABLED 61440 - ACTIVEpl r5demo-Dp02 r5demo ENABLED 1920 - LOGsd dgX06-03 r5demo-Dp02 ENABLED 1920 0 -pl r5demo-01 r5demo ENABLED 61440 - ACTIVEsd dgX03-04 r5demo-01 ENABLED 12288 0 -sd dgX04-03 r5demo-01 ENABLED 12288 0 -sd dgX05-03 r5demo-01 ENABLED 12288 0 -sd dgX01-03 r5demo-01 ENABLED 12288 0 -sd dgX02-03 r5demo-01 ENABLED 12288 0 -sd dgX06-02 r5demo-01 ENABLED 12288 0 -

By default, the vxassist layout=raid5 specification adds a log to theRAID-5 volume. Due to performance degradation, RAID-5 volume logsshould not reside on the same disk drives as the RAID-5 data.

To prevent the unintentional addition of a RAID-5 log, the vxassistlayout specification must be more specific.

# vxassist -g dgX relayout r5demo layout=raid5,nolog alloc="dgX06” ncol=6



Volume Relayout Using the VEA GUI

To perform the same process as in the previous section, complete thefollowing steps:

1. Display the RAID-5 volume in the grid area, and select ChangeLayout from its pop-up menu.

2. Complete the Change Volume Layout form, as shown in Figure 6-11.

Figure 6-11 VEA Change Volume Layout Form

In the preceding example, the volume Layout was set to RAID 5 andthe Number of Columns was increased to 6. Use the Show Optionsbutton to enter additional relayout criteria, such as which disksdrives to use.



Relayout Status Monitor

After you fill out the Change Volume Layout form and start the relayoutprocess, a Relayout Status window, shown in Figure 6-12, appears. Youuse the controls in the Relayout Status window to:

● Temporarily stop the relayout process (pause)

● Abort the relayout process

● Continue the process after a pause

● Undo the relayout changes (reverse)

The VEA Relayout Status Monitor window also displays the percentagecomplete status.

Figure 6-12 VEA Relayout Status Monitor Window

Note – The relayout task could fail if the target volume was not originallycreated using the VEA GUI or the vxassist command.

Creating Layered Volumes



A layered volume is built on one or more other volumes. The underlyingvolumes are mirrored. Layered volume are RAID 1+0 format and arecalled striped mirror or concatenated mirror configurations.

Layered Volume Disk Requirements

By default, the mirrors in layered volumes are created with DRLs. For bestperformance and reliability, all components of a layered volume should beon different disk drives.

A striped mirror volume, shown in Figure 6-13, requires a minimum offour disk drives to implement. Placing the DRLs on separate disk drivesincreases the disk requirement to six.

Figure 6-13 Striped Mirror Disk Requirements

The disk requirements for a concatenated mirror volume structure, shownin Figure 6-14, are the same as for striped mirror structures.

Figure 6-14 Concatenated Mirror Disk Requirements

MirrorMirrorLog Log

StripeStripe

Volume

MirrorMirror

Log

Log

Volume

Subdisk

Subdisk

MirrorMirror

MirrorMirror



Evaluating Available Disk Space

There are several methods available for evaluating available disk space.The vxdg free command is a good tool for performing a basicevaluation. In the following example, each of the disk drives in the dgXdisk group have approximately 8 Gbytes of available space. To calculatethe available free space in Mbytes on a given disk drive, divide its LENGTHvalue by 2048 (17674896/2048 = 8630.32 Mbytes = 8.63 Gbytes).

# vxdg -g dgX freeDISK DEVICE TAG OFFSET LENGTH FLAGSdgX01 c2t16d0s2 c2t16d0 0 17674896 -dgX02 c2t18d0s2 c2t18d0 0 17674896 -dgX03 c2t1d0s2 c2t1d0 0 17674896 -dgX04 c2t3d0s2 c2t3d0 0 17674896 ndgX05 c2t5d0s2 c2t5d0 0 17674896 -dgX06 c2t20d0s2 c2t20d0 0 17674896 -

You can also use the vxassist maxsize option as follows to calculatemaximum available space for a specific structure. The example defaults totwo mirrors and logging.

# vxassist -g dgX maxsize \layout=stripe-mirror ncolumn=2 \dgX01 dgX02 dgX03 dgX04 dgX05 dgX06

Maximum volume size: 35348480 (17260Mb)

Note – If you use the VEA GUI New Volume wizard to configure layeredvolumes, you can use the Max Size button to estimate maximum availablespace.



Creating Layered Volumes From the Command Line

If you create layered volumes using the vxassist command, you shouldspecify the layout, number of columns, and whether logging is to beconfigured. The default is to not create DRLs.

An example of using the vxassist command to create a layered volumefollows.

# vxassist -g dgX make strmirr 500m \layout=stripe-mirror ncolumn=2 \dgX01 dgX02 dgX03 dgX04 dgX05 dgX06

Use the vxprint command as follows to display the layered volumestructure. Output not related to the volume is omitted for clarity.

# vxprint -g dgX......v strmirr fsgen ENABLED 1024000 - ACTIVEpl strmirr-03 strmirr ENABLED 1024000 - ACTIVEsv strmirr-S01 strmirr-03 ENABLED 512000 0 -sv strmirr-S02 strmirr-03 ENABLED 512000 0 -

v strmirr-L01 fsgen ENABLED 512000 - ACTIVEpl strmirr-P01 strmirr-L01 ENABLED 512000 - ACTIVEsd dgX03-02 strmirr-P01 ENABLED 512000 0 -pl strmirr-P02 strmirr-L01 ENABLED 512000 - ACTIVEsd dgX05-02 strmirr-P02 ENABLED 512000 0 -

v strmirr-L02 fsgen ENABLED 512000 - ACTIVEpl strmirr-P03 strmirr-L02 ENABLED 512000 - ACTIVEsd dgX04-02 strmirr-P03 ENABLED 512000 0 -pl strmirr-P04 strmirr-L02 ENABLED 512000 - ACTIVEsd dgX01-02 strmirr-P04 ENABLED 512000 0 -

Note – Layered volume structures are difficult to understand in thevxprint output format. Use the VEA GUI Layout View function to see avisual representation of layered volume structures.



Creating Layered Volumes Using the VEA GUI

Use the VEA GUI New Volume wizard, show in Figure 6-15, to configurelayered volumes. Select either Concatenated Mirrored or Striped Mirroredin the Layout section.

By default, the New Volume wizard configures two columns, two mirrorsfor each column, and enables logging (DRLs).

Figure 6-15 Creating Layered Volumes Using the VEA GUI



Identifying Layered Volume Subcomponents

The striped mirror volume components, shown in Figure 6-16, consist ofseveral layers. The lower levels of the layered volumes are ready-madeconfigurations designed to provide the highest level of availabilitywithout increasing the administrative complexity.

Figure 6-16 Striped Mirror Volume Structure

The concatenated mirror volume components shown in Figure 6-17 alsoconsist of several layers.

Figure 6-17 Concatenated Mirror Volume Structure

Striped Plexvol01-03

10 GBCol/Width:2x64

Volumevol0110 GB

Subdiskvol01-S01Column: 0

Sub-Volumevol01-L01

5 GB

Sub-Plexvol01-P01

5 GB

Subdiskdisk01-01

5 GB

Subdiskdisk02-01

5 GB

Subdiskdisk03-01

5 GB

Subdiskdisk04-01

5 GB

Sub-Plexvol01-P02

5 GB

Sub-Plexvol01-P03

5 GB

Sub-Plexvol01-P04

5 GB

Sub-Volumevol01-L02

5 GB

Subdiskvol01-S02Column: 1

Volumevol01

10.5 GB

Concat Plexvol01-03

10.5 GB

Subdiskvol01-S01

1.6 GB

Sub-Volumevol01-L01

1.6 GB

Sub-Plexvol01-P01

1.6 GB

Subdiskdisk01-01

1.6 GB

Subdiskdisk02-01

1.6 GB

Subdiskdisk03-01

8.9 GB

Subdiskdisk04-01

8.9 GB

Sub-Plexvol01-P02

1.6 GB

Sub-Plexvol01-P03

8.9 GB

Sub-Plexvol01-P04

8.9 GB

Sub-Volumevol01-L02

8.9 GB

Subdiskvol01-S02

8.9 GB

Basic Intelligent Storage Provisioning Administration



The VxVM Intelligent Storage Provision (ISP) feature is specificallytargeted at managing large storage installations composed of a hugenumber of hardware RAID LUNs or SAN devices.

Primary ISP Components

Traditional VxVM volume creation, using software such as the vxassistcommand or the VEA GUI, relies on the administrator. The administratormust analyze storage resources and configure volumes in a manner thatsatisfies the required performance or reliability needs.

With the advent of hardware RAID LUN technology, such as the SunStorEdge 3510/6020/9910 models and related SAN technology, a systemadministrator might be faced with analyzing thousands of devices whoseunderlying characteristics and capabilities are hidden and unknown.

The ISP system provides a storage allocation engine that automaticallychooses which designated storage to use based on the capabilities youspecify when creating new ISP volumes (called application volumes).

The components, commands, volumes, and storage designated for ISP usecannot be used by traditional VxVM commands such as vxassist ,vxdiskadm , and vxvol .

Basic ISP components include the following:

● Data pools (only for data volumes)

● Clone pools (only for snapshots)

● Application volumes

● Pre-defined templates

All ISP operations are performed using the vxvoladm command, thevxpool command, or the VEA GUI. There are no other commands ortools used to create ISP pools or application volumes.



ISP Storage Pools

You configure both data pools and clone pools within a disk group.

A data storage pool is created within a standard disk group. One or moreLUNs from a disk group are assigned to a named storage pool. Anysubsequent storage pools created in the same disk group areautomatically defined as clone pools.

Clone pools are used only to hold full-sized instant snapshots of data poolvolumes in the same disk group. If the instant snapshot feature is notlicensed on your system, clone pools have no use.

ISP Application Volumes

Application volumes reside only in an ISP data storage pool. You createapplication volumes using either the vxvoladm command or VEA GUI.

Predefined ISP Templates

You associate ISP templates with storage pools so that volumes created ina storage pool are restrained by a fixed set of configuration rules. Thehierarchy of ISP templates is shown in Figure 6-18.

Figure 6-18 ISP Template Hierarchy

Storage Pool Template Sets (6)

Storage Pool Templates (11)

Volume Templates (21)

Capabilities (25)

Variables (8)

Policies

autogrowselfsufficient

2 Each

5-15 Each

1-2 Each

0-2 Each



Using Storage Pool Set Templates

Storage pool set templates describe both data pool and clone poolcharacteristics within a disk group. There are six pre-defined storage poolsets. By default, the first pool created in a disk group is a data pool and allsubsequent pools created in a disk group are clone pools.

Each storage pool set provides two storage pool definitions. For example,the storage pool set, mirrored_data_striped_clones , provides themirrored_volumes storage pool definition for the data pool and thestriped_volumes storage pool definition for the clone pool.

Creating Storage Pool Sets Using the VEA GUI

As shown in Figure 6-19, the VEA GUI Organize Disk Group Wizardorganizes existing disk groups using one of the pre-defined storage pool settemplates. Default data pool and clone pool names can be modified. Youassign disk group disks to the pools afterwards.

Figure 6-19 Selecting Storage Pool Sets



Creating Storage Pool Sets Using the vxpool Command

The vxpool command associates a storage pool set template with anexisting disk group. In the following example, the storage pool settemplate mirrored_data_striped_clones is associated with the dgSPdisk group.

# vxpool listpoolsetsmirrored_data_striped_clonesmirrored_prefab_raid5_data_mirrored_clonesmirrored_prefab_stripe_data_striped_clonesprefab_mirrored_data_prefab_striped_clonesstripe_mirrored_data_striped_clonesstriped_prefab_mirrored_data_striped_clones

# vxpool -g dgSP organize mirrored_data_striped_clones

By default, the data and clone pool names are a variation of the pooltemplates, mirrored_volumes and striped_volumes .

After initial pool creation, you use the vxpool command to associate diskgroup media names with each pool. You can also modify the defaultstorage pool names. An example follows.

# vxpool -g dgSP adddisk mirrored_volumes1 \dm=dgSP01,dgSP03,dgSP04,dgSP02,dgSP07,dgSP06

# vxpool -g dgSP adddisk striped_volumes1 \dm=dgSP08,dgSP12,dgSP13,dgSP09,dgSP10,dgSP11

# vxpool -g newDG2 rename mirrored_volumes1 dp_01# vxpool -g newDG2 rename striped_volumes1 cp_01



Using Storage Pool Templates

When first created, data pools or clone pools are associated with a singlestorage pool template such as mirrored_volumes or raid5_volumes . Eachstorage pool template references a list of predefined volume templates. Youcan create storage pools without assigning a template, and later use thevxpool command to associate a particular template.

Each storage pool template references 5-15 volume templates.

The following example shows the use of the vxpool command to create astorage pool, assign disks to it, and associate a template with the pool.The first pool created in a disk group is automatically a data pool.

# vxdisk -g dgX listDEVICE TYPE DISK GROUP STATUSc2t1d0s2 auto:cdsdisk dgX03 dgX onlinec2t3d0s2 auto:cdsdisk dgX04 dgX onlinec2t5d0s2 auto:cdsdisk dgX05 dgX onlinec2t16d0s2 auto:cdsdisk dgX01 dgX onlinec2t18d0s2 auto:cdsdisk dgX02 dgX onlinec3t32d0s2 auto:cdsdisk dgX06 dgX onlinec3t33d0s2 auto:cdsdisk dgX07 dgX onlinec3t35d0s2 auto:cdsdisk dgX08 dgX onlinec3t37d0s2 auto:cdsdisk dgX09 dgX onlinec3t50d0s2 auto:cdsdisk dgX10 dgX online

# vxpool listpooldefinitionsany_volume_typemirror_stripe_volumesmirrored_prefab_raid5_volumesmirrored_prefab_striped_volumesmirrored_volumesprefab_mirrored_volumesprefab_raid5_volumesprefab_striped_volumesraid5_volumesstripe_mirror_volumesstriped_prefab_mirrored_volumesstriped_volumes

# vxpool -g dgX create r5pool \dm=dgX01,dgX02,dgX03,dgX06,dgX07,dgX08 \pooldefinition=raid5_volumes



# vxpool -g dgX listr5pool

# vxpool -g dgX printTY NAME TYPE AUTOGROW SELFSUFFICIENT DESCRIPTIONst r5pool data diskgroup pool Volume has paritydm dgX08 - - - -dm dgX07 - - - -dm dgX06 - - - -dm dgX03 - - - -dm dgX02 - - - -dm dgX01 - - - -

By default, the autogrow policy for pools is set to 2 (diskgroup). The poolcan be grown by bringing in additional storage from the disk groupoutside of the storage pool.

By default, the selfsufficient policy is set to level 1 (pool). It only allowsthe use of templates that have been manually assigned to the storage pool.

Storage pool attributes can be modified after initial pool creation. See thevxpool man page for more details.

The vxprint command provides a method for evaluating current storagepools. In the following example, a disk group named dgX, containing 10disk drives, has a data pool named r5pool that uses 6 of the disk group’sdrives.

# vxprint -g dgXTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg dgX dgX - - - ALLOC_SUP

dm dgX04 c2t3d0s2 - 17679776 - -dm dgX05 c2t5d0s2 - 17679776 - -dm dgX09 c3t37d0s2 - 17679776 - -dm dgX10 c3t50d0s2 - 17679776 - -

st r5pool - - - - DATAdm dgX01 c2t16d0s2 - 17679776 - -dm dgX02 c2t18d0s2 - 17679776 - -dm dgX03 c2t1d0s2 - 17679776 - -dm dgX06 c3t32d0s2 - 17679776 - -dm dgX07 c3t33d0s2 - 17679776 - -dm dgX08 c3t35d0s2 - 17679776 - -



Using Application Volume Templates

There are currently 21 predefined application volume templates. Eachstorage pool template references 5-15 volume templates.

Each predefined volume template references one or two volumecapabilities. For instance, the storage pool template raid5_volumes refersto a set of ten volume templates.

Most of the volume templates contain the name of a single capability.Many of the capabilities use the same name as the volume templates. Thecapabilities define one or more variables that can be used when creatingapplication volumes. Many of the variables have default values. Thefollowing are capabilities for the Raid5Volume template.

Volume Template: Raid5Volume

Provides capabilities: Raid5Capability , Raid5LogMirroring

Variables: ncols Minimum number of columns

nlogs Number of logs

nmaxcols Maximum number of columns

The Raid5Capability capability provides the ncols and nmaxcolsvariables which define the minimum and maximum number of columnsto create for a RAID-5 volume. The default value for ncols is 8 and fornmaxcols is 20.

The Raid5LogMirroring capability provides the nlogs variable whichdefines the number of logs to create for a RAID-5 volume. The defaultvalue is 1.

Another volume template named LogsOnSeparateComponents has thefollowing capabilities:

Volume Template: LogsOnSeparateComponents

Provides capabilities: LogsOnSeparateComponents

Variables: component Name of component

The component variable can be set to Controller or Enclosure toseparate multiple logs. The default is Enclosure .



Deriving Volume Template Capabilities

Table 6-2 shows capabilities resulting from applying themirrored_volumes storage pool template to a disk group. Many of thevariables have default values.

Table 6-2 Volume Capability Derivation

Storage Pool: mirrored_volumes

Associate VolumeTemplates

CapabilityName Capability Variables

ArrayProductId Same name Volume uses storage with the sameproductid

ConfineLogsToSimilarStorage

Same name Logs confined to same storage byvariable: namedefault: Enclosure

ConfineMirrorsToSimilarStorage

Same name Mirrors confined to same storage byvariable: namedefault: Enclosure

ConfineToSimilarStorage

Same name Volumes confined to same storage byvariable: namedefault: VendorName

ConfineToSpecificStorage

Same name Volume confined to specific storage byvariable: name (no default)variable: value (no default)

DataMirroring Same name Number of mirror set byvariable: nmirs default: 2

DCOLogMirroring Same name Number of DCO log mirrors set byvariable: nlogs default:1

InstantSnapshottable Same name This volume supports instant snapshots.

LogsOnSeparateComponents

Same name Separate volume logs by component.variable: component default:Enclosure

MirrorsOnSeparateComponents

Same name Separate volume mirrors by component.variable: component default:Enclosure

MultipathingThroughMultiplePaths

Same name Number of paths that can fail.variable: npaths default=2



Creating Application Volumes Using the vxvoladmCommand

You use the vxvoladm command to create application volumes. Thefollowing example shows the use of the vxvoladm command to create adefault RAID-5 volume in a storage pool named r5pool .

# vxvoladm -g dgX make r5vol 10m volume_template=Raid5Volume

# vxprint -g dgX r5volTY NAME ASSOC KSTATE LENGTH PLOFFS STATEv r5vol raid5 ENABLED 20480 - ACTIVEpl r5vol-01 r5vol ENABLED 20480 - ACTIVEsd dgX03-01 r5vol-01 ENABLED 2560 0 -sd dgX05-01 r5vol-01 ENABLED 2560 0 -sd dgX04-01 r5vol-01 ENABLED 2560 0 -sd dgX02-01 r5vol-01 ENABLED 2560 0 -sd dgX06-01 r5vol-01 ENABLED 2560 0 -sd dgX08-01 r5vol-01 ENABLED 2560 0 -sd dgX07-01 r5vol-01 ENABLED 2560 0 -sd dgX09-01 r5vol-01 ENABLED 2560 0 -sd dgX10-01 r5vol-01 ENABLED 2560 0 -pl r5vol-02 r5vol ENABLED 8640 - LOGsd dgX01-01 r5vol-02 ENABLED 8640 0 -

The initial results are unexpected because the autogrow policy allowedthe use of disk group disks outside of the pool and the default for thenmaxcols variable is 20. The result is a 9 column RAID-5 volume thatuses all of the disk group disks, even those outside of the defined pool.

# vxvoladm -g dgX remove volume r5vol

When the r5vol volume is removed, the extra disks are automaticallyremoved from the r5pool .

# vxpool -g dgX getpolicy r5poolAUTOGROW SELFSUFFICIENTdiskgroup pool

# vxpool -g dgX setpolicy r5pool autogrow=pool

# vxvoladm -g dgX make r5vol 10m volume_template=Raid5Volume \capability=’Raid5Capability(nmaxcols=4)’



Creating Application Volumes Using the VEA GUI

You create application volume using the VEA GUI by clicking a diskgroup that has storage pools configured and selecting New Volume fromits pop-up menu.

When you create a new volume in a disk group that contains configuredstorage pools, the New Volume Wizard is aware that a storage pool exists.It automatically displays all possible volume configuration capabilitiesbased on templates associated with the data pool in the disk group.Initially, none of the capabilities are enabled. If you do not enable any ofthe capabilities, the completed volume is a simple concatenation.

Selecting Volume Capabilities

As shown in Figure 6-20, some capabilities display variables, such asnumber of mirrors, that you can modify if needed.

Figure 6-20 Selecting Volume Capabilities



Saving Custom Volume Templates

The next application volume window display, shown in Figure 6-21,summarizes your capability choices and offers the opportunity to save thecapability configuration as a custom volume template.

If you save a custom template, the next time you create a volume, you areoffered the opportunity to use your custom template and bypass themanual capability process.

Figure 6-21 Saving Custom Volume Templates



Interpreting Application Volume Configurations

As shown in the following example, disk group disks, pools, and theirrelated application volumes are grouped together by the vxprintcommand.

# vxprint -g dgSPTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg dgSP dgSP - - - ALLOC_SUP

dm dgSP05 c2t22d0s2 - 17679776 - -

st mirrored_volumes1 - - - - DATAdm dgSP01 c2t16d0s2 - 17679776 - -dm dgSP02 c2t18d0s2 - 17679776 - -dm dgSP03 c2t1d0s2 - 17679776 - -dm dgSP04 c2t20d0s2 - 17679776 - -dm dgSP06 c2t3d0s2 - 17679776 - -dm dgSP07 c2t5d0s2 - 17679776 - -

v datapool_vol01 fsgen ENABLED 2097152 - ACTIVEpl datapool_vol01-01 datapool_vol01 ENABLED 2097152 - ACTIVEsd dgSP04-01 datapool_vol01-01 ENABLED 2097152 0 -pl datapool_vol01-02 datapool_vol01 ENABLED 2097152 - ACTIVEsd dgSP01-01 datapool_vol01-02 ENABLED 2097152 0 -

st striped_volumes1 - - - - CLONEdm dgSP08 c3t32d0s2 - 17679776 - -dm dgSP09 c3t33d0s2 - 17679776 - -dm dgSP10 c3t35d0s2 - 17679776 - -dm dgSP11 c3t37d0s2 - 17679776 - -dm dgSP12 c3t50d0s2 - 17679776 - -dm dgSP13 c3t52d0s2 - 17679776 - -

Replacing Failed Disk Drives



One of the most common VxVM errors that you might see is detachedplex messages. This error message usually means that one of the mirrorsin a volume has encountered a disk drive error while being accessed.

You must identify the physical path to the failed disk drive before youproceed. The most common tools you use to do this are:

● The vxprint command

● The vxdisk command

● The /var/adm/messages file

Failure Behavior

A plex (mirror) is detached if a persistent I/O error is encountered. Thereare several things to be aware of before proceeding:

● Disk block read errors might affect one subdisk, while other subdiskson the same physical disk drive remain functional.

● Errors are not detected until read or write operations are attempted.

● Severe disk drive errors, such as general access failures, result inrelocation of all redundant subdisks associated with the failed disk.They are relocated to either designated hot spare disks or to any diskthat does not have the nohotuse flag set.

Hardware RAID Behavior

Hardware RAID storage units, such as the Sun StorEdge T3 array, presentLUNs to attached systems. Each LUN is actually a portion of a hardwareRAID structure that is monitored internally in the storage array for diskfailures. Typically, the internal RAID volumes are redundant, such asRAID 5 or RAID 1, and the storage array internally relocates the failingdata to a designated spare drive. Hardware RAID internal failures areusually transparent to the VxVM software. Hardware RAID storagedevices usually notify the user root mail account of internally detectedproblems.



Evaluating Failure Severity

By default, the VxVM software sends failure notification email to the userroot account. The extent of failures must be evaluated in a timelymanner.

It is important to assess failures in a timely manner before a problemescalates to a much worse situation. The first line of defence is to examinesystem error logs and verify volume status.

Use root Email Notification

Typically, a series of root account email error notifications follow a devicefailure. After the initial error notification, several more message areusually sent as VxVM attempts to relocate failed volume components. Thefollowing example shows an initial error notification.

# mail?From [email protected] Fri Nov 7 21:11:59 2003Date: Fri, 7 Nov 2003 21:11:58 -0500 (EST)From: Super-User <[email protected]>Message-Id: <[email protected]>To: [email protected]: Volume Manager failures on host ns-east-104Content-Length: 239

Failures have been detected by the VERITAS Volume Manager:

failed disks: dgX04

failed plexes:mirvol_01-02

The Volume Manager will attempt to find spare disks, relocate failedsubdisks and then recover the data in the failed plexes.



Examining Errors in the /var/adm/messages File

The following messages are seen when a disk drive is experiencing accessproblems, such as hard write errors.

Nov 7 21:11:40 ns-east-104 vxio: [ID 245403 kern.warning] WARNING: VxVMvxio V-5-0-151 error on Plex mirvol_01-02 while writing volume mirvol_01offset 16 length 4Nov 7 21:11:40 ns-east-104 vxio: [ID 786473 kern.warning] WARNING: VxVMvxio V-5-0-4 Plex mirvol_01-02 detached from volume mirvol_01Nov 7 21:11:40 ns-east-104 vxio: [ID 628984 kern.warning] WARNING: VxVMvxio V-5-0-386 dgX04-01 Subdisk failed in plex mirvol_01-02 in volmirvol_01Nov 7 21:11:40 ns-east-104 vxvm:vxconfigd: [ID 976563 daemon.notice] V-5-1-768 Offlining config copy 1 on disk c2t3d0s2:Nov 7 21:11:40 ns-east-104 vxvm:vxconfigd: [ID 672837 daemon.notice]Reason: Disk write failureNov 7 21:11:41 ns-east-104 vxvm:vxconfigd: [ID 905431 daemon.notice] V-5-1-7909 Detached disk dgX04

Checking Volume Status

The vxprint command is the easiest way to check the status of all volumestructures. In the following excerpt, the status of two plexes in a volume isbad. One of the plexes is a log.

# vxprint

Disk group: sdg0

TY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg sdg0 sdg0 - - - -

dm disk0 c4t0d0s2 - 8368512 - -dm disk7 c5t0d0s2 - 8368512 - -

v vol0 fsgen ENABLED 524288 - ACTIVEpl vol0-01 vol0 DISABLED 525141 - IOFAILsd disk0-01 vol0-01 ENABLED 525141 0 -pl vol0-02 vol0 ENABLED 525141 - ACTIVEsd disk7-01 vol0-02 ENABLED 525141 0 -pl vol0-03 vol0 DISABLED LOGONLY - IOFAILsd disk0-02 vol0-03 ENABLED 5 LOG -



Note – If a hot spare is available and the volume is redundant, thestructure is relocated to a different disk drive and you never see suchvolume states.

When VxVM detects a disk drive failure, it can place a failed plex in anumber of different states. The two most common states for a failed plexare:

● DETACHED/IOFAIL

● DISABLED/NODEVICE

The DETACHED/IOFAIL state is not so severe. It is relatively easy toidentify the failed device path by examining the vxprint -ht commandoutput as follows:

# vxprint -ht | grep sd0b-01sd sd0b-01 lv0b-01 disk7 0 525141 0 c5t0d0 ENA

In the previous example, the VxVM media name is disk7 and thephysical path is c5t0d0 .

When the VxVM software loses complete contact with a disk drive, thephysical path in the vxprint -ht command’s output might be blank. Atthose times, you must determine the media name of the failed disk drivefrom the vxprint command, and then use the vxdisk list command toassociate the media name with the physical device.

# vxdisk listDEVICE TYPE DISK GROUP STATUSc0t0d0s2 auto:sliced rootdg01 rootdg onlinec0t1d0s2 auto:sliced rootdg02 rootdg online nohotusec2t1d0s2 auto:sliced dgX03 dgX onlinec2t3d0s2 auto:sliced - - onlinec2t5d0s2 auto:sliced dgX05 dgX onlinec2t16d0s2 auto:sliced dgX01 dgX onlinec2t18d0s2 auto:sliced dgX02 dgX onlinec2t20d0s2 auto:sliced dgX06 dgX online- - dgX04 dgX failed nohotuse was:c2t3d0s2

When a disk drive fails and becomes detached, the VxVM software cannotfind the disk drive, but it still knows the physical path. This information isthe origin of the failed was status. This status means that the disk drivehas failed and that the physical path is the value displayed in the STATUScolumn.



General Disk Drive Replacement Process

When a disk drive containing redundant volumes fails, usually all of thesubdisks are relocated to a hot spare and the redundant volume continuesto function normally. Subdisk relocation messages are sent to the userroot email account.

The VxVM software requires that a specific recovery process is followedto replace the failed disk drive, and then relocate the subdisks back to thenew disk drive.

Some storage arrays also require that specific disk replacement processesare followed.

Disk drive failures in hardware RAID storage, such as the Sun StorEdgeT3 arrays, are frequently transparent to VxVM. This is because theirinternal LUN structures are redundant and an internal spare isautomatically substituted for the failed drive.

The overall replacement process is typically a mixture of software andhardware replacement procedures. Following is a summary of theprocess.

1. If appropriate, try to resolve temporary problems before taking morecomplex action.

If the disk drive is detached due to a secondary problem, such as aloose cable, you can try to reattach it using the vxreattachcommand.

Reattachment might resolve all problems. If you feel the disk is stillat risk, you can evacuate it and perform a hardware replacement.

2. Remove the failed disk drive from VxVM control.

Use the vxdiskadm utility option 4, Remove a disk forreplacement , to remove the physical disk from its disk group whileretaining its media name.

If you intend to hot-swap the failed disk without rebooting thesystem, you might also use the vxdiskadm utility option 11, Disable(offline) a disk device , to stop all VxVM access, such aspolling.

3. Perform the storage-specific hardware replacement procedure.

The hardware replacement procedures vary widely depending onthe storage type and model. Always consult the technical manualsfor your particular array model.



4. Return the failed disk drive to VxVM control.

Use the vxdiskadm utility option 10, Enable (online) a diskdevice , to enable VxVM access to the replacement disk.

Use the vxdiskadm utility option 5, Replace a failed orremoved disk , to associate the new disk with its media name. Youinitially select None as a replacement device. You are then asked ifyou want to initialize the new disk drive.

Note – In some cases you might need to scan for new disk drives usingeither the vxdiskconfig or vxdctl enable command.

5. Use the vxdiskadm utility option 14, Unrelocate subdisks backto a disk , to move the relocated volume components back to theoriginal disk location.

The only information you furnish is the media name of thereplacement disk, VxVM.

Note – If a failed volume is non-redundant, you cannot recover it. Youmust manually recreate the volume structure, using tools such asvxconfigrestore , and restore the data from backup tapes.

Exercise: Performing Advanced Operations





● Encapsulate the system boot disk

● Mirror the system boot disk

● Perform an online volume relayout

● Evacuate a disk drive

● Move a populated volume

● Perform a snapshot backup

● Create a layered volume

● Replace a failed disk drive

● Use Intelligent Storage Provisioning

● Configure a best practice boot disk

Preparation

If your lab environment uses a central VxVM server instead of standaloneworkstations, the ‘‘Encapsulating the System Boot Disk’’ section on page6-4 and the ‘‘Mirroring the System Boot Disk’’ section on page 6-5 must beperformed as a demonstration on the VxVM server.

The demonstration is typically performed by the instructor. The tasksrequire a second internal disk drive that is the same model and capacityas the system boot disk.




Answer the following questions about boot disk mirroring, hot devices,and layered volumes.

1. Which of the following are features of a preferred boot diskconfiguration?

a. The boot disk should be located in a storage array.

b. The boot disk should be mirrored.

c. The boot disk should not be part of the rootdg disk group.

d. The boot disk should have at least two unused slices.

e. The boot disk should be at least 18 Gbytes in size.


2. Which answer most accurately describes the best practice boot diskconfiguration process?

a. Initialize, copy, mirror, delete

b. Encapsulate, mirror, delete, copy

c. Copy, delete, encapsulate, mirror

d. Mirror, encapsulate, initialize, copy

The answer is b.

3. What is the primary benefit of a best-practice boot diskconfiguration?

a. It improves boot disk performance

b. It standardizes recovery procedures

c. It simplifies boot disk prerequisites

The answer is b.

4. What is the key feature of hot relocation?

a. Whole disk drive relocation

b. Cylinder group relocation

c. Subdisk relocation

d. Sub-volume relocation

The answer is c.



5. Can designated hot spares be used for hot relocation?

a. Yes

b. No

The answer is a.

6. What is a key prerequisite for both hot spares and hot relocation?

a. Volumes must be striped

b. Volumes must be failure tolerant

c. Volumes must be mirrored

d. Volumes must be striped mirror

The answer is b.

7. Can a designated hot spare be used for typical volume creation?

a. Yes

b. No

The answer is b.

8. What is the default hot-device mode of operation?

a. Hot relocation

b. Hot spare

The answer is a.

9. Which of the following commands performs hot-deviceadministration?

a. vxdisk set

b. vxtask set

c. vxedit set

d. vxdisksetup

The answer is c.



10. Which of the following commands starts a layered volume?

a. vxvol

b. vxrecover

c. vxdctl

d. vxdiskadm

The answer is b.

11. What is the primary purpose of the ISP software?

a. To create complex volume structures

b. To increase overall system performance

c. To manage complex SAN environments

The answer is c.

12. Which of the following commands is used to create ISP applicationvolumes?

a. vxvol

b. vxvoladm

c. vxassist

d. vxdiskadm

The answer is b.



Task 2 – Encapsulating the System Boot Disk

If your lab configuration uses a central VxVM server, this task must beperformed as a demonstration (probably by the instructor).

1. Before proceeding with this task, ensure that your boot disk meetsthe following requirements:

● The boot disk must have at least two unused slices.

● Ideally, the boot disk does not have any slices in use other thanthe root and swap partitions.

● There should be a minimum of 2048 sectors (blocks) at thebeginning or end of the boot disk that are not assigned to apartition.

A standard lab boot disk should either have one or twounassigned cylinders or have them assigned to an unusedpartition, usually slice 7. If necessary, use the format utility tozero out the small partition at the end of the boot disk. Do notproceed until your boot disk meets this requirement.

2. Start the vxdiskadm utility, select option 2, and answer thepreliminary questions as follows. Substitute the logical path to yourdisk drive.

# vxdiskadm


Encapsulate one or more disksMenu: VolumeManager/Disk/Encapsulate

Select disk devices to encapsulate:[<pattern-list>,all,list,q,?] c0t0d0

Which disk group [<group>,list,q,?] rootdgThere is no active disk group named rootdg.Create a new group named rootdg? [y,n,q,?] (default: y) yUse a default disk name for the disk? [y,n,q,?] (default: y) y

A new disk group rootdg will be created and the disk device c0t0d0 willbe encapsulated and added to the disk group with the disk name rootdg01




3. Reboot the Solaris OS.

# init 0ok boot

4. Observe the following system messages as the encapsulation processproceeds through reconfiguration reboots.

VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup...

VxVM vxvm-reconfig INFO V-5-2-324 The Volume Manager is nowreconfiguring (partition phase)...

VxVM vxvm-reconfig INFO V-5-2-499 Volume Manager: Partitioning c0t0d0as an encapsulated disk.

/dev/dsk/c0t0d0s1 was dump device --invoking dumpadm(1M) -d swap to select new dump device

VxVM vxvm-reconfig INFO V-5-2-323 The Volume Manager is nowreconfiguring (initialization phase)...

VxVM vxvm-reconfig INFO V-5-2-497 Volume Manager: Adding rootdg01(c0t0d0) as an encapsulated disk.

VxVM vxcap-vol INFO V-5-2-89 Adding volumes for c0t0d0...Starting new volumes...VxVM vxcap-vol INFO V-5-2-444 Updating /etc/vfstab...Remove encapsulated partitions...

VxVM vxroot INFO V-5-2-328 The Volume Manager will now set up your BootDisk as a managed disk.VxVM vxroot INFO V-5-2-290 Saving original configuration...80 blocks

VxVM vxvm-reconfig NOTICE V-5-2-393 The system will now be rebooted.syncing file systems... done

rebooting...Rebooting with command: boot...VxVM INFO V-5-2-3247 starting special volumes ( swapvol rootvol )...VxVM vxvm-startup2 INFO V-5-2-503 VxVM general startup...vxvm: NOTE: Setting partition /dev/dsk/c0t0d0s1 as the dump device.



5. Log into the system as user root and verify the boot diskenvironment is correctly configured.

# vxprint# eeprom nvramrc# eeprom “use-nvramrc?”=true# vxdg defaultdgnodg# vxdg bootdgrootdg

Task 3 – Mirroring the System Boot Disk

Perform the following steps only if you have already encapsulated yourboot disk and have a disk drive available that is identical to the primarysystem boot disk.

In a production system, it would be best to take the system offline beforestarting a procedure such as the following. Single-user mode would beideal.

Caution – This procedure assumes two identical disk drives. The primaryboot disk address is c0t0d0 , and the mirror disk is c0t1d0 . Ensure thatyou are using the correct address or disk media name for each step in thisprocedure.

1. Locate an appropriate mirror disk and add it to the rootdg diskgroup using the VxVM medianame rootdg02 .

# vxdisk list# vxdisksetup -i c0t1d0# vxdg -g rootdg adddisk rootdg02=c0t1d0

2. Start the vxdiskadm utility and select option 6, Mirror Volumes ona disk .

# vxdiskadmSelect an operation to perform: 6Mirror volumes on a disk

At the prompt below, supply the name of the disk containing the volumesto be mirrored.

Enter disk name [<disk>,list,q,?] listDisk group: rootdg



DM NAME DEVICE TYPE PRIVLEN PUBLEN STATEdm rootdg01 c0t0d0s2 auto 2048 8380799 -dm rootdg02 c0t1d0s2 auto 4063 8374320 -

Enter disk name [<disk>,list,q,?] rootdg01Enter destination disk [<disk>,list,q,?] (default: any) rootdg02VxVM vxmirror INFO V-5-2-22 Mirror volume rootvol ...VxVM vxmirror INFO V-5-2-22 Mirror volume swapvol ...

3. Use the vxprint command to examine your completed boot diskconfiguration.

# vxprint -g rootdgns-east-104# vxprint -g rootdgTY NAME ASSOC KSTATE LENGTH PLOFFS STATEdg rootdg rootdg - - - -


v rootvol root ENABLED 17139843 - ACTIVEpl rootvol-01 rootvol ENABLED 17139843 - ACTIVEsd rootdg01-B0 rootvol-01 ENABLED 1 0 -sd rootdg01-02 rootvol-01 ENABLED 17139842 1 -pl rootvol-02 rootvol ENABLED 17139843 - ACTIVEsd rootdg02-01 rootvol-02 ENABLED 17139843 0 -

v swapvol swap ENABLED 531468 - ACTIVEpl swapvol-01 swapvol ENABLED 531468 - ACTIVEsd rootdg01-01 swapvol-01 ENABLED 531468 0 -pl swapvol-02 swapvol ENABLED 531468 - ACTIVEsd rootdg02-02 swapvol-02 ENABLED 531468 0 -

4. Examine the OpenBoot PROM boot device aliases and verify that analias for the boot disk mirror was added.

# eeprom nvramrcnvramrc=devalias vx-rootdg01/pci@1f,4000/scsi@3/disk@0,0:adevalias vx-rootdg02/pci@1f,4000/scsi@3/disk@1,0:a

5. Halt the Solaris OS and boot from the VxVM boot disk device alias.

# init 0...ok boot vx-rootdg01



Task 4 – Performing an Online Volume Relayout

In this task, you use the volume relayout feature of the VEA GUI to addtwo columns to an existing 3-column RAID-5 volume. This task can alsobe accomplished using the command line, but the syntax is complex.

It is best to unmount the volume file system before performing an onlinerelayout operation.

To perform a volume relayout, complete the following steps:

1. Click the third mouse button on your RAID-5 volume in the VEAGUI grid area and select Change Layout from the pop-up menu.

Let the VxVM software select additional disk drives.

2. In the Change Volume Layout form, change the Columns entry to 5(as shown in Figure 6-22) and click OK.

Figure 6-22 Change Volume Layout Form



3. When the status window appears, as shown in Figure 6-23, trypausing and continuing the relayout operation.

Figure 6-23 Relayout Status Monitor

4. After the relayout has completed, examine the new volume structureusing the vxprint command and verify that the results are whatyou anticipated.

5. Unmount the RAID-5 volume and delete it.

6. Create a two-disk mirrored volume, 200 Mbytes in size, with amounted file system.

7. Use the mkfile command to create some test files in the mirror-volume’s file system.

# mkfile 10m /Test/file1# mkfile 20m /Test/file2



Task 5 – Evacuating a Disk Drive

To evacuate a disk drive, complete the following steps:

1. Start the vxdiskadm utility on the VxVM server, and select option 7,Move volumes from a disk .

2. Move the contents of one of the disk drives in your new mirroredvolume to another unused disk drive in your disk group.

Caution – If you do not specify a destination disk drive for the move,VxVM uses any available disk space in the disk group. This might resultin a poorly configured volume with performance problems.

Task 6 – Moving a Populated Volume

In this task, you move your mirrored file system volume into anotherexisting disk group. You must first create a second disk group. Completethe following steps:

1. On the VxVM server, use the vxprint command to determine themedia names and logical paths of the two disk drives being used inyour mirrored file system.

# vxprint -g old_dg -ht

2. Record the media names and logical paths of the two disk drivesbeing used in your mirrored volume. For example: dgB01, c3t4d0 .

Media name: __________ __________

Logical path: __________ __________

3. Remove an unused disk drive from your current disk group and useit to create a second disk group.

4. Use the vxprint command to save the mirrored volumeconfiguration.

# vxprint -hmQq -g old_dg volumename > volumename .save

Note – Do not locate the volumename .save file on a disk drive that isbeing relocated.

5. Unmount any file systems associated with the mirrored volume.

6. Stop the mirrored volume.

# vxvol -g old_dg stop volumename



7. Remove the definitions of the structures (volume, plexes, andsubdisks) from the configuration database.

# vxedit -g old_dg -r rm volumename

8. Remove the disk drives from the original disk group.

# vxdg -g old_dg rmdisk olddg01 olddg02

Note – The disk drives should now be in the free disk pool.

9. Add the two disk drives to the new disk group.

10. Rename the disk drives so that they have their original media names.

Caution – Step 10 is critical. If the disk drives do not have their originalmedia names, the configuration reload fails.

11. Use the vxmake command to reload the saved configuration for thevolumename volume.

# vxmake -g new_dg -d volumename .save

The -d option specifies the description file to use for buildingsubdisks, plexes, and volumes.

12. Use the vxvol command to bring the mirrored volume back online.

# vxvol -g new_dg init active volumename

13. Mount the mirrored volume file system to return the mirroredvolume to service.

14. Unmount the mirrored volume and destroy its disk group.

15. Add all of your disk drives into a single disk group again.



Task 7 – Performing a Snapshot Backup

To perform a snapshot backup, complete the following steps:

1. Create a two-disk mirrored volume, 200 Mbytes in size, with amounted file system.

2. Use the mkfile command to create some test files in the mirror-volume’s file system.

# mkfile 10m /Test/file1# mkfile 20m /Test/file2

3. Click the mirrored volume in the VEA GUI grid area, and selectSnapshot Interactive from the pop-up menu.

Note – Although it is not absolutely necessary, you can assign disk drivesfor temporary use in the Volume Snapshot form.

4. Click Snapstart on the Volume Snapshot form to start the snapshotprocess.

This can take some time depending on the volume size.

5. Click Snapshot when the mirror copy is completed.

This detaches the new mirror and creates a separate volumefrom it.

6. Use the vxprint command to verify that a volume structure similarto the following is present:

v SNAP-vol02 fsgen ENABLED 204800 - ACTIVE - -pl vol02-02 SNAP-vol02 ENABLED 208278 - ACTIVE - -sd dgY07-01 vol02-02 ENABLED 208278 0 - - -

7. Back up the new snapshot volume to tape (if possible). The followingexample shows the process.

# fsck -y /dev/vx/rdsk/dgY/SNAP-vol02# mkdir /vol02_backup_081202# mount /dev/vx/dsk/dgY/SNAP-vol02 /vol02_backup_081202# cd /# tar cvf /dev/rmt/0 ./vol02_backup_081202

8. Unmount and delete the snapshot volume.

# umount /vol02_backup_081202# vxedit -g dgY -rf rm SNAP-vol02



Task 8 – Creating a Layered Volume

To perform this task, you need a minimum of five unused disk drives in adisk group.

To create a striped-mirror RAID 1+0 volume, complete the followingsteps:

1. Unmount all volume-related file systems.

2. Stop all volumes in your disk group and delete them.

3. Display the disk drives in your disk group in the VEA GUI grid area,and select all six of them by simultaneously clicking the mousebutton and pressing the Control key.

4. Click New Volume in the toolbar.

5. Fill out the New Volume Wizard form as follows:

a. Enter a volume name, such as strmir_vol01 .

b. Enter a size of 200m or larger.

c. Select the Stripe Mirrored layout.

d. Leave the number of columns at 2.

e. Leave the number of mirrors at 2.

f. Leave logging enabled

g. Add a file system that mounts at boot time.

6. Click Next and Finish in the next two forms.

It can take quite a while to complete a large layered volume. Keepchecking the status of the volume completion either from thecommand line or by using the Task Monitor.

7. Use the vxprint command to verify that your RAID 1+0 volume isstructured using sub-volume components.

8. Click the layered volume in the VEA GUI grid area and selectLayout View from its pop-up menu.

9. Resize the Volume Layout window until you can see all of thelayered volume components.

10. Close the Volume Layout window when you finish examining thevolume’s structure.

11. Test your new volume’s file system by copying data into it.



Task 9 – Replacing a Failed Disk Drive

You can decided whether or not to perform this procedure as a group demonstration.

To simulate a failed disk drive replacement, complete the following steps:

1. Unmount and delete your RAID 1+0 volume.

Caution – Before proceeding, ensure that all of the disks in the disk groupare initialized in a sliced format. If the disks are in a cdsdisk format, thefailure simulation will not work. If necessary, destroy the disk group andrecreate it using the vxdiskadm utility.

2. Create a two-disk mirrored volume, 200 Mbytes in size, with amounted file system, with logging disabled (no DRLs).

3. Use the mkfile command to create a test file in the mirroredvolume’s file system.

# mkfile 10m /Test/file1

4. Double-click on the volume in the VEA GUI object tree to displaydetailed information about the volume’s disk drives.


a. Simulate a disk drive failure by zeroing-out the VTOC of one ofthe mirrored volume disks.

b. Substitute the physical path to one of your mirrored volumedisks.

# fmthard -s /dev/null /dev/rdsk/ c2t3d0s2

6. Create a second test file in your mirrored volume’s file system.

# mkfile 20m /Test/file2

You should see some plex-related errors.


a. Type mail on the VxVM server and press the Return key toview each new message.

b. Type q when you are done.

Unless you disabled hot-relocation use on all of your disk drive,VxVM relocates the mirror on the failed disk drive to a differentdisk drive.



8. Use the vxprint command to search for disabled plexes.

The mirrored volume should show a normal status because thefailed plex was relocated to a different disk drive.


a. Use the vxdisk list command to verify the failed disk drive.

b. Record the logical path to the failed disk drive.

Failed disk drive logical path: __________________________________

10. Remove the failed disk drive from VxVM control.

11. Use the vxdiskadm utility option 4, Remove a disk forreplacement , to remove the physical disk from its disk group whileretaining its media name. Do not select a replacement disk.

Note – If you intend to hot-swap the failed disk without rebooting thesystem, you might also use the vxdiskadm utility option 11, Disable(offline) a disk device , to stop all VxVM access, such as polling.

12. Perform the storage-specific hardware replacement procedure.

The hardware replacement procedures vary widely depending onthe storage type and model. Always consult the technical manualsfor your particular array model.

13. Return the failed disk drive to VxVM control.

a. Use the vxdiskadm utility option 10, Enable (online) adisk device , to enable VxVM access to the replacement disk.

b. Use the vxdiskadm utility option 5, Replace a failed orremoved disk , to associate the new disk with its media name.

You initially select none as a replacement device. You are thenasked if you want to initialize the new disk drive.

Do not use FMR for plex resync.

Note – In some cases you might need scan for new disk drives using thevxdctl enable command.



14. Use the vxdiskadm utility option 14, Unrelocate subdisks backto a disk , to move the relocated volume components back to theoriginal disk location and complete the following steps:

● The only information you furnish is the media name of thereplacement disk, VxVM

● Answer no to Unrelocate to a new disk.

● Answer no to Use -f option.

15. Use the vxprint command to verify the mirrored volume isreturned to its original configuration.

The letters UR are added to the relocated subdisk name.

Task 10 – Using Intelligent Storage Provisioning

Perform the following steps to configure ISP storage pools and ISPapplication volumes.

1. Unmount and delete all volumes in your disk group.

2. Click your disk group in the VEA GUI object tree and selectOrganize DiskGroup from its pop-up menu.

3. Click on the Mirrored Data and Striped Snapshot organizationcategory in the Organize DiskGroup Wizard window.

4. Use the default data pool and clone pool names.

5. Review the summary information and click Finish when you aresatisfied with the configuration.

6. Display your disk group in the VEA GUI grid area and click theStorage Pools tab.


a. Click the mirrored_volumes1 data pool with the third mousebutton and select Properties from its pop-up menu.

b. Examine the properties and capabilities associated with themirrored_volumes1 data pool.


a. Click the mirrored_volumes1 data pool with the third mousebutton and select Add/Remove Disks from its pop-up menu.

b. Add all six of your disk drive to the mirrored_volumes1 datapool.



9. Click your disk group in the VEA GUI object tree and select NewVolume from its pop-up menu.

10. Create a mirrored ISP application volume as follows:

a. Click Next in the Select User Template window.

b. Click Data Redundancy and Data Mirroring capabilities, andthen click Next.

c. Leave the Let Volume Manager Decide button enabled, andthen click Next.

d. Complete the following steps:

1. Enter a volume name.

2. Set the size to 200 Mbytes

3. Click Next.

e. Do not create a file system on the volume.

f. Complete the following steps:

1. Enter a user template name.

2. Click Save.

3. Click Finish.

g. Use the VEA GUI and the vxprint command to examine theresulting volume structure and storage pool organization.

h. Delete the mirrored ISP application volume.

11. Create another application volume in your data storage pool usingadditional capabilities.

a. Click Next in the Select User Template window.

b. Click Mirrored DCO Logs, Data Redundancy and DataMirroring capabilities, and then click Next.

c. Leave the Let Volume Manager Decide button enabled, andthen click Next.

d. Complete the following steps:

1. Enter a volume name.

2. Set the size to 200 Mbytes.

3. Click Next.

e. Do not create a file system on the volume.

f. Click Finish.



Note – DCO logging is associated with a feature not currently licensed orsupported by Sun Microsystems.

12. Click your disk group in the VEA GUI object tree and select NewStorage Pool from its pop-up menu.

13. Create another storage pool in your disk group as follows:

a. Enter a storage pool name and click Next.

b. Click the RAID-5 storage pool template, and then click Next.

c. Examine the summary information, and then click Finish.

14. Display your storage pools in the VEA GUI grid area and verify thatthe new storage pool is a clone pool.

15. Delete all of your storage pools and organize your disk group againusing a different storage pool set template, such as the StripedMirrored Data and Striped Snapshots template.

16. Create one more application volume using the new storage poolcapabilities.



Task 11 – Configuring a Best Practice Boot Disk

Do not perform this task unless you have encapsulated and mirrored yourboot disk in an earlier task.

The following section assumes that all disk drives and volumecomponents follow the default VxVM naming conventions.

To reorganize the disks into a best practice configuration, complete thefollowing steps:

1. Un-mirror the primary boot disk by removing the mirror plexes fromthe mirror disk. Use the Bourne shell.

# vxassist -g rootdg remove mirror rootvol !rootdg02# vxassist -g rootdg remove mirror swapvol !rootdg02

2. To ensure the correct mirror placement, manually mirror therootvol and swapvol volumes again in the order shown.

# vxrootmir rootdg02# vxassist -g rootdg mirror swapvol rootdg02

3. Disassociate all primary boot disk plexes and recursively removethem.

# vxplex -g rootdg -v rootvol dis rootvol-01# vxplex -g rootdg -v swapvol dis swapvol-01# vxedit -g rootdg -r rm rootdg01Priv# vxedit -g rootdg -r rm rootvol-01# vxedit -g rootdg -r rm swapvol-01

4. Verify that only the mirror disk plexes remain.



v rootvol root ENABLED 7808400 - ACTIVEpl rootvol-02 rootvol ENABLED 7808400 - ACTIVEsd rootdg02-01 rootvol-02 ENABLED 7808400 0 -




5. Remove the rootdg01 disk from the rootdg disk group, andcomplete the following steps:

a. Reinitialize the rootdg01 disk.

b. Add the rootdg01 disk back into the rootdg disk group.

# vxdg -g rootdg rmdisk rootdg01# vxdisksetup -i c0t0d0 format=simple# vxdg -g rootdg adddisk rootdg01=c0t0d0

6. Use the vxdiskadm option 6, Mirror volume on a disk , to mirrorrootvol and swapvol on the rootdg02 disk drive back to the newlyinitialized rootdg01 disk drive.

# vxdiskadm...At the prompt below, supply the name of the diskcontaining the volumes to be mirrored.

Enter disk name [<disk>,list,q,?] rootdg02

Enter destination disk [<disk>,list,q,?] (default: any)rootdg01

7. Modify the boot-device parameter and enable nvramrc usage.

# eeprom boot-device="rootdisk rootmirror"# eeprom "use-nvramrc?"=true

8. Use the prtvtoc command and the format utility to verify that theprimary and mirror boot disk drives have exactly the same partitionmaps.

9. Boot the system from each of the available system devices.

ok boot disk0ok boot disk1 .ok boot vx-rootdg01ok boot vx-rootdg02

You can now replace either the primary boot disk drive or its mirror in thesame manner as any other VxVM disk drive, and just resynchronize themirrors.

Exercise Summary


Exercise Summary

?!


Manage the discussion here based on the time allowed for this module, which was given in the “About ThisCourse” module. If you find you do not have time to spend on discussion, highlight just the key conceptsstudents should have learned from the lab exercise.

● Experiences

Ask students what their overall experiences with this exercise have been. Go over any trouble spots orespecially confusing areas at this time.

● Interpretations


● Conclusions


● Applications



Module 7

VERITAS File System Basic Operations

Objectives


● Describe basic VxFS features

● Install the VxFS software

● Create VxFS file systems

● Use extended VxFS mount options

● Perform online VxFS administration tasks

Basic VxFS Features


Basic VxFS Features

The VxFS software is targeted at commercial environments where highperformance and availability are important, and large volumes of datamust be managed.

The VxFS extent-based space allocation scheme increases performance byreducing the number of I/O operations required to read and write largeamounts of data.

The VxFS intent log feature provides fast recovery following a systemcrash or reboot. A file system check can be completed in seconds,regardless of the file system size.

Extent-Based Space Allocation

Standard UFS file systems use block-based allocation schemes andprovide good random access to files and reasonable latency on small files.For larger files, however, this block-based architecture limits throughput.The VxFS file system improves file system performance by using adifferent allocation scheme name extent-based allocation.

Disk space is allocated by the system in 512-byte sectors, which aregrouped together to form a logical block. VxFS supports logical blocksizes of 1024, 2048, 4096, or 8192 bytes. The default block sizes varyaccording to file system size as follows:

● 1024 block size for file systems up to 4-Tbytes in size



● 8192 block size for file systems beyond 16-Tbytes in size

An extent is one or more adjacent blocks of data within a file system. It ispresented as an address-length pair, which identifies the starting blockaddress and the length of the extent (in blocks). When data is added to afile on a VxFS system, it is grouped in extents as opposed to beingallocated a block at a time (as is done with UFS file systems).

By allocating disk space in extents, disk I/O to and from a file can be donein units of multiple blocks considerably faster than block-at-a-timeoperations.

Basic VxFS Features

VERITAS File System Basic Operations 7-3Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

File System Intent Logging

After a system failure, the structural integrity of UNIX file systems isverified by the fsck program during the system reboot operation. Thismeans checking the entire structure of a file system, verifying that it isintact, and correcting any inconsistencies that are found. This process canbe very time consuming.

The VxFS file system provides recovery only seconds after a systemfailure by using a tracking feature called intent logging. Intent logging is alogging scheme that records pending changes to the file system structure.During system recovery from a failure, the intent log for each file systemis scanned and operations that were pending are completed. The filesystem can then be mounted without a full structural check of the entiresystem.

When the disk has a hardware failure, the intent log might not be enoughfor recovery and, in such cases, a full fsck check must be performed.However, when the failure is due to software rather than hardware, asystem can be recovered in seconds.

The default intent log size is currently 64 Mbytes. The fsadm commandcan be used to dynamically modify the intent log size. Larger intent logscan improve system performance because they reduce the number oftimes the log wraps around. An intent log that is too large can increase filesystem recovery time after a system failure.

Note – See the fsadm_vxfs and mkfs_vxfs man pages for moreinformation.

Installing the VxFS Software


Installing the VxFS Software

You use the pkgadd command to install the VxFS software. The VxFSpackages should be installed in the order shown.

# pkgadd -d .The following packages are available: 1 VRTSalloc VERITAS Volume Manager: VERITAS Intelligent Storage

Provisioning (sparc) 4.0,REV=12.06.2003.03.07 2 VRTSap VERITAS Action Provider (sparc) 2.00.015 3 VRTSddlpr VERITAS Device Discovery Layer Services Provider (sparc) 4.0,REV=12.06.2003.02.03 4 VRTSfppm VERITAS File Placement. (sparc) 4.0,REV=GA03 5 VRTSfsdoc VERITAS File System Documentation (sparc) 4.0,REV=GA04 6 VRTSfsman VERITAS File System - Manual Pages (sparc) 4.0,REV=GA04 7 VRTSfspro VERITAS File System Management Services Provider (sparc) 4.0,REV=GA04 8 VRTSmuob VERITAS Enterprise Administrator Service Localized

Package (sparc) 3.2.514.0 9 VRTSob VERITAS Enterprise Administrator Service (sparc) 3.2.514.0 10 VRTSobgui VERITAS Enterprise Administrator (sparc) 3.2.514.0 11 VRTStep VERITAS Task Exec Provider (sparc) 1.20.025 12 VRTSvlic VERITAS License Utilities (sparc) 3.02.005d 13 VRTSvmdoc VERITAS Volume Manager (user documentation) (sparc) 4.0,REV=12.06.2003.01.34 14 VRTSvmman VERITAS Volume Manager, Manual Pages (sparc) 4.0,REV=12.06.2003.01.34 15 VRTSvmpro VERITAS Volume Manager Management Services Provider (sparc) 4.0,REV=12.06.2003.01.35 16 VRTSvxfs VERITAS File System (sparc) 4.0,REV=GA04 17 VRTSvxvm VERITAS Volume Manager, Binaries (sparc) 4.0,REV=12.06.2003.01.35

Select package(s) you wish to process (or ’all’ to processall packages). (default: all) [?,??,q]: 16 4 5 6 11 2

Creating VxFS File Systems


Creating VxFS File Systems

You use the same process as with the UNIX UFS file system to create VxFSfile systems. To create VxFS file system on top of VxVM volumes, you canuse either the VEA GUI New File System Wizard or the standard filesystem command-line programs, such as mkfs .

The primary difference is that you must specify a file system type of vxfs .You create a VxFS file system using the mkfs command as follows.

# mkfs -F vxfs /dev/vx/rdsk/dgX/mirvol

The following mkfs command options can affect system performancewhen used appropriately:

● mkfs -F vxfs -o bsize

You use the bsize option to decrease or increase the file systemblock size during initial file system creation. For most applications itis best to use the default VxFS block size, which is automaticallycalculated based on file system size.

● mkfs -F vxfs -o logsize

You use the logsize option to decrease or increase the size of theintent log during initial VxFS file system creation. Increasing the sizeof the intent log can improve performance, but the log must never belarger than half of virtual memory (physical memory + swap).

● mkfs -F vxfs -o largefiles/nolargefiles

If the largefiles option is enabled, a persistent largefiles bit iswritten on the file system. The largefiles bit (flag) allows thecreation of files larger than 2 Gbytes.

The default mode of operation is largefiles .

Note – Consult the mkfs_vxfs man pages for more information on otherVxFS mkfs command options, including inosize and ninode .

Extended VxFS Mount Options



The VxFS software provides a number of mount options that might beappropriate under special circumstances. You should not use many of theoptions unless there is a clearly defined need. The most commonly usedVxFS mount options control intent log behavior and how the VxFSsoftware responds to media errors.

Intent Log Behavior

The log , delaylog , tmplog , logiosize , and nodatinlog mount optionshave varying effects on VxFS file system performance and integrity,depending on specific needs.

● mount -F VxFS -o log

The log option guarantees all file system changes are logged beforea system call returns. Recent changes are not lost.

● mount -F VxFS -o delaylog

The default logging mode is delaylog . Most system call changes arelogged before returning. Some changes are logged shortly after thesystem call returns. After a system failure, the most recent changemight be lost. This behavior is similar to UNIX UFS file systembehavior.

● mount -F VxFS -o tmplog

When the tmplog mode of operation is enabled, most intent logchanges are made after the system calls return. This mode canincrease file system performance, but places recent data changes at ahigher level of risk.

● mount -F VxFS -o logiosize= size

The logiosize option can increase performance by adjusting intentlog write size to match the stripe size of read-modify-write (RAID 5)devices. Valid size values are 512, 1024, 2048, 4096, or 8192.

● mount -F VxFS -o nodatinlog

The nodatainlog option should not be used on systems capable ofrelocating bad disk blocks. The Solaris OS uses bad block relocationto relocate disk blocks that are having recoverable read or writeerrors. If enabled, nodatainlog decreases the file system performanceby approximately 50 percent.



Error Handling Behavior

The ioerror option has several sub-options that are designed to handleerrors in a way that is appropriate for the related storage technology.

● mount -F VxFS -o ioerror=disable

The disable sub-option disables all file system data, and metadatareads and writes when an error is detected. You use this sub-optionwhen the underlying storage is redundant, such as mirroring orRAID 5.

You should use the disable policy for cluster file systems.

● mount -F VxFS -o ioerror=nodisable

The nodisable sub-option does not disable reads or writes on errordetection. It posts appropriate error messages and sets numerous filesystem error flags.

This behavior is similar to previous versions of VxFS.

● mount -F VxFS -o ioerror=wdisable /mwdisable

The wdisable sub-option disables all write operations to file systemand metadata, but read status is set to degraded.

The mwdisable sub-option disables writes to metadata, and all otherread and write status is degraded.

The wdisable and mwdiable sub-options are used in environmentswhere read errors are more likely to persist than write errors, such aswhen using non-redundant storage.

The mwdisable policy is the default for locally mounted file systems.

Other VxFS Mount Options

There is also a blkclear option that is designed for high securityenvironments. The blkclear mode of operation guarantees thatuninitialized storage (old data) never remains in a file system. This modeof operations can degrade file system performance by as much as10 percent.

Note – Consult the mount_vxfs man pages for further details about otheravailable mount options, some of which relate to VxFS features that arenot licensed by Sun.

Online File System Administration


Online File System Administration

You can defragment, resize, and back up VxFS file systems while they areonline and in use.

Online Defragmentation

The UFS software uses the concept of cylinder groups to limitfragmentation. These are self-contained sections of a file system that arecomposed of inodes, data blocks, and bitmaps, that indicate free inodesand data blocks. Allocation strategies in UFS attempt to place inodes andrelated data blocks near each other. This strategy reduces fragmentation,but does not eliminate it. Over time, the original ordering of free resourcescan be lost. As files are added and removed, gaps between used areas ofthe disk can still occur.

The VxFS software fsadm utility is used to defragment a disk withoutrequiring that the disk be unmounted first. The fsadm utility removesunused space from directories, makes small files contiguous, andconsolidates free blocks for use.

Online Resizing

When UFS file systems are too small or become too large for theirassigned disk space, the following methods are used to correct theproblem:

● Users are moved to new or different file systems.

● Subdirectories are moved to other file systems.

● The file systems are backed up and restored to a different file system.

VxFS with VxVM enables a file system to be expanded or reduced in sizewhile it is being accessed.

Online Backup and Restore

You use the VxFS vxdump utility to dump volume snapshots to tape.Volumes are restored using the vxrestore utility.

Exercise: Configuring VxFS





● Install the VxFS software

● Create a VxFS file system

● Resize a VxFS file system

● Defragment a VxFS file system

● Backup and restore a VxFS file system

● Use VxFS extended mount options

Preparation

VxFS is a separately licensed option. You must install a license key toactivate the software. Ask your instructor for a VxFS temporary licensekey. Record the temporary key.

VxFS license key: __________________________________________________

You can get VxVM, VxFS, and Shared Disk Group temporary licenses at the Sun Business Partners Web siteat http://webhome.ebay/partnersoftware/ .




Answer the following questions about VxFS features and mount options.

1. What is the primary advantage of extent-based space allocation?

a. Faster recovery times after system failures

b. Reduced data fragmentation

c. Increased throughput in write-intensive applications

d. Reduced data loss after system failures

The answer is c.

2. What is the primary advantage of intent logging?

a. Simplified file system administration

b. Enhanced space utilization

c. Improved file system performance

d. Faster recovery time after system failures

The answer is d.

3. Which of the following VxFS mount options is used to increase datasecurity?

a. log

b. delaylog

c. blkclear

d. nodatinlog

e. convosync

The answer is c.

4. Which of the following VxFS mount options should be used withcluster file systems?

a. log

b. delaylog

c. blkclear

d. nodatinlog

The answer is a.



Task 2 – Installing the VxFS Software

To install the VxFS software, complete the following steps:

1. Log in as user root on the system attached to the storage arrays.

2. Change to the VxVM installation package location furnished by yourinstructor.

3. Install the VxFS packages in the order shown and answer yes to allquestions.

# pkgadd -d . VRTSvxfs VRTSfppm VRTSfsdoc VRTSfsman \VRTStep VRTSap

4. Install the VxFS license key furnished by your instructor.

# vxlicinst

VERITAS License Manager vxlicinst utility version3.02.005Copyright (C) VERITAS Software Corp 2002. All Rightsreserved.

Enter your license key : 3JZU-YBPO-CR28-RWWP-P5RO-P12

5. Reboot the VxFS server.

There are no VxFS-specific boot messages.

6. Log in to the VxFS server and verify that the VxFS program files areinstalled.

# ls /opt/VRTSvxfs/sbincp qiostat vxedquotacpio qlogadm vxenableffcladm qlogckd vxfsckdfsadm qlogclustadm vxfsconvertfsapadm qlogdb vxfsstatfscat qlogdetach vxlsinofsckpt_restore qlogdisable vxquotfsckptadm qlogenable vxquotafsclustadm qlogmk vxquotaofffsvoladm qlogprint vxquotaongetext qlogrm vxrepquotals qlogstat vxrestoremv qlogtrace vxtunefsqioadmin setext vxupgradeqiomkfile vxdump



7. Verify that the VxFS technical manuals are installed.

# ls /opt/VRTS/docsvxfs_ag.pdf vxfs_ig.pdf

8. Verify that the VxFS man pages have been added to the/opt/VRTS/man directory.

# man vxquot# man vxdump

9. Verify that the VxFS driver is loaded.

# modinfo | grep vxfs

10. Verify that the special file system programs that are required early inthe boot process are present.

# ls /etc/fs/vxfsmount qlogck system.preinstallqlogattach qlogrec

11. Move to the VxFS administration guide directory.

# cd /opt/VRTS/docs

12. View the contents of the manuals with the Adobe Acrobat Reader(acroread ).


a. Examine Appendix B, “Kernel Messages,” of the VERITAS FileSystem 4.0 Administrator’s Guide.

b. Examine the Description and Action sections associated witheach of the error messages.

Note – Use the Control-F sequence to enable the Adobe Acrobat Findwindow.



Task 3 – Creating a VxFS File System

You use VEA to perform the following file system operations. Study VxFSoptions in the VERITAS File System 4.0 Administrator’s Guide as necessary.

To create a VxFS file system, complete the following steps:

1. Delete all volumes in your disk group and add all of your disks tothe group.

2. Use the VEA GUI to create a mirrored volume as follows:

a. Use all 6 of your disk drives for the volume.

b. Click Concatenated layout and select the Mirrored check box.

c. Set the total number of mirrors at 2.

d. Disable logging.

e. Click Max Size to calculate the maximum size and round-off theresult that is returned.

f. Do not create a file system on the volume.

3. Click your new volume and select File System then New File Systemfrom its pop-up menu.

4. Configure the New File System window as follows:

a. Set the file system type to vxfs.

b. Leave the block size set to the default value (1024).

c. Enter a file system mount point.

d. Click Add to file system table check box, and complete thefollowing steps:

1. Select Mount at boot.

2. Set the fsck pass number to 2.

e. Click OK.

5. In the VEA GUI grid area, complete the following steps:

a. Click your VxFS volume and select Layout View from its pop-up menu.

b. Examine the concatenated mirror volume structure and closethe Layout View window when you have finished.



6. On the VxVM server, complete the following steps:

a. Examine the VEA GUI command log.

b. Observe the volume size and the log size information in theOutput section of the mkfs command.


c. Verify that the largefiles option was specified with the mkfscommand.

Task 4 – Resizing a VxFS File System

A unique feature of VxFS is the ability to decrease the size of an existingfile system that is overly large for its intended application.

To decrease the size of the VxFS file system created previously in thisexercise, complete the following steps.

1. Use the df -kl command to verify the amount of space available inyour new VxFS file system.

# df -klFilesystem kbytes used avail capacity Mounted on..../dev/vx/dsk/dgX/mirvol 26214400 39890 24538611 1 /VXFS

In the preceding example, the file system is approximately 26-Gbytesin size and about 40-Mbytes of space is in use by inode and intentlog space.

2. Use the mount command to verity that your VxFS file system wasmounted using the delaylog , largefiles , andioerror=mwdisable mount options.

3. Verify that you can create a file larger than 2-Gbytes in your filesystem.

# mkfile 2500m /VXFS/file1

4. Delete all test files from your VxFS file system.

5. Click on the VxFS volume in the VEA GUI Object tree, and selectResize Volume from its pop-up menu.



6. Complete the Resize Volume form as follows:

a. Enter a new volume size of 1 Gbyte.

b. Let volume manager decide which disks to use.

c. Click on the OK button to complete the resize operation.

d. Verify the new VxFS file system size by using the df -klcommand.

Task 5 – Defragmenting a VxFS File System

See the fsadm_vxfs man page for further information on the fsadmdefragmentation utility.

Perform the following steps to defragment your VxFS file system:

1. Use the fsadm command to verify the current fragmentation in yourVxFS file system. Substitute the name of your file system.

# fsadm -D /VXFS Directory Fragmentation Report Dirs Total Immed Immeds Dirs to Blocks to Searched Blocks Dirs to Add Reduce Reduce total 2 0 0 2 0 0

The Blocks to Reduce value is a general indicator thatdefragmentation might be of value. Some directories are fragmentedin a way that does not respond to defragmentation.

2. Perform the following command sequence to create somefragmentation in your VxFS file system. Substitute the name of yourfile system.

# cp -r /usr/dt /VXFS (151 Mbytes)# cp -r /usr/appserver /VXFS (75 Mbytes)# rm -r /VXFS/dt# cp -r /usr/iplanet /VXFS (65 Mbytes)# cp -r /usr/share /VXFS (78 Mbytes)# cp -r /usr/dt /VXFS

3. Verify the level of fragmentation and available space in your VxFSfile system.

# fsadm -D /VXFS# df -kl



4. Defragment your VxFS file system. Substitute the name of your filesystem.

# fsadm -de /VXFS

5. Verify your VxFS file system’s fragmentation and available spacehave improved.

# fsadm -D /VXFS# df -kl

Task 6 – Backing Up and Restoring a VxFS FileSystem

To use the Snapshot feature to prepare for a VxFS file system backup,complete the following steps:

1. In the VEA grid area, click your VxFS volume and select Snap thenSnap Start from its pop-up menu.

2. Let volume manager decide what disks to use for the Snapshot, andthen click OK.

3. On the VxVM server, use the vxprint command to verify that theSnapshot structure is complete.

4. In the VEA grid area, click your VxFS volume and select Snap thenSnap Shot from its pop-up menu.

5. Examine the Snap Shot Volume form and complete the followingsteps:

a. Enter a custom volume name or use the default volume name.

b. Click OK.

The Snapshot mirror is detached and used to create theSnapshot volume.

6. In the VEA GUI grid area, click the Snapshot volume and select FileSystem then Mount File System from its pop-up menu.

7. Enter a mount point name in the Mount File System form andclick OK.

8. Verify that the Snapshot volume’s file system is mounted andavailable.

# df -kl



9. Ask your instructor if there is a tape unit available to perform thefollowing dump and restore procedure. Substitute the name of yourSnapshot file system and VxFS file system.

# vxdump -0f /dev/rmt/0 /SNAP# cd /VXFS# rm -r appserver dt iplanet share# lslost+found# vxrestore -f /dev/rmt/0

Note – The vxdump and vxrestore utilities are functionally the same asthe Solaris OS ufsdump and ufsrestore utilities.

Task 7 – Using VxFS Extended Mount Options

To investigate the effects of selected VxFS file system mount options,complete the following steps:

1. If you have not done so already, remove all of the files from yourVxFS file system.

# cd /VXFS# rm -r appserver dt iplanet share# lslost+found# cd /

2. Create a test file in your VxFS file system and record the amount oftime it takes to complete the operation.

# ptime mkfile 100m /VXFS/file1real 7.348user 0.076sys 3.459# rm /VXFS/file1

3. Unmount your VxFS file system and mount it again using theblkclear security option.

# umount /VXFS# mount -F vxfs -o blkclear /VXFS



4. Use the mount command to verify that the VxFS file system’s mountoptions are now read , write , setuid , blkclear , delaylog ,largefiles , and ioerror=mwdisable .

5. Run a performance test on your VxFS file system again.

# ptime mkfile 100m /VXFS/file1real 1:10.047user 0.078sys 5.145# rm /VXFS/file1

6. Unmount your VxFS file system and mount it again with no specialoptions.

# mount -F vxfs /VXFS

Exercise Summary


Exercise Summary

?!



● Experiences


● Interpretations


● Conclusions


● Applications



Module 8

VERITAS Volume Manager PerformanceManagement

Objectives


● Describe performance improvement techniques

● Use the vxstat and vxtrace performance analysis tools

● Describe RAID-5 write performance characteristics

Performance Improvement Techniques



Periodic reassessment of volume performance is necessary on any system.The access to any data structure can degrade over time to the point ofpoor performance.

Data Assignment Strategies

When deciding where to locate a file system, you usually attempt tobalance I/O load among available disk drives. The success of this processis limited by the difficulty of anticipating future usage patterns. Figure 8-1shows how data assignment mistakes can lead to a performance problem.

Figure 8-1 Data Assignment Bottleneck

In general, do not place file systems that have heavy I/O loading on thesame disk drives. Separate them into different storage arrays on variouscontrollers.

Also, the placement of logs can be critical to performance. This isespecially true of RAID-5 logs.

Heavy-UseVolume

Array

Heavy-UseVolume

Heavy-UseVolume

Heavy-UseVolume

Low-UseVolume

c3

Array

Low-UseVolume

Low-UseVolume

Low-UseVolume

Controller c4Controller


VERITAS Volume Manager Performance Management 8-3Copyright 2004 Sun Microsystems, Inc. All Rights Reserved. Sun Services, Revision D

The following solutions can be used to resolve the problem demonstratedin Figure 8-1 on page 8-2:

● Swap some of the heavy-use volumes with the low-use volumes.

● Move one of the heavy-use subdisks to a different location.

Note – Swapping volume locations is probably a better solution because iteliminates having two heavily used volumes on a single disk drive.

Another type of performance problem can occur when a log plex is placedon the same disk drive as its associated data plex. In the case of RAID-5logs, you should always consider that the data written to all RAID-5columns must also be written to the log.

In a six-column RAID-5 volume, this configuration could increase the I/Orate of the log disk drive by as much as 600 percent.

As shown in Figure 8-2, leaving unused space on all disk drives ensuresthat you always have alternate locations to which to move logs. This iswhy the Maxsize calculations of the vxassist command or VEA mightnot be wise to use.

Figure 8-2 RAID-5 Log Placement

The log placement shown in Figure 8-2 would not work well if bothvolumes were heavily accessed. The configuration would work best if atleast one of the volumes has low write activity.

Column 0

Vol02_log

Column 1

Log Space

Column 2

Log Space

Column 3

Log Space

Volume 01

Column 0

Log Space

Column 1

Log Space

Column 2

Log Space

Column 3

Vol01_log

Volume 02



Volume Structure Strategies

Sometimes, performance problems are not due to physical volumelocations and can be greatly reduced by reconfiguring the volumestructures. In many cases, this can be accomplished using the VxVMVolume Relayout feature.

Striping

Striping distributes data across multiple devices to improve accessperformance. Striping improves performance for both read and writeoperations.

If you can identify the most heavily accessed volumes (for file systems ordatabase tables) during the initial design stages, then you can eliminateperformance bottlenecks by striping them across several devices. Theexample in Figure 8-3 shows a volume (Hot_Vol ) that was identified asbeing a data-access bottleneck. The volume is striped across four diskdrives, leaving the remainder of those four disk drives free for use by lessheavily used volumes.

Figure 8-3 Using Striping to Improve Performance

Hot_Vol

Light Use

Stripe 0Hot_Vol

Light Use

Stripe 1Hot_Vol

Light Use

Light Use

Stripe 2Hot_Vol

Light Use

Stripe 3



Mirroring

Mirroring stores multiple copies of data on a system. Mirroring isprimarily used to protect against data loss due to physical media failure.It also improves the chance of data recovery in the event of a systemcrash.

In some cases, mirroring can also be used to improve systemperformance. Mirroring heavily accessed data not only protects the datafrom loss due to disk drive failures, but it can also improve I/Operformance. Unlike striping, however, performance gained through theuse of mirroring depends on the read/write ratio of the disk driveaccesses. If the system workload is primarily write-intensive (for example,greater than 30 percent writes), then mirroring can result in somewhatreduced performance.

RAID 0+1

RAID 0+1 is also referred to as a mirror-stripe configuration. When usedtogether, mirroring and striping provide the advantage of both spreadingthe data across multiple disk drives and providing redundancy of data.

Layered Volumes

Striped mirror (RAID 1+0) volume configurations can improveperformance significantly. Additionally, striped mirror configurations cantolerate a higher percentage of disk drive failures and recovery times aregreatly reduced.

RAID 5

RAID 5 provides the advantage of read performance that is similar to thatof striping, while also providing data protection using a distributed parityscheme. The disadvantage of RAID 5 is relatively slow write performance.

RAID 5 is not generally seen as a performance improvement mechanismexcept in cases of read-intensive applications. However, RAID-5 volumestructures can be adjusted to dramatically improve performance for somewrite-intensive applications.



Read Policy Strategies

To provide optimal read performance for different types of mirroredvolumes, you can select the following read policies:

● The round-robin read policy (round )

Read requests to a mirrored volume are satisfied in a round-robinmanner from all plexes in the volume. You select this policy whenthere is no significant performance advantage by using anyparticular mirror.

● The preferred-plex read policy (prefer )

Read requests to a mirrored volume are satisfied from one specificplex (presumably the plex with the highest performance), unless thatplex has failed.

● The default read policy (select )

The appropriate read policy is automatically selected for theconfiguration. For example, preferred-plex is selected when there isonly one striped plex associated with the volume, and round-robin isselected in most other cases.

In the example in Figure 8-4, you set the read policy of the volume labeledHot_Vol to prefer for the striped plex labeled Plex 1. In this way, readrequests are directed to the striped plex that has the best performancecharacteristics.

Figure 8-4 Preferred-Plex Read Policy

You can change volume read policies either from the command line orusing the VEA GUI.

Hot_Vol

Disk 1

Stripe 0Hot_Vol

Disk 2

Stripe 1Hot_Vol

Disk 3

Stripe 2Hot_Vol

Disk 4

Plex 1Plex 2

Plex 1Plex 1



In the VEA GUI, highlight the volume in the grid area, and click Props inthe toolbar. In the General properties tab, you can choose one of followingfixed read policy options:

● Based on Layout

● Round Robin

● Prefer (preconfigured)

From the command line, use the vxvol command as follows:

# vxvol -g dgY rdpol prefer vol01 vol01-03

Hardware Configuration Strategies

For increased performance, availability, or both, striping and mirroringshould be done across system boards, controllers, and targets. You cangain the highest level of performance or reliability by striping ormirroring across system boards, as shown in Figure 8-5.

Figure 8-5 High Availability and Performance Cabling

System Board

Host System

t1

t2

t3

t1

t2

t3

Controller c3

System Board Controller

Perferred stripe ormirror configuration

Array Array

c4

Using Performance Analysis Tools



The VxVM software continuously gathers performance statistics about alldevices and objects under its control. The types of information include:

● A count of operations

● The number of blocks transferred

● The average operation time (which reflects the total time through theVxVM software and is not suitable for comparison against otherstatistics programs)

The statistics include the number of reads, writes, atomic copies, verifiedreads, verified writes, plex reads, and plex writes for each volume. As aresult, one write to a two-plex volume results in at least five operations:one for each plex, one for each subdisk, and one for the volume.

VxVM also maintains other statistical data, such as information aboutread and write failures. The statistics are continuously gathered startingwith the system boot operation. The statistics are reset prior to a testingoperation.

Gathering Volume Performance Statistics

The vxstat command displays statistical information about differenttypes of VxVM physical and logical objects. You can use the followingoptions to control the output:

● vxstat -g disk_group

Displays volume statistics for the specified disk group.

● vxstat -g disk_group vol01

Displays statistics for the specified volume.

● vxstat -g disk_group -d

Displays disk drive level statistics for the specified disk group.

● vxstat -g disk_group -d disk01

Displays statistics for the specified disk drive.

After performance data has been gathered, you can analyze it todetermine, and to optimize, your system configuration for efficient use ofsystem resources.



A volume or disk drive with elevated read or write access times is notnecessarily a problem. If the slow response is not causing any apparentproblems for users or applications, then there might not be anything thatneeds fixing.

Preparing for Analysis

Before obtaining statistics, clear (reset) all existing statistics by using thevxstat -r command. Clearing statistics eliminates any differencesbetween volume or disk drives due to volumes being created. It alsoremoves statistics that are not typically of interest, such as informationabout booting.

After clearing the statistics, allow the system to run during typical systemactivity. When monitoring a system that is used for multiple purposes, trynot to exercise any one application more than it would usually beexercised.

It can also be beneficial to take periodic snapshots of the volume statisticsto help identify the source of irregular system-load problems.

A single volume with excessive I/O rates can cause performancedegradation on other volumes associated with the same disk drives.

Analyzing Volume Statistics

Use the vxstat command as follows to help identify volumes with anunusually large number of operations or excessive read or write times:

# vxstat -g benchOPERATIONS BLOCKS AVG TIME(ms)

TYP NAME READ WRITE READ WRITE READ WRITEvol acct 473 11 57252 44 4.0 20.9vol brch 23 11 92 44 33.0 20.0vol ctrl 92773 121312064799783 78102463 276.9 457.2vol hist1 23 11 92 44 97.0 24.5vol hist2 23 11 92 44 54.8 22.7vol hist3 23 11 92 44 103.5 25.5vol log1 9 27217 9 409716 16.7 21.1vol log2 7 8830 7 159769 15.7 24.3vol rb1 123 13 492 52 30.7 83.1vol rb2 23 11 92 44 149.1 25.5vol sys 26933 86156 17768834463215.0310.7vol t11r 23 11 92 44 39.6 24.5



Analyzing Disk Drive Statistics

You can also use the vxstat command to summarize operationsaccording to physical disk drives. For example:

# vxstat -g bench -dOPERATIONS BLOCKS AVG TIME(ms)

TYP NAME READ WRITE READ WRITE READ WRITEdm c3t98d0 14330 140370 120348 986785 15.4 185.6dm c3t100d0 13881 140370 117971 986785 15.4 187.7dm c3t113d0 0 0 0 0 0.0 0.0dm c3t115d0 0 0 0 0 0.0 0.0dm c3t117d0 0 0 0 0 0.0 0.0



Gathering Application Performance Statistics

The vxtrace command displays detailed trace information about errorsor I/O operations. This level of detail is generally not necessary, but isincluded here for completeness. You can use the following options tocontrol the display:

● vxtrace -o disk

Traces all physical disk drive I/O operations.

● vxtrace -o disk c3t98d0

Traces all I/O operations to the physical disk drive c3t98d0 .

● vxtrace hist2

Traces all virtual device I/O operations associated with the volumehist2 .

● vxtrace -o dev hist2

Traces virtual disk device I/O to the device associated with volumehist2 .

After identifying a volume that has an I/O-related problem, you can usethe vxtrace command to determine which system process is responsiblefor the I/O requests. The volume of interest in this example is namedctrl .

# vxtrace -o dev ctrl40122 START write vdev ctrl block 16 len 4 concurrency 1 pid 1068940122 END write vdev ctrl op 40122 block 16 len 4 time 140123 START write vdev ctrl block 16 len 4 concurrency 1 pid 1068940123 END write vdev ctrl op 40123 block 16 len 4 time 240124 START write vdev ctrl block 16 len 4 concurrency 1 pid 1068940124 END write vdev ctrl op 40124 block 16 len 4 time 440125 START write vdev ctrl block 16 len 4 concurrency 1 pid 1068940125 END write vdev ctrl op 40125 block 16 len 4 time 0^C

# ps -ef |grep 10689oracle 10689 1 0 20:05:21 ? 0:03 ora_ckpt_bench

RAID-5 Write Performance



The RAID-5 write process is controlled according to how much data isgoing to be written into a full stripe width. The optimum writeperformance is obtained when full stripes are written.

Read-Modify-Write Operations

When less than 50 percent of the data disk drives are undergoing writeoperations in a single I/O, the read-modify-write sequence is used.

This is the default operation for RAID-5 volumes.

As shown in Figure 8-6, the read-modify-write sequence involves severalsteps, including:

1. The stripes to be modified are read into a buffer.

2. The parity information is read into a buffer.

3. Exclusive OR (XOR) operations are performed.

4. The new data and parity are written in a single write operation.

Figure 8-6 Read-Modify-Write Operation

StripeUnit 0

StripeUnit 1

StripeUnit 2

StripeUnit 3

StripeUnit 4

Parity

1 0 1 1 0 1

XOR

New Data

1 1

1 1 0



At least three I/O operations are necessary in the example shown inFigure 8-6 on page 8-12. Also, additional XOR calculations arenecessary to account for the data in stripe units 2, 3, and 4 that wasnot read.

Generally, the read-modify-write method is the least efficient way ofwriting to RAID-5 structures.

Reconstruct-Write Operations

If more than 50 percent of the data stripe will be modified, use thereconstruct-write method.

As shown in Figure 8-7, the reconstruct-write method involves differentsteps, including:

1. Only unaffected data is read into a buffer.

2. XOR is applied to the new data and the unaffected data.

3. New parity and data are written in a single write operation.

Figure 8-7 Reconstruct-Write Operation

Only two I/O operations are necessary in the example shown inFigure 8-7.

Generally, the reconstruct-write operation is more efficient than theread-modify-write sequence.

StripeUnit 0

StripeUnit 1

StripeUnit 2

StripeUnit 3

StripeUnit 4

Parity

1 0 1 1

0 1 1 0

0 1 1 0

0 1

New Data

XOR

0



Full-Stripe Write Operations

When large write operations that cover an entire data stripe are issued,the read-modify-write and reconstruct-write procedures are bypassed infavor of a full-stripe-write operation. A full-stripe write operation is fasterthan the other RAID-5 write procedures because it does not require anyread operations. As shown in Figure 8-8, a full-stripe write operationconsists of the following steps:

1. XOR is applied to the new data to produce new parity.

2. The new data and parity are written in a single write operation.

Figure 8-8 Full-Stripe-Write Operation

Only a single write operation is necessary in the example shown inFigure 8-8.

Note – In some cases, it is beneficial to reduce the number of RAID-5columns to force more full-stripe write operations. This reduction canenhance overall write performance for some applications that userandom-length write operations.

StripeUnit 0

StripeUnit 1

StripeUnit 2

StripeUnit 3

StripeUnit 4

Parity

1 0 1 1

0 1 1 0

1 00 0 1

0

1

1

New Data

XOR

0

Exercise: Demonstrating Performance Differences





● Perform a RAID-5 write performance test

● Perform a striped volume write performance test

Preparation

Unless your instructor says otherwise, the instructor conducts theperformance demonstrations on the VxVM server system.

First complete the questions in ‘‘Task 1 – Reviewing Key Lecture Points’’on page 8-16.




Answer the following questions about performance planning andmonitoring.

1. What are the two most important areas of interest when researchingvolume placement?

a. Volume structure

b. Volume activity levels

c. Volume size

d. Volume log locations


2. Why is it important to leave a small amount of free space on eachdisk drive?

a. To relocate the private region

b. To adjust cylinder boundaries

c. To relocate log space

d. To create snapshot volumes

The answer is c.

3. Which RAID structure usually has the best write performance?

a. RAID 5

b. Mirrored

c. Concatenated

d. Striped

The answer is d.

4. When is the preferred-plex read policy used?

a. When a RAID-5 stripe is on a faster disk drive

b. When one mirror has better read performance

c. When no mirror has a performance advantage

The answer is b.



5. Which of the following commands sets read policies?

a. vxedit

b. vxvol

c. vxdisk

The answer is b.

6. What are the advantages of mirroring across controllers?

a. Availability

b. Performance

c. Administration

d. Tuning

The answers are a and b.

7. What is the primary VxVM command for displaying performancestatistics?

a. vxtrace

b. vxtask

c. vxstat

The answer is c.

8. What should be done prior to starting a VxVM volume performancetest?

a. Clear the VxVM statistics

b. Empty the system logs

c. Initialize the syslogd daemon

The answer is a.

9. What data pattern gives the best RAID-5 write performance?

a. Less than 30 percent stripe-width writes

b. Near 80 percent stripe-width writes

c. Full stripe-width writes

The answer is c.



Task 2 – Performing a RAID-5 Write Performance Test

The following demonstration is performed by the instructor unless youhave been given other directions. The performance test uses the ddcommand to write directly to an unmounted raw VxVM volume.

This demonstration is designed to be performed on a 30-Mbyte RAID-55+0 volume named r5vol that is in a disk group named dgX, and a5-column striped volume named stdemo that is also 30 Mbytes in size.

There is a lab demonstration file named r5demo.sh that is used toperform a series of performance tests. Alternately, you can enter thecommand manually to perform the performance test.

Create the Demonstration Volume and Test Data




a. Create a disk group named dgX containing a minimum of 5disks.

b. Use the vxdiskadm utility to ensure the disk type is sliced .

3. Create a 5-column, no log, RAID-5 volume with 30 Mbytes.

# vxassist -g dgX make r5demo 30m \layout=raid5,nolog \dgX01 dgX02 dgX03 dgX04 dgX05

Note – The default stripe unit size is 32 blocks (16 Kbytes).

4. Check for space, and then create a 20-Mbyte test file.

# df -kl /# mkfile 20m /testfile



Enable Classroom-Wide Display


Note – In this task, all steps are directed toward the instructor. The wordsyou and your refer to the instructor, not the students.

The instructor should complete the following steps:

1. Verify that the students remotely log in to the VxVM server.

2. Verify that each workgroup gives you their terminal identifier bytyping the tty command. Record the identifiers here:

__________ __________ __________

__________ __________ __________

3. Direct the output of your window to all of the students terminals.

For example:

# script /dev/null | tee /dev/pts/2 | tee /dev/pts/3



Perform a RAID 5 Volume Demonstration


1. Run the r5demo.sh demonstration test file on the VxVM server.

You can also manually repeat the following command sequence,each time incrementing the block size of the dd command in thefollowing sequence: 17, 33, 49, 65, 80, 64.

# vxstat -g dgX -r r5demo# /usr/proc/bin/ptime dd if=/testfile \of=/dev/vx/rdsk/dgX/r5demo bs=17k# vxstat -g dgX -f MWF -v r5demo

Note – As you move past the 50-percent stripe write into full-stripe write,the I/O should move through the three write categories (M, W, and F). Thefull stripe width is 16 Kbytes times 5 columns, which is equal to80 Kbytes.

Tell the class that a full-stripe write is only four of the five columns. One column is always used for parity. Thisis evident in the last test loop using 64-Kbyte transfer size.

The vxstat -f MWF options have the following meanings:

M Read-modify-write statistics

W Reconstruct write statistics

F Full-stripe write statistics

2. Kill the tee processes to end the demonstration. Do not kill thescript process, it terminates after the tee processes are gone.



Task 3 – Performing a Striped Volume WritePerformance Test

If time permits, create a second volume in a striped format with5 columns, and run the performance test again. Use the same test file data.


1. Use the vxassist command to create a 5-column striped volume.

# vxassist -g dgX make stdemo 30m \layout=striped \dgX01 dgX02 dgX03 dgX04 dgX05

2. Run the r5demo.sh script again, but substitute the name and path ofthe new striped volume, stdemo .

Enter disk group name (default: dgX)Enter the name of the demo volume (default: r5demo) stdemoEnter the raw path to the demo volume (default: /dev/vx/rdsk/dgX/r5demo)/dev/vx/rdsk/dgX/stdemoEnter data file location (default: /testfile)

Note – The vxstat command does not display any statistics for thestriped volume, but the ptime results are informative.

Exercise Summary


Exercise Summary

?!


Manage the discussion here based on the time allowed for this module, which was given in the “About ThisCourse” module. If you find you do not have time to spend on discussion, highlight just the key conceptsstudents should have learned from the lab exercise.

● Experiences

Ask students what their overall experiences with this exercise have been. Go over any trouble spots orespecially confusing areas at this time.

● Interpretations


● Conclusions


● Applications


Documents

Veritas Volume Manager 1