HP 3PAR Cluster Extension Software Administrator Guidekmcs-service.austin.hp.com/km-ext/kmcsdirect/emr_na-c03393844-1.pdf · Configurationtool(clx3parconfig.exe)forWindows ... Thefailoverpackageisthemostcommontypeofpackage.Itrunsononenodeatatime.Ifa

HP 3PAR Cluster Extension SoftwareAdministrator Guide

AbstractThis guide contains detailed instructions for configuring and troubleshooting HP 3PAR Cluster Extension Software in Windowsand Linux environments. The intended audience has independent knowledge of related software and of the HP 3PAR storagesystem and its software.

HP Part Number: 5697-2047Published: June 2012Edition: 2

© Copyright 2012 Hewlett-Packard Development Company, L.P.

Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, CommercialComputer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government undervendor's standard commercial license.

The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the expresswarranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shallnot be liable for technical or editorial errors or omissions contained herein.

Acknowledgments

Intel®, Itanium®, Pentium®, Intel Inside®, and the Intel Inside logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries

in the United States and other countries.

Microsoft®, Windows®, Windows® XP, and Windows NT® are U.S. registered trademarks of Microsoft Corporation.

Adobe® and Acrobat® are trademarks of Adobe Systems Incorporated.

Oracle and Java are registered trademarks of Oracle and/or its affiliates.

UNIX® is a registered trademark of The Open Group.

Contents1 HP 3PAR Cluster Extension features...............................................................7

Integration into cluster software (Graphical user interface)..............................................................7CLI for easy integration..............................................................................................................7Disaster tolerance through geographical dispersion.......................................................................7Automated redirection of mirrored disks.......................................................................................8Metropolitan distance support....................................................................................................8Fully Automatic Failover and Failback..........................................................................................9No server reboot......................................................................................................................9No single point of failure...........................................................................................................9Synchronous mode support........................................................................................................9Storage system configurations.....................................................................................................9

1-to-1 configuration.............................................................................................................10N-to-1 and 1-to-N configurations..........................................................................................11

Processes and components.......................................................................................................11HP 3PAR Remote Copy.......................................................................................................11

Synchronous replication.................................................................................................13User configuration file.........................................................................................................13Force Flag.........................................................................................................................13

Planning for HP 3PAR Cluster Extension.....................................................................................13Cluster setup considerations................................................................................................13

Windows clustering.......................................................................................................13Windows Server 2008/2008 R2...............................................................................13Windows Server 2003..............................................................................................13MNS or Node Majority quorum in Windows Clustering................................................14Majority Node Set with File Share Witness..................................................................14

SLE HA cluster setup considerations......................................................................................15Quorum.......................................................................................................................15STONITH.....................................................................................................................15Networking in an SLE HA cluster.....................................................................................15Resource constraints......................................................................................................15Resource operation attribute...........................................................................................16HP 3PAR Cluster Extension resource dependency...............................................................16Failover order...............................................................................................................16Failback option.............................................................................................................16Migration-threshold.......................................................................................................16Disk monitoring.............................................................................................................17

RHCS cluster setup considerations........................................................................................17Quorum.......................................................................................................................17Fencing........................................................................................................................17Qdisk configuration.......................................................................................................17Failover domains...........................................................................................................17Failback policy.............................................................................................................18Recovery policy............................................................................................................18Service hierarchical structure and resource dependency.....................................................18Disk monitoring.............................................................................................................18

HP Serviceguard for Linux cluster setup considerations............................................................19Quorum.......................................................................................................................19Using Generic Resources to Monitor Volume Groups..........................................................19Disk monitoring.............................................................................................................19Cluster timeout considerations.........................................................................................19failover packages..........................................................................................................20

Contents 3

failover policy...............................................................................................................20failback policy..............................................................................................................20

Configuration tool (clx3parconfig.exe) for Windows...............................................................20HP 3PAR InForm Management Console or HP 3PAR InForm CLI ...............................................20IP network considerations....................................................................................................20SAN fabric considerations..................................................................................................20

2 Configuring HP 3PAR Cluster Extension for Windows....................................21Integrating HP 3PAR Cluster Extension with MSCS.......................................................................21Configuring HP 3PAR Cluster Extension.....................................................................................21

Starting the HP 3PAR Configuration Tool...............................................................................21Defining the HP 3PAR configuration information using the GUI.................................................22Defining the HP 3PAR configuration information using the CLI..................................................24Importing and exporting configuration information.................................................................24

Exporting configuration settings using the GUI..................................................................25Exporting configuration settings using the CLI....................................................................25Importing configuration settings using the GUI..................................................................25Importing configuration settings using the CLI....................................................................25

Adding a HP 3PAR Cluster Extension resource............................................................................25Adding a resource using the Cluster Administrator GUI (Windows Server 2003)........................26Adding HP 3PAR Cluster Extension resource using the Failover Cluster Management GUI (WindowsServer 2008/2008 R2)......................................................................................................26Adding HP 3PAR Cluster Extension resource using the Microsoft CLI cluster commands................26

Example......................................................................................................................26Configuring more than one resource in a service or application...............................................27

Changing a HP 3PAR Cluster Extension resource name................................................................27Changing a HP 3PAR Cluster Extension resource name (Windows Server 2003)........................27Changing a HP 3PAR Cluster Extension resource name (Windows Server 2008/2008 R2).........27

Configuring HP 3PAR Cluster Extension resources........................................................................27Setting Microsoft cluster-specific resource and service or application properties...............................28Setting HP 3PAR Cluster Extension-specific resource properties......................................................31

Setting HP 3PAR Cluster Extension resource properties using the GUI........................................31Using Cluster Administrator to set resource properties (Windows Server 2003)......................32Using Failover Cluster Management to set resource properties (Windows Server 2008/2008R2)..............................................................................................................................32Using the resource configuration tool to set resource properties (Server Core and Hyper-VServer).........................................................................................................................33Configuring cluster node data center assignments..............................................................33Configuring HP 3PAR storage system...............................................................................34Selecting a volume group...............................................................................................34Changing Remote Copy volume group settings..................................................................35Configuring takeover actions..........................................................................................36

Setting HP 3PAR Cluster Extension resource properties using the MMC ....................................36Setting HP 3PAR Cluster Extension resource properties using the CLI.........................................36Setting HP 3PAR Cluster Extension properties using a UCF......................................................37

Adding dependencies on a HP 3PAR Cluster Extension resource...................................................37Adding dependencies using Cluster Administrator (Windows Server 2003)...............................37Adding dependencies using Failover Cluster Management (Windows Server 2008/2008 R2).....38Adding dependencies using the CLI.....................................................................................39

Disaster-tolerant configuration example using a file share.............................................................39Managing HP 3PAR Cluster Extension resources.........................................................................41

Bringing a resource online..................................................................................................41Taking a resource offline.....................................................................................................41Deleting a resource............................................................................................................41

Using Hyper-V Live Migration with HP 3PAR Cluster Extension.......................................................42

4 Contents

Timing considerations for Windows Clustering............................................................................42Bouncing service or application................................................................................................43Creating array password file....................................................................................................43Administration........................................................................................................................43Remote management of HP 3PAR Cluster Extension resources in a cluster.......................................44System resources....................................................................................................................44Logs......................................................................................................................................44Hyper-V Live Migration log entries.............................................................................................45

3 Configuring HP 3PAR Cluster Extension for Linux...........................................46HP 3PAR Cluster Extension for Linux: Sample configuration...........................................................46Configuring HP 3PAR Cluster Extension.....................................................................................48

Defining HP 3PAR configuration information using the CLI.......................................................48Importing and exporting configuration information.................................................................48

Importing configuration settings using the CLI....................................................................48Exporting configuration settings using the CLI....................................................................48

Configuring HP 3PAR Cluster Extension with RHCS......................................................................48Configuration overview.......................................................................................................49Creating an RHCS HP 3PAR Cluster Extension shared resource................................................49

Using Conga to create a shared resource.........................................................................49Using system-config-cluster to create a shared resource.......................................................49

Creating an RHCS service using the HP 3PAR Cluster Extension shared resource........................50Configuration overview..................................................................................................50Using Conga to create a service.....................................................................................50Using system-config-cluster to create a service...................................................................52Creating the HP 3PAR Cluster Extension configuration file...................................................53Testing the service configuration......................................................................................54

Managing HP 3PAR Cluster Extension services (RHCS)............................................................55Starting an RHCS service...............................................................................................55Stopping or disabling an RHCS service............................................................................55

Configuring HP 3PAR Cluster Extension with SLE HA...................................................................55Configuration overview.......................................................................................................55Creating and configuring a HP 3PAR Cluster Extension resource..............................................55

Creating the HP 3PAR Cluster Extension configuration file...................................................56Creating a HP 3PAR Cluster Extension resource for Pacemaker............................................56Testing the configuration................................................................................................58

Managing HP 3PAR Cluster Extension services (SLE HA)..........................................................58Configuring HP 3PAR Cluster Extension with HP Serviceguard on Linux..........................................58

Configuring HP 3PAR Cluster Extension.................................................................................59Creating a new HP Serviceguard package and adding HP 3PAR Cluster Extension (CLI).........59Adding HP 3PAR Cluster Extension to an existing HP Serviceguard package (CLI)..................61

Starting an HP Serviceguard package..................................................................................61Halting an HP Serviceguard for Linux package......................................................................62Removing HP 3PAR Cluster Extension from an HP Serviceguard package...................................62

Removing HP 3PAR Cluster Extension from an HP Serviceguard package (CLI).......................62Rescanning multipath devices...................................................................................................62

Configuring the rescan script...............................................................................................63Finding the user-friendly name of a multipath device...............................................................63

Timing considerations..............................................................................................................644 User configuration file and HP 3PAR Cluster Extension objects........................65

User configuration file.............................................................................................................65File structure......................................................................................................................65Specifying object values.....................................................................................................66

COMMON objects.................................................................................................................67APPLICATION objects..............................................................................................................67

Contents 5

Sample configuration file.........................................................................................................725 CLI commands and utilities........................................................................73

CLI commands.......................................................................................................................73clx3PARrun.......................................................................................................................73

Configuring the CLI.................................................................................................................74Creating the HP 3PAR Remote Copy environment...................................................................74Configuring the HP 3PAR storage system...............................................................................74Installing HP 3PAR InForm Command Line Interface (CLI).........................................................75Creating and configuring the user configuration file................................................................75Timing considerations.........................................................................................................75Restrictions for customized implementations...........................................................................76

Pre-execution and post-execution programs................................................................................76Pre-execution return codes...................................................................................................77Post-execution return codes..................................................................................................77

6 Troubleshooting........................................................................................79Logs......................................................................................................................................79

Windows logs...................................................................................................................79Linux logs.........................................................................................................................79

Log facility.............................................................................................................................80HP 3PAR Cluster Extension logs for Windows.............................................................................80HP 3PAR Cluster Extension logs for Linux....................................................................................81HP 3PAR storage system log or sysmgr log.................................................................................83Error return codes...................................................................................................................83Log files.................................................................................................................................84Start errors.............................................................................................................................84Failover error handling............................................................................................................85Failing physical disk resources during online attempt of the resource group....................................85Cannot connect to HP 3PAR storage system................................................................................85Resource disk failure when HP 3PAR Cluster Extension is online....................................................86Cluster Extension and HP 3PAR Command Line (CLI) Install/Upgrade............................................86Change of HP 3PAR storage system IP or password file................................................................86MNS needs its own resource monitor........................................................................................86Host persona settings..............................................................................................................86no_fail_wrt_on_err settings.......................................................................................................87Remote Copy and 3PAR Virtual Domains...................................................................................87Promote issue.........................................................................................................................87Cluster Extension Autopass troubleshooting................................................................................87The FC link is down (RHCS).....................................................................................................88A storage replication link is down (RHCS)..................................................................................88A data center is down (SLE HA and RHCS)................................................................................88HP 3PAR Target arrays not configured with Remote Copy Links.....................................................89

7 Support and other resources......................................................................90Contacting HP........................................................................................................................90Related information.................................................................................................................90

Whitepapers.....................................................................................................................90HP websites......................................................................................................................90

Typographic conventions.........................................................................................................91Glossary....................................................................................................92Index.........................................................................................................93

6 Contents

1 HP 3PAR Cluster Extension featuresHP 3PAR Cluster Extension offers protection against application downtime due to a fault, failure,or site disaster by extending a local cluster between data centers over metropolitan distance. HP3PAR Cluster Extension reinstates critical applications at a remote site within minutes of an adverseevent, integrating your open-system clustering software and HP 3PAR Remote Copy to automatefailover and failback between sites. This dual integration enables the cluster software to verify thestatus of the storage and the server cluster. The cluster software can then make correct failover andfailback decisions, thus minimizing downtime and accelerating recovery.

Integration into cluster software (Graphical user interface)HP 3PAR Cluster Extension Software provides tight integration with the cluster software and ismanaged as a resource of the clustered application service (like a disk or IP address).HP 3PAR Cluster Extension integrates with the following cluster software products:• Microsoft Cluster Service (MSCS)

• Microsoft Failover Cluster (MSFC)

• SUSE Linux Enterprise High Availability Extension (SLE HA)

• Red Hat Cluster Suite (RHCS)

• HP Serviceguard for Linux (Only CLI based configuration is supported by CLX)For supported cluster software versions, see the HP SPOCK website: http://www.hp.com/storage/spock.HP 3PAR Cluster Extension for Windows can be configured using the Cluster Administrator(Windows Server 2003) or Failover Cluster Management (Windows Server 2008 /2008 R2) GUI.HP 3PAR Cluster Extension offers full integration into the GUI so that you can easily set and changeresource values.

CLI for easy integrationHP 3PAR Cluster Extension provides a CLI to enable disaster tolerant environments if no clustersoftware is available for your operating system or if you want to integrate HP 3PAR Remote Copywith your choice of cluster software. This feature is useful if you use custom software to migrateapplication services from one system to another or if you want HP 3PAR Cluster Extension to checkthe Remote Copy volume group member states to ensure that you can automatically start yourapplication service on the local storage system. For detailed information, see “CLI commands andutilities” (page 73).

Disaster tolerance through geographical dispersionUsing two or more storage systems, HP 3PAR Remote Copy feature copies data to a remote datacenter. Cluster solutions using HP 3PAR Remote Copy disk mirroring are called metropolitan clustersor geographically dispersed clusters. In this arrangement, a server is a member of the same clusterdispersed over two or more sites. In such clusters the server is relieved from writing any I/O requestto the disk more than one time because the storage system controls the replication process (seeFigure 1 (page 8)).

Integration into cluster software (Graphical user interface) 7

http://www.hp.com/storage/ spock

http://www.hp.com/storage/ spock

Figure 1 Physical replication using HP 3PAR Remote Copy

HP 3PAR Remote Copy-mirrored disks have a read/write-enabled primary (local) disk and aread-only secondary (remote) disk. Current cluster software products cannot distinguish betweenread-only and write-enabled disks, and cannot enable disk access if the disk is not write-enabledduring the server boot process.With HP 3PAR Cluster Extension, the consistency and concurrency of the data can be checkedwhen the resource comes online in the remote data center. The capability for restoring theapplication service after the failure of the server, the array, or the datacenter is called disastertolerance.

Automated redirection of mirrored disksStorage systems with HP 3PAR Remote copy automatically redirect the mirroring destination. Thismeans that HP 3PAR Remote Copy almost instantaneously swaps the primary/secondary relationshipof Remote Copy volume group members if the application must access the destination disk (virtualvolume). This feature ensures that the disks are always accessible when failover to a remote datacenter occurs.

Metropolitan distance supportDetermined by the distance and latency requirements for the HP 3PAR Remote Copy Software andcluster server.

8 HP 3PAR Cluster Extension features

Fully Automatic Failover and FailbackAutomated failover and failback reduces the complexity involved in a disaster recovery situation.It is protection against the risk of downtime, whether planned or unplanned.

No server rebootDisks on the server on both the primary and secondary sites are recognized during the initial systemboot in a HP 3PAR Cluster Extension environment; therefore LUN presentation and LUN mappingchanges are not necessary during failover or failback - for a truly hand-free disaster tolerant solution.

No single point of failureSupreme redundancy: identical configuration of established SAN infrastructure redundancies areimplemented on remote site.

Synchronous mode supportHP 3PAR Remote Copy software offers synchronous data transfer mode between storage systemsand host. Synchronous data transfers offer the highest levels of data protection. With synchronousoperations, both the primary and secondary copies are identical and concurrent at all times.Synchronous mirroring ensures that data copies are identical, to prevent critical data loss in theevent of a failure or disaster. In this mode, data is written simultaneously to the mirrored cache ofthe primary storage system and the secondary storage system, in real time, before the applicationI/O is completed, thus ensuring the highest possible data consistency. Synchronous replication isappropriate when exact consistency is critical to the business application.

Storage system configurationsCluster configurations typically consist of two or more server systems connected to a shared storagesystem. HP 3PAR Cluster Extension allows the dispersion of data center resources by enablingcluster systems to take advantage of HP 3PAR storage systems configured for Remote Copyoperations. HP 3PAR Cluster Extension uses HP 3PAR InForm OS Command Line Interface (CLI) tocontrol the HP 3PAR storage systems (and Remote Copy) with the cluster software, and uses clustersoftware to react to system hardware and application failures.HP 3PAR Cluster Extension behavior is based on four major considerations:• Cluster software failover behaviors

• HP 3PAR Cluster Extension user settings

• The replication mode setting for HP 3PAR Remote Copy, which is used to configure the remotereplication feature of a 3PAR storage system environment based on your needs for applicationservice availability, data concurrency, and replication performance

• Remote Copy volume group and virtual volume status informationHP 3PAR Cluster Extension supports the following configurations:• 1-to-1

• N-to-1 (the logical configuration must be a one-to-one configuration). See Figure 2 (page 10).

• 1-to-N (the logical configuration must be a one-to-one configuration). See Figure 2 (page 10).

NOTE: Synchronous long distance and unidirectional remote copy configurations are notsupported.

Fully Automatic Failover and Failback 9

1-to-1 configurationWhen setting up a one-to-one configuration (see Figure 2 (page 10)), consider the following:• Cluster nodes are split between two or more geographically dispersed data center sites and

use redundant, diversely routed network connections for intra-cluster communications. Thoselinks must be as reliable as possible to prevent false failover operations or split-brain situations.

• Each cluster node connected to a HP 3PAR storage system should have redundant I/O pathsto the HP 3PAR storage system. Connections to both the source (primary) and the destination(secondary) copy of the application disk set are not allowed from the same host. Configureyour servers only in the zone that allows access to the local HP 3PAR storage system.

• HP recommends a minimum of two cluster nodes per site. This allows for preferred local failoverin case of server system failure. Local failover operations are faster than remote failoverbetween HP 3PAR storage systems because you do not need to change the mirroring directionof the Remote Copy volume group members.

• You can deploy HP 3PAR Cluster Extension in environments where several clusters use thesame HP 3PAR storage system pair.

• You can use HP 3PAR Remote Copy in synchronous replication mode between two HP 3PARstorage systems connected using FC switches and extender hardware.

• HP 3PAR Cluster Extension supports synchronous replication mode in all the configurationssupported by HP 3PAR Remote Copy Software.

• The HP 3PAR Remote Copy links must have redundant, separately routed links. The clusternetwork must have redundant, separately routed links. However, cluster networks and HP3PAR Remote Copy can share the same links if the link technology is protocol independent(for example, WDM) or if the FC protocol is transformed into IP.

Figure 2 1-to-1 configuration


N-to-1 and 1-to-N configurations

NOTE: The restrictions outlined in “1-to-1 configuration” (page 10) also apply to N-to-1configurations.

When setting up an N-to-1 or 1-to-N configuration (see Figure 3 (page 11)), consider the following:• HP 3PAR Cluster Extension uses the unit of Remote Copy volume groups as the failover unit.

This means all virtual volumes or members of a Remote Copy volume group will be failed overif the cluster software moves disk access from one cluster node to another.

• A given Windows HP 3PAR Cluster Extension resource can manage only one Remote Copygroup.

Figure 3 N-to-1 and 1-to-N configuration

Processes and componentsFor Windows Clustering, HP 3PAR Cluster Extension is shipped as a resource DLL. For RHEL, HPServiceguard, and SUSE, HP 3PAR Cluster Extension is shipped as a binary executable that iscalled by native clustering software when data replication is enabled through the package controlscript.With both Windows and Linux, the installation of HP 3PAR Cluster Extension also includes a CLIfor custom HP 3PAR Remote Copy integration. Customized solutions for failover application servicesmust implement HP 3PAR Cluster Extension through the CLX CLI (clx3PARrun.exe) prior to thedisk activation procedure on the server.

HP 3PAR Remote CopyHP 3PAR Remote Copy is a product that allows you to copy virtual volumes from one HP 3PARstorage system to another. The copy can be used for disaster recovery, backup, or data migration.

NOTE: HP 3PAR Remote Copy requires 3PAR Remote Copy licenses for all storage serversparticipating in a Remote Copy replication. See the 3PAR InForm OS Concepts Guide for additionalinformation on licensing and features.

Processes and components 11

Figure 4 Relationship of primary versus secondary volume groups on primary and backup storageservers

Remote Copy pairsRemote Copy configurations are based on the relationship between a storage system pair, alsoknown as the Remote Copy pair. Within this pair, the primary storage system is the system thatholds the volumes that are copied to a backup or secondary storage system, also known as aremote storage system.

Remote Copy volume groupsRemote Copy operations are performed on groups of virtual volumes called Remote Copy volumegroups. A Remote Copy volume group is a group of volumes on the same storage system that arelogically related and for which there is a cross-volume ordering of writes. Remote Copy volumegroups are used when data needs to be consistent across a group of volumes in order for databasesor other applications to process data correctly.Remote Copy uses volume groups to define a set of volumes for which applications might issuedependent writes.Remote Copy ensures that the data in the volumes within a group maintain write consistency. WhenRemote Copy operations are started or stopped, this is done for the whole group. When point-in-timesnapshots of such volumes are created, writes to all volumes in the group are blocked for a briefmoment to assure a consistent point-in-time copy of the whole Remote Copy volume group.You must configure a single Remote Copy volume group for a cluster group in Windows clustering.This Remote Copy volume group must include all virtual volumes being used for the applicationservice. The Remote Copy volume group is the unit in which HP 3PAR Cluster Extension failoveroperations are carried out. A given HP 3PAR Cluster Extension cluster resource can manage onlyone Remote Copy volume group.


Synchronous replicationUsing synchronous mode, all write requests from the server are first transferred to the remote storagesystem. After each I/O has been mirrored in the cache area of the remote storage system, it isacknowledged to the local storage system. The write request is then acknowledged to the server.

User configuration fileHP 3PAR Cluster Extension provides a user configuration file to customize failover behavior. Youcan specify all customizable objects of HP 3PAR Cluster Extension in this file.See “User configuration file” (page 65) for more information.

Force FlagThe force flag forces HP 3PAR Cluster Extension to skip the internal logic and enables write accessto the local virtual volumes regardless of the Remote Copy volume group member state. You canset this flag when you are sure that the current site contains the latest data, even though a previousapplication service startup process failed because HP 3PAR Cluster Extension discovered a RemoteCopy volume group member state that could not be handled automatically.To use the force flag feature, before starting the application service that uses HP 3PAR ClusterExtension, you must create a file called application_name.forceflag in the directory specifiedby the ApplicationDir property. While using clx3PARrun to use the force flag feature, use theoperand –forceflag to eliminate the need of creating the application_name.forceflag file. The application_name.forceflag is the name of the HP 3PAR Cluster Extension resource.Before creating this file, ensure that the application service is not running elsewhere. This file isremoved after it is detected by HP 3PAR Cluster Extension.You cannot use the force flag if the local virtual volume state is a combination of secondary andsyncing, which indicates that a copy operation is in progress. A disk cannot be put in read/writeaccess mode when a write operation is in progress to that virtual volume; in this case, HP 3PARCluster Extension returns a global error.

Planning for HP 3PAR Cluster ExtensionConsider the dependencies described in this section before configuring HP 3PAR Cluster Extension.

Cluster setup considerations

Windows clusteringWindows Clustering depends on a quorum resource to maintain a persistent log of clusterconfiguration changes and status, as well as a single point to resolve any possible events thatcould result in a split-brain situation. HP 3PAR Cluster Extension supports the following quorumconfigurations in which one site failure will not cause the entire cluster to fail.

Windows Server 2008/2008 R2

• Node Majority

• Node and File Share Majority

Windows Server 2003

• Majority Node Set (MNS)

• Majority Node Set with File Share Witness

NOTE: HP 3PAR Cluster Extension does not support shared disk quorum with Windows Clustering.

Planning for HP 3PAR Cluster Extension 13

MNS or Node Majority quorum in Windows ClusteringAn MNS is a single quorum resource that allows you to build clusters with nodes that aregeographically separated. In an MNS cluster, unlike in a quorum disk cluster, data is stored onmultiple disks across the cluster. The MNS resource ensures that the cluster's configuration data isstored locally on the nodes and that it is kept consistent across the different disks. This allows clustertopologies where the nodes do not need shared access to a quorum disk.(Number of nodes in the cluster/2) +1This ensures that more than 50% of the nodes will have an up-to-date copy of the configurationinformation.The cluster service does not start (and bring resources online) if there are 50% or less of theconfigured nodes up and running. The cluster service waits, trying to restart, until a quorum isestablished when more nodes join. This feature guarantees that the cluster has the latest and mostup-to-date configuration. This also means that, in a geographically dispersed cluster, you mustdistribute the nodes evenly between two data centers and have an arbitrator node in a third siteor separate protected area to be able to survive a single data center failure.

Majority Node Set with File Share WitnessThe file share witness feature is an improvement to the Majority Node Set (MNS) quorum model.This feature lets you use a file share that is external to the cluster as an additional "vote" to determinethe status of the cluster in a MNS quorum cluster deployment.Consider a two-node MNS quorum cluster. Because an MNS quorum cluster can only run whenthe majority of the cluster nodes are available, a two-node MNS quorum cluster is unable to sustainthe failure of any cluster node. This is because the majority of a two-node cluster is two. To sustainthe failure of any one node in an MNS quorum cluster, you must have at least three devices thatcan be considered as available. The file share witness feature enables you to use an external fileshare as a witness. This witness acts as the third available device in a two-node MNS quorumcluster. Therefore, with this feature enabled, a two-node MNS quorum cluster can sustain the failureof a single cluster node. Additionally, the file share witness feature provides the following twofunctions:• It helps protect the cluster against a problem that is known as a split brain. This problem occurs

if the two nodes in a MNS quorum cluster cannot communicate with each other. In this situation,each cluster node is unable to determine whether the loss of communication occurred becausethe other cluster node failed, or whether the loss of communication occurred because of aproblem with the network. The file share witness can designate one of the cluster nodes asthe surviving cluster node. That cluster node can then determine that it should continue to runthe cluster. In this scenario, the surviving cluster node can determine that the other cluster nodefailed, or that the other cluster node was not sanctioned by the file share witness.

• It helps protect the cluster against a problem that is known as a partition in time. This problemoccurs if the following conditions are true:

◦ Cluster node A is running, but cluster node B is not running.

◦ Cluster node A stops running.

◦ Cluster node B tries to run the cluster.In this situation, cluster node B may not have the cluster state information that was updatedon cluster node A. Therefore, cluster node B may run the cluster by using incorrect stateinformation. The file share witness feature helps prevent this problem by detecting thatthe cluster state has changed. The file share witness feature prevents the cluster node thatcontains outdated cluster state information from running the cluster.


NOTE: File Share Witness support:• Windows Server 2003 (Service Pack 1 and later) supports only two node clusters when using

File Share Witness.• Windows Server 2003 Service Pack 2 supports MNS with File Share Witness by default.

• When using Microsoft Windows Server 2003 Service Pack 1 (SP1)-based server cluster orWindows Server 2003 R2-based server cluster, you must apply a Microsoft patch to supportMNS with File Share Witness. See your Microsoft documentation for more information.

SLE HA cluster setup considerationsFollow the guidelines in this section when you configure clusters for use with HP 3PAR ClusterExtension.

QuorumIn an SLE HA cluster, quorum is defined as a strict majority of the defined cluster (more than 50%).With certain failures, a cluster might be divided into two subclusters. In an SLE HA cluster, asubcluster with more than 50% of the nodes wins the quorum. The subcluster that wins the quorumre-forms the cluster and fences the subcluster that lost the quorum. The behavior of the subclusterthat lost the quorum depends on the defined no-quorum policy. This behavior is in effect until thecluster is fenced. When the cluster is fenced, the resources owned by the fenced nodes fail overto active cluster nodes.

STONITHSTONITH is an SLE HA cluster fencing method. SLE HA cluster provides STONITH plug-ins fordevices such as UPS, PDU, Blade power control devices, and lights out devices. Some plug-ins canSTONITH more than one node (for example, Split Brain Detector STONITH) and some can STONITHonly one node (for example, HP iLO STONITH).HP iLO STONITH uses the power control functions of an HP iLO device to STONITH a node thathas lost quorum and needs to be fenced.

IMPORTANT: If all of the iLO devices in a cluster are connected using a single network, a singleswitch failure might disable iLO, preventing nodes from being fenced. This failure might be difficultto detect, especially before a node failure where iLO features would be required.

The STONITH action can be set to power off or reset, depending on the environment requirements.• Power off: The STONITH agent powers off the nodes in the errant subcluster.

• Reset: The STONITH agent resets the nodes in the errant subcluster, and the nodes try toautomatically rejoin the cluster.

NOTE: IPMI fencing can be used for Integrity servers that do not support RIBCL scripting.

Networking in an SLE HA clusterConfiguring redundant and independent cluster communication paths is a good way to avoid SplitBrain conditions. With redundancy in communication paths, the loss of a single interface or switchdoes not break the communication between nodes and prevents Split Brain conditions.Administrators can configure multiple independent communication paths. HP recommends usingbonded Ethernet channels.

Resource constraintsResource constraints allow administrators to specify which cluster nodes resources can run on, theorder resources are loaded, and the other resources a specific resource is dependent on.


There are three types of resource constraints:• Resource location: Defines the nodes on which a resource can run, cannot run, or is preferred

to be run.• Resource colocation: Defines which resources can or cannot run together on a node.

• Resource order: Defines the sequence of actions for resources running on a node.

Resource operation attributeSLE HA does not monitor resource health by default. To enable this feature, add the monitoroperation to the resource definition. You can specify the interval attribute and the timeout attributefor a monitor operation. The interval attribute defines the time interval in which the monitor operationis executed. The timeout attribute determines how long to wait before considering the resource asfailed.Define start, stop, and monitor operations for the HP 3PAR Cluster Extension resource.

HP 3PAR Cluster Extension resource dependencyA Group resource in an SLE HA cluster ensures that the member resource agents are started andstopped in the required order. A HP 3PAR Cluster Extension resource must be added as the firstmember of the group. This way, all primitive resources added after the HP 3PAR Cluster Extensionresource are dependent on HP 3PAR Cluster Extension.Since the primitive resources within a resource group can be failed over independently, set acollocation constraint for each resource group ID with the last resource in the group to achieve thefailover of the entire group when any primitive resource fails.

Failover orderUse location constraints to define the failover order for a resource group.For each node, define a location constraint with the appropriate score to prioritize the resourcegroup on that particular node. During failover, the cluster calculates the score of the resource groupon the available nodes, and the node with the highest score is considered the next preferred owner.For more information, see the SLE HA documentation.

Failback optionHP does not recommend auto failback in configurations with HP 3PAR Cluster Extension becausethe resource failovers due to storage failure can cause resources to go into an unstable state(failover/failback might toggle the resource between the nodes).SLE HA provides the meta-attribute resource-stickiness to determine how much a resource agentprefers to stay where it is. To disable auto failback, set resource-stickiness to the lowest valuecompared to the other resource location constraints.

Migration-thresholdA resource is automatically restarted if it fails. If a restart cannot be achieved on the current nodeor it fails to start a certain number of times on the current node, it tries to fail over to another node.You can define the number of failures for resources (a migration-threshold) after which they migrateto a new node. If you have more than two nodes in your cluster, the high availability softwarechooses the node a particular resource fails over to.When a HP 3PAR Cluster Extension resource fails, HP recommends configuring your cluster to failover the resource without restarting on the local node. To set this preference, set themigration-threshold to 1.


Disk monitoringFor the situations in which disk access is lost or read/write protection is in effect due to storagefencing, application monitoring agents, file system agents, or LVM resource agents detect the IOfailure. HP 3PAR Cluster Extension does not monitor the disk access status.

RHCS cluster setup considerations

QuorumIn RHCS, the quorum is based on a simple voting majority of the defined nodes in a cluster. Tore-form successfully, a majority of all possible votes is required.Each cluster node is assigned a number of votes, and they contribute to the cluster while they aremembers. If the cluster has a majority of all possible votes, it has quorum (also called quorate);otherwise, it does not have quorum.

FencingCluster software adjusts the node membership based on various failure scenarios. The concept ofquorum defines which set of nodes continue to define the cluster. To protect data, nodes that donot have quorum are removed from the cluster. The non-quorate nodes that are removed must beprevented from accessing the shared resources. This process is called fencing.HP iLO fencing is one method that can be used with RHCS to restrict cluster node access to sharedresources.Observe the following guidelines when using HP iLO network configurations with RHCS clusters:• HP iLO can be connected to the client access network or to a different network, but the network

must be routable.• HP iLO should not be on the network that is used for cluster communication.

• The HP iLO of each cluster system must be accessible over the network from every other clustersystem.

To handle infrequent failures of the HP iLO fencing (such as a switch failure), you can set up abackup fence method for redundancy.HP iLO fencing can be used on HP Proliant systems with built-in iLO hardware. For third-partysystems, other power control fencing methods can be used.

NOTE: IPMI fencing can be used for Integrity servers that do not support RIBCL scripting.

Qdisk configurationRed Hat recommends the use of a Qdisk configuration to bolster quorum to handle failures suchas half (or more) of the members failing, a tie-breaker in equal split partition, and a SAN failure.In a HP 3PAR Cluster Extension configuration with multiple storage arrays, a Qdisk configurationis not supported.

Failover domainsA cluster service is associated with a failover domain, which is a subset of cluster nodes that areeligible to run a particular cluster service. To maintain data integrity, each cluster service can runon only one cluster node at a time. By assigning a cluster service to a restricted failover domain,you can limit the nodes that are eligible to run a cluster service in the event of a failover, and youcan order the nodes by preference to ensure that a particular node runs the cluster service (as longas that node is active).


A failover domain can have the following characteristics:• Unrestricted: Specifies that the subset of members is preferred, but the cluster service assigned

to this domain can run on any available member.• Restricted: The cluster service is allowed to run only on a subset of failover domain members.

• Unordered: The member on which the cluster service runs is chosen from the available list offailover domain members with no preference order.

• Ordered: The failover domain member on which the cluster service runs is selected based onpreference order. The member at the top of the list (as specified in /etc/cluster/cluster.conf) is the most preferred, followed by the second member, and so on.

For an orderly failover, HP recommends using the Ordered and Restricted options for your failoverdomains.

Failback policyHP does not recommend auto failback in configurations with HP 3PAR Cluster Extension becausethe resource failovers due to storage failure can cause resources to go into an unstable state(failover/failback might toggle the resource between the nodes). In this situation, HP recommendscorrecting the failure and then manually failing back to the intended data center or server.To disable the auto failback, set the nofailback flag for the failover domain.Enabling this option for an ordered failover domain prevents automated failback after amore-preferred node rejoins the cluster.

Recovery policyWhen a resource inside the service fails, the default action is to restart the service on the localnode before the failover. In a HP 3PAR Cluster Extension environment, it is always expected torelocate the service during restart. To enable this functionality, set the service recovery policy torelocate.

Service hierarchical structure and resource dependencyIn RHCS, a service is a collection of cluster resources configured into a single entity that is managed(started, stopped, or relocated) for high availability. A service is represented as a resource treethat specifies each resource, its attributes, and its relationship among other resources in the resourcetree. The relationships can be parent, child, or sibling. Even though a service is seen as a singleentity, the hierarchy of the resources determines the order in which each resource within the serviceis started and stopped.In the case of a child-parent relationship, the startup or shutdown is simple. All parents are startedbefore children, and children must all stop cleanly before a parent can be stopped. For a resourceto be considered in good health, all of its children must be in good health.A service is considered failed if any of its resources fail. In this case, the expected course of actionis to restart the entire service, including the failed resource and the other resources that did notfail.In a HP 3PAR Cluster Extension environment, configure the HP 3PAR Cluster Extension resource asthe parent resource in the service so that HP 3PAR Cluster Extension can control the service behaviorbased on the user configuration and storage device status. This means that the HP 3PAR ClusterExtension resource must be configured at the highest level in the dependency hierarchy.

Disk monitoringFor the situations in which disk access is lost or read/write protection is in effect due to storagefencing, application monitoring agents, file system agents, or LVM resource agents detect the IOfailure. HP 3PAR Cluster Extension does not monitor the disk access status.


HP Serviceguard for Linux cluster setup considerations

QuorumIn general, the algorithm for cluster re-formation requires a cluster quorum of a strict majority (thatis, more than 50%) of the nodes previously running.Although a cluster quorum of more than 50% is generally required, exactly 50% of the previouslyrunning nodes may re-form as a new cluster provided that the other 50% of the previously runningnodes do not re-form. This is guaranteed by the use of a tie-breaker to choose between the twoequal-sized node groups, allowing one group to form the cluster and forcing the other group toshut down. This tie-breaker is known as a cluster lock. The cluster lock is implemented either bymeans of a lock LUN or a quorum server. A cluster lock is required on two-node clusters.The cluster lock is used as a tie-breaker only for situations in which a running cluster fails and, asServiceguard attempts to form a new cluster, the cluster is split into two sub-clusters of equal size.Each sub-cluster will attempt to acquire the cluster lock. The sub-cluster which gets the cluster lockwill form the new cluster, preventing the possibility of two sub-clusters running at the same time. Ifthe two sub-clusters are of unequal size, the sub-cluster with greater than 50% of the nodes willform the new cluster, and the cluster lock is not used.If you have a two-node cluster, you are required to configure a cluster lock.

Using Generic Resources to Monitor Volume GroupsYou can monitor a particular disk that is a part of an LVM volume group used by packages.You can do this by using the disk monitor capabilities of the System Fault Management, availableas a separate product, and integrating it in Serviceguard by configuring generic resources inpackages.Monitoring can be set up to trigger a package failover or to report disk failure events toServiceguard by writing monitoring scripts, which can be configured as a service in a Package.

Disk monitoringServiceguard provides disk monitoring for the shared storage that is activated by packages in thecluster. The monitor daemon on each node tracks the status of all the disks on that node that youhave configured for monitoring.The configuration must be done separately for each node in the cluster, because each node monitorsonly the group of disks that can be activated on that node, and that depends on which packagesare allowed to run on the node.

NOTE: For disk monitoring we should ensure that the multipath device names (either default oruser friendly) should be same for both source and destination LUNs of the Remote Copy volumegroup across all the nodes.For more details, see Red Hat Enterprise Linux DM Multipath Configuration and Administrationguide and SUSE Linux Enterprise Server Storage Administration guide .

Cluster timeout considerationsEach node sends its heartbeat message at a rate calculated by Serviceguard on the basis of thevalue of the MEMBER_TIMEOUT parameter, set in the cluster configuration file, which you createas a part of cluster configuration.The duration of the safety timer depends on the cluster configuration parameter MEMBER_TIMEOUT,and also on the characteristics of the cluster configuration, such as whether it uses a quorum serveror a cluster lock (and the type of lock) and whether or not standby LANs are configured.


failover packagesThe failover package is the most common type of package. It runs on one node at a time. If afailure occurs, it can switch to another node listed in its configuration file. If switching is enabledfor several nodes, the package manager will use the failover policy to determine where to startthe package.HP 3PAR Cluster Extension supports only failover packages.

failover policyIt specifies how Serviceguard decides where to start the package, or restart it if it fails. It can beset to configured_node or min_package_node. The default is configured_node.• configured_node means that Serviceguard will attempt to start the package on the first

available node in the list you provide under node_name.• min_package_node means that Serviceguard will start the package on whichever node in

the node_name list that has the fewest packages running at the time.This parameter can be set for failover packages only.

failback policyIt specifies whether or not Serviceguard will automatically move a package that is not running onits primary node (the first node on its node_name list) when the primary node is once againavailable. It can be set to automatic or manual. The default is manual.• manual means that the package will continue to run on the current node.

• min_package_node means that Serviceguard will move the package to the primary nodeas soon as that node becomes available, unless doing so would also force a package with ahigher priority to move.

For more information on HP Serviceguard for Linux cluster setup considerations, see the latestedition of Managing HP Serviceguard for Linux document.

Configuration tool (clx3parconfig.exe) for WindowsHP 3PAR Cluster Extension for Windows uses the configuration tool (clx3parconfig.exe) tostore configuration and access information for each configured management server. This tool storesinformation in the clx3parcfg file. However, because the clx3parcfg file cannot be editedwith a standard text editor, management server entries can only be added, modified, and deletedusing the configuration tool that is included with the standard installation of HP 3PAR ClusterExtension.

HP 3PAR InForm Management Console or HP 3PAR InForm CLIHP 3PAR InForm Management Console or HP 3PAR InForm CLI is used to create virtual volumesand configure the HP 3PAR storage system.

IP network considerationsHP 3PAR CLI is essential to Cluster extension which communicates with the HP 3PAR storage systemover the IP network. HP recommends that you provide reliable network connections for cluster nodeto HP 3PAR storage system communications.

SAN fabric considerationsEnsure that cluster nodes in the zoning configuration recognize their respective (local) HP 3PARstorage system, but not the remote HP 3PAR storage system.


2 Configuring HP 3PAR Cluster Extension for WindowsAfter installing HP 3PAR Cluster Extension, use the configuration tool to define the HP 3PAR ClusterExtension setup configuration. Use Cluster Administrator (Windows Server 2003), Failover ClusterManagement (Windows Server 2008/2008 R2), or cluster commands in the CLI to add andconfigure resources.

Integrating HP 3PAR Cluster Extension with MSCSTo integrate HP 3PAR Cluster Extension with MSCS:1. Define the setup configuration for the HP 3PAR Cluster Extension application. For instructions,

see “Configuring HP 3PAR Cluster Extension” (page 21).2. Add a HP 3PAR Cluster Extension resource. For instructions, see “Adding a HP 3PAR Cluster

Extension resource” (page 25).3. Configure the HP 3PAR Cluster Extension resource. For more information, see “Configuring

HP 3PAR Cluster Extension resources” (page 27) .Add dependencies on the HP 3PAR Cluster Extension resource. For instructions, see “Addingdependencies on a HP 3PAR Cluster Extension resource” (page 37).

Configuring HP 3PAR Cluster ExtensionAfter installation, you must define the configuration setup using the HP 3PAR Cluster ExtensionConfiguration Tool.You can configure HP 3PAR Cluster Extension with the GUI or the CLI. Use the following instructionsfor the GUI. For instructions on performing HP 3PAR Cluster Extension configuration tasks with theCLI, see “Defining the HP 3PAR configuration information using the GUI” (page 22).

Starting the HP 3PAR Configuration ToolTo start the HP 3PAR Cluster Extension Configuration Tool:• ForWindows Server 2003 or Windows Server 2008/2008 R2: Double-click the HP 3PAR

CLX Configuration Tool icon on the desktop, or select Start > Programs > Hewlett-Packard >HP 3PAR CLX Configuration Tool.

• For Server Core or Hyper-V Server: Open a command window and enter clx3PARconfig–I.

The HP 3PAR CLX Configuration Tool window appears.

Integrating HP 3PAR Cluster Extension with MSCS 21

Defining the HP 3PAR configuration information using the GUITo define configuration information using the GUI:1. Open the configuration tool.

For instructions, see “Starting the HP 3PAR Configuration Tool” (page 21)2. Click Add in the Array Configuration section to open the Add Array dialog box.

22 Configuring HP 3PAR Cluster Extension for Windows

3. Enter the Array Network Name.4. Click Browse to select the password file.5. Click Connect to connect to the array.

The array details (Array Name, Array Serial Number, Array Model, and Array ID) and thearray target information appears on the Add Array dialog box.

NOTE: If the Remote Copy link is down between the array and the defined targets, ClusterExtension will not be able to obtain the target array information. The value in the target arrayID will be NA and a warning message is displayed. However, you can continue with the arrayconfiguration, but you cannot configure Cluster Extension resources between the array andthe failed targets. Verify the Remote Copy link between the array and the failed targets. Launchthe HP 3PAR Cluster Extension Configuration Tool and use the Modify array option, and clickConnect on the Modify HP 3PAR storage system Connection Settings dialog to obtain thecorrect target information.

6. Click OK to add the array information into the cluster configuration.The HP 3PAR CLX Configuration Tool window appears.

7. Click Add in the Cluster Node Configuration section to open the Add Server dialog box.The available servers are listed.

8. Select the servers that are possible owners of the HP 3PAR Cluster Extension-managed disks,and click OK.

NOTE: See the Microsoft Windows Clustering documentation for more information aboutpossible owners.

9. Click OK.You will be prompted to confirm if you want to automatically copy the configuration file onall the cluster nodes used by Cluster Extension.

a. If you select Yes, the configuration information is automatically copied to all cluster nodesused by Cluster Extension.

Configuring HP 3PAR Cluster Extension 23

NOTE: You must create or copy the array password file on all cluster nodes used byCluster Extension. Ensure that the appropriate path is reflected in the configurationinformation of each node. During configuration, you will be prompted to confirm if youwant to copy the configuration file on all cluster nodes. If you choose to copy the HP3PAR Cluster Extension configuration file automatically, the same password file locationis retained on all cluster nodes. If the password file location on each cluster node isdifferent, ensure that you change the password file location by running the HP 3PARCluster Extension Configuration Tool locally.

b. If you select No, configuration information is saved locally.You can use the import and export features to copy the HP 3PAR Cluster Extensionconfiguration file clx3PARcfg from one cluster node to another.

Your configuration information is saved in the clx3PARcfg file in the %HPCLX_3PAR_PATH%\bin directory.

NOTE: To verify the connectivity between array and cluster nodes, select one or moreconfigured HP 3PAR storage systems and click Connection Test.

Defining the HP 3PAR configuration information using the CLIYou can configure HP 3PAR Cluster Extension using the CLI command clx3PARconfig.Enter clx3PARconfig /?, or clx3PARconfig /help to view usage information.1. Use the following command to add arrays:

clx3PARconfig ARRAY /ADD [{NAME=<array network name|array IPaddress>PWF=<password file path>}] [{NAME=<array network name |array IP address>PWF=<password file path>}]...]

2. Use the following command to add the available cluster nodes:clx3PARcofig SERVER /ADD [NAME=servername] [NAME=servername...]

3. Use the procedures in “Importing and exporting configuration information” (page 24) to copythe configuration information to the other cluster nodes.

Importing and exporting configuration informationThe import feature allows you to define the configuration setup using an existing configuration file.The export feature allows you to save a copy of an existing configuration.Use the import and export features to copy the HP 3PAR Cluster Extension configuration file(clx3PARcfg) from one cluster node to another.


Exporting configuration settings using the GUI1. Open the configuration tool.

For instructions, see “Starting the HP 3PAR Configuration Tool” (page 21).2. Click Export.3. When prompted, choose a save location, enter a file name, and then click Save.4. Click OK to save and close the configuration tool.

Exporting configuration settings using the CLI1. Open a command window.2. Enter clx3PARconfig EXPORT /FILE=filepath, where filepath specifies the save

location and file name.

Importing configuration settings using the GUI1. Open the configuration tool.2. For instructions, see “Starting the HP 3PAR Configuration Tool” (page 21).3. Click Import.4. When prompted, choose the configuration file, and then click Open.5. Click OK to save and close the configuration tool.

Importing configuration settings using the CLI1. Open a command window.2. Enter clx3PARconfig IMPORT /FILE=filepath, where filepath specifies the file

location and name.

Adding a HP 3PAR Cluster Extension resourceIMPORTANT: In Cluster Administrator (Windows Server 2003), resources are added to resourcegroups. In Failover Cluster Management (Windows Server 2008/2008 R2), the term resourcegroups are changed to services and applications. In this guide, the term services and applicationsrefers to resource groups for Windows Server 2003 and services and applications for WindowsServer 2008/2008 R2.

To use 3PAR Cluster Extension, you must add an HP 3PAR Cluster Extension resource in the clustermanagement application. The procedure to use depends on the installed version of Windows.

• For Windows Server 2003, use the Cluster Administrator GUI or cluster commands in the CLI.For instructions, see “Adding a resource using the Cluster Administrator GUI (Windows Server2003)” (page 26)or “Adding HP 3PAR Cluster Extension resource using the Microsoft CLIcluster commands” (page 26).

• For Windows Server 2008/2008 R2, use the Failover Cluster Management GUI or clustercommands in the CLI. For instructions, see “Adding HP 3PAR Cluster Extension resource usingthe Failover Cluster Management GUI (Windows Server 2008/2008 R2)” (page 26) or“Adding HP 3PAR Cluster Extension resource using the Microsoft CLI cluster commands”(page 26).

• For Server Core or Hyper-V Server, use one of the following:

Failover Cluster Management GUI on the remote management station◦◦ HP 3PAR Resource Configuration Tool on CLX nodes

◦ Cluster commands in CLX nodes

Adding a HP 3PAR Cluster Extension resource 25

See “Adding HP 3PAR Cluster Extension resource using the Microsoft CLI cluster commands”(page 26)for instructions.

CAUTION: Do not use the following characters in Cluster Extension resource names: \ / : * ? "< > |. Using these characters might affect the creation of the resourcename.online file, whichis used for the HP 3PAR Cluster Extension resource health check mechanism.

Adding a resource using the Cluster Administrator GUI (Windows Server 2003)Use the procedure in this section to add a resource using the Cluster Administrator GUI. Forinstructions on using the CLI, see “Adding HP 3PAR Cluster Extension resource using the MicrosoftCLI cluster commands” (page 26).1. Create a new resource as described in your Microsoft documentation.2. Enter values for the following fields:

• Name: A resource name, for example, clx01.

• Description: As appropriate for the resource.

• Resource type: Select Cluster Extension 3PAR.

• Group: The resource group to be associated with the new resource, for example,Application_ A.

3. Click Next.4. Add or remove resource owners.5. Click Next. Do not add any dependencies.6. Click Next.7. Modify resource property values of the new resource if needed.8. Click Finish.

Adding HP 3PAR Cluster Extension resource using the Failover Cluster ManagementGUI (Windows Server 2008/2008 R2)

Use the procedure in this section to add a resource using the Failover Cluster Management GUI.For instructions on using the CLI, see “Adding HP 3PAR Cluster Extension resource using theMicrosoft CLI cluster commands” (page 26).1. Add a service or application in the Failover Cluster Management GUI, as described in your

Microsoft documentation.2. Right-click the service or application and select Add a resource > More resources > Add Cluster

Extension 3PAR.

Adding HP 3PAR Cluster Extension resource using the Microsoft CLI cluster commandsYou can use the cluster commands in this section with Windows Server 2003, Windows Server2008/ 2008 R2, Server Core, and Hyper-V Server.Use the following command to add an HP 3PAR Cluster Extension resource:cluster resource resource_name /create /group:service_or_application_name/type:"Cluster Extension 3PAR"

ExampleThis command adds an HP 3PAR Cluster Extension resource called clx_fileshare to theCLX_SHARE service or application.cluster resource clx_fileshare /create /group:CLX_SHARE /type:"ClusterExtension 3PAR"


Configuring more than one resource in a service or applicationTo create more than one HP 3PAR Cluster Extension resource in a service or application:1. Create two (or more) Remote Copy volume groups and create a resource for each Remote

Copy volume group.2. Make every physical disk resource in the cluster service or application depend on both (all)

resources within the cluster service or application. This ensures that any disk in the clusterservice or application can go online only if both (all) Remote Copy volume groups are failedover correctly.

Changing a HP 3PAR Cluster Extension resource nameThis section describes how to change a HP 3PAR Cluster Extension resource name.

Changing a HP 3PAR Cluster Extension resource name (Windows Server 2003)In this procedure, you use the Cluster Administrator GUI to change a resource name. For instructionson using CLI commands to change a resource name, see “Setting HP 3PAR Cluster Extensionresource properties using the CLI” (page 36).1. Open Cluster Administrator.2. Open the resource Properties window and click the General tab.3. Enter a new name in the Name field.4. Click OK to save your changes and close the window.

Changing a HP 3PAR Cluster Extension resource name (Windows Server 2008/2008R2)

In this procedure, you use the Failover Cluster Management GUI to change a resource name. ForServer Core or Hyper-V Server, use the MMC to run the Failover Cluster Management GUI froma remote node or use cluster commands in the CLI to change the resource name. See “Setting HP3PAR Cluster Extension resource properties using the CLI” (page 36) for instructions.1. Open Failover Cluster Management.2. Open the resource Properties window and click the General tab.3. Enter a new name in the Resource Name field.4. Click OK to save your changes and close the window.

Configuring HP 3PAR Cluster Extension resourcesHP 3PAR Cluster Extension resource properties are configured using the Cluster Administrator(Windows Server 2003) or Failover Cluster Management (Windows Server 2008/2008 R2) GUI,or cluster commands in the CLI.• For information about MSCS and Microsoft Failover Cluster Service properties that affect HP

3PAR Cluster Extension, see “Setting Microsoft cluster-specific resource and service orapplication properties” (page 28).

• For information on HP 3PAR Cluster Extension-specific properties, see “Setting HP 3PAR ClusterExtension-specific resource properties” (page 31).

Before configuring HP 3PAR Cluster Extension resources, review the HP 3PAR Cluster Extensionobjects described in “User configuration file and HP 3PAR Cluster Extension objects” (page 65).

Changing a HP 3PAR Cluster Extension resource name 27

When configuring HP 3PAR Cluster Extension resources, note the following:• If the Cluster Administrator or Failover Cluster Management GUI is used to configure a HP

3PAR Cluster Extension resource, configuring the resource using a user configuration file (UCFfile) is not required.

• HP 3PAR Cluster Extension creates a file called resource_name.online to indicate thatthe HP 3PAR resource is online on the local node. This file is created in the directory specifiedby the ApplicationDir resource property. If the resource is taken offline, this file is deleted.

• Windows Server 2008/2008 R2 only: If a HP 3PAR Cluster Extension resource is notconfigured, the resource icon in the Failover Cluster Management GUI shows the message“not configured” next to the resource status.

• The HP 3PAR Cluster Extension resource must be the first resource for all other disk resources.

• The resource's PendingTimeout value must be equal to or greater than the value specified forthe ResyncWaitTimeout attribute.

Setting Microsoft cluster-specific resource and service or applicationproperties

Microsoft allows you to set specific failover parameter and threshold values for a service orapplication, and for a resource. Some of these values must be changed for HP 3PAR ClusterExtension to enable manual recovery actions in case of a disaster.To set Microsoft cluster-specific resource properties:• For Windows Server 2003, use the Cluster Administrator GUI or cluster commands in the CLI.

• For Windows Server 2008/2008 R2, use the Failover Cluster Management GUI or clustercommands in the CLI.

• For Server Core or Hyper-V Server, use cluster commands in the CLI.

TIP:You can use the GUI option for Server Core or Hyper-V Server by using the MMC to manage acluster remotely. For more information about using the MMC, see your Microsoft documentation.

HP 3PAR Cluster Extension requirements for Cluster Administrator and Failover Cluster Managementresource properties are described in Table 1 (page 28). If there is no required value for a property,the valid and/or default values are specified. Set these properties in the resource properties windowor the CLI. If you use the CLI, use the following command:cluster.exe resource<ResourceName> /prop PropertyName="PropertyValue".For more information about setting resource properties, see your Microsoft documentation.

Table 1 Setting resource properties and values in the GUI

ValueDescriptionFormatProperty

Windows Server2008/2008 R2 GUI:01:00 mm:ss (Default)Windows Server 2003 GUI:60000 milliseconds (Default)

Used to poll “Alive” state forthe resource. Also used byHP 3PAR Cluster Extensionto perform HP 3PAR RemoteCopy start operations ifauto_recover policy is set

IntegerThorough Resource HealthCheck Interval (WindowsServer 2008/2008 R2)“Is Alive” poll interval(Windows Server 2003)IsAlivePollInterval (CLI)


Table 1 Setting resource properties and values in the GUI (continued)


CLI: 60000 milliseconds(Default)

for a particular RemoteCopy volumegroup.Decreasing this valueallows faster resource failuredetection but also consumesmore system resources.Set this value in theAdvanced Policies tab of theresource properties windowin Failover ClusterManagement, or in theAdvanced tab of theresource properties windowin Cluster Administrator.

Windows Server2008/2008 R2 GUI:00:05 mm:ss (Default)

Used to poll “Alive” state forthe resource. Decreasing thisvalue allows for fasterresource failure detection but

IntegerBasic Resource Health CheckInterval (Windows Server2008/2008 R2)“Looks Alive” poll interval(Windows Server 2003)

Windows Server2008/2008 R2 CLI: 5000milliseconds (Default)

also consumes more systemresources.Set this value in theAdvanced Policies tab of the

LooksAlivePollInterval (CLI)Windows Server 2003 GUI:60000 milliseconds (Default)resource properties window

in Failover Cluster Windows Server 2003 CLI:60000 milliseconds (Default)Management, or in the

Advanced tab of theresource properties windowin Cluster Administrator.

0 (Required)Defines whether a resourcecan be 0 (Required)

IntegerIf a resource fails, attemptrestart on current node –

automatically restarted afterit has failed.

Maximum restarts in thespecified period (WindowsServer 2008/2008 R2) Set this value in the Policies

tab of the resourceRestart Threshold (WindowsServer 2003) properties window in

Failover ClusterRestartThreshold (CLI)Management, or in theAdvanced tab of theresource properties windowin Cluster Administrator.

Windows Server2008/2008 R2: Check(Required)

Defines whether resourceswill be failed over if a restartis unsuccessful.

IntegerIf restart is unsuccessful, failover all resources in thisservice or application

Windows Server 2003:Restart and affect the group(Required, Default)

Set this value in the Policiestab of the resourceproperties window in

(Windows Server2008/2008 R2)RestartAction (WindowsServer 2003) CLI: 2 restart and

affect the group(Required)

Failover ClusterManagement, or in theAdvanced tab of theresource properties windowin Cluster Administrator.

RestartAction (CLI)

Windows Server 2003 only:This value must “affect thegroup.” This ensures that theresource group fails over toanother system if a resourceis reported FAILED.

Setting Microsoft cluster-specific resource and service or application properties 29

Table 1 Setting resource properties and values in the GUI (continued)


Windows Server2008/2008 R2: 15:00mm:ss (Default)

Determines the amount oftime for restart.Set this value in the Policiestab of the resource

IntegerIf a resource fails, attemptrestart on current node –Period for restarts (WindowsServer 2008/2008 R2)RestartPeriod (WindowsServer 2003)

Windows Server 2003: 900seconds (Default)CLI: 900000 milliseconds(Default)

properties window inFailover ClusterManagement, or in theAdvanced tab of the

RestartPeriod (CLI)

resource properties windowin Cluster Administrator.

Windows Server2008/2008 R2: 03:00mm:ss

Used to specify the timeoutfor status resolution. Formore information, see“Timing considerations for

IntegerPending timeout (GUI)PendingTimeout (CLI)

Windows Server 2003: 180seconds (Default)Windows Clustering” (page

36) .CLI: 180000 milliseconds(Default)Set this value in the Policies

tab of the resourceproperties window inFailover ClusterManagement, or in theAdvanced tab of theresource properties windowin Cluster Administrator.

HP 3PAR Cluster Extension requirements for service or application properties are described inTable 2 (page 31). If no specific value is required, the default value is listed. Set these values inthe Failover tab of the service or application properties window (Windows Server 2008/2008R2), the resource group properties window (Windows Server 2003), or in the CLI. For moreinformation about setting service or application properties, see your Microsoft documentation.

TIP: To change the properties in Table 2 (page 31) with the CLI, use the following command:cluster group groupname /prop propertyname="propertyvalue".


Table 2 Service or application properties and values


GUI: Prevent failbackCLI: 0 (required)

Prevents automatic failbackof a service or application

IntegerGUI: Failback (Preventfailback or Allow failback)

to its primary system.CLI: AutoFailbackTypeTransfer the service orapplication back manuallyafter the failure has beenrecovered. This allowsrecovery of all possiblefailure sources and pairresynchronization (ifnecessary) while theapplication service is stillrunning.

6 (Default)Determines the time (inhours) over which the cluster

StringGUI: PeriodCLI: FailoverPeriod

6 (Default) service attemptsto failover a service orapplication. See “Timingconsiderations for WindowsClustering” (page 36) formore information.

Windows Server2008/2008 R2: Number ofnodes in the cluster minus 1.

Determines the number offailover attempts. In case ofsubsequent system failure,the default value allows

IntegerGUI: Maximum failures inthe specified period(Windows Server2008/2008 R2), Threshold(Windows Server 2003)

Windows Server 2003, CLI:10 (Default)cluster service to transfer the

service or application toCLI: FailoverThresholdeach system. If all clustersystems are not members ofthe cluster, the service orapplication might restartautomatically.If this value is set to anumber higher than thecurrent number of clusteredsystems for the cluster group,the service or applicationwill continue to restart untileither the FailoverThresholdvalue or the FailoverPeriodtimeout value is reached.

Setting HP 3PAR Cluster Extension-specific resource propertiesChanges to resource properties take effect when the resource is brought online again. For instructionson changing resource properties, see:• “Setting HP 3PAR Cluster Extension resource properties using the GUI” (page 31)

• “Setting HP 3PAR Cluster Extension resource properties using the MMC ” (page 36)

• “Setting HP 3PAR Cluster Extension resource properties using the CLI” (page 36)

• “Setting HP 3PAR Cluster Extension properties using a UCF” (page 37)

Setting HP 3PAR Cluster Extension resource properties using the GUIThis section describes the procedures for setting HP 3PAR Cluster Extension resource propertieswith a GUI. You can perform these procedures through the resource configuration utility using thecluster software GUI (Cluster Administrator for Windows Server 2003 and Failover Cluster

Setting HP 3PAR Cluster Extension-specific resource properties 31

Management for Windows Server 2008/2008 R2) or the standalone resource configuration utilityin Server Core or Hyper-V Server.

TIP: For information on managing HP 3PAR Cluster Extension resources from a remote managementstation through the MMC, see“Setting HP 3PAR Cluster Extension resource properties using theMMC ” (page 36) .

Using Cluster Administrator to set resource properties (Windows Server 2003)To set resource properties for Windows Server 2003, use the Cluster Administrator GUI:.1. Open Cluster Administrator.2. Double-click the Resource folder in the console tree.3. Double-click the HP 3PAR Cluster Extension resource in the details pane to open the Properties

window.4. Click the Parameters tab.5. Make the necessary parameter changes, and then click OK.

Using Failover Cluster Management to set resource properties (Windows Server 2008/2008 R2)To set resource properties for Windows Server 2008/2008 R2, use the Failover ClusterManagement GUI:1. Open Failover Cluster Management.2. Double-click the HP 3PAR Cluster Extension resource in the summary pane to open the Properties

window.3. Click the Parameters tab.

4. Make the necessary parameter changes, and then click OK.


Using the resource configuration tool to set resource properties (Server Core and Hyper-V Server)To set resource properties for Server Core or Hyper-V Server, use the HP 3PAR Cluster Extensionresource configuration tool. When using the resource configuration tool:• You must run the tool on a Server Core or Hyper-V cluster node. You cannot run the tool on

a remote management station.• You cannot use the resource configuration tool to add or delete a resource.

• You can use the tool to configure multiple resources at one time. This saves time because youcan switch resources from the tool menu.

• The resource configuration tool is recommended for Hyper-V and Server Core environmentsbecause the properties you enter are validated. When you configure HP 3PAR Cluster Extensionresource properties from a remote management station or through the CLI, the properties youenter are not validated.

To use the resource configuration tool:1. Open a command window and enter CLX3PARRESCONFIG.EXE.2. Select the resource you want to change in the 3PAR CLX resource menu.3. Make the necessary parameter changes, and then click OK.

Configuring cluster node data center assignmentsTo configure cluster node data center assignments from the cluster software Parameters tab or theresource configuration tool:1. To add a cluster node:

a. Click Add to open the Add Server Data center Assignment window.


b. Select a cluster node from the Pre-Configured host list.c. Assign the cluster node to a data center selected from the Data center list.d. Click OK to save and close the window.

2. To remove a cluster node, select the node, and then click Remove.3. To modify a cluster node, select the node, and then click Modify.4. Click Apply to finish your modifications and leave the window open, or OK to save your

changes and close the Properties window.

Configuring HP 3PAR storage systemTo configure HP 3PAR storage systems from the cluster software Parameters tab or the resourceconfiguration tool:1. To add a storage system:

a. Click Add to open the Add Array Data center Assignment window.

b. Select a storage system from the Storage System Name list.c. Assign the storage system to a data center selected from the Data center list.d. Click OK to save and close the window.

2. To remove a storage system, select the storage system, and then click Remove.3. To modify a storage system, select the storage system, and then click Modify.4. Click Apply to finish your modifications and keep the window open, or click OK to save your

changes and then close the Properties window.

Selecting a volume groupUse the cluster software Parameters tab or the resource configuration tool to select the RemoteCopy volume group containing the disk that will be managed by the HP 3PAR Cluster Extensionresource.1. Click Browse.


2. Select a volume group, and then click OK.3. Click Apply to finish your modifications and keep the window open, or click OK to save your

changes and then close the Properties window.

Changing Remote Copy volume group settingsThe cluster software resource configuration GUI and the resource configuration tool displays theRemote Copy volume group name, replication mode, and replication status. Use the cluster softwareParameters tab or the resource configuration tool to edit the advanced Remote Copy volume groupsettings to control failover behavior for the Remote Copy volume group that you use with HP 3PARCluster Extension.1. Click Advanced Settings.

The Advanced settings dialog box appears.

Use this dialog box to configure the Remote Copy volume group behavior.• The settings in the Replication link up scenario section affect the failover behavior when

the HP 3PAR Remote Copy link is up.• The settings in the Replication link down scenario section affect the failover behavior

when the HP 3PAR Remote Copy link is down.2. Select RESYNCWAIT or FASTFAILBACK for the Application Startup Behavior.3. If you selected RESYNCWAIT in Step 2, set the RESYNCWAIT timeout value.

• Enter the RESYNCWAIT timeout value, in seconds.4. Select YES or NO as the Use non-current data OK value in the Replication link down scenario

section.5. Enter the path of the application directory. The application directory is where HP 3PAR Cluster

Extension searches for application-specific files, such as the force flag or online file.


6. Click OK to save your changes and close the Advanced settings window.7. Click Apply to save your changes and leave the window open, or click OK to save your

changes and close the Properties window.

Configuring takeover actionsPre-executables and post-executables can be defined to be executed before or after HP 3PARCluster Extension invokes its takeover functions. Use the cluster software Parameters tab or theresource configuration tool to add or remove pre/post executable scripts.1. Click Pre/Post Exec to display the Pre/Post Executable Properties dialog box.

2. Update the PreExecScript, PostExecScript, and PostExecCheck values as needed, and thenclick OK.

3. Click Apply to finish your modifications and leave the Properties window open, or OK to saveyour changes and close the window.

Setting HP 3PAR Cluster Extension resource properties using the MMCIf you are using Server Core or Hyper-V Server, you can manage a cluster remotely by using theMMC to run Failover Cluster Management.

NOTE: When you configure HP 3PAR Cluster Extension resource properties from a remotemanagement station through the MMC, which uses the standard Microsoft Properties tab, theproperties you enter are not validated, so you must enter the property values accurately, and verifythem against the HP 3PAR Cluster Extension documentation.

When you use this option, you will see the default Microsoft properties page instead of the HP3PAR Cluster Extension Parameters tab. For more information about using the MMC, see “Remotemanagement of HP 3PAR Cluster Extension resources in a cluster” (page 44)and your Microsoftdocumentation.

Setting HP 3PAR Cluster Extension resource properties using the CLIThe cluster commands in this section can be used with Windows Server 2003, Windows Server2008/2008 R2, Server Core, or Hyper-V Server.The Windows cluster default properties for a resource can be changed using the following command:cluster resource <resource_name> /privprop [<object_name=value|"value1value2 ...">]

NOTE: When you configure HP 3PAR Cluster Extension resource properties using the CLI, theproperties you enter are not validated, so you must enter the property values accurately, and verifythem against the HP 3PAR Cluster Extension documentation.

You can display all attributes of the HP 3PAR Cluster Extension resource clx_fileshare with thefollowing command:cluster resource clx_fileshare /privprop


The following example sets the UseNonCurrentDataOk property for the resource clx_fileshareto NO:cluster resource clx_fileshare /privprop UseNonCurrentDataOk=NO

For more information on the available properties, see “User configuration file and HP 3PAR ClusterExtension objects” (page 65).The following example changes the name of the resource 3PAR Cluster Extensionresource1 to 3PAR Cluster Extension resource2:cluster resource "3PAR Cluster Extension resource1" /ren:"3PAR ClusterExtension resource2"

Setting HP 3PAR Cluster Extension properties using a UCFYou can use a UCF to configure certain HP 3PAR Cluster Extension properties for Windows.Properties that you can configure in a UCF include:• LogLevel

• ClusterNotifyCheckTime

• ClusterNotifyWaitTime

• StatusRefreshTime

• HP3PARCLICommandTimeout

IMPORTANT: If you plan to use the default values for these properties, no UCF is required.

To configure properties using a UCF:1. Take the HP 3PAR Cluster Extension resource offline.2. Open the sample UCF.cfg file located in %ProgramFiles%\Hewlett-Packard\Cluster

Extension 3PAR\sample.3. Update the file with the property values you want to use.

For more information on the available properties, see “User configuration file and HP 3PARCluster Extension objects” (page 65).

4. Save the file and copy it to the following directory on all cluster nodes: %ProgramFiles%\Hewlett-Packard\Cluster Extension 3PAR\conf.

5. Bring the HP 3PAR Cluster Extension resource online.

Adding dependencies on a HP 3PAR Cluster Extension resourceHP 3PAR Cluster Extension Software must be the first resource in the resource chain of a Windowscluster service or application. All resources that depend on the disk resource, such as a file share,and all disk resources (physical disks), must be configured for dependency on the HP 3PAR ClusterExtension resource..To add dependencies:• For Windows Server 2003, use the Cluster Administrator GUI or cluster commands in the CLI.

• For Windows Server 2008/2008 R2, use the Failover Cluster Management GUI, clustercommands in the CLI, or the MMC for remote management.

• For Server Core or Hyper-V Server, use cluster commands in the CLI or the MMC.

Adding dependencies using Cluster Administrator (Windows Server 2003)To add dependencies using Cluster Administrator:1. Double-click the Resource folder in the console tree.2. Double-click a disk resource in the details pane.

Adding dependencies on a HP 3PAR Cluster Extension resource 37

3. Click the Dependencies tab, and then click Modify.4. Add the resource to the Dependencies list.

5. Click OK to save your changes.

Adding dependencies using Failover Cluster Management (Windows Server2008/2008 R2)

You can add dependencies with the GUI on a local node or by using the MMC to run the FailoverCluster Management application.1. Open Failover Cluster Management.2. Select a service or application that has HP 3PAR Cluster Extension resource.3. Double-click a disk in the summary pane.4. Click the Dependencies tab, and then click Insert.5. Select the HP 3PAR Cluster Extension resource in the Resource menu.

6. Click OK to add the selected dependency.


Adding dependencies using the CLIThe cluster commands in this section can be used with Windows Server 2003, Windows Server2008/2008 R2, Server Core, or Hyper-V Server.To add a dependency on a HP 3PAR Cluster Extension resource using the CLI, use the followingcommand:cluster resource physical_disk_resource/adddependency:Cluster_Extension_3PAR_resource

The following command adds a dependency on the HP 3PAR Cluster Extension clx_fileshareresource to the physical disk resource Disk_32b_00b:cluster resource Disk_32b_00b /adddependency:clx_fileshare

Disaster-tolerant configuration example using a file shareThe following example assumes that your environment includes:• Four nodes (host1_DCA, host2_DCA, host3_DCB and host4_DCB)

• Two HP 3PAR storage systems with Serial number 1301811 and 1301812

• A Remote Copy volume group configured as VG_CLX01

• An MNS arbitrator node in a third site or separate protected area

• A pre-executable clxpre.exe to be invoked by the softwareFigure 5 (page 39) illustrates failover options and shows a second cluster group CLX_IIS. Theresource CLX_FILESHARE is part of the service group CLX_SHARE and must be brought onlinebefore the physical disk resources Disk_3PAR_LUN25.

Figure 5 Service or application example

Disaster-tolerant configuration example using a file share 39

Figure 6 Resource tree for service or application CLX_SHARE

HP 3PAR Cluster Extension is configured as a single resource to enable read/write access to thephysical disk resource used for the CLX_SHARE cluster group. The physical disk resource dependson the HP 3PAR Cluster Extension resource and can be brought online only when the HP 3PARCluster Extension resource is already online. The network card is not attached to this resource treeand is configured with the CLX_SHARE service or application IP address and network name.The file share can be started when all the resources are online.To configure the resource according to the configuration shown in the Figure 5 (page 39):1. Log in to the host3_DCB system with the Administrator account.2. Create the file share service or application with all previously mentioned resources and its

dependencies, except the HP 3PAR Cluster Extension resource on host3_DCB.3. Create a new resource of type HP 3PAR Cluster Extension and add systems host2_DCA,

host3_DCB and host4_DCB to its possible owners.4. Change the restart behavior of the resource so that the resource can be restarted and that the

restart affects the service or application. Set the number of restarts to 0.5. Edit the properties in the Parameters tab window to configure your resource. Configure Remote

Copy volume group, the respective HP 3PAR storage mapping for DCA and DCB, theDC_A_Hosts, and the DC_B_Hosts.

6. (Optional step) Click the Pre/Post Exec button and add clxpre.exe with its full path. (Theclxpre.exe program is an example. It is not included with this product.

7. Add a dependency on the resource CLX_FILESHARE to the physical disk resourceDisk_3PAR_LUN25.

8. Check the cluster service group and resource settings using the following commands:C:\>cluster group CLX_SHARE /prop

C:\>cluster resource CLX_FILESHARE /prop

9. For Windows Server 2003 only:Set the resource property RestartAction to zero (0) or 2, or select the Do not restart checkbox in the resource's Advanced tab, and then verify that the value has changed. For example,C:\>cluster resource CLX_FILESHARE /prop RestartAction=0


C:\>cluster resource CLX_FILESHARE /prop

10. For Windows Server 2008/2008 R2 only:a. Click the Policies tab of the Failover Cluster Management Properties window.b. For the HP 3PAR Cluster Extension resource property response to resource failure, select

If resource fails, do not restart.11. Bring the service or application online on host3_DCB, and verify that the resource and all

other CLX_SHARE service or application resources are brought online.12. Take the service or application offline, and verify that all resources are stopped.13. Bring the service or application online again, and verify that all resources are available.14. Check the cluster service settings of nodes host4_DCB, the service or application and resource

settings.15. Move the service or application to node host4_DCB and verify that all resources are available.16. Check the cluster service settings of systems host2_DCA, the service or application and

resource settings.17. Move the service or application to node host2_DCA and verify that all resources are available.18. Check the cluster service settings of node host1_DCA, the service or application and resource

settings.19. Take the service or application offline and verify that all resources are stopped.20. Change the resource to be able to restart on another system.

Managing HP 3PAR Cluster Extension resourcesYou can manage resources by bringing them online and offline, or by deleting them.

Bringing a resource onlineResources are usually brought online automatically when the service or application is broughtonline. You might need to move the service or application to the node where you want to bringthe resource online.To bring resources online:

• For Windows Server 2008/2008 R2, use the GUI, MMC, or CLI.

• For Server Core or Hyper-V Server, use the CLI or the MMC.

• For Windows Server 2003, use the GUI or CLI.For more information on using this command, see your Microsoft documentation.

Taking a resource offlineResources are usually taken offline automatically when the service or application is taken offline.Taking a resource offline causes resources that depend on that resource to go offline.When taking resources offline:• For Windows Server 2008/2008 R2, use the GUI, MMC, or CLI.


• For Windows Server 2003, use the GUI or CLI.For more information on using this command, see your Microsoft documentation.

Deleting a resourceDeleting a running resource causes the resource and its dependents to go offline.

CAUTION: Deleting a running resource does not remove the resource_name.online file.

Managing HP 3PAR Cluster Extension resources 41

When deleting resources:• For Windows Server 2008/2008 R2, use the GUI or CLI.


• For Windows Server 2003, use the GUI or CLI.For more information on deleting resources, see your Microsoft documentation.

Using Hyper-V Live Migration with HP 3PAR Cluster ExtensionLive migration is a managed failover of VM resources. Live migration should be performed whenall of the solution constituents are in a healthy state, all the servers and systems are running, andall the links are up. Ensure that the underlying infrastructure is in a healthy state before performinglive migration.HP 3PAR Cluster Extension has the capability of discovering unfavorable storage-level conditionsfor performing live migration. In response to these conditions, HP 3PAR Cluster Extension will stopor cancel the live migration process and inform the user. This is accomplished with no VM downtime.For example, if live migration is initiated while VM data residing on the storage system is stillmerging and not in sync, HP 3PAR Cluster Extension will proactively cancel the live migration andinform the user to wait until the disks are fully in sync. Without this feature, live migration mightfail or the VM might come online in the remote data center with inconsistent data.The HP 3PAR Cluster Extension StatusRefreshInterval property, which you can configure in a UCFfor each application, specifies the time interval between consecutive array status gatheringoperations before the live migration to the target cluster node occurs. By adjusting this property,you can increase the probability of getting the correct HP 3PAR storage system status to ensure asuccessful live migration. The default StatusRefreshInterval value is 300 seconds. For moreinformation about configuring this property, see “Setting HP 3PAR Cluster Extension propertiesusing a UCF” (page 37).

NOTE: Using Hyper-V Live Migration with Cluster Shared Volumes is not supported with HP 3PARCluster Extension.

TIP: For more information about using Hyper-V Live Migration with HP 3PAR Cluster Extension,see the white paper Live Migration across data centers and disaster tolerant virtualizationarchitecture with HP StorageWorks Cluster Extension and Microsoft Hyper-V on the white paperswebsite: www.hp.com/storage/whitepapers.

Timing considerations for Windows ClusteringHP 3PAR Cluster Extension gives priority to storage system operations over cluster softwareoperations. If HP 3PAR Cluster Extension is invoked during a virtual volume synchronization/fullsynchronization operation, or is gathering information about the remote storage system, it waitsuntil the requested status information is reported. This ensures the priority of data integrity overcluster software failover behavior. However, this behavior can cause a failure of HP 3PAR ClusterExtension resources in some cases:• When HP 3PAR Cluster Extension communicates with the HP 3PAR storage system, online

operations could time out depending on the settings of the command time out configuration.• When a virtual volume resynchronization/ full synchronization is in progress, the software

waits until the Remote Copy volume group is in a synced state if the ApplicationStartupresource property is set to RESYNCWAIT. In case of resynchronization, the delta between thesource and destination virtual volume could be large enough for the resynchronization processto exceed the resource PendingTimeout value.

• The values of the ResyncWaitTimeout properties must be less than the configured resourcePendingTimeout value.


www.hp.com/storage/whitepapers

In general, because the failover environment is dispersed into two (or more) data centers, thefailover time cannot be expected to be the same as that in a single data center with a single shareddisk device. Therefore, the following values of the HP 3PAR Cluster Extension resource and theservice or application using that resource, must be adjusted, based on failover tests performed toverify the proper configuration setup: FailoverPeriod, RestartPeriod, PendingTimeout, LookAlive,and IsAlive.In addition, FailoverPeriod value of the service or application must be higher than the RestartPeriodvalue of the resource and both must be higher than the resource's PendingTimeout value. MicrosoftCluster provides two properties to adjust state change recognition or resolution:• IsAlive

• LookAlive

NOTE: For Windows Server 2008/2008 R2, the IsAlive and LookAlive properties are replacedby the Thorough Resource Health Check Interval and Basic Resource Health Check Intervalproperties.

Bouncing service or applicationHP 3PAR Cluster Extension will alternate (start and fail) between local nodes if the ApplicationStartupproperty has been set to FASTFAILBACK and no remote system is available until the service orapplication restart limit has been reached.

Creating array password fileHP 3PAR Cluster Extension uses HP 3PAR CLI commands to manage the HP 3PAR storage systemobjects. HP 3PAR CLI can be installed on the client machine to manage the HP 3PAR storagesystems remotely. Each CLI session requires a user name and an encrypted password for successfulcommand execution. Alternatively, a onetime password file can be generated for subsequent use.Generated password file is specific to the HP 3PAR storage system. However, it can be copied toother systems for use. HP 3PAR Cluster extension requires this password file for its operations. Foreach array managed by HP 3PAR Cluster Extension, appropriate password files must be generatedand configured during the HP 3PAR Cluster Extension configuration.After you log into the HP 3PAR command shell (using the valid username and password credentials),use following command to generate the password file:setpassword -u <user name> -saveonly -file <file name>

The generated password file is stored in the directory from where the command shell is launched.For example, if you use %HP 3PAR CLI PATH% to login to the command shell, the generatedpassword file location will be <HP 3PAR CLI PATH>\bin.

IMPORTANT: If there is any change in the configured IP address of the array or password filelocation and content, Cluster Extension will fail to communicate with the array and will not performthe required actions. At any point of time, you can launch the HP 3PAR Configuration tool anduse the Connection Test option to verify connectivity and array credentials, and take necessaryaction.

AdministrationCluster Extension administration includes remote management of resources and monitoring ofsystem resources and logs.

Bouncing service or application 43

Remote management of HP 3PAR Cluster Extension resources in a clusterYou can use the MMC with Failover Cluster Management to manage clusters and configure HP3PAR Cluster Extension resources. Note the following when configuring HP 3PAR Cluster Extensionresources by using the MMC from a remote management station:• When you use the MMC to remotely configure HP 3PAR Cluster Extension resource properties

in a Server Core or Hyper-V Server cluster node, the Failover Cluster Management GUI onthe remote management station displays the standard Microsoft Properties tab instead of thecustomized HP 3PAR Cluster Extension Parameters tab. For more information about theProperties tab, see “Setting HP 3PAR Cluster Extension resource properties using the MMC ”(page 36).

• When you install HP 3PAR Cluster Extension into a Windows Server 2008/2008 R2environment, the resource extension DLL is registered by default, which prevents you fromconfiguring a HP 3PAR Cluster Extension resource from a remote management station. If youneed to remotely configure a HP 3PAR Cluster Extension resource in a Windows Server2008/2008 R2-based cluster, unregister clx3PARmscsEx.dll from the cluster node, whichallows you to configure the HP 3PAR Cluster Extension resource using the standard MicrosoftProperties tab. Use the command cluster /UNREGADMINEXT:Clx3PARmscsEx.dll tounregister the DLL.

CAUTION: Configuring HP 3PAR Cluster Extension resources using the MMC from a remotemanagement station is supported using only the standard Microsoft Properties tab. Do not tryto use the customized HP 3PAR Cluster Extension Parameters tab for this purpose.

• If you see the customized HP 3PAR Cluster Extension Parameters tab when you try to configureHP 3PAR Cluster Extension resource from a remote management station using the MMC, youmust unregister clx3PARmscsEx.dll from the cluster node. Use the command cluster/UNREGADMINEXT:clx3PARmscsEx.dll to unregister the DLL. Unregistering the DLL allowsyou to configure the resource using the standard Microsoft Properties tab. This situation mightoccur if you have a cluster with both Server Core or Hyper-V Server and Windows Server2008/2008 R2 cluster nodes.

• When you configure HP 3PAR Cluster Extension resource properties from a remote managementstation through the MMC, which uses the standard Microsoft Properties tab, the propertiesyou enter are not validated, so you must enter the property values accurately, and verify themagainst the HP 3PAR Cluster Extension documentation.

System resourcesMonitor the system resources on a regular basis as part of Windows administration. If any systemresource usage by the cluster service is reaching maximum levels, stop and then restart the clusterservice. This action automatically fails over the resources and resets system resources. See theWindows cluster documentation for information about how to stop a cluster service.An alternate method is to manually move all resources to another node in the cluster before stoppingthe cluster service. After all resources are successfully moved to another node, stop and then restartthe cluster service; then, manually move back all resources.

LogsIf the HP 3PAR Cluster Extension log files need to be cleared and reset (for example, to reducedisk space usage), you can delete the files. HP 3PAR Cluster Extension automatically creates newlog files.

TIP: Archive the log files before deleting them.


Hyper-V Live Migration log entriesIn the HP 3PAR Cluster Extension log file (clxmscs.log), live migration messages include theprefix CLX_LM to help you differentiate live migration issues from HP 3PAR Cluster Extension logmessages.For example:[10/12/09 20:13:02][2136][CLX_LM: CLXVMDISK04-App01][INFO] CLX detectedthat Live Migration for VM "Virtual Machine VM04" has begun.

[10/12/09 20:13:02][2136][CLX_LM: CLXVMDISK04-App01][INFO] CLX startedgathering VM "Virtual Machine VM04" specific storage information.

Hyper-V Live Migration log entries 45

3 Configuring HP 3PAR Cluster Extension for LinuxHP 3PAR Cluster Extension supports integration with the following cluster software for Linux:• “Configuring HP 3PAR Cluster Extension with RHCS” (page 48)

• “Configuring HP 3PAR Cluster Extension with SLE HA” (page 55)

• “Configuring HP 3PAR Cluster Extension with HP Serviceguard on Linux” (page 58)

NOTE: For a list of HP 3PAR Cluster Extension versions and the cluster software versions theysupport, see the HP SPOCK website: http://www.hp.com/storage/spock.

HP 3PAR Cluster Extension for Linux: Sample configurationFigure 7 (page 46) shows a sample configuration with RHCS, HP Serviceguard, or SLE HA andHP 3PAR Cluster Extension.

Figure 7 Sample configuration

The configuration example in Figure 7 (page 46) assumes the following information about thecluster:• There are four nodes in the cluster: Host1, Host2, Host3, and Host4.

• The Remote Copy volume group clxwebdrg exists on the storage systems, and has virtualvolumes that are presented to all cluster nodes.

• HP 3PAR Cluster Extension may invoke the pre-executable script clxweb_pre_takeover.shand the post-executable script clxweb_post_takeover.sh. These files can be an executablescript or a program of your choice.

• For RHCS, the configuration file /etc/opt/hpclx3par/conf/CLX3PAR.config isassociated with the RHCS service CLXWEB that is configured to use the HP HP 3PAR ClusterExtension resource agent script. RHCS invokes the resource agent script to start the CLXWEBservice. The CLXWEB service checks the Remote Copy volume group member states before

46 Configuring HP 3PAR Cluster Extension for Linux

http://www.hp.com/storage/spock

the LVM volume groups vgweb and vghtdocs are activated and the web server is started.The HP 3PAR Remote Copy volume group clxwebdrg includes all virtual volumes for theLVM volume groups vgweb and vghtdocs. The sample CLX3PAR.config file shows thecontents of the configuration file with the described failover behavior.For SLE HA, the configuration file /etc/opt/hpclx3par/conf/CLX3PAR.config isassociated with the SLE HA resource CLXWEB. SLE HA invokes the resource agent script/usr/lib/ocf/ resource.d/heartbeat/CLX3PAR, which checks the Remote Copyvolume group member states before the LVM volume groups vgweb and vghtdocs areactivated and the web server is started. The Remote Copy volume group clxwebdrg includesall virtual volumes for the LVM volume groups vgweb and vghtdocs. The sampleCLX3PAR.config file shows the contents of the configuration file with the described failoverbehavior.For HP Serviceguard on Linux cluster, the configuration file /etc/opt/hpclx3par/conf/CLX3PAR.config is associated with the package CLXWEB that is configured to use the HP3PAR Cluster Extension package configuration script. HP SG invokes the package configurationscript to start the CLXWEB package. The CLXWEB package checks the Remote Copy volumegroup member states before the volume groups vgweb and vghtdocs are activated and theweb server is started. The 3PAR Remote Copy volume group, \Data Replication\clxwebdrg includes all vdisks for the LVM volume groups vgweb and vghtdocs. The sampleCLX3PAR.config file shows the contents of the configuration file with the described failoverbehavior.Sample CLX3PAR.config file.#COMMON#LogLevel info # error|info default: error (optional)# The APPLICATION part defines failover information for the application# specified as value of the APPLICATION tag. # For example, the command - clx3PARrun CLXTEST1 — will use the below # settings to check and allow access to the disk set specified as volume # Group CLX_VGTEST1. Example includes similar kind of settings for another # cluster group CLXTEST2.# Necessary object when clx3PARrun is used are:# APPLICATION, DC_A_InServStorage_SerNum, DC_B_InServStorage_SerNum, # RCVolumeGroupA, RCVolumeGroupB, DC_A_Hosts and DC_B_Hosts#APPLICATION CLXTEST1 # package/service/cluster group test_application#DC_A_InServStorage_SerNum 1302871 # 1302871 is the data center A HP 3PAR array serial number#DC_B_InServStorage_SerNum 1302835 # 1302835 is the data center B HP 3PARarray serial number#RCVolumeGroupA CLX_VGTEST1 # CLX_VGTEST1 is the data center AHP 3PAR Remote Copy volume group name#RCVolumeGroupB CLX_VGTEST1.r2835 # CLX_VGTEST1.r2835 is the datacenter B HP 3PAR Remote Copy volume group name#DC_A_Hosts clx_node1 clx_node2 # clx_node1, clx_node2 are data centre A cluster nodes#DC_B_Hosts clx_node3 clx_node4 # clx_node3, clx_node4 are data centre B cluster nodes#ApplicationStartup fastfailback # values: fastfailback | resyncwait#UseNonCurrentDataOk yes # values: yes | no #Default value is yes which causes the failover operation to succeed at the destination site when the data at the destination site might not be current. On setting the value to No the resource will not come online at the destination site since the data might not be current.

#APPLICATION CLXTEST2 # package/service/cluster group test_application#DC_A_InServStorage_SerNum 1302871 # 1302871 is the data center A HP 3PAR array serial number#DC_B_InServStorage_SerNum 1302835 # 1302835 is the data center B HP 3PAR array serial number#RCVolumeGroupA CLX_VGTEST2 # CLX_VGTEST2 is the data center A HP 3PAR Remote Copy volume group name#RCVolumeGroupB CLX_VGTEST2.r2835 # CLX_VGTEST2.r2835 is the data center B HP 3PAR Remote Copy volume group name#DC_A_Hosts clx_node1 clx_node2 # clx_node1, clx_node2 are data centre A cluster nodes#DC_B_Hosts clx_node3 clx_node4 # clx_node3, clx_node4 are data centre B cluster nodes#ApplicationStartup resyncwait # values: fastfailback | resyncwait#UseNonCurrentDataOk yes # values: yes | no #Default value is yes which causes the

HP 3PAR Cluster Extension for Linux: Sample configuration 47

failover operation to succeed at the destination site when the data at the destination site might not be current. On setting the value to No the resource will not come online at the destination site since the data might not be current.

• The service (RHCS), package (HP Serviceguard), or resource group (SLE HA) is configured towait for a volume group merge or full copy by setting the ApplicationStartup value toRESYNCWAIT.

• HP 3PAR Cluster Extension enables read/write access to the Remote Copy volume group usedfor the CLXWEB service. Activation of the volume groups depends on a successful return codefrom HP 3PAR Cluster Extension. The logical volumes can be mounted only when their volumegroups are active and HP 3PAR Cluster Extension allows read/write access to the disk group.After the file system for the web server's executable files and content data is mounted andchecked, the NIC is configured with the web server's IP address.

Configuring HP 3PAR Cluster ExtensionAfter installation, you must define the configuration setup using the HP 3PAR Cluster ExtensionConfiguration Tool for RHCS, HP Serviceguard, and SLES HA cluster software.

Defining HP 3PAR configuration information using the CLIYou can configure HP 3PAR Cluster Extension using the CLI command clx3PARconfig.Enter clx3PARconfig /?, or clx3PARconfig /help to view usage information.1. Use the following command to add arrays:

clx3PARconfig ARRAY /ADD [{NAME=<array network name|array IPddress>PWF=<password file path>}] [{NAME=<array network name | arrayIP address>PWF=<password file path>}]...]

2. Use the following command to add the available cluster nodes:clx3PARconfig SERVER /ADD [NAME=servername] [NAME=servername...]

3. Use the procedures in “Importing and exporting configuration information” (page 48) to copythe configuration information to the other cluster nodes.

Importing and exporting configuration informationThe import feature allows you to define the configuration setup using an existing configuration file.The export feature allows you to save a copy of an existing configuration.Use the import and export features to copy the HP 3PAR Cluster Extension configuration file(clx3PARcfg) from one cluster node to another.

Importing configuration settings using the CLI1. Open a command window.2. Enter clx3PARconfig IMPORT /FILE=filepath, where filepath specifies the file location

and name.

Exporting configuration settings using the CLI1. Open a command window.2. Enter clx3PARconfig EXPORT /FILE=filepath, where filepath specifies the file location

and name.

Configuring HP 3PAR Cluster Extension with RHCSHP 3PAR Cluster Extension Software is integrated with the RHCS using an RHCS shared resource.HP 3PAR Cluster Extension provides a resource agent script (clx3par.sh) that allows you tomanage HP 3PAR Cluster Extension resources.


The executable clx3PARlxcs is called by HP 3PAR Cluster Extension before volume groupactivation. This checks the status of a Remote Copy volume group associated with a volume groupconfigured for use with an RHCS service. If necessary, HP 3PAR Cluster Extension takes appropriateactions to allow access to the volume group's physical disks before attempting to activate andmount the logical volume on a cluster node.

Configuration overview1. Create an RHCS shared resource. For instructions, see “Creating an RHCS HP 3PAR Cluster

Extension shared resource” (page 49)2. Create an RHCS service using the HP 3PAR Cluster Extension shared resource. For instructions,

see “Creating an RHCS service using the HP 3PAR Cluster Extension shared resource” (page 50)

Creating an RHCS HP 3PAR Cluster Extension shared resourceAfter HP 3PAR Cluster Extension is installed, as described in the HP 3PAR Cluster ExtensionInstallation Guide, use Conga or the Cluster Configuration Tool (system-config-cluster) to create aHP 3PAR Cluster Extension shared resource. This procedure is required as part of the initial HP3PAR Cluster Extension configuration procedure. After you complete this procedure, you do notneed to repeat it when you add services. Use one of the following procedures:

• “Using Conga to create a shared resource” (page 49)

• “Using system-config-cluster to create a shared resource” (page 49)

Using Conga to create a shared resourceTo create a HP 3PAR Cluster Extension shared resource using Conga:1. Log in to Conga.2. Click the Cluster tab, and then select Cluster List.3. Click the name of the cluster you want to administer.4. Click Resources.5. Click Add a Resource.6. Select Script in the Select a Resource Type box.7. Enter a name for the HP 3PAR Cluster Extension shared resource in the Name box. For example:

CLX3PAR.8. Enter /usr/share/cluster/clx3par.sh in the Full path to script file box.

9. Click Submit.

Using system-config-cluster to create a shared resourceTo create a HP 3PAR Cluster Extension shared resource using system-config-cluster:1. Start system-config-cluster.2. Click the Cluster Configurationtab.3. Expand the Managed Resources tree.4. Select the Resources tree.5. Click Create a Resource to open the Resource Configuration dialog box.

Configuring HP 3PAR Cluster Extension with RHCS 49

6. Select Script in the Select a Resource Type box.7. Enter CLX3PAR in the Name box.8. Enter /usr/share/cluster/clx3par.sh in the File (with path) box.

9. Click OK.10. Select File > Save to save the configuration changes.

The service configuration in /etc/cluster/cluster.conf is updated.

11. Click Send to Cluster to propagate the cluster configuration to the other cluster nodes.

Creating an RHCS service using the HP 3PAR Cluster Extension shared resourceAfter you create a shared resource, create an RHCS service using the HP 3PAR Cluster Extensionshared resource.

Configuration overview1. Create a service at the root of the dependency tree using the HP 3PAR Cluster Extension shared

resource created in “Creating an RHCS HP 3PAR Cluster Extension shared resource” (page 49).This ensures that the HP 3PAR Cluster Extension resource is the first resource to start in aservice. All other resources in this service should be configured as child resources to HP 3PARCluster Extension. Use one of the following procedures:• “Using Conga to create a service” (page 50)

• “Using system-config-cluster to create a service” (page 52)2. Create a configuration file. For instructions, see “Creating the HP 3PAR Cluster Extension

configuration file” (page 53)3. Test the service configuration. For instructions, see “Testing the service configuration” (page 54)

Using Conga to create a serviceTo create a HP 3PAR Cluster Extension service using Conga:1. Log in to Conga.2. Click the Cluster tab, and then select Cluster List.3. Click the name of the cluster you want to administer.4. Click Services.


5. Click Add a Service.The Add a Service page appears.

6. Enter the service name in the Service name box.

IMPORTANT: The service name must match the name that is defined for the APPLICATIONproperty in the HP 3PAR Cluster Extension configuration file CLX3PAR.config.

7. Select a failover domain. For information about the failover domain requirements, see Failoverdomains .

8. Select Relocate for the recovery policy.9. Click Add a resource to this service to add the HP 3PAR Cluster Extension shared resource.

The Add a resource page appears.10. Select a HP 3PAR Cluster Extension shared resource from the Use an existing global resource

menu.


11. Click Submit.Conga saves the configuration information and updates all of the other cluster nodes.

NOTE: To add additional resources to the service, use the Add a child feature.

Using system-config-cluster to create a serviceTo create a HP 3PAR Cluster Extension service using system-config-cluster:1. Start the Cluster Configuration tool.2. Click the Cluster Configuration tab.3. Expand the Managed Resources tree.4. Select Services.

The Service properties page appears.5. Click Create a Service.

The Add a Service dialog box appears.6. Enter the service name in the Name box, and then click OK.

IMPORTANT: The service name must match the name that is defined for the APPLICATIONproperty in the configuration file CLX3PAR.config.

The Service Management dialog box appears.


7. Click Add a Shared Resource to this service.The Resource Configuration dialog box appears.

8. Select CLX3PAR in the Select a Resource Type menu, and then click OK.9. To add additional resources to the service, select the HP 3PAR Cluster Extension resource and

click Attach a new Private Resource to the Selection. Select the resource to be configured andprovide the required resource agent parameters.

10. Click Close to close the Service Management window.11. Select File > Save to save the configuration changes.

The service configuration in /etc/cluster/cluster.conf is updated.12. Click Send to Cluster to propagate the cluster configuration to the other cluster nodes.

Creating the HP 3PAR Cluster Extension configuration fileThe procedure in this section is based the sample configuration in “HP 3PAR Cluster Extension forLinux: Sample configuration” (page 46).Use this procedure as a guide for configuring your environment.1. Log in to system Host1 as root.2. Create the configuration file CLX3PAR.config in the /etc/opt/hpclx3par/conf

directory, by copying and editing the sample file CLX3PAR.config provided in the /opt/hpclx3par/ sample directory.$cp /opt/hpclx3par/sample/CLX3PAR.config /etc/opt/hpclx3par/conf/CLX3PAR.config


3. In the configuration file (CLX3PAR.config), enter the appropriate values for:• DC_A_InServStorage_SerNum

• DC_B_InServStorage_SerNum

• RCVolumeGroupA

• RCVolumeGroupB

• DC_A_Hosts

• DC_B_HostsSee “User configuration file” (page 65) for more information.For example:COMMON#LogLevel infoAPPLICATION CLXWEB#DC_A_InServStorage_SerNum 1302871#DC_B_InServStorage_SerNum 1302835#RCVolumeGroupA CLX_VG#RCVolumeGroupB CLX_VG.r2835#DC_A_Hosts clx_node1 clx_node2 # clx_node1, clx_node2 are data centre A clusternodes#DC_B_Hosts clx_node3 clx_node4 # clx_node3, clx_node4 are data centre B clusternodes#ApplicationStartup fastfailback # values: fastfailback | resyncwait#UseNonCurrentDataOk yes # values: yes | no

IMPORTANT: If you are using Device Mapper Multipath, configure themultipath_rescan.sh script as a PostExecScript. For more information, see “Rescanningmultipath devices” (page 62).

4. Copy the updated /etc/opt/hpclx3par/conf/CLX3PAR.config file to the other clusternodes.

Testing the service configurationThe procedure and commands in this section are based the sample configuration in “HP 3PARCluster Extension for Linux: Sample configuration” (page 46)Use this procedure as a guide for configuring your environment.1. Use the Cluster User Service Administration Utility (clusvcadm) to start the service on Host1.

#clusvcadm -e CLXWEB –m Host1

2. Verify that the service started successfully.#clustat –s CLXWEB

3. Stop the service and verify that the service stopped successfully.#clusvcadm –s CLXWEB Or #clusvcadm –d CLXWEB

4. Start the service on Host2.#clusvcadm –e CLXWEB –m Host2

5. Relocate the service to a remote data center node.a. Verify that the Remote Copy volume group is in destination mode on the remote HP 3PAR

storage system using the HP 3PAR Inform Management GUI or CLI.For more information, see HP 3PAR Remote Copy User Guide.

b. Relocate the service using the cluster administration command.#clusvcadm –r CLXWEB –m Host3


c. Verify that the Remote Copy volume group is in source mode on the remote storagesystem.

d. Stop the service and verify that the service stopped successfully.#clusvcadm –d CLXWEB

6. After verifying that the CLXWEB service can be moved with RHCS and that the softwareperforms automatic failover of the Remote Copy volume group on each cluster node, start theservice on its primary cluster node.#clusvcadm –e CLXWEB -m Host1

Managing HP 3PAR Cluster Extension services (RHCS)This section includes the instructions for starting or stopping an RHCS service.

Starting an RHCS serviceTo start a HP 3PAR Cluster Extension service using Cluster User Service Administration (clusvcadm),enter the following commands:#clusvcadm –e service name

#clusvcadm -e service name -m cluster node

For instructions on starting a HP 3PAR Cluster Extension service using Conga or the ClusterConfiguration Tool, see the RHCS documentation.

Stopping or disabling an RHCS serviceTo stop a HP 3PAR Cluster Extension service using Cluster User Service Administration (clusvcadm),enter the following command:#clusvcadm –d service name

To disable a HP 3PAR Cluster Extension service using Cluster User Service Administration(clusvcadm), enter the following command:clusvcadm -d service name

NOTE: For instructions on stopping or disabling a HP 3PAR Cluster Extension service using Congaor the Cluster Configuration Tool, see the RHCS documentation.

Configuring HP 3PAR Cluster Extension with SLE HAHP 3PAR Cluster Extension Software is integrated with SLE HA using a configuration file and acustom resource agent.The executable clx3PARlxcs is called by HP 3PAR Cluster Extension before volume groupactivation. This checks the status of a 3PAR Remote Copy volume group associated with a volumegroup configured for use with an SLE HA agent. If necessary, HP 3PAR Cluster Extension takesappropriate actions to allow access to the volume group's physical disks before attempting toactivate and mount the logical volume on a cluster node.

Configuration overview1. Create and configure a HP 3PAR Cluster Extension resource.

For instructions, see “Creating and configuring a HP 3PAR Cluster Extension resource”(page 55)

Creating and configuring a HP 3PAR Cluster Extension resourceUse the following procedure to create a HP 3PAR Cluster Extension SLE HA resource:

Configuring HP 3PAR Cluster Extension with SLE HA 55

1. Create the configuration file. For instructions, see “Creating the HP 3PAR Cluster Extensionconfiguration file” (page 56)

2. Create a HP 3PAR Cluster Extension resource using the SLE HA GUI. Use the procedure for“Creating a HP 3PAR Cluster Extension resource for Pacemaker” (page 56)

3. Test the configuration. For instructions, see “Testing the configuration” (page 58)

Creating the HP 3PAR Cluster Extension configuration fileCreate the HP 3PAR Cluster Extension configuration file. The procedure in this section is based onthe sample configuration in “HP 3PAR Cluster Extension for Linux: Sample configuration” (page 46).Use this procedure as a guide for configuring your environment.1. Log in to system Host1 as root.2. Create the configuration file CLX3PAR.config in the /etc/opt/hpclx3par/conf

directory, by copying and editing the sample file CLX3PAR.config provided in the /opt/hpclx3par/ sample directory.$cp /opt/hpclx3par/sample/CLX3PAR.config /etc/opt/hpclx3par/conf/CLX3PAR.config

3. In the configuration file (/etc/opt/hpclx3par/conf/CLX3PAR.config), enter theappropriate values for:• DC_A_InServStorage_SerNum


• RCVolumeGroupA

• RCVolumeGroupB

• DC_A_Hosts

• DC_B_HostsSee “User configuration file” for more information.For example:COMMON#LogLevel infoAPPLICATION CLXWEB#DC_A_InServStorage_SerNum 1302871#DC_B_InServStorage_SerNum 1302835#RCVolumeGroupA CLX_VG#RCVolumeGroupB CLX_VG.r2835#DC_A_Hosts clx_node1 clx_node2 # clx_node1, clx_node2 are data centre A clusternodes#DC_B_Hosts clx_node3 clx_node4 # clx_node3, clx_node4 are data centre B clusternodes#ApplicationStartup fastfailback # values: fastfailback | resyncwait#UseNonCurrentDataOk yes # values: yes | no

IMPORTANT: If you are using Device Mapper Multipath, configure themultipath_rescan.sh script as a PostExecScript. For more information, see “Rescanningmultipath devices” (page 62).

4. Copy the updated /etc/opt/hpclx3par/conf/CLX3PAR.config file to the other clusternodes.

Creating a HP 3PAR Cluster Extension resource for PacemakerThis procedure uses the Linux HA Management Client and Pacemaker. For specific instructions onusing the GUI, see the SuSE Linux Enterprise High Availability Extension documentation.1. Start the Linux HA Management Client.2. Select Add group from the Resources menu, and enter a group ID.


3. Add HP 3PAR Cluster Extension as a primitive group's first resource.

NOTE: The resource hierarchy depends on the order in which resources are added. Alwaysadd HP 3PAR Cluster Extension resources as the first resource in a group.

4. Select the following options for the HP 3PAR Cluster Extension resource:

ValueName

ocfClass

heartbeatProvider

CLX3PARType

5. Configure the instance attributes for the resource by selecting the app parameter. In the Valuebox, enter the APPLICATION tag name configured in the HP 3PAR Cluster Extensionconfiguration file (/etc/opt/hpclx3par/conf/CLX3PAR.config).

6. Configure the start, stop, and monitor operations for the HP 3PAR Cluster Extension resource.7. Add additional primitive resources to the group.

For example: If LVM and File System are used as the second and third resources of the group,the Summary dialog box is similar to the following:

8. Add a resource colocation constraint between the resource group ID assigned in Step 2 andthe last resource in the group hierarchy.

9. Set location constraints for the group ID to achieve the required failover order for the group.

Configuring HP 3PAR Cluster Extension with SLE HA 57

10. Set the operation defaults to control failover behavior. To specify that when a resource fails,the resources attempts to restart on the same node or another node in the cluster, use thefollowing settings:

ValueName

nothingrequires

restarton-fail

30timeout

11. Set the migration-threshold value. This value defines the number of failures that can occur ona node before the node becomes ineligible to host the resource and the resource fails over toanother node. Set this value to 1 for HP 3PAR Cluster Extension.

12. Disable automatic failback by using resource constraints and setting resource-stickiness to thelowest value compared with the other resource location constraints.

Testing the configurationThe procedure and commands in this section are based the sample configuration in “HP 3PARCluster Extension for Linux: Sample configuration” (page 46). Use this procedure as a guide forconfiguring your environment.Test the configuration by migrating the resource group to the remote data center nodes:1. Verify that the Remote Copy volume group is in destination mode on the remote storage system.2. In the SLE HA GUI, click Management in the left pane. Right-click the HP 3PAR Cluster Extension

resource and select Start.This will bring the resource group online on one of the cluster hosts, based the configuredresource constraints.

3. To migrate the resource, click Management in the left pane. Right-click the HP 3PAR ClusterExtension resource, and then select Migrate Resource. Select a target node in the remote datacenter in the Migrate Resource dialog box, and then click OK.

4. Verify that the Remote Copy volume group is in source mode on the remote storage systemafter a successful migration in Step 3.For more information, see HP 3PAR Remote Copy User Guide.

5. Migrate the resource to a node in the same data center and verify that the Remote Copyvolume group has not changed the volume group mode.

Managing HP 3PAR Cluster Extension services (SLE HA)To manage a HP 3PAR Cluster Extension resource:1. Click Management in the left pane of the Linux HA Management Client.2. Right-click the HP 3PAR Cluster Extension resource and select Start or Stop to automatically

initiate the requested operation on the each resource in the dependency tree. For moreinformation, see the SLE HA documentation.

Configuring HP 3PAR Cluster Extension with HP Serviceguard on LinuxHP 3PAR Cluster Extension Software is integrated with the HP Serviceguard cluster software onLinux using the customization scheme provided by Serviceguard. HP 3PAR Cluster Extensionprovides a package control script (clxsg.sh) that allows you to manage HP 3PAR Cluster Extensionpackages.The executable clx3PARlxcs is called by HP 3PAR Cluster Extension before volume groupactivation. This checks the status of a Remote Copy volume group associated with a volume groupconfigured for use with Serviceguard package. If necessary, HP 3PAR Cluster Extension takes


appropriate actions to allow access to the volume group's physical disks before attempting toactivate and mount the logical volume on a cluster node.Information on the latest version of HP Serviceguard for Linux is available at http://www.hp.com/info/sglx.For more information on cluster package configuration with HP Serviceguard for Linux, seeManaging HP Serviceguard for Linux on the HP High Availability website http://docs.hp.com/en/ha.html.For information on installing HP 3PAR Cluster Extension, see the HP 3PAR Cluster Extension SoftwareInstallation Guide.

Configuring HP 3PAR Cluster ExtensionYou can configure HP 3PAR Cluster Extension objects in an environment file for all the clusterpackages. This file must be located in the /etc/opt/hpclx3par/conf directory.You can configure a package to use HP 3PAR Cluster Extension by editing the package configurationfile.Use one of the following procedures to configure HP 3PAR Cluster Extension:• “Creating a new HP Serviceguard package and adding HP 3PAR Cluster Extension (CLI)”

(page 59)• “Adding HP 3PAR Cluster Extension to an existing HP Serviceguard package (CLI)” (page 61)

Creating a new HP Serviceguard package and adding HP 3PAR Cluster Extension (CLI)Before configuring the software parameters in the CLX configuration file, review the objects describedin “User configuration file and HP 3PAR Cluster Extension objects” (page 65)While integration with HP Serviceguard for Linux, CLX uses a single configuration file which canbe used to configure all the packages.The configuration procedure in this section is based on a sample configuration only. Use thisprocedure as a reference for configuring the packages and hosts. To configure HP 3PAR ClusterExtension as described in “HP 3PAR Cluster Extension for Linux: Sample configuration” (page 46),create the package Package_name and complete the following procedure.1. Log in to system Host1 as root.2. Source the HP Serviceguard for Linux environment variables before you use them.

#. /etc/cmcluster.conf

3. Create the package configuration file and edit its contents. For example#cmmakepkg -m sg/failover -m sg/priority -m sg/dependency -msg/weight -m sg/monitor_subnet -m sg/package_ip -m sg/service -msg/generic_resource -m sg/volume_group -m sg/filesystem -m sg/pev-m sg/external_pre -m sg/external -m sg/acp –m clx/clxsg >USER_DEFINED_DIR/USER_DEFINED_FILE

Here USER_DEFINED_DIR, is any path preferred by the user.USER_DEFINED_FILE is any file name preferred by the user.For details on how to create a package configuration file and package control file, seeManaging HP Serviceguard for Linux.

4. Copy the CLX3PAR.config file from the /opt/hpclx3par/sample/ to /etc/opt/hpclx3par/conf.#cp /opt/hpclx3par/sample/CLX3PAR.config /etc/opt/hpclx3par/conf

Configuring HP 3PAR Cluster Extension with HP Serviceguard on Linux 59

http://www.hp.com/info/sglx

http://www.hp.com/info/sglx

http://docs.hp.com/en/ha.html


5. In the CLX configuration file (CLX3PAR.config), enter the appropriate values for:• DC_A_InServStorage_SerNum


• RCVolumeGroupA

• RCVolumeGroupB

• DC_A_Hosts

• DC_B_HostsSee “User configuration file” for more information.For exampleCOMMON#LogLevel infoAPPLICATION CLXWEB#DC_A_InServStorage_SerNum 1302871#DC_B_InServStorage_SerNum 1302835#RCVolumeGroupA CLX_VG#RCVolumeGroupB CLX_VG.r2835#DC_A_Hosts clx_node1 clx_node2 # clx_node1, clx_node2 are data centre A clusternodes#DC_B_Hosts clx_node3 clx_node4 # clx_node3, clx_node4 are data centre B clusternodes#ApplicationStartup fastfailback # values: fastfailback | resyncwait#UseNonCurrentDataOk yes # values: yes | no

NOTE: The new CLX config file, CLX3PAR.config would contain information pertainingto all the packages.

6. For the parameter clx/clxsg/data_rep in the package configuration fileUSER_DEFINED_DIR/USER_DEFINED_FILE, change its value to clx3par.clx/clxsg/data_rep clx3par

7. Verify and apply the new package information to the cluster database.#cmapplyconf –v -P USER_DEFINED_DIR/USER_DEFINED_FILE

Here USER_DEFINED_FILE is the package configuration file which was created earlier.

8. Include a service entry for disk device or volume group monitoring if the package dependson monitored disks/volume groups. Edit the service parameters in the package configurationfile USER_DEFINED_DIR/USER_DEFINED_FILE similar to the following:service_name service_1aservice_cmd "/usr/local/cmcluster/bin/cmresserviced /dev/mapper/mpatha"

NOTE: For more information, see “Disk monitoring” (page 19)For multipath configuration, see “Rescanning multipath devices” (page 62)

9. Start the package on Host1.#cmrunpkg CLXWEB

10. Verify that the package has started successfully.cmviewcl –v –p CLXWEB

11. Halt the package and verify that all services for the package are halted.#cmhaltpkg CLXWEB

#cmviewcl –v –p CLXWEB


12. After creating the mount points for the shared directories on all configured nodes, start thepackage on Host2. The volume groups will be imported automatically during the first packagestart on each system.#cmrunpkg –n Host2 CLXWEB

13. Transfer the package to the remote data center nodes and verify that the Remote Copy volumegroup is in destination mode on the remote storage system.

14. After verifying that the package Package_name can be moved with HP Serviceguard forLinux, and that the software performs automatic failover of the Remote Copy volume groupon each cluster node, move the package to its primary cluster node.#cmrunpkg –n Host1 CLXWEB

#cmviewcl –v –p CLXWEB

NOTE: For configuration of failover packages' switching behavior and for configuring volumegroups and filesystems to be used in this package, see Managing HP Serviceguard for Linux onthe HP High Availability website http://docs.hp.com/en/ha.html.

Adding HP 3PAR Cluster Extension to an existing HP Serviceguard package (CLI)This procedure uses the CLI to add HP 3PAR Cluster Extension to an existing Serviceguard package.To integrate HP 3PAR Cluster Extension with an existing package:1. Copy the file CLX3PAR.config from /opt/hpclx3par/sample/ to /etc/opt/

hpclx3par/conf.#cp /opt/hpclx3par/sample/CLX3PAR.config /etc/opt/hpclx3par/conf

2. Add information pertaining to the package in the CLX config file, CLX3PAR.config, whichis located in the /etc/opt/hpclx3par/conf/ directory.

3. Change the parameters in the package config file based on your cluster and storage systemconfiguration.For more information, see “Creating a new HP Serviceguard package and adding HP 3PARCluster Extension (CLI)” (page 59)

4. For the DATA_REP parameter in the package configuration file, USER_DEFINED_DIR/USER_DEFINED_FILE, change its value to clx3par.clx/clxsg/data_rep clx3par

Here USER_DEFINED_FILE is the package configuration file which was created earlier forthis package and USER_DEFINED_DIR is the path in which the package configuration filewas created.

5. Update the package information in the HP Serviceguard for Linux cluster database.#cmapplyconf -v -P USER_DEFINED_DIR/USER_DEFINED_FILE

Starting an HP Serviceguard packageYou can start a package using the cmrunpkg package_name command. You can also configurea package to start automatically when a cluster node reboots (see Managing HP Serviceguard forLinux for details on automatic package startup.)To start a cluster package manually, you must enable the package to run on the cluster node.

NOTE: Ensure that the mount points for your file systems exist on each cluster node configuredto run the package.

To enable and start a package configured with HP 3PAR Cluster Extension, use the following HPServiceguard for Linux commands:cmmodpkg –e –n cluster_node package_name

Configuring HP 3PAR Cluster Extension with HP Serviceguard on Linux 61


cmrunpkg –v –n cluster_node package_name

Where cluster_node is the name of the cluster node running the package, and package_name isthe name of the package. The following commands enable and start the Serviceguard package,CLXWEB.#cmmodpkg –e -n Host1 CLXWEB

#cmrunpkg –v –n Host1 CLXWEB

Halting an HP Serviceguard for Linux packageIf there is a maintenance stop of the package, you must disable package switching.To disable and halt a package configured with HP 3PAR Cluster Extension, use the following HPServiceguard for Linux commands:cmmodpkg –d package_name

cmhaltpkg –v package_name

For example:#cmmodpkg –d CLXWEB

#cmhaltpkg –v CLXWEB

Removing HP 3PAR Cluster Extension from an HP Serviceguard packageYou can use the CLI to remove HP 3PAR Cluster Extension from a Serviceguard package, asdescribed in “Removing HP 3PAR Cluster Extension from an HP Serviceguard package (CLI)”(page 62)

Removing HP 3PAR Cluster Extension from an HP Serviceguard package (CLI)This procedure uses the CLI to remove HP 3PAR Cluster Extension from an existing Serviceguardpackage.To delete the HP 3PAR Cluster Extension software from a package:1. Halt the package and disable package switching.

cmhaltpkg package_name

cmmodpkg –d package_name

Where package_name is the name of the package. For example:cmhaltpkg CLXWEB

cmmodpkg –d CLXWEB

2. Remove the CLX config file, CLX3PAR.config, which is located in the /etc/opt/hpclx3par/conf/ package directory on all cluster nodes.

3. Comment out or change the value of clx/clxsg/data_rep in the package configurationfile USER_DEFINED_DIR/USER_DEFINED_FILE from clx3par to none, and then copy it tothe USER_DEFINED_DIR directory on all other cluster nodes. For example:clx/clxsg/data_rep none

Rescanning multipath devicesIMPORTANT: The information in this section applies to Device Mapper Multipath Software usersonly.

When a Remote Copy volume group failover occurs, the permission settings of the LUs in the RemoteCopy volume group change from read-only to read-write at the destination site. In Linuxconfigurations with the Device Mapper Multipath Software, the hosts do not dynamically detectthe LU permission change. In this situation, the disks used in the HP 3PAR Cluster Extension setup


fail to come online when the host OS does not detect the LU permission change. As a workaround,configure the HP 3PAR Cluster Extension script multipath_rescan.sh as a PostExecScript torescan the disks before they are brought online.

Configuring the rescan scriptThe multipath rescan script is available in the directory /opt/hpclx3par/sample/multipath_rescan.sh.To configure the script to run as a PostExecScript:1. Copy the multipath_rescan.sh script to the /etc/opt/hpclx3par/conf folder, and

rename the file as follows:• RHCS: multipath_rescan_ServiceName.sh

• SLE HA: multipath_rescan_ResourceGroupName.sh

• HP SG on Linux: multipath_rescan.sh2. Open the script file and enter the user-friendly names of all multipath devices that are in the

volume groups configured for the RHCS service, HP Serviceguard package, or SLE HA resourcegroup.For instructions on finding the user-friendly name of a multipath device, see “Finding theuser-friendly name of a multipath device” (page 63)In the following example, you specify the user-friendly names (mpathab, mpathac, andmpathad) for the variable MULTIPATH_DEVICES:MULTIPATH_DEVICES=( mpathab mpathac mpathad )

3. Enter the multipath_rescan.sh script for the PostExecScript object in the HP 3PAR ClusterExtension resource configuration file.You must specify the full path name of the multipath_rescan.sh script. For example:

Finding the user-friendly name of a multipath deviceThe multipath_rescan.sh script requires that you enter the user-friendly names of the multipathdevices.To obtain the user-friendly name of a multipath device:1. Run the pvs command to obtain the mapping between the physical volume and volume groups.

[root@node1 ~]# pvsPV VG Fmt Attr PSize PFree/dev/mpath/mpath18p1 vg06 lvm2 a- 5.00G 0

2. Run the ll /dev/mapper/ command to check the physical volume created for the multipathdevice[root@ Node1 ~]# ll /dev/mapper/

total 0crw------- 1 root root 10, 63 Nov 16 12:41 controlbrw-rw---- 1 root disk 253, 9 Nov 18 12:42 mpath18

Rescanning multipath devices 63

brw-rw---- 1 root disk 253, 17 Nov 18 12:42 mpath18p13. Run the multipath -ll command to view the multipath device names and SCSI ID for your

volume groups for Red Hat Enterprise Linux.[root@ Node1 ~]# multipath -ll

mpath18 (350002ac0192e0f50) dm-9 3PARdata,VV[size=5.0G][features=0][hwhandler=0][rw]\_ round-robin 0 [prio=2][active]\_ 1:0:0:8 sdh 8:112 [active][ready]\_ 1:0:1:8 sdp 8:240 [active][ready]

In the above example mpath18 needs to be put in the multipath_rescan.sh file.

Timing considerationsHP 3PAR Cluster Extension Software gives priority to storage system operations over cluster softwareoperations. If HP 3PAR Cluster Extension is invoked during a Remote Copy volume group groupmember resynchronization (merge), or is gathering information about the remote storage system,it waits until the requested status information is reported. This ensures the priority of data integrityover cluster software failover processes.This behavior can lead to failed resources:• When a Remote Copy volume group member resynchronization (merge) is in progress, the

software waits until the Remote Copy volume group is in a normal state if the ApplicationStartupresource property is set to RESYNCWAIT or until the resyncwaittimeout value.

• In an SLE HA environment, if the ResyncWaitTimeout object value is higher than the resourcestartup timeout value.

NOTE: Because the failover environment is dispersed over two or more data centers, the failovertime cannot be expected to be the same as that of a single data center with a single shared diskdevice. Therefore, you must adjust the resource startup timeout value and the resyncwaittimeoutCLX property value based on failover tests you perform to verify the proper configuration setup.In an SLE HA environment, the timeout values for the start operation can be adjusted. In an RHCSenvironment, the timeout value for the resource depends on the standard script resource agent(/usr/ share/cluster/script.sh) timeout values.In a Serviceguard for Linux environment, the timeout values for the start operation can be adjusted.


4 User configuration file and HP 3PAR Cluster Extensionobjects

Objects (also called properties in this document) define the HP 3PAR storage system environmentand failover/failback behavior. You can customize HP 3PAR Cluster Extension objects in the userconfiguration file or in the cluster software.

User configuration fileFor the CLI implementation of HP 3PAR Cluster Extension, the user configuration file UCF.cfg isused to configure application service-specific information. UCF.cfg describes the dependenciesbetween application services and HP 3PAR Remote Copy volume groups in one file for allapplication services in the cluster.A UCF.cfg file is available in the HP 3PAR Cluster Extension sample directory. This file can becopied and modified to suit a specific configuration.• Windows: %ProgramFiles%\Hewlett-Packard\Cluster Extension 3PAR\sample

• Linux:/opt/hpclx3par/sampleAfter the user configuration file has been created, it must be copied to the /conf directory on allcluster nodes.• Windows: %ProgramFiles%\Hewlett-Packard\Cluster Extension 3PAR\conf

• Linux: /etc/opt/hpclx3par/confFor more information, see “Sample configuration file” (page 72)and “Creating and configuringthe user configuration file” (page 75).

Windows ClusteringHP 3PAR Cluster Extension integration with Windows cluster does not require a user configurationfile when the standard environment for HP 3PAR Cluster Extension is used. The HP 3PAR ClusterExtension objects that are integrated with Windows cluster can be configured as resource privateproperties in the cluster software. See “Configuring HP 3PAR Cluster Extension for Windows”(page 21)for more information.

RHCS, HP Serviceguard and SLE HAHP 3PAR Cluster integration with RHCS and SLE HA uses a HP 3PAR Cluster Extension resourceconfiguration file. The objects and format in the configuration file are the same as the UCF.cfgfile.

File structureThe configuration file consists of a COMMON section and an APPLICATION section. These sectionsare distinguished by control tags. HP 3PAR Cluster Extension uses the following objects as controltags:• COMMON

• APPLICATIONObjects have one of the following formats:

A definition of an object; for example, COMMON or APPLICATIONtag

A number; for example, a timeout valueinteger

User configuration file 65

A name, which can include alphabetic and numeric characters and underscores; forexample, an application startup value

string

A list of space-separated strings, for example, a list of host names (lists of numbers arestored as lists of strings)

list

Text that is a comment starts with the pound (#) symbol and continues until the end of the line.Comments can start on a new line or be part of a line specifying an object.

Specifying object valuesYou do not need to change the default settings unless you want to change the degree of protectionfor your Remote Copy volume group members. If you change an object, you may need to changeadditional objects as well. For example, if RESYNCWAIT has been specified for theApplicationStartup object, the ResyncWaitTimeout object must be specified.Objects are supported according to the requirements or capabilities of the cluster software, asindicated in Table 3 (page 66).

Table 3 Common and application objects

LinuxWindows ClusterCLIObject

••COMMON

••LogDir

•••LogLevel

•••APPLICATION

•••ApplicationDir

•••ApplicationStartup

•ClusterNotifyCheckTime

•ClusterNotifyWaitTime

•••DC_A_Hosts+

•••DC_B_Hosts+

•••DC_A_InServStorage_SerNum+

•••DC_B_InServStorage_SerNum+

•••RCVolumeGroupA+

•••RCVolumeGroupB+

•••PostExecCheck

•••PostExecScript

•••PreExecScript

•••ResyncWaitTimeout

•StatusRefreshInterval

•••UseNonCurrentDataOk

••HP3PARCLICommandTimeout

+ Required• Supported

66 User configuration file and HP 3PAR Cluster Extension objects

COMMON objectsThe COMMON section is used to set the environment of HP 3PAR Cluster Extension.The COMMON tag can appear in the configuration file only once. The COMMON object doesnot require any value.Objects of the type COMMON can appear only one time. Those objects must be placed after theCOMMON tag in the configuration file.If the default values fit your environment, there is no need to specify them in the file.

COMMONFormat tagDescription Distinguishes between general (common) and application-specific objects.

LogDirFormat StringDescription (Optional) Defines the path to the HP 3PAR Cluster Extension log file.Defaultvalue

Linux/var/opt/hpclx3par/logWindows %ProgramFiles%\Hewlett-Packard\Cluster Extension 3PAR\log

LogLevelFormat StringDescription (Optional) Defines the logging level used by HP 3PAR Cluster Extension.Valid values error (default): Logs only error messages for events that are unrecoverable.

warning: Logs error messages and warning messages for events that arerecoverable.info: Logs error messages, warning messages, and additional information, suchas disk status.debug: Logs error messages, warning messages, info messages, and messages thatreport on execution status, useful for troubleshooting.

APPLICATION objectsThe APPLICATION section defines the failback behavior of HP 3PAR Cluster Extension for eachapplication service. APPLICATION is a multitag that can appear in the configuration file for eachapplication service using HP 3PAR Cluster Extension.The APPLICATION object requires the name of the application service as its value. The objectsspecified after an APPLICATION tag must appear only once per application. As with COMMONobjects, APPLICATION objects have predefined default values.HP 3PAR Cluster Extension uses the following rules to define objects:• If you use the default value, you do not need to specify the object.

• The pre-execution and post-execution functions in HP 3PAR Cluster Extension will not beprocessed if the associated object values are empty. This is the default setting.

To set APPLICATION object values for the CLI, modify the user configuration file.To set APPLICATION object values for MSCS, use the MSCS Cluster Administrator GUI (WindowsServer 2003) or Failover Cluster Management (Windows Server 2008/2008 R2)

COMMON objects 67

APPLICATIONFormat TagDescription Distinguishes between general and application-specific objects. Specify the name

of the application service. The format of its value is equivalent to a string value.

ApplicationDirFormat StringDescription Specifies the directory where HP 3PAR Cluster Extension searches for

application-specific files, such as the force flag or online file.If ApplicationDir is set to a nonexistent drive, HP 3PAR Cluster Extension is unableto create the online file and cannot bring the resource online.WindowsIf ApplicationDir is not set, HP 3PAR Cluster Extension uses the local%HPCLX_3PAR_PATH% values as defined in the registry.

Defaultvalues

Linuxonline file: /etc/opt/hpclx3parforce flag: /etc/opt/hpclx3par/confWindows%HPCLX_3PAR_PATH%

Files application_name.forceflag

CLX_application_name.online

If specified in a user configuration file, resource_name is the value of theAPPLICATION tag; otherwise, resource_name is the value of the HP 3PAR ClusterExtension resource name.

ApplicationStartup (Optional)Format StringDescription Specifies where a cluster group should be brought online.

The ApplicationStartup object can be customized to determine whether an applicationservice starts locally or is transferred back to the remote data center (if possible) tostart immediately without waiting for disk pair resynchronization. This object is usedonly if an application service has already been transferred to the secondary siteand no recovery procedure has been applied to the Remote Copy volume group(the Remote Copy volume group has not been recovered and is not in a normalstate). This process is considered a failback attempt without prior Remote Copyvolume group recovery.HP 3PAR Cluster Extension can detect the most current copy of your data based onthe Remote Copy volume group information. If HP 3PAR Cluster Extension detectsthat the remote HP 3PAR storage system has the most current data, it orders aresynchronization of the local disk from the remote disk, or it stops the startup processto enable the cluster software to fail back to the remote HP 3PAR storage system.If a resynchronization is ordered, HP 3PAR Cluster Extension monitors the progressof the copy process. If the application service was running on a secondary HP 3PARstorage system without a replication link, a large number of records may need tobe copied. If the copy process takes longer than the configured application startuptimeout value, the application startup will fail.Windows cluster


If the ApplicationStartup resource property is set to FASTFAILBACK, and theFailoverThreshold value is set to a number higher than the current number of clusteredsystems for the service or application, the service or application will restart onconfigured nodes until one of the following conditions is met:

• The resource is brought online in the remote data center.

• The resource failed because the FailoverThreshold value has been reached.

• The resource failed because the FailoverPeriod timeout value has been reached.

CAUTION: Disable subsequent automated failover procedures for recovery failbackoperations. If you want to use HP 3PAR Cluster Extension capabilities to recover asuspended Remote Copy volume group state during a failover, ensure that subsequentfailover operations are disabled in case the recovery attempt fails. To disablesubsequent failover attempts, set the FailoverThreshold to zero (0) until the recoveryoperation has finished successfully.

Valid values FASTFAILBACK (default)The cluster group is brought online in the remote data center (if possible) withoutwaiting for resynchronization. The application startup process is stopped locally andHP 3PAR Cluster Extension reports a data center error. Depending on the clustersoftware, the application service cannot start on any system in the local data center,and the cluster software transfers the application service back to the remote datacenter. Use this value to provide the highest level of application service availability.In a two-node cluster, this process does not work because the target failback systemis not available. In this case, the application service must be started manually, orthe ApplicationStartup object must be set to RESYNCWAIT.RESYNCWAIT

The online operation in the local cluster group must wait until the Remote Copyvolume group status is no longer in syncing state. HP 3PAR Remote Copy initiatesa resynchronization of the local virtual volume based on the remote virtual volume.HP 3PAR Cluster Extension monitors the syncing process. If no copy progress is madeafter configured ResyncWaitTimeout value, HP 3PAR Cluster Extension returns aglobal error. If RESYNCWAIT has been specified for the ApplicationStartup object,the ResyncWaitTimeout object must be specified, in case HP 3PAR Cluster Extensionshould wait for resynchronization changes for more or less than 90 seconds, whichis the default.

Files application_name.forceflag

CLX_application_name.online

If specified in a user configuration file, resource_name is the value of theAPPLICATION tag; otherwise, resource_name is the value of the HP 3PAR ClusterExtension resource name.

ClusterNotifyCheckTimeFormat IntegerDescription Specifies how often HP 3PAR Cluster Extension will check for VM live migration state

changes.Defaultvalue

10 seconds

APPLICATION objects 69

ClusterNotifyWaitTimeFormat IntegerDescription Specifies the amount of time that HP 3PAR Cluster Extension will monitor for VM live

migration state changes.Defaultvalue

5 seconds

DC_A_Hosts (Required)Format ListDescription This space-separated list defines the cluster nodes in data center A.

DC_B_Hosts (Required)Format ListDescription This space-separated list defines the cluster nodes in data center B.

DC_A_ InServStorage_SerNum (Required)Format StringDescription The serial number of the HP 3PAR storage system located in data center A.

DC_B_ InServStorage_SerNum (Required)Format StringDescription The serial number of the HP 3PAR storage system located in data center B.

RCVolumeGroupA (Required)Format StringDescription HP 3PAR Remote Copy volume group name on configured DCA HP 3PAR storage

system, containing the application service disk set.

RCVolumeGroupB (Required)Format StringDescription HP 3PAR Remote Copy volume group name on configured DCB HP 3PAR storage

system, containing the application service disk set.

NOTE: HP 3PAR InForm Management Console (IMC) will not provide details of RCVolumeGroupAand RCVolumeGroupB. Use HP 3PAR CLI command showrcopy groups to find out the exact namefor RCVolumeGroupA and RCVolumeGroupB. For Example, for a group clx_vg_01 displayedin the Management Console GUI, RCVolumeGroupA can be obtained from logging into DCA HP3PAR storage system command line and executing the command showrcopy groups.

PostExecCheck (Optional)Format StringDescription The PostExecCheck object is used to configure HP 3PAR Cluster Extension to gather

3PAR virtual volumes status information after the takeover procedure. That informationwill be passed to the post-executable. In case of a remote data center failure, it couldbe time consuming to gather that information, especially if your post-executable doesnot need any Remote Copy volume group status information. The arguments passedto the post-executable will include only the local disk status if the PostExecCheckobject is set to NO.


Valid values YES

NO (default)

PostExecScript (Optional)Format StringDescription Specifies an executable with its full path name to be invoked after the failover action

or failover procedure.You can include environment variables in path names.Do not specify arguments to be passed to the executable.

PreExecScript (Optional)Format StringDescription Specifies an executable with its full path name to be invoked before the failover

action or failover procedure.You can include environment variables in path names.Do not specify arguments to be passed to the executable.

ResyncWaitTimeout (Optional)Format IntegerDescription Specifies the value (in seconds) for checking the status of the Remote Copy volume

group resynchronization. It might take some time to resynchronize member virtualvolumes. You can set this value if the ApplicationStartup property is set toRESYNCWAIT.

Defaultvalue

90

StatusRefreshIntervalFormat IntegerDescription Specifies how often HP 3PAR Cluster Extension will gather HP 3PAR storage array

volume group information.Defaultvalue

300 seconds

UseNonCurrentDataOk (Optional)Format StringDescription When the replication link is down, this property specifies whether HP 3PAR Cluster

Extension can fail over to the destination site when the data at the destination sitemight not be current.If a HP 3PAR Remote Copy link goes down, and the resource is moved to the remotedata center:

• Setting this value to YES means that, regardless of whether or not the data iscurrent in the remote data center, the resource will come online.

• Setting this value to NO means that the resource will not come online on theremote data center since data might not be current.

Valid values YES (default)NO

APPLICATION objects 71

HP3PARCLICommandTimeoutFormat IntegerDescription Specifies timeout value (in seconds) for HP 3PAR storage system command execution.

The timeout value is used whenever HP 3PAR Cluster Extension communicates withthe HP 3PAR storage system. If the storage system is not responding within theconfigured timeout value, the command will time out and returns an error to theapplication.

Defaultvalue

90 seconds

Sample configuration fileThe following is a sample UCF.cfg file:COMMON#LogLevel info # error|info default: error (optional)#HP3PARCLICommandTimeout 60

# The APPLICATION part defines failover information for the application specified# as value of the APPLICATION tag. For example, the command - clx3PARrun CLXSer -# will use the below settings to check and allow access to the disk set specified# as volume Group CLX_VG.# Necessary object when clx3PARrun is used are:# APPLICATION, DC_A_InServStorage_SerNum, DC_B_InServStorage_SerNum, #RCVolumeGroupA, RCVolumeGroupB, DC_A_Hosts and DC_B_HostsAPPLICATION CLXSer # package/service/cluster group test_applicationDC_A_InServStorage_SerNum 1302871 # 1302871 is the data center A HP 3PAR array serial number#DC_B_InServStorage_SerNum 1302835 # 1302835 is the data center B HP 3PAR array serial number#RCVolumeGroupA CLX_VG # CLX_VG is the data center A HP 3PAR Remote Copy volume group name#RCVolumeGroupB CLX_VG.r2835 # CLX_VG.r2835 is the data center B HP 3PAR Remote Copy volume group name#DC_A_Hosts clx_node1 clx_node2 # clx_node1, clx_node2 are data centre A clusternodes#DC_B_Hosts clx_node3 clx_node4 # clx_node3, clx_node4 are data centre B clusternodes#ApplicationStartup resyncwait # values: fastfailback | resyncwait # Setting the fastfailback property will behave as resyncwait property when clx3parrun is used#UseNonCurrentDataOk yes # values: yes | no #Default value is yes which causes the failover operation to succeed at the destination site when the data at the destination site might not be current. On setting the value to No the resource will not come online at the destination site since the data might not be current.

NOTE: When you configure a UCF file, remove the # symbol for the properties you want toconfigure.


5 CLI commands and utilitiesUsing its CLI and command-line utilities, HP 3PAR Cluster Extension can be integrated with almostany cluster software for Windows operating systems.

CLI commandsThe HP 3PAR Cluster Extension CLI command clx3PARrun can be:• Used as a standalone utility

• Integrated into any cluster software or custom application for Linux and Windows

clx3PARrunDescription Use clx3PARrun to manually prepare the application service's

disk set before an existing application service start procedure isinvoked. You can use clx3PARrun in custom solutions and startupscripts.When using clx3PARrun, the status of the associated volumegroup is checked to ensure that access to the virtual volume(s) willoccur under data consistency and concurrency situations only.You must invoke clx3PARrun before the application service diskset can be activated; it is considered an online-only program.However, the CLI features provide the same disaster recoveryfeatures as the integrated versions of HP 3PAR Cluster Extension.

Syntax clx3PARrun [–version] [–forceflag] app_nameOperands Displays the HP 3PAR Cluster Extension version–version

–forceflag Force startupapp_name Application name configured in the UCF.cfg file

Example 1 This example assumes you have defined an APPLICATION tagnamed my-exchange in the UCF.cfg file, and have specified allnecessary objects, including the RCVolumeGroupA andRCVolumeGroupB objects, to map the Remote Copy volume groupto the application service my-exchange. HP 3PAR Cluster Extensionwill check the virtual volume set mapped to the application servicemy-exchange, run the necessary failover procedure, and generatea return code.# clx3PARrun my-exchange

Example 2 This example assumes you have defined an APPLICATION tagnamed my-exchange in the UCF.cfg file and have specified allnecessary objects, including the RCVolumeGroupA andRCVolumeGroupB objects, to map the Remote Copy volume groupto the application service my-exchange. HP 3PAR Cluster Extensionwill check the virtual volume set mapped to the application servicemy-exchange, and run the necessary failover procedure to enableread/write access to the virtual volumes.# clx3PARrun -forceflag my-exchange

CLI commands 73

Related information clx3PARrun expects only the app_name parameter, whichuniquely identifies an application service in the APPLICATIONsection of the user configuration file.

•

• To force a Remote Copy volume group failover, use the–forceflag option.

CAUTION: The forceflag option is an emergency switch usedto manually activate your virtual volume. If the forceflag optionhas been specified, HP 3PAR Cluster Extension will not check anyconsistency or concurrency rules before activating the virtual volumein the associated volume group.

Return codes clx3PARrun exits with one of the following return codes:

OK Application service can be started.0

ERROR_GLOBAL Application service could not start on any node in eithersite on either HP 3PAR storage system.

1

ERROR_DC Application service could not start on any node in the local siteon the local HP 3PAR storage system.

2

ERROR_LOCAL Application service could not start on this node.3

ERROR_REPLICATION_IO_NOT_STARTED Application service can be startedbut Remote Copy replication is not started for the Remote Copy volumegroup.

13

ERROR_REPLICATION_IO_STARTING Application service can be started butRemote Copy replication is being started for the Remote Copy volume group.

14

Configuring the CLIThe following configuration steps are required before using the HP 3PAR Cluster Extension CLI:1. “Creating the HP 3PAR Remote Copy environment” (page 74)2. “Configuring the HP 3PAR storage system” (page 74)3. “Installing HP 3PAR InForm Command Line Interface (CLI)” (page 75)4. “Creating and configuring the user configuration file” (page 75)

Creating the HP 3PAR Remote Copy environmentHP support personnel are trained to set up HP 3PAR Remote Copy environments. You can createand configure Remote Copy volume groups by either using the HP 3PAR InForm ManagementConsole GUI or CLI. For more information, see HP 3PAR Remote Copy User Guide.

Configuring the HP 3PAR storage systemTo configure the HP 3PAR storage system you must:• Create hosts

• Create virtual volumes and add a LUN presentation

• Create Remote Copy volume groups

• Add a LUN presentation to the secondary virtual volumesFor detailed procedures, see the HP 3PAR InForm Management Console User Guide or the HP3PAR InForm CLI Administrator’s Manual.

74 CLI commands and utilities

Installing HP 3PAR InForm Command Line Interface (CLI)HP 3PAR Cluster Extension communicates with the HP 3PAR storage system using the HP 3PARInForm Command Line Interface (CLI). Ensure that you install the supported version of HP 3PARCommand Line Interface (CLI) on a host where Clx3PARrun can be used.

Creating and configuring the user configuration fileThe CLI expects as an argument the name configured as the APPLICATION tag value.The following is an example of a customized user configuration file when using clx3PARrun:#COMMON#LogLevel info # error|info default: error (optional)# The APPLICATION part defines failover information for the application# specified as value of the APPLICATION tag. # For example, the command - clx3PARrun CLXTEST1 — will use the below # settings to check and allow access to the disk set specified as volume # Group CLX_VGTEST1. Example includes similar kind of settings for another # cluster group CLXTEST2.# Necessary object when clx3PARrun is used are:# APPLICATION, DC_A_InServStorage_SerNum, DC_B_InServStorage_SerNum, # RCVolumeGroupA, RCVolumeGroupB, DC_A_Hosts and DC_B_Hosts#APPLICATION CLXTEST1 # package/service/cluster group test_application#DC_A_InServStorage_SerNum 1302871 # 1302871 is the data center A HP 3PAR array serial number#DC_B_InServStorage_SerNum 1302835 # 1302835 is the data center B HP 3PARarray serial number#RCVolumeGroupA CLX_VGTEST1 # CLX_VGTEST1 is the data center AHP 3PAR Remote Copy volume group name#RCVolumeGroupB CLX_VGTEST1.r2835 # CLX_VGTEST1.r2835 is the datacenter B HP 3PAR Remote Copy volume group name#DC_A_Hosts clx_node1 clx_node2 # clx_node1, clx_node2 are data centre A cluster nodes#DC_B_Hosts clx_node3 clx_node4 # clx_node3, clx_node4 are data centre B cluster nodes#ApplicationStartup fastfailback # values: fastfailback | resyncwait#UseNonCurrentDataOk yes # values: yes | no #Default value is yes which causes the failover operation to succeed at the destination site when the data at the destination site might not be current. On setting the value to No the resource will not come online at the destination site since the data might not be current.

#APPLICATION CLXTEST2 # package/service/cluster group test_application#DC_A_InServStorage_SerNum 1302871 # 1302871 is the data center A HP 3PAR array serial number#DC_B_InServStorage_SerNum 1302835 # 1302835 is the data center B HP 3PAR array serial number#RCVolumeGroupA CLX_VGTEST2 # CLX_VGTEST2 is the data center A HP 3PAR Remote Copy volume group name#RCVolumeGroupB CLX_VGTEST2.r2835 # CLX_VGTEST2.r2835 is the data center B HP 3PAR Remote Copy volume group name#DC_A_Hosts clx_node1 clx_node2 # clx_node1, clx_node2 are data centre A cluster nodes#DC_B_Hosts clx_node3 clx_node4 # clx_node3, clx_node4 are data centre B cluster nodes#ApplicationStartup resyncwait # values: fastfailback | resyncwait#UseNonCurrentDataOk yes # values: yes | no #Default value is yes which causes the failover operation to succeed at the destination site when the data at the destination site might not be current. On setting the value to No the resource will not come online at the destination site since the data might not be current.

Timing considerationsHP 3PAR Cluster Extension gives priority to storage system operations over cluster softwareoperations. If HP 3PAR Cluster Extension is invoked during a virtual volumes of Remote Copyvolume group are in resynchronization or full synchronization operation or is gathering informationabout the remote storage system, it waits until the requested status information is reported. Thisensures the priority of data integrity over cluster software failover behavior. Because the failovertiming depends on the configuration of your Remote Copy environment, load on the HP 3PARstorage system and the settings in the UCF.cfg file, you must evaluate the following considerations:• HP 3PAR Cluster Extension uses HP 3PAR Command Line Interface (CLI) to communicate with

the HP 3PAR storage system, depending on the settings of HP 3PAR command timeout property,

Configuring the CLI 75

and the number of remote Copy volume groups managed by HP 3PAR storage systems, theonline operation could time out. This can also happen if clx3PARrun is used in a script orcalled by another program or 3PAR storage system IP cannot be reached.

• In the UCF.cfg file, if the ApplicationStartup attribute is set to RESYNCWAIT, HP 3PAR ClusterExtension tries to resynchronize virtual volumes and waits until the Remote Copy volume groupis in a synced state. Depending on the amount of data to be transferred, it could take hoursto resynchronize. If this is the case, clx3PARrun may take some time to return. Do not stopclx3PARrun, use the HP 3PAR Management Console GUI or CLI to check the progress ofsyncing.

• Depending on the amount of data that is out of sync between the source and destination virtualvolumes while the Remote Copy link(s) is unavailable, a resynchronization or a complete copyoperation is initiated when the Remote Copy link(s) is re-established. The amount of delta datato be transferred between the source and destination virtual volume could be large enoughfor the copy process to take some time.

• To prevent the failover timeout trigger from terminating a failover command before it is finished,the time to copy an average amount of daily processed (write) data should be measured andthe resource online timeout interval should be adjusted accordingly. Use only one (the slowest)link used for Remote Copy to measure the copy time. This ensures that the average dailygenerated data can be transferred fast enough from the remote HP 3PAR storage system, evenin the event of a single surviving replication link between the HP 3PAR storage systems. Ingeneral, because the failover environment is dispersed into two (or more) data centers, thefailover time cannot be expected to be the same as it would be in a single data center witha single shared disk device.

Restrictions for customized implementationsThe following restrictions apply when using the HP 3PAR Cluster Extension CLI:

• The HP 3PAR Cluster Extension CLI command clx3parrun must be invoked before theassociated disk resources are activated on a server.

• Associated disk resources must not be activated on any other system. If other disk resourcesare activated, HP 3PAR Cluster Extension may remove write-access rights for those disk devices(putting them in read-only mode).

Pre-execution and post-execution programsHP 3PAR Cluster Extension can invoke pre-execution and post-execution programs prior to or aftera HP 3PAR Cluster Extension failover/failback function. These programs can be any executable,and must be able to provide return codes to HP 3PAR Cluster Extension. If the programs addsignificant execution time to the application service startup process, the timeout values for thestartup process must be adjusted in the cluster software.HP 3PAR Cluster Extension transfers information as CLI arguments to the pre-execution andpost-execution programs. You must not specify CLI arguments. Pre-executables and post-executablesmust be specified by full path in the PreExecScript and PostExecScript objects. If no executable isspecified (there is an empty value for the object), no preprocessing or postprocessing is performed.If the object is not defined, then the argument value is empty.The following arguments are transferred to the scripts in this order:1. Resource name2. DC A HP 3PAR storage system serial number3. DC B HP 3PAR storage system serial number4. DC A Remote Copy volume group5. DC B Remote Copy volume group6. DC A host (server) names


7. DC B host (server) names8. Replication mode9. Local replication role10. Remote replication role11. Remote copy link status12. Failsafe policy13. Remote Copy volume group status14. Remote Copy volume group virtual volume status15. AutoRecover16. Application directory path (ApplicationDir object)17. Log file location (LogDir object)Pre-executables and post-executables must supply a return code. The return code is used to determinewhether a failover function is called.

Pre-execution return codes

DescriptionTextPre-execution return code

Pre-executable OK and failover action allowed.PRE_OK_TAKEOVER0

Pre-executable failed; no failover; stop application servicecluster-wide. Microsoft Cluster service does not allow

PRE_ERROR_GLOBAL1

cluster-wide or data-center-wide disabling of resources.This is local error behavior.

Pre-executable failed; no failover; stop application servicein this data center. MSCS does not allow cluster-wide or

PRE_ERROR_DC2

data-center-wide disabling of resources. This is local errorbehavior.

Pre-executable failed; no failover; stop application serviceon this system.

PRE_ERROR_LOCAL3

Pre-executable failed; failover action allowed.PRE_ERROR_TAKEOVER4

Pre-executable OK; no failover; no post-exec.PRE_OK_NOTKVR_NOPST5

CAUTION: If the pre-execution program returns 1, 2, 3 or 5, a properly configured post-executableis not executed. If a failover function fails, the post-executable is not executed.

Post-execution return codes

DescriptionTextPre-execution return code

Post-executable OK; continue.POST_OK0

Post-executable failed; stop application servicecluster-wide. MSCS does not allow cluster-wide or

POST_ERROR_GLOBAL1

data-center-wide disabling of resources. This is localerror behavior.

Post-executable failed; stop application service in thisdata center. MSCS does not allow cluster-wide or

POST_ERROR_DC2

data-center-wide disabling of resources. This is localerror behavior.

Post-executable failed; stop application service on thissystem.

POST_ERROR_LOCAL3

Post-executable failed; continue without error.POST_ERROR_CONTINUE4

Pre-execution and post-execution programs 77

CAUTION: Operating system scripts and/or batch files return 0 if the program was successfullyexecuted. To return a different value, use the following line as the last statement in your pre- orpost-executable: Windows batch file: exit /b exitcode.


6 TroubleshootingTo troubleshoot problems with HP 3PAR Cluster Extension, one must understand HP 3PAR RemoteCopy environments. See the Remote Copy User's Guide for more information on Remote Copyconfiguration and volume group states. See the HP 3PAR Cluster Extension log for any clusterspecific information. However, it always keeps its own log files in its default log location.

CAUTION: HP 3PAR Cluster Extension cannot handle Remote Copy volume group statesautomatically and correctly, if they result from manual manipulations (either through HP 3PARInForm GUI or CLI or through any other means).

LogsThe following logs are available for troubleshooting in Windows and Linux:

Windows logsThe following logs are available for troubleshooting in Windows:• Windows event logs

• Cluster related log messages logged by Windows Cluster (%ClusterLog%)

• Messages logged by Cluster Extension (%HPCLX_3PAR_PATH%\log\clxmscs.log)

• 3PAR storage system log file (sysmgr log file)Follow the order depicted in the flow diagram to troubleshoot problems.

Linux logsThe following logs are available for troubleshooting in Linux:• Cluster related log messages logged by Linux Cluster (/var/log/messages)

• Messages logged by Cluster Extension /var/opt/hpclx3par/log (clx3PARlxcs.log)

• 3PAR storage system log file (sysmgr log file)

Logs 79

Log facilityThe logging module of HP 3PAR Cluster Extension provides log messages to the cluster softwareas well as to the HP 3PAR Cluster Extension log file. The HP 3PAR Cluster Extension log file includesdisk status information and is located by default in this directory:

Windows%ProgramFiles%\Hewlett-Packard\Cluster Extension 3PAR\log\

Linux/var/opt/hpclx3par/log

If you need to clear and reset the log file, for example to reduce disk space usage, archive the logfile and then delete it. A new log file is automatically generated.Redhat cluster logs are available at /var/log/messages.By default, package logs in HP Serviceguard are available at /usr/local/cmcluster/run/log/package_name.log.

HP 3PAR Cluster Extension logs for WindowsThe logging module of HP 3PAR Cluster Extension provides log messages to the cluster softwareand to the HP 3PAR Cluster Extension log file. The HP 3PAR Cluster Extension log file includesRemote copy volume group status information.By default, the HP 3PAR Cluster Extension log file for Windows is located in the %ProgramFiles%\Hewlett-Packard\Cluster Extension 3PAR\log\ directory.To clear and reset the log file, for example, to reduce disk space usage, archive the log file andthen delete it. A new log file is automatically generated.The following shows a typical log entry and description of the fields, including timestamps,component IDs, and message IDs.

Sample log file

80 Troubleshooting

Line 1: The process begins for a Cluster Extension resource CLX03 using HP 3PAR Cluster Extensionversion number 2.00.00.Line 2: Cluster Extension internal configuration file version.Line 3 - 5: The warning message for a temporary license.Line 6: The host name where a Cluster Extension resource is being brought online in the data center(A or B).Line 7: The remote data center nodes status.Line 8-28 : Configuration details of the Cluster Extension resource before failover. The configurationdetails will be used to take failover decisions.In the current example, the state of the local Remote Copy (RC) group is “primary” (virtual volumeshaving read/write access) and the state of the remote RC group is “secondary” (virtual volumeshaving read-only access). The RC link status is UP and the replication status is started.Line 30: The status of the Cluster Extension takeover action.Line 31: The encoded Cluster Extension execution path.Line 32: Online status of Cluster Extension resources.Line 33: Creating <CLX resource name>.online file to check resource health.Line 34: End of the Cluster Extension execution process.

HP 3PAR Cluster Extension logs for LinuxThe logging module of HP 3PAR Cluster Extension provides log messages to the cluster softwareand to the HP 3PAR Cluster Extension log file. The HP 3PAR Cluster Extension log file includesRemote copy volume group status information.By default, the HP 3PAR Cluster Extension log file for Linux is located in the /var/opt/hpclx3par/log directory.To clear and reset the log file, for example, to reduce disk space usage, archive the log file andthen delete it. A new log file is automatically generated.The following shows a typical log entry and description of the fields, including timestamps,component IDs, and message IDs.

HP 3PAR Cluster Extension logs for Linux 81

Sample log file

Line 1: The process begins for a Cluster Extension resource CLX03 using HP 3PAR Cluster Extensionversion number 2.00.00.Line 2: Cluster Extension internal configuration file version.Line 3 - 5: The warning message for a temporary license.Line 6: The host name where a Cluster Extension resource is being brought online in the data center(A or B).Line 7: The remote data center nodes status.Line 8-28 : Configuration details of the Cluster Extension resource before failover. The configurationdetails will be used to take failover decisions.In the current example, the state of the local Remote Copy (RC) group is “primary” (virtual volumeshaving read/write access) and the state of the remote RC group is “secondary” (virtual volumeshaving read-only access). The RC link status is UP and the replication status is started.Line 30: The status of the Cluster Extension takeover action.Line 31: The encoded Cluster Extension execution path.Line 32: Online status of Cluster Extension resources.

82 Troubleshooting

Line 33: Creating CLX_application_name.online file to check resource health.Line 34: End of the Cluster Extension execution process.

HP 3PAR storage system log or sysmgr logA task, identified by task IDs is generated for each operation in the 3PAR storage system. Taskmanager commands provides information regarding the tasks in the system.For more information, see Managing Tasks section in HP 3PAR CLI Administrator’s Manual.

NOTE: Match the 3PAR storage system time with the cluster node time so that tasks can be easilymapped to Cluster Extension operations.

For detailed log messages from HP 3PAR storage system, do the following:1. Login with the 3parsvc user using HP 3PAR CLI.2. Run showsys.

This will indicate the master node on that system.3. On the CLI shell, enter copyfile <nodeid>:/var/log/tpd/sysmgr /tmp/xxxsysmgr.

If master node ID is 1, then copyfile 1:/var/log/tpd/sysmgr /tmp/sysmgr.4. In the above example, the file sysmgr will be copied to the <SystemDrive:>\tmp directory.

Error return codesHP 3PAR Cluster Extension provides the following error return codes for failover operations:

DescriptionError return codes

Prohibits an application service startup on the local system.This can be caused by the inability of HP 3PAR Cluster

local error

Extension to enable disk access, or the misconfigurationof the storage system environment.

Prohibits an application service startup on any system inthe local data center. This error is returned if the disk state

data center error

indicates that it makes no sense to allow any other systemconnected to the same storage system to access the disks.Windows clustering does not allow cluster-wide ordata-center-wide disabling of resources. This is local errorbehavior.

Returned if the configuration or the disk state do not allowan automatic application service startup process. Manual

global error

intervention is required in such cases. Windows clusteringdoes not allow cluster-wide or data-center-wide disablingof resources. This is local error behavior.

Replication I/O between the primary and secondaryRemote Copy volume group is not started. Use HP 3PAR

Replication I/O not started error

Management GUI or CLI to check if the replication statusis started or not. You can manually start the RemoteCopy replication using HP 3PAR Management GUI or CLIwhen required.

Replication I/O between the primary and secondaryRemote Copy volume group is started. Use HP 3PAR

Replication I/O starting error

Management GUI or CLI to check if the replication statusis started or not. You can manually start the Remote Copyreplication using HP 3PAR Management GUI or CLI whenrequired.

HP 3PAR storage system log or sysmgr log 83

When HP 3PAR Cluster Extension is integrated into the cluster software, an error message stringand integer value are displayed. For the CLI, a return code is displayed. For more information,see “CLI commands” (page 73).

Log filesLog files assist you in locating and identifying problems. Timestamps, component IDs, and messageIDs are included.

• HP 3PAR Cluster Extension-related messages are logged by MSCS to the file %ClusterLog%for Windows 2003. For Windows 2008, HP 3PAR Cluster Extension-related messages arelogged by MSCS to a file in the cluster log directory %SystemRoot%

• HP 3PAR Cluster Extension-related messages are logged by RHCS and SLE HA for each clusterresource to the file /var/log/messages.

• For each system, HP 3PAR Cluster Extension keeps its own log file in its default log location.

Log locationLog fileSystem

%HPCLX_3PAR_PATH%\logclx3PARmscs.logWindows

/var/opt/hpclx3par/logclx3PARlxcs.logLinux

/var/opt/hpclx3par/logclx3PARrun.logCLI

• HP 3PAR Cluster Extension messages can also be found in the following:

MSCS log file◦◦ RHCS, HP Serviceguard, and SLE HA: /var/log/messages

Always disable automatic service or application/package failover when merging orresynchronizing Remote Copy volume groups. A failure of the resynchronization source whileresynchronizing can lead to noncurrent data on the resynchronization target.

Start errors• In Windows, generic startup or initialization logic errors can occur when the path to the HP

3PAR Cluster Extension binaries has not been set in the PATH environment variable and theprocesses (for example the cluster service) have not recognized the %HPCLX_3PAR_PATH%variable. To fix this problem, restart the cluster service.

• A start error occurs if the APPLICATION name tag value in the HP 3PAR Cluster Extensionresource configuration file does not match the service name (RHCS) or the HP 3PAR ClusterExtension resource App (SLE HA) or the package name (HP Serviceguard).

• If the required components of HP 3PAR Cluster Extension are not available, a file (clxlog.log) is generated that can be used for troubleshooting. The file is usually in the working directoryof the operating system (%SYSTEMROOT%).

• HP 3PAR Cluster Extension returns a local error if any of the following occurs:

HP 3PAR Cluster Extension does not find the user configuration file for clx3PARrun inthe correct directory location.

◦

◦ HP 3PAR Cluster Extension does not find the HP 3PAR Cluster Extension resourceconfiguration file for RHCS, HP Serviceguard, or SLE HA integration in the correct directorylocation.

◦ An error occurs after initialization.

84 Troubleshooting

Failover error handlingWindows Cluster automatically fails over resource groups if the system where resource group isrunning on becomes unavailable. This is part of the cluster functionality. Also, this means that if aproblem occurs with the HP 3PAR storage system, a resource group online process will be stopped.The behavior of HP 3PAR Cluster Extension is highly configurable. Depending upon the customersetting, Cluster Extension is used to prevent resource groups from going online automatically underthe wrong conditions.Cluster Extension will return local, data center-wide or even cluster-wide errors to prevent accidentalaccess to the resource group’s virtual volumes. HP does not recommend restarting a failed resourcegroup without investigating the problem. A failed Cluster Extension resource indicates the need tocheck the status of the Remote Copy volume group and its member virtual volumes and decidewhether it is safe to continue or not.HP 3PAR Cluster Extension services, resources, or resource groups return a data center error andfail the resource if the Remote Copy volume group status indicates that the problem experiencedlocally would not be solved on another system connected to the same HP 3PAR storage system.Depending on the resource group and resource property values, the resource tries to start ondifferent nodes several times. If the remote data center is down, this would look like the resourcegroup is alternating between the surviving systems. This happens until the previously mentionedresource and resource group property values are reached or you disable the restarting of theresource. This could be also the case if the ApplicationStartup resource property has been set toFASTFAILBACK. If a 3PAR storage system state has been discovered that does not allow bringingthe resource group online on any system in the cluster, a cluster error would be reported and theresource would fail on all systems. This could lead to the same behavior as described for a HP3PAR Cluster Extension data center error.

Failing physical disk resources during online attempt of the resource groupWhen resource groups that use HP 3PAR Cluster Extension to failover Remote Copy volume groupare brought online, physical disk resources may fail due to the following reasons:• The physical disk resource does not have a dependency on its HP 3PAR Cluster Extension

resources/packages configured. Review the setup steps for HP 3PAR Cluster Extensionresources.

• The fibre channel path or connectivity between the servers and the storage systems may bebroken. So user has to review the FC connectivity between the servers and the storage systems.

Cannot connect to HP 3PAR storage systemDuring HP 3PAR Cluster Extension configuration, if you are unable to connect to the 3PAR storagesystem, ensure that the storage system is up and running, and the network ports are functioningproperly. To check for response from the storage system over the network, use the ping commandfrom cluster nodes to the storage system's network name or IP address.ping <storage system network name or IP address>

If you are using storage system network name, verify it is resolving to proper IP address usingnslookup command from the cluster nodes.nslookup <storage system network name>

Failover error handling 85

Resource disk failure when HP 3PAR Cluster Extension is onlineIn a cluster resource group, if the Cluster Extension resource is online but dependent disk resourceis failing:• Ensure that the fibre channel connectivity between the servers and the storage systems is up

and running.• Verify that the disk resource is dependent to the Cluster Extension resource.

• Ensure that the Remote Copy volume group managed by the Cluster Extension resource containsthe virtual volume which is correctly mapped to the configured disk resource.

• On a cluster node where the Cluster Extension resource is online, ensure that the presentedvirtual volume is having read/write access on the HP 3PAR storage system.

• Using HP 3PAR Management GUI, verify that the virtual volume is having read/write accesson HP 3PAR storage system where Cluster Extension resource is online.

• Rescan disks using Disk Management.

Cluster Extension and HP 3PAR Command Line (CLI) Install/UpgradeWhile installing or upgrading HP 3PAR Command Line Interface (CLI) on Windows, the environmentvariable Path will be updated with the CLI install directory. Occasionally, this change does nottake effect and you may have to re-login to Windows. HP recommends that you test the CLIcommands after you install or upgrade HP 3PAR CLI and try to re-login, if the commands fail dueto the binary not found error.

Change of HP 3PAR storage system IP or password fileIf there is change in the configured HP 3PAR storage system's IP address in the Cluster Extensionconfiguration or if the configured password file is missing or corrupt, Cluster Extension will fail tocommunicate with the storage system and cannot perform any Cluster Extension failover operations.You can use the Connection Test option of HP 3PAR Cluster Extension configuration tool to verifyconnectivity of configured storage systems. To change HP 3PAR storage systems settings, use theModify option in the HP 3PAR Cluster Extension configuration tool and save the configuration fileon all cluster nodes.

MNS needs its own resource monitorThe MNS quorum is a regular cluster resource that is monitored by the cluster resource monitor(ResrcMon.exe). To avoid cluster failures because of other hanging resources, HP recommendsthat the MNS resource run in its own resource monitor process. This can be changed by selectingthe MNS quorum resource in the Cluster Administrator GUI. Right-click and select Properties fromthe menu or double-click the MNS resource and select the box next to Run this resource in a separateResource Monitor, and then click OK.

Host persona settingsHost personas are a set of behaviors that permit hosts connected to FC or iSCSI ports on the HP3PAR storage system to deviate from the default host behavior. By assigning a persona to a host,multiple host types that require distinct customized responses can share a HP 3PAR storage systemport. For example, hosts running Windows, Linux, and AIX operating systems can all connect tothe same HP 3PAR storage system port.You can set the host persona number when the host is created or modify it later. To display, create,modify, and remove host personas, use the HP 3PAR CLI commands or the HP 3PAR ManagementConsole. For instructions on displaying, creating, modifying, and removing host personas, see theHP 3PAR InForm CLI Administrator’s Manual or the HP 3PAR InForm Management Console OnlineHelp.

86 Troubleshooting

NOTE: Host persona setting varies based on the Windows operating system installed on thecluster nodes, For more information, see the HP 3PAR Implementation Guide.

no_fail_wrt_on_err settingsThe no_fail_wrt_on_err policy apply only to synchronous Remote Copy.The no_fail_wrt_on_err group policy (default policy) allows writes to a primary Remote Copyvolume group even though the link between the systems in the Remote Copy pair is broken.Continuing to write changes despite a broken link leads to a loss of synchronization betweenprimary and secondary Remote Copy volume groups. The no_fail_wrt_on_err policy is usedwhen you would rather keep the applications that are writing the data running, even when theRemote Copy has failed and become out of sync. After the error that caused the broken link isresolved, you must manually restart the Remote Copy operation between the members of the RemoteCopy pair to bring the Remote Copy in sync again.

NOTE: HP 3PAR Cluster Extension supports only the Remote Copy volume groups withno_fail_wrt_on_err policy. For information on setting the Remote Copy volume group policies,see the “Remote Copy Commands” section in HP 3PAR Remote Copy User’s Guide .

Remote Copy and 3PAR Virtual DomainsRemote Copy checks HP 3PAR Virtual Domains (domains) on the remote system to ensure that thevirtual volume is mirrored to the same domain name as the local domain name. The domain needsto be called a correct domain name. For more information on domains, see the HP 3PAR ConceptsGuide and for instructions on setting up domains, see the HP 3PAR CLI Administrator’s Manual.

NOTE: When configuring the HP 3PAR storage system password file for Cluster Extension usage,user in the password file should have access to all the domains of Remote Copy virtual volumesmanaged by Cluster Extension.

Promote issueWhen the data copy is going on between the primary and the secondary Remote Copy volumegroups, if the Remote Copy link is broken for some reason, the Remote Copy volume groups goto the stopped state and the snapshots of the secondary volumes start getting promoted to thebase volumes. This is the design of 3PAR Remote Copy. This activity may take some time to complete.At this time, if the start or restore operation is attempted on the Remote Copy volume groups,then this operation may fail with the error Promote operation is going on. At this time, ifthe local replication role is secondary, the remote replication role is primary, and the remote copylink is up, Cluster Extension executes stop, reverse, and start operations for the Remote Copyvolume group. If the stop and reverse operation succeed, then the secondary volumes becomeread-write and Cluster Extension resource comes online even though the start operation mayfail. If the start operation fails, replication I/O does not start from the new primary volumes tothe new secondary volumes. In case of Windows OS, Cluster Extension will continuously attemptto start the group during the monitoring interval of the Cluster Extension resource. In case of RHELand SUSE, manual start of the group is necessary in order to resume the replication IO betweenthe primary and secondary RC volume groups. At this time, if the local replication role issecondary-rev, the remote replication role is primary-rev, and the remote copy link is up,the restore operation fails due to promoting of the snapshot, and the Cluster Extension resourcedoes not come online.

Cluster Extension Autopass troubleshootingCluster Extension uses Autopass as a framework for licensing checks. Autopass provides GraphicalUser Interface and a Command Line Interface to perform licensing specific operations, and theyare well integrated in to Cluster Extension. For the GUI, Autopass needs a compatible JRE version

no_fail_wrt_on_err settings 87

installed on the system. For the supported JRE version refer to Cluster Extension SPOCK. In casethe GUI is not working due to environmental issues related to JRE, CLI can be used to perform thelicensing specific operations like install and uninstall.

The FC link is down (RHCS)In RHCS, the detection of a storage outage due to failure of all paths to the storage depends onthe monitoring capability of resources configured in the RHCS service. For example, the LVM andfilesystem resource agents distributed with RHCS can detect the loss of storage and take appropriateactions. The stop operation on a service might fail due to the inability to stop individual resourcescleanly. This may be caused by the loss of paths to the storage. When the stop operation on aservice fails, RHCS marks the service as failed and the service does not automatically fail over toanother node.To recover from this situation, use the following procedure:1. Remove the node that lost access to the storage by shutting down the node.2. Follow the steps required to bring up a service in a failed state, as documented in the RHCS

administration guide. This process involves disabling the service, and then enabling it on thenode where the service is allowed to come online.

3. Restart the node that was shut down.

NOTE: The time to detect a storage outage due to failure of all paths to storage dependson the setting for no_path_retry in the multipath software configuration. A value of faildoes not queue I/O in the event of a failure in all paths and returns an immediate failure. Forinformation about the recommended value for your environment, see the DM-Multipathdocumentation.Some resource agents, such as LVM, offer a mechanism called self_fence to take themselvesout of a cluster through node reboot when an underlying logical volume can no longer beaccessed. For supported options, see the RHCS documentation.

A storage replication link is down (RHCS)If an HP 3PAR Cluster Extension configuration uses Remote Copy volume groups with failsafemodeenabled, the array disables access to the disk when it cannot replicate the I/O to the remote array.In this situation, if a replication link is broken, the resource agents of configured resources, suchas lvm or fs, may be able to detect and take appropriate actions. The stop operation on a servicemight fail due to the inability to stop individual resources cleanly because the disk is no longeraccessible for read/write operations. When the stop operation on a service fails, RHCS marks theservice as failed and the service does not automatically fail over to another node.To recover from this situation, use the following procedure:1. Remove the node that lost access to the storage by shutting down the node.2. Follow the steps required to bring up a service in a failed state, as documented in the RHCS

administration guide. This process involves disabling the service, and then enabling it on thenode where the service is allowed to come online.

A data center is down (SLE HA and RHCS)RHCS and SLE HA expect an acknowledgement from the fencing device before services are failedover to another node. In the event of complete site failure, including fencing devices, clusters donot automatically fail over services to surviving cluster nodes at the remote site. Manual interventionis required in this situation. For instructions on bringing a service online, see the cluster softwaredocumentation.

88 Troubleshooting

HP 3PAR Target arrays not configured with Remote Copy LinksSometimes CLX Configuration Tool is not able to add an array even through the password fileexists and PATH variable is set properly for the HP 3PAR Inform CLI and you can execute HP 3PARInform CLI command locally. The root cause for this problem is that the HP 3PAR Inform CLI commandshowrcopy for this array does not show any target information (in case if target array is notconfigured). So configure the 3PAR remote copy for the array first, then add the array to CLXconfiguration through CLX configuration tool.

HP 3PAR Target arrays not configured with Remote Copy Links 89

7 Support and other resourcesContacting HP

For worldwide technical support information, see the HP support website:http://www.hp.com/support

Before contacting HP, collect the following information:

• Product model names and numbers

• Technical support registration number (if applicable)

• Product serial numbers

• Error messages

• Operating system type and revision level

• Detailed questions

Related informationThe following documents [and websites] provide related information:

• HP 3PAR Cluster Extension Software Installation Guide

• HP 3PAR Cluster Extension Software Release NotesYou can find these documents on the Manuals page of the HP Business Support Center website:

http://www.hp.com/support/manuals

In the Storage section, click Storage software and then select your product.

WhitepapersThe following white paper is available at www.hp.com/storage/whitepapers:• Implementing HP StorageWorks Cluster Extension for Windows in a VMware Virtual Machine

NOTE: The above whitepaper is specific to HP P6000 (EVA) and HP P9000 (XP) storage systems,but the functionally is also applicable for HP 3PAR storage systems.

HP websitesFor additional information, see the following HP websites:

• http://www.hp.com

• http://www.hp.com/go/storage

• http://www.hp.com/service_locator

• http://www.hp.com/support/manuals

• Single Point of Connectivity Knowledge (SPOCK)

• http://www.hp.com/support/downloads

• http://www.hp.com/storage/whitepapers

90 Support and other resources

http://www.hp.com/support


www.hp.com/storage/whitepapers

http://www.hp.com

http://www.hp.com/go/storage

http://www.hp.com/service_locator


http://www.hp.com/storage/spock

http://www.hp.com/support/downloads

http://www.hp.com/storage/whitepapers

Typographic conventionsTable 4 Document conventions

ElementConvention

Cross-reference links and e-mail addressesBlue text: Table 4 (page 91)

Website addressesBlue, underlined text: http://www.hp.com

Bold text • Keys that are pressed

• Text typed into a GUI element, such as a box

• GUI elements that are clicked or selected, such as menuand list items, buttons, tabs, and check boxes

Text emphasisItalic text

Monospace text • File and directory names

• System output

• Code

• Commands, their arguments, and argument values

Monospace, italic text • Code variables

• Command variables

Emphasized monospace textMonospace, bold text

WARNING! Indicates that failure to follow directions could result in bodily harm or death.

CAUTION: Indicates that failure to follow directions could result in damage to equipment or data.

IMPORTANT: Provides clarifying information or specific instructions.

NOTE: Provides additional information.

TIP: Provides helpful hints and shortcuts.

Typographic conventions 91

http://www.hp.com

Glossary3PAR Remote Copy A product that allows you to copy virtual volumes from one HP 3PAR storage system to another.CLI Command-line interface. An interface comprised of various commands which are used to control

operating system responses.cluster A group of controller nodes connected via the same storage server backplane. The nodes in a

cluster operate as a unified system, separate from any other clusters that may share the sameservice processor.

CLX Cluster Extension.data distribution Pushing copies of data to geographic locations to make it more easily accessible to many

customers.data migration Moving data to a new location or to one with a larger capacity.fabric A network of Fibre Channel switches or hubs and other devices.failover A role-reversal process where the backup Remote Copy volume group is converted to the primary

Remote Copy volume group.heartbeat A periodic synchronization signal issued by cluster software or hardware to indicate that a node

is an active member of the cluster.high availability Redundant systems, software, and information technology processes to reduce the risk of downtime.

No single point of failure.LU Logical UnitMMC Microsoft Management Console.MNC Majority node set quorum. A quorum-capable resource introduced by Microsoft with Windows

Server 2003. A single quorum resource that allows you to build clusters with nodes that aregeographically separated.

MSCS Microsoft Cluster Service.primary volumegroup

A Remote Copy volume group that reside on the local or primary HP 3PAR storage system.

quorum In MSCS, a cluster resource that has been configured to maintain essential cluster data andrecovery information. If a node fails, the quorum acts as a tie-breaker and is transferred to asurviving node to ensure that data remains consistent within the cluster.

Remote Copy links The method by which information is sent and received between Remote Copy targets.secondary volumegroup

A Remote Copy volume group that reside on the remote or backup HP 3PAR storage system.

split-brainsyndrome

A state of data corruption can occur if a cluster is reformed as subclusters of nodes at each site,and each subcluster assumes authority, starting the same set of applications and modifying thesame data.

target The description of a Remote Copy system on one HP 3PAR storage system in the Remote Copypair.

virtual volume A virtual storage unit created by mapping data from one or more logical disks.volume group A group of virtual volumes that are logically related and for which there is a cross-volume ordering

of writes.WorldWide Name(WWN)

A 64- or 128-bit identifier that uniquely identifies the address of a component on the fabric.

92 Glossary

Index

Symbols3PAR Cluster Extension

components, 11features, 7integration, 7

3PAR Remote Copysupport for 3PAR Cluster Extension, 11volume groups, 12

AAPPLICATION

description, 67ApplicationDir

description, 68ApplicationStartup

description, 68AutoFailbackType

description, 31

BBasic Resource Health Check Interval

description, 29

CCLI

configuration, 7ClusterNotifyCheckTime

description, 69UCF requirement, 37

ClusterNotifyWaitTimedescription, 70UCF requirement, 37

clx3PARrun command, 73command line interface

configuring, 74restrictions, 76

components3PAR Cluster Extension, 11

configurationCLI, 7configuration tool Windows, 21GUI, 7Windows, 20

configuration informationexporting, 24importing, 24

configuration toolWindows, 21

contacting HP, 90conventions

document, 91text symbols, 91

DDC_A_ InServStorage_SerNum

description, 70DC_A_Hosts

description, 70DC_B_ InServStorage_SerNum

description, 70DC_B_Hosts

description, 70dependencies

adding (CLI), 39adding (Windows Server 2008/2008 R2), 38

disaster tolerance, 7document

conventions, 91related information, 90

documentationHP website, 90

Eexporting

configuration information, 24

FFailoverPeriod

description, 31FailoverThreshold

description, 31FASTFAILBACK value

description, 69features

3PAR Cluster Extension, 7CLI, 7disaster tolerance, 7GUI, 7integration with cluster software, 7redirection of mirrored disks, 8

filesconfiguration

Windows , 20force flag, 13user configuration, 13

Force flag, 13

GGUI

configuration, 7

Hhelp

obtaining, 90HP

technical support, 90HP 3PAR Remote Copy

synchronous replication, 13HP3PARCLICommandTimeout

description, 72Hyper-V Live Migration, 42, 45

93

Iimporting

configuration information, 24integration

with cluster software, 7IsAlivePollInterval

description, 28

Llive migration, 42, 45log files

Microsoft Cluster Service, 43MSCS, 44

LogDirdescription, 67

LogLeveldescription, 67UCF requirement, 37

LooksAlivePollIntervaldescription, 29

MMergeCheckInterval

UCF requirement, 37Microsoft Cluster Service

administration, 43, 44changing resource names, 27

Microsoft Management Console, 28mirrored disks

automated redirection, 8

Nnames

changing (Microsoft Cluster Service), 27changing (Windows Cluster), 27

Oobjects

APPLICATION, 67COMMON section, 67

PPendingTimeout

description, 30post-execution programs, 76

return codes, 77PostExecCheck

description, 70PostExecScript

description, 71pre-execution programs, 76

return codes, 77PreExecScript

description, 71

Qquorum considerations

Windows, 13

RRCVolumeGroupA

description, 70RCVolumeGroupB

description, 70related documentation, 90remote management, 28, 36

Windows Server 2008/2008 R2, 44resources

bringing online, 41deleting for MSCS, 41properties (CLI), 36properties (UCF), 37taking offline, 41

Response to resource failuredescription, 29

RestartActiondescription, 29

RestartPerioddescription, 30

RestartThresholddescription, 29

RESYNCWAIT valuedescription, 69

ResyncWaitTimeoutdescription, 71

return codespost-execution, 77pre-execution, 77

Sservice or application

bouncing, 43StatusRefreshInterval

description, 71UCF requirement, 37

storage systemsconsolidated volume group site configuration, 11one-to-one configuration, 9, 10supported configurations, 9

symbols in text, 91synchronous replication mode, 13

Ttechnical support

HP, 90service locator website, 90

text symbols, 91Thorough Resource Health Check Interval

description, 28typographic conventions, 91

UUseNonCurrentDataOK

description, 71user configuration file

ApplicationDir, 68ApplicationStartup, 68creating, 75

94 Index

DC_A_Hosts, 70DC_B_Hosts, 70FASTFAILBACK value, 69PostExecCheck, 70PostExecScript, 71PreExecScript, 71RCVolumeGroupA, 70RCVolumeGroupB, 70requirements, 13RESYNCWAIT value, 69ResyncWaitTimeout, 71sample, 72

Vvolume groups, 12

Wwebsites

HP , 90product manuals, 90

Windows Clusterchanging resource names, 27

95

Documents

HP 3PAR Cluster Extension Software Administrator Guidekmcs-service.austin.hp.com/km-ext/kmcsdirect/emr_na-c03393844-1.pdf · Configurationtool(clx3parconfig.exe)forWindows ... Thefailoverpackageisthemostcommontypeofpackage.Itrunsononenodeatatime.Ifa