HACMP Design

© 2005 Mt Xia Technical Consulting Group - All Rights Reserved.

HACMP – High Availability

Design and ConfigurationNovember, 2005

HACMP Design© 2005 Mt Xia Technical Consulting Group - All Rights Reserved.

Introduction

Scope

HACMP – Cluster Design

HACMP – Redundant Architecture

HACMP – Single Points of Failure (SPOF's)

Preparing for HACMP Configuration

HACMP Configuration


Scope

Scope

– This presentation covers HACMP standards, design and concepts in an example environment.

– This presentation is designed as a step by step procedure for the configuration of a 1-way cascading HACMP cluster running a database and application.


1-way Cascading Failover

– Two Node cluster, both nodes running– One active node, one hot-standby node– Both nodes running HACMP– Active node is running database and application

– Second node is waiting to takeover application processing if necessary

HACMP – Cluster Design


HACMP – Hot-standby configuration


Redundant network adapters on each node

– Gigabit ethernet adapter dedicated to database traffic

– Gigabit ethernet adapter dedicated to application traffic

– Gigabit ethernet standby adapter for failover of failed adapter

Systems

– One active node

– One standby node

Power

– Multiple connections to power source

Disk Array

– Raid 10 level protection of data

Application

– Multiple systems to provide business function

HACMP – Redundant Architecture


Example SPOF's

– Network switch (single network design)

• Switch failure will result in business function outage until switch is fixed or replaced

– Single storage communication adapter per node

• Adapter failure will cause node fail over with short outage to business function

HACMP – Single Points of Failure


Preparing for HACMP Configuration

Identify cluster name and node namesIdentify boot IP address for database adapterIdentify service IP address for databaseIdentify boot IP address for application adapterIdentify service IP address for applicationIdentify standby IP address for standby adapter

Create boot and service DNS names for databaseCreate boot and service DNS names for applicationCreate standby DNS nameCreate service DNS alias name for databaseCreate service DNS alias name for application


Preparing for HACMP Configuration (cont)

Review database startup script

– /usr/local/scripts/whmpctt0_appl01_start.shReview database shutdown script

– /usr/local/scripts/whmpctt0_appl01_stop.sh

Review application startup script

– /usr/local/scripts/whmpctt0_appl02_start.shReview application shutdown script

– /usr/local/scripts/whmpctt0_appl02_stop.sh



Download all latest fixes for the AIX operating system and HACMP via “suma”

– smitty suma

• Download

– All latest fixes

Commit all applied updates of current system

– smitty commit

Update the AIX operating system and HACMP with all downloaded updates

– smitty update_all

• Do not commit updates

• Save copy

• Accept all license agreements



Modify “/etc/hosts” on both nodes to contain

– IP Addresses

– Fully Qualified Domain Names (FQDN's)

– hostnames

172.19.30.17 ttc520-1a.mtxia.com ttc520-1a

172.19.30.33 ttc520-1b.mtxia.com ttc520-1b

172.19.30.1 ttc520-1a-boot.mtxia.com ttc520-1a-boot

172.19.30.2 ttc520-1b-boot.mtxia.com ttc520-1b-boot

172.19.30.3 ttc520-1c-boot.mtxia.com ttc520-1c-boot

172.19.30.4 ttc520-2a-boot.mtxia.com ttc520-2a-boot

172.19.30.5 ttc520-2b-boot.mtxia.com ttc520-2b-boot

172.19.30.6 ttc520-2c-boot.mtxia.com ttc520-2c-boot



Modify “/.rhosts” on both nodes to allow root access via– service and boot hostnames (short names)– service and boot Fully Qualified Domain Names (FQDN's)– Boot IP addresses– Service IP addresses

ttc520-1a rootttc520-1a.mtxia.com rootttc520-1b rootttc520-1b.mtxia.com root172.19.30.1 root172.19.30.2 rootttc520-1a-boot rootttc520-1a-boot.mtxia.com rootttc520-1b-boot rootttc520-1b-boot.mtxia.com root172.19.30.17 root172.19.30.33 rootttc520-2a-boot rootttc520-2a-boot.mtxia.com rootttc520-2b-boot rootttc520-2b-boot.mtxia.com root172.19.30.4 root172.19.30.5 root



Modify “/etc/netsvc.conf”on both nodes to resolve in the following order

– Local

– Bind

hosts = local, bind



Modify “/usr/es/sbin/cluster/etc/rhosts” on both nodes to contain the following

– service names for all resource groups in cluster

– boot names for both nodes in cluster

ttc520-1a

ttc520-1a-boot

ttc520-1b

ttc520-1b-boot

ttc520-2a-boot

ttc520-2b-boot


Create cluster and identify participating nodes

smitty hacmpInitialization and Standard ConfigurationAdd Nodes to an HACMP Cluster

Cluster name: ttc520-cl01Node names: ttc520-1 ttc520-2

HACMP Configuration


Create service IP labels

smitty hacmp

Initialization and Standard Configuration

Configure Resources to Make Highly Available

Add a Service IP/Label Address

IP/Label Address: ttc520-1a

Network Name: Press F4 and Select

HACMP Configuration


Synchronize the HACMP configuration to ensure correctness up to this point.

smitty hacmp

Extended Configuration

Extended Verification and Synchronizatoin

HACMP Configuration


Create resource group for database and application volume groups

smitty hacmp


Extended Resource Configuration

HACMP Extended Resource Group Configuration

Add a Resource Group

Resource Group Name: whmpctt0

Participating Nodes: ttc520-1a ttc520-2a

Startup Policy: Online on Home Node Only

Fallover Policy: Fallover to Next Priority Node

Fallback Policy: Fallback to Higher Priority Node

HACMP Configuration


Create resource group for the database service IP label

smitty hacmp





Resource Group Name: whmpctta





HACMP Configuration


Create resource group for the application service IP label

smitty hacmp





Resource Group Name: whmpcttb





HACMP Configuration


Configure the database and application resource group

smitty hacmp




Change/Show Resources and Attributes...

Select from list: whmpctt0

Volume Groups: whmpctt000vg

HACMP Configuration


Configure the database service IP label resource group

smitty hacmp





Select from list: whmpctta

Service IP Labels/Addresses: ttc520-1a

HACMP Configuration


Configure the application service IP label resource group

smitty hacmp





Select from list: whmpcttb

Service IP Labels/Addresses: ttc520-1b

HACMP Configuration


Arrange resource group activation in the following order

– whmpctt0

– whmpctta

– whmpcttb

This will start up the database and application then activate the database service label, and lastly, activate the application service label

HACMP Configuration



smitty hacmp


Extended Verification and Synchronization

HACMP Configuration


Add an application server to startup and shutdown the database

smitty hacmp



HACMP Extended Resources Configuration

Configure HACMP Applications

Configure HACMP Application Servers

Add an Application Server

Server Name: whmpctt0_appl01

Start Script: /usr/local/scripts/whmpctt0_appl01_start.sh

Stop Script: /usr/local/scripts/whmpctt0_appl01_stop.sh

HACMP Configuration


Add an application server to startup and shutdown the application

smitty hacmp



HACMP Extended Resources Configuration

Configure HACMP Applications

Configure HACMP Application Servers

Add an Application Server

Server Name: whmpctt0_appl02

Start Script: /usr/local/scripts/whmpctt0_appl02_start.sh

Stop Script: /usr/local/scripts/whmpctt0_appl02_stop.sh

HACMP Configuration


Modify the database and interface application resource group to include the application servers

smitty hacmp





Select from List: whmpctt0

Application Servers: whmpctt0_appl01 whmpctt0_appl02

HACMP Configuration



smitty hacmp


Extended Verification and Synchronization

Automatically Correct Errors: Yes

HACMP Configuration


Start HACMP on both nodes, one node at a time, allowing the cluster to stabilize in between

smitty hacmp

System Management

Manage HACMP Services

Start Cluster Services

Start now, on system restart, or both: now

Start Cluster services on these nodes: ttc520-1a

Broadcast message at startup: True

Startup cluster information daemon: True

Reacquire resources after forced down: False

Ignore verification errors: False

Automatically correct errors...: Interactively

HACMP Configuration


Start HACMP on both nodes, one node at a time, allowing the cluster to stabilize in between

smitty hacmp

System Management

Manage HACMP Services

Start Cluster Services

Start now, on system restart, or both: now

Start Cluster services on these nodes: ttc520-2a

Broadcast message at startup: True

Startup cluster information daemon: True

Reacquire resources after forced down: False

Ignore verification errors: False

Automatically correct errors...: Interactively

HACMP Configuration

Documents

HACMP Design