HACMP Configuration in AIX

CONFIGURATIONHACMP Step to Step guide

1. Convert 32 bit kernel to 64 bit kernel with JFS as default fils systemSteps: -Creating link in root#ln –fs /usr/lib/boot/unix_64 /unixCreating link at unix kernel location#ln –fs /usr/lib/boot/unix_64 /usr/lib/boot/unixPutting image on the device from which system boots i.e at cris its hdsik3.#bosboot –ad /dev/ipldevicebosboot creates boot image-d device Specifies the boot device.-a Creates complete boot image and device.#shutdown –Fr or reboot

2. Give IP, Hostname to local machines

3. Increasing Paging spacSteps: -#smit lvm - Paging Space -> Change / Show Characteristics of a Paging Space – > hd6 -> Just enter thenumber of additional LP’sEX: if current size of paging is 512 MB and you want to increase to 32 GB, then you have to calculatehow many additional PP’s are required to be 32 GB.Size of PP for rootvg is 128 mb, our 1 LP is equal to 1 PP therefore LP size is 128 MB each.LP = Total disk size / PP size32000/128250 LPSince paging is of 512 MB, it uses 4 PP where each pp is of 128 MB.Therefore additional LP’s we required to creaste 32 GB paging is250 – 4 PP’s ( 512 mb current paging size)246 LP’s are required to be 32 GB of paging space

4. Timming the network interface for concurrency OR Creating EtherchannelFor this we have to create a etherchannel and in that we have to add 2 network interfaces,ehterchannel is like a creating a single group of network by assiging a single IP.Step a: -Change the setting of Network adapters to 10/100/100/ MBPS Full/Half duplex. Please verifywhat setting is to be keep from network administrator.Steps below are the conversion from Auto-Negotitiion mode to 100 MBPS Full Duplex, you canhave your own seeting also,If the network adapter you choosen for etherchannel is up and running, make it down andremove it from system or else use the adapters which are not in use currently.#ifconfig –aen4:flags=5e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAS

T,GROUPRT,64BIT,CHECKSUM_OFFLOAD,PSEG,CHAIN>inet 192.168.100.12 netmask 0xffffff00 broadcast 192.168.100.255inet 10.129.1.12 netmask 0xffffff00 broadcast 10.129.1.255UP means network adapter en4 is up and running.#ifconfig network adapter name i.e en0, en2 downor#ifconfig en2 down#ifconfig <network adapter name i.e en0, en2> detachSmitty Mode of operation: -smit ethernet -> Adapters -> Change / Show Characteristics of an Ethernet Adapter ->select ent0 or ent1 or ent3, of whom you want to change settings.Select media speed=100_Full_Duplexand press enterCommand Mode of operation: -Network media speed options available can be check usiong below command.bash-3.00$ lsattr -E -l ent2 -a media_speed -R10_Half_Duplex10_Full_Duplex100_Half_Duplex100_Full_DuplexAuto_NegotiationI want to change to 100 MBPS Full duplex, below is the command#chdev -l ent2 -a media_speed=100_Full_DuplexSteps b: -Smit devices -> Communicaation -> EtherChannel / IEEE 802.3ad Link Aggregation -> Add An EtherChannel / Link Aggregation ->Or use Smitty Fast path#smit etherchannelSelect ent0 and ent2 i.e 1 port from network PCI card and 1 port from onboard network interface. In case of any error i.e you are working currently on ent0 netowork card, then you cant use it for etherchannel since its in use. So go for second port from PCI card and any port from onboard.Select ent1 and ent2> ent1 │> ent2You will get this screen

EtherChannel / Link Aggregation Adapters ent1,ent2 Just press enter, it will say ent4 is created.Step c: -Know start the etherchannel it will name it as ent4 if available, ent5 or anyOur etherchannel logical name is ent4.First check the name of your etherchannel.#Smit etherchannel - > List All EtherChannels / Link Aggregations – > EtherChannel / Link Aggregation: ent4Status: AvailableAttributes:adapter_names ent0,ent2 EtherChannel AdaptersFrom above output you came to know that your etherchannel is ent4 which consist 2

adapters ent0 and ent2.Know configure the newly created etherchannel adapter ent4.Smit mktcpipà select en4 à Enter the below information given by your networkadministrator.* HOSTNAME [utscstm2]* Internet ADDRESS (dotted decimal) [192.168.100.12]Network MASK (dotted decimal) [255.255.255.0]* Network INTERFACE en4NAMESERVERInternet ADDRESS (dotted decimal) []DOMAIN Name []Default GatewayAddress (dotted decimal or symbolic name) []Cost [0]Do Active Dead Gateway Detection? noYour CABLE Type N/ASTART Now YesPress enter and do similar setting on primary/secondary server.Step d: -Go to /etc/hostsEntry will be there: – 192.168.100.11 utscstm1 change utscstm1 to utsvcstm1_bootInternet Address Hostname # Comments192.168.100.11 utscstm1_boot192.168.100.12 utscstm2_boot10.129.1.11 utscstm1_svc utscstm110.129.1.12 utscstm2_svc utscstm2Put the entry in .rhosts which is in /utscstm1utscstm2utscstm1_bootutscstm2_bootutscstm1_svcutscstm2_svcDo the same above for secondary or other server.Test both the server by removing cables one by one for concurrency.Etherchannel is created know move to the installation of Cluster software and configuration.

CLUSTER PRE-REQUISTE

Before installing cluster software, use installp and preview the cluster cd content, if any pre-requitesfilesets are required install it first from IBM AIX O.S CD.Install HACMP software on both the server, this cluster requires the following filesets*.adt.libm 5.1.0.0, *.adt.syscalls 5.1.0.0, *.adt.data 5.1.0.0, *.rsct.compat.client.HACMP 2.3.1.0,*.rsct.compat.basic.HACMP 2.3.1.0, *.rsct.compat.client.HACMP 2.2.1.30, *.rsct.compat.client.HACMP2.2.1.30,1. First Create Disk based/ NON IP based heartbeat.

STEP IEnsure that the disks to be used for disk heartbeating are assigned and configured toeach cluster NODE.Enter: -lspv -> ensure that a PVID is assigned to the disk on each cluster NODEIf a PVID is not assigned, run one of the following commands:chdev -l hdisk8 -a pv=yes

STEP IICreate heartbeat as VG smit VG à Add original vg à vg Name = heartbeatvg,PP size =8 MB, PV name =hdisk8, create VG concurrency =yesFirst verify the major number of heartbeatvg#ls –l /dev/heartbeatvgyou will get major number as 45 , 46, ………if its 46 the execute this command on secondary server# importvg –V 46 –y heartbeatvg hdisk8ORCreate an enhanced concurrent mode volume group on the disk or disks in question usingSMIT. Enter: smitty HACMP -> System Management (C-SPOC) -> HACMP Concurrent Logical Volume Management – > Concurrent Volume Groups -> Create a Concurrent Volume Group (with Datapath Devices, if applicable) ->Press F7 to select each cluster NODE. Select the PVID of the disk to be added to the Volume Group. Enter the Volume Group Name, Desired Physical Partition Size, and major number.Enhanced Concurrent Mode should be set to True.

STEP III (Skip this step if already done).Put the entry of boot-IP’s and service-IP and there label int /etc/hosts192.168.100.11 utscstm1_boot192.168.100.12 utscstm2_boot10.129.1.11 utscstm1_svc utscstm110.129.1.12 utscstm2_svc utscstm2Put the entry in .rhostsutscstm1utscstm2utscstm1_bootutscstm2_bootutscstm1_svcutscstm2_svc.STEP IVCopy the .rhosts file to from root to the below location#cp /.rhosts /usr/es/sbin/cluster/etc/rhostsCreate same above for secondary server also you can just import the vg that is created inprimary serverCreate a diskhb network. Enter: -smitty HACMP -> Extended Configuration -> Extended Topology Configuration -> Configure HACMP Networks -> Add a Network to the HACMP cluster -> Enterwhen prompted to select, Choose diskhb.Enter the network name or accept the default.

Network Name net_diskhb_01* Network Type [diskhb]

STEP VAdd each disk-NODE pair to the diskhb network. Enter:smitty HACMP -> Extended Configuration -> Extended Topology Configuration -> Configure HACMP Communication Interfaces/Devices -> Add Communication Interfaces/Devices -> Add Pre-Defined Communication Interfaces and Devices -> Communication Devices -> Choose your diskhb Network Name.For Devices Name, enter a unique name; For device path, enter /dev/vpath# or /dev/hdisk#;For NODEname, enter the NODE on which this device resides.Repeat step VI for second NODE in the cluster.Device name :- utscstm1_hbNewtwork type :- diskhbNetwork name :- net_diskhb_01Device path :- /dev/hdisk8NODE Name :- utscstm1

STEP VIVerify communication of heartbeat from both the servers.Run the following command on the first NODE to put it in Receive Mode:#/usr/sbin/rsct/bin/dhb_read -p hdisk8 -r (replace hdisk# with rvpath# if using SDD)The following should be displayed:Receive Mode:Waiting for Response . . .Run the following command on a different NODE to put it in Transmit Mode:#/usr/sbin/rsct/bin/dhb_read -p hdisk8 -t (replace hdisk# with rvpath if using SDD)If communication is successful, the following should be displayed:Link operating normally.Or#/usr/sbin/rsct/bin/dhb_read –p –rhdisk8 –r ( on main server }#/usr/sbin/rsct/bin/dhb_read –p –rhdisk8 –t ( on secondary server)========================= DISK BASED HEART BEAT CREATION DONE =================GENERAL CONFIGURATION OF CLUSTER/HACMP

Step1:Add entries in rhosts, clhosts filesEdit Files rhosts, clhosts, clhosts.clientUse Commands to copy

First check files

# cd /usr/es/sbin/cluster/etc

/usr/es/sbin/cluster/etc=>ls –lrt

total 296

-

-rwxr-xr-x 1 root system 0 Aug 02 15:34 clhosts.client.orig

-rwxr-xr-x 1 root system 78 Aug 06 18:47 clhosts.client.08_06_2011

-rwxr-xr-x 1 root system 181 Aug 06 18:47 clhosts.client

-rwxr-xr-x 1 root system 7 Aug 07 13:31 auto_versync.pid

-rw-r–r– 1 root system 12 Aug 07 13:31 auto_versync.newpid.running

-rw-r–r– 1 root system 12 Aug 07 13:31 auto_versync.newpid

-rw——- 1 root system 0 Aug 07 14:48 rhosts

drwxr-xr-x 3 root system 256 Aug 07 14:48 config

-rwxr-xr-x 1 root system 3749 Aug 07 14:48 clhosts

srw——- 1 root system 0 Aug 07 14:48 ha_sec

/usr/es/sbin/cluster/etc=>sudo cp clhosts.client rhosts /usr/es/sbin/cluster/etc=>sudo cp clhosts.client clhosts /usr/es/sbin/cluster/etc=>sudo cat rhosts

172.168.3.15172.168.3.16172.168.3.18172.168.3.19172.169.3.15172.169.3.16192.168.3.15 #NODE1-hb192.168.4.15 #NODE1-bkup-hb192.168.3.16 #NODE2-hb192.168.4.16 #NODE2-bkup-hbNow entries are added in clhosts and rhosts files.

Step2:

Define a cluster

/usr/es/sbin/cluster/etc=>sudo smit HACMP

The following options will appear, then select.

Move cursor to desired item and press Enter.Initialization and Standard ConfigurationSystem Management (C-SPOC)

Problem Determination Tools Extended Configuration

• Select: Extended ConfigurationThen again following options will appear, select

Extended ConfigurationMove cursor to desired item and press Enter.Discover HACMP-related Information from Configured NODEsExtended Topology ConfigurationExtended Resource ConfigurationExtended Cluster Service SettingsExtended Event ConfigurationExtended Performance Tuning Parameters ConfigurationSecurity and Users ConfigurationSnapshot ConfigurationExport Definition File for Online Planning WorksheetsImport Cluster Configuration from Online Planning Worksheets FileExtended Verification and SynchronizationHACMP Cluster Test Tool

• Select: Extended Topology ConfigurationThen again following options will appear, select

Extended Topology ConfigurationMove cursor to desired item and press Enter.Configure an HACMP ClusterConfigure HACMP NODEsConfigure HACMP SitesConfigure HACMP NetworksConfigure HACMP Communication Interfaces/DevicesConfigure HACMP Persistent NODE IP Label/AddressesConfigure HACMP Global NetworksConfigure HACMP Network ModulesConfigure Topology Services and Group ServicesShow HACMP Topology

• Select: Configure an HACMP ClusterThen again following options will appear, select

Configure an HACMP ClusterMove cursor to desired item and press Enter.Add/Change/Show an HACMP ClusterRemove an HACMP ClusterReset Cluster Tunable

• Select: Add/Change/Show an HACMP Cluster

Repeat same steps for NODE 2.span class=san><Step3: Define NODEs to clusterRepeat same procedure as above two times to select NODEs


•Select: Configure HACMP NODEs

Configure HACMP NetworksMove cursor to desired item and press Enter.Add a Network to the HACMP ClusterChange/Show a Network in the HACMP ClusterRemove a Network from the HACMP ClusterManage Concurrent Volume Groups for Multi-NODE Disk Heartbeat

•Select: Add a NODE to the HACMP ClusterNODE Name [NODE1]• Communic ation Path to NODE [NODE1] +• Press here Esc 4 to list and then select NODE1 (172.168.3.15)

Repeat same steps for NODE 2.Step4:

Add networks to cluster.


• Select: Configure HACMP Networks


• Select: Add a Network to the HACMP Cluster

Select a Network TypeMove cursor to desired item and press Enter.Discovery last performed: (Not Performed)Discovered IP-based Network TypesDiscovered Serial Device TypesPre-defined IP-based Network TypesXD_dataXD_ipatmetherfddihpsibtoken# Pre-defined Serial Device TypesXD_rs232diskhbrs232tmscsitmssaF1=Help F2=Refresh F3=CancelF8=Image F10=Exit Enter=DoF1=Help│ /=Find n=Find Next

• Select ether as in green color (Not in actual).

Repeat steps for second IP:• Sudo smit HACMP• Extended Configuration• Configure HACMP Networks• Select ether• Add an IP-Based Network to the HACMP Cluster

Each time check command cltopinfo and observe changes.

Step 5: Now assign communication path for network to cluster.• Sudo smit HACMP• Extended Configuration• Extended Topology Configuration• Configure HACMP Communication Interfaces/Devices


• Select: Configure HACMP Communication Interfaces/Devices

Configure HACMP Communication Interfaces/DevicesMove cursor to desired item and press Enter.Add Communication Interfaces/DevicesChange/Show Communication Interfaces/DevicesRemove Communication Interfaces/DevicesUpdate HACMP Communication Interface with Operating System Settings

• Select: Add Communication Interfaces/Devices

Select a categoryMove cursor to desired item and press Enter.Add Discovered Communication Interface and DevicesAdd Pre-defined Communication Interfaces and Devices

• Select: Add Pre-defined Communication Interfaces and Devices• Select: Communication Interfaces• Select: net_ether_01

NODE1 (172.168.3.15) // For IP Label/Address press Esc4

Repeat steps for NODE2

NODE2 (172.168.3.16) // For IP Label/Address press Esc4

Check using cltpinfo as• /usr/es/sbin/cluster/etc=>cltopinfo -w

Network net_ether_01NODE NODE1:NODE1 172.168.3.15NODE NODE2:NODE2 172.168.3.16

Network net_ether_02NODE NODE1:NODE NODE2:

Step6: Now assign communication interfaces for backup• sudo smit HACMP

• Select Extended Configuration

• Select Extended Topology Configuration• Select Configure HACMP Communication Interfaces/Devices

Configure HACMP Communication Interfaces/DevicesMove cursor to desired item and press Enter.Add Communication Interfaces/DevicesChange/Show Communication Interfaces/DevicesRemove Communication Interfaces/DevicesUpdate HACMP Communication Interface with Operating System Settings

• Select: Add Communication Interfaces/Devices• Select: Add Pre-defined Communication Interfaces and Devices• Select: Communication Interfaces• Select: net_ether_02

Press Esc4 and select NODE Name NODE1-bkup (172.169.3.15)Press Esc4 and select NODE Name NODE2-bkup (172.169.3.16)

Repeat steps for NODE2:

Step7: Now add IP based network using heart beats:• Sudo smit HACMP• Select: Extended Configuration• Select: Extended Topology Configuration• Select: Configure HACMP Networks


Select: Add a Network to the HACMP Cluster

Select a Network TypeMove cursor to desired item and press Enter.[TOP]# Discovery last performed: (Aug 07 15:53)# Discovered IP-based Network Types# Discovered Serial Device Typesdiskhb# Pre-defined IP-based Network TypesXD_dataXD_ipatmetherfddihpsibtoken# Pre-defined Serial Device TypesXD_rs232diskhb[MORE...3]F1=Help F2=Refresh F3=CancelF8=Image F10=Exit Enter=DoF1=Help│ /=Find n=Find Next

Select: diskhb

Repeat steps for NODE2.

Check using cltopinfo as,• /usr/es/sbin/cluster/etc=>cltopinfo -wNetwork net_diskhb_01NODE NODE1:NODE NODE2:

Network net_diskhb_02NODE NODE1:NODE NODE2:


Network net_ether_02NODE NODE1:NODE1-bkup 172.169.3.15NODE NODE2:NODE2-bkup 172.169.3.16

Step 8: Now add communication devices to non-IP disk heartbeat network:• Sudo smit HACMP• Select: Extended Configuration• Select: Extended Topology Configuration• Select: Configure HACMP Communication Interfaces/Devices• Select: Add Communication Interfaces/Devices• Select: Add Pre-defined Communication Interfaces and Devices

Now here select Communication Devices instead Communication interfaces• Select: Communication Devices• Select: net_diskhb_01

Now repeat steps for second NODE:Select: net_diskhb_01 this time also

Give command:• /usr/es/sbin/cluster/etc=>cltopinfo -w

Network net_diskhb_01NODE NODE1:NODE1-dhb /dev/hdisk0NODE NODE2:NODE2-dhb /dev/hdisk4


Network net_ether_02NODE NODE1:NODE1-bkup 172.169.3.15NODE NODE2:NODE2-bkup 172.169.3.16

Step9: Now Create Service IP Labels:• Sudo smit HACMP• Select: Initialization and Standard Configuration• Select: Configure Resources to Make Highly Available• Select: Configure Service IP Labels/Addresses• Select: Add a Service IP Label/Address(standard)

Press Esc4: and select wilson_apache (172.168.3.18)Press Esc4: and select net_ether_01 (172.168.3.0/24)Repeat steps for other NODEPress Esc4: and select wilson_nweb (172.168.3.19)Press Esc4: and select net_ether_01 (172.168.3.0/24)

Step10: Now Create Application ServersSudo smit HACMP• Select: Initialization and Standard Configuration• Select: Configure Resources to Make Highly Available• Select: Configure Application Servers• Select: Add an Application Server

Repeat steps for 2nd application server.

• Step11: Now create a Resource Group• Sudo smit HACMP• Select: Initialization and Standard Configuration• Select: Configure HACMP Resource Groups• Select: Add a Resource Group• Select: Add an Application Server

To select both NODEs press F7 as following screen.• Select: Option Never Fallback for this press Esc4 and select.

Repeat steps for other resource group

• /usr/es/sbin/cluster/etc=>cltopinfoCluster Name: HACLUSTER

Cluster Connection Authentication Mode: StandardCluster Message Authentication Mode: NoneCluster Message Encryption: NoneUse Persistent Labels for Communication: NoThere are 2 NODE(s) and 4 network(s) definedNODE NODE1:Network net_diskhb_01NODE1-dhb /dev/hdisk0Network net_diskhb_02

Network net_ether_01wilson_apache 172.168.3.18wilson_nweb 172.168.3.19NODE1 172.168.3.15Network net_ether_02

NODE1-bkup 172.169.3.15NODE NODE2:Network net_diskhb_01NODE2-dhb /dev/hdisk4Network net_diskhb_02Network net_ether_01wilson_apache 172.168.3.18wilson_nweb 172.168.3.19NODE2 172.168.3.16Network net_ether_02NODE2-bkup 172.169.3.16

Resource Group WAPACHEStartup Policy Online On Home NODE OnlyFallover Policy Fallover To Next Priority NODE In The ListFallback Policy Never FallbackParticipating NODEs NODE1 NODE2

Resource Group WNWEBStartup Policy Online On Home NODE OnlyFallover Policy Fallover to Next Priority NODE in the ListFallback Policy Never FallbackParticipating NODEs NODE1 NODE2sh: /usr/es/sbin/cluster/utilities/clhbs: 0403-006 Execute permission denied.

Step12: Now Configure Concurrent Volume Groups:• Sudo smit HACMP• Select: System Management (C-SPOC)• Select: HACMP Concurrent Logical Volume Management• Select: Concurrent Volume Groups• Select: Create a Concurrent Volume Group• Select: Select here both NODEs using F7NODE1 and NODE2

So now here concurrent Volume Group is created.

Now Create a Shared Volume Groups for ApplicationStep 13:Sudo smit HACMPSelect: System Management (C-SPOC)Select: HACMP Logical Volume ManagementSelect: Shared Volume GroupsSelect: Create a Shared Volume GroupSelect: both NODEs

Now Add Resources to Resource Groups:

Step14:

Add Resources to Resource GroupsSudo smit HACMPSelect: Initialization and Standard ConfigurationSelect: Configure HACMP Resource GroupsSelect: Change/Show Resources for a Resource Group (standard)

Select WAPACHE

Now verify Cluster:Step15:Very and Sync Cluster• Sudo smit HACMP• Select: Initialization and Standard Configuration• Select: Verify and Synchronize HACMP Configuration

Documents

HACMP Configuration in AIX