RACGuides_Rac10gR2OnSolaris

Table of ContentsRac10gR2OnSolaris............................................................................................................................................1

1. Introduction.........................................................................................................................................1 1.1. What you need to know......................................................................................................1 1.2. Install Steps.........................................................................................................................1 1.3. Schematic............................................................................................................................2

1.3.1. Hardware/software configuration before Oracle software install.......................2 1.3.2. Hardware/software configuration after Oracle software install..........................2

1.4. Installation Method.......................................................................................................................................2 2. Prepare the cluster............................................................................................................................................3

2.1. Groups and Users Accounts.............................................................................................................3 2.1.1. Create necessary Groups and User Accounts..................................................................4 2.1.2. Modifying "oracle" user...................................................................................................4

2.2. Configuring Kernel Parameters....................................................................................................................4 2.3. Hardware Requirements................................................................................................................................5

2.3.1. Memory Requirements..................................................................................................................5 2.3.2. Disk Space Requirements.............................................................................................................6

2.4. Software Requirements.................................................................................................................................6 2.5. Network Requirements.................................................................................................................................7 2.6. Time Sync.....................................................................................................................................................9 2.7. Configuring SSH...........................................................................................................................................9 2.8. Creating Required Directories....................................................................................................................10 2.9. CVU stage check.........................................................................................................................................10 3. Prepare the shared storage for Oracle RAC...................................................................................................12

3.1. Shared Disk Layout and Partition Creation...................................................................................12 4. Install Oracle Clusterware..............................................................................................................................17

4.1. Run CVU Pre Clusterware Install Checks.....................................................................................17 4.2. Install Oracle Clusterware..............................................................................................................20 4.3. Run CVU Post Clusterware Install Checks...................................................................................34

5. Oracle Clusterware patching..........................................................................................................................36 6. Install Oracle ASM Software only Home......................................................................................................39 7. Oracle ASM Software Home Patching..........................................................................................................46 8. Create Node Specific Network Listeners.......................................................................................................50 9. Create ASM Instances and initial ASM disk groups.....................................................................................55 10. Install Oracle RAC Database Software only Home.....................................................................................62

10.1. Run CVU Pre Database Install Check.........................................................................................62 10.2. Install Oracle RDBMS Software.................................................................................................66

11. Oracle RAC Software Home Patching........................................................................................................72 12. Create RAC database...................................................................................................................................76

i

Rac10gR2OnSolaris

1. Introduction

1.1. What you need to know

Oracle Clusterware: With Oracle Database 10g Release 2 (10.2), Cluster Ready Services, or CRS, is nowcalled Oracle Clusterware.

Cluster Verification Utility (CVU): This utility (also known as cvu or cluvfy) checks the state of yourcluster at various stages of the install, from initial hardware and operating system configuration topostinstallation cluster database configuration. It should catch operating system configuration issues thatotherwise would have caused an unnecessary installation failure. See Appendix I for more details on cluvfy.You can download the latest version of CVU from otn.

Clusterware Home: Starting with Oracle Database 10g Release 2 (10.2), Oracle Clusterware should beinstalled in a separate Oracle Clusterware home directory. You should not install Oracle Clusterware in arelease-specific Oracle home mount point, typically /u01/app/oracle/product/10.2.0.

Automatic Storage Management (ASM): In this document, ASM is the standard method of storage for thedatabase files (not the Oracle Home). ASM can provide Automatic striping, mirroring, and VolumeManagement. This document does not cover using OCFS and RAW devices for storage, though they are stillavailable.

ASM Instance: With Oracle Database 10g Release 2 (10.2), a single ASM instance for each node is now ableto serve disk groups to all the database instances in the cluster, whether or not the database is a RAC databaseor a Single Instance database. Automatic Storage Management should be installed in a separate ASM homedirectory.

Virtual IP (VIP): Oracle Database uses the VIP address to provide a secondary IP for the main networkadapter for the node. Clients connect to the VIP to gain access to the database. The purpose of the VIP is toimprove detection of node failure by clients, and facilitate failover. The VIP is not a cluster IP.

1.2. Install Steps

To summarise the install steps are:

Pre-reqs, to make sure the cluster is setup OK.1. Stage all the software on one node, typically Node12. Install Oracle Clusterware (using the push mechanism to install on the other nodes in the cluster)3. Patch Oracle Clusterware – when available4. Install Oracle ASM Software only Home (using the push mechanism to install on the other nodes inthe cluster)

5.

Patch Oracle ASM Software Home – when available6. Create Node Specific Network Listeners7. Create ASM Instances and initial ASM disk group8. Install Oracle RAC Database Software only Home (using the push mechanism to install on the othernodes in the cluster)

9.

Patch Oracle RAC Database Software Home – when available10. Create RAC database11.

Rac10gR2OnSolaris 1

For a base release, without patches, it is possible to combine steps 5, 6 & 8 into 1 integrated step, and thesame with steps 9 & 10 as another integrated step.

1.3. Schematic

The following is a schematic of the software & hardware layout of a 2node RAC cluster. As explained in thisdocument. The actual number of LUNs required will vary depending on your mirroring requirements.

1.3.1. Hardware/software configuration before Oracle software install

Notice that there are a number of LUNs present to both machines from the shared storage. 5 * 270 MB LUNs- 2 are used for the OCR devices and 3 are used for the vote devices 4 * ~10 GB LUNs - 2 are used for+DATA ASM diskgroup and 2 for +FLASH diskgroup. Also notice there is a dedicated switch on the clusterinterconnect.

1.3.2. Hardware/software configuration after Oracle software install

1.4. Installation Method

The method of installing a 2 node Oracle 10gR2 RAC cluster on Sun Solaris detailed in this note uses localUFS (UNIX File System) for the Oracle Clusterware, ASM & Oracle Database Homes. The files required byOracle Clusterware are on RAW partitions. The database data-files are on ASM. The local Homes for OracleClusterware, ASM and RAC take up a total of 6.5GB on each node.

There are other possible combinations although Oracle recommends the above install method unless you havea specific requirement.

1.2. Install Steps 2

2. Prepare the cluster

This installation routine presumes that you have a 2-node Sun Solaris cluster. There are a number of itemswhich require checking before the install commences. Getting this bit right will enhance your installexperience.

Groups and Users Accounts1. Configuring Kernel Parameters2. Hardware Requirements3. Software Requirements4. Network Requirements5. Shared Disk Layout and Partition Creation6. Time Synchronization7. Configure SSH on all nodes8. Creating Required Directories9. Run CVU10.

It is essential that these items are checked and are configured correctly before the install commences.

2.1. Groups and Users Accounts

Typically, we need 2 main UNIX groups, The Oracle Inventory group (typically, oinstall) and the DBA group(typically, dba).

We must create Oracle clusterware software owner user (typically, oracle). For the sake of this document, wewill create a single software owner user, "oracle", for all Oracle softwares (crs, asm and db). Another optioncan be to create 3 users, (crs, asm and oracle), and separate operating system privileges groups for differentOracle software installations.

To determine whether the oinstall and dba group exists, and whether the oracle user also exist enter thefollowing commands:

On the First node

fabushab-sunclu1:/ # grep dba /etc/group fabushab-sunclu1:/ # grep oinstall /etc/group fabushab-sunclu1:/ # grep oracle /etc/passwd

-OR-

fabushab-sunclu1:/ # id -a oracle id: invalid user name: "oracle"

On the second Node

fabushab-sunclu2:/ # grep dba /etc/group fabushab-sunclu2:/ # grep oinstall /etc/group fabushab-sunclu2:/ # grep oracle /etc/passwd

-OR-

fabushab-sunclu2:/ # id -a oracle id: invalid user name: "oracle"

2. Prepare the cluster 3

2.1.1. Create necessary Groups and User Accounts

If the "oinstall","dba"; groups and "oracle"; user does not exist or if you require a new set of groups and users,then you can create them as follows.

On the First node

fabushab-sunclu1:/ # /usr/sbin/groupadd oinstall fabushab-sunclu1:/ # /usr/sbin/groupadd dba fabushab-sunclu1:/ # /usr/sbin/useradd -g oinstall -G dba –d /home/oracle oracle fabushab-sunclu1:/ # id -a oracle uid=102(oracle) gid=100(oinstall) groups=101(dba)

On the second Node

fabushab-sunclu2:/ # /usr/sbin/groupadd oinstall fabushab-sunclu2:/ # /usr/sbin/groupadd dba fabushab-sunclu2:/ # /usr/sbin/useradd -g oinstall -G dba –d /home/oracle oracle fabushab-sunclu2:/ # id -a oracle uid=102(oracle) gid=100(oinstall) groups=101(dba)

After that we need to set the password for the "oracle" user

On the First node

fabushab-sunclu1:/ # passwd oracle New Password: Re-enter new Password: passwd: password successfully changed for oracle fabushab-sunclu1:/ #

On the second Node

fabushab-sunclu2:/ # passwd oracle New Password: Re-enter new Password: passwd: password successfully changed for oracle fabushab-sunclu2:/ #

2.1.2. Modifying "oracle" user

If the "oracle" user exists, but its primary group is not oinstall or it is not a member of the appropriate OSDBAor OSOPER groups, then the following command can be used to modify it. Specify the primary group usingthe -g option and any required secondary group using the -G option:

On the First node

fabushab-sunclu1:/ # /usr/sbin/usermod -g oinstall -G dba oracle

On the Second Node

fabushab-sunclu2:/ # /usr/sbin/usermod -g oinstall -G dba oracle

2.2. Configuring Kernel Parameters

We need to set the following kernel parameters to values greater than or equal to the recommended valuesshown below.

set noexec_user_stack=1 set semsys:seminfo_semmni=100 set semsys:seminfo_semmns=1024 setsemsys:seminfo_semmsl=256 set semsys:seminfo_semvmx=32767 setshmsys:shminfo_shmmax=4294967296 set shmsys:shminfo_shmmni =100

2.1.1. Create necessary Groups and User Accounts 4

Since we are using Solaris 10, we are not required to make changes to the /etc/system file to implement theSystem V IPC. Solaris 10 uses the resource control facility for its implementation. However, Oraclerecommends that you set both resource control and /etc/system/ parameters. Operating system parameters notreplaced by resource controls continue to affect performance and security on Solaris 10 systems. For furtherinformation, contact your Sun vendor

On the First node

fabushab-sunclu1:/ # prctl -n project.max-sem-ids -v 100 -r -i project user.root fabushab-sunclu1:/ # prctl –n project.max-shm-ids -v 100 -r -i project user.root fabushab-sunclu1:/ # prctl -n project.max-shm-memory -v 4 gb -r -i project user.root fabushab-sunclu1:/ # vi /etc/system

set noexec_user_stack=1 set semsys:seminfo_semmni=100 set semsys:seminfo_semmns=1024 set semsys:seminfo_semmsl=256 set semsys:seminfo_semvmx=32767 set shmsys:shminfo_shmmax=4294967296 set shmsys:shminfo_shmmni =100

fabushab-sunclu1:/ # init 6

On the Second Node

fabushab-sunclu2:/ # prctl -n project.max-sem-ids -v 100 -r -i project user.root fabushab-sunclu2:/ # prctl -nproject.max-shm-ids -v 100 -r -i project user.root fabushab-sunclu2:/ # prctl -n project.max-shm-memory -v 4.3gb -r -i project user.root fabushab-sunclu2:/ # vi /etc/system

set noexec_user_stack=1 set semsys:seminfo_semmni=100 set semsys:seminfo_semmns=1024 set semsys:seminfo_semmsl=256 set semsys:seminfo_semvmx=32767 set shmsys:shminfo_shmmax=4294967296 set shmsys:shminfo_shmmni =100

fabushab-sunclu2:/ # init 6

2.3. Hardware Requirements

2.3.1. Memory Requirements

We have to have at least 1 GB of RAM, To determine the physical RAM size, enter the following command:

On the First node

fabushab-sunclu1:/stage # /usr/sbin/prtconf | grep "Memory size" Memory size: 2048 Megabytes

On the Second Node

fabushab-sunclu2:/stage # /usr/sbin/prtconf | grep "Memory size" Memory size: 2048 Megabytes

If the size of the physical RAM is less than the required size, then you must install more memory beforecontinuing.

Oracle recommends if installed RAM is less than 2G, to configure the swap space 1.5 times the RAM size, ifthe RAM size is more than 2 G, then swap space equals RAM size

To determine the size of the configured swap space, enter the following command:

2.2. Configuring Kernel Parameters 5

On the First node

fabushab-sunclu1:/stage # /usr/sbin/swap -s

total68712k bytes allocated + 8264k reserved = 76976k used, 5681472k available

On the Second Node

fabushab-sunclu2:/stage # /usr/sbin/swap -s

total68712k bytes allocated + 8265k reserved = 76977k used, 5681486k available

If necessary, configure additional swap space.

2.3.2. Disk Space Requirements

Disk space requirements for installing Oracle Database 11g Release 1 is:

Between 225 and 275 MB of disk space in the /tmp directory (using "df -h /tmp")1. around 4.5 G per Oracle Home per node2.

2.4. Software Requirements

The following packages (or later versions) must be installed:

SUNWarc SUNWbtool SUNWhea SUNWlibC? SUNWlibm SUNWlibms SUNWsprot SUNWtooSUNWi1of SUNWi1cs SUNWi15cs SUNWxwfnt

On the First node

fabushab-sunclu1:/stage # pkginfo -i SUNWarc SUNWbtool SUNWhea SUNWlibm SUNWlibms SUNWsprot SUNWtoo \ SUNWi1of SUNWi1cs SUNWi15cs SUNWxwfnt system SUNWarc Lint Libraries (usr) system SUNWbtool CCS tools bundled with SunOS system SUNWhea SunOS Header Files system SUNWi15cs X11 ISO8859-15 Codeset Support system SUNWi1cs X11 ISO8859-1 Codeset Support system SUNWi1of ISO-8859-1 (Latin-1) Optional Fonts system SUNWlibm Math & Microtasking Library Headers & Lint Files (Usr) system SUNWlibms Math & Microtasking Libraries (Usr) system SUNWsprot Solaris Bundled tools system SUNWtoo Programming Tools system SUNWxwfnt X Window System platform required fonts

On the Second Node

fabushab-sunclu2:/ # pkginfo -i SUNWarc SUNWbtool SUNWhea SUNWlibm SUNWlibms SUNWsprot SUNWtoo \ SUNWi1of SUNWi1cs SUNWi15cs SUNWxwfnt system SUNWarc Lint Libraries (usr) system SUNWbtool CCS tools bundled with SunOS system SUNWhea SunOS Header Files system SUNWi15cs X11 ISO8859-15 Codeset Support system SUNWi1cs X11 ISO8859-1 Codeset Support system SUNWi1of ISO-8859-1 (Latin-1) Optional Fonts

2.3.1. Memory Requirements 6

http://globaldc.oracle.com/perl/twiki/edit/RACGuides/SUNWlibC?topicparent=RACGuides.Rac10gR2OnSolaris

system SUNWlibm Math & Microtasking Library Headers & Lint Files (Usr) system SUNWlibms Math & Microtasking Libraries (Usr) system SUNWsprot Solaris Bundled tools system SUNWtoo Programming Tools system SUNWxwfnt X Window System platform required fonts

If a package is not installed, then install it.

Depending on the components you want to use, you want to ensure the necessary OS packages are alsoinstalled. E.g if you are planning to use Pro*C/C++, Oracle Call Interface, Oracle C++ Call Interface, orOracle XML Developer’s Kit (XDK), you need to install Sun ONE Studio 8 (C and C++ 5.5) and gcc 3.4.2.

But for the sake of this document we will not install any additional components to the ones installed bydefault with Oracle Enterprise Edition (EE).

2.5. Network Requirements

You will need a minimum of 2 network adapters per machine, one is used for the public interface, and theother adapter is used for the private interface, which is used for Oracle RAC node membership andinter-nodes traffic. The private interface adapters should be connected to each other via a private andnon-routable network switch which is used exclusively for the interconnect traffic. It is not supported to usecross-over cables between the interconnect adapters. A total of 3 IP addresses per node will be required:

The public IP address, which should be recorded in hosts file on each node and, if available, DNS.This IP Address should be bound to the public adapter before starting the install. It should be a static,not DHCP, address

•

The private IP address, which should be from a different subnet than the public IP address. Thisaddress does not require registering in DNS but you should place an entry in the hosts file on eachnode. This IP Address should be bound to the private adapter before starting the install. It should be astatic, not DHCP, address

•

A VIP address, which should be from the same subnet as the public IP address and should be recordedin DNS and the hosts file on each node. This IP Address should NOT be bound to the public adapterbefore starting the install. Oracle Clusterware is responsible for binding this address. It should be astatic, not DHCP, address

•

If you do not have a DNS server then make sure both the public and the VIP addresses are entered into all therelevant hosts file that are normally found in the /etc/hosts. For the sake of this document, we have our hostsname and IPs configured in the /etc/hosts and also has DNS entries.

/etc/hosts file On the First node

fabushab-sunclu1:/ # cat /etc/hosts # # Internet host table #

# Public IPs

127.0.0.1 localhost 10.148.60.238 fabushab-sunclu1 loghost 10.148.60.240 fabushab-sunclu2

# Private IPs 192.168.20.3 fabushab-sunclu1-priv 192.168.20.4 fabushab-sunclu2-priv

# VIPS

10.148.60.239 fabushab-sunclu1-v 10.148.60.241 fabushab-sunclu2-v

fabushab-sunclu1:/ # ifconfig -a

2.4. Software Requirements 7

lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 eri0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.148.60.238 netmask fffffc00 broadcast 10.148.63.255 ether 0:3:ba:1d:9b:63 rge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 192.168.20.3 netmask fffffc00 broadcast 192.168.23.255 ether 0:14:d1:38:7d:67

/etc/hosts file On the Second Node

fabushab-sunclu2:/ # cat /etc/hosts

# Internet host table #

# Public IPs

127.0.0.1 localhost 10.148.60.238 fabushab-sunclu1 10.148.60.240 fabushab-sunclu2 loghost

# Private IPs 192.168.20.3 fabushab-sunclu1-priv 192.168.20.4 fabushab-sunclu2-priv

# VIPS

10.148.60.239 fabushab-sunclu1-v 10.148.60.241 fabushab-sunclu2-v

fabushab-sunclu2:/ # ifconfig -a lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 eri0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.148.60.240 netmask fffffc00 broadcast 10.148.63.255 ether 0:3:ba:f:1a:83 rge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3 inet 192.168.20.4 netmask fffffc00 broadcast 192.168.23.255 ether 0:14:d1:38:82:b6

There are a series of 'ping' tests that should be completed. You should ensure that the public IP addressesresolve correctly and that the private addresses resolve on both nodes.

Public Ping test

Pinging Node1 from Node1 should return Node1's public IP address♦ Pinging Node2 from Node1 should return Node2's public IP address♦ Pinging Node1 from Node2 should return Node1's public IP address♦ Pinging Node2 from Node2 should return Node2's public IP address♦

•

Private Ping test

Pinging Node1 private from Node1 should return Node1's private IP address♦ Pinging Node2 private from Node1 should return Node2's private IP address♦ Pinging Node1 private from Node2 should return Node1's private IP address♦ Pinging Node2 private from Node2 should return Node2's private IP address♦

•

VIP Ping test

Pinging the VIP address at this point should fail. VIPs will be activated at the end of theOracle Clusterware install.

♦

•

If any of the above tests fail you should fix name/address resolution by updating the DNS or local hosts files

2.5. Network Requirements 8

on each node before continuing.

2.6. Time Sync

There is a general requirement for Oracle RAC that the time on all the nodes be the same. Typically this canbe achieved using Network Time Server protocol. At the very least you should ensure that the system clockson all nodes are as close as possible.

2.7. Configuring SSH

The following steps are to be run by the software owner ("oracle" user in our case)

On the First Node

fabushab-sunclu1:/ # su - oracleSun Microsystems Inc. SunOS 5.10 Generic January 2005

fabushab-sunclu1:/u01/oracle $ mkdir ~/.sshfabushab-sunclu1:/u01/oracle $ chmod 700 ~/.sshfabushab-sunclu1:/u01/oracle $ /usr/bin/ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/u01/oracle/.ssh/id_rsa):Enter passphrase (empty for no passphrase): <<press enter here>>Enter same passphrase again: <<press enter here>>Your identification has been saved in /u01/oracle/.ssh/id_rsa.Your public key has been saved in /u01/oracle/.ssh/id_rsa.pub.The key fingerprint is:0d:61:8e:db:7a:cc:44:96:d3:79:f2:52:82:04:01:5e oracle@fabushab-sunclu1

fabushab-sunclu1:/u01/oracle/.ssh $ ssh fabushab-sunclu1 "cat ~/.ssh/id_rsa.pub" >> ~/.ssh/authorized_keysPassword:fabushab-sunclu1:/u01/oracle/.ssh $ ssh fabushab-sunclu2 "cat ~/.ssh/id_rsa.pub" >> ~/.ssh/authorized_keysThe authenticity of host 'fabushab-sunclu2 (10.148.60.240)' can't be established.RSA key fingerprint is 62:c2:a4:60:77:b1:9b:ce:17:59:22:8a:08:50:f2:fa.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'fabushab-sunclu2,10.148.60.240' (RSA) to the list of known hosts.Password:

fabushab-sunclu1:/u01/oracle/.ssh $ scp ~/.ssh/authorized_keys fabushab-sunclu2:/u01/oracle/.ssh/Password:authorized_keys 100% |*****************************| 466 00:00

fabushab-sunclu1:/u01/oracle/.ssh $ ssh fabushab-sunclu2

Last login: Sat Jun 14 08:34:41 2008 from fabushab-suncluSun Microsystems Inc. SunOS 5.10 Generic January 2005

fabushab-sunclu2:/u01/oracle $ logoutConnection to fabushab-sunclu2 closed.

fabushab-sunclu1:/u01/oracle/.ssh $ ssh fabushab-sunclu1Last login: Sat Jun 14 08:32:34 2008 from fabushab-suncluSun Microsystems Inc. SunOS 5.10 Generic January 2005

fabushab-sunclu1:/u01/oracle $ logoutConnection to fabushab-sunclu1 closed.

On the Second Node

fabushab-sunclu2:/ # su - oracleSun Microsystems Inc. SunOS 5.10 Generic January 2005

fabushab-sunclu2:/u01/oracle $ mkdir ~/.ssh

fabushab-sunclu2:/u01/oracle $ chmod 700 ~/.ssh

fabushab-sunclu2:/u01/oracle $ /usr/bin/ssh-keygen -t rsaGenerating public/private rsa key pair.Enter file in which to save the key (/u01/oracle/.ssh/id_rsa):

2.6. Time Sync 9

Enter passphrase (empty for no passphrase): <<press enter here>>Enter same passphrase again: <<press enter here>>Your identification has been saved in /u01/oracle/.ssh/id_rsa.Your public key has been saved in /u01/oracle/.ssh/id_rsa.pub.The key fingerprint is:e8:1a:04:23:79:c3:99:4f:52:64:ae:cf:4b:06:8d:e4 oracle@fabushab-sunclu2

fabushab-sunclu2:/u01/oracle $

<<Wait until you copy the ssh/authorized_keys to this node >>

fabushab-sunclu2:/u01/oracle $ ssh fabushab-sunclu1The authenticity of host 'fabushab-sunclu1 (10.148.60.238)' can't be established.RSA key fingerprint is 4d:43:62:2a:28:7c:97:28:1e:01:3d:2c:25:f3:f4:1e.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'fabushab-sunclu1,10.148.60.238' (RSA) to the list of known hosts.Last login: Sat Jun 14 08:34:15 2008 from fabushab-suncluSun Microsystems Inc. SunOS 5.10 Generic January 2005

fabushab-sunclu1:/u01/oracle $ exitlogoutConnection to fabushab-sunclu1 closed.

fabushab-sunclu2:/u01/oracle $ ssh fabushab-sunclu2The authenticity of host 'fabushab-sunclu2 (10.148.60.240)' can't be established.RSA key fingerprint is 62:c2:a4:60:77:b1:9b:ce:17:59:22:8a:08:50:f2:fa.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added 'fabushab-sunclu2,10.148.60.240' (RSA) to the list of known hosts.Last login: Sat Jun 14 08:35:01 2008 from fabushab-suncluSun Microsystems Inc. SunOS 5.10 Generic January 2005

fabushab-sunclu2:/u01/oracle $ exitlogoutConnection to fabushab-sunclu2 closed.

2.8. Creating Required Directories

For our installed we will use the following directoriesOracle BASE = /u01/app/oracleCRS HOME = /u01/crsDB HOME = /u01/app/oracle/product/11.1/dbASM HOME = /u01/app/oracle/product/11.1/asm

On the First Node

fabushab-sunclu1:/ # mkdir -p /u01/app/oraclefabushab-sunclu1:/ # mkdir -p /u01/crsfabushab-sunclu1:/ # chown -R oracle:oinstall /u01/app/oraclefabushab-sunclu1:/ # chown -R oracle:oinstall /u01/crs

On the Second Node

fabushab-sunclu2:/ # mkdir -p /u01/app/oraclefabushab-sunclu2:/ # mkdir -p /u01/crsfabushab-sunclu2:/ # chown -R oracle:oinstall /u01/app/oraclefabushab-sunclu2:/ # chown -R oracle:oinstall /u01/crs

2.9. CVU stage check

Now you can run the CVU to check the state of the cluster prior to the install of the Oracle Software. CVUcan be run from the installation media, I have staged the media into"/stage/core/SOLARIS_SPARC/64bit/rdbms/10.2.0.1.0/" directory. It's also recommended that you run thelatest CVU available from OTN. at the time when this document was written a newer version of CVU wasreleased, and I used the latest version of CVU

fabushab-sunclu1:/stage/core/SOLARIS_SPARC/64bit/rdbms/10.2.0.1.0/clusterware/cluvfy

2.7. Configuring SSH 10

$ ./runcluvfy.sh stage -post hwos -n fabushab-sunclu1,fabushab-sunclu2 -verbose

Performing post-checks for hardware and operating system setup

Checking node reachability...

Check: Node reachability from node "fabushab-sunclu1" Destination Node Reachable? ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yesResult: Node reachability check passed from node "fabushab-sunclu1".

Checking user equivalence...

Check: User equivalence for user "oracle" Node Name Comment ------------------------------------ ------------------------ fabushab-sunclu2 passed fabushab-sunclu1 passedResult: User equivalence check passed for user "oracle".

Checking node connectivity...

Interface information for node "fabushab-sunclu2" Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address ---------------- ------------ ------------ ------------ ------------ ------------ eri0 10.148.60.240 10.148.60.0 10.148.60.240 UNKNOWN 00:03:ba:0f:1a:83 rge0 192.168.20.4 192.168.20.0 192.168.20.4 UNKNOWN 00:14:d1:38:82:b6

Interface information for node "fabushab-sunclu1" Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address ---------------- ------------ ------------ ------------ ------------ ------------ eri0 10.148.60.238 10.148.60.0 10.148.60.238 UNKNOWN 00:03:ba:1d:9b:63 rge0 192.168.20.3 192.168.20.0 192.168.20.3 UNKNOWN 00:14:d1:38:7d:67

Check: Node connectivity of subnet "10.148.60.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- fabushab-sunclu2:eri0 fabushab-sunclu1:eri0 yes Result: Node connectivity check passed for subnet "10.148.60.0" with node(s) fabushab-sunclu2,fabushab-sunclu1.

Check: Node connectivity of subnet "192.168.20.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- fabushab-sunclu2:rge0 fabushab-sunclu1:rge0 yes Result: Node connectivity check passed for subnet "192.168.20.0" with node(s) fabushab-sunclu2,fabushab-sunclu1.

Interfaces found on subnet "10.148.60.0" that are likely candidates for VIP:fabushab-sunclu2 eri0:10.148.60.240fabushab-sunclu1 eri0:10.148.60.238

Interfaces found on subnet "192.168.20.0" that are likely candidates for VIP:fabushab-sunclu2 rge0:192.168.20.4fabushab-sunclu1 rge0:192.168.20.3

WARNING:Could not find a suitable set of interfaces for the private interconnect.

Result: Node connectivity check passed.

Checking shared storage accessibility...

WARNING:Unable to determine the sharedness of c0t1d0 on nodes: fabushab-sunclu2,fabushab-sunclu1



WARNING:

2.9. CVU stage check 11

Unable to determine the sharedness of c1t3d0 on nodes: fabushab-sunclu2,fabushab-sunclu1



Shared storage check failed on nodes "fabushab-sunclu2,fabushab-sunclu1".

Post-check for hardware and operating system setup was unsuccessful on all the nodes.

The following warning message can be ignored, it's because the Serial number of my disks (c0t4d0 andc0t5d0) didn't return any number, so CVU assume its not same disk. Check note 372358.1 for more details


3. Prepare the shared storage for Oracle RAC

This section describes how to prepare the shared storage for Oracle RAC

For all installations, you must choose the storage option that you want to use for Oracle Clusterware files,Automatic Storage Manager (ASM) and Oracle Real Application Clusters databases (Oracle RAC). You donot have to use the same storage option for each file type.

Oracle Clusterware files include:

* Voting disks, used to monitor cluster node status. * Oracle Cluster Registry (OCR) which containsconfiguration information about the cluster.

As root, now configure storage for cluster registry, voting disk and database files. You are presented with abunch of disks from the storage array.

3.1. Shared Disk Layout and Partition Creation

It is assumed that the two nodes have local disk primarily for the operating system and the local OracleHomes (CRS,ASM and DB homes). In this setup we are using 2 local disks (c0t0d0 and c0t2d0) where wehave all the local directories and 2 iSCSI disks, one for Voting files and OCR files and the second iSCSI diskfor ASM disks. Also for the sake of this document only, we have partitioned the first iSCSI disk to 6Partitions (or slices) and the second disk to 4 partitions to simulate multiple disks. This is not recommendedpractice especially for production machines. It's recommended to create one partition per Disk or LUN.

On Solaris 10, you can use format or smc utilities to carve disk or LUNs partitions/slices. It is very importantto skip the first Cylinder on the disk to avoid ASM or Oracle Cluster ware to overwrite the partition table. Soyou always start partitioning from cylinder number 1. falling to do so, you will find out after rebooting yourmachines that data on your disks is erased and Oracle Clusterware will not start and ASM will not be able torecognize any disks. Also Don't use slice 2 because this slice represent whole disk (backup or overlap slice)

On the First node

fabushab-sunclu1:/ # formatSearching for disks...done

3. Prepare the shared storage for Oracle RAC 12

AVAILABLE DISK SELECTIONS: 0. c0t0d0 <ST320011A cyl 38790 alt 2 hd 16 sec 63> /pci@1f,0/ide@d/dad@0,0 1. c0t2d0 <WDC WD800BB-00CAA1 cyl 38307 alt 2 hd 16 sec 255> /pci@1f,0/ide@d/dad@2,0 2. c1t4d0 <Openfile-Virtualdisk-0 cyl 374 alt 2 hd 64 sec 256> /iscsi/[email protected]%3Aunix_vg2.vote_ocr0001,0 3. c1t5d0 <Openfile-Virtualdisk-0 cyl 4998 alt 2 hd 64 sec 256> /iscsi/[email protected]%3Aunix_vg2.asm0001,0Specify disk (enter its number): 2selecting c1t4d0[disk formatted]

FORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels save - save new disk/partition definitions inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return quitformat> p

PARTITION MENU: 0 - change `0' partition 1 - change `1' partition 2 - change `2' partition 3 - change `3' partition 4 - change `4' partition 5 - change `5' partition 6 - change `6' partition 7 - change `7' partition select - select a predefined table modify - modify a predefined partition table name - name the current table print - display the current table label - write partition map and label to the disk !<cmd> - execute <cmd>, then return quitpartition> prinCurrent partition table (original):Total disk cylinders available: 374 + 2 (reserved cylinders)

Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 1 16.00MB (2/0/0) 32768 1 swap wu 2 - 64 504.00MB (63/0/0) 1032192 2 backup wu 0 - 373 2.92GB (374/0/0) 6127616 3 unassigned wm 65 - 127 504.00MB (63/0/0) 1032192 4 unassigned wm 128 - 190 504.00MB (63/0/0) 1032192 5 unassigned wm 191 - 253 504.00MB (63/0/0) 1032192 6 usr wm 254 - 310 456.00MB (57/0/0) 933888 7 unassigned wm 311 - 373 504.00MB (63/0/0) 1032192

partition>


AVAILABLE DISK SELECTIONS: 0. c0t0d0 <ST320011A cyl 38790 alt 2 hd 16 sec 63> /pci@1f,0/ide@d/dad@0,0 1. c0t2d0 <WDC WD800BB-00CAA1 cyl 38307 alt 2 hd 16 sec 255> /pci@1f,0/ide@d/dad@2,0 2. c1t4d0 <Openfile-Virtualdisk-0 cyl 374 alt 2 hd 64 sec 256>

3.1. Shared Disk Layout and Partition Creation 13

/iscsi/[email protected]%3Aunix_vg2.vote_ocr0001,0 3. c1t5d0 <Openfile-Virtualdisk-0 cyl 4998 alt 2 hd 64 sec 256> /iscsi/[email protected]%3Aunix_vg2.asm0001,0Specify disk (enter its number): 3selecting c1t5d0[disk formatted]



Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 1 16.00MB (2/0/0) 32768 1 swap wu 2 - 1281 10.00GB (1280/0/0) 20971520 2 backup wu 0 - 4997 39.05GB (4998/0/0) 81887232 3 unassigned wm 1282 - 2561 10.00GB (1280/0/0) 20971520 4 unassigned wm 2562 - 3841 10.00GB (1280/0/0) 20971520 5 unassigned wm 3842 - 4993 9.00GB (1152/0/0) 18874368 6 usr wm 4994 - 4997 32.00MB (4/0/0) 65536 7 unassigned wm 0 0 (0/0/0) 0

partition>

On the Second Node


AVAILABLE DISK SELECTIONS: 0. c0t0d0 <ST320011A cyl 38790 alt 2 hd 16 sec 63> /pci@1f,0/ide@d/dad@0,0 1. c0t2d0 <WDC WD800BB-00CAA1 cyl 38307 alt 2 hd 16 sec 255> /pci@1f,0/ide@d/dad@2,0 2. c1t4d0 <Openfile-Virtualdisk-0 cyl 374 alt 2 hd 64 sec 256> /iscsi/[email protected]%3Aunix_vg2.vote_ocr0001,0 3. c1t5d0 <Openfile-Virtualdisk-0 cyl 4998 alt 2 hd 64 sec 256> /iscsi/[email protected]%3Aunix_vg2.asm0001,0


Specify disk (enter its number): 2selecting c1t4d0[disk formatted]

FORMAT MENU: disk - select a disk type - select (define) a disk type partition - select (define) a partition table current - describe the current disk format - format and analyze the disk repair - repair a defective sector label - write label to the disk analyze - surface analysis defect - defect list management backup - search for backup labels verify - read and display labels save - save new disk/partition definitions inquiry - show vendor, product and revision volname - set 8-character volume name !<cmd> - execute <cmd>, then return quit

format> p


Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 1 16.00MB (2/0/0) 32768 1 swap wu 2 - 64 504.00MB (63/0/0) 1032192 2 backup wu 0 - 373 2.92GB (374/0/0) 6127616 3 unassigned wm 65 - 127 504.00MB (63/0/0) 1032192 4 unassigned wm 128 - 190 504.00MB (63/0/0) 1032192 5 unassigned wm 191 - 253 504.00MB (63/0/0) 1032192 6 usr wm 254 - 310 456.00MB (57/0/0) 933888 7 unassigned wm 311 - 373 504.00MB (63/0/0) 1032192

partition>


AVAILABLE DISK SELECTIONS: 0. c0t0d0 <ST320011A cyl 38790 alt 2 hd 16 sec 63> /pci@1f,0/ide@d/dad@0,0 1. c0t2d0 <WDC WD800BB-00CAA1 cyl 38307 alt 2 hd 16 sec 255> /pci@1f,0/ide@d/dad@2,0 2. c1t4d0 <Openfile-Virtualdisk-0 cyl 374 alt 2 hd 64 sec 256> /iscsi/[email protected]%3Aunix_vg2.vote_ocr0001,0 3. c1t5d0 <Openfile-Virtualdisk-0 cyl 4998 alt 2 hd 64 sec 256> /iscsi/[email protected]%3Aunix_vg2.asm0001,0Specify disk (enter its number): 3selecting c1t5d0[disk formatted]




Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 1 16.00MB (2/0/0) 32768 1 swap wu 2 - 1281 10.00GB (1280/0/0) 20971520 2 backup wu 0 - 4997 39.05GB (4998/0/0) 81887232 3 unassigned wm 1282 - 2561 10.00GB (1280/0/0) 20971520 4 unassigned wm 2562 - 3841 10.00GB (1280/0/0) 20971520 5 unassigned wm 3842 - 4993 9.00GB (1152/0/0) 18874368 6 usr wm 4994 - 4997 32.00MB (4/0/0) 65536 7 unassigned wm 0 0 (0/0/0) 0

partition>

After That we need to ensure that raw devices has the right permissions, and change it as following

On the First Node

fabushab-sunclu1:/dev/rdsk # chown -h root:oinstall c1t4d0s1fabushab-sunclu1:/dev/rdsk # chown -h root:oinstall c1t4d0s3fabushab-sunclu1:/dev/rdsk # chown -h oracle:oinstall c1t4d0s4fabushab-sunclu1:/dev/rdsk # chown -h oracle:oinstall c1t4d0s5fabushab-sunclu1:/dev/rdsk # chown -h oracle:oinstall c1t4d0s7fabushab-sunclu1:/dev/rdsk # chown -h oracle:oinstall c1t5d0*

On the Second Node

fabushab-sunclu1:/dev/rdsk # chown -h root:oinstall c1t4d0s1fabushab-sunclu1:/dev/rdsk # chown -h root:oinstall c1t4d0s3fabushab-sunclu1:/dev/rdsk # chown -h oracle:oinstall c1t4d0s4fabushab-sunclu1:/dev/rdsk # chown -h oracle:oinstall c1t4d0s5fabushab-sunclu1:/dev/rdsk # chown -h oracle:oinstall c1t4d0s7fabushab-sunclu1:/dev/rdsk # chown -h oracle:oinstall c1t5d0*


4. Install Oracle Clusterware

Next we will install the Oracle Clusterware layer. Oracle Clusterware is an essential component of the OracleRAC database infrastructure. Oracle Clusterware does not require any other clustering software.

To install you will need the following information

The local drive which will host the Oracle Clusterware software• The shared devices that will be used by Oracle Clusterware• The public node names and IP address for the nodes to be used by the cluster• The private node names and IP address for the nodes to be used by the cluster• The Virtual Public IP names and IP address for the nodes to be used by the cluster•

Before starting the Oracle Clusterware install it is prudent to complete a CVU check for this stage.

4.1. Run CVU Pre Clusterware Install Checks

$./runcluvfy.sh stage -pre crsinst -n fabushab-sunclu1,fabushab-sunclu2 -r 10gR2 -verbose


Check: Node reachability from node "fabushab-sunclu1" Destination Node Reachable? ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yes Result: Node reachability check passed from node "fabushab-sunclu1".


Check: User equivalence for user "oracle" Node Name Comment ------------------------------------ ------------------------ fabushab-sunclu2 passed fabushab-sunclu1 passed Result: User equivalence check passed for user "oracle".

Checking administrative privileges...

Check: Existence of user "oracle" Node Name User Exists Comment ------------ ------------------------ ------------------------ fabushab-sunclu2 yes passed fabushab-sunclu1 yes passed Result: User existence check passed for "oracle".

Check: Existence of group "oinstall" Node Name Status Group ID ------------ ------------------------ ------------------------ fabushab-sunclu2 exists 100 fabushab-sunclu1 exists 100 Result: Group existence check passed for "oinstall".

Check: Membership of user "oracle" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Comment ---------------- ------------ ------------ ------------ ------------ ------------ fabushab-sunclu2 yes yes yes yes passed fabushab-sunclu1 yes yes yes yes passed Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed.

Administrative privileges check passed.


Interface information for node "fabushab-sunclu2" Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address ---------------- ------------ ------------ ------------ ------------ ------------ eri0 10.148.60.240 10.148.60.0 10.148.60.240 UNKNOWN 00:03:ba:0f:1a:83 rge0 192.168.20.4 192.168.20.0 192.168.20.4 UNKNOWN 00:14:d1:38:82:b6

4. Install Oracle Clusterware 17

Interface information for node "fabushab-sunclu1" Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address ---------------- ------------ ------------ ------------ ------------ ------------ eri0 10.148.60.238 10.148.60.0 10.148.60.238 UNKNOWN 00:03:ba:1d:9b:63 rge0 192.168.20.3 192.168.20.0 192.168.20.3 UNKNOWN 00:14:d1:38:7d:67

Check: Node connectivity of subnet "10.148.60.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- fabushab-sunclu2:eri0 fabushab-sunclu1:eri0 yes Result: Node connectivity check passed for subnet "10.148.60.0" with node(s) fabushab-sunclu2,fabushab-sunclu1.


Interfaces found on subnet "10.148.60.0" that are likely candidates for VIP:fabushab-sunclu2 eri0:10.148.60.240fabushab-sunclu1 eri0:10.148.60.238


WARNING: Could not find a suitable set of interfaces for the private interconnect.


Checking system requirements for 'crs'...

Check: Total memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 1.12GB (1179648KB) 1GB (1048576KB) passed fabushab-sunclu1 2GB (2097152KB) 1GB (1048576KB) passed Result: Total memory check passed.

Check: Free disk space in "/tmp" dir Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 1.14GB (1196720KB) 400MB (409600KB) passed fabushab-sunclu1 5.2GB (5456440KB) 400MB (409600KB) passed Result: Free disk space check passed.

Check: Swap space Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 3.91GB (4095496KB) 1.5GB (1572864KB) passed fabushab-sunclu1 3.91GB (4095496KB) 1.5GB (1572864KB) passed Result: Swap space check passed.

heck: System architecture Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 64-bit sparcv9 kernel modules 64-bit passed fabushab-sunclu1 64-bit sparcv9 kernel modules 64-bit passedResult: System architecture check passed.

Check: Kernel version Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 10,REV=2006.11.14.15.02 10,REV=2005.01.22.19.14 passed fabushab-sunclu1 10,REV=2006.11.14.15.02 10,REV=2005.01.22.19.14 passed Result: Kernel version check passed.

Check: Package existence for "SUNWarc" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWarc:11.10.0 passed fabushab-sunclu1 SUNWarc:11.10.0 passed Result: Package existence check passed for "SUNWarc".

Check: Package existence for "SUNWbtool" Node Name Status Comment ------------------------------ ------------------------------ ----------------

4.1. Run CVU Pre Clusterware Install Checks 18

fabushab-sunclu2 SUNWbtool:11.10.0 passed fabushab-sunclu1 SUNWbtool:11.10.0 passed Result: Package existence check passed for "SUNWbtool".

Check: Package existence for "SUNWhea" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWhea:11.10.0 passed fabushab-sunclu1 SUNWhea:11.10.0 passed Result: Package existence check passed for "SUNWhea".

Check: Package existence for "SUNWlibm" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWlibm:5.10 passed fabushab-sunclu1 SUNWlibm:5.10 passed Result: Package existence check passed for "SUNWlibm".

Check: Package existence for "SUNWlibms" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWlibms:5.10 passed fabushab-sunclu1 SUNWlibms:5.10 passed Result: Package existence check passed for "SUNWlibms".

Check: Package existence for "SUNWsprot" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWsprot:5.10 passed fabushab-sunclu1 SUNWsprot:5.10 passed Result: Package existence check passed for "SUNWsprot".

Check: Package existence for "SUNWtoo" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWtoo:11.10.0 passed fabushab-sunclu1 SUNWtoo:11.10.0 passed Result: Package existence check passed for "SUNWtoo".

Check: Package existence for "SUNWi1of" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWi1of:6.6.2.7400 passed fabushab-sunclu1 SUNWi1of:6.6.2.7400 passed Result: Package existence check passed for "SUNWi1of".

Check: Package existence for "SUNWi1cs" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWi1cs:2.0 passed fabushab-sunclu1 SUNWi1cs:2.0 passed Result: Package existence check passed for "SUNWi1cs".


Check: Package existence for "SUNWxwfnt" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWxwfnt:6.6.2.7400 passed fabushab-sunclu1 SUNWxwfnt:6.6.2.7400 passed Result: Package existence check passed for "SUNWxwfnt".

Check: Package existence for "SUNWlibC" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWlibC:5.10 passed fabushab-sunclu1 SUNWlibC:5.10 passed Result: Package existence check passed for "SUNWlibC".

Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ fabushab-sunclu2 exists passed fabushab-sunclu1 exists passedResult: Group existence check passed for "dba".

Check: Group existence for "oinstall"

4.1. Run CVU Pre Clusterware Install Checks 19

Node Name Status Comment ------------ ------------------------ ------------------------ fabushab-sunclu2 exists passed fabushab-sunclu1 exists passedResult: Group existence check passed for "oinstall".

Check: User existence for "nobody" Node Name Status Comment ------------ ------------------------ ------------------------ fabushab-sunclu2 exists passed fabushab-sunclu1 exists passedResult: User existence check passed for "nobody".

System requirement passed for 'crs'

Pre-check for cluster services setup was successful.

4.2. Install Oracle Clusterware$ cd :/stage/core/SOLARIS_SPARC/64bit/rdbms/10.2.0.1.0/clusterware $ ./runInstaller

ActionClick Next

4.2. Install Oracle Clusterware 20

ActionIf the defaults for Inventory directory are not acceptable modify them, in this case we used/u01/app/oracle/oraInventoryClick Next

NotesThe OUI will name the Oracle Clusterware Home 'OraCrs10g', It will select the local drive that has the mostfree space and place the Oracle Clusterware home in. If you choose to modify the path for the home then youshould ensure that there is sufficient free space on that local drive on all nodes.ActionIf the defaults are not acceptable modify them.Click Next


NotesThe installer will then complete some Product-Specific Prerequisite checks. These should all pass OK - as youhave already run the CVU 'pre crs' stage checkActionClick Next

NotesHere we specify node specific details for the node that will make up the cluster. The installer assumes that thePrivate node name will be the %HOSTNAME%-priv and that the Virtual node name will be the%HOSTNAME%-vip. If these are not correct click Edit and modify as required. You need to add the othernode(s) to this screenActionClick Add


NotesHere we specify the networking details for the other node(s) in the cluster.ActionEnter the Public Node NameEnter the Private Node NameEnter the Virtual Host NameClick OK

NotesIf you have more than 2 nodes, repeat the previous ‘Add’ cycle to add more node details You should now seea list of the nodes and network names in the clusterActionClick Next


NotesHere we see a list of the detected network adapters on the nodes in the cluster. You may only have 2 listed.You must set one of these Interface Types to Public and one to Private. Oracle will guess, based on theassigned IP address, the usage of the network. If it guesses incorrectly you must change the usage. Here it hasguessed that all adapters are Private, which is incorrect.ActionSelect the Public Network AdapterClick Edit

NotesHere you specify the ‘Interface type’ Oracle will use this specific network forActionSelect the Private radio buttonClick OK


NotesNow we have set interface types correctly.ActionClick Next

NotesHere we specify the shared storage devices that will be used by Oracle Clusterware. Ideally you will have 2devices for the OCR, Oracle will mirror to these devices to protect you from a single OCR device failure.Also you will have an additional 3 vote devices to protect your cluster from the failure of a single vote device.ActionSelect the device to be used for the first OCRClick Next


NotesHere we specify the shared storage devices that will be used by Oracle Clusterware. You will need 3 devicesfor the Voting Files, Oracle will mirror to these devices to protect you from device failure.ActionEnter the names of the raw devices to be used for voting disksClick Next

NotesHere we can see a summary of the install.ActionClick Install


NotesHere the installer copies the software to all nodes in the clusterAction

NotesAt the end of the software copy a number of Configuration Assistants are run. It is essential that thesecomplete successfullyActionRun the following scripts on each node, one node at a time

Run the following on the first node

fabushab-sunclu1:/ # /u01/app/oracle/oraInventory/orainstRoot.shChanging permissions of /u01/app/oracle/oraInventory to 770.Changing groupname of /u01/app/oracle/oraInventory to oinstall.The execution of the script is complete

fabushab-sunclu1:/ # /u01/crs/root.sh


WARNING: directory '/u01' is not owned by rootChecking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directorySetting up Network socket directoriesOracle Cluster Registry configuration upgraded successfullyThe directory '/u01' is not owned by root. Changing owner to rootSuccessfully accumulated necessary OCR keys.Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.node <nodenumber>: <nodename> <private interconnect name> <hostname>node 1: fabushab-sunclu1 fabushab-sunclu1-priv fabushab-sunclu1node 2: fabushab-sunclu2 fabushab-sunclu2-priv fabushab-sunclu2Creating OCR keys for user 'root', privgrp 'root'..Operation successful.Now formatting voting device: /dev/rdsk/c1t4d0s4Now formatting voting device: /dev/rdsk/c1t4d0s5Now formatting voting device: /dev/rdsk/c1t4d0s7Format of 3 voting devices complete.Startup will be queued to init within 30 seconds.Adding daemons to inittabExpecting the CRS daemons to be up within 600 seconds.Cluster Synchronization Services is active on these nodes. fabushab-sunclu1Cluster Synchronization Services is inactive on these nodes. fabushab-sunclu2Local node checking complete. Run root.sh on remaining nodes to start CRS daemons.

Run the following on the Second node

fabushab-sunclu2:/ # /u01/app/oracle/oraInventory/orainstRoot.shChanging permissions of /u01/app/oracle/oraInventory to 770.Changing groupname of /u01/app/oracle/oraInventory to oinstall.The execution of the script is complete

fabushab-sunclu2:/ # /u01/crs/root.shChecking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directorySetting up NS directoriesOracle Cluster Registry configuration upgraded successfullyclscfg: EXISTING configuration version 3 detected.clscfg: version 3 is 10G Release 2.Successfully accumulated necessary OCR keys.Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.node <nodenumber>: <nodename> <private interconnect name> <hostname>node 1: fabushab-sunclu1 fabushab-sunclu1-priv fabushab-sunclu1node 2: fabushab-sunclu2 fabushab-sunclu2-priv fabushab-sunclu2clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.-force is destructive and will destroy any previous clusterconfiguration.Oracle Cluster Registry for cluster has already been initializedStartup will be queued to init within 30 seconds.Adding daemons to inittabExpecting the CRS daemons to be up within 600 seconds.CSS is active on these nodes. fabushab-sunclu1 fabushab-sunclu2CSS is active on all nodes.Waiting for the Oracle CRSD and EVMD to startWaiting for the Oracle CRSD and EVMD to startOracle CRS stack installed and running under init(1M)Running vipca(silent) for configuring nodeappsThe given interface(s), "eri0" is not public. Public interfaces should be used to configure virtual IPs.

fabushab-sunclu2:/ #

The above error in running vipca in silent mode is due to bug 4437727 that affects 10.2.0.1.0, to work aroundthe problem we need to run VIPCA on the second node

fabushab-sunclu2:/u01/crs/bin # ./vipca


Action

ActionClick Next


NotesHere you select the supported interfaces to run your VIP onActionSelect the Public interface and then click next

ActionHere you enter your VIP host name, IP Address and subnet and click next


ActionHere verify that the data is correct and click finish

NotesHere you see the progress of creating resources and starting them upAction


ActionClick OK

NotesThis page show you the summery and configuration resultsActionClick exit

Now verify every thing is up and running

fabushab-sunclu2:/u01/crs/bin # ./crs_stat -tName Type Target State Host------------------------------------------------------------ora....lu1.gsd application ONLINE ONLINE fabu...clu1ora....lu1.ons application ONLINE ONLINE fabu...clu1ora....lu1.vip application ONLINE ONLINE fabu...clu1ora....lu2.gsd application ONLINE ONLINE fabu...clu2ora....lu2.ons application ONLINE ONLINE fabu...clu2ora....lu2.vip application ONLINE ONLINE fabu...clu2


------------ ------------------------

NotesAfter that configuration assistants have to runAction

NotesAfter the configuration assistants have run a final end if installation screen is displayedActionClick Exit


NotesAre you SureActionClick Yes

Congratulations, You have completed your Oracle Clusterware install.

4.3. Run CVU Post Clusterware Install Checks$ ./runcluvfy.sh stage -post crsinst -n fabushab-sunclu1,fabushab-sunclu2 -verbose

Performing post-checks for cluster services setup





Checking Cluster manager integrity...

Checking CSS daemon... Node Name Status ------------------------------------ ------------------------ fabushab-sunclu2 running fabushab-sunclu1 running Result: Daemon status check passed for "CSS daemon".

Cluster manager integrity check passed.

Checking cluster integrity...

4.3. Run CVU Post Clusterware Install Checks 34

Node Name ------------------------------------ fabushab-sunclu1 fabushab-sunclu2

Cluster integrity check passed

Checking OCR integrity...

Checking the absence of a non-clustered configuration...All nodes free of non-clustered, local-only configurations.

Uniqueness check for OCR device passed.

Checking the version of OCR...OCR of correct Version "2" exists.

Checking data integrity of OCR...Data integrity check for OCR passed.

OCR integrity check passed.

Checking CRS integrity...

Checking daemon liveness...

Check: Liveness for "CRS daemon" Node Name Running ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yes Result: Liveness check passed for "CRS daemon".


Check: Liveness for "CSS daemon" Node Name Running ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yes Result: Liveness check passed for "CSS daemon".


Check: Liveness for "EVM daemon" Node Name Running ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yes Result: Liveness check passed for "EVM daemon".

Liveness of all the daemons Node Name CRS daemon CSS daemon EVM daemon ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 yes yes yes fabushab-sunclu1 yes yes yes

Checking CRS health...

Check: Health of CRS Node Name CRS OK? ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yes Result: CRS health check passed.

CRS integrity check passed.

Checking node application existence...

Checking existence of VIP node application Node Name Required Status Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 yes exists passed

4.3. Run CVU Post Clusterware Install Checks 35

fabushab-sunclu1 yes exists passed Result: Check passed.

Checking existence of ONS node application Node Name Required Status Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 no exists passed fabushab-sunclu1 no exists passed Result: Check passed.

Checking existence of GSD node application Node Name Required Status Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 no exists passed fabushab-sunclu1 no exists passed

Result: Check passed.

Post-check for cluster services setup was successful.

5. Oracle Clusterware patching

At this point we have installed Oracle Clusterware 10.2.0.1. In this section we will patch the OracleClusterware to the latest release of Oracle 10gR2 - 10.2.0.4. The patchset can be downloaded from metalink.Note: The Oracle Clusterware software must be at the same or newer level as the Oracle software in theOracle RAC Oracle home. Therefore, you should always upgrade Oracle Clusterware before you upgradeOracle RAC. This patch does not allow you to upgrade Oracle RAC before you upgrade Oracle Clusterware.We will apply this patchset in a non rolling upgrade.

The first step in the application of this patchset requires to shutdown the Oracle Clusterware on all nodes inthe RAC configuration. Login as the root and issue the following command on all the nodes:

#$CRS_home/bin/crsctl stop crs

Enter the following commands to start Oracle Universal Installer, where patchset_directory is the directorywhere you unpacked the patch set software:

$ cd patchset_directory/Disk1$ ./runInstaller

On the Welcome screen, click Next.

5. Oracle Clusterware patching 36

Notes

Specify the name and the location of the clusterware home♦

•

Action

Check that the name and location are correct♦

•

Notes

Both nodes in the RAC Cluster are displayed♦

•

Action

Click Next♦

•

Notes

The installer will validate the state of the cluster before continuing.♦

•

Action•


If there are issues you should rectify them before continuing.♦ Click Next♦

Notes

the installer lists a summary of the planned actions♦

•

Actions

Click Install♦

•

Notes

the installer installs the software♦

•

Actions

none required♦

•


Notes

The installer requires to run root102.sh as root on each the node♦

•

Action (to be performed on each node, one after the other)

Log in as the root user and enter the following command to shut down the OracleClusterware:

# CRS_home/bin/crsctl stop crs

♦

Run the root102.sh script. It will automatically start the Oracle Clusterware on the patchednode:

# CRS_home/install/root102.sh

♦

•

6. Install Oracle ASM Software only Home$ cd /stage/core/SOLARIS_SPARC/64bit/rdbms/10.2.0.1.0/database$ ./runInstaller

Here we establish the Oracle software for the ASM home.

6. Install Oracle ASM Software only Home 39

ActionClick Next

NotesHere we can choose the type of database software to be installedActionSelect the Enterprise Edition radio buttonClick Next


NotesHere we specify the location of various components. Modify as required. Also this is where you specify thelocation of the ASM software home. I usually modify the Home name and Home Path to include the wordASM. This makes it easier to identify later on. If you change the path you should ensure that you do not usethe exact same path as the Oracle Clusterware homeActionConfirm entries are OKClick Next

NotesThe installer has detected the presence of Oracle Clusterware and uses this to populate this dialog box. Tobuild a cluster which includes all nodes you must ensure that there are check-boxes next to the node namesActionClick Select AllClick Next


NotesThe installer will then complete some Product-Specific Prerequisite checks. These should all pass OK – asyou have already run the CVU checkActionClick Next

NotesWe are going to install a Software only home and then subsequently configure the softwareActionSelect the Install Software Only radio buttonClick Next





ActionRun the scripts on each node as root, when finished, come back here and click OK

fabushab-sunclu1:/ # /u01/app/oracle/product/10.2.0/asm/root.shRunning Oracle10 root.sh script...

The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/10.2.0/asm

Enter the full pathname of the local bin directory: [/usr/local/bin]:The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying dbhome to /usr/local/bin ...The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying oraenv to /usr/local/bin ...The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying coraenv to /usr/local/bin ...

Creating /var/opt/oracle/oratab file...Entries will be added to the /var/opt/oracle/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.


On the Second Node

fabushab-sunclu2:/ # /u01/app/oracle/product/10.2.0/asm/root.shRunning Oracle10 root.sh script...

The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/10.2.0/asm


Creating /var/opt/oracle/oratab file...Entries will be added to the /var/opt/oracle/oratab file as needed by


Database Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.


NotesAfter the software install completes you will see this End of Installation dialogActionClick Exit



7. Oracle ASM Software Home Patching

Once ASM software has been installed, the next step in the process is to apply the 10.2.0.4 patchset to theASM ORACLE_HOME. If an Automatic Storage Management instance is present, then shut down theAutomatic Storage Management instance. To shut down an Automatic Storage Management instance, enterthe following command where node is the name of the node where the Automatic Storage Managementinstance is running:

# srvctl stop asm -n node




Notes

Specify the name and the location of the asm home♦

•

Action


•

7. Oracle ASM Software Home Patching 46

Notes


•

Action

Click Next.♦

•

Notes

The installer will run Product-specific Prerequisite Checks .♦

•

Action


•


Notes

On this screen, you will register with Oracle Configuration Manager (OCM)♦

•

Action

If you want to register with OCM, enter details regarding the CSI Number, OracleMetaLink?Account user name and Country code (optional - The registration and configuration can alsobe done manually after patchset installation)

♦

Click Next♦

•

Notes

Here we can see a summary of the install.♦

•

Actions

Click Install♦

•


http://globaldc.oracle.com/perl/twiki/edit/RACGuides/OracleMetaLink?topicparent=RACGuides.Rac10gR2OnSolaris

Notes

The installer pauses, root.sh need to be run as root on both nodes of the cluster♦

•

Action

open a shell window on each node and run root.sh (one after the other)♦ then return to the installer and Click OK♦

•

Action

The installation is done. Click Exit to exit the installer♦ Click Yes in the box♦

•

Oracle ASM software home has been patched to 10.2.0.4


8. Create Node Specific Network Listeners

The Oracle network listeners traditionally run from the ASM home. Here we are going to create the listenersusing netca from the ASM home

fabushab-sunclu1:/u01/oracle $ cd /u01/app/oracle/product/10.2.0/asm/binfabushab-sunclu1:/u01/app/oracle/product/10.2.0/asm/bin $ ./netca

NotesNetca detects that the Oracle Clusterware layer is running and offers Cluster or Single Node configurationActionSelect the Cluster configuration radio buttonClick Next

NotesNetca uses Oracle Clusterware to determine all the nodes in the clusterAction

8. Create Node Specific Network Listeners 50

Click Select all nodesClick Next

NotesYou get various options - we need to configure listenersActionSelect the Listener configuration radio buttonClick Next

NotesWe need to add a listenerActionSelect the Add radio buttonClick Next


NotesHere you get the opportunity to name the listener – Do not change this. The listeners will eventually be calledLISTSNER_nodename1 & LISTENER_nodename2. This is important for RACActionClick Next

NotesOracle Net supports various network protocols, although TCP is the most common.ActionEnsure the Select Protocols list includes TCPClick Next


NotesIt is possible to choose a non-default Port – I do not recommend thisActionEnsure the Use the Standard port number of 1521 radio button is setClick Next

NotesAfter configuring the node listeners you get the opportunity to configure more network componentsActionSelect the No radio buttonClick Next


ActionClick Next

ActionClick Finish

You have now completed the RAC cluster nodes Network listener configuration. A new managed resourcewill have been added to Oracle Clusterware OCR

fabushab-sunclu1:/ # crs_stat -tName Type Target State Host------------------------------------------------------------ora....U1.lsnr application ONLINE ONLINE fabu...clu1ora....lu1.gsd application ONLINE ONLINE fabu...clu1ora....lu1.ons application ONLINE ONLINE fabu...clu1ora....lu1.vip application ONLINE ONLINE fabu...clu1ora....U2.lsnr application ONLINE ONLINE fabu...clu2ora....lu2.gsd application ONLINE ONLINE fabu...clu2ora....lu2.ons application ONLINE ONLINE fabu...clu2ora....lu2.vip application ONLINE ONLINE fabu...clu2



9. Create ASM Instances and initial ASM disk groups

After installing ASM Software and creating listeners, we can now create the ASM instances and diskgroups tobe used by the RAC databases. We use the dbca from the ASM home to create the ASM instances

fabushab-sunclu1:/u01/crs/bin $ cd /u01/app/oracle/product/10.2.0/asm/binfabushab-sunclu1:/u01/app/oracle/product/10.2.0/asm/bin $ ./dbca

Notesdbca detects the Oracle Clusterware layer is running and offers to create either cluster or single instancedatabaseActionSelect the Oracle Real Application Clusters database radio buttonClick Next

9. Create ASM Instances and initial ASM disk groups 55

NotesWe use dbca to create RAC databases and ASM instancesActionSelect the Configure Automatic Storage Management radio buttonClick Next

NotesYou need to make sure you create ASM instances on all the cluster nodesActionClick Select AllClick Next

NotesHere we specify the password for the ASM Oracle SYS userActionEnter the SYS password


Enter the same password for the Confirm SYS passwordClick Next

Notesdbca will create and start ASM instancesActionClick OK

NotesThis should take a few secondsAction


NotesASM requires disks to be group together into diskgroups. This section will be used to create 2 disk groups+DATA and +FLASHActionClick Create New

NotesAt the moment we have 5 candidate disksActionCreate +DATA diskgroup


NotesNow we will assign disks to specific disk groups and create the DATA diskgroupActionIn the Disk Group Name enter DATASelect the External Redundancy radio buttonSelect the Show Candidate disks radio buttonSelect the first 2 disksClick OK

NotesA progress message – this should not take very longAction


NotesHere we can see the DATA diskgroup has been created and is mounted on 2/2 instances. We now need tocreate the FLASH diskgroupActionClick Create New

NotesNow we will assign disks to specific disk groups and create the FLASH diskgroupActionIn the Disk Group Name enter FLASHSelect the External Normal radio buttonSelect the Show Candidate disks radio buttonSelect the remaining 2 disks,Click OK


NotesA progress message – this should not take very longAction

NotesHere we can see the DATA and FLASH diskgroups have been created and are mounted on 2/2 instances. Thiscompleted the ASM configuration.ActionClick Finish


NotesA confirmation dialog boxActionClick No

10. Install Oracle RAC Database Software only Home

10.1. Run CVU Pre Database Install Check$ ./runcluvfy.sh stage -pre dbinst -n fabushab-sunclu1,fabushab-sunclu2 -r 10gR2 -verbose

Performing pre-checks for database installation





Checking administrative privileges...

Check: Existence of user "oracle" Node Name User Exists Comment ------------ ------------------------ ------------------------ fabushab-sunclu2 yes passed fabushab-sunclu1 yes passed Result: User existence check passed for "oracle".

Check: Existence of group "oinstall" Node Name Status Group ID ------------ ------------------------ ------------------------ fabushab-sunclu2 exists 100 fabushab-sunclu1 exists 100 Result: Group existence check passed for "oinstall".

10. Install Oracle RAC Database Software only Home 62

Check: Membership of user "oracle" in group "oinstall" [as Primary] Node Name User Exists Group Exists User in Group Primary Comment ---------------- ------------ ------------ ------------ ------------ ------------ fabushab-sunclu2 yes yes yes yes passed fabushab-sunclu1 yes yes yes yes passed Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed.

Check: Existence of group "dba" Node Name Status Group ID ------------ ------------------------ ------------------------ fabushab-sunclu2 exists 101 fabushab-sunclu1 exists 101 Result: Group existence check passed for "dba".

Check: Membership of user "oracle" in group "dba" Node Name User Exists Group Exists User in Group Primary Comment ---------------- ------------ ------------ ------------ ------------ ------------ fabushab-sunclu2 yes yes yes no passed fabushab-sunclu1 yes yes yes no passed Result: Membership check for user "oracle" in group "dba" passed.

Administrative privileges check passed.


Interface information for node "fabushab-sunclu2" Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address ---------------- ------------ ------------ ------------ ------------ ------------ eri0 10.148.60.240 10.148.60.0 10.148.60.240 UNKNOWN 00:03:ba:0f:1a:83 eri0 10.148.60.241 10.148.60.0 10.148.60.240 UNKNOWN rge0 192.168.20.4 192.168.20.0 192.168.20.4 UNKNOWN 00:14:d1:38:82:b6

Interface information for node "fabushab-sunclu1" Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address ---------------- ------------ ------------ ------------ ------------ ------------ eri0 10.148.60.238 10.148.60.0 10.148.60.238 UNKNOWN 00:03:ba:1d:9b:63 eri0 10.148.60.239 10.148.60.0 10.148.60.238 UNKNOWN rge0 192.168.20.3 192.168.20.0 192.168.20.3 UNKNOWN 00:14:d1:38:7d:67

Check: Node connectivity of subnet "10.148.60.0" Source Destination Connected? ------------------------------ ------------------------------ ---------------- fabushab-sunclu2:eri0 fabushab-sunclu2:eri0 yes fabushab-sunclu2:eri0 fabushab-sunclu1:eri0 yes fabushab-sunclu2:eri0 fabushab-sunclu1:eri0 yes fabushab-sunclu2:eri0 fabushab-sunclu1:eri0 yes fabushab-sunclu2:eri0 fabushab-sunclu1:eri0 yes fabushab-sunclu1:eri0 fabushab-sunclu1:eri0 yes Result: Node connectivity check passed for subnet "10.148.60.0" with node(s) fabushab-sunclu2,fabushab-sunclu1.


Interfaces found on subnet "10.148.60.0" that are likely candidates for VIP:fabushab-sunclu2 eri0:10.148.60.240 eri0:10.148.60.241fabushab-sunclu1 eri0:10.148.60.238 eri0:10.148.60.239


WARNING: Could not find a suitable set of interfaces for the private interconnect.


Checking system requirements for 'database'...

Check: Total memory Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 1.12GB (1179648KB) 1GB (1048576KB) passed fabushab-sunclu1 2GB (2097152KB) 1GB (1048576KB) passed Result: Total memory check passed.

10.1. Run CVU Pre Database Install Check 63

Check: Free disk space in "/tmp" dir Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 4.25GB (4456152KB) 400MB (409600KB) passed fabushab-sunclu1 4.25GB (4456152KB) 400MB (409600KB) passed Result: Free disk space check failed.

Check: Swap space Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 3.91GB (4095496KB) 1.5GB (1572864KB) passed fabushab-sunclu1 3.91GB (4095496KB) 1.5GB (1572864KB) passed Result: Swap space check failed.

Check: System architecture Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 64-bit sparcv9 kernel modules 64-bit passed fabushab-sunclu1 64-bit sparcv9 kernel modules 64-bit passed Result: System architecture check passed.

Check: Kernel version Node Name Available Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 10,REV=2006.11.14.15.02 10,REV=2005.01.22.19.14 passed fabushab-sunclu1 10,REV=2006.11.14.15.02 10,REV=2005.01.22.19.14 passed Result: Kernel version check passed.

Check: Package existence for "SUNWarc" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWarc:11.10.0 passed fabushab-sunclu1 SUNWarc:11.10.0 passed Result: Package existence check passed for "SUNWarc".

Check: Package existence for "SUNWbtool" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWbtool:11.10.0 passed fabushab-sunclu1 SUNWbtool:11.10.0 passed Result: Package existence check passed for "SUNWbtool".

Check: Package existence for "SUNWhea" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWhea:11.10.0 passed fabushab-sunclu1 SUNWhea:11.10.0 passed Result: Package existence check passed for "SUNWhea".

Check: Package existence for "SUNWlibm" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWlibm:5.10 passed fabushab-sunclu1 SUNWlibm:5.10 passed Result: Package existence check passed for "SUNWlibm".

Check: Package existence for "SUNWlibms" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWlibms:5.10 passed fabushab-sunclu1 SUNWlibms:5.10 passed Result: Package existence check passed for "SUNWlibms".

Check: Package existence for "SUNWsprot" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWsprot:5.10 passed fabushab-sunclu1 SUNWsprot:5.10 passed Result: Package existence check passed for "SUNWsprot".

Check: Package existence for "SUNWtoo" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWtoo:11.10.0 passed fabushab-sunclu1 SUNWtoo:11.10.0 passed Result: Package existence check passed for "SUNWtoo".

Check: Package existence for "SUNWi1of" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWi1of:6.6.2.7400 passed


fabushab-sunclu1 SUNWi1of:6.6.2.7400 passed Result: Package existence check passed for "SUNWi1of".



Check: Package existence for "SUNWxwfnt" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWxwfnt:6.6.2.7400 passed fabushab-sunclu1 SUNWxwfnt:6.6.2.7400 passed Result: Package existence check passed for "SUNWxwfnt".

Check: Package existence for "SUNWlibC" Node Name Status Comment ------------------------------ ------------------------------ ---------------- fabushab-sunclu2 SUNWlibC:5.10 passed fabushab-sunclu1 SUNWlibC:5.10 passed Result: Package existence check passed for "SUNWlibC".

Check: Kernel parameter for "project.max-sem-ids" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 128 100 passed fabushab-sunclu1 128 100 passed Result: Kernel parameter check passed for "project.max-sem-ids".

Check: Kernel parameter for "process.max-sem-nsems" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 512 256 passed fabushab-sunclu1 512 256 passed Result: Kernel parameter check passed for "process.max-sem-nsems".

Check: Kernel parameter for "project.max-shm-memory" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 4294967295 4294967295 passed fabushab-sunclu1 4294967295 4294967295 passed Result: Kernel parameter check failed for "project.max-shm-memory".

Check: Kernel parameter for "project.max-shm-ids" Node Name Configured Required Comment ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 128 100 passed fabushab-sunclu1 128 100 passed Result: Kernel parameter check passed for "project.max-shm-ids".

Check: Group existence for "dba" Node Name Status Comment ------------ ------------------------ ------------------------ fabushab-sunclu2 exists passed fabushab-sunclu1 exists passed Result: Group existence check passed for "dba".

Check: User existence for "nobody" Node Name Status Comment ------------ ------------------------ ------------------------ fabushab-sunclu2 exists passed fabushab-sunclu1 exists passed Result: User existence check passed for "nobody".

System requirement failed for 'database'

Checking CRS integrity...


Check: Liveness for "CRS daemon" Node Name Running ------------------------------------ ------------------------


fabushab-sunclu2 yes fabushab-sunclu1 yes Result: Liveness check passed for "CRS daemon".


Check: Liveness for "CSS daemon" Node Name Running ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yes Result: Liveness check passed for "CSS daemon".


Check: Liveness for "EVM daemon" Node Name Running ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yes Result: Liveness check passed for "EVM daemon".

Liveness of all the daemons Node Name CRS daemon CSS daemon EVM daemon ------------ ------------------------ ------------------------ ---------- fabushab-sunclu2 yes yes yes fabushab-sunclu1 yes yes yes

Checking CRS health...

Check: Health of CRS Node Name CRS OK? ------------------------------------ ------------------------ fabushab-sunclu2 yes fabushab-sunclu1 yes Result: CRS health check passed.

CRS integrity check passed.

Pre-check for database installation was successful.

10.2. Install Oracle RDBMS Software$ cd /stage/core/SOLARIS_SPARC/64bit/rdbms/10.2.0.1.0/database$ ./runInstaller

ActionClick Next

10.2. Install Oracle RDBMS Software 66

NotesHere we can choose the type of database software to be installedActionSelect the Enterprise Edition radio buttonClick Next

NotesHere we specify the location of various components. If you change the path you should ensure that you do notuse the exact same path as the Oracle Clusterware or ASM homesActionConfirm entries are OKClick Next


NotesThe installer has detected the presence of Oracle Clusterware and uses this to populate this dialog box. Tobuild a cluster which includes all nodes you must ensure that there are check-boxes next to the node namesActionClick Select AllClick Next

NotesThe installer will then complete some Product-Specific Prerequisite checks. These should all pass OK - as youhave already run the CVU checkActionClick Next


NotesWe are going to install a Software only home and then subsequently configure the softwareActionSelect the Install Software Only radio buttonClick Next




ActionRun the scripts on each node as root, when finished, come back here and click OK

fabushab-sunclu1:/ # /u01/app/oracle/product/10.2.0/db/root.shRunning Oracle10 root.sh script...

The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/10.2.0/db

Enter the full pathname of the local bin directory: [/usr/local/bin]:The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying dbhome to /usr/local/bin ...The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying oraenv to /usr/local/bin ...


The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n) [n]: y Copying coraenv to /usr/local/bin ...

Entries will be added to the /var/opt/oracle/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.

On the Second Node

fabushab-sunclu1:/ # /u01/app/oracle/product/10.2.0/db/root.shRunning Oracle10 root.sh script...

The following environment variables are set as: ORACLE_OWNER= oracle ORACLE_HOME= /u01/app/oracle/product/10.2.0/db


Entries will be added to the /var/opt/oracle/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.

NotesAfter the software install completes you will see this End of Installation dialogActionClick Exit



11. Oracle RAC Software Home Patching

Once RAC software has been installed, the next step in the process is to apply the 10.2.0.4 patchset to theRAC ORACLE_HOME.

Ensure that all databases using the to-be-patched Oracle Home are fully shutdown on all nodes, then proceed.




11. Oracle RAC Software Home Patching 72

Notes

Specify the name and the location of the RAC home♦

•

Action


•

Notes


•

Action

Click Next.♦

•

Notes

The installer will run Product-specific Prerequisite Checks .♦

•

Action


•


Notes

On this screen, you will register with Oracle Configuration Manager (OCM)♦

•

Action

If you want to register with OCM, enter details regarding the CSI Number, OracleMetaLink?Account user name and Country code (optional - The registration and configuration can alsobe done manually after patchset installation)

♦

Click Next♦

•

Notes

Here we can see a summary of the install.♦

•

Actions

Click Install♦

•


http://globaldc.oracle.com/perl/twiki/edit/RACGuides/OracleMetaLink?topicparent=RACGuides.Rac10gR2OnSolaris

Notes

The installer pauses, root.sh need to be run as root on all nodes of the cluster♦

•

Action

open a shell window on each node and run root.sh (one after the other)♦ then return to the installer and Click OK♦

•


Action

The installation is done. Click Exit to exit the installer♦ Click Yes in the box♦

•

Oracle RAC software home has been patched to 10.2.0.4

12. Create RAC database

We can now create the DB instances. We use the dbca from the DB home to create the DB

fabushab-sunclu1:/u01/crs/bin $ cd /u01/app/oracle/product/10.2.0/db/binfabushab-sunclu1:/u01/app/oracle/product/10.2.0/db/bin $ ./dbca

NotesYou must use the dbca from the RAC database home - not the ASM home. dbca detects the OracleClusterware layer is running and offers to create either cluster or single instance databaseActionSelect the Oracle Real Application Clusters database radio buttonClick Next

12. Create RAC database 76

NotesWe use dbca to create RAC databases and ASM instancesActionSelect the Create a Database radio buttonClick Next

NotesYou need to make sure you create RAC database instances on all the cluster nodesActionClick Select AllClick Next


NotesHere you get to select the type of databaseActionSelect the General Purpose... radio buttonClick Next

NotesNow you specify the preface for the SIDs for the RAC database instances - recommendation is to keep itsimpleActionEnter RAC in the Global Database name, The SID should autofillClick Next


NotesHere you get to configure Oracle Enterprise Manager. If you have no Grid Control server then the bestmethod is to use dbcontrol as detailed here.ActionCheck the Configure Enterprise manager checkboxSelect the Configure Database Control for local management radio buttonClick Next

NotesHere we specify the password for the Oracle Database users. In this example we are setting the samepassword for all users. You should set a password scheme that meets your requirements.ActionSelect the Use the Same Administrative Password.... radio buttonEnter the passwordEnter the same password for the Confirm passwordClick Next


NotesHere you specify where you would like your database datafiles stored. We are going to use ASMActionSelect the Automatic Storage Management (ASM) radio buttonClick Next

Notesdbca displays the diskgroups we previously created. We are going to use the DATA diskgroupActionCheck the box on the DATA diskgroup lineClick Next


NotesHere we specify recovery configuration information. We are going to use a flash recovery area.ActionCheck the Specify Flash Recovery Area checkboxIn the Flash recovery area enter +FLASHSet the flash recovery size - remember the size of the +FASH diskgroupCheck the Enable Archiving checkboxClick Next

NotesHere you can choose to add the Oracle Sample schemasActionClick Next


NotesHere you can choose to add DB ServicesActionFor this installation, we will not create any service while creating the DBClick Next

NotesHere you can modify the default initialisation parameters.ActionClick Next


NotesHere you can review the placement of various database filesActionClick Next

NotesYou can decide to generate database creation scriptsActionCheck the Create Database checkboxClick Finish


NotesA summary screenActionClick OK

NotesThe database is in the process to get createdAction


NotesAfter the database is created the summary screen is displayed - note the URL for the Database ControlActionClick Exit

NotesThe Database instances will be started on all nodesAction

The RAC database install is complete, Let's verify that every thing is up and running on all nodes

fabushab-sunclu1:/u01/oracle $ crs_stat -tName Type Target State Host------------------------------------------------------------ora....SM1.asm application ONLINE ONLINE fabu...clu1ora....U1.lsnr application ONLINE ONLINE fabu...clu1ora....lu1.gsd application ONLINE ONLINE fabu...clu1ora....lu1.ons application ONLINE ONLINE fabu...clu1


ora....lu1.vip application ONLINE ONLINE fabu...clu1ora....SM2.asm application ONLINE ONLINE fabu...clu2ora....U2.lsnr application ONLINE ONLINE fabu...clu2ora....lu2.gsd application ONLINE ONLINE fabu...clu2ora....lu2.ons application ONLINE ONLINE fabu...clu2ora....lu2.vip application ONLINE ONLINE fabu...clu2ora.rac.db application ONLINE ONLINE fabu...clu1ora....c1.inst application ONLINE ONLINE fabu...clu1ora....c2.inst application ONLINE ONLINE fabu...clu2


Documents

RACGuides_Rac10gR2OnSolaris