12c_RAC_on_OEL7 - · PDF file5/12/2016 · The Oracle Database 12c R1 for Linux x86-64: ... installing required rpms, creating oracle user, required groups, setting parameters in

12cR1 RAC | OEL 7.2 Page 1

This RAC installation article is based on RAC Attack hands-on learning lab

https://en.wikibooks.org/wiki/RAC_Attack_-_Oracle_Cluster_Database_at_Home/RAC_Attack_12c

The article performs the following task:-

1. Deploying Oracle Grid Infrastructure 12cR1 (12.1.0.2.0)

2. Deploying Oracle database software 12cR1 (12.1.0.2.0)

3. Using Oracle ASM with udev rules

Below the hardware details

Windows 7 64 bit

8 CPUs

16Gb memory, 8GB is used for the RAC

50Gb for VDI (Dynamic Allocation)

30Gb for ASM disk (Fixed Size)

Below are the software components

The VirtualBox for Windows: https://www.virtualbox.org/wiki/Downloads

The Oracle Linux (OEL 7.2 OS -- V100082-01.iso) : https://edelivery.oracle.com/linux

The Oracle Database 12c R1 for Linux x86-64:

http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html

1. Complete the Installation of Oracle VM VirtualBox

2. Run VirtualBox Manager.

3. From the main screen, choose File > Preferences > Network.

4. Double click on VirtualBox Host-Only Ethernet Adapter. VirtualBox will suggest a default

address of 192.168.56.1. Please update the settings as shown and click Ok twice.

https://en.wikibooks.org/wiki/RAC_Attack_-_Oracle_Cluster_Database_at_Home/RAC_Attack_12c

https://www.virtualbox.org/wiki/Downloads

https://edelivery.oracle.com/linux

http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html

https://en.wikibooks.org/wiki/RAC_Attack_-_Oracle_Cluster_Database_at_Home/RAC_Attack_12c/VirtualBox_Setup


IPv4 Address: 192.168.78.1

IPv4 Network Mask: 255.255.255.0

5. In VirtualBox, from the main screen, click the New icon in the upper left hand corner.


6. Set the memory size. I have set to 4G.

7. Create virtual hard drive.


8. Leave VDI selected and click next.

9. Leave dynamically allocated selected and click Next.


10. Type in the full path or choose through the browse icon the Location for the disk file to be

created. Have named the file “racnode1.vdi” and all the vdi files created are in location

“D:\rac12c\”.


11. From the main screen, select the virtual machine racnode1 and click the Settings icon in the

upper left hand corner.


12. Open the Storage sub-menu. Delete the Empty disk under the IDE Controller.

13. Click on the Add CD/DVD/Optical Device.


14. Locate and open the Oracle_Linux_7_64.iso (V100082-01.iso) file.


15. Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to: dropdown to

Host-only Adapter.

16. Choose the Adapter 2 tab. Check the box for Enable Network Adapter. Change the Attached to:

dropdown to Internal Network and type the name “rac-priv” in the Name field.


17. Choose the Adapter 3 tab. Check the box for Enable Network Adapter. Change the

Attached to: dropdown to NAT.

18. Select the Shared Folders tab. Click Add Shared Folder. Enter the path where you have

downloaded the Oracle installation media and enter a name for your folder.


19. Open the USB sub-menu. Uncheck the Enable USB Controller check box.

20. Click on Start to boot the virtual machine.


21. The Oracle Linux install screen will automatically come up.

22. Select the language and click on “Continue” .


23. Next is the “Installation Summary” screen.

24. Select the “Installation Destination”. If you are happy with automatic partitioning of

the whole disk, click “Done” button to return to previous screen.


25. On “Software Selection” screen select “Server with GUI” for your “Base Environment”.

Then select extra software you want to install by selecting it in “Add-Ons for Selected

Environment”. For Oracle software installation and general development on Oracle

Linux 7 please choose the below option and click “Done” button.


26. Select the “Network & Host Name” section from “Installation Summary”.


27. Select Ethernet (enp0s3), the “Public IP” and click on Configure. In earlier releases of

OEL we had eth0, eth1, eth2, which is now enp0s3, enp0s8, enp0s9

a. Click the “Connect Automatically” checkbox in General Tab

b. Click on IPv4 Settings and select “Manual” method.

c. Click “Add” and type “192.168.78.51” for the address and “255.255.255.0” for

Netmask

d. Type 192.168.78.51, 192.168.78.52 in DNS servers and “racattack” in search domain.

e. In Hostname type “racnode1.racattack”.

f. Click on Save


28. Select Ethernet (enp0s8) and click on “Configure”

a. Click on “Connect Automatically” checkbox in General Tab

b. Move to “IPv4 Settings” , change to “Manual” method and click on “Add”

c. Type “172.16.100.51” in Address and “255.255.255.0” in Netmask

d. Save the settings


29. Select Ethernet (enp0s9) and click on “Configure”

a. Click on “Connect Automatically” checkbox in General Tab

b. Move to “IPv4 Settings” , change to “Automatic (DHCP) addresses only”

method



30. Click on “Begin Installation” and the screen changes to “CONFIGURATION” window.

31. Set the “ROOT PASSWORD” and “USER CREATION” while the installation is going on.


32. Completion Screen: - After the Oracle Linux is successfully installed, go ahead and

reboot it. After reboot you would need to complete “License Information”.



33. Turn off and disable the Firewall. “systemctl” and “systemd” have replaced chkconfig

and init under OEL7.

34. Change the setting of SELinux to “disabled” by editing “/etc/selinux/config”.


35. Check the “ntpd” status. Either configure NTP, or make sure it is not configured so that

CTSSd can synchronize the times of RAC node. In my installation I de-configured NTP.

# systemctl disable ntpd

# mv /etc/ntp.conf /etc/ntp.conf.orig

36. Verify that all the network interfaces are up.


37. Configure DNS.


Original “/etc/named.conf” file before any changes -

Add “192.168.78.51” to “listen-on port 53” and “192.168.78.0/24” to “allow-query” and “allow-

transfer”.


At the end of file add the “zone” and “reverse zone” as shown below.

Create the zone file for the racattack domain on racnode1.


Create the reverse zone file

Verify the changes made to “/etc/named.conf” file as shown below. In case of any errors, rectify it

and re-check.


Enabled the named service and started it

Check the forward and reverse lookup using “nslookup” command.



38. Install the database preinstall package “oracle-rdbms-server-12cR1-preinstall”.

Select the “Devices > Optical Drives > oracle_linux_7_64.iso”


Run the following commands.

Make a backup copy of file, delete the content and add the below lines to /etc/yum.repos.d/public-

yum-ol7.repo.

Install the “oracle-rdbms-server-12cR1-preinstall” package. It performs all prerequisite setup like

installing required rpms, creating oracle user, required groups, setting parameters in

/etc/sysctl.conf etc.


One thing to note is, Oracle user limit does not reflect values in /etc/security/limit.conf if oracle-

rdbms-server-12cR1-preinstall RPM is used. The file to be checked is “/etc/security/limits.d/oracle-

rdbms-server-12cR1-preinstall.conf”.


39. Unmount/Eject the device (oracle_linux_7_64.iso) and Make the VirtualBox guest

additions available to the OS by clicking Devices->Install Guest Additions.


40. After the guest addition, the Shared Folders added in step 18 should be visible. The folder is visible as /media/sf_12cR1.

41. Create the directories in which the Oracle Software will be installed and add oracle to

“vboxsf” group.


42. Create Virtualbox Shared Storage (ASM Disks)

a. Shutdown the server.

b. In the VirtualBox Manager, select the machine racnode1, click Settings -> Storage.

c. Select Controller SATA and click on the Add Hard Disk button. Click “Create new disk”.



43. Repeat the same steps to create three more disks:

44. Make all the asm devices sharable: from VirtualBox Manager, click File -> Virtual Media

Manager.


Repeat the same above steps for all the asm*.vdi disk.


45. Start the racnode1 and you must see 4 disks named /dev/sd*.

46. Once the new disks are visible in the server, add primary partition on each of them using

“fdisk”.


47. Add the following to the "/etc/scsi_id.config" file to configure SCSI devices as trusted.

Create the file if it doesn't already exist.

# echo “options=-g” > /etc/scsi_id.config

Check if scsi_id returns data.

# /usr/lib/udev/scsi_id –g –u –d /dev/sdb

Prepare the “/etc/udev/rules.d/99-oracle-asmdevices.rules” by running the below script.

#!/bin/bash

#file ~/createOracleAsmUdevRules.sh

i=1

# ol7 / rhel7

cmd="/usr/lib/udev/scsi_id -g -u -d"

for disk in sdb sdc sdd sde ; do

cat <<EOF >> /etc/udev/rules.d/99-oracle-asmdevices.rules

KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="$cmd /dev/\$parent", \

RESULT=="`$cmd /dev/$disk`", SYMLINK+="asm-disk$i", OWNER="oracle", GROUP="dba", \

MODE="0660"

EOF

i=$(($i+1))

done


Load updated block device partition tables.

Test the rules are working as expected.

Restart UDEV Service.


48. Clone VirtualBox VM

a. Shutdown the VM.

# shutdown –h now

b. Clone the disk racnod1.vdi: from VirtualBox Manager, click File -> Virtual Media

Manager.



Once the copy is complete, click Close to exit from the Virtual Media Manager.


49. In VirtualBox, click New icon in the upper left corner to create the new VM.



50. Open the Network sub-menu. Under the Adapter 1 tab, change the Attached to:

dropdown to Host-only Adapter.


Attached to: dropdown to Internal Network and type the name “rac-priv” in the Name

field.


Attached to: dropdown to NAT.


53. Open the USB sub-menu. Uncheck the Enable USB Controller check box.

54. Select the Shared Folders tab. Click Add Shared Folder. Enter the path where you have downloaded the Oracle installation media and enter a name for your folder.


55. Click on Controller SATA and “Choose existing disk” to add the asm disk vdi file.

56. Repeat the operation to add all asm disks and then click OK to save the modifications.


57. Review the summary of new virtual machine.

58. Start the “racnode2” and login as root.


59. On the top right had side end you would see “Ethernet” setting. Choose one and click

on “Wired Settings”.

60. Below is the screen-shot for enp0s3 (Public IP). Click on the right end bottom mark.


61. Change the Address to “192.168.78.52” for enp0s3.

62. In the similar way change the Address to “172.16.100.52” (Private-IP) for enp0s8.


63. For enp0s9 uncheck “Connect Automatically”

64. On terminal as root type “nmtui” (NetworkManager Text User Interface)

65. Change the Hostname


66. Delete the UUID entry for ifcfg-enp0s[389].

# sed –i –e ‘/UUID/d’ ifcfg-enp0s[389]


67. Reboot the server and then verify all the network interfaces are up and ASM devices are

visible.

68. Check the firewall status (should be inactive) and stop the DNS.


69. Delete the “racattack.zone” and “racattack.revzone” from /var/named.

70. Edit the “/etc/named.conf” and change .51 to .52 and the end of the file as below.


71. Review the “/etc/resolv.conf”.

72. Start DNS and check the forward and reverse lookup using “nslookup” command.


73. Check that both the master on racnode1 and slave on racnode2 DNS servers are

working. Run all the below statements to verify correct configuration of both DNS

servers :-

[root@racnode2 ~]# dig @racnode1 racnode1.racattack


[root@racnode2 ~]# dig @racnode1 racnode1-vip.racattack


[root@racnode2 ~]# dig @racnode1 racnode1-priv.racattack


[root@racnode2 ~]# dig @racnode1 rac-cluster-scan.racattack







[root@racnode2 ~]# dig @racnode2 rac-cluster-scan.racattack


74. When both nodes have started, check they can both ping all the public and private IP

addresses using the following commands.


75. Install the Grid Infrastructure

a. Install the following package for the Oracle Grid media as root user on both the nodes.

b. Setup the passwordless ssh for as Oracle user.

./sshUserSetup.sh –user oracle –hosts “racnode1 racnode2” –noPromptPassphrase –advanced –

exverify –logfile /home/oracle/sshsetup.log

c. At this point you can run the “runcluvfy.sh” (optional)


d. Run the Grid Infrastructure runInstaller as Oracle

[oracle@racnode1 grid] ./runInstaller

e. Below are the screen-shots for the configuration.



f. In the SCAN Name box, enter “rac-cluster-scan.racattack”. In the central box, only the first

node is displayed. Click on Add and enter the network names of the second node.

Public Hostname: - “racnode2.racttack”

Virtual Hostname: - “racnode2-vip.racattack”

Since the SCAN Name chosen is more than 15 character, Oracle proposes different name as Cluster

Name. Click Yes.


g. Check the correct addresses are set as below.

h. Make sure the “Software Location” is correct and change “Cluster Registry Storage Type”

to Oracle Automatic Storage Management. Enter SYSASM password and confirm. Click Yes.


i. Set the redundancy to “external”, click the “Change Discovery Path” button and set the

path to “/dev/asm-disk*”. I have changed the disk group name to DATA01.Once completed

click on “Next”.


j. If you want the root script to run automatically, enter the relevant credentials. I prefer to

run them manually.


k. Wait while the prerequisite checks complete.

l. The below checks can be ignored for this type of installation.


m. Save the response file and review the summary. Click the “Install” button.

n. Installation progress


o. When prompted, run the configuration scripts in each node. Once both the scripts are run

on all the nodes, click “OK”.


p. The installation continues and may take longtime to complete. “MGMTDB” creation may

take some time.


q. Optional step: - You can increase the timeout of CRS before it causes fencing of the node.


r. You can check the crs patch details.

76. As oracle user add new diskgroup for FRA

The asmca opens on the Disk Groups tab, showing the DATA01 Diskgroup created by the

installer. Click on create and enter “FRA” in Disk Group Name. Choose “External (node)”

redundancy. Check one disk to create 10GB diskgroup. Click “OK”


77. After the diskgroup is created

78. Install Database Software

a. You can use “cluvfy” to perform prerequisite checks (Optional)

cluvfy stage –pre dbinst –n “racnode1,racnode2” –d /u01/app/oracle/product/12.1.0.2/db_1 -verbose


b. Make sure all nodes are started and then login to racnode1 as oracle user and

start the Oracle Installer.


c. Make sure both nodes are selected, then click “Next” button



d. Make sure the “Oracle Base” and “Software Location” are as needed


e. For my installation I ignored both the warnings.


f. Review the summary information and click the “Install” button.

g. Run the configuration script on each node. When the script has been run on each

node, click “OK” button.


79. Create Database

a. Run the Database Configuration Assistance (DBCA)


b. Global Database Name:- cdbrac, Pluggable Database Name:- PDB


c. Review the “Create Database” Summary and Click “Finish”



Documents

12c_RAC_on_OEL7 - · PDF file5/12/2016 · The Oracle Database 12c R1 for Linux x86-64: ... installing required rpms, creating oracle user, required groups, setting parameters in