Provisioning Oracle RAC in a Virtualized Environment ...Introduction to Oracle Enterprise Manager and Oracle VM 4 • New Challenges for IT organization –Quick deployment of Oracle

Provisioning Oracle RAC in a

Virtualized Environment, Using

Oracle Enterprise Manager

Kai Yu, Sr. System Engineer Consultant

Dell Global Solutions Engineering

Rajat Nigam, Senior Product Manager,

Oracle Corporation

• Kai Yu

• Senior System Engineer, Dell Oracle Solutions Lab– 15 years Oracle DBA and Solutions Engineering

– Specialized in Oracle RAC, Oracle EBS and OVM

– Oracle ACE Director, Oracle papers author and frequent presenter at OOW and Collaborate

– IOUG Oracle RAC SIG President

• Rajat Nigam

• Senior Product Manager, Oracle Corporation

Focus areas

– Enterprise Manager’s Datacenter Automation solution–Provisioning, Patching, Upgrade, Migration

– End user experience of Enterprise Manager install and upgrade

About Authors

2

• Introduction to Oracle Enterprise Manager and Oracle VM

• Oracle Virtualization Infrastructure Implementation

• Provisioning Oracle 11g RAC using Oracle Enterprise Manager

• Extending Oracle 11g RAC using Oracle Enterprise Manager

• QA

Agenda

3

Introduction to Oracle Enterprise Manager

and Oracle VM

4

• New Challenges for IT organization

– Quick deployment of Oracle service at low cost

– Deploy using tested and patched gold images

– An ability to response demand in short order

– Server consolidation and partition

• Oracle Enterprise Manager Grid Control

– Life cycle management of the entire IT stack

– Oracle Database/ RAC Provisioning and Patching

– Oracle VM Provisioning and Patching

5

Introduction to Oracle Enterprise Manager

and Oracle VM• Automated provisioning, patching, lifecycle management

• Gold images based & reference based deployment

6

RAC PROVISIONING

Oracle VM Technology

7

– Virtual Server: Xen Hypervisor, management domain

dom0 Linux kernel with support of devices, IO, networking,

etc.

– Virtual Server Pool: an autonomous region of VM servers

– VM: guest OS with applications running on domU

8

Oracle VM and Oracle RAC

9

VM and Virtualization Management

• VM Management Pack features

• Lifecycle Automation for Guest Virtual Machines

OVM Infrastructure Implementation

1

0

• Grid Infrastructure Project– Dell/Oracle Joint POC Project: Oracle Grid/ RAC Provisioning

a. Expand the Grid to include virtualization

b. Implement the Grid on Commodity hardware and SAN storage

c. Grid control as the unified management for the grid

• Oracle Grid Architecture– Grid : Physical Grid and Virtual Grid

– Grid based on Commodity Hardware Dell Blade Servers

– Refer to Session#705: Building Oracle Grid with Oracle VM on blades and iSCSI storage.

– Focus of this presentation:

a. Establish the VM infrastructure for running 11g RAC on VM

b. Provision 11g RAC on VMs using Enterprise Manager Grid Control


11

• Virtualization Infrastructure Design

– Virtual Servers: Dell blade servers

– Shared storage: Dell EqualLogic iSCSI SAN

– Unified Management: Grid Control with VM Management

1

2

• Oracle VM server Installation– Enable virtualization for CPUs on BIOS

– Install Oracle VM server 2.1.2

– Configure dom0 memory by editing /boot/grub/menu.lst

– Secure OVM agent is running: service ovs-agent status

• Oracle VM server network configuration



1

3

– Customize the default Xen bridge configuration

a. Disable the default Xen bridge settings.

b. Create ifcfg-ethn, incfg-bond0, ifcfg-xenbr0, ifcfg-xenbr1

c. Restart the network services

• Oracle VM server shared storage configuration

– Shared storage partitions added into OVM repositories

/dev/mapper/ovs_data4p1; /dev/mapper/ovs_data6p1

– Additional disk partitions for guest VMs through physical

disks mappings for RAC shared disks

2 X OCRs, 3 X Votingdisks, ASM diskgroup

– Detailed implementation referred to Session #705

1

4

• Grid Control Management Infrastructure

– Install EM Grid control 10.2.0.3,Upgraded to 10.2.0.5

– Enabled Virtual Management Pack

– Detailed implementation referred to Session #705

• Connect the VM servers to Grid Control

– Pre-configure steps on VM servers

– Create VM server pool with the VM server



1

5

• Create Guest VMs using the VM template– Download/Register OVM_EL5U2_X86_64_11GRAC_PVM

– Create VMs from the template


1

6

Specify the parameters Guest VM is created and running

for the guest VM on virtual server kblade10

Create an additional VM for node 2


1

7

– Present xenbr0 and xenbr1 as eth0 (public) and eth1(private)

vm.cfg: vif =

['bridge=xenbr0,mac=00:16:3E:11:8E:CE,type=netfront',

'bridge=xenbr1,mac=00:16:3E:50:63:25,type=netfront', ]

– Add the additional disk and shared storages to the VMsa. Attached the shared disks from VM repositories


1

8

vm.cfg: disk = [''file:/OVS/sharedDisk/racdb.img,xvdc,w!',

b. Attached the physical partitions to the guest VM

vm.cfg: disk = ['‘phy: /dev/mapper/vmracdbp1, xvdc,w!',

Provision Oracle 11g RAC using EM

1

9

• Install/configure Grid control Agent on RAC node VMsa. Grid control agent 10.2.5 installed on two guest VMs

b. Disable firewall services

c. Check host/cluster preferred credentials on the guest VMs


2

0

– Setup software library: DeploymentProvisioning

Administration: Software Library configuration

– Load the Oracle RAC shipped software to the software library

Download 11g clusterware/11g RAC software; DeploymentComponents


2

1

Upload Oracle Database binary and Oracle clusterware binary

Provision Oracle 11g RAC EM

2

2

• Prepare guest VM nodes for RAC provisioning– Install/configure Grid control Agent on RAC node VMs

– Prepare shared devices: 2 X OCR and 3 X votingdisk, ASM disk

– /udev/rules.d/65-raw.rules and 89-raw.rules: map raw devices

– Change kernel parameters for RAC install

– Extend OS partition by adding another 20 GB disks using LVM

– Prepare /etc/hosts to include public, private, VIPs: hostnames/IPs


2

3

• Provision Oracle RAC on guest VM

– Start the provisioning procedure

Select source from Software library

Click Schedule deployment and specify the source of software

Provision Oracle 11g RAC Using EM

2

4

Select

Target

Host:


2

5

Target: clusterware/ASM RAC Database details


2

6

Shared Storage configuration for RAC

Submit the provision job and check the job status


2

7

Check the Job completion status:

Extend Oracle 11g RAC using EM

MANAGER

2

8

• Two ways to extend RAC

– Single Click Extend Deployment Procedure

– Clone from the existing RAC node’s Template

• Single Click Extent Deployment Procedure

– Single node RAC database vmracdb


ORACLE ENTERPRISE MANAGER

2

9

– Prepare the guest VM for the new node

a. Create a new guest VM

b. Same network, disks, kernel settings, rpms. disable firewall

c. Install Grid control agent 10.2.0.5

d. Add host and cluster preferred credentials of Oracle users

– Schedule the provisioning procedure to extend the RAC

DeploymentProvisioning Deployment Procedure Manager

RAC Provisioning Procedure

Select “One Click Extend Cluster Database”, click “Schedule

Deployment” select the RAC database to extend


MANAGER

3

0

Extend Oracle 11g RAC using


3

1

Specify the new node to add and the user credentials


MANAGER

3

2

• Check the job status: procedure completed successfully

Extend Oracle 11g RAC using


3

3

• Verify the RAC extension results:– Check clusterware status:[root@bvmrac2 bin]# ./crsctl check crs

Cluster Synchronization Services appears healthy

Cluster Ready Services appears healthy

Event Manager appears healthy

– Check database service:[oracle@bvmrac2 bin]$ srvctl status database -d vmracdb

Instance vmracdb1 is running on node bvmrac1

Instance vmracdb2 is running on node bvmrac2

Summary

3

4

• Use Grid control to manage the virtual servers

• Use Grid Control Provisioning pack to provision 11g RAC on VMs

and extend the RAC to additional VM node.

• Acknowledge the support of Oracle engineers: Daniel Dibbets, Kurt

Hackel, Channabasappa Nekar, Premjith Rayaroth,

• Related Collaborate 10 Presentation:

– ID#: 705, Oracle Grid with Oracle VM on Blade servers and

iSCSI storage Monday, April 19, 2010, 2:30-3:30 PM, Kai Yu

• Dell Oracle Whitepapers: www.dell.com/oracle

– Efficient Deployment of Oracle® RAC in a Virtualized Environment

Using Enterprise Manager Provisioning Pack

– A Reference configuration for Oracle Grid leveraging Oracle VM

• Visit my Oracle Blog at http://kyuoracleblog.wordpress.com/

3

5

QA

3

6

Oracle RAC SIG Expert Panel

• Time: Wed, April 21, 1:00 pm - 2:00 pm

• Room: Lagoon D

• Panelists:

Marcus Michalewicz (Oracle RAC PACK)

Arup Nanda

Mike Ault

Mark W. Farnham

Syamal Bandyopadhyay

3

7

Thank You

• Please Complete the Survey

Title: Provisioning Oracle RAC in a

Virtualized Environment, Using Oracle

Enterprise, #349

Presenter: Kai Yu

• Visit my Oracle Blog:

http://kyuoracleblog.wordpress.com

Documents

Provisioning Oracle RAC in a Virtualized Environment ...Introduction to Oracle Enterprise Manager and Oracle VM 4 • New Challenges for IT organization –Quick deployment of Oracle