Virtualization, Oracle RAC 11g and more.. Vijay Ragunathan Guidewire Software Inc...

Virtualization, Oracle RAC 11g and more..

Vijay RagunathanGuidewire Software Incvragunathan@guidewire.com

Guidewire Software Inc

Agenda

Introduction Set up VM and RAC Take a look at a few 11g RAC/ASM features Introduce Cache fusion and trace Cache

Fusion

Virtualization

Abstraction layer between applications and hardware

Methodology to divide the resources of a computer hardware into multiple operating environments

Partition at various levels like Storage, Network, Server, Application

Why Virtualize

Consolidate IT infrastructure Improve QoS Reduce cost Provide more reliable working environment for

development and testing Ease replication of environments Partition applications to get security, reliability

and flexibility

Where can we Virtualize

Development and Test Virtual Desktops PoC deployments Disaster Recovery and Business continuity Technology refresh

Evolution

Virtualization in place for three decades Mainframes RISC based systems And now commodity hardware

Commodity Virtualization Software

VmWare Xen Oracle VM (Xen) Microsoft Virtual PC

And probably many more…

Oracle Real Application Cluster

Cluster is two or more nodes sharing some common resources

They communicate with each other to check health and guarantee data integrity

For Oracle, it is two or more Oracle instances sharing a physical database

Each instance use their own SGA and background processes

Database components like Datafiles, control files and redo log files are shared

Real World Oracle RAC Implementations

RISC or CISC based processers on two or more servers

A SAN or NAS for storing database Interconnect mechanisms like Infiniband,

Gigabit Ethernet, Memory channel

Node1 Node2

Storage

Real Application Clusters

VM and Oracle RAC

Hardware for clustering too costly Difficult to always build dev and test

environments with clustering Typically Applications built on single instance

and tested and deployed on RAC VM is Beneficial for small, medium and large

companies and cost effective

Virtual Node1

VirtualNode2

Same Disks in

Real Application Clusters

Physical Machine

Setup Xen

Xen Architecture

Xen hypervisor has management and virtual hardware API

Dom0 hosts the management code and has control interfaces to manage other domains

All guest Virtual machines are referred as domU

Packages

xen-3.0.3-41.el5 kernel-xen-2.6.18-53.el5 xen-libs-3.0.3-41.el5 kernel-xen-devel-2.6.18-53.el5

Dom0 Host

Default Centos 5.1 install Virtual Machine Manager GUI Command line tool virt-install

Virtual Machine Manager

The GUI VMM displays all the domains and their resource usage

Create a VM

File -> New Machine

Name the VM

We name the first VM as dtrac41

Virtualization Method

ParaVirtualization

ParaVirtualization uses a modified kernel Not very portable Not all hardware is simulated Best performing for Network and IO Unique address map for each VM Easy to implement

Fully Virtualized

Complete Hardware simulation Any supported guest OS can be used Not supported by all hardware architecture VT support may need to be enabled in BIOS

for some hardware Use techniques to trap and emulate

instructions

Install using PXE PXE server enables storing images in central location and install large

number of machines Using centos 5.1 for guestVM Kickstart Template file instructs what to install

OS and Swap for GuestVM

Using simple files Default path /var/lib/xen/images

Network for the GuestVM

Memory and CPU

Do not over allocate as we need resources for the dom0 Host and other GuestVMs

Summary of the Options and Resources

Vm Installation operations

VMM will start allotting the disk storage Create virtual network Map memory to the Dom0 Virtual memory

map Will start installing the Guest OS

PXE install looks at dhcp server

Xen Utilities

xm utility - can be used to create domains

- useful for cloning virsh

- can be used to change CPU, memory- define config, dump config in xml form for cloning- start, shutdown and reboot domains

Xen Utilities

xentop- similar to top but customized to list all virtual resources and domains

xentop - 21:55:32 Xen 3.1.0-53.el53 domains: 2 running, 1 blocked, 0 paused, 0 crashed, 0 dying, 0 shutdownMem: 15723456k total, 15721284k used, 2172k free CPUs: 8 @ 2992MHz NAME STATE CPU(sec) CPU(%) MEM(k) MEM(%) MAXMEM(k) MAXMEM(%) VCPUS

NETS NETTX(k) NETRX(k) VBDS VBD_OO VBD_RD VBD_WR SSID Domain-0 -----r 9576 2.9 13498460 85.8 no limit n/a 8

4 398061 38880 0 0 0 0 0 dtrac41 -----r 2755 2.1 1048152 6.7 1048576 6.7 1

2 883749 857104 0 0 0 0 0 dtrac42 --b--- 1302 8.1 1048080 6.7 1048576 6.7 1

2 850349 813954 0 0 0 0 0

XenStore

Configuration database for all domain Multiple commands to manage this database xenstore-ls

- can be used to list details of all hardware components in each domain

Add Virtual Hardware Add more storage for Oracle clusterware

Add Virtual Storage

Specify Simple file or disk partition (if available) Min size of 256Mb for Clusterware

Add virtual network interface For Interconnect use

Add virtual network interface Choose virtual network as this interconnect does not need to talk to the

external world

Virtual hardware of one guestVM

Oracle Install Preparation

Update sysctl.conf with the new kernel parameters rmem_default and wmem_default should be 4MB

min Update limits.conf Install any additional RPMs needed as listed in

Oracle Install documentation Edit /etc/hosts and add localhost and 6 other

addresses to indicate public, private and virtual networks for both guestVMs

Create users and groups

groupadd –g 10000 oinstall groupadd –g 10001 dba groupadd –g 10002 asmdba useradd –g oinstall –Goinstall,dba,asmdba –d

/home/oracle –u 10000 oracle passwd oracle

Mapping raw devices

Create something like /etc/rc3.d/S90raw and/or /etc/rc5.d/S90raw based on boot run level with contents like:raw /dev/raw/raw1 /dev/xvdbraw /dev/raw/raw2 /dev/xvdcraw /dev/raw/raw3 /dev/xvddraw /dev/raw/raw4 /dev/xvdechown root:dba /dev/raw/raw2chmod 660 /dev/raw/raw2chown oracle:dba /dev/raw/raw1chmod 660 /dev/raw/raw1chown oracle /dev/raw/raw3chown oracle /dev/raw/raw4chmod 660 /dev/raw/raw3chmod 660 /dev/raw/raw4

Cloning GuestVM

In dom0, /etc/xen/dtrac41 has list of devices created for this guest VM

Copy /etc/xen/dtrac41 to /etc/xen/dtrac42 Change the mac address of the two virtual network

interfaces in dtrac42 and edit the primary disk Modify the name and uuid for the new VM Ensure the order of the network is right in the config

files Copy the virtual disk of dtrac41(dtrac41.img) to

/var/lib/xen/images/dtrac42.img

Config file for guestVM

name = "dtrac41"uuid = "3656ca7c24e378bd2c63d380206a5f8d"maxmem = 1024memory = 1024vcpus = 1bootloader = "/usr/bin/pygrub"kernel = "/var/lib/xen/boot_kernel.qFE_y1"ramdisk = "/var/lib/xen/boot_ramdisk.HlhabB"extra = "ro root=/dev/vgSys/lvRoot rhgb quiet console=tty0 console=ttyS1,57600n8"on_poweroff = "destroy"on_reboot = "restart"on_crash = "restart"vfb = [ "type=vnc,vncdisplay=0,keymap=en-us" ]disk = [ "tap:aio:/var/lib/xen/images/dtrac41.img,xvda,w",

"tap:aio:/var/lib/xen/images/dtrac4x-ocr11g-shared.img,xvdb,w", "tap:aio:/var/lib/xen/images/dtrac4x-vote-shared.img,xvdc,w", "tap:aio:/var/lib/xen/images/dtrac4x-asm1-shared.img,xvdd,w", "tap:aio:/var/lib/xen/images/dtrac4x-asm2-shared.img,xvde,w" ]

vif = [ "mac=00:16:3e:0c:81:f5,bridge=xenbr0,script=vif-bridge", "mac=00:16:3e:75:41:84,bridge=virbr0,script=vif-bridge" ]

Cloning GuestVM

Use xm to create the domain

xm create dtrac42 Start the new domain

xm start dtrac42 You should see both the GuestVMs in the

Virtual Machine Manager

Setup GuestVMs

Reboot both the virtual machines Ensure fdisk –l is showing all the disks Edit the network configuration files to setup

the IP addresses and hostname and start the network

Ping the IP addresses of the other node and confirm that the network works fine

User equivalence Setup

Use ssh-keygen and generate keys Concatenate the public keys for both

machines and store as ~/.ssh/authorized_keys

Check you can login to all combinations without prompts. For e.g.,ssh dtrac41 datessh dtrac42 datessh dtrac41p datessh dtrac42p date

Run cluster verify tool and do pre CRS inst check

Vmware

Install Vmware server Create two virtual machines using Vmware

Server Console For the shared disks, edit the config files

(suffixed vmx) and add:disk.locking = "false“

scsi0:1.shared = "true"

Oracle install media

Download the clusterware and database software

Unzip the filesunzip linux_11gR1_clusterware.zip

unzip linux_11gR1_database.zip

If using cdrom/dvd, mount the clusterware media first

Start the Installercd clusterware./runInstaller

Clusterware Software Home

Cluster Configuration Cluster name is dtrac4_cluster Edit details of dtrac41 Add details of dtrac42

Network Interface Usage

Select the right Interface for use as Public and Private network Here eth0 is chosen as Public and eth1 as private

OCR and Voting disk

Both OCR in the previous installer screen and voting disk here should have min of 256MB space

Specify mirror copies – here we use single voting disk location

Clusterware Install Summary

Ensure all cluster nodes are seen

Root Scripts

Run orainstRoot.sh and root.sh on all nodes (one at a time)

Root.sh on node1 Configures OCR and Voting Disk and starts CRS stack on node1

Root.sh on node2 Starts CRS stack on node2 and creates cluster resources

Config Assistants for Clusterware

Clusterware Install Completion

That completes the clusterware install OCR and Voting disk are configured Resources like ONS and VIP should be created by

now and can be seen using crs_stat Processes like evmd, crsd, ocssd and oprocd should

be running now Ready to Install RAC software

cd database

./runInstaller

Installation Type

Node Selection Choose all nodes in the cluster – both VMs

OS Authentication

RAC Install Summary Check for selection of all Cluster nodes

Create RAC database

Create ASM Disk Group Change disk discovery path and check permissions if disks are

not seen

Starts Cluster database Database Creation is complete

CRS status

crs_stat -tName Type Target State Host------------------------------------------------------------ora....SM1.asm application ONLINE ONLINE dtrac41ora....41.lsnr application ONLINE ONLINE dtrac41ora....c41.gsd application ONLINE ONLINE dtrac41ora....c41.ons application ONLINE ONLINE dtrac41ora....c41.vip application ONLINE ONLINE dtrac41ora....SM2.asm application ONLINE ONLINE dtrac42ora....42.lsnr application ONLINE ONLINE dtrac42ora....c42.gsd application ONLINE ONLINE dtrac42ora....c42.ons application ONLINE ONLINE dtrac42ora....c42.vip application ONLINE ONLINE dtrac42ora.xenrac.db application ONLINE ONLINE dtrac42ora....c1.inst application ONLINE ONLINE dtrac41ora....c2.inst application ONLINE ONLINE dtrac42

RAC and ASM

11g New Features

Automatic Diagnostic Repository should be placed in shared disk if using shared oracle home

Kill sessions from anywhere in the cluster ADDM for RAC – dbms_addm.analyze_db

procedure and dbms_addm.get_report function Inter-node parallelism can use services to determine

the instances that it can use Grid provisioning and EM changes Clone ASM and RAC homes – clone.pl

11g ASM New Features

asm_preferred_read_failure_groups parameter ASM rolling migration

– alter system start rolling migration to <pnumber>; Convert single instance ASM to cluster using

rconfig/EM ASM compatibility attributes Variable size extents Higher AU values for Coarse striping 1M – 64M

11g ASM New features

Ability to drop ASM instances using dbcadbca –silent –deleteasm –nodelist dtrac41,dtrac42

ASM fast mirror resync SYSASM and OSASM ASMcmd command additions – cp,

md_backup, lsdsk, remap Mount/drop diskgroup with Force option

Cache Fusion

Shared cache architecture Cache coherency – preserve integrity of data

across different caches for the same data Employs high-speed Interconnects Created in Oracle OPS 8i to reduce pings 8i handled read/write contention 9i RAC handles other contentions like write-

Cache Fusion

Locks – mode, role, past image

E.g lock : SG1 Modes: Null, Shared, Exclusive Role: Local, Global Past Image component: 0 or 1 Maximum 3-way protocol (master, holder and

requestor)

Events

Event 10432 – for diagnosing unexpected GCS events and lock requests

Event 10708 – shows block request and grants (level 7 or 103 is common)

Event 10046 – common sql trace and combines well with event 10708

Trace commands

SQL> alter system set events '10432 trace name context forever, level 15:10708 trace name context forever, level 103';

System altered.

SQL> conn scott/tigerConnected.SQL> alter session set events '10046 trace name

context forever, level 8';

Session altered.

First Read session

Select on node 1*** 2008-05-15 02:40:36.228kclscrs: req=0 block=5/36 *** 2008-05-15 02:40:36.228

kclscrs: req=0 typ=immediate mode=SWAIT #3: nam='db file sequential read' ela= 786 file#=5 block#=36 blocks=1 obj#=16215 tim=1210844436231630 *** 2008-05-15 02:40:36.231 KCL:

P02: cr prewarm *** 2008-

05-15 02:40:36.231kclscrs: req=0 block=5/28 *** 2008-05-15 02:40:36.231

kclscrs: req=0 typ=immediate mode=S ***

2008-05-15 02:40:36.231kclscrs: req=1 block=5/29 *** 2008-05-15 02:40:36.231

kclscrs: req=1 typ=immediate mode=S…*** 2008-05-15 02:40:36.232kclscrs: req=7 block=5/25

*** 2008-05-15 02:40:36.232kclscrs: req=7 typ=immediate mode=SWAIT #3: nam='db file scattered read' ela= 7342 file#=5 block#=25 blocks=8 obj#=16214 tim=1210844436239474

Second read from same node

PARSE #9:c=0,e=1478,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=1,tim=1210847692139409

EXEC #9:c=0,e=37,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=1210847692139492

WAIT #9: nam='SQL*Net message to client' ela= 1 driver id=1650815232 #bytes=1 p3=0 obj#=523 tim=1210847692139514

FETCH #9:c=0,e=29,p=0,cr=2,cu=0,mis=0,r=1,dep=0,og=1,tim=1210847692139568

No communication to the other node

Write Operation Tracing..

update emp set sal=sal+100 where empno = 7654END OF STMTPARSE

#3:c=20001,e=44395,p=1,cr=55,cu=0,mis=1,r=0,dep=0,og=1,tim=1210848457075954

WAIT #3: nam='enq: TM - contention' ela= 226 name|mode=1414332419 object #=16214 table/partition=0 obj#=-1 tim=1210848457076262

*** 2008-05-15 03:47:37.076kclscrs: req=0 block=5/36

*** 2008-05-15 03:47:37.076kclscrs: bid=1:3:2:0:2:c:0:0:2:0:0..

Tracing Contd..

Lms trace of remote node *** 2008-05-15 03:47:37.077kcl_snd_cur: block=5/36 time=54

Back to update session trace*** 2008-05-15 03:47:37.077kclwcrs: got 1 blocks from ksxprcvimdWAIT #3: nam='gc current block 2-way' ela= 413

p1=5 p2=36 p3=1 obj#=16215 tim=1210848457077694*** 2008-05-15 03:47:37.077kclwcrs: req=0 typ=cur wtyp=2hop tm=489

Tracing Contd..

*** 2008-05-15 03:47:37.140kcllkopb: req=7 block=5/25*** 2008-05-15 03:47:37.140kcllkopb: bid=1:3:2:0:2:14:0..WAIT #3: nam='gc current multi block request' ela= 27265

file#=5 block#=25 id#=33554446 obj#=16214 tim=1210848457226586

• Nothing in the LMS trace• Back to sessions trace*** 2008-05-15 03:47:37.254kcllkopb: req=0 typ=cur wtyp=2hop tm=174615WAIT #3: nam='db file scattered read' ela= 83230 file#=5

block#=25 blocks=3 obj#=16214 tim=1210848457337661

Causing Write-Write Contention

update emp set sal=sal+100 where empno != 7654

PARSE #5:c=4000,e=49748,p=1,cr=9,cu=0,mis=1,r=0,dep=0,og=1,tim=1210848625816852

*** 2008-05-15 03:50:25.817

kclscrs: req=0 block=5/28

*** 2008-05-15 03:50:25.817

kclscrs: bid=1:3:1:0:2:5:0:0:4:0:..

Write-Write Contention Tracing..

Remote LMS*** 2008-05-15 03:50:25.816

kclgcr: req=231 block=5/28

*** 2008-05-15 03:50:25.816

kclgcr: bid=1:3:1:0:2:5:0:0..

*** 2008-05-15 03:50:25.825

kclqsnd: req=231 status=2 fscn=0:b49c2 time=8523

*** 2008-05-15 03:50:25.826

kclpdc: le=1c/50000 rls=49

Write-Write Contention Tracing..

Back to local session*** 2008-05-15 03:50:25.825kclwcrs: got 1 blocks from ksxprcvimdWAIT #5: nam='gc cr block busy' ela= 8846 p1=5 p2=28

p3=1 obj#=16214 tim=1210848625825992*** 2008-05-15 03:50:25.825kclwcrs: got 1 blocks from ksxprcvimdWAIT #5: nam='gc cr block busy' ela= 8846 p1=5 p2=28

p3=1 obj#=16214 tim=1210848625825992

*** 2008-05-15 03:50:25.826kclwcrs: req=0 typ=cr wtyp=busy tm=9178

Write - Write contention tracing

*** 2008-05-15 03:50:25.826kcllkopb: req=0 block=5/28*** 2008-05-15 03:50:25.826kcllkopb: bid=1:3:1:0:2:6:0:0:

LMS trace*** 2008-05-15 03:50:25.826kcl_snd_cur: block=5/28 time=64

Local Session Trace*** 2008-05-15 03:50:25.827kcllkopb: got 1 blocks from ksxprcvimdWAIT #5: nam='gc current block 2-way' ela= 270 p1=5 p2=28 p3=33554433 obj#=16214

tim=1210848625827163*** 2008-05-15 03:50:25.827kcllkopb: req=0 typ=cur wtyp=2hop tm=417

Virtualization, Oracle RAC 11g and more.. Vijay Ragunathan Guidewire Software Inc...

Documents

Choice And Use Of Appropriate Guidewire in PCI

Guidewire updates: Concept and Technology

Leading services and solutions to Guidewire initiatives€¦ · We value the importance of Guidewire training and certification. In 2011, Capgemini financial services professionals

Enabling Cannulation and Smooth Stricture Passage · About the NaviPro Guidewire The NaviPro Guidewire is a 100 percent Hydrophilic guidewire designed for cannulation and smooth passage

Marlabs Capabilities Overview: Guidewire Services

Achieving Claims Excellence with Guidewire ClaimCentersuccess.guidewire.com/rs/guidewire/images/WP-Guidewire-ClaimCen… · Customer Retention – Carriers achieving high levels of

Accenture Global Services v. Guidewire Software, Inc

SSS Guidewire - Technowood · 2020-01-22 · Along with traditional guidewire types including straight, angle and J shape, the SSS Guidewire comes in additional shapes like the BK

Brochure Guidewire Policy Painless Business Case

Double Guidewire Technique (DGT) For Difficult ERCP Cannulation

Presented by: RAGUnathan Kanagaretnam CEO Quality Associates, Malaysia · Presented by: RAGUnathan Kanagaretnam CEO Quality Associates, Malaysia RANJINI Ragunathan Perioperative Fellow,

Guidewire BillingCenter 8 Functional Certification Upgrade Exam

Hierarchical Learning of Curves Application to Guidewire ...vlm1.uta.edu/~athitsos/publications/barbu_cvpr2007.pdfHierarchical Learning of Curves Application to Guidewire Localization

Guidewire Final Presentation Apr 12

Dallas Best Places to Work Roadshow | Guidewire

ClaimCenter - Guidewire Softwaresuccess.guidewire.com/.../images/Brochure-Guidewire-ClaimCenter.pdf · Guidewire ClaimCenter® is the property and casualty insurance indus-try’s

guidewire training | guidewire training online | guidewire training videos | agile development

6 Reasons to Extend Guidewire

Arun Ragunathan Product Manager Product Marketing Manager · 2011-03-15 · © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1 Arun Ragunathan –Product

Guidewire PaaS