Oracle 11gR2 New Features for RAC - aturquia.comnzoug.org/system/files/private/2011_otn-apac_11gr2newfeaturesinrac.pdf · Oracle 11gR2 New Features for RAC OTN APAC Tour - 2011 (Thailand,

Oracle 11gR2 New Features for RAC

OTN APAC Tour - 2011(Thailand, China, Philippines, New Zealand)

About me…Independent Oracle Consultant - Summersky Enterprisese-mail: [email protected]

blog: www.muralivallath.comtwitter/ym/skype: muralivallath

email: [email protected]

Oracle Clusterware Components

Grid Infrastructure

HAS Stack

CRS Stack

CRS CSS EVM CTSS ONS ASM oraagent orarootagent

GNS mDNS GIPCD GPnPD crflogd

crfmond

New in 11.2.0.2

ClusterWare Files

• Oracle Cluster Registry (OCR)

• Oracle Local Registry (OLR)

• Voting Disk

• GPnP Profile

New in 11.2.0.2

ORADB2ORADB1

Cluster Interconnect

SSKY1 SSKY2

ORADB4

SSKY4

ORADB3

SSKY3

Public Network

HAS

CRS

CSS

EVM

VIP VIP VIP VIP

NM

GM

OCR

Network Switch

Interconnect Switch

SAN switch

Oracle Clusterware

© Summersky Enterprises LLC | | Murali Vallath | Slide: 5

CRS

CSS

EVM

HAS

CTSS

GPnP

DISKMON

CRS

CSS

EVM

HAS

CTSS

GPnP

DISKMON

CRS

CSS

EVM

HAS

CTSS

GPnP

DISKMON

OLR OLR OLR OLR

AA_00 AC_02AB_01 AD_03 AE_04 AF_05 AG_06 AH_07

prddb

Vote Disk

OCR

CRFMOND/LOGD

GIPCD

New in 11.2.0.2

GPnPD

• Grid Plug and Play Daemon• Started before CSS• Allows easy addition and deletion of nodes• Dynamic management of VIPs• Maintains essential cluster information in an XML profile:

– Cluster name and unique cluster identifier– Public and private network interface mappings– Location for the OCR and voting disks– Location for the ASM parameter file

• Manages and distributes the profile using the GPnP daemon

• Defines cluster name, networks and storage

New in 11.2.0.2


GPnP ProfileNew in 11.2.0.2


CTSSD

• Provides automatic time synchronization across cluster nodes

• If NTP is present, CTSS will run in observer mode• Starts in active mode.• Installed as part of Grid Infrastructure installation.• CTSS runs in two ways:

– Observer mode: If NTP is enabled, CTSS only observes.– Active mode: If NTP is not enabled, CTSS designates one node as

a master clock and synchronizes the other nodes.• Started by OHAS daemon


Cluster Stack Invocation

initO

HA

SD

orar

oota

gent

cssdagent

oraa

gent

ASM

EVMD

MDNSD

GPNPD

CTSSD

CRSD

CSSD Monitor

GNSD

CSSD

SCAN

Listener

DB Instance

eONS

ONS

ASM Instance

VIP

SCAN VIP

GNS VIP

Network

GNSD


Level Resource Name Description

1 SYSTEM Version Grid Infrastructure Version

WALLET Security Wallet Manager

GNS Grid Naming Services

CSS Cluster Synchronization Services

EVM Event Manager

CRS Cluster Ready Services

ACFS ASM Cluster File System

GPnP Grid Plug & Play

ORA_CRS_HOME GRID_HOME

CRSD Cluster Ready Services Deamon

DIAG Diagnostics

CRSADMIN Grid Infrastructure Administrator

CRSUSER Grid Infrastructure User

2 DATABASE NODEAPPS Node apps

VIP_RANGE VIP range address

LOG

DATABASES

© Summersky Enterprises LLC | 11gR2 RAC | Murali Vallath | Slide: 10

OCRNew in 11.2.0.2

ORADB2ORADB1


ORADB4ORADB3

Public Interface

OCR Process OCR Process OCR Process OCR Process

OCR Cache OCR Cache OCR Cache OCR Cache

OCR(repository)

OEM Agent OEM AgentsrvctlOUI

OCR

© Summersky Enterprises LLC | Murali Vallath | Slide: 11

Level Resource Name Description1 SYSTEM crs Cluster Ready Services

ORA_CRS_HOME GRID_HOME

WALLET Security Wallet Manager

GNS Grid Nam

version Grid Infrastructure Version

GPnP Grid Plug & Play

css Cluster Synchronization Services

network Network

OHASD Oracle High Availability Services Deamon

OLR Oracle Local Repository – manual backup info

OCR OCR – Auto Backup info

2 DATABASE NODEAPPS Node Apps

VIP_RANGE IP ranges used for VIP

LOG Logs

ASM ASM

DATABASES List of Databases© Summersky Enterprises LLC | 11gR2 RAC | Murali Vallath | Slide: 12

OLRNew in 11.2.0.2

OCRDUMP

Dump contents of Oracle Cluster/Local Registry to a file.

ocrdump [-local] [<filename> [-xml]

Clusterware Log Directories

GRID HOME log<nodename>

e.g.. prddb1

evmd

agent

gpnpd

gnsd

client

gipcd

diskmon

srvm

ohasd

crsd

cssd

admin

ctssd

mdnsd

racg

ohas

dcr

sd

orarootagent_root

oragent_oracle

oracssdmonitor_root

oracssdagent_root

orarootagent_root

oragent_oracle

oracssdmonitor_root

racgmain

racgevtf

racgeut

alert<nodename>.log

crs

diag

Clusterware Log DirectoriesG

RID

HO

ME

log diag

tnsl

snr

clie

nts

<nod

e>us

er_r

oot

listener_scan3

listener_scan2

listener_scan1

host

_<nn

nnn>

adrci_dir.mif

crs

node

sweep

stage

metadata

lck

incpkg

incident

cdump

alert

trace

stage

metadata

lck

incpkg

incident

cdump

alert

trace

sweep

Clusterware Log Directories

GRID HOME log<nodename>

e.g.. prddb1

evmd

agent

gpnpd

gnsd

client

gipcd

diskmon

srvm

ohasd

crsd

cssd

admin

ctssd

mdnsd

racg

ohas

dcr

sd

orarootagent_root

oragent_oracle

oracssdmonitor_root

oracssdagent_root

orarootagent_root

oragent_oracle

oracssdmonitor_root

racgmain

racgevtf

racgeut

alert<nodename>.log

crs

diag

New in 11.2.0.2

acfssec

acfsreplroot

acfsrepl

crflogd

crfmond


VIP’s

• Database VIP (also called RAC VIP)

• Application VIP

• HAIP (IC VIP)


New in 11.2.0.2


• Oracle uses the interconnect for cache synchronization

• Interconnect is also used for– Data blocks– Locks– SCN numbers– Messages


New in 11.2.0.2

Redundant Interconnects

• CLUSTER_INTERCONNECT

• NIC Bonding

• HAIP


New in 11.2.0.2

Verify Deamons$GRID_HOME/bin/crsctl stat res -t -init

-----------------------------------------------------------------------------NAME TARGET STATE SERVER STATE_DETAILS ---------------------------------------------------------------------------ora.asm 1 ONLINE ONLINE devsrv1 Startedora.cluster_interconnect.haip 1 ONLINE ONLINE devsrv1ora.crf 1 ONLINE ONLINE devsrv1ora.crsd 1 ONLINE ONLINE devsrv1ora.cssd 1 ONLINE ONLINE devsrv1ora.cssdmonitor 1 ONLINE ONLINE devsrv1ora.ctssd 1 ONLINE ONLINE devsrv1 OBSERVER


Part II

Oracle Real Application Cluster | Murali Vallath | Slide: 21

RAC Background Process• ACMS – Atomic Controlfile to Memory Service

• GTX0-j – Global Transaction Process

• RMSn – Oracle RAC Management Processes (RMSn)

• RMSN – Remote Slave Monitor

• LMS – Global Cache Service Processes

• LMON – Global Enqueue Service Monitor

• LMD – Global Enqueue Service Daemon

• LCK – Lock Process

• DBRM – Database Resource Manager

• PING – Response time agent

© Summersky Enterprises LLC | Oracle Real Application Cluster | Murali Vallath | Slide: 22

New in 11.2.0.2

ACMS – Atomic Controlfile to Memory Service

ACMS is behaves like an agent background process that contributes to ensuring a distributed SGA memory update is either globally committed on success or globally aborted if a failure occurs

GTX0-j – Global Transaction Process

GTX0-j process provides transparent support for XA global transactions in a RAC environment. The database autotunes the number of these processes based on the workload of XA global transaction.

RMSn – Oracle RAC Management Process

RMSn processes perform manageability tasks for RAC. Primarily around resource management when instances are added or removed from the cluster.

RMSN – Remote Slave Monitor

RMSN are slave background processes and perform tasks on behalf of a coordinating process running in another instance.

LMS – Global Cache Service

When spawned by Oracle, copy blocks directly from the holding instance’s buffer cache and send a read consistent copy of the block to the requesting foreground process on the requesting instance to be placed into the buffer cache. LMS rolls back any uncommitted transactions for any blocks that are being requested for consistent read by the remote instance.

RAC software provides for up to 36 LMS processes, depending on the amount of messaging traffic. Oracle uses new algorithm to allocate LMS processes

© Summersky Enterprises LLC | Oracle Real Application Cluster | Murali Vallath | Slide: 27

New in 11.2.0.2

SCAN

• SCAN –Single Client Access Number• Similar to a cluster alias available on Tru64 and VMS

clusters• Helps load balance across the database cluster• Clients do not require VIP information• Removes the requirement to change the client connection

if cluster changes • Must resolve to atleast one address on the public network• Best practice .. resolve to 3 addresses for the entire cluster

SCAN

• Allows clients to use EZConnect or simple JDBC connections

• Each cluster will have 3 SCAN listeners, each having a SCAN VIP defined as cluster resources on network 1

• A SCAN VIP/LISTENER will failover to another node in the cluster

• Instance registers with local listener on its node• Database “REMOTE_LISTENER” registers

instances with all SCAN listeners

Advanced Installation

SCAN/GNS Configuration

GNS configuration

SCAN configuration

Server Pools

Node 5 Node 6 Node 7 Node 8 Node 9 Node 10

oltp5 oltp6 oltp7 dwh8 dwh9 dwh10

Node 1 Node 2 Node 3 Node 4

ssky1 ssky2 ssky3 ssky4

sskypool2Min=3Max=3

sskypool3Min=3Max=3

sskypool1Min=2Max=4

Server Pools

• Logical division of the cluster into pools of servers • Applications (e.g. databases) can be configured to

run in one or more server pools• Managed by crsctl (applications), srvctl

(Oracle) • Defined by 3 attributes:

– MIN - minimum number of servers (default 0)– MAX – maximum number of servers (default 0 or -1) – IMPORTANCE – 0 (least important) to 1000

Server Pools

• One-to-one mapping between a database service and server pool

• A database service can only be defined to operate in one pool – UNIFORM – all servers in the pool– SINGLETON – one server in the pool

• Resources are contained in logical groups of server pools

Types of Server Pools

• System defined– Free– Generic

• User defined

Server Pools

Server Pools

Server Pools





sskypool2Min=3Max=3

sskypool3Min=3Max=3

sskypool1Min=2Max=4

Node 4

oltp5

Server Pools





sskypool2Min=3Max=3Importance = 20



Node 4

oltp5

Node 8

dwh8

crsctl

• Check if autostart of ohasd is configured – crsctl config crs

$GRID_HOME/bin/crsctl config crsCRS-4622: Oracle High Availability Services autostart is enabled.

crsctl

• Check the health of Oracle Clusterware – crsctl check crs

[root@prddb1 bin]# ./crsctl check crsCRS-4638: Oracle High Availability Services is onlineCRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is online

crsctl

• Add additional voting disk to CFS– crsctl add css votedisk

• E.g.. crsctl add css votedisk /u03/oradata/CssVoteDisk

[root@prddb4 root]# crsctl add css votedisk /u03/oradata/CssVoteDiskNow formatting voting disk: /u03/oradata/CssVoteDiskRead -1 bytes of 512 at offset 0 in voting device (CssVoteDisk)

successful addition of votedisk /u03/oradata/CssVoteDisk.

crsctl

• Query css voting disk– crsctl query css votedisk

[root@prddb1 bin]# ./crsctl query css votedisk## STATE File Universal Id File Name Disk group-- ----- ----------------- --------- ---------1. ONLINE b1240356cc244fb0bf79f51c9fbb1f60 (ORCL:ASM_SSKY1) [PRD_GRID]

Located 1 voting disk(s).

crsctl

• Debug to trace crs activity– crsctl debug statedump crs

e.g.. [root@prddb1 root]# crsctl debug statedump crsdumping State for crs objects

Dynamic state dump information is appended to the crsd log file

$GRID_HOME/log/<node name>/crsd/crs.log

crsctl

• Query crs software version– crsctl query crs softwareversion

• Query crs active version– crsctl query crs activeversion

• Query Grid Infrastructure administrator– crsctl query crs administrator

ACFS

• General Purpose file system integrated with– ASM– Oracle Clusterware– Oracle Enterprise Manager

• File system features across multiple vendor platforms

• Single Node and cluster wide shared Oracle homes

• Uniform, coherent shared file access

ASM Dynamic Volume Manager

• Provides volume management services• Disk device interface to ACFS• Volume is created using management tool• Should be enabled to generate an OS device• Devices reside at /dev/asm/<volume-name>• ADVM driver maps IO requests against a

volume to the corresponding volume file located within an ASM diskgroup

ACFSUTIL

• Command line utility for ACFS

• Similar to ASMCMD

• Set ORACLE_SID=<asm sid> before executing acfsutil

• Located in /sbin/acfsutil

ACFS Replication

• Asynchronous replication from primary to standby

• Replication across network to another site.

• Provides DR option to the filesystem

• Used in conjunction with Oracle dataguard

New in 11.2.0.2

ACFS Tagging

• Method for relating a group of files

• Use alone or with other features

New in 11.2.0.2

Create ACFS Volumes

• Create ASM diskgroup • Enable ASM diskgroup as Volume• Create filesystem with ACFS mkfs

command• Register file system with acfsutil registry• Mount the filesystem• Test the file system

Q U E S T I O N SA N S W E R S

Complete your evaluations



[email protected]

| Murali Vallath | Slide: 53

Thank You



Documents

Oracle 11gR2 New Features for RAC - aturquia.comnzoug.org/system/files/private/2011_otn-apac_11gr2newfeaturesinrac.pdf · Oracle 11gR2 New Features for RAC OTN APAC Tour - 2011 (Thailand,