Oracle RAC 10g on HP-UX - decus.de · PDF file2/4 (1,6GHz) 200,829 Oralce 10g EE 2.75 US $ Montecito Integrity ... File System for RAC T2777BA HP Serviceguard Storage Management Suite

HP IT-Symposium 2006

1www.decus.de

5/17/2006 Copyright © 2003 HP corporate presentation. All rights reserved. 1

Oracle RAC 10g on HP-UX –now featuring HP SG CFS

Rebecca SchlechtEMEA HP/Oracle CTChttp://www.hporaclectc.com


Agenda• HP/Oracle Alliance• RAC10g with HP CFS for RAC• Supported Configurations with RAC10g on HP-UX• What option to choose?• Further Links• Q & A


2www.decus.de


HP/Oracle Alliance

= 25 years of collaborative partnership


Oracle and HP – a Strong Alliance• 25 years of collaborative

partnership • 100,000+ combined

customers… $6B business• Market leading 41% of Oracle

database customers use HP systems

• 70% mid-range HP-UX servers and 90% of Superdomes run Oracle

• Hundreds of dedicated resources to Oracle …. 13 worldwide technology & competency centers


3www.decus.de


See it @ http://www.hp.com/go/hplive

Oracle committed to HP IntegrityPrompt Oracle releases on IntegrityE-Business Suite ‘split-tier’ on Integrity now, pure Integrity by early 2007


Oracle’s Development Platforms …• Linux IA32 + Windows IA32 are Base

Development Platforms for Oracle• HP-UX Integrity, SUN Solaris, IBM AIX, ... are

Strategic Development Platforms for Oracle.• New Oracle versions are first available on Base

Development platforms and within 30 days on all Strategic Development Platforms.

• Up to now, HP-UX PA-RISC was Strategic Development Platform, with the next Oracle Releases this is HP-UX Integrity.


4www.decus.de


Current TPMC Status 4 Core and 8 Core

HP UX 11.iv2

MS 2003 EE

MS 2003 EE

AIX 5L 5.3

OS

2.75 US $ Oralce 10g EE200,8292/4 (1,6GHz) Montecito

Integrity rx4640

HP

2.30 US $ SQL 2005206,1814/8 (2.4GHz)DL585HP

2.02 US $IBM DB2 8.2236.0544/8 (2.4GHz)DL585HP

3.93 US $Oralce 10g EE203.4402 / 4 (1.9GHz)P5 570IBM

Price/PerfDatabaseTpmcProc/CoreSystemVendor

Integrity Benchmark = zx1 Chipset; DDR1 Memory; only 2 CPU sockets filled.

Even more PERFORMANCE with 8 Cores in rx4640Even more PERFORMANCE with zx2Price / Performance already now below P5 !!

Source: http://www.tpc.org/tpcc/results/tpcc_last_ten_results.asp

NEW


Recent Oracle Licensing Change(1) Virtualization

• All Virtualization Software tools of HP will be treated as a hard partition− nPpar− vPar−Secure Resource Partitions−Virtual Machine

Refer to http://www.oracle.com/corporate/pricing/partitioning.pdf


5www.decus.de


• Oracle’s recent dual core pricing announcements will level the playing− Multicore RISC processors 0.75

e.g. IBM Power5, Sun UltraSPARC IV, and HP PA-8800− Multicore Intel &AMD processors 0.5

e.g. Montecito ☺− Multicore Sun T1 processors 0.25− Single core CPU 1

• Example: # of Oracle licenses required for 32 core server− IBM = 32 x 0.75 x $40k = $960k− HP Montecito = 32 x 0.5 x $40K = $640k

• Standard Edition based on sockets (not cores) 8 Dual Core

Recent Oracle Licensing Change(2) Dual Core

Refer to http://www.oracle.com/corporate/pricing/databaselicensing.pdf


Oracle RAC10g on HP


6www.decus.de


shared storage

clusterinterconnect

Oracle Cache Fusion

S3 S4 Sn

more nodes = higher

availabilityOperating System

Oracle RACinstance 1

concurrent access from every node =

“scale out”

Oracle RAC10g ArchitectureServer 1 Server 2

redo log instance 1 …

redo log instance 3 …

control files

database files

ocr

voting

Operating System

Oracle Clusterware

Oracle RACinstance 2

opt: Vendor Clusterware

opt: CVM / CFS

Optional:quorum

Oracle Clusterware

opt: Vendor Clusterware

opt: CVM / CFS


Oracle Cluster Synchronization Services (CSS)Node Monitoring & Membership

Hardware / OS Kernel

Oracle Cluster Ready Services (CRS)HA Framework

virtual IP addresses database instances processes

140.84.20.02140.84.20.01

140.84.20.03SQL Cache

Large Pool

Buffer Cache

Java Pool

SGA Pool

SQL Cache

Large Pool

Buffer Cache

Java Pool

SGA Pool

Oracle Clusterware Architecture


7www.decus.de


Hardware- / OS-KernelCFS; SLVM; SLVM/ASM; CVM

SG / SGeRAC- Resource Monitoring- Resource Management- Cluster Control/ Failover APIs- Cluster Membership

Oracle Clusterware-CRS-EVM- CSS

Oracle Clusterware-CRS- EVM- CSS

Oracle ASM

Oracle RAC10g Architecture


Oracle RAC10g runs on …

OpenVMS *

Windows 64

OpenVMSHP-UX 11iv1Windows 32/64Linux 64

Tru64HP-UX 11i v2Linux 32/64HP-UX 11i v2OS

AlphaPA-RISCIntel Xeon & AMD Opteron

Itanium2Processor

HP Alpha Server

HP 9000 Server

HP ProLiant Server

HP Integrity Server

Server Family

* planned for H2CY06


8www.decus.de


Oracle RAC10g on HP

RAC10g with HP CFS for RAC


Commitment: HP and symantec partner to provide integrated clustering solutions

HP and VERITAS product integration BETTER integrated management for ease of use and strengthening

the foundation for leading high availability and disaster tolerant solutions

• HP introduces HP Serviceguard integrated with symantecVERITAS cluster file system on HP-UX 11i v2 scheduled for end of September

• VERITAS reaffirms HP-UX 11i as a strategic platform, including HP Integrity Servers and HP 9000


9www.decus.de


Oracle RAC10g with HP CFS for RAC• One of HP’s new Serviceguard

Storage Management Suite Bundles

• Includes the newest version of VRTS software 4.1 and SG/SGeRAC 11.17

• Provides consistent support for both Oracle 9i and 10g environments, for both single-instance Oracle and RAC (9iRAC and RAC10g R2)

• Applications and the database are managed the same way

• Provides initial support for 4-node clusters with SG CFS for RAC, 8 nodes will follow

• Ability to see the various db files (and associated sizes) with normal UNIX commands

• CFS can be used for Oracle data files and for Oracle binaries, configfiles, and logs

• Ability to use all normal backup utilities with db files on a CFS

• Currently only local cluster supported. Please contact us if you have customers who want Extended Cluster with CFS!

RAC 10g

SG/SGeRAC & Oracle Clusterware

CVM

CFS


Serviceguard Cluster File System

Serviceguard Cluster File System for Oracle

T2771BA T2773BA

T2775BA T2776BA

Serviceguard Storage

Management


Management Premium

T2772BA T2774BA


Management for Oracle


Management for Oracle Premium

Serviceguard Cluster File System for RAC

T2777BA

HP Serviceguard Storage Management Suite


10www.decus.de


Management of RAC databases –Advantages using CFS• Best management option

for all kind of files− addition/deletion of files− possibility to grow Oracle

data files− possibility for file system backup

• Support for installation of binaries into CFS− Oracle is full cluster file system aware− configuration file names are node/instance specific:

• CRS log files: [nodename].log• CSS log files: ocssd[Nodenumber].log


Backup & Recovery with RAC –Advantages using CFS

• Having archive logs on a CFS simplifies the backup significantly.− No need of cross-

instance archiving− No NFS cross

mounts− No manual

intervention in case a node/instance goes down

SGAInstance A

SGA

Instance Binterconnect

Node 1 Node 2

ARCH

Redo Thread1

RedoRedoshared Oracle Data filesRedoRedo

Redo Thread2

LGWR ARCHLGWR

Arch Files, Configuration & Logfiles


11www.decus.de


HP Serviceguard Cluster File System for RAC

Product Number: T2777BA

Specific to this bundle• SGeRAC• Cluster File System• Clustered Volume Manager• Quality of Storage • Instant Volume Snapshots • Checkpoint Restore • Oracle Disk Manager – for Oracle 9i & 10g

(known as Quick I/O for pre-9i)• Storage Mapping to LUN Level • Storage rollback• Storage Checkpoints• Disk Group Split and Join • Fast Mirror Resync

Common to all bundles• Serviceguard A.11.17• VxVM• Enterprise Cluster Master Toolkit• 2TB File System• 2-32 File System• >32 File System• VERITAS Enterprise Administrator (GUI)New with 4.1:• Dynamic Multi-pathing (enhanced with 4.1)• Intelligent Storage Provisioning • Configuration Backup • Storage Expert • History log (new with 4.1)• Online Intent Log Resize • Named Data Streams • Online LUN Resize• Portable Data Containers • Hardware Assisted E-Copy • Multi-Volume File System From 3.5:• Online Administration


What is ODM?• API defined by Oracle to enhance File

Management and Disk I/O throughput – this API is available to vendors.

• ODM API co-developed by Oracle and VERITAS• Designed for Oracle9i or later• Replaces Quick I/O which is up to Oracle8i• The ODM interface provides unified I/O and file

management for Oracle servers and reduces the number of system calls that Oracle servers generate. All Oracle processes use a single I/O library.

• Available in all Serviceguard Storage Management Suite for Oracle Bundles


12www.decus.de


RAC using CFSRAC using CFS

CVM/CFS 4.x SMNP(SG-CFS-pkg)

CVM/CFS 4.x SMNP(SG-CFS-pkg)

Disk group MNP(SG-CFS-DG-#id)

Disk group MNP(SG-CFS-DG-#id)

CFS Mount MNP(SG-CFS-MP-#id)

CFS Mount MNP(SG-CFS-MP-#id)

9i RAC instance or

10g RAC CRS startup package*

Per CFS mount point (may need multiple CFS mount per

RAC instance)

Per CVM Disk Group (may need multiple CFS DG per

RAC instance)

CVM/CFS subsystem

RAC using CVMRAC using CVM

* Requires Oracle 10.1.0.4 or later patch

Simple dependency

STORAGE_GROUP

Architecture and Design - SGeRAC 11.17


Oracle RAC10g on HP

Other Supported Configurations besides SG CFS for RAC


13www.decus.de


Oracle RAC10g on HP-UX

RAC 10g

SG/SGeRAC & Oracle Clusterware Oracle Clusterware Only

SLVM CVM

RAW ASM RAW CFS

ASM


Why HP SG / SGeRAC together with Oracle Clusterware• HP Serviceguard as proven

cluster layer with 150,000+ licenses

• HP Serviceguard is integrated with the HP Virtual Server Environment

• Continue to use HP Serviceguard to make your complete cluster high available including 3rd party applications

• HPs Enterprise Cluster Master Toolkits (ECMT) to rapidly implement and deploy applications under the control of HP Serviceguard

• Increase network reliability: Oracle Clusterware does not provide any protection for the critical private interconnect.

• Integration with HP Extended Cluster (aka Campus Cluster) and HP ContinentalClusters.

• Make use of a Cluster File System

• Make use of HP's shared logical volumes

• Co-existence of Oracle9i RAC and RAC10g on the same cluster


14www.decus.de


HP Serviceguard Extension for RAC (SGeRAC)• Support for 9i RAC & RAC10g• Cluster management is provided with SG• Shared Logical Volume Manager features are enabled

with SGeRAC• Support for 16 nodes with SLVM• SGeRAC runs on top of SG starting with 11.15• Support for SG Extension for Fast Failover• Support for Rolling Upgrade starting with 11.15• SGeRAC does not support mixed clusters. All nodes must

either be Itanium2 or PA-RISC, but not both in the same cluster.

• Online Node Reconfiguration Support (SNOR)− Being able to reconfigure SLVM while VG is activated in only one

administrative node in the cluster• Online Node Addition Support

− This feature requires changes in SG and Kernel LVM


Integration SGeRAC & Oracle Clusterware• In a combined SGeRAC & Oracle Clusterware

environment−SGeRAC is responsible for

• Cluster membership info provided to Oracle Clusterware CSS (Cluster Synchronization Service) daemon

• Management of SGeRAC provided storage options (SLVM, CVM, CFS)

• Startup of Oracle Clusterware within an HP Serviceguard package (required 10.1.0.4).

−Oracle Clusterware CRS (Cluster Ready Services) is responsible for• Management of database and associated resources (instances,

services, VIPs, listeners, ..)• Management of Oracle ASM


15www.decus.de


SGeRAC and Oracle Clusterware -best practices• Disable automatic start-up of Oracle Clusterware

− /sbin/init.d/init.crs disable• Ensure that Oracle does not activate/deactivate any

volumes− no /var/opt/oracle/oravg.conf

• Configure SG package to activate/deactivate shared VGsand to start/stop

• Let Oracle Clusterware start/stop all registered resources such as db instances, listener, VIPs, ...

• Configure same heartbeat interfaces for SGeRAC & Oracle

• Oracle VIP must be configured for a public network that is not used by any other SG application

• SG/SGeRAC will provide high availability for all networks


Oracle RAC10g on HP-UX with SG/SGeRAC & RAW• Oracle Clusterware running

integrated on top of SG/SGeRAC 11.16+

• Cluster Membership is derived from SG/SGeRAC: mature and proven cluster layer

• SG Package Manager to provide high availability of other applications

• Provides high availability for all networks

• Integration with HP Extended Cluster (aka Campus Cluster) and Continental Clusters.

• Support for shared logical volume manager (OCR, VOTING, data files, …)

• Optimal performance• Lacks file system capabilities• Slightly more complex to handle

RAC 10g


SLVM CVM

RAW RAW


16www.decus.de


Oracle RAC10g on HP-UX with SG/SGeRAC & SLVM/ASM

• Prior to SGeRAC 11.17, ASM is NOT supported in SG/SGeRAC configurations

• Only support for RAC 10g R2:− no support for RAC10g R1− no support for single instance− no support for Extended Cluster

(irrespective of whether the inter-site mirroring is via ASM or HP-provided volume manager)

• Each ASM disk group member must be a SLVM raw logical volume

• Max # of nodes in cluster: without MNP/Simple Dependency Framework - 16, with Framework – 4

• SLVM provides active/standby multi-pathing support for RAC and single instance (“pvlinks”)

• Leverage cluster-wide device name feature of SLVM

RAC 10g


SLVM

ASM


Oracle RAC10g on HP-UX with Oracle Clusterware only and ASM

RAC 10g

Oracle Clusterware Only

ASM

• The Oracle Clusterware to provide RAC cluster membership

• Oracle ASM for storage management• High availability for other applications

running on same cluster can be implemented with HP SG (no SGeRAC) or with RAC10gR2 Cluster API

• CRS monitors the virtual IP address (VIP) and not other network interfaces. Therefore, additional software necessary to provide HA features for private interconnect (e.g. HP APA)

• No shared logical volume manager supported for Oracle files (OCR, VOTING, data files, …)

• Supported with Oracle RAC 10gR1+


17www.decus.de


RAC10g on HP-UX: What to choose?

– additional software licenses for SG CFS bundle

+ best management option for all kind of files+ ability to use all normal file system utilities+ optimal performance with ODM driver+ includes many desirable file system + volume management features such as Flashsnap, Dynamic multipathing, …– supported with 9i and 10g

RAC10g withHP SG/SGeRAC …and CFS

– additional software licenses for SG/SGeRAC– no support for heterogeneous environments (HP-UX only)

+ SG/SGeRAC provides mature and proven cluster layer+ SG Package Manager to provide HA for other applications+ provides high availability for all networks+ integration with HP Extended Cluster

RAC10g with HP SG/SGeRAC ...

Con’sPro’s


RAC10g on HP-UX: What to choose?

– no file system capabilities– more complex to handle

+ optimal performance+ no additional software licenses for volume manager / file systems+ supported for 9i + 10g

RAC10g with HP SG/SGeRAC.... and RAW

– no HA for networks-– ASM lacks multi-pathingcapabilities– you must use ASM, no SLVM– only supported with 10g

+ easy support model (all from Oracle)

RAC10g with Oracle Clusterware Only and ASM

– no support before 10gR2– no support for single instance– no support for Extended Cluster– performance impact of extra layer of volume management

+ no additional licenses for volume manager / file systems+ easy management with ASM+ SLVM provides active/standby multi-pathing support for RAC and single instance (“pvlinks”)

RAC10g withHP SG/SGeRAC ... and ASM over SLVM

Con’sPro’s


18www.decus.de


Further Links …


to get a copy of this paper please send an email to [email protected]

/ [email protected]


19www.decus.de


Further Links• HP & Oracle Alliance Web Site

− http://www.hp.com/go/oracle

• Optimze for Agility− http://h71028.www7.hp.com/enterprise/ca

che/261009-0-0-0-121.html

• Oracle Metalink:− http://metalink.oracle.com

• Oracle Technology Network:− http://otn.oracle.com

• Product Availability:− http://www.oracle.com/technology/support

/metalink

• Oracle Grid Computing optimized in HP BladeSystems:− http://h71028.www7.hp.com/enterprise/do

wnloads/15391%20Oracle%20Blade%20BR%204b.pdf

• HP High availability technology− http://www.hp.com/go/ha

• Serviceguard HA and DTS solutions for HP-UX 11i and Linux− http://h71028.www7.hp.com/enterpri

se/cache/6469-0-0-0-121.html

• HP Parallel Database Clusters− http://h18004.www1.hp.com/solution

s/enterprise/highavailability/oracle

• Automatic Storage Management with HP Storage Arrays on HP-UX− http://h71028.www7.hp.com/enterpri

se/downloads/Oracle_10g_ASM_w_HP_StorageWorks_Arrays_on_HP-UX.pdf

• HP Offers Oracle Patch Parity:− http://h71028.www7.hp.com/enterpri

se/downloads/HP%20Oracle%20Patch%20Parity.pdf


Q & A


20www.decus.de


Oracle RAC10g on HP

Interconnect


Interconnect for RAC on HP-UX• Crossover Cables are not supported by Oracle as an interconnect

with 9iRAC/10gRAC on any platform.• Important criteria for interconnect:

− Throughput of interconnect: 1GB Ethernet / 10GB Ethernet / InfiniBand(today 10GB/s) / Hyperfabric2 (4 GB/s)

− Latency of protocol• UDP: available for all network adapters; ~ 50 microseconds• Low latency implementations:

− RDS planned− RDMA out of scope− HMP over Hyperfabric: not any longer strategic ...

− Distance• InfiniBand: max 10m copper cable• Hyperfabric2: max 200m cable

− High Availability• For most customer implementations UDP over Ethernet is sufficient.• Goal : Replace HMP/HyperFabric with Infiniband or 10Gigabit

Ethernet.


21www.decus.de


No

Yes, (SLVM only, no CFS yet)

No

No

No

IPoIB

Yes

Yes

Yes

Yes

Yes

1Gb 10Gb

NoNoNo10gR2 CRS only

Currently Planned

NoNo10gR2 SG


NoNoNo10gR1 SG

NoYesYes9.2

IB/RDSHF2/UDPHF2/HMPIntegrity

Oracle RAC Interconnect for HP-UX


No

No, under discussion

No

No

No

IPoIB

Yes

Yes

Yes

Yes

Yes

1Gb 10Gb


Currently Planned

NoNo10gR2 SG


NoYesYes10gR1 SG

NoYesYes9.2

IB/RDSHF2/UDPHF2/HMPPA-RISC

Oracle RAC Interconnect for HP-UX


22www.decus.de


Oracle RAC10g on HP

Disaster Tolerant Configurations


Extended Distance SGeRAC Cluster• Single Cluster over two data

centers - Active/Active• Disaster tolerance as servers and

storage reside in two separate data centers

• Automatic failover to second data center

• Software mirroring− 2 nodes supported with SLVM and

MirrorDisk/UX− 8 nodes supported with Veritas

VxVM/CVM up to 10km; 2 nodes up to 100km

• Support for single-instance & RAC• Support for Oracle9i & 10g

Data Center 1 Data Center 2

Storage Storage

RAC

Node A

SLVM

Node B

SLVM


23www.decus.de


SAN

Extended Distance SGeRAC Cluster: German Logistic Customer• 6x HP rx4640 HP-UX

11iv2 • HP SG & SGeRAC • Raw Devices -- Veritas

CVM with mirroring across two HP EVA8000

• Servers and storage placed in two data centers with a distance < 150m

• Migration project from Oracle8i OPS

• Need to make additional applications high available

• Possibility to upgrade to SG CFS later

< 150m

in production since October 2005


HP Continentalclusters & Oracle RAC• Single Clusters in separate

data centers –Active/Passive

• Support up to 32 HP 9000 and Integrity servers per Continentalclusters

• Supported with Oracle single instance and Oracle RAC

• „Push-Button“ Failover• Support for RAC 10g

introduced in December 2005 with SG/SGeRAC 11.17

Active DB

Primary Cluster

Recovery Cluster

Instance1

Instance2

XP CA

Active DB

Primary Cluster

Recovery Cluster

Instance1

Instance2

Instance1

Instance2

XP CA

Before Failure:

After Failure:


24www.decus.de


HP Continentalclusters & Oracle RACGerman Customer Example

PU SH

RE AD Y

AL ARM

M ESS AG E

hp S t o r a g e W o r k s x p 1 2 0 0 0 d is k a r r a y

hp S to r a g e W o r k s x p 1 2 0 0 0 d i s k a r r a yhp S t o r a g e W o r k s x p 1 2 0 0 0 d i s k a r r a y

PU SH

RE AD Y

AL ARM

M ESS AG E

hp S t o r a g e W o r k s x p 1 2 0 0 0 d is k a r r a y

hp S to r a g e W o r k s x p 1 2 0 0 0 d i s k a r r a yhp S t o r a g e W o r k s x p 1 2 0 0 0 d i s k a r r a y

Cab in e t Nu mb er

HKP

4 8 V

GSP

Atten t io n

Re mote

RAC11

Apps Server

Cab in e t Nu mb er

HKP

4 8 V

GSP

Atten t io n

Re mote

RAC22

Apps Server

Cab in e t Nu mb er

HKP

4 8 V

GSP

Atten t io n

Re mote

RAC12

Apps Server

Cab in e t Nu mb er

HKP

4 8 V

GSP

Atten t io n

Re mote

RAC21

Apps Server

DB1

DB2’

DB1’

DB2

ContinentalClusters

local SGeRAC cluster

local SGeRAC cluster

RAC11 & RAC12 access DB1RAC21 & RAC22 access DB2

InterclusterMonitoring

bi-directional HP CA XP/EVAsynch. or async.


Failure Group 1 Failure Group 2

EVA1

ASM Disk Group with normal redundancy

Oracle Clusterware: Oracle Mirroring across Data Centers

SAN

OCR VoteEVA2

OCR Vote

Vote

3rd data center

Documents

Oracle RAC 10g on HP-UX - decus.de · PDF file2/4 (1,6GHz) 200,829 Oralce 10g EE 2.75 US $ Montecito Integrity ... File System for RAC T2777BA HP Serviceguard Storage Management Suite