- DBCloudShifu€¦ · • Oracle Database 11g Release 2 ... Extending ASM to Manage ALL Data Automatic Storage Management

<Insert Picture Here>

•

Oracle Database 11g Release 2

ASM Top 5 Essential FeaturesNitin Vengurlekar - Grid Product Strategy

2

The following is intended to outline our general product

direction. It is intended for information purposes only,

and may not be incorporated into any contract. It is not a

commitment to deliver any material, code, or

functionality, and should not be relied upon in making

purchasing decisions.

The development, release, and timing of any features or

functionality described for Oracle’s products remains at

the sole discretion of Oracle.

Safe Harbor Statement

3


Audience Poll

How many of you have standardized on ASM?

How many of you are running ASM for non-RAC?

How many of you are running 11.1? 10.2? 9.2?

How many of you are System Administrators and

manage ASM? DBAs who manage ASM?

4


Agenda

• Brief ASM Overview

• Simplified Technology Stack

• Simplified Storage Provisioning

• Simplified user interface for manageability

• asmca, asmcmd, acfsutil and EM

• ASM Clustered File System

• Disk Group Rename

What is Oracle ASM?

Operating SystemOperating System

HardwareHardware

Oracle DatabaseOracle Database

ASMASM

Operating SystemOperating System

HardwareHardware

Logical Volume ManagerLogical Volume Manager

File SystemFile System

ApplicationApplication

6

ASM Benefits

• I/O is spread evenly across diskgroup disks • Prevents hot spots and maximizes performance.

• ASM eliminates need for over provisioning and maximizes

storage utilization • Great driver for database consolidation.

• Rebalance - online redistribution of extents • Initiate with incremental addition/removal of storage

• Can leverage ASM redundancy or 3rd party RAID

functionality.

• Supports single instance databases as well as RAC.

• Various tools to manage ASM• EM, asmcmd, asmca, Sql*Plus

• Reduces overall Oracle Database TCO without

compromising performance or availability.

ASM Instance Architecture

Screen reader hint: This slide contains a diagram that is explained in the speaker

notes.

11


Agenda








12

ASM Cluster File System (ACFS)Overview - Metalink Doc ID: 948187.1

• General purpose scalable file system

� Journaling, extent based

� Single node and cluster

� POSIX, X/OPEN file system solution for UNIX/Linux

� Accessible through NAS protocols (NFS, CIFS)

• Integrated with Oracle Clusterware for cluster support

• Multi OS platform (Supported on Red Hat 5 and Windows at initial

release)

• Integrated with Oracle system management tools

� Oracle installation and configuration

� Enterprise Manager and ASM Storage mgt tools

� Native OS File System Management tools

ACFS Architecture

Oracle

Database

3rd Party

Application

3rd Party

File System

ASM

Disk Group

Database &

Clusterware

Files

Dynamic Volume

ACFS

Files

3rd Party

File System

Files

ASM Instance

Dynamic Volume Manager

ASM Cluster

File System

14

Practical Usages of ACFS

•Oracle database installation binaries

•RAC or single instance ORACLE_HOME directories

•Database exports

•Database logs and trace files

•File based database objects

�UTL_FILE_DIR location

�Directories used by extproc routines

�Directory objects for external BFILES and external tables

•Application logs

•Shared documents

•Report output

•Data dumps

•Oracle database installation binaries

•RAC or single instance ORACLE_HOME directories

•Database exports

•Database logs and trace files

•File based database objects

�UTL_FILE_DIR location

�Directories used by extproc routines

�Directory objects for external BFILES and external tables

•Application logs

•Shared documents

•Report output

•Data dumps

15

ACFSCost Savings Use Cases

• Customer running In house application needs cluster

filesystem for shared report and batch loading

capabilities. Additionally, they want to replace existing

3rd party high availability solution

• Replace existing NAS appliance for their test/Q/A. ACFS

can be leveraged as a “Poor Man’s NAS”

• Export ACFS file system from a designated Grid Infrastructure

server and mount as NFS or CIFS mount point on the clients

16

ACFS for E-Business Suite Customers

RAC DB TierRAC DB Tier

Forms ServerForms Server

Concurrent Manager Server

Concurrent Manager Server

Web App

Server

Web App

Server

NFS Share - /etc/exports

Each node NFS Mount File System

NFS

Shared Apps Tier:

•ORA_TOP

•COMMON_TOP

•APPLTOP

Concurrent Manager

Output and

UTL_FILE_DIR

•APPLPTMP

•APPLTMP

•APPLCFS/OUT

•APPLCFS/LOG

Shared Apps Tier:

•ORA_TOP

•COMMON_TOP

•APPLTOP

Concurrent Manager

Output and

UTL_FILE_DIR

•APPLPTMP

•APPLTMP

•APPLCFS/OUT

•APPLCFS/LOG

18

Dynamically Resize ACFS Reduce Application Downtime

• Privileged Users can increase the size of an active file

system (acfsutil)

• Implicitly will grow the ASM Volume

• No need to unmount the file system to extend/shrink

the file system

#1: Extend DIAG_DEST FS on the fly

Ease of Management - increase space immediately rather than

hunt and peck and delete log, core and trace files

#2: Extend ETL FS on the fly

Running out of space in the ETL file system. Cannot acquire

downtime to unmount/mount file system to increase

19

Command Function

acfsutil infoDisplay new ACFS file and file system features (ACFS, snapshots, registry, size) and information

acfsutil size Resize an ACFS file system

Resize ACFS on the flyNo need to kick off the users to increase file system size

Command-line syntax

# /sbin/acfsutil size +4G -d

/dev/asm/lobdata_db1-212

/apps/oracle/product/11.2.0/db_1acfsutil size:

new file system size: 10737418240 (10240MB)

Command-line syntax

# /sbin/acfsutil size +4G -d

/dev/asm/lobdata_db1-212

/apps/oracle/product/11.2.0/db_1acfsutil size:

new file system size: 10737418240 (10240MB)

20

ACFS Read-Only Snapshots

•ACFS snapshot is an online, read-only, point in time copy of an

ACFS file system

•The snapshot copy is initially sparse and merely references the

storage allocation information maintained by the file system

•Automatically built when filesystem is created

•Snapshot storage is maintained within the file system

•Before an ACFS file extent is modified/deleted, its current value is

copied to the snapshot.

•Copy-On-Write (COW) technique to maintain the snapshot's

point-in-time view of the file system

•Snapshots can be created on demand to deliver a current,

consistent, online view of an active file system

21

ACFS Read-Only Snapshots

•An enabler for:• On-line, disk-based, file backup model using snapshots and

individual file recoveries

• Up to 64 snapshot images per ASM file system• Policy based snapshots:

• Schedule snapshots on an interval basis: every 5 seconds,

every 30 minutes, daily, … with recycling (using EM)

• ACFS CLIs support creation and removal of snapshots

• ACFS Snapshot functions integrated with EM

• ACFS Snapshots are always available in a mounted

file system

23

ACFS SnapshotsThe New Upgrade and Patching Strategy

File System

1.Flashback capability of

ACFS file system

2.ACFS Snapshots should

be part of every database

upgrade or patch

3.ACFS Snapshots should

be part of every application

server patch or upgrade

Database

• Database Guaranteed

Restore Points (GRP)

• Flashback Recovery Area

(FRA)

Database

• Database Guaranteed

Restore Points (GRP)

• Flashback Recovery Area

(FRA)

For SAs: The most convenient way to create or remove snapshots is

with the acfsutil OS command

For SAs: The most convenient way to create or remove snapshots is

with the acfsutil OS command

24


Agenda








25

Storage Provisioning Pitfalls Pre 11.2Reduces provisioning and planning for RAC

•Requires many distinct requests for space

•Error-prone to define permissions and config

for OCR/Vote

•Potentially requires odd LUN size requests

•Many Admins fdisk/partition the disks

for the OCR/Vote files

•Requires many distinct requests for space

•Error-prone to define permissions and config

for OCR/Vote

•Potentially requires odd LUN size requests

•Many Admins fdisk/partition the disks

for the OCR/Vote files

26

Storage Provisioning 11.2+Simplified Storage Provisioning

•New Way - Reduces provisioning and

planning for RAC

•Provision storage for ASM Disks only

No need to nickel and

dime SAN Architects

for trivial space

•Only manage ASM Disks

•New Way - Reduces provisioning and

planning for RAC

•Provision storage for ASM Disks only

No need to nickel and

dime SAN Architects

for trivial space

•Only manage ASM Disks

•ASM can now store: OCR, Vote, ASM

Spfile, database home

•All allocated from the requested disk pool

•ASM can now store: OCR, Vote, ASM

Spfile, database home

•All allocated from the requested disk pool

27

Everything Inside ASM Model

3rd Party FSApplication

Extending ASM to Manage ALL Data

Automatic Storage Management (ASM)

ASM Instance Managing

Oracle DB Files

ASM Cluster & Single Node File System

(ACFS)

Database RAC

ACFS Snapshot

ASM Files

Dynamic Volume Manager

Dynamic Volumes

ACFS FSOracle Bin

3rd PartyFS

DB DatafilesDB DatafilesDB DatafilesDB DatafilesDB DatafilesDB Datafiles

OCR & Voting filesOCR & Voting files

ASM

Disk

Group

29


Agenda








30

Managing ASMWhich management interface should I use?

• ASMCA (GUI)

• Lightweight tool to create ASM Disk Groups, Volumes and ACFS

• Command line option is available in silent mode

• Enterprise Manager Grid Control

• Easy ASM Volumes and ACFS creation and management

• HTTP Access via Database Control

• Designed to be the all inclusive tool

• ASMCMD

• Great for reports and listings

• SQL*Plus Command Line

31

Storage Manageability - asmcaEven a caveman can do it !

• Just like we have the database configuration and network configuration assistants, we now have an ASM configuration assistant

• Simplifies ASM Maintenance

• Extremely low learning curve• Lightweight• Simplistic• GUI

32

• Recommended tool to configure ASM,

Dynamic Volumes and ACFS

• Upgrade previous versions of ASM to the

current version (11g R2)

• Create & manage Dynamic Volumes

• Create ACFS file systems

• Ready for Oracle DB Home

binaries to be installed

ASM Configuration Assistant (ASMCA)

33

asmca – Create Disk Group

1. Recommended to create

disk groups with asmca

2. Recommended AU in 11g

is 4MB

• ADVM Compatibility must

be set to 11.2 to utilize

ACFS

34

asmca – Create ACFS for Database HomeCreate a shared file system for Oracle Binary

35

Linux-Unix ExtensionsEase of Management

Identical to other Unix file system management commands

Create an ACFS file system mkfs.acfs [-vf] [-n name ] device

# mkfs.acfs /dev/asm/vol1-123

Mount an ACFS file system mount.acfs [-v] [-o options] device dir

# mount.acfs /dev/asm/vol1-123

/u01/oracle/acfsmounts

Unmount an ACFS file system umount [-v] device|dir

Check and repair an ACFS file

system

fsck.acfs [-avnf] [info] device

# fsck.acfs /dev/asm/vol1-123

36

Extensions for asmcmd

Everything you can do in SQL*Plus, you can do in asmcmd

• Privileged Users are encouraged to manage the Cluster / ASM and ACFS

• Single point of entry for ASM maintenance• No more need to go back and forth between asmcmd

and SQL*Plus

• Now, asmcmd is an attractive tool that Privileged Users can swallow

37

Extensions for asmcmd As a Privileged User – life is easy

1. You can startup and shutdown ASM instance – (non-RAC)

2. Create (mkdg) / Drop (dropdg) / Mount (mount) / Dismount

(umount) ASM Disk Group

• Add / remove disk from ASM disk group (chdg)

• View Disk I/O Statistics (iostat)

• View Disk Group Level Information (lsdg)

• View Disk Level Information (lsdsk)

• Create (volcreate) / Resize (volresize) / Enable (volenable)/

Disable (voldisable) Volumes

• Rebalance Disk Group(rebal)

1. You can startup and shutdown ASM instance – (non-RAC)

2. Create (mkdg) / Drop (dropdg) / Mount (mount) / Dismount

(umount) ASM Disk Group

• Add / remove disk from ASM disk group (chdg)

• View Disk I/O Statistics (iostat)

• View Disk Group Level Information (lsdg)

• View Disk Level Information (lsdsk)

• Create (volcreate) / Resize (volresize) / Enable (volenable)/

Disable (voldisable) Volumes

• Rebalance Disk Group(rebal)

39

Enterprise ManagerCreate Snapshot

1. Create

Snapshots

2. Delete

Snapshots

• Manage up to

63 Snapshots

Command-line syntax

# /sbin/acfsutil snap create 11oct2009_db_1.before.patch

/apps/oracle/product/11.2.0/db_1acfsutil snap create:

Snapshot operation is complete.

Command-line syntax

# /sbin/acfsutil snap create 11oct2009_db_1.before.patch

/apps/oracle/product/11.2.0/db_1acfsutil snap create:

Snapshot operation is complete.

40


Agenda








41

Rename Disk GroupClone databases on same node or another node

• Change the name of a cloned disk group

• Simplify server relocation

• With triple mirroring technologies, you can split the third

mirror and mount it locally

• Rename DG / Mount DG

• Fast way to clone a database

42

Rename Disk GroupRecommended Method: Phase One followed by Phase Two

Phase One Creates a Config File

# renamedg phase=one dgname=dg_prod_data newdgname=data2

config=rename_to_data2.conf verbose=true

# cat rename_to_data2.confORCL:DATA1 DG_PROD_DATA DATA2ORCL:DATA2 DG_PROD_DATA DATA2ORCL:DATA3 DG_PROD_DATA DATA2

Phase Two – Apply the config file generated from Phase One# renamedg phase=two dgname=dg_prod_data newdgname=data2

config=rename_to_data2.conf verbose=true

Parsing parameters..

Parameters in effect:

Old DG name : DG_PROD_DATA New DG name : DATA2

…

43


Agenda








44

•Install Clusterware

•Install ASM in a separate ORACLE_HOME

•Install Database home in a separate ORACLE_HOME

Pre-11gR2 Installation

45

GI Tech StackSimplified Technology Stack – only 2 Homes to manage

Grid Infrastructure - FREE

• Clusterware

• ASM

• ACFS

Grid Infrastructure Home

i.e. /apps/11.2.0/grid

Managed by privileged user

Grid Infrastructure Home

i.e. /apps/11.2.0/grid

Managed by privileged user

Database Home

i.e. /apps/oracle/product/11.2.0/db_1

Managed by DBAs

Database Home

i.e. /apps/oracle/product/11.2.0/db_1

Managed by DBAs

46

•OHASD replaces inittab entries and manages stack

•OHASD manages startup and shutdown of entire stack

• ACFS modules get loaded

• ASM gets started and diskgroups mounted

• ACFS filesystems get mounted

• Database(s) get started

GI Tech Stack Benefits of a Simplified Technology Stack

47

•Components of GI stack run as privileged user

•Oracle recommends GI stack to be managed by privileged

user (i.e. System Administrator)

•Clustering, file system and storage volume management

roles are traditionally handled by SAs

� Significant changes in GI stack manageability that

cater to SAs

� Rich command-line features in ASMCMD and a

compact GUI tool (ASMCA) are available for SA

disposal

Grid Infrastructure StackRoles and Responsibilities


•

49

ACFS – ADVM Operational

Overview

50

ACFS Installation

• No separate install for ACFS software

• Installed as part of the Oracle Clusterware – Grid

Infrastructure installation along with ASM

• Includes kernel drivers and userspace tools

• Kernel drivers are installed as part of the GI installation

• The utilities for ACFS and ADVM are located in /sbin.

• Drivers are located in /lib/modules/<os>/extra/usm.

51

Dynamic Volume Manager and

Dynamic Volumes

• Provides volume management services and standard disk device driver interface to ACFS

• Loadable kernel driver

• Once the volume is created anOS system device file is also created

• Can be seen as /dev/asm/<volume-name>

• File systems issue I/O requests to the ADVM volume devices.

• ADVM driver maps I/O requests against a volume device to the corresponding volume file located within an ASM disk group

• Managed using SQL, ASMCMD, EM, ASMCA

Note: An AVD volume device may not be used as an OS boot or swap device, nor used as a root file system of the OS

ASM DG

/dev/asm/vol1-123

+DATA/

OS

52

ADVM Concepts

• ADVM supports all storage solutions supported for ASM

except Exadata

• Inherits and leverages underlying ASM redundancy

• On Linux platforms Oracle ADVM volume devices are created

as block devices

• Do not use raw (8) to map ADVM volume block devices into

raw volume devices.

• All device management is performed against the ASM disks

underneath ADVM, so need to manage ADVM devices

• Do not create multipathed devices over ADVM devices

• Do not create ASMLIB devices on ADVM devices

• Device partitioning (fdisk) not supported on ADVM

devices

53

Space Management with ACFS

• Conventional LUN, volume and

file system management

• Define SAN LUNs

• Present LUNs to OS

• Group LUNs into volume group

or pool

• Create logical volume

• Create file system

• ASM based space, volume and

file system management

• Define SAN LUNs

• Present LUNs to OS

• Group LUNs into ASM disk

group

• Create ASM logical volume

• Create ACFS file system

55

ACFS Build Walk-through

• Planning

• Create a ASM diskgroup

• Create a Shared DB Home Filesystem

• Create local volume

• Create a local filesystem

• Create a ACFS snapshot

57

Build Walk-thru - Planning

A typical deployment work flow:

• Systems Administrator installs the Oracle Clusterware

software

• Oracle recommends that the GI stack be installed using the

grid user, with a primary group as oinstall

• Sysadmin via ASMCMD or ASMCA (recommended),

create appropriate disk groups

• Flash Recovery Area, disk group for database binaries,

database external data, etc

• Create an ASM volume

• Create a ACFS fileystem

58

Build Walk-thru - Planning

• Creation of ASM diskgroups, volumes, or filesystems

does not require root access

• Mounting the filesystem will require root access.

• Once the ACFS filesystem or diskgroup is created,

the DBA can use these containers for the appropriate

purposes

• Install Oracle database software in the ACFS filesystem

The DBA should install database software using a

username different than grid; eg, oracle with primary group

as oinstall

• Create databases in ASM

59

ACFS – ASM Deployment Practices

Two Disk Groups• DATA diskgroup -

• Database datafiles

• Oracle clusterware file (OCR and Voting Disk)

• ACFS-ADVM volumes for:

• ORACLE_HOME (Oracle binaries)

• Oracle external files types

– BFILEs

– UTL_DIR

– ETL data and external tables

• FRA diskgroup – holds database recovery related files.

• Archive logs

• Backup and dumpsets

• Flashback logs

60

ACFS – ASM Deployment

Alternative

• DATA diskgroup - database datafiles Oracle Clusterware files

(OCR/Vote files)

• FRA diskgroup – holds database recovery related files.

• Archived log files, backups, dumpsets, flashback logs,

etc.

• DATAOH diskgroup – houses ASM volumes.

• ORACLE_HOME (Oracle binaries)

• Oracle external file types, such as BFILEs , UTL_DIR

directory contents, or ETL data/external tables

61

Build Walk-thru

•ASMCMD

•ASMCA

• Shared Oracle Home

• Non-db home (local filesystem)

62

ACFS-ADVM Build - ASMCMD

• asmcmd> mkdg diskgroup_create_crsdata.xml

• asmcmd> mkdg diskgroup_create_data.xml

• ASMCMD> lsdg

• State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name

• CONNECTED NORMAL N 512 4096 1048576 157479 151671 52493 49589 0 Y CRSDATA/

• MOUNTED NORMAL N 512 4096 1048576 209972 176839 52493 62173 0 N DATA/

63


• $ asmcmd

• ASMCMD [+] > volcreate -G DATA -s 5g ohdg1

• ASMCMD [+] > volinfo -a

• Diskgroup Name: DATA

• Volume Name: OHDG1

• Volume Device: /dev/asm/ohdg1-296

• State: ENABLED

• Size (MB): 5120

• Resize Unit (MB): 256

• Redundancy: UNPROT

• Stripe Columns: 4

• Stripe Width (K): 128

• Usage:

• Mountpath:

64


• $ ls -l /dev/asm/*

• brwxrwx--- 1 root oinstall 252, 151553 Jun 30 12:40 /dev/asm/ohdg1-

296

• # mkfs.acfs /dev/asm/ohdg1-296

• mkfs.acfs /dev/asm/ohdg1-296

• mkfs.acfs: version = 11.2.0.0.2.0

• mkfs.acfs: on-disk version = 38.0

• mkfs.acfs: volume = /dev/asm/ohdg1-296

• mkfs.acfs: volume size = 5368709120

• mkfs.acfs: Format complete.

• # mkdir /u01/app/grid/product/11.2.0/db1

• # mount /dev/asm/ohdg1-296 /u01/app/grid/product/11.2.0/db1

65

ACFS-ADVM Build - ASMCMD ACFS resize - Walk-thru

• $ /acfsutil size +200G -d /dev/asm/ohdg1-296

/u01/app/grid/product/11.2.0/db1

• acfsutil size: new file system size: 322122547200

(307200MB)

66

ACFS-ADVM Build - ASMCMD ACFS Snapshots - Walk-thru

• $ acfsutil snap create oh_db_snap \/u01/app/grid/product/11.2.0/db1

• acfsutil snap create: Snapshot operation is

complete.

• $ ls –l /u01/app/grid/product/11.2.0/db1/.ACFS/snaps]#

• drwxrwxr-x 69 oracle oinstall 8192 Jan 5 00:23

oh_db_snap

67

• ACFS-ADVM Build Walk Thru –

using ASMCA

68

Create ASM Diskgroup

70

Create Shared DB Home –cont’d

71

Shared DB Home Post-exec script

72

Create non-DB home volume

73

Verify Volume is created/enabled

75

Create filesystem over volume

YOUR DONE !!

76

ADVM-ACFS and Oracle Clusterware -

Startup

• The ADVM-ACFS drivers are loaded

• The CRS ASM agent will start the ASM instance

• As part of ASM instance startup, disk groups will be

mounted

• All ASM Dynamic volumes are enabled.

• The OS ADVM volume device node in /dev is created

• The CRS agent mounts any ACFS filesystems

• The appropriate CRS agents will start their respective

resources.

• RDBMS CRS agent will startup the database.

77

ADVM-ACFS and Oracle Clusterware -

Shutdown

• The ACFS CRS agent will attempt to perform umounts for

filesystems on ASM volumes.

• The pids of active processes using that filesystem will be

displayed and logged for sysadmin action

• Eventually CRS shutdown will terminate active pids.

• The important lesson: CRS should be seen as moving closer to

the operating system rather than just part of the Oracle stack.

• Shutting down the clusterware will stop ASM, thus shutting down

all ACFS file systems.

78

ASM 11gR2 How-Toshttp://www.dbaexpert.com/blog/category/asm/

1. ACFS command-line tools available in Oracle Database 11g Release 2

2. Configure ACFS for auto-startup for non-RAC database servers

3. Volume Management with asmcmd

4. Rename Disk Group

5. Ways to mount and dismount disk groups in Oracle Database 11g

Release 2

6. New ways to create disk groups in Oracle Database 11g Release 2

ASMCA Command-line Interface

7. Maintaining ASM Volumes and Cluster File Systems in Oracle

Database 11g Release 2

8. Creating ASM Volumes and Cluster File Systems with Enterprise

Manager in 11g Release 2

9. Creating ASM Snapshots with Enterprise Manager in 11g Release 2

79

Documents

- DBCloudShifu€¦ · • Oracle Database 11g Release 2 ... Extending ASM to Manage ALL Data Automatic Storage Management