The OFA Standard—Oracle for Open Systems · 2.1 Oracle Software and Administrative Data ... sacrificing performance is given in section 6 in this paper. Resist the temptation to

Oracle System Performance Group Technical Paper, September 24, 1995

The OFA Standard—Oracle for Open Systems

Cary V. MillsapOracle Corporation

September 24, 1995

The OFA Standard is a set of configuration guidelines that will give you faster, more reliable Ora-cle databases that require less work to maintain. The OFA Standard is written by the founder ofthe Oracle team responsible for installing, tuning, and upgrading several hundreds of sitesworldwide since 1990—this paper is based on the best practices of those hundreds of sites. Todaythe “Optimal Flexible Architecture’’ described in the OFA Standard is built into the Oracle con-figuration tools and documentation on all open systems ports.

This paper formally defines the OFA Standard for configuring complex Oracle systems at sitesdemanding high performance with low maintenance under continually evolving requirements. Italso details how the OFA is derived from requirements essential to successful implementation ofcomplicated software on any system. Along with the definition of the OFA, this paper will alsoreveal the strategy and analysis that motivate the individual recommendations of Oracle Services’Optimal Flexible Architecture. By reading this paper, you will more fully understand the chal-lenges that confront the Oracle Server configuration planner.

Permission to copy without fee all or part of this material is granted provided that the copies are not made ordistributed for direct commercial advantage, the Oracle Corporation copyright notice and the title of thepublication and its data appear, and notice is given that copying is by permission of Oracle Corporation. To copyotherwise, or to republish, requires a fee and/or specific permission. 1993, 1996 Oracle Corporation. Oracle part number A19308-1

2 • Cary V. Millsap


Contents1. OPERATING SYSTEM CONFIGURATION

1.1 Mount Points1.2 Login Home Directories1.3 User Profiles1.4 Zero Maintenance Administration

2. ORACLE FILES2.1 Oracle Software and Administrative Data2.2 Oracle Database Files2.3 Exploiting the OFA Structure for Oracle Files

3. ORACLE TABLESPACES4. RAW DEVICES VS. BUFFERED I/O5. ORACLE PARALLEL SERVER ADMINISTRA-TIVE FILES6. MOUNT POINTS FOR VLDB SITES7. QUICK REFERENCE SUMMARY

7.1 System Requirements7.2 OFA Standard Recommendations

8. REFERENCES9. UNIX UTILITIES

IntroductionAt the 1991 International Oracle User Week inMiami, Oracle Services presented a paper de-scribing a low maintenance way to configurehigh performance, growing Oracle data-bases [Millsap 1993]. The Optimal FlexibleArchitecture (OFA) described at that presenta-tion had already helped solve devastatingproblems at several production Oracle sites.outlines some of the problems that motivatedthe original OFA work. The promise of the OFAwas that an entire class of problems could beavoided by applying tested knowledge of howto optimize Oracle’s relationship with its hostoperating system (Figure 1).

A large worldwide audience welcomed theOFA because it delivered on its promise. Exist-ing installations adopted it because theirconfiguration problems were taking moneyfrom their bottom lines. New customers wel-comed the OFA because it filled a desperateneed for knowledge in areas that only a handfulof specialists in the world knew anything about.Oracle Services’ Optimal Flexible Architecturewas the first widely published document thatmade specific configuration recommendationsbased on actual field experience. The OFA hassince become the world’s most popular specifi-cation for configuring high performance, lowmaintenance Oracle database systems.

The original OFA paper described how to ex-ploit the capacity of modern operating systemsto organize massive amounts of data and, gen-erally, how to make configuration decisions thatminimize the cost of administering a database.In 1992, Oracle Services published a compre-hensive technical OFA Standard for UNIX thataddressed a broad range of configuration issuesincluding: naming standards, file protections,UNIX logins, raw devices, revision manage-ment, storage parameters, resource privileges,and rollback segments. This document was dis-tributed only to a few configuration specialists,but the work motivated several Oracle productand documentation refinements. By early 1993Oracle had integrated the OFA Standard intoOracle7.

A good standard should act as a strong floor,without becoming a ceiling that inhibits“creative magic.” Creating a useful standardrequires its author to master the precarious bal-ance between appropriate flexibility and actualusefulness to people who just want to knowwhat to type. In an attempt to achieve this bal-ance, The OFA Standard—Oracle for Open Systemsidentifies universal requirements to motivate ageneric standard, which is then illustrated witha specific example. The resulting document isintended to be useful both as a configuration

• Attempting to organize large amounts ofcomplicated software and data on diskleads to device bottlenecks and poor per-formance.

• Routine administrative tasks like softwareand data backup are performed incorrectlyor inefficiently, causing vulnerability to cor-ruptive influences.

• Attempting to switch among multiple Ora-cle databases results in corruption ofproduction data.

• Inability to adequately administer databasesegment growth results in recurrent appli-cation failures.

Figure 1. These configuration problems motivatedthe original OFA.

OFA Standard • 3


requirements definition for sites of all sizes andas the formal definition of the OFA Standard.

This paper does not address all the questionsyou will encounter as you plan your configura-tion. However, we hope that by studying theOFA Standard, you will understand the impor-tance of investing care into the optimal andflexible configuration of the system to whichyou will entrust your strategic data. We hopealso that you will want to keep learning afterhaving read this monograph, and that you willgive our team the opportunity to execute ourmission: To help you make the most efficientuse of the resources you pay for by sharingknowledge with you at your own site.

1. Operating System ConfigurationIf you are interested in installing a relationaldatabase management system, it is likely thatyou will be installing the largest applicationyour computer is capable of supporting. Beforeyour system can serve your Oracle application,you must connect and configure peripherals,configure disks for swap space and data stor-age, name your file systems, configure yournetwork, prepare your UNIX kernel for the spe-cific demands your new applications will have,and create logins [Frisch 1991, Loukides 1991,Nemeth et al 1989]. The following sections willassist you and your systems engineer to under-stand some of the issues your Oracleconfiguration planner must consider.

1.1 Mount Points

One of the first tasks undertaken to perma-nently configure a UNIX system is to selectsizes and names for file system mount points. Amount point is a directory name denoting wherethe file subsystem for a single disk slice will belinked into an existing UNIX file system. Thesample /etc/vfstab excerpt from a Solaris systemin Figure 2 shows the mapping of two mount

points, / and /usr, to two disk slices. Althoughphysically the contents of these two directorieslive in parallel slices on one disk, a UNIX userregards usr hierarchically as a subdirectory of /.

Regardless of whether you use “raw devices’’for Oracle data (sec. 4), you will have to choosemount point names for slices containing users’files and software. The file system makes UNIXeasier to administer by hiding device details,and selection of mount point names sets thestage for exactly how easy it will be. In selectingnames, the configuration planner must find anappropriate balance among these importantrequirements:

Requirement 1. The file system must beorganized so that it is easy to adminis-ter growth from: adding data intoexisting databases, adding users, creat-ing databases, and adding hardware.

Requirement 2. It must be possible to dis-tribute I/O load across sufficientlymany disk drives to prevent a perform-ance bottleneck.

Requirement 3. It may be necessary tominimize hardware cost.

Requirement 4. It may be necessary toisolate the impact of drive failure acrossas few applications as possible.

The way to balance requirements 2 and 3 is toname every one of your UNIX mount points insuch a manner that it is possible to lay a filefrom any database there without violating re-quirement 1. The following rule accomplishesthis balance.

OFA 1 Name all mount points that will hold site-specific data to match the pattern /pm where p is astring constant chosen not to misrepresent the con-tents of any mount point, and m is a unique fixed-length key that distinguishes one mount point fromanother.

#device device mount FS fsck mount mount#to mount to fsck point type pass atboot options/dev/dsk/c0t3d0s0 /dev/rdsk/c0t3d0s0 / ufs 1 yes -/dev/dsk/c0t3d0s6 /dev/rdsk/c0t3d0s6 /usr ufs 2 yes -

Figure 2. This Solaris /etc/vfstab excerpt shows the mapping of mount point names to device names. Eachrow in this file represents one disk slice.



If files from two or more applications will livetogether in a mount point subtree, it is impor-tant that the mount point name not denote thepresence of one application to the exclusion ofanother. For example, consider a situation inwhich you would like to balance I/O load bystoring both an Oracle database file and an on-line backup of some other data within a singledisk slice. You shouldn’t call that disk slice’smount point /oracle because that would put amisleading name in the path of the non-Oraclefiles stored there. And you shouldn’t call themount point /backup because you wouldn’twant to keep Oracle database files in a directorythat looks like it has been made exclusively forstorage of backed up files.

Mount points that contain files from multipleapplications are given names only to distinguishone mount point from another, like /u01 and/u02, or even /ulna/disk01 and /ulna/disk02.Using zeros to pad distinguishing keys to afixed length makes sorted lists of mount pointnames come out right, like

… /u1/u08 /u10/u09 instead of /u11/u10 /u2 … …

Unfortunately, storing files from two or moredatabases on the same drive contradicts re-quirement 4. The only way to satisfyrequirements 2 and 4 simultaneously is to buymore disk drives than your initial sizing esti-mates probably showed. Administrators withtight hardware budget constraints normallysacrifice maximal fault resilience (req. 4), be-cause performance (req. 2) is almost alwaysmore important. So, the degree to which youare willing to solve this conflict with hardwarequantity determines the strategy you shouldtake for naming your mount points. An alterna-tive strategy for the small percentage of Oraclesites that can pursue fault resilience withoutsacrificing performance is given in section 6 inthis paper.

Resist the temptation to encode controller ordisk identification characters into a mount pointname. Putting specific hardware configurationdetail into directory names causes exactly thetypes of problems that mount point names were

designed to avoid. Mount point names allow usto “abstract away’’ the details of hardware im-plementation that are irrelevant to thechallenges that a system’s users face.

Administrators can be tempted to match mountpoint names with device names because, to bal-ance I/O load, they have to make decisionsabout directory contents based on I/O statisticsprinted for devices. However, denoting hard-ware information within a mount point namecauses trouble if you ever upgrade your diskhardware. New hardware that uses differentdevice names than you had before (consider thecase of exchanging a SCSI drive for an IPI drive)will require you to invest time into changingpath names in your applications if you intend tofollow your own hardware-bound naming tra-dition. Your best mount point naming decisionis to use names unrelated to your hardware de-vice names and to examine system files (orwrite a program to do it for you) on the rareoccasion when you need to balance your sys-tem’s I/O load.1

1.2 Login Home Directories

On very old UNIX systems, home directorieswere placed in /usr. That worked well enoughfor single-user systems, but having home direc-tories in /usr did cause unnecessary challenges.For example, having directories in /usr in-creases risk of accidental file loss at upgradetime when the entire /usr subtree is replaced;also, if the /usr file system fills, UNIX will crash.UNIX books today usually recommend thatlogin home directories be placed in a directorycalled /u or /home. Using /u or /home worksfine for sites with average data volume. How-ever, a challenge arises if the set of files to becontained within a single home directory is toolarge to fit on any single disk slice. For example,Oracle Financials and Manufacturing softwareconsumes almost a gigabyte, all of which issupposed to reside in some directory calledapplmgr, but no single disk slice on your sys-

1 The requirement implicit here is very similar torequirement 6 (to which you will be introducedshortly): It must be possible to exchange hardwarecomponents without having to revise programs thatrefer to them.

OFA Standard • 5


tem is big enough to hold all of it. Hence, thefollowing requirement:

Requirement 5. It must be possible to dis-tribute across two or more disk drivesboth (a) the collection of home directo-ries and (b) the contents of anindividual home directory.

The following rule offers an elegant solution torequirement 5:

OFA 2 Name home directories matching the pattern/pm/h/u, where pm is a mount point name, h is se-lected from a small set of standard directory names,and u is the name of the owner of the directory.

Placing all home directories at the same level inthe UNIX file system means that we can puthome directories on different mount points, yetstill be able to refer to the collection of loginhomes with a single pattern. We meet require-ment 5.a without violating requirement 1 byplacing two large home subtrees at the samelevel on separate mount points. For example,the Oracle Server software owner home direc-tory might be /u01/app/oracle, and the OracleApplications software owner home directorymight be /u02/app/applmgr. Even though twoenormous subtrees are on different disk slices,we can still use a single pattern—in this exam-ple, /*/app/*—to refer to all applications ownerlogin home directories on the system.

To illustrate requirement 5.b, consider the Ora-cle Financial and Manufacturing Applicationssoftware owner, typically named applmgr,which can own more than a gigabyte of UNIXfiles on a system whose largest disk slice is600 megabytes. A simple solution is to use sym-bolic links to make directories appear in a single

subtree, even though they physically reside ondifferent mount points. Figure 3 shows thesymbolic link required to enable the OracleGeneral Ledger software to live on a separatemount point from the other applications soft-ware, yet appear to live in /u02/app/applmgr.All applmgr files are still identifiable as resi-dents of subtrees whose names match thepattern /*/app/applmgr.

Different values of h in rule OFA 2 enable sys-tem administrators to simplify their backupprocedures by using different home directoryroots for different types of users. For example,Oracle Server software files might be owned bya UNIX login called oracle, just like the file ré-sumé.tex might be owned by a login calledcmillsap. However, the administrator of theUNIX system housing both users may wish toback up the two types of files represented hereon different tapes or different schedules. Topartition a system’s users into the two classes(1) applications owners and (2) interactive log-ins, the administrator could choose homedirectory subtree names with, for example,h chosen from the list {app, home}. For exam-ple, you might use the following UNIXcommand to back up your applications soft-ware directories:

find /*/app/* -print | \bar cvf /dev/rst0

…And this UNIX command to back up yourlogin user home directories:

find /*/home/* -print | \bar cvf /dev/rst0

$ ln -s /u03/app/applmgr/gl /u02/app/applmgr/gl$ ls -l /u02/app/applmgr-rw-r--r-- 1 applmgr 1119 Jul 5 01:16 AOL.envdrwxrwxr-x 2 applmgr 2048 Jul 5 01:16 alrdrwxrwxr-x 2 applmgr 2048 Jul 5 01:16 fndlrwxrwxrwx 1 applmgr 5 Jul 5 01:16 gl -> /u03/app/applmgr/gl...$ ls -l /u03/app/applmgrdrwxrwxr-x 1 applmgr 2048 Jul 5 01:16 gl

Figure 3. In this figure, we use a symbolic link to distribute the applmgr login’s files across disks. The gldirectory in /u02/app/applmgr is actually a named (symbolic) link referring to a directory subtree that isphysically located on the /u03 mount point. Thus we can treat the gl directory as if it were a part of /u02 inour administrative programming.



1.3 User Profiles

Oracle Server is designed well to enable users tochoose which of several simultaneously activeversions of server software to run against any ofseveral databases, without sophisticated pro-gramming and without complicated userprofiles. An Oracle database user’s profileshould (1) assign a UNIX path value so that theuser’s shell can execute Oracle’s programs forsetting up the environment; (2) define an in-stance name (SID) for planned databaseconnections; and (3) execute the program thatsets the UNIX path for running Oracle software.Insert the following lines into your users’.profile to accomplish these tasks:2

# set UNIX pathPATH=/bin:/usr/bin:/var/opt/bin

# set default instanceORACLE_SID=sab

# set ORACLE_SID, ORACLE_HOME,# and PATH without askingORAENV_ASK=NO. oraenvORAENV_ASK=

By ensuring that oraenv, coraenv, and dbhomereside in /var/opt/bin,3 independent of any Ora-cle software home directory, you remove alldependence on Oracle versions from your loginprofiles.

The steps shown here should be executed foreach user connecting to an Oracle Server in-stance. A user may want to perform additionalsetup steps in the profile, such as assigningterminal settings, inserting additional softwaredirectories in the UNIX path, assigning ORA-CLE_PATH and additional environmentvariables, or selecting among various instancesfor connection.

1.4 Zero Maintenance Administration

Oracle Services still occasionally visits an unfor-tunate client whose backup programs didn’tkeep pace with the file system changes made

2 The example shown here is what you will need forBourne shell or KornShell users. For C Shell users,you will need to make functionally equivalent entriesinto the .cshrc file for each user.3 The standard directory for site local programs iscalled /usr/local/bin or /usr/lbin on most systemsbefore UNIX System V Release 4.

when someone cured an I/O bottleneck or fer-reted out free space to feed a growingapplication. We have been asked to visit severalsites at which a routine I/O balancing exercisethat should have consumed ten minutes actu-ally consumed hours of database downtimebecause administrative programs containedhard-coded references to path names that wereno longer valid after the surgery. These kinds ofproblems motivate the following requirement.

Requirement 6. It must be possible toadd or move login home directorieswithout having to revise programs thatrefer to them.

Conforming to the following rule satisfies re-quirement 6.

OFA 3 Refer to explicit path names only in filesdesigned specifically to store them, such as theUNIX /etc/passwd file and the Oracle oratab file;refer to group memberships only in /etc/group.

Hard-coded references to a file’s path namemust be systematically identified and modifiedif it ever becomes necessary to relocate that fileto a new directory. Fortunately, it is completelyunnecessary to record a path name in any fileother than a central reference file, because auser’s home directory name can be derived.C Shell and KornShell users can use ~login torefer to the home directory for login; and al-though the Bourne shell has no built-in ability tocalculate home directory names, it is easy toconstruct a program to do it. Such a program isgiven later in this document (lhd, section 9).

Similarly, group memberships should never berecorded in an administrative program, becausegroup member names can be computed from/etc/group, as is also shown later (grpx, sec-tion 9). By always using programs like lhd(or ~) and grpx instead of explicit references,your administrative tools will not require modi-fication when you move user home directoriesor change group memberships.

OFA Standard • 7


Figure 4 illustrates programming that exploitsthe ability to refer to classes of objects withsimple patterns that remain constant when youadd data, users, databases, or disks; even whenyou move files around your system. Considerthe change you would have to make to the one-line backup program shown here if you were tochange the physical location of the applmgruser’s login home directory. How would add-ing a new user to the dba or clerk group affectthe behavior of the second or third example?The OFA Standard ensures that no maintenanceon these programs would be required to ac-commodate these changes. You will see later(Figure 7) that using the OFA also gives a zeromaintenance way to manipulate all the files as-sociated with a given Oracle database as a unit,even though the files might be distributed uni-formly across many disks.

2. Oracle FilesBefore the original OFA recommendations werepublished, many people placed database files in$ORACLE_HOME/dbs [Millsap 1993]. How-ever, putting control files, redo log files, or datafiles in a subtree of the directory holding Oraclesoftware caused problems: having all databasefiles on a single disk bottlenecked the server;and having long-term database data in the Ora-cle software subtree made upgradesunnecessarily difficult because administratorshad to spend time and money to carefully pre-serve and move the data. A goal of the originalOFA was to solve these difficult problems witha flexible set of recommendations that allowed

files to be distributed across multiple diskdrives without disorganization, yet preventedsoftware and data from interfering with eachother during software upgrades. The separationof data from software is just one specific case ofa broader requirement:

Requirement 7. Categories of files mustbe separated into independent directorysubtrees so that files in one category areminimally affected by operations uponfiles in the other categories.

The following classification of Oracle files en-ables us to build a standard that meets thisrequirement:

• Product files consist of Oracle Server soft-ware and tools that are supplied on theOracle Corporation distribution media.

• Administrative files are files containing dataabout the database or instance, includingarchived redo log files, server process diag-nostic output, database creation scripts,online exports, instance parameter files, etc.

• Local software is software used with Oraclethat is written on site or purchased sepa-rately from the Oracle distributionsoftware.

• Database files consist of control files, redolog files, and data files.

This classification partitions Oracle files alongthe boundaries of different lifespans, mainte-nance schedules, and security privileges. Thefollowing sections describe a model that fulfillsrequirement 7 by placing each of these four

# Back up all files in login home subtrees of the ‘applmgr’ userfind /*/app/applmgr -print | tar cvf /dev/rst0

# Propagate a centrally administered .profile to ‘clerk’ usersfor user in `grpx clerk` ; do f=`lhd $user`/.profile ln -s `lhd applmgr`/.profile.clerk $f chown applmgr $f ; chgrp oaa $f ; chmod 644 $fdone

Figure 4. This figure shows an example of zero maintenance programming. Regardless of thephysical location of the applmgr subtrees on the system, the first find command will find thembecause of the wildcard in the mount point directory name component of the path. The secondUNIX command will create a properly protected file into each member of the clerk group. Nei-ther of these programs require modification if login home directories are added, moved, ordeleted. (Source code for the grpx and lhd programs used here is supplied later in this paper.)



types of files in subtrees independent of theother types.

2.1 Oracle Software and Administrative Data

Figure 5 is a graphical representation of theOFA Standards that will be detailed below forstoring Oracle product, administrative, and lo-cal software files. All Oracle software andadministrative data reside in subtrees of theOracle Server owner’s login home directory.The name of this directory is the value to whichOracle advises that you set the environmentvariable ORACLE_BASE.

Product Files Mature production sites requiremeans to test a new version of Oracle softwarewithout impacting daily operations. Stagingupgrades has been a difficult challenge formany Oracle sites because the focus of a defaultinstallation is that there is exactly one centralrepository of Oracle software. An inexperiencedtechnician trying to complete an Oracle installa-tion will rarely consider the requirements of anupgrade that isn’t even planned yet. Building astructure to make upgrades easy is simplework, but it requires foresight and a completeunderstanding of the Oracle version switchingmechanism discussed briefly in section 1.3.

Requirement 8. It must be possible toexecute multiple versions of applica-tions software simultaneously. Cutoverafter upgrade must be as simple for theadministrator and as transparent for theuser community as possible.

Conforming to the following rule yields a struc-ture that helps fulfill requirement 8:

OFA 4 Store each version of Oracle Server distribu-tion software in a directory matching the patternh/product/v, where h is the login home directory ofthe Oracle software owner, and v represents the ver-sion of the software.

Most OFA for UNIX implementations use val-ues of v like 7.0.16. Oracle Server patchesinvolving changes only to version numbersright of the third decimal point (e.g., from6.0.36.3 to 6.0.36.5) usually take place withoutelaborate staging, and thus most sites do notuse values of v significant beyond the thirddecimal point.

An Oracle upgrade at an OFA compliant sitethen undertakes the following steps: (1) a newversion directory is created in product; (2) thenew Oracle Server version is installed in thatdirectory; (3) a test database is created for con-firmation of the new software; (4) afterconfirmation, the home directory column oforatab is updated for the row associated withthe production instance; and (5) to cut over tothe new version, all users exit and re-enterUNIX. Executing a fresh login (normally ac-complished by leaving work one afternoon toreturn the next morning) causes the oraenv callin a user’s .profile4 to set the UNIX environ-ment properly for operation with the newlyvalidated software.

After an upgrade is judged successful, the nextstep becomes removal of the old version sub-tree, so that the space it consumes can bereclaimed. The safety with which the old ver-sion directory can be removed varies inversely 4 …Or a coraenv call in a C shell user’s .cshrc.

/u01/app/oracle

product admin local

7.0.16 TAR sab sabt6.0.37 aps6intl aps7

Figure 5. This is a graphical representation of the Oracle software owner's home directory structure. In thisexample, ORACLE_BASE would be set to the value /u01/app/oracle, and ORACLE_HOME would be setto either /u01/app/oracle/product/6.0.37 or /u01/app/oracle/product/7.0.16.

OFA Standard • 9


with the amount of custom material storedthere. The Oracle administrator should ensurethat the only files permitted to reside within theproduct subtree either are either copied or me-chanically derived from the Oracle distributionmedia (such as executables created by linkingdistributed object files). Following this policygreatly simplifies upgrade cutover and cleanupby eliminating the need for intricate file-copysurgery to preserve original work. The lifespanof the product/v subtree matches the lifespan ofa version of Oracle. Don’t put files there if theywill outlive version v of the software.

Administrative Files A key Oracle administra-tor skill is the ability to manage large amountsof data about an Oracle system. In the normalcourse of operation, for example, installers storeprograms that create databases; Oracle Serveritself produces trace files; and administratorssave structural records, instance parameters,performance statistics, backups, archives, andgeneral logbook entries on each database. Thevolume of data to be administered increases asthe number of databases on the system in-creases. These facts motivate the followingrequirement:

Requirement 9. Administrative informa-tion about one database must beseparated from that of others; theremust be a reasonable structure for or-ganization and storage ofadministrative data.

In exclusive mode environments,5 conformingto the following rule fulfills requirement 9.

OFA 5 For each database with db_name=d, storedatabase administration files in the following subdi-rectories of h/admin/d:

• adhoc—ad hoc SQL scripts for a given database

• adump—audit trail trace files

• arch—archived redo log files

• bdump—background process trace files

• cdump—core dump files

• create—programs used to create the database

• exp—database export files

5 For sites using Oracle Parallel Server, see section 5.

• logbook—files recording the status and historyof the database

• pfile—instance parameter files

• udump—user SQL trace files

where h is the Oracle software owner’s login homedirectory.

Figure 5 shows the h/admin/d directories forthree databases in a sample system; becausespace is limited, the picture does not show theadhoc, adump, etc. directories that reside be-neath each named database directory.6 Thesimple classification named here gives the ad-ministrator sufficiently many “file folders’’ tostore the necessary data about a given database.By keeping all original work pertinent to a spe-cific database in the administrative subtreeinstead of in the product subtree, as so manypeople have implicitly done in the past by stor-ing files in the dbs or rdbms/admin subtrees of$ORACLE_HOME, the administrator accom-plishes the goal of keeping the product subtreefree from files that must be carefully preservedat Oracle upgrade time.

Some administrative directories, such as archand exp, are typically too large to store on thedisk slice housing the Oracle owner’s loginhome directory. These directories can be con-nected easily into the administrative subtreewith symbolic links similar to the one shownearlier in Figure 3. Using symlinks gives a sim-ple mechanism for storing a file anywhere youneed without sacrificing the organization ofyour file system to physical size constraints.

Local Software UNIX is an especially open en-vironment designed to enable addition of newcapabilities into the operating system. Generalpurpose administrative utilities like those listedin section 9 are typically executed from a direc-tory created exclusively for site-specific utilities,such as /var/opt/bin. The OFA Standard simi-larly enables the Oracle administrator to addsite-specific Oracle capabilities into the systemin the local subtree of the Oracle owner’s login 6 The TAR directory shown in Figure 5 is created atsome sites to enable site staff to record informationabout technical assistance requests (TARs) loggedwith Oracle Worldwide Support. The upper-casename distinguishes this directory from that of a da-tabase named ‘tar,’ if one were to exist.



home directory. During Core Technologies on-site engagements, for example, an Oracle con-sultant will populate this subtree withadministrative utilities.

2.2 Oracle Database Files

Intuition tells most installers that Oracle data-base files should be separated from other fileson a system. There are concrete reasons for do-ing so, among which: database files’ lifespansdiffer from all other files on your system; anddatabase files will require a different backupstrategy than the other files on your system. Athoughtful naming strategy for database fileseliminates a whole class of administrativeproblems. Experience yields the following re-quirement:

Requirement 10. Database files should benamed so that (a) database files are eas-ily distinguishable from other files;(b) files of one database are easily dis-tinguishable from files of another;(c) control files, redo log files, and datafiles are easily distinguishable from oneanother; and (d) the association of datafile to tablespace is easily identifiable.

The following rule meets requirement 10:

OFA 6 Name Oracle database files using the fol-lowing patterns:

• /pm/q/d/control.ctl—control files

• /pm/q/d/redon.log—redo log files

• /pm/q/d/tn.dbf—data files

where pm is a mount point name, q is a string de-noting the separation of Oracle data from all otherfiles, d is the db_name of the database, n is a distin-guishing key that is fixed-length for a given file type,and t is an Oracle tablespace name. Never store anyfile other than a control, redo log, or data file associ-ated with database d in /pm/q/d.

In section 1.1 we discussed selection of mountpoint names, to which we refer here as /pm. Thetwo directory layers beneath the mount pointlevel are valuable agents of organization. Theq layer fulfills requirement 10.a. It is homolo-gous to the home layer discussed in section 1.2,as it serves the same purpose of enabling anadministrator to refer to a collection of I/O bal-anced files as a unit, in a zero maintenance way.

Many people will choose q=ORACLE, as in theoriginal OFA paper [Millsap 1993], or other ob-vious values like q=oradata. The d layer satisfiesrequirement 10.b. It eliminates confusion aboutthe database association of a given file, becausethe name of the database will always be a com-ponent of the fully qualified path name of thefile. For example, it is easy to see to which data-base /u03/oradata/sab/system01.dbf belongs.

Figure 6 is a graphical representation of a sam-ple OFA compliant directory structure forstoring I/O balanced database files that are in-dependent of the Oracle software andadministrative files. In this example, q=oradataon a system with three databases. The figureshows only the specific mount points /u08, /u09,and /u10, but the OFA administrator replicatesthis structure on every /u[0-9][0-9] mount pointin the file system. It is a good idea to build theq/d structure for every database and everymount point, even if you do not intend to putdatabase files on every mount point. The cost isonly a few inodes,7 and the benefit is that in anemergency, any free disk space on your systemwill be ready to house a file from any databasethat could require more space.

This structure makes it easy to distinguish Ora-cle database files of one database from Oracledatabase files of another. Using distinct suffixesfor different types of database files makes iteasy to distinguish one type of file from an-other, fulfilling requirement 10.c.

7 The cost is n(k+1) inodes, where n is the number ofuser data mount points on your system, and k is thenumber of databases on your system.

OFA Standard • 11


Control Files Oracle control files contain struc-tural information about the database, includingrelatively static data such as UNIX file names,and more dynamic data like the current redolog sequence number. For safety’s sake, it is es-sential that the administrator create at least twocontrol files on two different disks. Havingthree control file copies ensures that even if onefile is lost, there remains a safe duplication. Be-cause control file copies are always stored ondifferent disks, they can have identical ba-senames; the /pm component distinguishes onecontrol file from the others.

Redo Log Files Oracle redo log files recordinformation necessary to roll forward a data-base in case of CPU or disk failure. Redo logfiles are written sequentially during transactionprocessing, and they are read only during ar-chiving and recovery. Since redo log files for adatabase may all exist on a single drive, it isnecessary to encode a distinguishing key intothe basename of the file. This key n for redo logfiles is normally a three- or four-digit string de-noting the redo log file’s group and sequencewithin that group. For example, you might usea name like redo0201.log or redo01b.log to de-note redo log member 1 of group 2.

Data Files Oracle data files are the physicalmanifestation of tablespaces. The OFA Standardmeets requirement 10.d by using the tablespacename as the root of the data file’s basename.Because two or more files from a given ta-blespace can reside within the same directory, it

is necessary to encode a distinguishing key intothe basename of each data file. The length of thekey n is an almost universally accepted two.Numerals alone generate 100 unique keys in therange 00, 01, …, 99. Most databases use farfewer than 100 files per tablespace. Commondata file names are system01.dbf, glx04.dbf,and so on. However, if having 100 files per ta-blespace is not enough, then using two-digitalphanumeric lower-case keys from the set [0-9a-z][0-9a-z], for example, yields (10+26)2,or 1,296, unique values.

Personal Preferences Enthusiasts of the origi-nal OFA [Millsap 1993] may reel in algebraicanaphylaxis upon seeing the familiar/un/ORACLE/d changed into the symbolic/pm/q/d. The early OFA paper was designed tosupply a simple, specific standard that installerscould use straight out of the wrapper. How-ever, when the material proved to be popularenough that administrators began to constructsite standards around the OFA recommenda-tions, the simple way of expressing a directoryname recommendation proved to be inade-quate.

Several rounds of discussion with our custom-ers have made it clear that not all sites haveexactly the same tastes in naming, and that theexistence of two different preferences doesn’tnecessarily mean that someone is wrong. Sothere is no single correct answer to whetherOracle data directories are best named ORA-CLE or oradata or something else, and science

/u08

oradata

sab sabt intl

......

/u10

oradata

sab sabt intl

......

/u09

oradata

sab sabt intl

......

Figure 6. This figure shows the OFA structure for storing Oracle database files. Each mount point on thesystem contains a directory for holding Oracle data, oradata in this particular example. Other directoriesresiding at this level might include subtrees named home, app, and so on. Beneath the Oracle data direc-tory is a level separating the files of the various Oracle databases on the system. This example showsdirectories for three databases: sab, sabt, and intl. Note that in this structure, files from each database maybe spread across as many disk drives as needed for I/O load balancing. Each mount point is equally wellsuited for storage of any database file.



does not dictate whether database files are bestkept four layers deep in the file tree or six.However, the requirements listed in this paperappear generally indisputable, and there seemsto be unanimous agreement among thinkersabout two assertions on storing Oracle data:

• The directory’s actual name doesn’t matter,as long as it is both (a) consistent with thenames of other similar directories, and(b) chosen carefully to not misrepresent thecontents of the directory.

• The level at which any type of I/O balancedfiles are stored doesn’t matter, as long asit’s the same level on every mount point.

The OFA Standard uses an algebraic-lookingregular expression notation in the database filenaming recommendation so that administratorsare free to exploit the freedom of individualtastes, as long as the two underlying basispoints expressed above are honored. Today, thesite naming standards represented by the fol-lowing names lie completely within theboundaries of OFA compliance:

/u01/ORACLE/sab/gld01.dbf/disk04/oradata/pdnt/gld01.dbf/db016/ora/mail/gld01.dbf/mars/data/disk31/bnr1/gld01.dbf/u08/app/oracle/data/pfin/gld01.dbf

Other standards are also appropriate in specialcircumstances. Responsible site administratorsmust consider the important points outlined insection 4 before deciding to implement Oracleon UNIX “raw devices.’’ Section 6 addressesnaming standards appropriate for considerationby very large database sites.

2.3 Exploiting the OFA Structure for OracleFiles

Figure 7 shows several useful UNIX patternsidentifying classes of files that can be manipu-lated by a find command like the ones shownearlier in Figure 4. Figure 8 is a complete pictureof the relationships among the Oracle for UNIXfiles in our familiar sample three-database sys-tem. In this example, the Oracle owner’s loginhome directory is in /u01/app, and the files forthis system’s three databases are I/O balancedthroughout subtrees named /*/oradata.

/u[0-9][0-9] user-data directories/*/home/* user home directories/*/app/* user application software directories/*/app/applmgr Oracle apps software subtrees/*/app/oracle/product Oracle Server software subtrees/*/app/oracle/product/6.0.37 Oracle Server 6.0.37 distribution files/*/app/oracle/admin/sab sab database administrative subtrees/*/app/oracle/admin/sab/arch/* sab database archived log files/*/oradata Oracle database directories/*/oradata/sab/* sab database files/*/oradata/sab/*.log sab database redo log files

Figure 7. These file name patterns are useful in an OFA compliant environment.

OFA Standard • 13


3. Oracle TablespacesTablespace is a name that was introduced inOracle Version 6 for an entity that relates mul-tiple database segments to multiple operatingsystem files.

Segment Partitioning The tablespace forms theinterface through which the logical database canbe partitioned into different physical files in theoperating system. Factors that affect decisionsabout separating Oracle segments into differenttablespaces include:

/ root mount point u01/ ‘user data’ mount point #1 app/ subtree for application software oracle/ login home for the Oracle Server software owner admin/ subtree for database administrative files TAR/ subtree for Support logs intl/ administrative subtree for ‘intl’ database sab/ administrative subtree for ‘sab’ database sabt/ administrative subtree for ‘sabt’ database local/ subtree for local Oracle software aps6/ an Oracle6 admin istrative software package aps7/ an Oracle7 administrative software package product/ Oracle Server distribution files 6.0.37/ ORACLE_HOME for 6.0.37 instances 7.0.16/ ORACLE_HOME for 7.0.16 instances home/ subtree for login home directories sbm/ home for a user oradata/ subtree for Oracle data intl/ subtree for ‘intl’ database files sab/ subtree for ‘sab’ database files sabt/ subtree for ‘sabt’ database files u02/ ‘user data’ mount point #2 app/ subtree for app software applmgr/ home for the Oracle Applications owner alr/ Oracle Alert fnd/ Application Object Library gl/ -> /u03/app/applmgr/gl symbolic link to General Ledger files ... more applications home/ subtree for login home directories mlm/ home for a user oradata/ subtree for Oracle data intl/ subtree for ‘intl’ database files sab/ subtree for ‘sab’ database files sabt/ subtree for ‘sabt’ database files u03/ ‘user data’ mount point #3 app/ subtree for applications software applmgr/ auxiliary directory for Oracle Applications gl/ Oracle General Ledger home/ subtree for login home directories vrm/ home for a user oradata/ subtree for Oracle data intl/ subtree for ‘intl’ database files sab/ subtree for ‘sab’ database files sabt/ subtree for ‘sabt’ database files ... more mount points, etc.

Figure 8. This is a hierarchical directory listing for a sample OFA compliant system. Indentation shows therelationships among directories and the files and directories contained within.



• Fragmentation character. Dropping of seg-ments causes tablespace free spacefragmentation that can lead you to thedoorstep of people who hope to “de-fragment’’ your database in exchange for afew thousands of your dollars. Free spacefragmentation is preventable. By separatingsegments with different lifespans into dif-ferent tablespaces, the database creatorprevents the problems associated with ta-blespace free space fragmentation.

• I/O distribution. Only at the tablespace levelcan the administrator determine what set ofoperating system files a segment will oc-cupy. By separating segments withcompeting I/O requirements into differenttablespaces, the database creator can makeit possible to ensure well-balanced I/Oloads across hardware components.

• Administrative needs. Only at the tablespacelevel can the administrator specify a collec-tion of segments for backup, or restrict auser’s privilege to consume database space.By separating segments with different ad-ministrative characteristics into differenttablespaces, the database creator can makegive the administrator an appropriate levelof control over collections of segments.

The following requirement motivates decisionsabout tablespace creation:

Requirement 11. Tablespace contentsmust be separated to (a) minimize ta-blespace free space fragmentation,(b) minimize I/O request contention;and (c) maximize administrative flexi-bility.

The next rule fulfills requirement 11.

OFA 7 Separate groups of segments with differentlifespans, I/O request demands, and backup frequen-cies among different tablespaces. For each Oracledatabase, create the following special tablespaces inaddition to those needed for applications segments:

• SYSTEM—data dictionary segments only

• TEMP—temporary segments only

• RBS—rollback segments only

• TOOLS—general-purpose tools only

• USERS—miscellaneous user segments

This standard has proven to have several tre-mendous benefits. For example, becausedictionary segments are never dropped, andbecause no other segments that can be droppedare allowed in the SYSTEM tablespace, thisscheme guarantees that the SYSTEM tablespacewill never require a rebuild due to tablespacefree space fragmentation. Because no rollbacksegment is stored in any tablespace holding ap-plications data, the administrator is neverblocked from taking an applications data ta-blespace offline for maintenance. Becausesegments are partitioned physically by type, theadministrator can record and predict data vol-ume growth rates without complicated tools.

Tablespace Names The OFA standard of em-bedding the name of a tablespace in thebasename of its associated data files (OFA 6)means that UNIX file name length restrictionsalso restrict tablespace name lengths. AlthoughOracle tablespace names can be thirty charac-ters long, portable UNIX file names arerestricted to fourteen characters. Recall that therecommended standard for a data file ba-sename is tn.dbf, where t is a tablespace nameand n is a two-digit string. The six-charactern.dbf suffix leaves eight characters remainingfor t. The recommended naming standard fortablespaces is thus:

OFA 8 Name tablespaces connotatively with eightor fewer characters.

Eight-character tablespace names not only sim-plify data file naming, they also makeadministrative reports about tablespaces muchmore “80-column friendly.’’ The total number oftablespaces in a database is generally on theorder of 100 or less, so inventing connotativenames with eight characters is usually easy.

Connotation enables the administrator to divinethe purpose of a tablespace by looking at itsname. It is sometimes useful to encode informa-tion about what type of segment a tablespace isdesigned to store into the tablespace name. Forexample, the names GLD and GLX might con-note that these tablespaces are designed to storeOracle General Ledger data and indexes, re-spectively. Figure 9 illustrates tablespace andfile naming in an OFA compliant system.

OFA Standard • 15


Resist the temptation to embed reminders of theword tablespace in your tablespace names. Ta-blespaces are distinguishable as tablespaces bycontext, and their names do not need to conveyinformation about their type. A competent Ora-cle administrator would not confuse atablespace with another database object, sonames like TEMP_TABLESPACE are patentlyunnecessary. Administrators learn as theygather experience that embedding type infor-mation into an object’s name is usually a wasteof motion.8

8 People who, after the sermon, insist upon continu-ing to embed type information into the names of newthings are of course free to do so. However, I am alsofree to fantasize that the grammar police will makethese people use their own naming convention athome until they see how ridiculous it is. The parentsof Billy the Kid, Attila the Hun, Winnie the Pooh,and Frosty the Snowman were evidently afflictedwith such a curse.

4. Raw Devices vs. Buffered I/OThe term raw device is an informal synonym forcharacter special file, which is an unmounted diskslice that Oracle can read and write withoutincurring the overhead of UNIX I/O buffer-ing [Bach 1986, Leffler et al 1990]. A characterspecial file has a fixed size because it is allo-cated an entire disk slice. Oracle Corporationuses raw devices extensively in TPC bench-marks to increase the number of transactionsper second that Oracle can perform. As tempt-ing as a potential performance gain sounds, useof raw I/O bears undeniable costs. The follow-ing sections identify some of the benefits andcosts of using raw devices.

Benefits of Using Raw Devices The followingfactors weigh in favor of using raw devices:

• If you intend to use Oracle Parallel Serveron a UNIX cluster, with multiple UNIXnodes manipulating a single database on a

File Type orTablespace Name

File Name Size (KB)

control /u01/oradata/sab/control.ctl



redo group 1 /u03/oradata/sab/redo0101.log 5,120redo group 1 /u05/oradata/sab/redo0102.log 5,120redo group 1 /u07/oradata/sab/redo0103.log 5,120redo group 2 /u04/oradata/sab/redo0201.log 5,120redo group 2 /u06/oradata/sab/redo0202.log 5,120redo group 2 /u08/oradata/sab/redo0203.log 5,120SYSTEM /u02/oradata/sab/system01.dbf 65,536TEMP /u04/oradata/sab/temp01.dbf 131,072RBS /u03/oradata/sab/rbs01.dbf 65,536TOOLS /u07/oradata/sab/tools01.dbf 16,384USERS /u07/oradata/sab/users01.dbf 32,768AOL /u05/oradata/sab/aol01.dbf 98,304GLD /u02/oradata/sab/gld01.dbf 524,288GLD /u04/oradata/sab/gld02.dbf 524,288GLX /u05/oradata/sab/glx01.dbf 524,288

Figure 9. This is a file map of a sample OFA compliant database. Note theease with which you can discern each file’s mount point, its application, itsdatabase, and its tablespace. Note also that each file’s type (control file,redo log file, or data file) is apparent by viewing its name.



shared disk subsystem, you must use rawdevices. UNIX vendors have not yet im-plemented a way for nodes in a cluster tosimultaneously mount a shared disk sub-system.

• Some platforms offer the capability forasynchronous I/O to boost output perform-ance. Asynchronous I/O is normallyavailable only with character special de-vices, outside the UNIX file system. Askyour hardware vendor or Oracle World-wide Support for more details about yourparticular implementation.

• Using raw devices will in some cases in-crease throughput if either your processesfrequently wait for I/O, or your systemstate CPU time is persistently excessive.9

• If you have variable disk partitioning,10 us-ing raw devices for redo log files isparticularly attractive, because: Raw de-vices benefit you most for write-intensive,sequentially accessed data; and online redolog files are not included in normal operat-ing system backup procedures.

Costs of Using Raw Devices The followingfactors weigh against using raw devices:

• Raw devices are more costly to administerthan files in the UNIX file system. Opera-tions that become more difficult are backupand recovery, I/O load balancing, and addi-tion of files to the database. If you do nothave the ability to thoroughly practice da-tabase recovery before taking your project

9 An interesting exception to the performance benefittraditionally attributed to raw device configurationsoccurs with applications that rely on table scans fordata access. These applications actually performbetter on mounted file systems than on raw devices.Improving the performance of an application thatdoes frequent table scans begins with analysis of theapplication’s data access methods, not with sweep-ing changes to the underlying operating systemconfiguration.10 Variable disk partitioning is a capability offered bymany UNIX vendors’ logical volume managementsoftware. With this capability, an administrator canmake any disk slice any size at all; without variabledisk partitioning, an administrator must choose froma limited (usually small) number of ways to cut adisk into slices.

live, then do not use raw devices. If you donot have sufficient disk volume that youcan leave two or more large unformatteddisk slices available for database growth orI/O balancing, then do not use raw devices.

• Raw I/O improves throughput in only asmall percentage of production site situa-tions. Applications in which I/O or I/O-related processing is not the bottleneck willnot deliver better throughput by improvingthe performance of I/O and I/O-relatedprocessing. If I/O is your bottleneck, thenbefore choosing to use raw devices, ensurethat you have taken full advantage of per-formance optimization techniques that areless costly to you:

− Carefully construct your application tominimize I/O.

− Configure your operating system andinstance parameters so that the OracleServer operates as efficiently as possi-ble.

− Balance your I/O load across enoughdisks that each drive’s data transfer rateis within the manufacturer’s recom-mended limits for the drive and itscontroller.

After these optimizations, you are likely tofind that I/O and the CPU overhead associ-ated with I/O is no longer your bottleneck,at which point moving to a raw device con-figuration will not improve throughput.

• If your UNIX implementation does not offervariable disk partitioning, a raw deviceOracle configuration requires more diskvolume than a similarly configured data-base using buffered I/O. Fixed partitioningconstraints make it difficult to find enoughsuitably sized disk slices for redo logs andsmall data files. A dangerous consequenceof any raw device configuration is thathaving only a few large slices tempts the in-experienced configuration planner tocombine database segments that should beseparated among Oracle tablespaces. Thebenefits of raw devices are not enough toovercome the user performance and server

OFA Standard • 17


administrator workload costs of a poorfundamental configuration decision.

If yours is a big site with a sophisticated Oraclefor UNIX administration staff with time for re-search, enough budget for several disk drives,and enough patience to leave some of your diskspace unused but reserved for unanticipatedgrowth and load balancing, then using raw de-vices may be economically feasible for you. Ifyou use raw devices, you will need to meet thefollowing requirement:

Requirement 12. It must be possible totune disk I/O load across all drives, in-cluding drives storing Oracle data incharacter special files.

The following rule enables the administrator toremedy a persistent I/O imbalance in a raw de-vice configuration, because two characterspecial files can trade places on disk only if theyare precisely the same size.

OFA 9 Choose a small set of standard sizes for allcharacter special files that may be used to store Ora-cle database files.

The Verdict Use of raw I/O does not necessar-ily increase a site’s throughput, so a responsibleadministrator will study the issues before usingthem. Simply stated, if you are not runningOracle Parallel Server and testing shows thereto be no performance improvement by usingraw devices in your specific environment, thenusing them would be all cost and no gain foryou. If testing shows raw devices to improveperformance by an amount that outweighs thecost of administering them, then use raw de-vices. If you lack the time or expertise to investin construction of a test to determine whetheror not to use raw devices, then you probablyalso lack the time or expertise to administerthem effectively.

When determining whether or not to use rawdevices, do not ignore the possibility of usingthem only for some database files. Hybrid sys-tems that use unbuffered I/O for sequentialwrite-intensive files (e.g., redo log files in highlyactive OLTP environments) and buffered I/Ofor other files (e.g., control and data files thathave to be backed up by a UNIX administrator)

can offer the best mixture of low cost and highgain.

5. Oracle Parallel ServerAdministrative FilesUsing Oracle Parallel Server (OPS) adds chal-lenges to the database administrative task,because the OPS environment highlights thedistinction between instance (a collection ofprocesses and memory) and database (a collec-tion of files). For example, assume that the sabdatabase is simultaneously held open by twoOPS instances on two UNIX nodes, sab1 onnode node1 and sab2 on node node2. Regard-less of which instance hosts the connection, areport on a data dictionary table (or view, suchas dba_users) will provide a consistent answer.However, a report produced on a dynamic per-formance table (or view, such as v$parameter)will give an answer wholly dependent uponwhich instance hosts the connection. Hence themotivation for the following requirement:

Requirement 13. (a) Database specificadministrative data must be stored in acentral place accessible to all instanceadministrators; and (b) instance specificadministrative data must be distin-guishable as associated with a giveninstance by file name.

In other words, you must never have to searchmultiple directories to find all the reports ondba_users for a given database, and you mustnever have to look through a list ofv$parameter reports to find the ones related tothe instance you are inspecting.

We can refine our understanding of how to ful-fill requirement 13 by observing the followingfeatures about OFA 5: the directories arch, cre-ate, and exp are database administrativedirectories; adump, bdump, cdump, pfile, andudump are instance administrative directories;and adhoc and logbook are curious mixtures ofboth.



One way of structuring the administrative di-rectory for the sab database is depicted inFigure 10. In this example, each administrativesubtree that must contain information specificto two or more instances uses another directorylayer to denote the distinction in the file name.For example, consider the admin/sab/logbookdirectory. Operationally, an Oracle administra-tor would use this as the working directory fora SQL*Plus, SQL*DBA, or Server Manager ses-sion while administering the sab database. Adata dictionary table report like users.lst wouldbe spooled to the current working directory,regardless of whether the administrator is con-nected to sab1 or sab2, to meet requirement 13.But a dynamic performance table report likeparams.lst would be spooled to the directorynamed for the instance to which the administra-

tor running the report is connected. In additionto flexibly accommodating the storage of data-base and instance report history, the Figure 10arrangement enables administrators to quicklyfind trace files created by a given instance.Therefore, the structure shown here fulfills re-quirement 13.b.

Further OPS environment complication derivesfrom the fact that any instance can be adminis-tered from any node in the cluster. Anadministrator is equally likely to have loggedinto node2 to administer the sab database asnode1, and, through SQL*Net, the node2 usercan administer the sab1 instance as sab2. How-ever, to fulfill requirement 13.a, theadministrator must use a single central adminis-trative directory to store database reports. Amechanism for accommodating this require-

/u01/app/oracle/admin/sab/ administrative directory for ‘sab’ database adhoc/ directory for miscellaneous scripts adump/ directory for audit file dumps sab1/ audit file dest for ‘sab1’ instance sab2/ audit file dest for ‘sab2’ instance arch/ log archive dest for all instances redo0001.arc archived redo log file bdump/ directory for background dump files sab1/ background dump dest for ‘sab1’ instance sab2/ background dump dest for ‘sab2’ instance cdump/ directory for core dump files sab1/ core dump dest for ‘sab1’ instance sab2/ core dump dest for ‘sab2’ instance create/ directory for creation scripts exp/ directory for exports 930920full.dmp Sep 20 full export dump file export/ directory for export parfiles import/ directory for import parfiles logbook/ directory for ‘sab’ logbook entries sab1/ directory for ‘sab1’ instance reports params.lst v$parameter report for ‘sab1’ instance sab2/ directory for ‘sab2’ instance reports params.lst v$parameter report for ‘sab2’ instance users.lst dba_users report pfile/ directory for instance parameter files sab1/ directory for ‘sab1’ instance parameters sab2/ directory for ‘sab2’ instance parameters udump/ directory for user dump files sab1/ user dump dest for ‘sab1’ instance sab2/ user dump dest for ‘sab2’ instance

Figure 10. Administrative directory structure for dual instance OPS

OFA Standard • 19


ment is to choose exactly one node to act as“administrative home’’ for maintenance of adatabase. Each node housing an instance thatconnects to that database may have a distinctproduct directory, but the admin/d directoryfor a given database must be a remotelymounted link to the admin/d directory on theadministrative home node. Thus, when an ad-ministrator logged onto node2 sets~oracle/admin/sab as the session’s currentworking directory, the working directoryphysically becomes the centralnode1:/u01/app/oracle/admin/sab, fulfillingrequirement 13.

From this discussion, we can induce the fol-lowing rule:

OFA 10 If you are using Oracle Parallel Server,select exactly one node N to act as Oracle adminis-trative home for the cluster to house theadministrative subtree defined in rule OFA 5. Let hbe the Oracle software owner’s login home directoryon node N. Create a directory named for each in-stance accessing database d within the adump,bdump, cdump, logbook, pfile, and udump direc-tories of N:h/admin/d. On every node n in thecluster except N, mount the remote directoryN:h/admin/d as the administrative directory fordatabase d (i.e., as n:h/admin/d).

6. Mount Points for VLDB SitesThere are at least two very different correct so-lutions to the mount point naming challenge,one for sites who can relax requirement 4 tosatisfy requirement 2 economically, and one forsites able to afford massive disk farms. Mostsites should use OFA 1. Only a handful of verylarge database (VLDB) sites worldwide shouldconsider the following strategy.

OFA 11 If you can afford enough hardware that:

1. You can guarantee that each disk drive11 willcontain database files from exactly one applica-tion; and

11 This is important: each disk drive, not disk slice.You violate requirement 4 if you place two applica-tions on the same disk, even if they’re stored in twodifferent slices.

2. You can dedicate sufficiently many drives toeach database to ensure that there will be no I/Obottleneck.

Then name mount points matching the pattern /qdmwhere q is a string denoting that Oracle data andnothing else is to be stored there, d is the value of thedb_name init.ora parameter for the single databasethat will be stored there, and m is a unique fixed-length key that distinguishes one mount point for agiven database from another.

Mount point names like /ora/intl01 connote acommitment to putting only control, redo log,and data files from the single Oracle databasecalled “intl’’ on a given disk slice.12 If you adoptthis standard, you commit never to need to useyour Oracle data mount points for anythingelse. Only a very small percentage of all Oraclecustomers worldwide can make this commit-ment, and you should not adopt this particularVLDB component of the OFA Standard if thereis a risk that you cannot.

7. Quick Reference SummaryThe following sections are intended for use as areference guide that summarizes the systemrequirements and OFA Standard recommenda-tions discussed in this paper.

7.1 System Requirements

1. The file system must be organized so that itis easy to administer growth from: addingdata into existing databases, adding users,creating databases, and adding hardware.

2. It must be possible to distribute I/O loadacross sufficiently many disk drives to pre-vent a performance bottleneck.

3. It may be necessary to minimize hardwarecost.

4. It may be necessary to isolate the impact ofdrive failure across as few applications aspossible.

5. It must be possible to distribute across twoor more disk drives both (a) the collectionof home directories and (b) the contents ofan individual home directory.

12 In this example, q=ora/, d=intl, and m=01.



6. It must be possible to add or move loginhome directories without having to reviseprograms that refer to them.

7. Categories of files must be separated intoindependent directory subtrees so that filesin one category are minimally affected byoperations upon files in the other catego-ries.

8. It must be possible to execute multiple ver-sions of applications softwaresimultaneously. Cutover after upgrademust be as simple for the administrator andas transparent for the user community aspossible.

9. Administrative information about one da-tabase must be separated from that ofothers; there must be a reasonable structurefor organization and storage of administra-tive data.

10. Database files should be named so that(a) database files are easily distinguishablefrom other files; (b) files of one database areeasily distinguishable from files of another;(c) control files, redo log files, and data filesare easily distinguishable from one another;and (d) the association of data file to ta-blespace is easily identifiable.

11. Tablespace contents must be separated to(a) minimize tablespace free space fragmen-tation, (b) minimize I/O request contention;and (c) maximize administrative flexibility.

12. It must be possible to tune disk I/O loadacross all drives, including drives storingOracle data in character special files.

13. (a) Database specific administrative datamust be stored in a central place accessibleto all instance administrators; and(b) instance specific administrative datamust be distinguishable as associated with agiven instance by file name.

7.2 OFA Standard Recommendations

1. Name all mount points that will hold site-specific data to match the pattern /pmwhere p is a string constant chosen not tomisrepresent the contents of any mountpoint, and m is a unique fixed-length key

that distinguishes one mount point fromanother.

2. Name home directories matching the pat-tern /pm/h/u, where pm is a mount pointname, h is selected from a small set of stan-dard directory names, and u is the name ofthe owner of the directory.

3. Refer to explicit path names only in filesdesigned specifically to store them, such asthe UNIX /etc/passwd file and the Oracleoratab file; refer to group membershipsonly in /etc/group.

4. Store each version of Oracle Server distribu-tion software in a directory matching thepattern h/product/v, where h is the loginhome directory of the Oracle softwareowner, and v represents the version of thesoftware.

5. For each database with db_name=d, storedatabase administration files in the follow-ing subdirectories of h/admin/d:

• adhoc—ad hoc SQL scripts for a givendatabase

• adump—audit trail trace files

• arch—archived redo log files

• bdump—background process trace files

• cdump—core dump files

• create—programs used to create the da-tabase

• exp—database export files

• logbook—files recording the status andhistory of the database

• pfile—instance parameter files

• udump—user SQL trace files

where h is the Oracle software owner’slogin home directory.

6. Name Oracle database files using the fol-lowing patterns:

• /pm/q/d/control.ctl—control files

• /pm/q/d/redon.log—redo log files

• /pm/q/d/tn.dbf—data files

where pm is a mount point name, q is astring denoting the separation of Oracledata from all other files, d is the db_name

OFA Standard • 21


of the database, n is a distinguishing keythat is fixed-length for a given file type, andt is an Oracle tablespace name. Never storeany file other than a control, redo log, ordata file associated with database d in/pm/q/d.

7. Separate groups of segments with differentlifespans, I/O request demands, andbackup frequencies among different ta-blespaces. For each Oracle database, createthe following special tablespaces in additionto those needed for applications segments:

• SYSTEM—data dictionary segmentsonly

• TEMP—temporary segments only

• RBS—rollback segments only

• TOOLS—general-purpose tools only

• USERS—miscellaneous user segments

8. Name tablespaces connotatively with eightor fewer characters.

9. Choose a small set of standard sizes for allcharacter special files that may be used tostore Oracle database files.

10. If you are using Oracle Parallel Server, se-lect exactly one node N to act as Oracleadministrative home for the cluster tohouse the administrative subtree defined inrule OFA 5. Let h be the Oracle softwareowner’s login home directory on node N.Create a directory named for each instanceaccessing database d within the adump,bdump, cdump, logbook, pfile, andudump directories of N:h/admin/d. Onevery node n in the cluster except N, mountthe remote directory N:h/admin/d as theadministrative directory for database d (i.e.,as n:h/admin/d).

11. If you can afford enough hardware that:

1. You can guarantee that each disk drivewill contain database files from exactlyone application; and

2. You can dedicate sufficiently manydrives to each database to ensure thatthere will be no I/O bottleneck.

Then name mount points matching the pat-tern /qdm where q is a string denoting that

Oracle data and nothing else is to be storedthere, d is the value of the db_name init.oraparameter for the single database that willbe stored there, and m is a unique fixed-length key that distinguishes one mountpoint for a given database from another.

8. ReferencesBACH, M. 1986. The Design of the UNIX Operating

System. Englewood Cliffs, New Jersey: Pren-tice Hall.

FRISCH, A. 1991. Essential System Administration.Sebastopol, California: O’Reilly & Associ-ates.

LEFFLER, S.; MCKUSICK, M.; KARELS, M.;QUARTERMAN, J. 1990. The Design and Im-plementation of the 4.3BSD UNIX OperatingSystem. Reading, Massachusetts: Addison-Wesley.

LOUKIDES, M. 1991. System Performance Tuning.Sebastopol, California: O’Reilly & Associ-ates.

MILLSAP, C. 1993. “An optimal flexible architec-ture for a growing Oracle database.” InOracle Magazine, vol. VII no. 1 (Winter1993): 41–46. Published originally as anOracle Corporation white paper, 1990. Alsopublished as “Configuring a growing Ora-cle V6 database for optimal performance” in1991 International Oracle User Week Proceed-ings, paper 513.

NEMETH, E; SNYDER, G; SEEBASS, S. 1989. UNIXSystem Administration Handbook. EnglewoodCliffs, New Jersey: Prentice Hall.

9. UNIX UtilitiesUNIX site administrators are always on thelookout for useful utilities that ease the burdenof administering complex applications. Thepages at the end of this paper give you portableutilities that are typical of the small, reliabletools used by OFA compliant Oracle sites. Localgeneral-purpose utilities such as these shouldnormally be stored in the directory called/var/opt/bin. Enjoy.



LHD(L) Oracle Services Utilities LHD(L)

NAME

lhd — print login home directory name for a given user

SYNOPSIS

lhd [login]

DESCRIPTION

lhd prints the name of the login home directory for a given UNIX login, to allow the administra-tor to refer to a user’s login home directory without hard-coding the path name. Using `lhdlogin` in the Bourne shell is the equivalent of ~login in the C shell or KornShell. lhd enablescreation of zero-maintenance administration programs that can survive file system changeswithout modification.

EXAMPLESexample$ . `lhd applmgr`/AOL.env

This example shows a line typical to that executed to set up a UNIX environment for OracleApplications. The .profile containing this line of code would not require modification if thelogin home directory for applmgr were to change.

AUTHOR

Cary V. Millsap, Oracle Services

SOURCE#!/bin/sh## lhd - print login home directory name for a given user## Cary Millsap, Oracle Services# @(#)1.1 (93/05/17)

prog=`basename $0ìf [ $# -eq 0 ] ; then login=`whoamièlif [ $# -eq 1 ] ; then login=$1else echo "Usage: $prog login" >$2 exit 2finawk -F: '$1==login {print $6}' login=$login /etc/passwd

OFA Standard • 23


GRPX(L) Oracle Services Utilities GRPX(L)

NAME

grpx — print the list of users belonging to a given group

SYNOPSIS

grpx group

DESCRIPTION

grpx prints the list of users belonging to a given group. grpx can be used to eliminate the needfor hard-coding group memberships into administrative programs used to manipulate (e.g.,back up, propagate files to) classes of users.

EXAMPLESexample$ for u in `grpx clerk` ; doexample> cp /etc/skel/.profile `lhd $uèxample> done

This example shows how the administrator can propagate a skeleton .profile to the home direc-tory for each member of a group. This code would not require modification if the membershiplist of the clerk group were to change.

AUTHOR

Cary V. Millsap, Oracle Services

SOURCE#!/bin/sh## grpx - print the list of users belonging to a given group## Cary Millsap, Oracle Services# @(#)1.1 (93/07/04)

prog=`basename $0ìf [ $# -ne 1 ] ; then echo "Usage: $prog group" >&2 exit 2fig=$1# calculate group id of ggid=`nawk -F: '$1==g {print $3}' g=$g /etc/group`# list users whose default group id is gidu1=`nawk -F: '$4==gid {print $1}' gid=$gid /etc/passwd`# list users who are recorded members of gu2=`nawk -F: '$1==g {gsub(/,/," "); print $4}' g=$g /etc/group`# remove duplicates from the union of the two listsecho $u1 $u2 | tr " " "\012" | sort | uniq | tr "\012" " "echo

Documents

The OFA Standard—Oracle for Open Systems · 2.1 Oracle Software and Administrative Data ... sacrificing performance is given in section 6 in this paper. Resist the temptation to