Oracle Optimized Solution for Secure Backup and …download.robotron.de/pdf/oracle/oos-backup-recovery-1973210_2.pdf · Best Practices for Performing Local Disk-Based Backups with

Oracle Optimized Solution for Secure Backup and Recovery O R A C L E W H I T E P A P E R | O C T O B E R 2 0 1 5

ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY

Table of Contents

Introduction 1

Solution Overview 1

Addressing Data Recovery Needs and Requirements 2

Creating More Secure Backup and Recovery Environments 3

Architecture Overview 4

Solution Software Components 4

Oracle Solaris 11 4

Oracle Linux 5

Oracle's StorageTek Automated Cartridge System Library Software 5

Oracle Key Manager 5

Oracle's StorageTek Tape Analytics Software 6

Oracle Enterprise Manager Cloud Control 7

Solution Hardware Components 7

Network Infrastructure 8

Oracle's SPARC Servers 8

Oracle's x86-Based Servers 9

Oracle ZFS Storage Appliance 9

Zero Data Loss Recovery Appliance 10

Oracle's StorageTek Tape Library Products 10

Oracle's StorageTek Tape Drives 12

Reference Architecture Components in Tape Environments 12

Oracle Secure Backup 13


Cloud-Based Database Backups 14

Backup Best Practices 14

Best Practices for Accomplishing Recovery Time and Recovery Point Objectives 15

Using a Combination of Software Tools 15

Best Practices for Backup and Recovery in Virtualized Environments or with

Unstructured Data 16

Best Practices for Backup and Recovery in Virtualized Environments and

Unstructured Data with NDMP 16

Best Practices for a Secure Backup and Recovery Implementation 16

Security Technical Implementation Guides 16

Component-Level Security Recommendations 17

Best Practices for Backup and Recovery of Operating Systems 19

Oracle Solaris 11 19

Backing Up Domains 20

Backing Up Oracle Solaris Zones 20

Best Practices for Database Backups (Oracle Database 11g Release 2 or Higher) 21

Recommended Database Backup Schedule 21

Always Use a Fast Recovery Area 21

Additional Database Configuration Best Practices 21

Oracle RMAN Configuration Best Practices 22

Restore and Recovery Best Practices 22

Best Practices for Performing Local Disk-Based Backups with Oracle RAC 22

Best Practices for Performing Local Disk-Based Backups with Oracle RAC One

Node 23


Database Best Practices for Disk-Based Backups 23

Database Best Practices for Tape-Based Backups 23

Best Practices for Tape-Based Backups 24

Network Configuration Best Practices 24

InfiniBand Network to Media Server Configuration Best Practices 24

1 GbE or 10 GbE Network to Media Server Configuration Best Practices 24

Configure Persistent Bindings for Tape Devices 25

Back Up the Oracle Secure Backup Catalog 25

Best Practices for Oracle Engineered Systems Backups 25

Oracle SuperCluster Backup Best Practices 25

Backing Up the Oracle SuperCluster Infrastructure 25

For More Information 26

Oracle Exadata Backup Best Practices 26

Oracle Database Appliance Backup Best Practices 26

Configuring Oracle Database Appliance for Local and External Backups 26


Best Practices for Oracle ZFS Storage Appliance 27

Use the Oracle Engineered Systems Backup Utility for Oracle ZFS Storage

Appliance 27

Network Configuration Best Practices 27


Storage Configuration Best Practices 28


Configuring Oracle RMAN Projects and Shares on Oracle ZFS Storage

Appliance 29

Database Backup and Restore Best Practices with Oracle ZFS Storage Appliance and

Oracle Engineered Systems 29

Configure Network Connectivity 29

Follow Recommended Best Practices for Database Backup and Restore 30

Tune the Oracle Database Instance for Oracle RMAN 30

Configure Oracle RMAN 31

Best Practices Using Oracle ZFS Storage Appliance 31

Use ZFS Snapshots 31

Using Non-Oracle Exadata Storage for Oracle SuperCluster and Oracle Exadata

Database Machine Backups 32

Providing Backup and Recovery Savings and High Availability with Remote DR Sites 32

Conclusion 33

References 34

1 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY

Introduction

Oracle engineered systems and Oracle Optimized Solutions offer massive scalability and performance

for today's larger and more complex workloads and increasingly diverse data environments.

Engineered, optimized, and integrated, these systems provide the radical processing speeds,

significantly faster deployments, and simplified data center operations needed to achieve unmatched

enterprise performance levels. Combining best-of-breed hardware and software components with

game-changing technical innovations, Oracle engineered systems and Oracle Optimized Solutions

deliver such massive capacity and extreme performance that they require a new way of thinking about

backup and recovery.

With ownership of both the hardware and software technology, Oracle is in the unique position of being

able to plan, design, and execute backup solutions for these powerful and scalable systems. This

paper describes Oracle Optimized Solution for Secure Backup and Recovery, with particular focus on

Oracle engineered systems, including Oracle Exadata Database Machine (Oracle Exadata), Oracle

Database Appliance, Oracle SuperCluster, and Oracle Optimized Solutions. This pretested, high-

performance backup solution can be used to accelerate data protection processing and management

with breakthrough cost structures using Oracle software powered by Oracle's servers, operating

systems, and tape storage.

Delivering next-generation data protection with redundancy, high availability, and the security of

encryption, Oracle Optimized Solution for Secure Backup and Recovery offers a flexible, secure,

multitier architecture for large and small environments. The solution also provides virtually unlimited

scalability with centralized management and end-to-end data protection across heterogeneous

technologies.

Solution Overview Oracle Optimized Solution for Secure Backup and Recovery is designed as a complete solution for performing network backups of heterogeneous clients. These clients include both Oracle engineered systems and Oracle Optimized Solutions. Leveraging built-in Oracle integration, Oracle Optimized Solution for Secure Backup and Recovery provides pretested, recommended configurations for complete backup and recovery solutions. This solution provides options for traditional disk and tape backups as well as cloud-based backups, and includes Oracle

as shown in Figure 1. The architecture is designed to simplify solution implementation and facilitate future upgrades as needed for data growth.

With complete end-to-end data protection for network backup clients, Oracle Optimized Solution for Secure Backup and Recovery can also complement the Oracle Recovery Manager (Oracle RMAN) and Oracle Active Data Guard data protection solutions using Oracle ZFS Storage Appliance for cost-effective, long-term storage on high-performing encrypted tape. Cloud-based database backups are also supported. Oracle Optimized Solution for


Secure Backup and Recovery is a comprehensive solution for Oracle Database backups with support for Oracle Database 9i or higher.

Figure 1. Oracle Optimized Solution for Secure Backup and Recovery supports different backup client types.

Oracle Optimized Solution for Secure Backup and Recovery is a unique offering in that it is both low cost and high performance, making it a clear industry leader in terms of price/performance for backup and recovery. Indeed, Oracle Secure Backup software licensing costs can be significantly less than the costs of comparable competitive products, because the tape management software component carries a low-cost, one-time software licensing fee per tape drive used. Using solutions that do not rely on tape backups can save on the costs of licensing tape management software as well. For example, backups that utilize Oracle ZFS Storage Appliances or Oracle Exadata storage systems are disk-only solutions and do not incur software licensing costs. In addition, the solution enables the use of tape the lowest-cost storage media, which is ideal for storing older backup copies. Capable of scaling nearly linearly, the solution uses a building block approach that consists of pretested server and storage configurations.

When this solution is deployed to back up Oracle engineered systems and Oracle Optimized Solutions, it dramatically simplifies backup and recovery processing and management. Typically deployed in replicated configurations, Oracle Optimized Solutions eliminate the need to deploy complex third party deduplication technologies, because simple tape and disk backups can be performed at one or both sites for complete data protection.

Addressing Data Recovery Needs and Requirements

When preparing data center backup and other data protection strategies, special attention must be paid to the recovery point objectives (RPOs) and recovery time objectives (RTOs) for all data and applications. An RPO defines the maximum acceptable level of data loss following an unplanned event natural or man-made that can cause data to be lost. Measured backwards in time, and starting from the instant the event occurs, the RPO is specified in seconds, minutes, hours, or days. It is used to help administrators select optimal disaster recovery technologies and procedures, and it also helps to determine the minimum frequency for performing backups. For example, Wall Street trading firms need an RPO of seconds or less, and must provide redundant, remote systems to ensure that transactions are not lost. Organizations with more static data will not have such granular RPOs.


An RTO is the period of time within which IT capabilities must be restored following an unplanned event. IT departments determine RTOs by calculating the amount of revenue lost per unit of time as a result of the event, or by determining the maximum amount of time that their organizations can afford to be without IT functionality.

Figure 2 illustrates various data center backup attributes, various Oracle backup and storage hardware, and which hardware addresses the various attributes. These factors should be kept in mind when evaluating backup solutions for Oracle engineered systems and Oracle Optimized Solutions.

Figure 2. Oracle Optimized Solution for Secure Backup and Recovery addresses a range of retention needs as well as requirements for RTOs and RPOs.

Creating More Secure Backup and Recovery Environments

Protecting data is an essential business concern for all enterprises. Enterprise-level backup environments typically use a centralized backup and recovery model, which simplifies administration as the size of data continues to grow. However, this centralization also presents a potential security threat: If an intruder gains access to a centralized backup or media server, data for many systems across the enterprise can be compromised. Physical security is, of course, fundamental, and access to servers and media libraries must be protected and audited. Backup servers, clients, networks, storage, and tape libraries should all be configured following recommended security guidelines to provide end-to-end security. Data encryption is also recommended to protect data at rest and while in transit across networks and to/from storage media.

The following steps can help create a more secure backup and recovery environment.

» Simplify the infrastructure. Most backup and recovery environments are based on a complex infrastructure, making implementation and management complicated. This complexity increases the risk of security vulnerabilities. A backup and recovery implementation as a whole is only as secure as its most vulnerable component, and it can be challenging to securely configure the myriad interacting components and products in a heterogeneous system. Oracle Optimized Solutions simplify backup and recovery implementations through the use of consolidation and virtualization technologies. Oracle also offers security guidelines and recommendations, and many Oracle components have security built-in by default.

» Reduce implementation flaws. Secure software is important but not sufficient by itself. Most security vulnerabilities arise from flawed implementation and architecture, including improper configuration and access control, lack of patch management, unencrypted communications, and inadequate security policies and processes. Based on current security best practices, Oracle Optimized Solutions provide proven and tested architecture recommendations for increased backup and recovery solution protection.


» Eliminate performance and cost penalties. Many security processes, such as on-the-fly encryption/decryption, can have a significant negative impact on the performance and cost of a backup and recovery solution. Oracle

-based systems, which offer high-performance security using cryptographic instruction accelerators that are directly integrated into the processor cores. By providing wire-speed security capabilities, Oracle systems eliminate the performance and cost penalties typically associated with real-time, secure computing.

Architecture Overview Oracle Optimized Solution for Secure Backup and Recovery features a scalable, multitier architecture that includes backup clients, media servers, administrative servers, Oracle disk devices, and encrypted tape devices. Designed to be software-agnostic, Oracle Optimized Solution for Secure Backup and Recovery can work with Oracle RMAN, Oracle Secure Backup, Symantec NetBackup, or other third-party backup software. For illustration purposes, this paper refers to use of Oracle Secure Backup software throughout.

Note that in disk-only environments such as those using Oracle Exadata storage, Oracle Exadata Storage Expansion Racks, or Oracle ZFS Storage Appliances no backup software is required. Disk backups can be completed using the operating system and Oracle Database tools such as Oracle RMAN alone.

Solution Software Components

The recommended architectures in this solution feature common software components, as shown in Table 1. The following sections describe these software components.

TABLE 1. SOLUTION SOFTWARE COMPONENTS

Software Component Description

Oracle Solaris 11, Oracle Linux Operating system

StorageTek Automated Cartridge System Library Software

A centralized library management solution

Oracle Key Manager A highly available encryption key management system for encrypted tape backups

StorageTek Tape Analytics Software that provides intelligent monitoring capabilities

Oracle Enterprise Manager Cloud Control

Software that provides fully automated, unified data protection management

Oracle Solaris 11

Oracle Solaris 11 is the first operating system engineered for enterprise clouds. With built-in virtualization, Oracle Solaris 11 offers fast, intelligent provisioning capabilities for rapid service setup and maintenance. The operating system provides fully integrated security for users, applications, and devices with fine-grained delegated management and the latest security standards. Scalable data management is achieved with Oracle Solaris ZFS, the innovative default file system in Oracle Solaris 11. Oracle Solaris ZFS brings advanced storage features such as built-in deduplication, encryption, and thin provisioning to enterprise servers, and Oracle Solaris ZFS snapshots help to deliver boot environments that enable fail-safe updates and self-recovery.

Oracle Solaris 11 offers mission-critical infrastructure that simplifies software lifecycle management and cloud-scale data management while providing advanced protection for public, private, and hybrid cloud environments. New features increase performance, reliability, security, efficiency, and virtualization, as well as preserve full compatibility


with a vast portfolio of existing third-party products and internally developed customer applications. Oracle Solaris 11 is engineered jointly with Oracle applications and middleware to deliver unique features that increase performance, streamline management, and automate support for Oracle deployments.

Oracle Linux

For those sites using Oracle x86 servers, Oracle Linux brings the latest Linux innovations to market, delivering extreme performance, advanced scalability, and reliability for enterprise applications and systems. Optimized for the Oracle stack, Oracle Linux is built and tested to run Oracle hardware, databases, and middleware and is recommended for all enterprise applications.

Oracle's StorageTek Automated Cartridge System Library Software

Oracle's StorageTek Automated Cartridge System Library Software (StorageTek ACSLS) provides a strategic, centralized library management solution. Figure 3 shows two environments one without StorageTek ACSLS and one after deploying StorageTek ACSLS.

Figure 3. StorageTek ACSLS provides centralized management and high availability for StorageTek tape libraries.

Without StorageTek ACSLS software, organizations must grapple with distributed management, multiple administrators and libraries, excess environmental consumption, poor resource utilization, and higher total cost of ownership. Deploying StorageTek ACSLS as part of a tape management solution enables centralized management of a single library by a single administrator, with lower environmental consumption, standard policy management, balanced workload and resource utilization, maximum storage efficiency, and lower cost of ownership. In addition, StorageTek ACSLS can help tape environments meet high-availability objectives.

Oracle Key Manager

Oracle Optimized Solution for Secure Backup and Recovery features Oracle Key Manager, which is a highly available encryption key management system for encrypted tape backups. Oracle Key Manager 2.5 makes it easy to implement and scale storage-based encryption for operational and archive data without unnecessary cost and complexity. A sample configuration is shown in Figure 4. Oracle Key Manager 2.5 is designed with an emphasis on simplicity, security, and scalability to help organizations realize the following advantages:

» High security. A hardened solution, Oracle Key Manager provides FIPS 140-2 Level 3 compliance (through Sun Crypto Accelerator 6000 PCIe Card) and secure key protection throughout the key lifecycle, with a

dedicated key management and delivery network.


» Interoperability. An open, standards-based architecture supports diverse storage devices from mainframes to open systems under a single storage key management system.

» High availability. Active n-node clustering, dynamic load balancing, and automated failover ensure high availability.

» Simplified management. A secure client GUI facilitates administration whether it is at one site or worldwidethrough user-defined, policy-based automatic key management.

» Scalability. A single clustered Oracle Key Manager appliance pair can be used to manage thousands of storage devices and millions of encryption keys, making the solution scale easily and nondisruptively.

Figure 4. An Oracle Key Manager sample configuration.

Oracle's StorageTek Tape Analytics Software

is an intelligent monitoring application that proactively captures library, drive, and media health metrics and runs analytical calculations on these data elements. As a result, the software can eliminate library, drive, and media errors and empower tape storage administrators to make proactive decisions about tape environments prior to device failures. Figure 5 illustrates how the StorageTek Tape Analytics software collects data from libraries, drives, and media.


Figure 5. StorageTek Tape Analytics software simplifies tape monitoring.

Oracle Enterprise Manager Cloud Control

Oracle Enterprise Manager Cloud Control is a centralized cloud management solution, with a single view of the entire IT environment from applications to disk. It features a pluggable framework, providing an easy way to update Oracle Enterprise Manager Cloud Control with support for the latest Oracle product releases. Several plugins, such as one for Oracle Database, are installed by default. Some newer product releases, such as Zero Data Loss Recovery Appliance, require an additional plugin to be downloaded and installed.

Oracle Enterprise Manager Cloud Control is used to provide end-to-end data protection management in this solution. This software provides a unified view from the time a backup is initiated to the time it is stored on disk or tape, or replicated to another storage device in a remote data center. All backup locations are tracked, so that any Oracle Recovery Manager (Oracle RMAN) restore and recovery operation can retrieve the most appropriate backups, wherever they reside.

Advanced storage monitoring and reporting helps administrators effectively manage current and future throughput and capacity. The amount of space needed for each database under management is predictively calculated based on its historical backup space usage and recovery window goal. The space needed is displayed for each database and the appliance aggregates the total space needed for all databases as a percentage of the total available storage, taking the guesswork out of accommodating data growth. Capacity reports provide summary and detailed information on storage utilization as well as average and maximum throughput, plus detailed information on CPU, memory, and IOPS. In addition, warnings can be generated if the space needed is within a user-configurable threshold of total available space.

Solution Hardware Components

All recommended architectures in the solution feature common hardware components, as shown in Table 2.

TABLE 2. SOLUTION COMMON HARDWARE COMPONENTS

Component Description

Storage Area Network Brocade switches

Administrative Servers -based systems or SPARC servers

Media Servers x86-based systems or SPARC servers

Tape Libraries StorageTek SL150, SL3000, and SL8500 tape libraries


Tape Drives Oracle s StorageTek T10000D or LTO 6 tape drives

StorageTek ACSLS A single Oracle server running the StorageTek ACSLS software

Oracle Key Manager Oracle Key Manager appliances

StorageTek Tape Analytics software A single Oracle server running the StorageTek Tape Analytics software

Oracle ZFS Storage Appliance Backup and recovery solution for Oracle engineered systems in customizable small, medium, and large configurations

Exadata Storage Expansion Rack Quarter-, half-, or full-rack configurations that add storage capacity to an Oracle Exadata Database Machine or Oracle SuperCluster

Network Infrastructure

Ethernet switches provide connections and link aggregation between the backup clients, media servers, and administrative servers. Network speeds of both 1 gigabit per second and 10 gigabits per second are supported. Connections from the media servers to the network clients are provided through a private 10 gigabit Ethernet (GbE) network or connections into the solution's InfiniBand fabric.

Built-in Oracle InfiniBand switches are used to provide connectivity within Oracle engineered systems. InfiniBand eliminates the physical complexity of multiple interconnects per system and provides immense bandwidth and high-speed connectivity to support high-speed transfers of massive amounts of data. With high throughput, low latency, and a scalable fabric suitable for fabric consolidation of interprocess communication, network, and storage, InfiniBand delivers up to 63 percent higher transactions per second for Oracle Real Application Clusters (Oracle RAC) than GbE networks.

SAN switches provide Fibre Channel connections between the Oracle Secure Backup media servers and the Fibre Channel archive devices.

Oracle's SPARC Servers

The solution features two members of Oracle's SPARC server family SPARC T7-1 and M10-1 servers (Figure 6). These servers are designed to provide breakthrough performance while maximizing reliability and minimizing power consumption and complexity. The SPARC T7-1 server features a single SPARC M7 processor (32 cores) and up to 512 GB of system memory in a 2U rackmount enclosure. The SPARC M10-1 server, a compact entry-level server, features processor (8 or 16 cores) and up to 512 GB of system memory in a space-saving 1U rackmount enclosure.

SPARC T7-1 and SPARC M10-1 servers are ideal for large-scale applications, enterprise-wide consolidation, and database projects that require extreme reliability, availability, and security.

Figure 6. SPARC T7-1 and Fujitsu M10-1 servers provide high levels of performance and throughput.


Oracle's x86-Based Servers

The solution also features members of Oracle's x86-based server family: Oracle Server X5-2 and Oracle Server X5-2L servers (Figure 7). Oracle's comprehensive, open standards based x86 systems provide the best platform to run Oracle software when x86 architectures are required, offering enhanced reliability for data center environments. Only Oracle provides an optimized hardware and software stack that comes complete with choice of OS, virtualization software, and cloud management tools all at no extra charge. Oracle's optimized hardware and software stack has enabled enormous performance gains in its engineered systems and delivered world-record benchmarks results.

Oracle Server X5-2 features up to two Intel Xeon processor E5-2600 v3 product family CPUs and up to 768 GB memory in a compact 1U enclosure. Oracle Server X5-2L offers the same processors and memory capacity, and adds expandability with support for up to 26 disk drives in a 2U enclosure.

Figure 7. Oracle x86-based servers provide high performance and reliability.

Oracle ZFS Storage Appliance

Oracle ZFS Storage Appliance (Figure 8) provides a flexible, high-performance, and low-cost backup and recovery solution for Oracle engineered systems. Offering unmatched Oracle integration, high performance, efficiency, simplified management, and low TCO, Oracle ZFS Storage Appliances are available in customizable small, medium, and large configurations to suit varied backup requirements.

Figure 8. Oracle ZFS Storage Appliance provides a flexible, low-cost backup and recovery option for Oracle engineered systems.


Zero Data Loss Recovery Appliance

Figure 9) is an engineered system for database backup that eliminates data loss exposure without impacting the performance of production environments. Compute, network, and storage are integrated into a massively scalable appliance with a cloud-scale architecture that provides fully automated database backup and recovery for multiple databases. The recovery appliance enables a centralized backup strategy for large numbers of databases using cloud-scale, fault-tolerant hardware and storage.

Integrated with Oracle RMAN and featuring an incremental-forever backup strategy, the recovery appliance provides minimal-impact backups. The databases send only changes, and all backup and tape processing is offloaded from the production servers to the recovery appliance for improved system performance. Real-time database redo block information is transmitted, eliminating potential data loss and providing instant protection for new transactions. Database recoverability is improved with end-to-end reliability, visibility, and control of a database as a whole, rather than as a disjoint set of files.

The recovery appliance features secure replication to help protect against disasters such as site outages or regional

replication can be set up in a simple one-way topology, or two recovery appliances can be set up to replicate each other, or a central recovery appliance can be used for replication from multiple satellite recovery appliances. In all topologies, only changed blocks are replicated, minimizing WAN network usage.

Use of secure replication to a recovery appliance can help speed recovery times in the event of an outage. If a local recovery appliance is not available, restore operations can run directly from a remote recovery appliance without first staging the data locally.

Figure 9 ata Loss Recovery Appliance eliminates data loss and provides minimal-impact backups.

Oracle's StorageTek Tape Library Products Oracle tape storage solutions maximize the bandwidth of Oracle Optimized Solution for Secure Backup and Recovery. Providing fault isolation from production systems storage, the tape storage solutions enable easy movement of backup media off-site and provide cost-effective retention of multiple backup copies. Tape media can also be the most cost-effective method for the initial movement of large amounts of data. In the event that a large database needs to be moved from site to site, such as to or from a secondary disaster recovery (DR) site, tapes can


be shipped across the country quickly and cheaply, and subsequent incremental backups can be applied to make the database current once again.

Oracle Optimized Solution for Secure Backup and Recovery features Oracle's StorageTek SL150, StorageTek SL3000, and StorageTek SL8500 modular library systems (Figure 10). These reliable modular library systems scale to suit a wide range of storage capacity requirements. The solution also features Oracle's StorageTek T10000D and StorageTek LTO 6 tape drives (Figure 11), where applicable. All the recommended architectures include the StorageTek SL3000 tape library, because this library offers the lowest-cost configuration with no single point of failure. The tape libraries and tape drives are described below.

» StorageTek SL150 modular tape library. The StorageTek SL150 tape library scales from 30 to 300 LTO slots with a maximum capacity of more than 750 terabytes of uncompressed data (using StorageTek LTO 6 drives) in a standard 19-inch rack-mounted cabinet. The library also supports up to 20 tape drives with a native throughput

designed for small and growing or mid-sized business environments.

» StorageTek SL3000 modular library system. The StorageTek SL3000 modular library system offers an innovative, eco-efficient approach to midrange storage, providing more choice and control for rapidly changing environments. The StorageTek SL3000 modular library system scales from 200 to 5,9251 cartridge slots and from 1 to 56 tape drives in a footprint that provides linear growth in a rack environment. Its RealTime Growth capability enables physical capacity to be installed in advance and tapped into incrementally with Capacity on Demand license keys. The StorageTek SL3000 modular library system delivers a native throughput rate of 48.4 TB per hour for maximum configurations.

» StorageTek SL8500 modular library system. The StorageTek SL8500 modular library system can be shared across supercomputer, mainframe, Oracle Solaris, AS/400, Windows, and Linux environments, and across a variety of other UNIX environments. Its Any Cartridge Any Slot technology enables any combination of supported drives and media. The choices include StorageTek T10000 and StorageTek T9840 tape drives, as well as LTO and SDLT tape drives. The StorageTek SL8500 modular library system has nearly limitless scalability, and can currently scale to 100,880 slots (554.8 PB native storage) and up to 640 tape drives. Many libraries can be consolidated onto a single StorageTek SL8500 modular library system, with modular and scalable capacity to meet rapid growth requirements. The StorageTek SL8500 modular library system delivers a native throughput rate of 580.6 TB per hour for maximum configurations.

1 Although the StorageTek SL3000 modular library system supports 5,925 cartridge slots, Oracle recommends moving to a StorageTek

SL8500 modular library system if capacity needs require over 3000 slots.


Figure 10. Oracle's StorageTek modular library systems come in different sizes to accommodate data center needs.

Oracle's StorageTek Tape Drives

Oracle Optimized Solution for Secure Backup and Recovery features Oracle's StorageTek T10000D and LTO6 tape drives, shown in Figure 11.

» StorageTekT10000D tape drive. The StorageTek T10000D tape drive delivers a capacity of 8 TB of uncompressed data and a throughput of 250 MB per second. These drive efficiencies help enable data centers to store more data in the same footprint and shorten backup and recovery windows. StorageTek T10000D tape drives support demanding, high-duty-cycle environments with dual-head, 32-channel technology, reducing the number of tape passes and extending head and media life. These tape drives connect using dual-port Fibre Channel options for compatibility with most storage environments.

» StorageTek LTO 6 tape drive. The StorageTek LTO 6 tape drive is Oracle's latest addition to its portfolio of tape drives. With a capacity of 2.5 TB per second and a throughput rate of 160 MB per second, the StorageTek LTO 6

It comes in full- and half-height platforms, as single drives, or as part of an automation environment. IT departments can choose the model that makes the most sense for the operating environment, application, capacity, and performance. Oracle's scalable tape automation solutions are designed to accommodate whatever drive and interface are selected.

Figure 11. StorageTek tape drives provide high-throughput capabilities.

Reference Architecture Components in Tape Environments

In environments using tape for backups, the primary logical components of Oracle Optimized Solution for Secure Backup and Recovery include software that fulfills the following three roles. The software used can be Oracle Secure Backup or other third-party backup software selected by IT staff.


» Backup software administrative server. Each administrative domain must have exactly one administrative server. The administrative server is configured with complete data for the other hosts in the domain, their roles, and their attached tape devices. This data is maintained in a set of configuration files stored on the server. The administrative server runs the scheduler, which starts and monitors each backup job. The scheduler keeps a backup catalog with metadata for all backup and restore operations performed in the administrative domain. Orac x86-based servers or SPARC servers are recommended for use as administrative servers in this solution because they deliver outstanding performance in the smallest possible configuration while providing considerable room for growth. For optimal performance and availability, the administrative and media server roles should not be mixed within the same server.

» Backup software media server. A media server is a host with at least one tape drive or library attached to it. The media server transfers data to or from a volume loaded on one of the attached tape devices. Single media servers can be attached to multiple tape libraries, and multiple media servers can share attachments to multiple tape libraries. The media server performs the backup operations under the direction of the administrative server, storing all backup data on its connected storage devices. x86-based servers or SPARC servers are recommended as media servers in this solution because they each offer the most I/O bandwidth for their platform type, a feature that dramatically improves media server performance. Both server families also provide a large amount of capacity for most environments. For optimal performance and availability, the administrative and media server roles should not be mixed within the same server.

» Backup clients. While this solution focuses on backup and recovery for Oracle engineered systems and Oracle Optimized Solutions, it can also be used for network backups of any client platform supported by Oracle Secure Backup. Although this solution includes multiple backup administrative and media servers, a single server can be deployed to support both the administrative and media server workloads, if desired. Using two systems provides high availability in a highly scalable solution, but the workloads can be combined on a single server. Additional media servers and archive devices can be added as needed to address scalability requirements.

Oracle Secure Backup

Oracle Secure Backup provides centralized backup management for Oracle Database, heterogeneous file systems, and Network Attached Storage (NAS) to disk and/or tape. Integrated with components of the Oracle technology stack and optimized for Oracle Database backup, this scalable end-to-end backup solution eliminates multivendor integration and support issues. With centralized, policy-based backup management and low-cost, single-component licensing, this software reduces cost and complexity for reliable data protection across complex, heterogeneous enterprise environments.

Oracle Secure Backup uses a central administrative server and one or more media servers to provide data -based or SPARC servers, and

using Oracle tape archive devices and Oracle ZFS Storage Appliance, this software scales to support large enterprise networks. The networks can contain a large number of clients (UNIX, Linux, or Windows), NAS devices, or high-performance engineered systems such as Oracle Exadata, Oracle Database Appliance, and Oracle SuperCluster. Data backup to both tapes and disk (new in Oracle Secure Backup 12.1) is supported. Oracle Secure Backup uses standard tape formats and supports most popular tape drives and tape libraries across SAN, Ethernet networks, Oracle InfiniBand networks, and SCSI environments. Dynamic tape-drive sharing ensures maximum utilization of tape drives.

Built-in integration with Oracle products provides significant performance benefits for these products. Integration with Oracle RMAN helps deliver optimized performance and achieve 25 40 percent faster backups for Oracle Database than comparable media management utilities with up to 10 percent less CPU utilization. In Oracle Exadata environments, optimized backup performance and support for Reliable Datagram Socket over Remote Direct Memory Access (RDS/RDMA) can deliver approximately 50 percent more throughput per InfiniBand port. And tight


integration with Oracle ZFS Storage Appliance and support for NDMP connectivity eliminates the need for an additional media server to write backups to disk.

Encrypted backups secure data at rest, on tape, or on disk. Backups can be encrypted using host-based backup

protects data while in transport and provides two-way server authentication for greater protection.

x86-based or SPARC servers are used to create the optimal backup and recovery solution that best fits an -in-class network backup

performance and best-in-class reliability. SPARC servers from Oracle can deliver up to 20 percent better network performance than the x86 server platforms. However, both types of platforms have similar bandwidth offerings in terms of the number of expansion slots. The recommended components in the solution, including servers, SAN, Ethernet and InfiniBand networks, tape library management, tape encryption management, and tape drives, are configured for redundancy. To facilitate interoperability and deliver more stability, the same platform found in

high-performing appliance solutions, such as Oracle engineered systems and Oracle Optimized Solutions, should be deployed wherever possible.

Cloud-Based Database Backups

Oracle Optimized Solution for Secure Backup and Recovery supports cloud-based database backups. Cloud-based backups can serve as a complement to or a replacement for local backup and recovery solutions. By using a cloud-based backup, an enterprise gains scalable capacity on demand and eliminates up-front capital investment in storage hardware.

Oracle Optimized Solution for Secure Backup and Recovery can be deployed as part of a private cloud solution, for enterprises requiring a dedicated internal private cloud implementation. Alternatively, the solution can be deployed as part of a hybrid cloud solution, with production databases maintained on premises and cloud storage used for database backups.

Two methods are supported when deploying Oracle Optimized Solution for Secure Backup and Recovery as part of a hybrid cloud:

» Oracle Database Backup Service. Oracle Database Backup Service is a secure, scalable, reliable, and high-performance public cloud storage solution for storing Oracle Database backups. A client-side Oracle Database Backup Module is used with Oracle RMAN to transparently handle the backup and restore operations.

» Oracle Secure Backup to Amazon Simple Storage Service (Amazon S3). The Oracle Secure Backup Cloud Module, compatible with Oracle Database version 9i Release 2 or later, can be used to back up databases to cloud-based storage services offered by Amazon S3. Oracle Secure Backup Cloud Module is implemented using the Oracle RMAN System Backup to Tape (SBT) interface, which enables external backup libraries to be seamlessly integrated with Oracle RMAN. With this cloud offering, local disk backups are sent directly to Amazon S3 for offsite storage and are fully integrated with Oracle RMAN.

For more information on Oracle Database Backup Service, see cloud.oracle.com/database_backup.

Backup Best Practices While in-depth coverage of backup best practices for each configuration of the solution is out of the scope of this document, some recommended best practices are discussed in the following sections, along with information on where to find thorough best practices for each Oracle solution.

https://cloud.oracle.com/database_backup


Best Practices for Accomplishing Recovery Time and Recovery Point Objectives Oracle offers a complete technology stack composed of hardware and software. Tightly integrated, the multiple layers of the stack contain tools for data protection and backup and recovery. Important tools can be found at the application, database, operating system, virtualization, and storage levels, as shown in Figure 12. Following best practices and combining these tools effectively can result in an optimal solution for backing up Oracle systems. In fact, by not relying on any single tool, it is possible to create an ideal data protection environment that reduces reliance on conventional backups.

Figure 12. Oracle offers data protection tools at various levels of the integrated stack.

Among the tools from each layer are the following:

» Application tools: Oracle Secure Backup and third-party backup software » Database tools: Oracle Archive Log Mode, Data Guard, Oracle Flashback Database, Data Recovery Advisor,

and Oracle RMAN » Operating system tools: Built-in backup and recovery tools, ZFS snapshots, and alternate boot environments (a

boot environment is an Oracle Solaris operating system tool). » Storage tools: Exadata Storage Expansion Racks, Oracle ZFS Storage Appliances, and StorageTek Tape

storage

Using a Combination of Software Tools

The tools within the different layers of Oracle's stack can be used to help achieve diverse and precise recovery time and recovery point objectives. By using various tools listed below, it is possible to protect valuable enterprise data and production environments without relying on conventional and lengthier backup and restore operations.

» Oracle Flashback Database and Data Guard can help IT staff recover from logical errors and database outages for recovery time objectives of 15 minutes to an hour.

» Oracle Archive Log Mode enables the database to be rolled back to any point in time within the recovery window to meet specific recovery point objectives.

» Oracle Enterprise Manager Ops Center enables administrators to deploy Oracle Solaris and Oracle Linux operating systems for fast operating system recovery.

» The default file system in Oracle Solaris 11 is ZFS, which supports snapshots, cloning, and replication. ZFS provides the ability at the operating system level to utilize built-in commands for snapshots, cloning, and replication.


» Oracle ZFS Storage Appliance supports snapshots, cloning, and replication for specific recovery points. Administrators can take a snapshot of an iSCSI LUN on Oracle ZFS Storage Appliance to create a backup. Alternatively, deploying a remote Oracle ZFS Storage Appliance enables recovery in seconds by switching from the primary to the remote appliance.

Best Practices for Backup and Recovery in Virtualized Environments or with Unstructured Data

When performing backup and recovery in virtualized environments or in environments with unstructured (non-database) data, traditional approaches consist of taking disks and applications offline and writing from disk to tape. In the real world, however, organizations require 24x7 availability and cannot afford to take applications offline at all. Oracle Solaris 11 uses Oracle Solaris ZFS (ZFS) as the default file system and ZFS provides features for capturing a system snapshot that can be sent to an external Oracle ZFS Storage Appliance or stored on tape.

Oracle Optimized Solution for Secure Backup and Recovery is configured with an internal Oracle ZFS Storage Appliance that can store the virtualized environments and file system snapshots. Oracle recommends the use of these snapshots to maintain application availability while obtaining a copy of vital data.

Best Practices for Backup and Recovery in Virtualized Environments and Unstructured Data with NDMP

NDMP is an open standard protocol for network-based backup of network-attached storage (NAS) devices. It facilitates interoperation of backup software and NAS hardware. Because NAS devices are not intended to host applications such as backup software agents and clients, they cannot be set up as backup clients for software such as Oracle Secure Backup. Instead, an NDMP service is created on the NAS device, using NDMP host types to designate Oracle ZFS Storage Appliance as the backup target. These host types require the use of specific path names. Please refer to My Oracle Support Note 1558851.1 for examples.

Best Practices for a Secure Backup and Recovery Implementation Backup and recovery systems cannot rely solely on perimeter security. A combination of system-wide security measures and best practices including the rule of least privilege, strong authentication, access control, encryption, auditing, disabling of unnecessary services, antimalware protections, and configuring system services for enhanced security should also be implemented for secure operations.

Oracle highly recommends leveraging existing recommendations and guidelines from product security guides, Center for Internet Security (CIS) benchmarks, ISACA publications, and Department of Defense (DoD) Security Technical Implementation Guides (STIGs) when designing a backup and recovery environment.

Security Technical Implementation Guides

STIGs are continually updated and currently available for many Oracle products. A list of STIGs relevant to this solution is shown in Table 3.

TABLE 3. EXAMPLES OF RELEVANT STIGS

STIG Location

Oracle Solaris iase.disa.mil/stigs/os/unix-linux/Pages/solaris.aspx

Oracle Database 11g Release 2 iasecontent.disa.mil/stigs/zip/Apr2015/U_Oracle_Database_11-2g_V1R3_STIG.zip

Oracle Integrated Lights Out Manager iase.disa.mil/stigs/app-security/database/Pages/exadata_lights.aspx

Oracle Exadata Storage Server iase.disa.mil/stigs/app-security/database/Pages/exadata_storage.aspx

http://iase.disa.mil/stigs/os/unix-linux/Pages/solaris.aspx

http://iasecontent.disa.mil/stigs/zip/Apr2015/U_Oracle_Database_11-2g_V1R3_STIG.zip

http://iasecontent.disa.mil/stigs/zip/Apr2015/U_Oracle_Database_11-2g_V1R3_STIG.zip

http://iase.disa.mil/stigs/app-security/database/Pages/exadata_lights.aspx

http://iase.disa.mil/stigs/app-security/database/Pages/exadata_storage.aspx


Oracle Sun Datacenter InfiniBand Switch 36 iase.disa.mil/stigs/app-security/database/Pages/exadata_infiniband.aspx

Oracle ZFS Storage Appliance iase.disa.mil/stigs/app-security/database/Pages/exadata_zfs.aspx

Oracle WebLogic Server 12c iase.disa.mil/stigs/Documents/u_oracle_weblogic_server_12c_v1r1_stig.zip

DoD Secure Telecommunications iase.disa.mil/stigs/net_perimeter/telecommunications/Pages/index.aspx

Oracle Linux 6 Manual STIG iasecontent.disa.mil/stigs/zip/Apr2015/U_Oracle_Linux_6_V1R2_STIG.zip

Storage Area Network (SAN) iase.disa.mil/stigs/Documents/u_storage_area_network_v2r2_stig.zip

For more STIGs, please see the website iase.disa.mil/stigs/Pages/index.aspx.

Component-Level Security Recommendations

Oracle recommends the following component-level security guidelines.

» Change system default passwords. Using known vendor-provided default passwords is a common way cyber criminals gain unauthorized access to infrastructure components. Changing all default passwords to stronger, custom passwords is a mandatory step during infrastructure deployment.

» Keep component patching current. Ensure that all components are using the most recent firmware and software versions to the extent possible. This tactic ensures that each component is protected by the latest security patches and vulnerability fixes.

» Leverage isolated, purpose-based network interfaces. Network interfaces, virtual or physical, should be used to separate architectural tiers, such as client access and management. In addition, consider using network interfaces to separate tiers within a multitier architecture. This enables per-tier security policy monitoring and enforcement mechanisms including network, application, and database firewalls as well as intrusion detection and prevention systems.

» Enable encrypted network communications. Ensure all endpoints use encrypted network-based communications, including secure protocols, algorithms, and key lengths. For Oracle WebLogic, use the UCrypto provider to ensure that cryptography leverages the hardware assist capabilities of the SPARC platform.

» Enable encrypted data-at-rest protections. » Use encrypted swap, /tmp, and ZFS datasets for any locations that could potentially house sensitive or

regulated data. This automatically takes advantage of cryptographic acceleration in Oracle Solaris. » Use tape drive encryption to protect data that must leave the data center for off-site storage. » For databases, use Transparent Data Encryption (TDE) to protect tablespaces that might store sensitive or

regulated data. TDE automatically takes advantage of cryptographic acceleration in Oracle Solaris on SPARC systems.

» Secure the database. Refer to Oracle Optimized Solution for Secure Oracle Database security best practices and recommendations.

» Deploy application services in Oracle Solaris non-global zones. Deploying applications within Oracle Solaris non-global zones has several security advantages, such as kernel root kit prevention, prevention of direct memory and device access, and improved control over security configuration (via zonecfg(1M)). This approach also enables higher assurance auditing, because audit data is not stored in the Oracle Solaris non-global zone, but rather in the Oracle Solaris global zone.

» Implement a baseline auditing policy. Use audit logs and reports to track user activity including individual transactions and changes to the system and to flag events that fall out of normal parameters. These should be implemented at both the Oracle Solaris and database levels. The baseline security audit policy should include login/logout activity, administrative actions, and security actions, as well as specific command executions for Oracle Solaris. This tactic enables auditing of a core set of security critical actions without overburdening the system or database.

» Follow the rule of least privilege. Increase access control by granting only those privileges that a given individual needs. This should be implemented at both the ERP system level and the infrastructure level.

http://iase.disa.mil/stigs/app-security/database/Pages/exadata_infiniband.aspx

http://iase.disa.mil/stigs/app-security/database/Pages/exadata_zfs.aspx

http://iase.disa.mil/stigs/Documents/u_oracle_weblogic_server_12c_v1r1_stig.zip

http://iase.disa.mil/stigs/net_perimeter/telecommunications/Pages/index.aspx

http://iasecontent.disa.mil/stigs/zip/Apr2015/U_Oracle_Linux_6_V1R2_STIG.zip

http://iase.disa.mil/stigs/Documents/u_storage_area_network_v2r2_stig.zip

http://iase.disa.mil/stigs/Pages/index.aspx


» Use strong authentication. Many intellectual property attacks use stolen credentials. Implementing strong authentication methods, such as Kerberos, RADIUS, and SSL, can help prevent unauthorized access.

» Leverage role-based access control. As the number of applications and users increases, user-based identity management can quickly become time consuming and labor intensive for IT staff. Consequentially, many users are granted inappropriate authorities. Though it requires increased efforts during the design and implementation phases, role-based access control (RBAC) is a popular option for low-maintenance, scalable access control, and it can help alleviate the burden of identity management.

Table 4 lists Oracle SuperCluster security recommendations. A full list of relevant component security recommendations is shown in Table 5.

TABLE 4. ORACLE SUPERCLUSTER SECURITY RECOMMENDATIONS

Title Location

Best Practices for Securely Deploying the SPARC SuperCluster T4-4

oracle.com/technetwork/articles/servers-storage-admin/supercluster-security-1723872.html

SPARC SuperCluster T4-4 Platform Security Principles and Capabilities

oracle.com/us/products/servers-storage/servers/sparc-enterprise/supercluster/supercluster-t4-4/ssc-security-pac-1716580.pdf

Oracle SuperCluster T5-8 Security Technical Implementation Guide (STIG) Validation and Best Practices on the Database Servers

oracle.com/technetwork/server-storage/hardware-solutions/stig-sparc-supercluster-1841833.pdf

Secure Database Consolidation Using the Oracle SuperCluster T5-8 Platform

oracle.com/technetwork/server-storage/sun-sparc-enterprise/documentation/o13-053-securedb-osc-t5-8-1990064.pdf

TABLE 5. EXAMPLES OF COMPONENT SECURITY RECOMMENDATIONS

Resource Location

Oracle Solaris 11 Security Guidelines docs.oracle.com/cd/E36784_01/html/E36837/index.html

Oracle Solaris 11.2 Security Compliance Guide docs.oracle.com/cd/E36784_01/pdf/E39067.pdf

oracle.com/technetwork/database/database-technologies/secure-backup/learnmore/osb-103-encryption-335238.pdf

Secure Deployment of Oracle VM Server for SPARC

oracle.com/technetwork/articles/systems-hardware-architecture/secure-ovm-sparc-deployment-294062.pdf

Oracle Solaris Cluster Security Guide docs.oracle.com/cd/E39579_01/html/E39649/index.html

User Authentication on the Solaris OS: Part 1 oracle.com/technetwork/server-storage/solaris/user-auth-solaris1-138094.html

Oracle ILOM Security Guide docs.oracle.com/cd/E37444_01/html/E37451/index.html

Database Advanced Security Administrator's Guide

docs.oracle.com/cd/E11882_01/network.112/e40393/toc.htm

Oracle Database 12c Security and Compliance oracle.com/technetwork/database/security/security-compliance-wp-12c-1896112.pdf

Best Practices for Deploying Encryption and Managing Its Keys on the Oracle ZFS Storage Appliance

oracle.com/technetwork/server-storage/sun-unified-storage/documentation/encryption-keymgr-1126-2373254.pdf

Securing the Network in Oracle Solaris 11.2 docs.oracle.com/cd/E36784_01/html/E36838/index.html

Securing Users and Processes in Oracle Solaris 11.2

docs.oracle.com/cd/E36784_01/html/E37123/index.html

http://oracle.com/technetwork/articles/servers-storage-admin/supercluster-security-1723872.html

http://oracle.com/technetwork/articles/servers-storage-admin/supercluster-security-1723872.html

http://oracle.com/us/products/servers-storage/servers/sparc-enterprise/supercluster/supercluster-t4-4/ssc-security-pac-1716580.pdf

http://oracle.com/us/products/servers-storage/servers/sparc-enterprise/supercluster/supercluster-t4-4/ssc-security-pac-1716580.pdf

http://oracle.com/technetwork/server-storage/hardware-solutions/stig-sparc-supercluster-1841833.pdf

http://oracle.com/technetwork/server-storage/hardware-solutions/stig-sparc-supercluster-1841833.pdf

http://oracle.com/technetwork/server-storage/sun-sparc-enterprise/documentation/o13-053-securedb-osc-t5-8-1990064.pdf

http://oracle.com/technetwork/server-storage/sun-sparc-enterprise/documentation/o13-053-securedb-osc-t5-8-1990064.pdf

http://docs.oracle.com/cd/E36784_01/html/E36837/index.html

http://docs.oracle.com/cd/E36784_01/pdf/E39067.pdf

http://www.oracle.com/technetwork/database/database-technologies/secure-backup/learnmore/osb-103-encryption-335238.pdf

http://www.oracle.com/technetwork/database/database-technologies/secure-backup/learnmore/osb-103-encryption-335238.pdf

http://oracle.com/technetwork/articles/systems-hardware-architecture/secure-ovm-sparc-deployment-294062.pdf

http://oracle.com/technetwork/articles/systems-hardware-architecture/secure-ovm-sparc-deployment-294062.pdf


http://oracle.com/technetwork/server-storage/solaris/user-auth-solaris1-138094.html


http://docs.oracle.com/cd/E11882_01/network.112/e40393/toc.htm

http://oracle.com/technetwork/database/security/security-compliance-wp-12c-1896112.pdf

http://oracle.com/technetwork/server-storage/sun-unified-storage/documentation/encryption-keymgr-1126-2373254.pdf

http://oracle.com/technetwork/server-storage/sun-unified-storage/documentation/encryption-keymgr-1126-2373254.pdf




Securing Systems and Attached Devices in Oracle Solaris 11.2


Securing Files and Verifying File Integrity in Oracle Solaris 11.2


Managing Encryption and Certificates in Oracle Solaris 11.2


Developer's Guide to Oracle Solaris 11 Security docs.oracle.com/cd/E36784_01/html/E36855/index.html

Configuring Oracle GoldenGate Security docs.oracle.com/goldengate/1212/gg-winux/GWUAD/wu_security.htm#GWUAD354

docs.oracle.com/cd/E26569_01/doc.104/e21477/network_security.htm#OBINS277

Best Practices for Backup and Recovery of Operating Systems Operating system backups are critical and should be made before and after every significant change to the Oracle SuperCluster software. This includes performing backups before and after completing the following procedures:

» Installation or reconfiguration of significant Oracle or non-Oracle software » Reconfiguration of significant operating parameters » Before and after each quarterly full stack download patch (QFSDP) for Oracle SuperCluster

Oracle Solaris 11

The conventional method for backing up an operating system is to install backup and recovery software and perform backups of the operating system to disk or tape, potentially an inefficient approach in 24x7 environments. In Oracle Solaris 10 and Oracle Solaris 11, the ZFS file system enables bare-metal recovery of the operating system using ZFS snapshots, the ZFS commands zfs send and zfs receive, and ZFS stream files. A ZFS stream file is an image of an entire ZFS file system placed into a file on an NFS share. The zfs send command creates a stream representation of a ZFS snapshot that is written either to a file or to a different system using standard output. The zfs receive command creates a snapshot whose contents are specified in the stream that is provided on standard input. If a full stream is received, a new file system is created as well. This functionality enables administrators to send and receive ZFS snapshot data and file systems using these commands.

Here is a high-level outline of the steps used to perform a backup and bare-metal recovery of the operating system. For details and commands, please see the Oracle SuperCluster Administrator's Guide, or My Oracle Support Note 1558851.1.

To perform a backup, do the following:

» To start, create an NFS share on an Oracle ZFS Storage Appliance on which to store the snapshots. » Take a ZFS snapshot of the operating system and send it to a stream file using the zfs send command.

To perform a restore of the snapshot, boot from the Oracle Solaris boot media, as follows:

» Create a new root pool, or rpool (a ZFS storage pool that contains a bootable ZFS root file system), on the replacement disks.

» Mount the NFS share. » To restore, issue a cat command on the stream file, piping it to the newly created pool name on the local disk

with the zfs receive command.

This procedure restores the zfs send file from the ZFS stream file back to the new ZFS pool. Using zfs send and zfs receive is the preferred method for restoring a bare-metal operating system. The previously





http://docs.oracle.com/goldengate/1212/gg-winux/GWUAD/wu_security.htm#GWUAD354

http://docs.oracle.com/cd/E26569_01/doc.104/e21477/network_security.htm#OBINS277


recommended method using a deployment system takes approximately eight hours contrasted with 45 minutes when using the ZFS stream file method.

Backing Up Domains

The physical resources within Oracle servers can be virtualized for greater agility and utilization. Two different methods can be used for virtualization. The first method creates domains, partitioning the hardware to use specified physical resources and then physically or logically isolating one domain from another. The second virtualization method uses Oracle Solaris Zones, where each zone is a virtualized operating system environment created within a single instance of Oracle Solaris. Implementing zones enables the installation of multiple versions of Oracle Solaris 10 and Oracle Solaris 11 on a single system.

Various domain configurations and implementations are supported on Oracle servers. Domain configurations can have different layouts due to the ability to install the domains on different disks. Disk resources can be allocated and shared among domains, while some domains might not own any disks. Disks can be shared from other file systems and appear to users as local hard drives. Single domain systems are typically fairly simple, as are two-domain configurations.

When backing up domains, the backup methods are the same as for the operating system. The restoration procedures vary, depending on what devices are installed with the domains, what disks are designated as targets, and where the rpool is created to receive the restore.

Domains also can be backed up with block-level backups using NDMP and an Oracle ZFS Storage Appliance. Note, however, that this method is limited to backing up entire data sets, and does not offer the ability to restore single files. Single-file backup and recovery can be accomplished by installing Oracle Secure Backup in a zone to back up the zone as a regular backup and recovery client.

Backing Up Oracle Solaris Zones

Backup and recovery of applications is supported in NFS, and backup and recovery procedures remain the same for snapshots and replication of NFS shares, as well as for tape using NDMP. To back up NFS shares, use Oracle ZFS Storage Appliance with NDMP. NDMP, sometimes referred to as dump mode, provides high-performance, block-level backups. Note, however, that the ZFS NDMP functionality does not permit restoration of individual files, so in order to restore a single file, the entire data set must be restored. For more information on using NDMP with Oracle ZFS Storage Appliance NDMP Implementation Guide for the Sun ZFS

Oracle Solaris Zones typically contain applications and databases for different use cases. Oracle Solaris Zones can be installed on internal server drives or on external iSCSI LUNs.

Zones stored on iSCSI LUNs cannot be backed up at the dump level with Oracle ZFS Storage Appliance they must be backed up in ZFS block mode. This is a high-performance method that works well for restoring the entire zone. The ZFS block mode method does not support restoration of single files or directories; the entire data set must be restored. Single-file backup and recovery can be accomplished by installing Oracle Secure Backup software in a zone to back up the zone as a regular backup and recovery client.

For disk-only environments, zones can be backed up through the creation of snapshots, clones, or replication to another Oracle ZFS Storage Appliance.


Best Practices for Database Backups (Oracle Database 11g Release 2 or Higher) The following sections outline database backup best practices designed for high availability, optimal performance, and maximum data protection. These best practices should be followed for instances of Oracle Database running on Oracle SuperCluster, Oracle Exadata, or Oracle Optimized Solutions.

Oracle Optimized Solution for Secure Backup and Recovery provides a choice of tape-based and disk-based backup methods for protecting databases. This section includes general best practices for database backup as well as recommendations for both disk-based and tape-based database backups.

For disk-based backups, this solution includes both Oracle ZFS Storage Appliance and Zero Data Loss Recovery Appliance. The different configurations are designed to meet a variety of backup solution requirements. In general, Zero Data Loss Recovery Appliance provides the highest performance and serviceability, while Oracle ZFS Storage Appliance provides flexibility and a lower-cost option.

Recommended Database Backup Schedule

Oracle recommends the following schedule for backups of Oracle Database:

» Perform weekly Oracle RMAN level 0 (full) backups of the database. » Perform daily cumulative Oracle RMAN incremental level 1 backups of the database and use block change

tracking. » Roll incremental backups into a full backup and delay by 24 hours. » Perform daily backups of the Oracle RMAN catalogs.

Always Use a Fast Recovery Area

One vital database backup and recovery best practice is to deploy the database with a Fast Recovery Area (FRA), the location where the database stores all of its information for Oracle Flashback technologies, snapshots, database backups, rollbacks, and more. The FRA enables the database to protect itself, by enabling the database to be rolled back to a certain point in time and restored within a matter of seconds instead of minutes or hours using conventional backup and recovery methods. Consequently, database recovery time is limited by the speed of the device on which the FRA resides, and it should be located on the fastest performing storage possible. Oracle engineered systems are designed with the FRA built in, because placing the FRA on anything but Oracle Exadata internal storage or an external Exadata Storage Expansion Rack will lengthen the amount of time it takes for recovery. Exadata Storage Expansion Racks can be used for extra space for the FRA, as needed.

Database backups can, if desired, be excluded from the FRA and stored instead on Oracle ZFS Storage Appliance. In this case, the FRA on Oracle Exadata can be configured much smaller, because space for the full and incremental backups is not required in the FRA. This configuration can provide cost savings, because lower-cost storage can be used for database backup, without sacrificing system performance.

Additional Database Configuration Best Practices

There are other configuration parameters for Oracle Database that must be set in order to ensure optimal backup performance:

» Set the initialization parameter _file_size_increase_increment=2143289344 to optimize the space used when incremental (level 1) backups are taken on the FRA.

» Reset or remove the initialization parameters _backup_ksfq_bufsz and _backup_ksfq_bufcnt on systems running Oracle Database release 11.2.0.2 or later releases.


» Use a backup Oracle Net service to run against all database servers in the cluster for better performance and high availability. The Oracle RMAN BACKUP command uses the service to automatically spread the backup load evenly among target the instances offering the service.

» Set DB_RECOVERY_FILE_DEST_SIZE to bound space in the FRA. It is important that the value of this parameter be set to less than the total free space in the disk group, which must take account of at least one disk failure and preferably one Oracle Exadata Storage Server failure. If multiple databases are sharing the FRA, ensure that the sum of the space allocated to the different databases is less than the free space in the disk group.

» Consider using Oracle Database Resource Manager to manage system resources, particularly I/O, on Oracle SuperCluster.

Oracle RMAN Configuration Best Practices » Parallelize backups across all database nodes, allowing all the disks, network connections, and system CPUs to

be leveraged for increased performance. Follow configuration instructions according to whether the backup targets are disks or tapes.

» Use Oracle RMAN backup scripts to automate weekly and daily backups. » Use two to eight Oracle RMAN channels per instance when performing backups to disks. When performing

backups to tape, configure one Oracle RMAN channel per tape drive and add tape drives to scale backup rates. » Use Oracle RMAN incremental backups and block change tracking.

» To reduce backup time and resources, perform nightly incremental backups to the FRA and merge them into the image copy backup on a regular basis. If recovery is needed, then the copies can be directly used as normal data files and recovered to a consistent point, without the need for a restore operation, thus significantly reducing overall recovery time.

» Enable Oracle RMAN block change tracking for fast incremental backups. This allows Oracle RMAN to avoid scanning blocks that have not changed when creating incremental backups. Oracle Exadata Storage Server also offloads block inspection from the database servers. Block change tracking is of greatest benefit for databases where fewer than 20 percent of the blocks are changed daily. For block change rates greater than 20 percent, testing is recommended to ensure that backup times are reduced.

» Set Oracle RMAN configuration setting FILESPERSET=1 when performing incremental backups to specify the maximum number of files in each backup set. A setting of 1 will allow for a faster single-file database restore operation.

» Use an external Oracle RMAN recovery catalog repository as long as it is located on a system that is configured to be highly redundant.

Restore and Recovery Best Practices

Oracle engineers performing testing in Oracle labs have discovered that optimum restore performance depends on proper management of the backup chunk size. While the chunk size setting is not important during backups, the restore operation must have Oracle RMAN backups broken into the best-performing chunk sizes for individual environments or performance can suffer. Oracle recommends running tests in each individual environment with the actual data to verify that the chunk size to be used for Oracle RMAN restores is going to provide optimal performance.

The layout of each individual database determines performance. In addition, to increase recovery rate performance, create a restore service that runs across all available database nodes. The service is used by the Oracle RMAN restore command. Oracle RMAN automatically balances the restore load among the targeted instances.

Best Practices for Performing Local Disk-Based Backups with Oracle RAC

Scale backup rates written to local disk in the FRA on an Oracle Database Appliance by using an Oracle RAC configuration.

» Use multiple instances and start with one Oracle RMAN channel per instance.


» Continue to add Oracle RMAN channels for performance per instance. Optimal backup rates have been observed by Oracle engineers with all Oracle RAC instances and one to four Oracle RMAN channels.

Best Practices for Performing Local Disk-Based Backups with Oracle RAC One Node

Scale backup rates written to local disk in the FRA on an Oracle Database Appliance by using a single instance and Oracle RAC One Node configuration.

» Start with one Oracle RMAN channel. » Continue to add Oracle RMAN channels to the single database instance to increase performance. Optimal

backup rates were observed with two to four Oracle RMAN channels.

Database Best Practices for Disk-Based Backups

Disk-only backups provide faster recovery times for data and logical corruptions and some tablespace point-in-time recovery (TSPITR) scenarios. Backups to disk also provide the ability to use backups directly with no restore needed by switching to a copy of the database, tablespace, or data file. Best practices for disk-based backup configurations include the following:

» Scale backup rates for disk: » Use all instances and start with two Oracle RMAN channels per instance. » Continue to add another two more Oracle RMAN channels for performance. » Utilize the automatic configuration of the Oracle Exadata Storage Server grid disk group layout during

deployment for better performance. This approach assigns the faster (outer) 40 percent of the disk to the DATA area and the slower (inner) 60 percent of the disk to the fast recovery area (RECO) area.

» An alternative strategy is to purchase additional SATA Oracle Exadata storage specifically for storing the FRA. This allows the application to leverage the full Oracle SuperCluster storage grid, allows the use of lower-cost storage for backups, and provides better failure isolation by using separate backup hardware. To reserve more space and bandwidth for the DATA disk group, Oracle recommends using a tape-based backup solution, or at the very least, a hybrid approach where full database backups are written to tape and incremental disk backups are written to the FRA.

» Configure a high-redundancy DATA disk group to contain the Oracle Cluster Registry (OCR) file, Oracle Cluster voting disk, spfiles, data files, redo log groups, and control files for any disk-based backup solution.

Database Best Practices for Tape-Based Backups

In addition to the general best practices for Oracle Database backups mentioned above, best practices for tape-based backups include the following:

» Perform daily backups of the Oracle Secure Backup or third-party backup and recovery software catalogs. » Keep the number of archive logs to a minimum to avoid adversely impacting backup rates. » Remember that heavy loads on an active database and fully consumed CPUs will affect backup rates. » Use Oracle Secure Backup for fast, low-cost tape backups. Oracle Secure Backup provides the fastest database

backup to tape due to its tight integration with Oracle RMAN. » Be aware that using Oracle Secure Backup enables the unused-block optimization capability. However, if the

backup is made directly to tape using a third-party media management product, then this does not have any effect because the unused-block optimization is available only with Oracle Secure Backup.

» Configure the Network Time Protocol (NTP) daemon on Oracle Secure Backup servers. This step ensures that the NTP daemon service on the Oracle Secure Backup administration and media servers are running and configured to use the same time source as the Oracle engineered system.

» Configure the Preferred Network Interface (PNI) to direct the Oracle Secure Backup traffic over the InfiniBand network interface. This parameter must be set when using a dedicated backup network.


» Configure one Oracle RMAN channel per tape drive and add tape drives to scale backup rates. Backup performance scales when more tape drives and Oracle RMAN channels are added, assuming there is available throughput on the media server.

» Configure dedicated 1 GbE, 10 GbE, or InfiniBand interfaces. Using a dedicated interface for the transport eliminates the impact on the client access network. Use InfiniBand for the best backup rates, especially for larger databases that require fast backup rates and low CPU overhead. When not using InfiniBand, use 1 GbE for smaller databases and 10 GbE for larger databases.

» Configure an Oracle RAC service for backup running on all database instances. This step reduces CPU utilization on the database and media server nodes and load balances the backups.

» Use SQL*Net service load balancing to distribute Oracle RMAN channels evenly among the allocated instances. » Tune the network communication when using a third-party media management vendor, and check with the

vendor the vendor should also be able to provide configuration best practices.

Best Practices for Tape-Based Backups In addition to the database best practices for tape-based backups mentioned above, there are some additional general best practices to consider when performing tape-based backups.

Network Configuration Best Practices

Care must be taken when configuring the network connections for tape-based backups. The following two sections detail configurations for connecting the media servers to the network.

InfiniBand Network to Media Server Configuration Best Practices

When connecting media servers to the InfiniBand network for tape-based backups, do the following:

» Directly connect media servers to the InfiniBand fabric by adding an InfiniBand Quad Data Rate (QDR) host channel adapter (QDR HCA) to the media server.

» Connect the HCA to two different Oracle SuperCluster InfiniBand switches to eliminate the switch as a single point of failure and provide high availability. This configuration ensures transparent failover if connectivity is lost to one of the ports.

» Configure bonding or IP network multipathing (IPMP) for the InfiniBand interfaces on the media server. » Update OpenFabrics Enterprise Distribution on the media server. » Configure IP over InfiniBand (IPoIB) connected mode for best performance. » Configure MTU Size=65520 on InfiniBand for faster data transmission. » Configure the media server to use the InfiniBand network.

1 GbE or 10 GbE Network to Media Server Configuration Best Practices

The media servers for tape-based backups can be connected with one or more Ethernet ports that are either 1 GbE or 10 GbE.

» Oracle Solaris enables link aggregation control protocol (LACP) bonding of interfaces for higher network performance. Alternatively, administrators can use Oracle Solaris to configure two interfaces with IP network multipathing (IPMP) for higher availability. LACP bonding requires a switch that can support LACP and enables the connection of multiple Ethernet ports to different switches, but having the same Ethernet address.

» IPMP supports multiple interfaces on the same IP link, connected to a single switch. This is a highly redundant configuration and increases availability, but does not provide high performance. This functionality is offered only by Oracle Solaris.

» Configure the switch configuration, using 1 GbE or 10 GbE on the database server to create either a dual-port 1 GbE or 10 GbE configuration that supports higher backup rates.

» Configure 1 GbE or 10 GbE on the media server.


Configure Persistent Bindings for Tape Devices

It is very important that the environment maintains consistent device addresses. In SAN environments running Oracle Linux, persistent bindings must be configured so that the device addresses do not change. If a device address changes, the media servers cannot access the device until the device configuration within Oracle Secure Backup is updated. Please see My Oracle Support Note 971386.1 for an example of creating persistent bindings for device attachments.

Back Up the Oracle Secure Backup Catalog

The Oracle Secure Backup catalog maintains backup metadata, scheduling, and configuration details for the backup domain. Just as it is important to protect the Oracle RMAN catalog or control file, the Oracle Secure Backup catalog should be backed up on a regular basis. In Oracle Secure Backup, the catalog backup has been preconfigured as follows:

» Media family: OSB_Catalog_MF writes all catalog backups to the same tape or tapes. » Job summary: OSB-CATALOG-SUM sends e-mail showing a daily report status of the catalog backup to users. » Data set: OSB-CATALOG-DS defines all directories and files to back up for file system backups. » Schedule: OSB-CATALOG-SCHED shows the schedule for the catalog backup.

The primary catalog backup configuration settings have been preconfigured, so only one step remains, which requires user intervention.

» Edit the OSB-CATALOG-SCHED triggers to specify when to perform the backup.

For detailed and up-to-date information, please refer to My Oracle Support Note 1558851.1.

Best Practices for Oracle Engineered Systems Backups The following sections detail best practices for backing up Oracle engineered systems along with pointers for more detailed information.

Oracle SuperCluster Backup Best Practices

The recommended backup solution for some Oracle SuperCluster and Oracle Optimized Solutions environments complements disk backups done on Oracle ZFS Storage Appliance by adding tape storage and tape storage management. Adding tape storage and tape storage management extends onsite backup storage capacity and enables the creation of off-site backup copies for long-term retention. Oracle recommends making tape backups of primary and remote site data using Oracle RMAN backups and NDMP on an Oracle ZFS Storage Appliance through Oracle Secure Backup media servers. For detailed procedures on performing a tape backup of Oracle SuperCluster, refer to the Oracle white Protecting SPARC SuperCluster Tape Backup with Symantec NetBackup

Backing Up the Oracle SuperCluster Infrastructure

The Oracle SuperCluster infrastructure information, including switch configurations for the InfiniBand fabric, is created when the system is first deployed. This configuration information should be backed up to disk or tape to enable quick recovery in case of human error or a disaster.

The Oracle SuperCluster backup tool, osc-backup, is used to create a snapshot of the entire Oracle SuperCluster infrastructure. This tool automates the process of backing up configuration information for all components in Oracle SuperCluster, including networking configurations plus Oracle Solaris domains and Oracle Solaris Zones on the

- at the following times:

» After initial deployment, to provide an initial backup snapshot of the Oracle SuperCluster infrastructure

http://www.oracle.com/technetwork/server-storage/sun-tape-storage/documentation/o13-016-1900890.pdf


» Before and after reconfiguration of significant operating parameters » Before and after installation of each quarterly full stack download patch (QFSDP) for Oracle SuperCluster » As part of a regular maintenance schedule, at a frequency that is practical and feasible (monthly, for example)

The backups created by the osc-backup tool are stored on the Oracle ZFS Storage Appliance that is internal to Oracle SuperCluster. It is critical that this storage be backed up as part of an enterprise backup and recovery strategy to safeguard against catastrophic failures and disasters. For more information on using the osc-backup tool, refer to My Oracle Support How to back up a SuperCluster using the osc-backup tool.

For More Information

For more information, p Oracle Optimized Solution for Secure Backup and Recovery of Oracle SuperCluster oracle.com/us/solutions/oos/oracle-backup-and-recovery/overview/index.html or refer to My Oracle Support Note 1558851.1.

Oracle Exadata Backup Best Practices

The Oracle Maximum Availability Architecture (MAA) group develops Oracle best-practices blueprints for Oracle Backup and Recovery Performance and

Best Practices for Exadata Cell and Oracle Exadata Database Machine, for backup and recovery of Oracle Exadata systems. The paper contains performance results for various Oracle Exadata configurations using both disk and tape backups. With detailed steps necessary for configuring the database, Oracle RMAN commands, and network settings for optimal backups, the white paper also includes recommended strategies for restore and recovery operations. In addition, readers can learn more about offloading backups using Data Guard and gaining additional benefits through the use of Oracle Active Data Guard.

Oracle Exadata systems can be backed up to tape or disk. While both media types provide extremely fast backup and restore rates, there are advantages to each. Tape-only solutions isolate faults from the Oracle Exadata Storage Servers and maximize Oracle Exadata capacity and bandwidth. Disk-only backups provide faster recovery times for data and logical corruptions and some tablespace point-in-time (TSPITR) scenarios. Disk backups also provide the ability to use backups directly with no restore by switching to a copy of the database, tablespace, or data file.

Oracle Database Appliance Backup Best Practices

As with the other Oracle engineered systems, both disk and tape backups provide extremely fast backup and restore rates, but there are advantages to each. Tape-only solutions isolate faults from the Oracle Database Appliance and maximize appliance capacity and bandwidth. Disk-only backups provide faster recovery times for data and logical corruptions and some tablespace point-in-time (TSPITR) scenarios. Backups to disk also provide the ability to use backups directly with no restore by switching to a copy of the database, tablespace, or data file.

Configuring Oracle Database Appliance for Local and External Backups

Choosing external versus local backup in the Oracle Database Appliance Manager during deployment will affect the size of the disk groups +DATA and +RECO, which will determine the FRA size. When configuring the appliance, selecting the external backup type option in the Oracle Database Appliance Manager Configurator utility assigns 80 percent of the disk to the DATA area and 20 percent of the disk to the RECO area. Selecting the local backup type option in the utility assigns 40 percent of the disk to the DATA area and 60 percent of the disk to the RECO area.

http://www.oracle.com/us/solutions/oos/oracle-backup-and-recovery/overview/index.html



For a more detailed discussion of the best practices for backing up Oracle Database running on Oracle Database Appliance, refer to Best Practices for Database Backups (Oracle Database 11g Release 2 or Higher) in this paper.

Backup and Recovery Best Practices for the Oracle Database Applianceon recommended architectures, suggested configuration parameters, best practices for backup and recovery of Oracle Database Appliances, and a detailed section on backing up appliances to network-attached storage (NAS) devices. Also included are performance results for Oracle Database Appliance configurations using local or external disks as backup targets, and the detailed steps necessary for configuring the various appliance components in order to obtain optimal backup results. For more information, please see the white paper, or refer to My Oracle Support Note 1558851.1 Protecting Oracle Database Appliance Tape Backup with Oracle Secure Backup oracle.com/technetwork/server-storage/engineered-systems/database-appliance/documentation/protecting-oda-with-osb-1674207.pdf.

Best Practices for Oracle ZFS Storage Appliance

Oracle recommends certain procedures for optimal configuration and performance of Oracle ZFS Storage Appliance with Oracle SuperCluster or Oracle Exadata. Some of these procedures are outlined here.

Use the Oracle Engineered Systems Backup Utility for Oracle ZFS Storage Appliance

Oracle recommends the use of the Oracle Engineered Systems Backup Utility for Oracle ZFS Storage Appliance. This tool, which can be used to configure the Oracle ZFS Storage Appliance that is internal to the Oracle SuperCluster and any additional external Oracle ZFS Storage Appliance used for backup, verifies the system configuration and automates the necessary configuration steps, including the following:

» Configuring Oracle ZFS Storage Appliance. This includes creating a project and shares for Oracle RMAN. » Configuring the Oracle SuperCluster database nodes. This includes enabling direct NFS (dNFS), creating mount

points, and adding backup services. » Creating the final Oracle RMAN scripts. (Note: The Oracle RMAN run block scripts are intended for samples only,

and should be reviewed and modified as necessary to meet site-specific requirements.)

Although these configuration steps can be performed manually, use of the Oracle Engineered Systems Backup Utility is recommended to reduce the risk of accidental misconfiguration.

The Oracle Engineered Systems Backup Utility for Oracle ZFS Storage Appliance can be downloaded from the Oracle ZFS Storage Appliance Plug-in Downloads web page at oracle.com/technetwork/server-storage/sun-unified-storage/downloads/zfssa-plugins-1489830.html.

Network Configuration Best Practices

Network configurations for Oracle ZFS Storage Appliance vary according to the environment where the appliance is deployed. If Oracle ZFS Storage Appliance is to be used to back up one or two Oracle engineered systems and it is located within 100 meters of the Oracle engineered systems, then InfiniBand networking can be used to connect all the systems. With this recommended configuration, each Oracle engineered system resides on its own InfiniBand fabric and different InfiniBand subnet.

When used with Oracle SuperCluster, Oracle ZFS Storage Appliance can be connected directly to the Oracle SuperCluster InfiniBand leaf switches if Oracle ZFS Storage Appliance is the only other appliance or device (other than Exadata Storage Expansion Rack) that will be connected to the infrastructure. Oracle ZFS Storage Appliance can be connected to external InfiniBand leaf switches if more appliances or devices will be connected. If Oracle ZFS Storage Appliance is to be used to back up more than two Oracle engineered systems, or Oracle ZFS Storage

http://www.oracle.com/technetwork/server-storage/engineered-systems/database-appliance/documentation/protecting-oda-with-osb-1674207.pdf

http://www.oracle.com/technetwork/server-storage/engineered-systems/database-appliance/documentation/protecting-oda-with-osb-1674207.pdf

http://www.oracle.com/technetwork/server-storage/sun-unified-storage/downloads/zfssa-plugins-1489830.html



Appliance cannot be located within 100 meters of the Oracle engineered systems, then 10 GbE should be used to connect the Oracle engineered systems to Oracle ZFS Storage Appliance.

Network Configuration for InfiniBand Connectivity

To provide the highest availability and performance, Oracle recommends that each Oracle ZFS Storage Appliance controller be configured with two dual port QDR InfiniBand HCA cards, providing a total of four ports per Oracle ZFS Storage Appliance controller. The controllers should also be connected to a pair of redundant 10 GbE switches to provide continuous connectivity in the event of an outage. This configuration requires the use of an active/standby IPMP group. For configuration details, see the Oracle white paper Configuring a Sun ZFS Backup Appliance with Oracle SPARC SuperCluster .

Network Configuration for 10 GbE Connectivity

To provide the highest availability and performance while not using the InfiniBand network, Oracle recommends that each Oracle ZFS Storage Appliance controller be configured with two dual port 10 GbE optical cards, providing a total of four ports per Oracle ZFS Storage Appliance controller. This configuration requires the use of an active/standby IPMP group. Additionally, for performance reasons, the 10 GbE network should be configured with a large MTU size, also known as jumbo frames. For configuration details, see the Oracle white paper Configuring a Sun ZFS Backup Appliance with Oracle SPARC SuperCluster

Configuring Network Cluster Resources

Once the network resources have been configured for either 10 GbE or InfiniBand, the interfaces need to be placed under the control of the Oracle ZFS Storage Appliance cluster. Clustering the network resources provides continuous access to the data on Oracle ZFS Storage Appliance in the event of a controller head failure in the appliance.

Configuring Networking for Oracle ZFS Storage Appliance

The InfiniBand ports on Oracle ZFS Storage Appliance must be configured for IP multipathing. Four IP addresses will be required for each Oracle ZFS Storage Appliance head for a total of eight addresses because the interfaces will be running in an active-active configuration.


F Backup and Recovery Performance and Best Practices Using the Sun ZFS Storage Appliance with the Oracle Exadata Database Machine y Oracle Support Note 1354980.1 to get details on the recommended software stack for Oracle ZFS Storage Appliance when used with Oracle Exadata. Refer to the Oracle white paper Configuring a Sun ZFS Backup Appliance with Oracle SPARC SuperCluster for more details on how to configure

Oracle ZFS Storage Appliance for use with Oracle SuperCluster to provide maximum throughput and availability.

Storage Configuration Best Practices

Configuring Oracle ZFS Storage Appliance storage pools assigns physical disk drive resources to logical storage pools for backup data storage. When backing up and recovering Oracle Database to Oracle ZFS Storage Appliance, Oracle recommends that Oracle ZFS Storage Appliance disk resources be configured in two equally sized pools. Configure the two pools by assigning half of the physical drives in each drive tray to each storage pool to maximize system throughput.

Once the pools have been created, they should be placed under the control of the Oracle ZFS Storage Appliance cluster to protect against the failure of a network component supporting the appliance or a component within the

http://www.oracle.com/technetwork/server-storage/sun-unified-storage/documentation/zba-ssc-config-043013-1948329.pdf




http://www.oracle.com/technetwork/database/features/availability/maa-wp-dbm-zfs-backup-1593252.pdf





appliance controller itself. Doing so ensures that an Oracle RMAN database backup or restore operation can continue to run. For more details, see the Oracle white p Backup and Recovery Performance and Best Practices Using the Sun ZFS Storage Appliance with the Oracle Exadata Database Machine Oracle white

Oracle Optimized Solution for Secure Backup and Recovery: Oracle SuperCluster Backup and Recovery

Configuring Oracle RMAN Projects and Shares on Oracle ZFS Storage Appliance

Share configuration is the process of setting up and running NFS mount points for client access. A project is an entity that provides a higher-level management interface point for a collection of shares. Two projects should be created for an Oracle Database 11g Release 2 Oracle SuperCluster configuration one project per pool. To optimize share management, update the default mount point for shares contained in the project to reference the database name. To optimize a system for performance, create four shares for each project in each pool, for a total of eight shares (four on each head).

Oracle provides the Engineered Systems Backup Configuration Utility downloadable from Oracle Technology Network at the Oracle ZFS Storage Appliance Plug-In Downloads page that automates the process of creating Oracle ZFS Storage Appliance projects and shares and creates the corresponding entries. The utility also creates Oracle RMAN command files that can be used for backing up the database. Running the utility ensures that all of the best practices of using Oracle ZFS Storage Appliance as an Oracle Exadata backup destination are followed.

» Create two projects, one per pool. » . » Configure the project s NFS Exceptions to allow access to the shares by the Oracle engineered system.

Database Backup and Restore Best Practices with Oracle ZFS Storage Appliance and Oracle Engineered Systems The following key practices should be implemented in order to obtain maximum availability and performance from Oracle ZFS Storage Appliance. Note, however, that for any backup operation or changes to backup practices, IT staff should perform testing to ensure that the backups complete within the available backup window and that the database restore and recovery operations can meet recovery time objective requirements.

Configure Network Connectivity

Do the following to configure network connectivity:

» Enable Direct NFS. An Oracle Database instance running on an Oracle engineered system is backed up to Oracle ZFS Storage Appliance using the Oracle Database 11g Release 2 Direct NFS option. Note, however, that by default, the Direct NFS option is not enabled, and enabling Direct NFS requires a database shutdown in order to relink the Oracle Database kernel. By enabling Direct NFS, the operating system kernel NFS processes are bypassed and the Direct NFS utility opens up 1 MB network buffers solely for use by Oracle RMAN, bypassing the system configured TCP network buffers, and speeding up the database backup and restore. For details, see the Oracle white Sun ZFS Storage Appliance Secure Backup and Recovery: Oracle SuperCluster Backup and Recovery

» Configure oranfstab. It is not essential to create an oranfstab file for Oracle Database to utilize Direct NFS. However, when using any of the following configurations, an oranfstab file must be created to fully utilize the network resources between the database nodes in Oracle Exadata or Oracle SuperCluster and Oracle ZFS Storage Appliance. » An Oracle engineered system is to be connected to the Oracle ZFS Storage Appliance using InfiniBand » An Oracle engineered system is running Oracle Solaris » An active/active IPMP group is created on Oracle ZFS Storage Appliance



http://www.oracle.com/us/solutions/oos/oracle-backup-and-recovery/overview/index.html



Refer to My Oracle Support Note 1517107.1, RMAN Backup from SPARC SuperCluster to Oracle ZFS Storage Appliance, for sample /etc/oranfstab files.

Follow Recommended Best Practices for Database Backup and Restore

Oracle recommends using a combination of Level 0 and Level 1 backup sets when backing up Oracle Database to Oracle ZFS Storage Appliance. The frequency of backups is dictated by recovery time and recovery point objective requirements. Note that when using Oracle ZFS Storage Appliance, Oracle does not recommend an incrementally updated backup strategy using a combination of data file copies and incremental backup sets that are subsequently merged into the data file copies. When an incrementally updated backup strategy is selected, the backup solution becomes I/O bound during the merge process, and alleviating this bottleneck requires a significant number of available disk spindles in order to achieve the required IOPS rate. For most IT departments, the following recommendations should be sufficient:

» The Oracle Database FRA should remain on the Oracle Exadata Storage Servers that are part of the Oracle Exadata or Oracle SuperCluster and should not be put on Oracle ZFS Storage Appliance.

» The weekly full level 0 backup should be written to Oracle ZFS Storage Appliance. » A daily incremental level 1 backup should be written to Oracle ZFS Storage Appliance. A maximum of 16 Oracle

RMAN channels should be allocated and configured for both weekly level 0 and daily level 1 backups to write to one of the 16 Oracle ZFS Storage Appliance shares created using the Oracle Engineered Systems Backup Utility mentioned earlier (or created manually).

» Two separate backup jobs should be submitted if Oracle RMAN compression is used to preserve space on the Oracle ZFS Storage Appliance but the database consists of a mixture of uncompressed data and compressed data or data that cannot be compressed further. One of the backup jobs will create an uncompressed backup, and the second will create a compressed backup. Otherwise, compressing data that does not typically yield good compression lengthens the backup window significantly, consumes unnecessary CPU resources, and results in very little space savings.

» Oracle RMAN offers an option to amortize the backup over a given time period and minimize the performance impact to critical applications. For example, if there is an eight-hour backup window available for the weekly level 0 backup of a 20 TB database, but critical database functions are still running during that backup window, the Oracle RMAN duration minimize load option can spread the work over the entire backup window.

» The use of Oracle RMAN options such as sectionsize is recommended. These options are included in the backup scripts generated by the Oracle Exadata Backup Configuration Utility. The sectionsize parameter breaks up BIGFILE tablespaces into more manageable amounts. Oracle recommends that the largest tablespaces be broken up into equal size sections for each Oracle RMAN channel. For instance, if there are two tablespaces of 10 TB and 14 TB, and there are 16 Oracle RMAN channels, then the sectionsize parameter should be set to approximately 320 GB.

» The Oracle Exadata Backup Configuration Utility does not enable Oracle RMAN or Oracle ZFS Storage Appliance compression, because this is something that is dependent on the data to be backed up and the database licensing options purchased. Enabling Oracle RMAN compression increases CPU utilization.

Tune the Oracle Database Instance for Oracle RMAN

Optimizing high-bandwidth backup and restore operations using Oracle RMAN and Oracle ZFS Storage Appliance requires adjusting the instance parameters that control I/O buffering. For information about how to tune these parameters, see My Oracle Support Note 1072545.1 RMAN Performance Tuning Using Buffer Memory Parameters at http://support.oracle.com.

http://support.oracle.com/


Configure Oracle RMAN

Performance benefits can be realized by configuring 16 or 32 Oracle RMAN channels to be evenly distributed over the Oracle Database instances and nodes in the Oracle RAC cluster. The channels also should be evenly distributed over the shares exported from Oracle ZFS Storage Appliance.

Best Practices Using Oracle ZFS Storage Appliance Traditional backups are performed with a base-level full backup and then backups at subsequent intervals to capture only the files that have changed. This can be an inefficient method because restore operations must start with the full backup and apply each incremental in the proper order. Also, any and all incremental backups must be applied or the restore will not be complete.

When using external Oracle ZFS Storage Appliances for backups, it is possible to follow a particular strategy as shown in Figure 13. Backup operations should start with a full level 0 backup on the first day. On the second day, IT staff can perform an incremental (level 1) backup to record just the blocks that have changed from the previous day, or they have the option to apply the level 1 incremental to the full level 0 backup, as shown in Figure 14 (using Day 3 as an example). The incremental applied backup merges any changes since the full backup and creates a single restore point. This is a much more efficient method than the traditional combination of full and incremental backups typically used to obtain an up-to-date restore.

Figure 13. Oracle suggested backup strategy for databases.

Use ZFS Snapshots

The previous procedure can be taken a step further by utilizing the snapshot capabilities of Oracle ZFS Storage Appliance, as illustrated in Figure 14. It starts again with a full level 0 backup on Day 1 and an incremental backup on Day 2. On Day 3, the incremental backup is applied to the level 0 backup from the first day, and a snapshot is taken of the applied incremental, resulting in an image that looks like a version of a full backup. This consumes a lot less space because it is recording only changes.


Figure 14. Oracle suggested backup strategy with Oracle ZFS Storage Appliance.

Using Non-Oracle Exadata Storage for Oracle SuperCluster and Oracle Exadata Database Machine Backups While not recommended, non-Oracle Exadata storage solutions might be necessary in some cases. Differences in network connectivity, storage configuration, and protocols used by non-Oracle Exadata storage will affect backup performance and contribute complexity. The Oracle Maximum Availability Architecture white paper at oracle.com/technetwork/database/features/availability/maa-tech-wp-sundbm-backup-final-129256.pdf contains suggestions on how to avoid the pitfalls of non-Oracle Exadata storage in these environments.

For configuration details and up-to-date changes, please see My Oracle Support Note 1558851.1.

For more information on Oracle's Maximum Availability Architectures, please see the following resource: oracle.com/au/products/database/maa-096107.html

Providing Backup and Recovery Savings and High Availability with Remote DR Sites Figure 15 shows the ultimate solution for providing the ability to restore IT operations while spending less for backup and recovery. Traditional solutions require expensive backup and recovery equipment to restore the production system as quickly as possible. An alternative solution is to have a second remote site with replicated applications and data. Using primary and secondary remote sites eliminates the need for high-performance systems to recover production operations. Instead of deploying Oracle SuperCluster and a high-performance disk array to back it up, a secondary Oracle SuperCluster with Oracle's lower cost Oracle ZFS Storage Appliance or tape can be deployed. In the event of a failure, production can be switched over immediately to the remote site. The primary site's recovery time is irrelevant because production continues on the secondary systems. This approach provides better business continuance and results in significant savings because backup and recovery system costs are lower.

When not running production operations as part of a business continuance effort, the remote replicated location can be used for testing, development, and other functions. Oracle is developing the ability to generate reports from a copy of the secondary site's production database, enabling IT staff to offload work from the production system and keep the secondary database current. This solution provides savings for backup and recovery and maximizes system availability. Many organizations distribute their recovery systems in this manner, with applications that can

http://www.oracle.com/technetwork/database/features/availability/maa-tech-wp-sundbm-backup-final-129256.pdf

http://www.oracle.com/au/products/database/maa-096107.html


fail over from city to city. In fact, this design can be used almost universally, with the exception of firms that are required to have large and elaborate backup and recovery systems for compliance reasons.

Figure 15. Oracle technologies enable maximum protection plus cost savings.

replication features of the recovery appliance. Backups on a local recovery appliance can be easily and quickly replicated via secure transport to a remote recovery appliance. Recovery operations can be run directly from the remote recovery appliance, without first staging the data locally.

Conclusion When deploying Oracle engineered systems, it is vital to update backup and recovery platforms at the same time, because the resulting production environment will exhibit superior performance with far greater data throughput and scalability. Backup and recovery solutions that are not designed to handle the unique performance characteristics of Oracle engineered systems are unable to keep pace and likely will become the limiting factor in protecting against data loss and maintaining suitable service levels.

Oracle Optimized Solution for Secure Backup and Recovery, powered by Oracle servers, provides an excellent backup solution for heterogeneous networks. A comprehensive end-to-end data archive solution with centralized management for simplified administration, Oracle Optimized Solution for Secure Backup and Recovery provides a scalable architecture that easily adapts to growing storage demands. Indeed, the high performance and capacity of this solution facilitates the consolidation and migration of existing SAN backups to more cost-efficient network backups.

With a highly flexible architecture, Oracle Optimized Solution for Secure Backup and Recovery features virtually unlimited scalability. The architecture supports multiple media servers working together under the administrative control of an administrative server. Additional media servers can be deployed or multiple backup domains can be configured to address evolving scalability requirements. Organizations using Oracle Optimized Solution for Secure


Backup and Recovery can implement cost-effective and powerful backup solutions for the largest, most diverse data center environments.

For configuration information, recommendations, sizing information, and all the latest information on Oracle Optimized Solution for Secure Backup and Recovery, please refer to My Oracle Support Note 1558851.1.

References For additional information, please visit the URLs listed in Table 6.

TABLE 6. REFERENCES FOR MORE INFORMATION

Description URL

Oracle Secure Backup software oracle.com/technetwork/products/secure-backup/overview

Oracle servers oracle.com/us/products/servers/overview

Oracle Solaris 11 oracle.com/technetwork/server-storage/solaris11/overview

Oracle Solaris Cluster oracle.com/technetwork/server-storage/solaris-cluster/overview

Oracle Linux oracle.com/us/technologies/linux/overview

Oracle Enterprise Manager oracle.com/us/products/enterprise-manager/index

StorageTek Automated Cartridge System Library Software

oracle.com/us/products/servers-storage/storage/tape-storage/acsls/overview

Oracle Key Manager oracle.com/us/products/servers-storage/storage/storage-software/oracle-key-manager/overview

Oracle storage systems oracle.com/us/products/storage/overview

Oracle Optimized Solutions oracle.com/optimizedsolutions

Oracle Support Document 1558851.1 (Oracle Optimized Solution for Secure Backup and Recovery)

https://support.oracle.com/epmos/faces/DocumentDisplay?id=1558851.1

http://www.oracle.com/technetwork/products/secure-backup/overview/index.html

http://www.oracle.com/us/products/servers/overview/index.html

http://www.oracle.com/technetwork/server-storage/solaris11/overview/index.html

http://www.oracle.com/technetwork/server-storage/solaris-cluster/overview/index.html

http://www.oracle.com/us/technologies/linux/overview/index.htm

http://www.oracle.com/us/products/enterprise-manager/index.html

http://www.oracle.com/us/products/servers-storage/storage/tape-storage/acsls/overview/index.html

http://www.oracle.com/us/products/servers-storage/storage/storage-software/oracle-key-manager/overview/index.html

http://www.oracle.com/us/products/servers-storage/storage/storage-software/oracle-key-manager/overview/index.html

http://www.oracle.com/us/products/storage/overview/index.html

http://oracle.com/optimizedsolutions

https://support.oracle.com/epmos/faces/DocumentDisplay?id=1558851.1

Oracle Corporation, World Headquarters Worldwide Inquiries 500 Oracle Parkway Phone: +1.650.506.7000 Redwood Shores, CA 94065, USA Fax: +1.650.506.7200

Copyright © 2015, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0615 Oracle Optimized Solution for Secure Backup and Recovery October 2015 Author: Dean Halbeisen

C O N N E C T W I T H U S

blogs.oracle.com/oracle

facebook.com/oracle

twitter.com/oracle

oracle.com