h13077 Wp Isilon Backup Using Commvault Bestpractices

Best Practices Guide

Abstract

This white paper outlines best practices for deploying EMC Isilon

scale-out storage with CommVault Simpana. EMC Isilon

seamlessly integrates with Simpana to provide a flexible and

scalable backup solution.

April 2014

BACKUP USING COMMVAULT SIMPANA

WITH EMC ISILON

2 Backup Using CommVault Simpana with EMC Isilon

2

Copyright © 2014 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without

notice.

The information in this publication is provided “as is.” EMC

Corporation makes no representations or warranties of any kind with respect to the information in this publication, and

specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in

this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC

Corporation Trademarks on EMC.com.

EMC2, EMC, the EMC logo, Isilon, OneFS, and SmartLock are registered trademarks or trademarks of EMC Corporation in the

United States and other countries. All other trademarks used herein are the property of their respective owners.

Part Number H13077


3

Table of Contents

Executive Summary ........................................................................................ 4

Audience ..................................................................................................... 5

Terminology ................................................................................................. 5

Solution Components .................................................................................... 6

Architecture Overview .................................................................................... 6

Best practices ................................................................................................. 7

Networking .................................................................................................. 7

Basic ....................................................................................................... 7

Advanced ................................................................................................. 9

Performance ................................................................................................ 9

Basic ....................................................................................................... 9

Advanced ............................................................................................... 10

Configuration ............................................................................................... 11

Isilon Storage Platform Configuration ............................................................ 11

Basic ..................................................................................................... 11

Advanced ............................................................................................... 12

CommVault Simpana Configuration ............................................................... 14

Basic ..................................................................................................... 14

Advanced ............................................................................................... 19

Conclusion .................................................................................................... 20

About EMC .................................................................................................... 21


4

Executive Summary

This paper describes the best practices and solution-specific configuration steps for

deployment of CommVault SimpanaTM with EMC® Isilon® for both basic and advanced

deployments. EMC Isilon scale-out NAS storage together with CommVault Simpana

provides a comprehensive, flexible and scalable backup and archive solution that lets

enterprises of all sizes address their backup needs; from data protection to

preservation while providing secure self-service access to data.

Backup challenges with traditional storage

1. Compounding effect of backups

Each year organizations are generating more data and keeping that data for longer.

This growth can be exponential, since traditionally many full backup copies are kept

weekly, monthly, and yearly. For example: If the most recent weekly backups are

kept for 8 weeks, monthly backups for 10 months (covering the remainder of the

year), and yearly backups for 7 years, then one file would have 25 copies just using

the backup algorithm. Using this example, just 40 TB of new data would need 1 PB of

storage. There are mechanisms to reduce some of the common data, like

compression, deduplication, and snapshots, but portions of the data will be unique or

not easily reduced.

This compounding growth affects the cost of tape even more, since many of these

data reduction mechanisms are not available, native tape drive compression is only

supported by tape. In addition, there are many often overlooked issues that need to

be considered when using tape – the cost of secure off-site storage, cost to

periodically retrieve backups from storage to perform test restores as needed for

compliance or business policy, and the risk of not being able to read older tapes if too

many tape drive generations have passed.

2. Management overhead

Individually monitoring performance and free space on traditional RAID-based volumes / LUNs becomes a huge burden. The constant juggling and adding of new

volumes / LUNs when capacity or hardware limits are reached consumes more and

more time, and each storage change often requires the application’s configuration to be updated as well.

3. Refresh cycle and data migration

The inevitable, and often overlooked, hardware refresh every 3 to 5 years will more

than likely take up many nights and weekends, data center resources, and budget

with traditional RAID-based storage systems. With extensive planning, a complete

re-evaluation of performance and capacity requirements, including future needs out

to 3 or more years; they will need a new infrastructure to be stood up alongside the

old infrastructure to allow the data to be migrated, likely with one or more outage

windows. Provisioning data center rack space, power, cooling, and network

infrastructure for this type of hardware refresh can be very expensive and time

consuming.


5

Backup Solutions with EMC Isilon and CommVault Simpana

Isilon scale-out technology removes the hurdles of multiple backup copies,

management overhead, application reconfiguration, and data migration; so you can

focus on your organization’s backup and archive strategy.

EMC Isilon OneFS®, the intelligence behind the Isilon scale-out NAS, combines the

three layers of traditional storage architectures—file system, volume manager, and

data protection—into one unified software layer, creating a single intelligent file

system that spans across all nodes within a cluster. The application only needs to be

configured once to use the single namespace provided by OneFS. New cluster

capacity can be added in 60 seconds and is immediately available for use by the

application without any manual intervention. To eliminate performance and capacity

hot-spots, and the juggling of volumes / LUNs, Isilon automatically distributes clients,

file data, and free space across the entire cluster. With Isilon’s utilization rate of over

80%, not achievable with RAID-based storage systems, fewer hard drives are needed

to satisfy the capacity demands while providing a comparable or higher level of data

protection. The ability to grow capacity to over 20 PB within a single cluster and

push-button retire older hardware greatly simplifies data migration on hardware

upgrades and eliminates the need to support two infrastructures simultaneously. This

saves valuable data center rack space, power, cooling, and network infrastructure,

not to mention time.

CommVault Simpana is a comprehensive information protection and preservation

platform that offers advanced features like Simpana OnePass™ which converges

backup, archive and reporting operations into a single job maximizing efficiency while

minimizing any impact of production infrastructure.

This document describes the best practices and solution-specific configuration steps

for deployment of CommVault Simpana with EMC Isilon for both basic and advanced

deployments. Basic best practices enable quick, easy, and straightforward

deployments using the fewest settings to get you started. The advanced best

practices identify opportunities to configure the system for performance, scalability,

or highly secure environments for a more optimized deployment model. This requires

a high level of knowledge, support, and time to plan the deployment of all the

components in advance.

Audience

This document is intended for administrators who will deploy and configure EMC Isilon

with CommVault Simpana. The assumed level of technical knowledge for the devices

and technologies described in this document is high.

Terminology

The abbreviations used in this document are summarized in Table 1.

Abbreviation Description

CS CommServe Server

MA MediaAgent Server

DB Database


6

Abbreviation Description

DDB Deduplication Database

SQL Microsoft SQL Server

Table 1. Abbreviations

Solution Components

The following solution components are described in this document:

CommVault Simpana 9 and 10, and EMC Isilon scale-out NAS with OneFS 7.0

and 7.1

The Isilon Third-Party Software & Hardware Compatibility Guide can be found at

https://support.emc.com/docu45932.

Architecture Overview

To understand the CommVault Simpana architecture, let us first discuss the

components and their function. Starting at the edge and working our way towards

the center, the iDataAgent software is deployed on all servers, workstations, and

laptops that need to be backed up; they are collectively called clients. There are

specific iDataAgents for each of the different File Systems, Applications, and

Databases supported by Simpana. The MediaAgent software is deployed on dedicated

servers that manage the transfer of data between the clients and media; there are

often multiple MediaAgents within an environment to distribute the load. The

CommServe software is deployed on a dedicated server that communicates and

coordinates all operations; backups, restores, copies, media management, etc. The

CommServe creates a logical grouping of MediaAgents and clients called a CommCell;

depending on the size of the environment, or network and physical boundaries, there

can be multiple CommCells within an environment.

An overview of the logical architecture for a CommVault Simpana and EMC Isilon

scale-out storage deployment is provided in Figure 1. This shows the Isilon cluster

defined as a single CommCell Disk Library that is being shared between the

CommServe and MediaAgents. With this configuration, the Disk Library settings

never need to be updated. When capacity is added to the Isilon cluster, the

CommServe and MediaAgents immediately see the additional capacity without

intervention; nodes can be added to an Isilon cluster in as little as 60 seconds with

just a few clicks.

Adding capacity to a CommCell that is using traditional RAID-based storage requires

many configuration steps on both the storage system and the CommCell. The

storage system needs to be manually configured to provision the additional storage

on existing or new volumes / LUNs. The CommCell would need an existing Disk

Library modified with an additional mount path or a new Disk Library created, which

requires a new Storage Policies to be defined. This could also require that Subclients

https://support.emc.com/docu45932


7

be manually rebalanced across the existing Storage Policies. These are time

consuming tasks that are eliminated when Isilon storage is used instead.

Figure 1. Logical architecture overview

Best practices

Networking

The networking sections that follow provide guidance and best practices to design and deploy the network connectivity and communication pathways of your environment.

Basic In this section we will discuss the basic information you should know to avoid firewall,

permissions, and latency pitfalls, and understand your connectivity options.


8

Firewall

During the installation, it is recommended to select the option to “Add programs to

the Windows Firewall Exclusion List”, even if the Windows Firewall is disabled. This

will add CommCell programs and servers to the exclusion list and ensure they

function, even if a disabled Windows Firewall is later enabled. Depending on the

Simpana version, this may be enabled by default.

Permissions

The CommServe, MediaAgent, and client software must be deployed using an

Administrator account or an account that is a member of the Administrator group on

these systems.

Similarly, the account used to define an Isilon cluster as a CommVault Disk Library

should have full control permissions on the share and directory.

Target Permissions Required

CommServe, MediaAgents, and

Clients

Local Administrator

Administrator Group

Disk Target (Isilon Share and Directory) Full Control

Table 2. Permissions Requirements

Bandwidth

Generally, more bandwidth available between services provides better performance.

Other factors account for overall performance; however, bandwidth can be an

important component

For most deployments, a pair of bonded Gigabit Ethernet connections on the

CommServe, MediaAgents, and Isilon cluster nodes will provide sufficient bandwidth

and resiliency. The CommVault recommendation for one exclusive 10 GbE connection

to a NAS Disk Library is based on a traditional NAS that has one or two filer

controllers with limited connectivity. Isilon is a scale-out NAS with CPU, RAM, drives,

and networking that scale-out together to provide more configuration options.

If switch ports are limited and bandwidth needs can be met with less than all the

Isilon cluster nodes, the excess nodes can remain disconnected on the external

Ethernet interfaces. This is possible due to the internal InfiniBand connections that

enable all of the nodes to continue to communicate and access resources, including

capacity. At a minimum, it is recommended to connect at least two different nodes in

the cluster to two different Ethernet switches to avoid a single point of failure.

Latency

Latency can affect transfer rate performance. Therefore, it is recommended that no

more than 25 ms of latency exist between the CommVault services and Isilon cluster.

Performance could be seriously affected if this latency is greater than 50 ms.


9

Advanced

In this section we will discuss the advanced information you should know regarding firewall

ports and connectivity option for high performance.

Firewall

Server Destination ports Comments

CommServe Server 81 CommCell Console – Management UI

CommServe, MediaAgent, and Clients

8400 CommVault Communications Service (CVD)

CommServe, MediaAgent,

and Clients

8401 CommVault Server Event Manager (EvMgrS)

CommServe, MediaAgent,

and Clients

8402 CommVault Client Event Manger (EvMgrC)

CommServe, MediaAgent, and Clients

8403 CommVault Tunnel HTTP/HTTPS

MediaAgent and Clients 1024 to 65525 CommVault Backup/Restore Range (Dynamic)

Table 3. Advanced firewall

Bandwidth

The use of 10 GbE connections should be considered for deployments where there are

enough excess CommVault and client resources to support higher transfer rates than

the dual bonded Gigabit Ethernet interfaces can provide, and a reduced

backup/restore window is needed.

Performance The performance sections that follow provide guidance and best practices to ensure the health

and performance of the CommCell components and their resources.

Basic

In this section we will discuss the basic information you should know about the resource

requirements for the CommServe and MediaAgent servers, CommServe database, and virtual deployments.

CommServe and MediaAgent Server Configurations

The CommVault server recommendations should be followed for your deployment

size. Consider stepping up to the next higher recommendation to ensure optimal DB

performance and to provide the most flexibility. For example, client resources may

be insufficient to perform deduplication and/or compression tasks on backup data at

an acceptable transfer rate, so it may be best to offload those tasks to the


10

MediaAgent servers. In which case, more network bandwidth may be consumed, so

that must be considered when making these types of adjustments.

SQL Configuration

On the CommServe server, the SQL memory size is typically set to 50% of the

physical RAM. With that in mind, if feasible, monitor the CommServe database size

so that there is enough RAM for the DB to run in memory. Also, it is recommended to

run the CS DB on fast disks, possibly SSDs or PCIe flash; such as an EMC XtremSF

PCIe flash card. For more information, see IOPs for CommServe Database Volumes

(http://documentation.commvault.com/commvault/v10/article?p=products/commserv

e/iops.htm).

Deploying in a Virtual Environment

The CommServe SQL data and log files should be stored on different virtual hard

drives (VMDKs or VHDs) and the log file should be placed on faster storage (RAID 10

or RAID 1).

CommVault states that MediaAgent virtualization may only achieve 60% of the

maximum number of concurrent streams supported in a physical deployment.

Local Server Storage

In most cases, two 15K drives (RAID 1) is the recommended local storage

configuration for the operating system on both the CS and MA servers, except for the

highest CS tier which recommends eight 15K drives (RAID 10) or enterprise SSDs

(RAID 5).

Advanced

In this section we will discuss the advanced information you should know regarding the

MediaAgent Deduplication Database performance.

Deduplication DB (DDB)

As a general guideline, you will need to store the DDB on four or more SSDs (RAID 5

or 10) or PCIe flash, such as EMC XtremSF PCIe Flash, on each MA server locally to

achieve the DDB performance necessary to ensure adequate backup/restore transfer

rates. The DDB should not be placed on an Isilon cluster; it must be local to each MA

server.

More information can be found in the following CommVault documents.

Deduplication to Disk - Best Practices:

http://documentation.commvault.com/commvault/v10/article?p=features/ded

up_disk/best_practice.htm

Deduplication Building Block Guide:

http://documentation.commvault.com/commvault/v10/article?p=features/ded

up_disk/building_block.htm

http://documentation.commvault.com/commvault/v10/article?p=products/commserve/iops.htm

http://documentation.commvault.com/commvault/v10/article?p=products/commserve/iops.htm

http://documentation.commvault.com/commvault/v10/article?p=features/dedup_disk/best_practice.htm

http://documentation.commvault.com/commvault/v10/article?p=features/dedup_disk/best_practice.htm

http://documentation.commvault.com/commvault/v10/article?p=features/dedup_disk/building_block.htm

http://documentation.commvault.com/commvault/v10/article?p=features/dedup_disk/building_block.htm


11

Configuration

An overview of the configuration steps for a CommVault Simpana and EMC Isilon

OneFS scale-out storage deployment is provided in Figure 2. The sections that follow

provide guidance on best practices or required settings for each step, where

applicable.

Figure 2. CommVault Simpana and Isilon OneFS configuration workflow

Isilon Storage Platform Configuration The sections that follow provide configuration guidance and best practices for configuring the

Isilon cluster.

Basic In this section we will discuss the basic requirement for creating the backup directory on the

Isilon cluster.


12

Create CommVault Backup Directory

In a typical IT environment, it is common to have multiple applications utilizing a

shared central enterprise storage system, such as an Isilon cluster. To ensure

simplified storage and application management, it is a recommended best practice to

create directories using a naming convention that easily represents the application

(for example, /ifs/CommVaultData). In this way, it should be clear which group owns

the data, if capacity or other questions arise.

When creating the CommVault backup directory, ensure the user account used by the

CommServe and MediaAgents has Full Control permissions on the directory.

The default /ifs share or export can be used to access the directory (for example,

\\IsilonClusterName\ifs\CommVaultData)

Advanced

In this section we will discuss the advanced options of creating a CommVault specific share or export on the Isilon cluster.

Create CommVault Specific Share or Export

Depending on the cluster configuration and other workloads being placed on the

cluster, it may be helpful to create a specific share or export for CommVault. This

could be useful to more easily expose the directory being use for this purpose, if not

immediately under the /ifs directory, or to more clearly show the workloads on a

cluster from a visibility / manageability perspective. This is facilitated by the

additional details that can be recorded within the share or export description on the

Isilon cluster.

When creating the CommVault share or export, ensure the user account used by the

CommServe and MediaAgents has the appropriate permissions on the share.

For example, the following CLI command can be used to create CommVault backup

directory and its SMB share.

isilon-1-1# mkdir /ifs/CommVaultData

isilon-1-1# isi smb shares create –name=CommVaultData –

path=/ifs/CommVaultData –browsable=true –description=”Share for

CommVault backups, managed by the Backup and Archive IT Group”

Isilon File Sharing

With its build-in multi-protocol support, Isilon OneFS provides the ability to share

directories and files to Windows client machines through SMB and to Linux client

machines through NFS.


13

Create CIFS/SMB Share

Refer to the Isilon OneFS Web Administration Guide for complete configuration

procedures.

Overview:

1. Enable the SMB Service, if not enabled already

2. Create a SMB share

3. Add the user account used by the CommServe and MediaAgents to the SMB

Share with Full Control permissions

4. Test this SMB share by browsing to it on the CommServe and MediaAgents

Figure 3. Isilon SMB WebUI

Create NFS Export

Refer to the Isilon OneFS Web Administration Guide for complete configuration

procedures.

Overview:

1. Enable the NFS service, if not enabled already

2. Create a NFS export

3. Add CommServe and MediaAgents Server IP Address in the NFS client list and

allow Read-Write access

4. Map users to user name ‘nobody’

5. Test the NFS export by mounting it on the CommServe and MediaAgents


14

Figure 4. Isilon NFS WebUI

CommVault Simpana Configuration

The sections that follow provide configuration guidance and best practices for configuring the CommVault Simpana CommCell.

Basic In this section we will discuss the basic best practice for configuring a single Isilon Disk Library

to be shared with all the MediaAgents, the CommCell policy configurations, and the necessary

alerts for virtual deployments.

Create an Isilon Disk Library

All MediaAgents can be configured to share a single Isilon Disk Library device using

the following method.

Steps:

1. CommCell console ribbon > Storage tab > Library and Drive – (see Figure 5)


15

2. Click “Add All >>” from the Available MediaAgents, click OK – (see Figure 5)

Figure 5. Create an Isilon Disk Library – Step 1 and 2

3. Click Start (in the Library and Drive Configuration window, bottom-left button) > Add > Disk Library… – (see Figure 6)

Figure 6. Create an Isilon Disk Library – Step 3


16

4. Enter an Alias and click OK – (see Figure 7)


5. Enter the Base Folder (is created), a MediaAgent, directory and share credentials, and

Folder (full UNC path to share and directory), then click OK – (see Figure 8)



17

6. Select the Shared Disk Device tab > right-click Folder > select “Configure for All Selected MediaAgents” – (see Figure 9)


7. Click Yes to configure for all MediaAgents – (see Figure 10)



18

8. This shows the MediaAgents configured with this Isilon Disk Library, the window can be closed now – (see Figure 11)


9. Once completed successfully, the Isilon Disk Library and Folder should be listed in the

CommCell Browser under Storage Resources > Libraries – (see Figure 12)



19

Create Storage Policy

A new Storage Policy should be created that uses the Disk Library created above with

the appropriate aging rules and other settings needed for the environment.

Depending on those needs, modifying the new storage policy and adding the other

MediaAgents as Data Paths to that storage policy with Round-Robin enabled in the

Data Paths Configuration may simplify storage policy management.

Subclient and Schedule Policy

The Subclients should be configured with the Storage Policy created above.

Depending on the needs of the environment, a new Subclient can be created or the

default Subclient can be used. To schedule the Subclient to run on a regular basis,

add it to a new or existing Schedule Policy with a Full and Incremental schedule that

is appropriate for the client.

Deploying in a Virtual Environment

For CommCell health, a vMotion and Storage vMotion like operation must never be

performed on an active CommServe DB. Any automatic mechanisms that trigger

these operations should be disabled on a CS virtual machine. Alerts should be

configured in your virtual environment to ensure you are aware of any such

operations.

Advanced

In this section we will discuss the advanced performance considerations.

CommVault Backup Performance

The backup process is a resource intensive operation, especially the compression and

deduplication process, so the amount of spare CPU, RAM, and hard drive performance

on a client all contribute to its backup performance.

If a client requires the total backup time to be reduced, in addition to monitoring the

CS and MA performance, the client performance should also be monitored during the

time of the backups. Additional CPU and RAM can help with the compression and

deduplication hashing algorithms and therefore reduce the backup time. Obviously

hard drive performance affects how quickly the data can be read during the backup

process and therefore the backup time, so ensure other workload or maintenance

operations are not hampering the backup. If resources cannot be easily increased,

test if offloading the compression and/or deduplication process from the client to the

MediaAgents helps. This will cause more network bandwidth to be consumed

between the client and the MediaAgent, so this must be considered when making

these adjustments.


20

Conclusion

The accelerating amount of data being generated today can have an exponential

effect on the amount of space needed for backups. This creates unprecedented

challenges on storage systems not built with scale-out as a core component of their

technology. Choosing the right storage solution that provides ease of management,

automated distribution, seamlessly scales, saves valuable data center resources, and

turns the inevitable hardware refresh / migration into a simple push-button affair is

critical to providing peace of mind to IT organizations already stretched thin on

resources.

Using the basic deployment method you are able to implement the CommVault

Simpana and EMC Isilon environment with the least amount of effort, provide good

resiliency and performance, and have the information necessary to avoid common

issues. The advanced deployment method provides additional information to

customize, administer, and optimize backup performance.


21

About EMC

EMC Corporation is a global leader in enabling businesses and service providers to

transform their operations and deliver IT as a service. Fundamental to this

transformation is cloud computing. Through innovative products and services, EMC

accelerates the journey to cloud computing, helping IT departments to store,

manage, protect, and analyze their most valuable asset—information—in a more

agile, trusted, and cost-efficient way. Additional information about EMC can be found

at www.EMC.com.

http://www.emc.com/

Documents

h13077 Wp Isilon Backup Using Commvault Bestpractices