83
Isilon OneFS CloudPools Version 8.1.0 Administration Guide

OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

  • Upload
    others

  • View
    6

  • Download
    0

Embed Size (px)

Citation preview

Page 1: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

IsilonOneFS CloudPoolsVersion 8.1.0

Administration Guide

Page 2: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Copyright © 2017 Dell Inc. or its subsidiaries. All rights reserved.

Published May 2017

Dell believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS-IS.“ DELL MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND

WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF

MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. USE, COPYING, AND DISTRIBUTION OF ANY DELL SOFTWARE DESCRIBED

IN THIS PUBLICATION REQUIRES AN APPLICABLE SOFTWARE LICENSE.

Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners.

Published in the USA.

EMC CorporationHopkinton, Massachusetts 01748-91031-508-435-1000 In North America 1-866-464-7381www.EMC.com

2 OneFS CloudPools 8.1.0 Administration Guide

Page 3: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Introduction to this guide 7About this guide........................................................................................... 8Where to go for support............................................................................... 8

Setting up CloudPools 9CloudPools overview...................................................................................10

CloudPools concepts......................................................................10Accessing CloudPools with IsilonSD Edge...................................... 12

Supported cloud providers.......................................................................... 12EMC Isilon...................................................................................... 12EMC ECS Appliance....................................................................... 13Virtustream Storage Cloud............................................................. 13Amazon S3..................................................................................... 13Microsoft Azure............................................................................. 14

CloudPools setup and management............................................................ 14Activating a CloudPools software license....................................... 14Configuring network proxy servers with CloudPools...................... 15Managing cloud storage accounts.................................................. 17Managing CloudPools.....................................................................21Managing CloudPools settings.......................................................24

Managing CloudPools policies 27CloudPools file processing..........................................................................28

Archiving files with file pool policies...............................................28Retrieving file data from the cloud.................................................33

Managing cloud policies..............................................................................34Create a file pool policy for cloud storage (Web UI)...................... 34Create a file pool policy for cloud storage (CLI)............................ 36Modify cloud attributes in a file pool policy (Web UI).................... 36Modify cloud attributes in a file pool policy (CLI)...........................37List file pool policies (CLI)..............................................................37View details of a file pool policy (CLI)............................................ 37Apply a file pool policy to a specified file or path (CLI).................. 38Archive files directly to the cloud (CLI)......................................... 38

Managing CloudPools with other OneFS functions 39Compression and encryption of cloud data.................................................40CloudPools protocol support...................................................................... 40

NFS inline access...........................................................................40SMB inline access.......................................................................... 41Other protocols supporting inline access........................................41

SyncIQ interoperability................................................................................41SyncIQ policies...............................................................................41CloudPools cloud data retention time............................................ 42Managing SmartLink files that have been replicated......................42SyncIQ deep copy..........................................................................43Configuring access to cloud data from a secondary cluster........... 44

NDMP backup and restore of cloud data.................................................... 46

Chapter 1

Chapter 2

Chapter 3

Chapter 4

CONTENTS

OneFS CloudPools 8.1.0 Administration Guide 3

Page 4: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Checking the version of SmartLink files.........................................46CloudPools and snapshots.......................................................................... 47CloudPools and SmartLock.........................................................................47CloudPools and SmartQuotas.....................................................................48CloudPools and SmartDedupe.................................................................... 48

CloudPools tips and troubleshooting 49CloudPools best practices.......................................................................... 50

Use time stamps for cloud data archival and recall........................ 50CloudPools archiving and file size..................................................50Create exclusive accounts for CloudPools purposes......................50

Managing cloud jobs...................................................................................50View a list of cloud jobs (CLI).........................................................51View a cloud job (CLI).................................................................... 51Pause a cloud job (CLI).................................................................. 51Resume a paused cloud job (CLI).................................................. 52Cancel a cloud job (CLI)................................................................ 52

CloudPools troubleshooting........................................................................52CloudPools limitations and expected behaviors............................. 52CloudPools logs............................................................................. 54Troubleshooting CloudPools..........................................................55

CloudPools CLI commands 57CloudPools command reference.................................................................58

isi cloud access add....................................................................... 58isi cloud access list........................................................................ 58isi cloud access remove................................................................. 60isi cloud access view......................................................................60isi cloud accounts create................................................................61isi cloud accounts delete................................................................62isi cloud accounts list.....................................................................63isi cloud accounts modify...............................................................64isi cloud accounts view.................................................................. 65isi cloud archive............................................................................. 66isi cloud jobs cancel....................................................................... 67isi cloud jobs files list......................................................................67isi cloud jobs list.............................................................................68isi cloud jobs pause........................................................................ 69isi cloud jobs resume...................................................................... 70isi cloud jobs view.......................................................................... 70isi cloud pools create..................................................................... 70isi cloud pools delete...................................................................... 71isi cloud pools list........................................................................... 72isi cloud pools modify.....................................................................73isi cloud pools view........................................................................ 74isi cloud proxies create...................................................................74isi cloud proxies delete...................................................................75isi cloud proxies list........................................................................ 76isi cloud proxies modify.................................................................. 77isi cloud proxies view..................................................................... 78isi cloud recall................................................................................ 78isi cloud restore_coi.......................................................................79isi cloud settings modify................................................................ 80isi cloud settings regenerate-encryption-key................................. 81

Chapter 5

Chapter 6

CONTENTS

4 OneFS CloudPools 8.1.0 Administration Guide

Page 5: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

isi cloud settings view.................................................................... 82

CONTENTS

OneFS CloudPools 8.1.0 Administration Guide 5

Page 6: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CONTENTS

6 OneFS CloudPools 8.1.0 Administration Guide

Page 7: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CHAPTER 1

Introduction to this guide

This section contains the following topics:

l About this guide................................................................................................... 8l Where to go for support.......................................................................................8

Introduction to this guide 7

Page 8: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

About this guideThis guide describes CloudPools, a licensed software module that works with Isilon'sOneFS operating system. This guide describes how the CloudPools interface providesaccess to OneFS cloud configuration, operation, and management.

Your suggestions help us to improve the accuracy, organization, and overall quality ofthe documentation. Send your feedback to https://www.research.net/s/isi-docfeedback. If you cannot provide feedback through the URL, send an email messageto [email protected].

Where to go for supportIf you have any questions about EMC Isilon products, contact EMC Isilon TechnicalSupport.

Online Support l Live Chat

l Create a Service Request

TelephoneSupport

l United States: 1-800-SVC-4EMC (1-800-782-4362)

l Canada: 1-800-543-4782

l Worldwide: 1-508-497-7901

l Local phone numbers for a specific country are available atEMC Customer Support Centers.

Supportregistration oraccess

For questions specific to EMC Online Support registration oraccess, email [email protected].

Isilon Info Hubs For the list of Isilon info hubs, see the Isilon Info Hubs page on theEMC Isilon Community Network. Isilon info hubs organize Isilondocumentation, videos, blogs, and user-contributed content intotopic areas, making it easy to find content about subjects thatinterest you.

Support for IsilonSD EdgeIf you are running a free version of IsilonSD Edge, support is available through the Isilon Community Network. If you purchased one or more IsilonSD Edge licenses,support is available through , provided you have a valid support contract for theproduct.

Introduction to this guide

8 OneFS CloudPools 8.1.0 Administration Guide

Page 9: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CHAPTER 2

Setting up CloudPools

This section provides conceptual information about, and procedures for, setting upCloudPools:

l CloudPools overview.......................................................................................... 10l Supported cloud providers..................................................................................12l CloudPools setup and management.................................................................... 14

Setting up CloudPools 9

Page 10: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CloudPools overviewCloudPools extends the capabilities of OneFS by enabling you to specify data to bemoved to lower-cost cloud storage. CloudPools can seamlessly connect to EMC-based cloud storage systems and to popular third-party providers, Amazon S3 andMicrosoft Azure.

CloudPools is a licensed module built on the SmartPools file pool policy framework,which gives you granular control of file storage on your cluster. CloudPools extendsthis file storage control to one or more cloud repositories, which act as additional tiersof OneFS storage.

Prior to the introduction of CloudPools, SmartPools enabled the grouping of nodesinto storage pools called node pools, and the classification of node pools as differentstorage tiers. SmartPools includes a policy framework that allows you to segregatefiles into logical groups called file pools, and to store those file pools in specific storagetiers.

CloudPools expands the SmartPools framework by treating a cloud repository as anadditional storage tier. This enables you to move older or seldom-used data to cloudstorage and free up space on your cluster.

File pool policiesAs with SmartPools, you define files to be stored in the cloud by creating file poolpolicies. These policies use file matching criteria to determine which file pools are tobe moved to the cloud.

File pool policies are applied when the SmartPools system job runs, by default on adaily basis. For each policy, all matched files on the cluster are handled according topolicy specifications.

When files match a file pool policy that contains CloudPools actions, OneFS moves thefile data of matched files to the cloud. Only metadata and a proxy file remain on thecluster, thus freeing up storage space.

SmartLink filesAlthough file data is moved to remote storage, the files remain visible in the OneFS filesystem. CloudPools accomplishes this by retaining a local SmartLink file, which is apointer to the location of data in the cloud. You can read, write, archive, and recallfiles from the cloud as needed.

When a user accesses a cluster and views the OneFS file system through a supportedprotocol (SMB, NFS, Swift, or HDFS), SmartLink files appear to be the original files.When the user opens a SmartLink file, OneFS automatically retrieves and caches asmuch data as needed from the cloud. This operation is called inline access. Anymodifications the user makes to a file during inline access are updated in the file datastored in the cloud.

In addition to inline access, CloudPools also provides a CLI command to enable fullrecall of files from the cloud, in which case the SmartLink files are replaced by theactual files.

CloudPools conceptsCloudPools is a licensed module that enables you to move file data on your Isiloncluster to the cloud, and to access or recall these files when needed. Taking advantage

Setting up CloudPools

10 OneFS CloudPools 8.1.0 Administration Guide

Page 11: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

of CloudPools requires you to configure cloud storage accounts and file pool policiesthat specify cloud storage targets.

You can configure CloudPools to move files to the cloud automatically, based on filepool policies. You can also use a OneFS command to archive individual files to, or recallfiles from, the cloud.

CloudPools uses a similar workflow to OneFS SmartPools. To store files in the cloud,you must have at least one account with a cloud storage provider. In addition, youmust configure OneFS for cloud storage, and create file pool policies that identify andmove files to the cloud.

When the SmartPools job runs, typically once a day, file pool policies are executed,and matched files are sent to the cloud storage target. To access file data stored inthe cloud, you can open its related SmartLink file through any supported protocol(SMB, NFS, Swift, or HDFS). This is referred to as inline access. To fully recall a filefrom the cloud, you can issue an isi cloud recall command from the OneFScommand-line interface.

Following are descriptions of key CloudPools concepts:

Archive

The CloudPools process of moving file data to the cloud. This process involvesextracting the data from the file and placing it in one or more cloud objects.CloudPools then moves these objects to cloud storage, and leaves in place on thelocal cluster a representative file called a SmartLink.

Recall

The CloudPools process of reversing the archival process. When you recall a filefrom the cloud, CloudPools replaces the SmartLink file by restoring the originalfile data on OneFS and removing the cloud objects from cloud storage.

SmartLink file

For every file archived to the cloud, OneFS maintains an associated SmartLink fileon the cluster. A SmartLink file contains metadata and map information so thedata in the cloud can be accessed or fully recalled. If allowed by a SmartLink file'sarchiving policy, accessing the SmartLink file on the cluster automaticallyretrieves and caches data from the cloud. Like other files, SmartLink files can bebacked up through NDMP or synchronized to other clusters with SyncIQ. WhenSmartLink files are retrieved from a backup or SyncIQ operation, CloudPoolsmaintains their links to related file data in the cloud.

File pool policies

File pool policies are the essential control mechanism for both SmartPools andCloudPools. OneFS runs all file pool policies on a regular basis. Each file poolpolicy specifies the files to be managed, actions to take on the files, protectionlevels, and I/O optimization settings.

If CloudPools has been enabled, each file pool policy can also contain cloud-specific parameters that specify the remote cloud account to archive files to, andhow to handle files prior to archiving them. Moreover, a policy can also specifySmartPools targets that specify where to store the locally-retained SmartLinkfiles related to the stored cloud data.

Cloud provider accounts

Making use of cloud storage requires you to set up one or more accounts with acloud provider. The types of cloud storage that are currently supported are EMCIsilon, EMC ECS Appliance, Virtustream Storage Cloud, Amazon S3, and

Setting up CloudPools

CloudPools concepts 11

Page 12: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Microsoft Azure. The account information from the cloud provider must matchthe information you use when configuring cloud accounts on your Isilon cluster.

Cloud storage accounts

A cloud storage account is a OneFS entity that defines access to a specific cloudprovider account. The cloud storage account settings must match the accountcredentials provided by the cloud provider.

CloudPool

A CloudPool is a OneFS entity that contains a single cloud storage account andprovides a conduit between OneFS and the cloud storage repository. Creating aCloudPool requires the availability of at least one cloud storage account. Thecloud storage account must be of the same type as the CloudPool.

Inline access

CloudPools enables users connecting to a cluster through supported protocols toaccess cloud data by opening associated SmartLink files. This process is referredto as inline access. To the user connecting to OneFS through a supportedprotocol, a SmartLink file appears to be the original file. When the user opens aSmartLink file, CloudPools retrieves and caches cloud data locally. The user canview and edit the file as usual. CloudPools automatically retrieves and sends anyupdated file data back to the cloud so that the cloud contains the latest version.

Note

CloudPools offers inline access as a user convenience. However, CloudPools isdesigned mainly as an archival solution, and is not intended for storing data that isfrequently updated. Such data should be left on the local cluster until it stabilizesand is ready for archival.

Accessing CloudPools with IsilonSD EdgeThe CloudPools software module is available only with a purchased license of IsilonSDEdge. It is not packaged with the free license of this product.

Supported cloud providersWith CloudPools, OneFS supports these cloud providers: EMC Isilon, EMC ECSAppliance, Virtustream Storage Cloud, Amazon S3, and Microsoft Windows Azure.

EMC IsilonCloudPools enables an EMC Isilon cluster to function as a cloud storage provider.

In this scenario, a secondary EMC Isilon cluster provides a private cloud solution. Thesecondary cluster archive files from your primary cluster, and is managed in yourcorporate data center.

To act as a cloud storage provider, an EMC Isilon cluster uses a set of APIs thatinclude the capabilities to configure CloudPools policies, define cloud storageaccounts, and retrieve cloud storage usage reports. These APIs are known collectivelyas the Isilon Platform API, and are described in the OneFS 8.0 API Reference.

The secondary cluster must be running the same version of OneFS as the primarycluster and must have active SmartPools and CloudPools licenses.

Setting up CloudPools

12 OneFS CloudPools 8.1.0 Administration Guide

Page 13: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

To configure a secondary EMC Isilon cluster as the cloud storage repository, you needto complete several tasks:

l On the secondary cluster, log on with system administrator privileges, and create anew user.

l On the secondary cluster, create a role with access to Console, Platform API,HTTP, License, Namespace Traverse, and Namespace Access privileges, and makethe new user a member of this role.

l On the secondary cluster, log on as the new user, and create the directory wherecloud data should be stored. For example: /ifs/data/HQ-Archive.

l On the primary cluster, set up the EMC Isilon cloud storage account, specifyingthe new user's credentials and the appropriate URI for the secondary cluster.Because the secondary cluster is within your corporate network, the URI wouldlook similar to the following example:

https://10.1.210.310:8080/namespace/ifs/data/HQ-Archive

l On the primary cluster, create a CloudPool that contains the EMC Isilon cloudstorage account.

EMC ECS ApplianceCloudPools supports EMC ECS (Elastic Cloud Service) appliance as a cloud provider.

ECS is a complete software-defined cloud storage platform deployed on a turn-keyappliance from EMC. It supports the storage, manipulation, and analysis ofunstructured data on a massive scale.

The ECS appliance is specifically designed to support mobile, cloud, big data, andnext-generation applications. As an appliance, it is simple to install and deploy withsupport for multi-tenancy, self-service access, usage metering, on-demand cloudstorage-as-a-service and dynamic application provisioning.

Virtustream Storage CloudCloudPools supports Virtustream Storage Cloud as a cloud provider.

Virtustream Storage Cloud (VSC) is a managed cloud computing service from EMCthat offers benefits associated both with a dedicated private cloud and a public multi-tenant cloud.

VSC enables enterprises to run complex, mission-critical applications with full cloudagility, economy, and automation, and to achieve enterprise-class service-levelagreements for both application performance and availability. In addition, Virtustreamprovides a foundation for meeting national and industry-level security, compliance andauditing requirements.

Virtustream Storage Cloud can be managed in your own data center, or provided as asolution from a third party.

Amazon S3CloudPools can be configured to store data on Amazon S3 (Simple Storage System),a public cloud provider.

When you configure CloudPools to use Amazon S3 for cloud storage, in addition toURI, username, and passkey, you must specify the following additional attributes.

l S3 Account ID

Setting up CloudPools

EMC ECS Appliance 13

Page 14: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

l S3 Telemetry Reporting Bucket

l S3 Storage Region

When you first establish an account with Amazon S3, the cloud provider gives you anaccount ID and allows you to choose a storage region. Amazon S3 offers multiplestorage regions in the U.S. and other regions of the world.

Note

CloudPools supports Amazon Web Services Signature V2 to authenticate queries toits cloud storage. CloudPools does not support Signature V4.

To work with CloudPools, you must also identify an S3 telemetry reporting bucket.This is where billing reports are stored on Amazon S3. This bucket must be accessibleto CloudPools.

To set up an S3 telemetry reporting bucket, navigate to Billing & Cost Managementpreferences in the S3 console. There you can indicate that you want to receive billingreports, and specify the bucket to which these reports should be saved. Use thisbucket name as the telemetry reporting bucket when setting up an S3 cloud storageaccount in CloudPools.

Microsoft AzureYou can configure CloudPools to store data on Microsoft Azure, a public cloudprovider

When you first establish an account with Microsoft Azure, you create a user name,and Microsoft provides you with a URI and a passkey. When you configure CloudPoolsto use Azure, you must specify the same URI, username and passkey.

CloudPools setup and managementSetting up and managing CloudPools includes activating licenses, configuring networkproxies, and managing accounts and storage pools.

Activating a CloudPools software licenseYou can activate a CloudPools license from either the web interface or the CLI.Running CloudPools also requires the activation of a SmartPools license.

For complete information about obtaining and activating OneFS licenses, see the IsilonOneFS Web Administration Guide.

Upload the updated license file (Web UI)After you receive an updated license file from EMC Software Licensing Central (SLC),upload the updated file to your cluster.

Procedure

1. Click Cluster Management > Licensing.

2. In the Upload and activate a signed license file area, click Browse and selectyour signed license file.

3. Click Upload and Activate.

Setting up CloudPools

14 OneFS CloudPools 8.1.0 Administration Guide

Page 15: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Upload the updated license file (CLI)After you receive an updated license file from EMC Software Licensing Central (SLC),upload the updated file to your cluster.

Procedure

1. Run the isi license add command.

The following command adds the /ifs/local license file to the cluster:

isi license add --path /ifs/local

Configuring network proxy servers with CloudPoolsYou can configure CloudPools so that data that is archived to, or recalled from, apublic cloud provider is routed through a proxy server.

By default, CloudPools communicates directly with the designated cloud provider. Ifthe cloud provider is private, such as another Isilon cluster or an EMC ECS appliancerunning on the same corporate network, the default communication protocol might beacceptable.

However, if CloudPools is archiving data to a public cloud provider, such as AmazonS3 or Microsoft Azure, communication happening directly through the public Internetmight violate security policies that are established by some organizations.

In a typical configuration, the Isilon cluster is installed in a data center behind one ormore firewalls. Ports that would enable communication to the public Internet often areclosed. To enable CloudPools to archive data to a public cloud provider, CloudPoolscan be configured to work with a proxy server.

CloudPools works with proxy servers running the following protocols:

l SOCKS v4

l SOCKS v5

l HTTP

Configuration on the CloudPools side includes creating the network proxy andconnecting the network proxy to a cloud storage account. Both SOCKS v5 and HTTPcan be configured with or without authentication. SOCKS v4 does not supportauthentication.

From OneFS, you can also list network proxies, view network proxy properties, modifyproxy settings, and delete proxies. Except for connecting the network proxy to a cloudstorage account, you must use the CLI to run all other proxy server commands.

Create a network proxy (CLI)You can create a network proxy to redirect CloudPools traffic to and from a publiccloud provider. CloudPools supports proxy servers running the SOCKS v4, SOCKS v5,and HTTP protocols.

Before you begin

The proxy server should be online and ready to accept a connection from an Isiloncluster.

Procedure

1. Run the isi cloud proxies create command.

Setting up CloudPools

Configuring network proxy servers with CloudPools 15

Page 16: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

The following command creates a proxy object named myproxy1 and links it toa specific proxy server URL, proxy type, and port:

isi cloud proxies create myproxy1 10.99.58.250 socks_5 1080

Results

When you later create or modify a cloud storage account, the myproxy1 networkproxy is available. At that time, when select the proxy, and save the changes,CloudPools verifies that the proxy server connection can be made.

View a list of network proxies (CLI)You can view a list of existing network proxies in CloudPools.

Before you begin

You or someone in your organization must first have created network proxies using theisi cloud proxies create command.

Procedure

1. Run the isi cloud proxies list command.

The command displays a list of proxy names, hosts, and types.

View network proxy properties (CLI)You can view the properties of a network proxy.

Before you begin

You or someone in your organization must have created a network proxy using theisi cloud proxies create command.

Procedure

1. Run the isi cloud proxies view command.

The following command displays the properties of a proxy named myproxy1:

isi cloud proxies view myproxy1

Properties shown include ID, name, host, type, and port.

Modify a network proxy (CLI)You can modify the properties of an existing network proxy in CloudPools.

Before you begin

You or someone in your organization must have created the network proxy using theisi cloud proxies create command.

Procedure

1. Run the isi cloud proxies modify command.

Setting up CloudPools

16 OneFS CloudPools 8.1.0 Administration Guide

Page 17: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

The following command adds a user name and password necessary to connectto a network proxy:

isi cloud proxies modify myproxy1 --username cloud1 --password @xy16+RZ20

Results

You can now add the network proxy to a cloud storage account.

Delete a network proxy (CLI)You can delete an existing network proxy in CloudPools. However, if the proxy isconnected to a cloud storage account, you cannot delete the proxy.

Before you begin

You or someone in your organization must have created the network proxy using theisi cloud proxies create command.

Procedure

1. Run the isi cloud proxies delete command.

The following command deletes the proxy named myproxy1:

isi cloud proxies delete myproxy1

OneFS asks you to confirm the deletion:

Are you sure? (yes/no):

2. Type yes and press ENTER.

Results

If the proxy is already connected to a cloud storage account in CloudPools, OneFSprevents you from deleting the proxy. Otherwise, the proxy is deleted.

Managing cloud storage accountsA cloud storage account provides OneFS with the information it needs to connect tothe remote cloud storage provider.

You can create and edit one or more cloud storage accounts in OneFS.

Create cloud storage accounts (Web UI)You define cloud storage accounts on your OneFS cluster as an essential part ofCloudPools configuration. The account username, password, and URI that you used toestablish an account with your cloud provider is required. You can also specify a proxyserver to redirect CloudPools archive and retrieval traffic to and from a public cloudprovider.

Procedure

1. Click File System > Storage Pools > CloudPools.

2. Click + Create a Cloud Storage Account.

3. In the Create a Cloud Storage Account dialog box, perform these actions:

Setting up CloudPools

Managing cloud storage accounts 17

Page 18: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

a. In the Name or Alias field, enter a name for the account.

b. In the Type drop-down menu, select a type of cloud account. Choices areEMC Isilon, EMC ECS Appliance, Virtustream Storage Cloud,Microsoft Azure, and Amazon S3.

c. In the URI field, enter a fully qualified URI for the account. The URI must usethe HTTPS protocol, and match the URI used to set up the account withyour cloud provider.

d. In the User Name field, enter the account user name, which must be thesame as the user name provided to or by the cloud provider.

e. In the Key field, enter the account password. The password must be thesame as the password that you provided to the cloud provider, or the keythat the cloud provider issued to you.

f. If you have already defined one or more network proxies, and want to useone for this cloud account, select the name from the Proxy drop-down box.

g. If you are creating an Amazon S3 account, you must also specify an S3Account ID, S3 Telemetry Reporting Bucket, and an S3 StorageRegion.

4. Click the Connect Account button.

The Create a Cloud Storage Account dialog box closes, and the new cloudaccount appears in the Cloud Storage Accounts list. A cloud account isrepresented by an orange user icon. The Name, Type, State, Username, andURI associated with the account is displayed.

Create a cloud storage account (CLI)You create cloud storage accounts to enable CloudPools to archive files to cloudstorage. The account username, password, and URI that you used to establish anaccount with your cloud provider is required. You can also specify a proxy server toredirect CloudPools archive and retrieval traffic to and from a public cloud provider.

Before you begin

Before creating a cloud storage account, you must establish an account with a cloudprovider, such as EMC Elastic Cloud Storage Service. When you create a cloudstorage account in OneFS, the system attempts to connect to the cloud providerusing the credentials you provide. Also, if you want to specify a proxy server with thecloud storage account, you must have already created the proxy server by means ofthe isi cloud proxies create command.

OneFS enforces the following requirements for cloud storage accounts.

l Each cloud storage account can only belong to a single CloudPool storagecontainer.

l A cloud storage account must be of the same type (EMC Isilon, EMC ECSAppliance, Virtustream Storage Cloud, Amazon S3, or Microsoft Azure) as theCloudPool.

To create a cloud storage account:

Procedure

1. Run the isi cloud accounts create command.

Setting up CloudPools

18 OneFS CloudPools 8.1.0 Administration Guide

Page 19: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

The following command creates a Microsoft Azure cloud storage account. Notethat this type of account requires a key provided by the cloud provider.

isi cloud accounts create --name=c-acct1 --type=azure --uri=https://admin2.blob.core.windows.net --account-username=adm1 --key=!$P@$$c0de998==

The following command creates the same account, while specifying a proxyserver.

isi cloud accounts create --name=c-acct1 --type=azure --uri=https://admin2.blob.core.windows.net --account-username=adm1 --key=!$P@$$c0de998== --proxy myproxy1

After you finish

After the cloud storage account successfully connects to the cloud provider, you mustadd the cloud storage account to a CloudPool in OneFS. OneFS is then able to archivefiles to the cloud.

Edit a cloud storage account (Web UI)You can edit some of the settings of an existing cloud storage account.

Procedure

1. Click File System > Storage Pools > CloudPools.

2. In the Cloud Storage Accounts list, click the View/Edit button to the right ofthe account that you want to edit.

3. In the View Cloud Storage Account Details dialog box, click the Edit Accountbutton.

4. In the Edit Cloud Storage Account Details dialog box, perform any of theseactions:

a. In the Name or Alias field, enter a new name for the account. You cannotchange the type of account.

b. In the URI field, enter a fully qualified URI for the account. The URI must usethe HTTPS protocol, and match the URI used to set up the account withyour cloud provider.

c. In the Username field, enter the account username, which must be the sameas the username provided to the cloud provider.

d. In the Key field, enter the account password. The password must be thesame as the password that you provided to the cloud provider, or the keythat the cloud provider issued to you.

e. If you want to use a different proxy server for this cloud account, select thename of the new proxy from the Proxy drop-down box.

f. If you are editing an Amazon S3 account, you can also specify a new S3Account ID and S3 Telemetry Reporting Bucket. You cannotchange the S3 Storage Region.

5. Click the Save Changes button.

Setting up CloudPools

Managing cloud storage accounts 19

Page 20: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CloudPools validates that your cloud data is still accessible. Otherwise, it alertsyou and does not save the changes.

Modify a cloud storage account (CLI)You can modify the information associated with a cloud storage account.

To modify a cloud storage account, you must specify the account name. You can runthe isi cloud accounts list command to list cloud storage accounts.

Procedure

1. Run the isi cloud accounts modify command.

This sample command changes the name of the cloud storage accountCloudAcct3 to CloudAcct5 and specifies a proxy server through whichcommunications with the public cloud provider are to be managed.

isi cloud accounts modify CloudAcct3 --name=CloudAcct5 --proxy myproxy1

Delete a cloud storage account (CLI)You can delete a cloud storage account. However, proceed with extreme caution, asdeleting an account can result in loss of access to cloud data.

You can stop archiving data to a cloud storage account without deleting it by runningthe isi cloud pools modify command and removing the account from its parentCloudPool. Previously archived files remain in cloud storage, and SmartLink files onthe local cluster still point to the cloud data.

Procedure

1. Run the isi cloud accounts delete command.

The following command deletes the cloud storage account OldRecords.

isi cloud accounts delete OldRecords --acknowledge yes

In this case, OneFS responds with the following message:

***************************************************WARNING: Deleting an account is extremely dangerous.Continuing with this operation will result in a permanent loss of data.Type 'confirm delete data' to proceed. Press enter to cancel:

2. Type the confirmation string confirm delete data, then press ENTER.

The cloud storage account is deleted. Although cloud data remains with yourcloud provider, the data will no longer be accessible in OneFS.

List cloud storage accounts (CLI)You can list all cloud storage accounts created on your cluster.

Procedure

1. Run the isi cloud accounts list command.

Setting up CloudPools

20 OneFS CloudPools 8.1.0 Administration Guide

Page 21: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

A sample list of cloud accounts is shown with name, type, account username,URI, and status.

View a cloud storage account (CLI)You can view detailed information about a cloud storage account.

Procedure

1. Run the isi cloud accounts view command.

The following command displays account information for the CloudAcct3account.

isi cloud accounts view CloudAcct3

Output from the command displays the properties of the cloud storage account,including account name, type, and more.

Managing CloudPoolsA CloudPool contains one or more cloud storage accounts and enables OneFS to usecloud storage as simply another tier of storage available to the cluster.

You can create, view, edit, and monitor CloudPools.

Create a CloudPool (Web UI)You can create a CloudPool and add a cloud storage account.

Procedure

1. Click File System > Storage Pools > CloudPools.

2. Click the + Create a CloudPool button.

3. In the Create a CloudPool dialog box, in the Name field, enter a name for theCloudPool. The name must be unique on your cluster.

4. From the Type drop-down menu, select a type of cloud account, one of EMCIsilon, EMC ECS Appliance, Virtustream Storage Cloud,Microsoft Azure, or Amazon S3.

5. Enter a vendor and description for the CloudPool.

6. From the Account in CloudPool list, select the cloud storage account that thisCloudPool should contain. The list is empty until you select a value from theType list. The Account in CloudPool list then shows only those cloud storageaccounts that match that type, for example, Azure.

7. Click Create a CloudPool .

The dialog box closes and, in the CloudPools list, the new CloudPool isdisplayed along with its type, state, vendor, and description.

Create a CloudPool (CLI)You can create a CloudPool and add a cloud storage account.

A CloudPool is the mechanism that connects a cloud storage account to OneFS. Whenyou create a CloudPool, OneFS enforces two requirements:

l The CloudPool may contain only one cloud storage account.

Setting up CloudPools

Managing CloudPools 21

Page 22: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

l The cloud storage account must be of the same type as the CloudPool. Forexample, an Azure CloudPool may only contain an Azure cloud storage account.

Procedure

1. Run the isi cloud pools create command.

When you create a CloudPool, you need to provide a unique name, theCloudPool type (isilon, ecs, virtustream, azure, or s3), and one cloud account.The following command creates an Azure-based CloudPool:

isi cloud pools create cp_az azure csa_azure1 --vendor Microsoft

Results

You can view the result of this operation by running the isi cloud pools viewcommand with the ID (name) of the CloudPool that you created, as shown in thefollowing example:

isi cloud pools view cp_az

The output displays the CloudPool ID, name, type, and other properties.

View information about a CloudPool (Web UI)You can view information about a CloudPool, including the cloud storage account,vendor, type, and description.

Procedure

1. Click File System > Storage Pools > CloudPools.

In the CloudPools list, each CloudPool is represented by a blue cloud.Associated cloud accounts are listed below each CloudPool, and represented byan orange user icon. The type, vendor, and description associated with eachCloudPool is displayed.

2. To further view the settings of a CloudPool, click View/Edit to the right of theCloudPool.

The View Cloud Storage Pool Details dialog box displays information about theCloudPool.

3. Click Close to close the dialog box.

View information about a CloudPool (CLI)You can view information about a CloudPool, including the cloud storage account,vendor, type, and description.

Before you begin

The CloudPool must already have been created.Procedure

1. Run the isi cloud pools view command.

The following command provides information on the CloudPool namedcah_s3_cp.

isi cloud pools view cah_s3_cp

Setting up CloudPools

22 OneFS CloudPools 8.1.0 Administration Guide

Page 23: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

The output of this command displays the ID, name, type, and other CloudPoolproperties.

Modify a CloudPool (Web UI)You can modify a CloudPool, changing the name, the account it contains, the cloudvendor, and the description.

Procedure

1. Click File System > Storage Pools > CloudPools.

In the CloudPools list, each CloudPool is represented by a blue cloud icon. Thecloud account associated with each CloudPool is listed and represented by anorange user icon. The type, vendor, and description are also displayed.

2. Click View/Edit to the right of the CloudPool that you want to modify.

The View Cloud Storage Pool Details dialog box appears.

3. Click the Edit CloudPool button.

The Edit CloudPool Details dialog box appears.

4. Modify the name, vendor, or description fields, as intended.

5. From the Account in CloudPool drop-down list, select a different account of thesame type.

6. Click the Save changes button.

7. In the View Cloud Storage Pool Details dialog box, click Close.

Results

Any changes that you made to the CloudPool are reflected in the CloudPools list.

Modify a CloudPool (CLI)You can modify a CloudPool, changing the name, the account it contains, the cloudvendor, and the description.

Before you begin

To determine the available CloudPools on your system, run the isi cloud poolsview command.

Procedure

1. Run the isi cloud pools modify command.

The following command modifies a CloudPool named c_pool_azure, removingits cloud storage account

isi cloud pool modify c_pool_azure --remove-accounts c_acct2 --description "Secondary archive"

Setting up CloudPools

Managing CloudPools 23

Page 24: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Delete a CloudPool (CLI)You can delete a CloudPool. However, you should proceed with caution. CloudPoolsprovide the mechanism to connect OneFS to your cloud storage accounts. If youdelete a CloudPool, the associated cloud storage account is no longer accessible.

Before you begin

Run the isi cloud pools list command to display the names of the CloudPoolson your cluster. Run the isi cloud pools view command, along with the name toget information about a CloudPool.

Procedure

1. Run the isi cloud pools delete command.

The following command deletes the CloudPool named c_pool_azure.

isi cloud pools delete c_pool_azure

OneFS asks you to confirm the deletion, as follows:

Are you sure? (yes/[no]):

2. Type yes and press ENTER.

The CloudPool is deleted.

Monitoring CloudPools (Web UI)You can monitor the health of CloudPools configured on your cluster.

Procedure

1. Click File System > Storage Pools > Summary.

2. In the Status list, check the status for CloudPools.

Status conditions for CloudPools are Good or Needs Attention. A status ofNeeds Attention appears when a CloudPool cannot connect to the remotecloud provider. This could indicate issues with the Internet connection or withthe cloud provider. If you confirm that your Internet connection is good, contactyour cloud provider for help.

Managing CloudPools settingsYou can manage CloudPools default settings, including snapshot archival, encryption,compression, cache settings, data retention settings, and the ability to regenerate anencryption key. An encryption key should only be regenerated if you suspect theexisting key has been compromised.

View cloud settings (CLI)You can view the top-level settings for CloudPools.

Procedure

1. Run the isi cloud settings view command.

The command displays CloudPools settings such as accessibility, cacheexpiration, whether compression and encryption is enabled, and so on.

Setting up CloudPools

24 OneFS CloudPools 8.1.0 Administration Guide

Page 25: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Modify default cloud settings (CLI)You can modify default CloudPools settings.

You can use the isi cloud settings view command to display current defaultsettings.

Procedure

1. Run the isi cloud settings modify command.

The following command disables archival of files that have snapshot versions.

isi cloud settings modify --default-archive-snapshot-files=no

The following command enables both encryption and compression of clouddata:

isi cloud settings modify --default-encryption-enabled=yes--default-compression-enabled=yes

Generate a new master encryption key (CLI)You can generate a new master encryption key. The key is used to encrypt data and isstored with cloud data objects.

Before you begin

Only generate a new master encryption key if you believe the existing key has beencompromised.

Procedure

1. Run the isi cloud settings regenerate-encryption-key command.

The following command generates a new encryption key in verbose mode.

isi cloud settings regenerate-encryption-key --verbose

In verbose mode, the system confirms the process:

Encryption key has been regenerated

Setting up CloudPools

Managing CloudPools settings 25

Page 26: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Setting up CloudPools

26 OneFS CloudPools 8.1.0 Administration Guide

Page 27: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CHAPTER 3

Managing CloudPools policies

This section describes how to define and maintain the file pool policies that archivefiles to cloud storage.

l CloudPools file processing................................................................................. 28l Managing cloud policies..................................................................................... 34

Managing CloudPools policies 27

Page 28: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CloudPools file processingCloudPools archives file data to the cloud, and enables you to access or fully recall thisdata whenever needed.

You can create file pool policies that identify the files to be archived to the cloud.When a file pool policy that contains cloud actions is run, CloudPools moves file datato the cloud and stores it in specialized cloud data objects, collectively referred to ascloud data. File data can be encrypted and compressed before it is archived to thecloud.

In place of each file that is archived, CloudPools retains a local proxy called aSmartLink file. SmartLink files include special metadata and maps to the actual filedata in the cloud.

When a user browses OneFS, typically through an SMB connection or NFS export,SmartLink files appear in place as the files they link to. When a user opens a SmartLinkfile, a process referred to as inline access, CloudPools retrieves file data from thecloud and caches it locally.

The user can view the file, and CloudPools continues caching as much of the data asneeded by the application. If the user modifies and saves the file, the changes are alsoheld in cache. Periodically, CloudPools scans SmartLink files for pending data changesand writes them to the appropriate objects in the cloud. In this way, the archived datais kept up to date.

You can also recall archived files from the cloud. When you do, SmartLink files are fullyreplaced by the recalled files.

Like any file in OneFS, SmartLink files are controlled either by the default file poolpolicy or by parameters included in a custom file pool policy. If you configure additionalfile pool policies, these policies have priority over the default file pool policy.

File pool policies contain instructions that determine how OneFS manages files acrossa cluster and in the cloud.

Because SmartLink files produced by CloudPools are retained on the cluster, OneFSapplies file pool policies to these files, as well.

When file pool policies run, the system compares each file on the system with each filepool policy. A file can match only some aspects of a custom file pool policy (forexample, SSD strategy and snapshot configuration). In this case, those aspects of filehandling are governed by the custom file pool policy, and all other aspects aregoverned by the default file pool policy.

Refer to the SmartPools section in the OneFS Web Administration Guide for additionalinformation about file pool polices.

Archiving files with file pool policiesYou can configure a file pool policy to identify the files you want to archive to thecloud and the CloudPools actions to apply to these files.

Specifying a file pool policy, you can archive files using either the OneFS webadministration interface or the command-line interface. A file pool policy that archivesfiles to the cloud must specify the following information:

l Files to manage: These can be files of a certain type, files in a specified path, orfiles that match specified criteria, such as size, creation date, or last modifieddate.

Managing CloudPools policies

28 OneFS CloudPools 8.1.0 Administration Guide

Page 29: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

l CloudPools actions: The cloud storage pool to send file data to, and whether thedata should be compressed or encrypted.

Sample policies with CloudPools actionsEach file pool policy identifies a set of files and the CloudPools actions to apply to thefile pool. You can identify files to be archived based on multiple criteria, including filetype, size, directory path, time of file creation, time of last file access, and time of lastfile modification.

File-matching criteria in a file pool policy enable you to define a logical group of filesreferred to as a file pool. After defining a file pool, you specify CloudPools actions toperform on the files, including the cloud storage target, compression, and encryption.

For example, you might define file pool policies that specify files to be archived basedon criteria similar to the following:

l Files of <type>, last accessed before <date>

l Files older than <date>, last accessed after <date>, and of <type>

l Files in <directory> that are older than <date>

l Files marked with <custom attribute>, that are older than <date>

You can specify file-matching criteria on a per-policy basis. Each file pool policy allowsyou to combine multiple criteria using AND statements and OR statements, providingsignificant flexibility and control for your workflow.

Combining cloud and local storage policy actionsYou can specify both cloud and a local storage actions in the same file pool policy. Thecloud actions are applied to the data of matching files, while the local actions apply tothe SmartLink files that are created in place.

SmartPools settings can determine the target storage pool or tier, file protection level,I/O optimization, and data access optimization. The SmartLink files are processedaccording to the specified SmartPools parameters. If some settings are not specifiedin the custom file pool policy, the default file pool policy settings are applied to theSmartLink files.

About file pool policy orderOneFS compares all files to file pool policies in order. The first custom policy thatmatches a file controls how that file is handled. All other custom file pool policies inthe ordered list are ignored. For any of the attributes that the matching custom policydoes not specify, the value from the default policy is applied.

This makes the order of file pool policies important. If two or more file pool policieswould match the same file, you must ensure that the policy order delivers yourpreferred file handling instructions.

After a file match with a file pool policy occurs, the system uses the settings in thematching policy to store and protect the file. However, a matching policy might notspecify all settings for the match file. In this case, the default policy is used for thosesettings not specified in the custom policy. For each file stored on the OneFS cluster,the system needs to determine the following:

l Requested protection level

l Data storage target for local data cache

l SSD strategy for metadata and data

l Protection level for local data cache

Managing CloudPools policies

Archiving files with file pool policies 29

Page 30: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

l Configuration for snapshots

l SmartCache setting

l L3 cache setting

l Data access pattern

l CloudPools actions (if any)

If no custom policy matches a file, the default policy specifies all storage settings forthe file. The default policy, in effect, matches all files not matched by any otherSmartPools policy. For this reason, the default policy is the last in the file pool policylist, and always the last policy the system applies.

Files that have been archived to the cloud are always governed by the original policy.

File pool policy cloud archive parametersCloudPools provides a specific set of file pool parameters that support archiving filesto the cloud. The following table lists and describes these parameters.

Web admin parameter CLI parameter Description Usage notes

CloudPool Storage Target cloud-pool An Isilon administrativecontainer for a cloud storageaccount.

Each CloudPool can contain only onecloud storage account with a cloudprovider. You must create a cloudstorage account before creating andconfiguring a CloudPool. A CloudPooland its contained cloud storageaccount must be of the same type:EMC Isilon, EMC ECS Appliance,Virtustream Storage Cloud, AmazonS3, or Microsoft Azure.

Encrypt data beforetransfer

cloud-encryption-enabled

Specifies whether CloudPoolsencrypts data prior toarchiving it

The default value isdisabled

Specifies whether data is encryptedprior to archiving to the cloud. Clouddata is decrypted when accessed orrecalled.

Compress data beforetransfer

cloud-compression-enabled

Specifies whether CloudPoolscompresses data prior toarchiving it.

The default value isdisabled

Specifies whether data is compressedprior to archiving to the cloud. Clouddata is decompressed when accessedor recalled.

Cloud Data RetentionPeriod

cloud-data-retention The length of time cloud filesare retained after the fileshave been fully recalled.

The default value is 1 week.

Specifies how long cloud objects areretained after a SmartLink file hasbeen replaced by the recalled file.When this happens, CloudPools cleansup local resources allocated for theSmartLink files, and also removes theassociated cloud objects. This work isperformed weekly by the cloud objectsgarbage collector job.

Managing CloudPools policies

30 OneFS CloudPools 8.1.0 Administration Guide

Page 31: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Web admin parameter CLI parameter Description Usage notes

Note

The system removes (garbage-collects) cloud objects when theirSmartLink files and all local referencesto them have been removed. If aSmartLink file has been backed up andthe original SmartLink file issubsequently deleted, associatedcloud objects are deleted only afterthe retention time of the backed-upSmartLink file has expired.

Incremental BackupRetention Period forNDMP IncrementalBackup and SyncIQ

cloud-incremental-backup-retention

Specifies the length of timethat OneFS retains cloud datareferenced by a SmartLinkfile that has been replicatedby SyncIQ or an incrementalNDMP backup.

The default value is 5years.

If a SmartLink file has been backed upand the original SmartLink file issubsequently deleted, associatedcloud objects are deleted only afterthe retention time of the backed-upSmartLink file has expired.

Full Backup RetentionPeriod for NDMP Only

cloud-full-backup-retention

Specifies the length of timethat OneFS retains cloud datareferenced by a SmartLinkfile that has been backed upby a full NDMP backup.

The default value is 5years.

If a SmartLink file has been backed upand the original SmartLink file issubsequently deleted, associatedcloud objects are deleted only afterthe original retention time, or a longerincremental or full backup retentionperiod, has expired.

Writeback Frequency cloud-writeback-frequency

Specified the interval atwhich the system writes thedata stored in the cache ofSmartLink files to the cloud.

The default value is9 hours

Specifies how often SmartLink filesmodified on the cluster are written totheir associated cloud data objects.

Accessibility cloud-accessibility Specifies how data is cachedin SmartLink files when a useror application accesses aSmartLink file on the cluster.Values are cached and no-cache.

The default value is cached

Determines whether cloud data iscached when a file is accessed on thelocal cluster.

cached

When cached is selected,

accessed cloud data is cached tothe SmartLink file on read orwrite access.

no-cache

When no-cache is selected, the

system does not cache data inthe SmartLink files on readaccess, but passes it through tothe local accessing application. If

Managing CloudPools policies

Archiving files with file pool policies 31

Page 32: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Web admin parameter CLI parameter Description Usage notes

you write to data accessed whenthis setting applies, the systemcaches your changes. Chooseno-cache if you want to limit

the use of cluster resources.

Cache Read Ahead cloud-readahead Specifies the cachereadahead strategy for cloudfiles (one of partial or full)

The default value is partial

Specifies whether cloud data is fully orpartially recalled when you access aSmartLink file on the cluster. Ifpartial is specified, the system only

recalls the file blocks needed when aSmartLink file is accessed. If full is

specified, all cloud data is fully cachedwhen the SmartLink file is accessed.

Cache Expiration cloud-cache-expiration Specifies the number of daysuntil the system purgesexpired cache information inSmartLink files.

The default value is 1 day.

Specifies how long the system retainscloud data that has been recalled inthe cache of associated SmartLinkfiles. The system purges theSmartLink file cache of data that hasnot been accessed for the number ofdays specified.

Archive Files withSnapshots

cloud-archive-snapshot-files

Specifies whether the policyshould archive files withsnapshots. The default is on.

Snapshots capture files at a momentin time. If a policy matches a file that isincluded in a snapshot, the file can bearchived or not depending on thissetting.

File matching options for cloud archival policiesEach file pool policy must provide match criteria to identify the files to archive and thecloud target where the files should be stored.

The following table describes the match criteria to use when creating file pool policies.

Match criteria Description

Web admininterface

Command lineinterface

Filename --name Includes or excludes files based on the file name.You can specify whether to include or exclude full or partial names thatcontain specific text. Wildcard characters are supported.

Path --path Includes or excludes files based on the file path.You can specify whether to include or exclude full or partial paths that

contain specified text. You can also include the wildcard characters *, ?,

and [ ].

File Type --file-type Includes or excludes files based on one of the following file-system objecttypes:

l Regular file

l Directory

Managing CloudPools policies

32 OneFS CloudPools 8.1.0 Administration Guide

Page 33: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Match criteria Description

Web admininterface

Command lineinterface

l Other

File Attribute --custom-attribute Includes or excludes files based on a custom user-defined attribute.

Modified --changed-time Includes or excludes files based on when the file was last modified.You can specify a relative date and time, such as "older than 2 weeks," ora specific date and time, such as "before January 1, 2012." Time settingsare based on a 24-hour clock.

Accessed --accessed-time Includes or excludes files based on when the file was last accessed.You can specify a relative date and time, such as "older than 2 weeks," ora specific date and time, such as "before January 1, 2012." Time settingsare based on a 24-hour clock.

Note

Because it affects performance, access time tracking as a file pool policycriterion is disabled by default.

Metadata Changed --metadata-changed-time

Includes or excludes files based on when the file metadata was lastmodified. This option is available only if the global access-time-trackingoption of the cluster is enabled.You can specify a relative date and time, such as "older than 2 weeks," ora specific date and time, such as "before January 1, 2012." Time settingsare based on a 24-hour clock.

Created --birth-time Includes or excludes files based on when the file was created.You can specify a relative date and time, such as "older than 2 weeks," ora specific date and time, such as "before January 1, 2012." Time settingsare based on a 24-hour clock.

Size --size Includes or excludes files based on their size.

Note

File sizes are represented in multiples of 1024, not 1000.

Retrieving file data from the cloudYou can retrieve file data from the cloud either by inline access through a supportedprotocol (SMB, NFS, Swift, or HDFS), or by fully recalling files.

Inline access of cloud dataUsers can retrieve file data in the cloud by accessing a SmartLink file on the localcluster through a supported protocol. This method is referred to as inline access.

When the user opens a SmartLink file, for example, through an SMB share, CloudPoolsretrieves and locally caches file data from the cloud. The amount of data that iscached is determined by the CloudPools Cache Read Ahead setting.

Managing CloudPools policies

Retrieving file data from the cloud 33

Page 34: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

If the user makes changes to the file, CloudPools maintains those changes in cacheand periodically updates file data to the cloud so that the latest version is alwaysarchived.

Recalling files from the cloudYou can fully recall a file from cloud storage. In this case, CloudPools restores the fullfile to the cluster and overwrites its associated SmartLink file. As part of a dailymaintenance routine, CloudPools also completely removes the recalled file data fromthe cloud.

You can recall files from cloud storage only with the CLI command isi cloudrecall. You can recall files individually by name or by specifying a fully recursivedirectory path.

Note

When you use the isi cloud recall command to recall a file from cloud storage,the full file is restored to its original directory. If the file pool policy that originallyarchived the file to the cloud is still in effect, the next time the SmartPools job runs,the recalled file is archived to the cloud again. If you do not want the recalled file to bere-archived, you can move the file to a different directory that would not be affectedby the file pool policy, or you can modify or delete the policy.

Managing cloud policiesCloudPools takes advantage of the SmartPools infrastructure, and applies file poolpolicies to determine which files are to be archived to the cloud.

Consequently, you must activate both a SmartPools and a CloudPools license to storedata in the cloud.

By defining file pool policies, you can have OneFS automatically archive files to thecloud when they match certain characteristics, such as age, size, type, or location.

File pool policies are both for SmartPools and CloudPools purposes. A file pool policycan specify a local storage target, a cloud storage target, or both. If you create apolicy that specifies both local and cloud targets, the policy moves file data to thecloud, and applies the local settings to the SmartLink files retained on the local cluster.

If the purpose of a file pool policy is to move files to a local node pool or tier, do notconfigure a cloud target. Conversely, if the purpose of a policy is to archive files to thecloud, configuring a local target, while allowed, is not necessary. In this case, thesystem uses the settings of the default file pool policy to store the local SmartLinkfiles.

Create a file pool policy for cloud storage (Web UI)You can create file pool policies that specify CloudPools actions to be applied toselected files.

Procedure

1. Click File System > Storage Pools > File Pool Policies.

2. Click the + Create a File Pool Policy button.

The Create a File Pool Policy dialog box displays.

3. Enter a policy name and, optionally, a description.

Managing CloudPools policies

34 OneFS CloudPools 8.1.0 Administration Guide

Page 35: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

4. In the Select Files to Manage area, use the pull-down menus to specify the fileselection criteria for cloud storage. The criteria you specify are used by OneFSto determine the files to archive. The criteria you specify for file selection caninclude the following attributes, combined with Boolean operators:

l Filename

l Path

l File Type

l File Attribute

l Modified

l Accessed

l Metadata Changed

l Created

l Size

5. In the Apply CloudPools Actions to Selected Files area, select Move to cloudstorage.

6. In the CloudPool Storage Target drop-down menu, select an existingCloudPool, and specify whether to encrypt and compress data before it isarchived to the cloud.

7. Click Show Advanced CloudPool Settings to specify additional cloud storageoptions, as described in the following table:

Setting Description

DataRetentionSettings

l Cloud Data Retention Period: Specifies how long cloudobjects are retained after a SmartLink file has beendeleted. When a SmartLink file is deleted on the localcluster, CloudPools cleans up local resources allocated forthe SmartLink files, and also removes the associated cloudobjects. This work is performed weekly by the cloudobjects garbage collector job.

l Incremental Backup Retention Period for NDMPIncremental Backup and SyncIQ: Specifies the backupretention period for SmartLink files created with anincremental NDMP backup policy or a SyncIQ policy. Thisvalue is only relevant for incremental NDMP backups andSyncIQ replication.

l Full Backup Retention Period for NDMP Only: Specifiesthe backup retention period for SmartLink files createdwith a full NDMP backup policy. This value is only relevantfor full NDMP backups.

Accessibilityand CacheSettings

l Writeback Frequency: Specifies how often SmartLink filesedited on the Isilon cluster are written to their associatedcloud data objects.

l Accessibility: Specifies whether or not to cache archiveddata locally. Local caching enables faster access of clouddata, but reduces space saving on your cluster.

l Cache Read Ahead: Specifies whether cloud data is fullyor partially cached when you access a SmartLink file on the

Managing CloudPools policies

Create a file pool policy for cloud storage (Web UI) 35

Page 36: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Setting Description

local file system. If the policy specifies partial caching, thesystem only caches the blocks needed when a file isaccessed. If the policy specifies full caching, cloud data isfully cached when the SmartLink file is accessed.

l Cache Expiration: Specifies the amount of time cloud datathat has been cached is retained in the local cache ofassociated SmartLink files. The system purges theSmartLink cache of data that has not been accessed forthe number of days specified.

Archive fileswithsnapshots

Specifies that CloudPools is to archive files that have one ormore snapshot versions. The latest versions of such files arearchived, and SmartLink files are left in their place on the localcluster. Because the full snapshot versions of files are retainedon the cluster, archiving these files does not save space on thecluster.

8. Click Create Policy.

The file pool policy appears under File Pool Policies in the File Pool Policieswindow.

Results

The next time the SmartPools system job is run, the file pool policy performs thespecified actions.

Create a file pool policy for cloud storage (CLI)You can create file pool policies that specify CloudPools actions to be applied toselected files.

Procedure

1. Run the isi filepool policies create command.

The following command creates a file pool policy with the name archive andthe CloudPool storage target, S3_pool. The command also specifies a file-matching pattern to archive all files in a directory path that have not beenaccessed after November 30, 2014.

isi filepool policies create archive --cloud-pool=S3_pool --begin-filter --name="*.*" --and --path="/ifs/home/users" --and --accessed-time=2014-11-30 --operator=lt --end-filter

Modify cloud attributes in a file pool policy (Web UI)You can modify a file pool policy. Each file pool policy for cloud archival specifies a file-matching pattern and the actions to perform on the matched files (file pool).

Procedure

1. Click File System > Storage Pools > File Pool Policies.

The File Pool Policies page appears.

2. In the File Pool Policies list, next to the file pool policy you intend to modify,click View/Edit.

Managing CloudPools policies

36 OneFS CloudPools 8.1.0 Administration Guide

Page 37: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

The View File Pool Policy Details dialog box appears.

3. Click Edit Policy.

The Edit File Pool Policy Details dialog box appears.

4. Make your changes in the appropriate areas and click Save Changes.

Results

Changes to the file pool policy are applied the next time the SmartPools system jobruns.

Modify cloud attributes in a file pool policy (CLI)You can modify a file pool policy. Each file pool policy for cloud archival specifies a file-matching pattern and the actions to perform on the matched files (file pool).

You can run the isi filepool policies list command to list available filepool policies.

Procedure

1. Run the isi filepool policies modify command.

The following example modifies the file-matching pattern in a file pool policynamed my_policy.

isi filepool policies modify my_policy --begin-filter --name="*.jpg" --and --accessed-time=2013-08-01 --operator=lt--end-filter

List file pool policies (CLI)You can list all file pool policies stored in OneFS.

Procedure

1. Run the isi filepool policies list command.

View details of a file pool policy (CLI)You can display detailed information about a file pool policy.

To list all available file pool policies, you can run the isi filepool policieslist command.

Procedure

1. Run the isi filepool policies view command.

The following command displays information about the policy my_policy,including status, associated CloudPool, whether encryption and compressionare enabled, and more.

isi filepool policies view my_policy

Managing CloudPools policies

Modify cloud attributes in a file pool policy (CLI) 37

Page 38: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Apply a file pool policy to a specified file or path (CLI)You can apply a file pool policy to specified files or directories manually, rather thanwaiting for the SmartPools job to run.

For isi filepool apply to execute, the file or directory specified must matchone of the defined file pool policies.

Procedure

1. Run the isi filepool apply command.

The following command applies the appropriate file pool policy to all files andsubdirectories in a given path.

isi filepool apply --path=/ifs/data/images --recurse

Archive files directly to the cloud (CLI)You can archive specific files directly to the cloud. To enable this, CloudPools mustmatch these files to an existing file pool policy.

Before you begin

A custom file pool policy that matches the specified file or files and points to cloudstorage must already exist on your system.

Procedure

1. Run the isi cloud archive command.

The following command specifies a directory and all of its subdirectories andfiles to be archived:

isi cloud archive /ifs/data/shared/images/*.* --recursive yes

Managing CloudPools policies

38 OneFS CloudPools 8.1.0 Administration Guide

Page 39: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CHAPTER 4

Managing CloudPools with other OneFSfunctions

CloudPools is designed to work seamlessly with other OneFS functions, including dataencryption and compression, SMB and NFS support, SyncIQ, snapshots, and NDMPbackup and recovery. This section includes the following topics:

l Compression and encryption of cloud data........................................................ 40l CloudPools protocol support..............................................................................40l SyncIQ interoperability....................................................................................... 41l NDMP backup and restore of cloud data............................................................46l CloudPools and snapshots..................................................................................47l CloudPools and SmartLock................................................................................ 47l CloudPools and SmartQuotas............................................................................ 48l CloudPools and SmartDedupe............................................................................48

Managing CloudPools with other OneFS functions 39

Page 40: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Compression and encryption of cloud dataYou can specify compression and encryption of data that is moved to the cloud.

With CloudPools, you can enable compression and encryption on a per-policy basis.Both encryption and compression are disabled by default.

Files encrypted or compressed when stored in the cloud are automatically decryptedand decompressed when data is cached (inline access) or the file is recalled from thecloud to local storage.

CloudPools uses a master encryption key to encrypt the data encryption keys.Encryption applies to both the SmartLink file and the file data archived to the cloud.Both the SmartLink file and the archived data include encrypted copies of the dataencryption keys. After a file is encrypted, it can only be decrypted by recalling it.

CloudPools keeps track of the encryption status of SmartLink files in snapshots andreferenced data in the cloud. If SmartLink files in snapshots are unencrypted and referto unencrypted cloud objects, the SmartLink files in the snapshots remainunencrypted even if you create a new CloudPools policy that encrypts the latestversion of the file.

OneFS stores the master encryption key in the local key management system. You cangenerate a new version of the key if you believe the key has been compromised. Ifregenerated, the new master key secures new data written to the cloud. Previouslywritten data is secured by the old data encryption keys, resident in the local SmartLinkfiles.

Note

CloudPools works seamlessly with nodes that are equipped with self-encrypting drives(SEDs). CloudPools can apply encryption to files that are archived to cloud storage.Similarly, any SmartLink files left on SEDs are handled like any other file.

CloudPools protocol supportCloudPools supports inline access of cloud data through SMB, NFS, and other filesystem protocols. Consequently, users who access files on an Isilon cluster from othersystems can also access files stored in the cloud.

NFS inline accessCloudPools enables access of SmartLink files from NFS exports.

When a user connects to a cluster through an NFS export, and browses the filesystem, SmartLink files appear to be the original files. When the user opens aSmartLink file, CloudPools retrieves and caches the original file data from the cloud.Depending on the Cache Read Ahead setting, either a portion of the file data, or theentire file, is cached.

If the user modifies the file, CloudPools stores the changes in the cache andperiodically writes the changes back to the cloud. In this way, cloud data is kept fullyup to date.

Managing CloudPools with other OneFS functions

40 OneFS CloudPools 8.1.0 Administration Guide

Page 41: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

SMB inline accessCloudPools enables access of SmartLink files from SMB shares.

When a user connects to a cluster through an SMB share, and browses the filesystem, SmartLink files appear to be the original files. When the user opens aSmartLink file, CloudPools retrieves and caches the original file data from the cloud.Depending on the Cache Read Ahead setting, either a portion of the file data, or theentire file, is cached.

If the user modifies the file, CloudPools caches the changes and periodically writes thechanges back to the cloud. In this way, the cloud data is kept fully up to date.

Other protocols supporting inline accessCloud data can be accessed programmatically through Swift and HDFS by accessingthe associated SmartLink files.

We recommend that you consult with your Isilon representative for more informationon these use cases.

SyncIQ interoperabilitySyncIQ enables you to synchronize data from your Isilon primary (source) cluster to asecondary (target) cluster. If your primary cluster becomes unavailable, you can failover to the secondary cluster, and users can continue to access data, including datastored in the cloud.

During SyncIQ replication, all files, including SmartLink files, are copied from thesource cluster to the target cluster. Users given access to the target cluster throughsupported protocols can retrieve cloud data or fully recall files from the cloud. In thesecases, CloudPools retrieves and caches data (inline access) or recalls the full fileexactly as it would from the original source cluster.

Unless you specifically grant cloud write access to the secondary cluster, CloudPoolsstores any changes to SmartLink files in the local cache, which is limited only byavailable space on the cluster.

SyncIQ policiesCloudPools supports SyncIQ replication of SmartLink files to one or more targetclusters. SyncIQ can also be used to restore backed up SmartLink files to their original(source) cluster.

The two types of SyncIQ policies are synchronization policies and copy policies. Thesepolicies can be run manually, or configured to run automatically, based on policysettings.

CloudPools supports both types of SyncIQ policy. When SyncIQ replicates SmartLinkfiles to a target cluster, secondary information associated with a SmartLink file, suchas local cache state and unsynchronized cache data, is also replicated.

If your source (primary) cluster goes down or is unavailable for any reason, and youfail over to your secondary cluster, users can continue to access SmartLink files and,therefore, cloud data, as they would normally.

If the failover is temporary and you plan to restore your source cluster to fulloperation, you do not need to enable cloud write access on the secondary cluster. Any

Managing CloudPools with other OneFS functions

SMB inline access 41

Page 42: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

changes that users make to SmartLink files are stored in the local cache, which islimited only by the amount of free space on your cluster.

When you fail back to your source cluster, and restore updated SmartLink files, onlythen will CloudPools write the cached modifications back to the cloud.

CAUTION

If your failover is long-term or permanent, you can grant cloud write access tothe secondary cluster by using the isi cloud access add command in theCLI. However, you should only ever allow cloud write access on one cluster at atime. Otherwise, corruption of cloud data can occur.

CloudPools cloud data retention timeIn CloudPools, cloud data is governed by a concept referred to as retention. Retentiontime defines an absolute time for cloud data to remain in cloud storage after therelated SmartLink file has been deleted.

When you use CloudPools to archive a file from your cluster to cloud storage, aSmartLink file is created on the Isilon cluster in place of the archived file. As long asdata from a file remains in the cloud, the SmartLink file remains in place to representand point to the cloud data.

Because a SmartLink file can be backed up to tape through NDMP and replicated toanother Isilon cluster through SyncIQ, more than one SmartLink file can be pointing tothe same cloud data at the same time.

However, importantly, cloud data can be deleted even if there is a related SmartLinkfile on another cluster that was restored by a SyncIQ or NDMP process. Cloud data ismaintained until the minimum duration of its retention times on all clusters hasexpired.

Retention time is affected by the following archive policy retention periods:

l Cloud Data Retention Period specifies the retention time of cloud data beyondthe time when an associated local Smartlink file is deleted. The default setting isone week.

l Backup Retention Period for NDMP Incremental Backup and SyncIQ specifiesthe retention time of cloud data whose SmartLink file has been backed up by anincremental NDMP backup, or replicated by a SyncIQ operation. If a localSmartLink file is deleted, the SmartLink file copy can be restored, and cloud datacan still be accessed. The default setting is five years.

l Full Backup Retention Period for NDMP Only is the retention time of cloud datawhose SmartLink file has been backed up by a full NDMP backup only. If a localSmartLink file is deleted, the SmartLink file copy can be restored from the backup,and cloud data can still be accessed. The default setting is five years.

Managing SmartLink files that have been replicatedIf you modify or delete a SmartLink file that has been replicated in a SyncIQ operation,CloudPools can modify or delete its associated cloud objects.

If you modify a SmartLink file on the primary cluster, changes are cached and,depending on the Writeback Frequency setting, are periodically written back to thecloud. In this way, cloud data is always kept up to date.

If you modify a SmartLink file on a secondary cluster, because the primary cluster istemporarily unavailable, changes will remain in cache. When you fail back to the

Managing CloudPools with other OneFS functions

42 OneFS CloudPools 8.1.0 Administration Guide

Page 43: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

primary cluster, only then are changes able to be written back to the cloud accordingto the Writeback Frequency setting.

Similarly, when a SmartLink file that has been replicated in a SyncIQ operation isdeleted, CloudPools appropriately manages the associated cloud data. Two retentionperiods can affect the cloud objects associated with a SmartLink file that has beenreplicated: the Cloud Data Retention Period and the Incremental Backup RetentionPeriod for NDMP Incremental Backup and SyncIQ.

When a SmartLink file has been replicated to a secondary cluster, and is then deletedfrom the primary cluster, CloudPools uses both the Cloud Data Retention Period andthe Incremental Backup Retention Period for NDMP Incremental Backup andSyncIQ settings to determine when the associated cloud objects should be deleted.CloudPools uses the longer of the two durations to determine when to delete clouddata.

For example, if the longer of the two retention periods is the Incremental BackupRetention Period for NDMP Incremental Backup and SyncIQ setting, thenCloudPools uses that setting to determine when to delete cloud data after itsassociated SmartLink file is deleted.

If you delete a SmartLink file on a secondary cluster, because the primary cluster istemporarily unavailable, the deleted state will remain in cache. When you fail back tothe primary cluster, CloudPools deletes the SmartLink file, and uses the retentionsettings to determine when to delete the associated cloud data.

SyncIQ deep copyYou can create a SyncIQ policy that restores full files rather than SmartLink files whencopying data from the primary (source) cluster to a secondary (target) cluster.

When you create a SyncIQ policy, you can modify the Deep Copy for CloudPoolssetting. The default setting is Deny, which means that, during a SyncIQ operation,SmartLink files are replicated to the target cluster.

Alternatively, you can select either the Allow or Force option for deep copy. Whenyou select Allow, SyncIQ still replicates SmartLink files to the target cluster unlessthere is a SmartLink version mismatch, in which case the full file data is retrieved fromthe cloud and replicated.

When you specify Force for deep copy, CloudPools retrieves and copies full file datafrom the cloud for all SmartLink files affected by the SyncIQ policy, and replicates thefull files to the target cluster.

Note

A SyncIQ operation that forces deep copy can take significantly more time andconsume more system resources. We recommend that you not specify deep copyunless you have a specific reason to do so. For example, if you are backing up datafrom the primary cluster to a secondary cluster that is running an older (pre-8.0)version of OneFS, then you should use deep copy. If you are unsure whether to usedeep copy, contact your EMC Isilon representative for guidance.

Managing CloudPools with other OneFS functions

SyncIQ deep copy 43

Page 44: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Configuring access to cloud data from a secondary clusterYou can make cloud data available on a secondary cluster if your primary clusterbecomes unavailable.

To configure such access, you need to have replicated the primary cluster's data on asecondary cluster using SyncIQ. Alternatively, you can restore an NDMP backup ofthe data to a secondary cluster.

The secondary cluster must have active SyncIQ, SmartPools, and CloudPools licenses.

With SyncIQ, when failover to a secondary cluster is required, two use cases aresupported: short-term failover versus long-term failover.

In the short-term failover use case, the intention is to restore and failback to theprimary cluster as quickly as possible. The secondary cluster is a temporary solution,enabling users to open SmartLink files from supported protocols and access clouddata as usual. Instead of writing any changes back to the cloud, however, CloudPoolscaches these changes locally in the SmartLink files on the secondary cluster. After theprimary cluster is restored to service, changes from the secondary cluster must bewritten back to the primary cluster to enable any cached data in SmartLink files to bewritten back to cloud storage.

In a long-term failover situation, in which the primary cluster is to be out of service foran extended period, or is to be decommissioned entirely, other considerations becomeimportant. In this scenario, because only one cluster can have write access to cloudstorage, you need to transfer write access to the failover cluster. From a CloudPoolsperspective in this scenario, the failover cluster effectively becomes the primarycluster.

With the NDMP approach, however, the short-term failover scenario is less practical.The secondary cluster should be given cloud write access to enable any cachedmodifications to SmartLink files to be written back to cloud storage. The alternativewould be to somehow write modified SmartLink files back to the primary cluster afterit is restored to service, but this might be more time-consuming.

CAUTION

Never allow write access to cloud data from more than one cluster at a timebecause it can result in data corruption.

List available clusters (CLI)You can list the GUIDs of clusters that are accessible for SyncIQ failover or restoreoperations.

Available clusters are the primary cluster and any other clusters to which data hasbeen replicated with SyncIQ or restored with NDMP.

Procedure

1. Run the isi cloud access list command.

OneFS displays a list of the GUIDs of eligible clusters.

Managing CloudPools with other OneFS functions

44 OneFS CloudPools 8.1.0 Administration Guide

Page 45: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Allow access to cloud data on a secondary cluster (CLI)You can enable write access of SmartLink files and their cloud data on a secondary(target) cluster. Data from the primary (source) cluster must have been replicated toor restored on the target cluster by a SyncIQ or NDMP process.

Before you begin

Run the isi cloud access list command to display available target clusters.The target cluster must also have both a SmartPools and CloudPools license.

CAUTION

This capability is offered to work around cases where the primary cluster is goingto be unavailable for an extended period. Never allow write access to cloud datafrom more than one cluster at a time because it can result in data corruption.Before allowing another cluster to have cloud write access, make sure that cloudwrite access is removed from the primary cluster, and that the primary cluster isoffline.

Procedure

1. Run the isi cloud access add command.

The following command enables a secondary cluster, identified by GUID, tohave write access to cloud data.

isi cloud access add ab9dd991261e11e382240800200c9a66

You can also specify the secondary cluster by name:

isi cloud access add isi2

If you know when the primary cluster will be restored to service, you can set anexpiration date. The following command adds an expiration date of December 1,2015:

isi cloud access add isi2 --expiration-date 12012015

Remove access to data from a source cluster (CLI)You can remove previously granted access to SmartLink files replicated from a sourcecluster or restored from an NDMP backup. This operation must be done on the targetcluster.

Before you begin

CAUTION

Never allow write access to cloud data from more than one cluster at a timebecause it can result in data corruption. Removing access to SmartLink files on atarget (secondary) cluster is typically done when the source (primary) clusterhas been or is about to be restored to service.

Managing CloudPools with other OneFS functions

Configuring access to cloud data from a secondary cluster 45

Page 46: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Procedure

1. Run the isi cloud access remove command on the target cluster.The following command removes SmartLink file access from a target cluster byspecifying its GUID.

isi cloud access remove ab9dd991261e11e382240800200c9a66

NDMP backup and restore of cloud dataYou can perform NDMP backup and restore operations on data that has been archivedto the cloud.

Backup and restore capabilities with CloudPools data include:

l Archive SmartLink files when backing up from a clusterl Restore data, including SmartLink files, to the same clusterl Restore data, including SmartLink files, to another cluster

With NDMP backup, by default, CloudPools supports backup of SmartLink files only.No cloud data is included in the backup. Secondary information such as accountinformation, local cache state, and unsynchronized cache data associated with theSmartLink file is also backed up.

However, you can force NDMP backup to store a full copy of file data rather thanSmartLink files. This is sometimes referred to as the deep copy option. You specifydeep copy by setting the BACKUP_OPTIONS environment variable to 0x00000100.

In CloudPools settings, you can set three retention periods that affect backed upSmartLink files and their associated cloud data:

l Full Backup Retention Period for NDMP takes effect when the SmartLink file isbacked up as part of a full backup. The default is five years.

l Incremental Backup Retention Period for Incremental NDMP Backup and SyncIQtakes effect when a SmartLink file is backed up as part of an incremental backup.The default is five years.

l Cloud Data Retention Period defines the duration that data in the cloud is keptwhen its related SmartLink file is deleted. The default is one week.

CloudPools ensures the validity of a backed-up SmartLink file within the cloud dataretention period. It is important for you to set the retention periods appropriately toensure that when the SmartLink file is restored from tape, it remains valid. CloudPoolsdisallows restoring invalid SmartLink files.

To check whether a backed-up SmartLink file is still valid, CloudPools checks theretention periods stored on tape for the file. If the retention time is past the restoretime, CloudPools prevents NDMP from restoring the SmartLink file.

CloudPools also makes sure that the account under which the SmartLink files wereoriginally created has not been deleted. If it has, both NDMP backup and restore ofSmartLink files will fail.

Checking the version of SmartLink filesDuring an NDMP backup session, version data for CloudPools SmartLink files isincluded in the backup stream.

When restoring data, a version check is performed on the SmartLink files. If theversion check determines that the SmartLink files are incompatible with the operating

Managing CloudPools with other OneFS functions

46 OneFS CloudPools 8.1.0 Administration Guide

Page 47: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

system version running on the target cluster, the NDMP restore session does notrestore the SmartLink files to the target cluster and reports the versionincompatibilities in the NDMP log.

CloudPools and snapshotsWith CloudPools, you can control whether files that have snapshot versions arearchived to the cloud.

The SnapshotIQ, SyncIQ, FSAnalyze, and NDMP Backup functions create point-in-time snapshots of directories in OneFS. Even as files are modified, the snapshotversions are maintained. As part of file matching, CloudPools can include files thathave snapshot versions.

CloudPools archives the latest versions of those files to the cloud, and creates localSmartLink files in place of the archived files. As long as the snapshots remain,however, OneFS also maintains the full-data versions of these files in snapshots.

The default CloudPools setting is to allow files with snapshot versions to be archived,but you can change the default setting.

Note

Allowing archival of files that have snapshot versions does not result in space savingson your local cluster until the snapshots are deleted. Some functions, such asFSAnalyze, SyncIQ, and NDMP Backup, automatically delete snapshots when they areno longer needed. However, snapshots created with SnapshotIQ are not automaticallydeleted. To free up space on your cluster, we recommend that you regularly deleteolder snapshots that you no longer need.

CloudPools also supports SnapRevert for SmartLink files. For example, suppose thatCloudPools archived a directory named /ifs/data/images to the cloud. The files inthe images directory would be replaced with SmartLink files.

If you create a SnapRevert domain for the directory, and run the SnapRevert job, theCloudPools archival process is reversed, and the original files are restored to thedirectory. CloudPools removes any cloud data that was created as part of the originalarchive process.

CloudPools and SmartLockNormally, CloudPools archives files to a cloud storage repository and leaves SmartLinkfiles in their place on local storage. However, CloudPools is not designed to be usedwith SmartLock domains, in which files are committed to a WORM (write-once, read-many) state.

When you create file pool policies to archive certain files to cloud storage, make surethat these policies avoid SmartLock domains and the WORM files they include.

If you inadvertently attempt to archive WORM files to the cloud, CloudPools willgenerate a "file system read-only" error. Although it is possible to archive a file to thecloud, then add the resulting SmartLink file to a WORM domain, with or withoutcommitting it to a WORM state, this is not recommended.

Managing CloudPools with other OneFS functions

CloudPools and snapshots 47

Page 48: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CloudPools and SmartQuotasThe administrator can enforce storage limits for users with SmartQuotas. In this case,users should be aware that recalling data from the cloud could potentially cause themto exceed those limits.

When CloudPools archives files to cloud storage, CloudPools creates SmartLink fileson local storage in place of the archived files. SmartLink files typically take upconsiderably less storage space than the archived files they replace.

When users recall archived files from the cloud, the full files replace the SmartLinkfiles in local storage. This could potentially cause users to exceed their quotas. Forexample, suppose a user's quota is 500 MB, and files older than six months arearchived to the cloud. This saves the user 250 MB of space, as the SmartLink filestake up relatively little local storage space. In the meantime, the user has added morefiles and now has 400 MB of data in local storage. Should the user recall files from thecloud that would take up more than 100 MB of storage, the user would exceed thequota.

As a storage administrator, you should make your users aware of this possibility andhow best to mitigate the issue.

CloudPools and SmartDedupeSmartDedupe scans the OneFS file system for files that contain identical blocks ofdata. If SmartDedupe finds duplicate blocks, SmartDedupe moves a single copy of theblocks to a hidden file called a shadow store. SmartDedupe then deletes the duplicateblocks from the original files and replaces the blocks with pointers to the shadowstore.

CloudPools interacts as follows with SmartDedupe:

l If a file pool policy specifies that de-duplicated files should be archived to cloudstorage, CloudPools archives those de-duplicated files and leaves SmartLink filesin their place in local storage.

l When an archived file that had been de-duplicated is recalled from the cloud, theSmartLink file is replaced and the recalled file placed back in local storage is nolonger de-duplicated.

l SmartDedupe does not de-duplicate SmartLink files.

Managing CloudPools with other OneFS functions

48 OneFS CloudPools 8.1.0 Administration Guide

Page 49: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CHAPTER 5

CloudPools tips and troubleshooting

This section provides best practices and other advanced information aboutCloudPools.

l CloudPools best practices..................................................................................50l Managing cloud jobs.......................................................................................... 50l CloudPools troubleshooting............................................................................... 52

CloudPools tips and troubleshooting 49

Page 50: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CloudPools best practicesFor best results using CloudPools, follow these best practices.

Use time stamps for cloud data archival and recallUse time matching patterns (creation, modification, last access) when you archivedata to and recall data from the cloud. This enables more efficient archival and recalloperations, therefore better performance.

When you create a file pool policy for archiving data to the cloud, several of the file-matching criteria involve time:

l Createdl Accessedl Modified

Therefore, you can specify file-matching criteria that specify when the files werecreated, when files were last accessed, or when they were last modified.

CloudPools can also more efficiently recall files based on time stamps.

CloudPools archiving and file sizeYou can gain the most benefit from CloudPools, in terms of freeing up storage spaceon your cluster, by archiving larger files. Archiving small files provides less, if any,benefit.

One of the benefits of archiving files to the cloud with CloudPools is how quickly youcan recall these files when needed.

To enable fast recall, CloudPools creates a SmartLink file for every file whose data isarchived to the cloud. SmartLink files each contain a map to the data in the cloud,meta data, and cache space. SmartLink files are generally small in size, but can grow ifdata is cached through inline access.

Therefore, if you archive small files to the cloud, SmartLink files are left in their placeon the cluster, and could approach, or even exceed, the size of the original file.

Create exclusive accounts for CloudPools purposesYou should create an account with your cloud provider that is exclusively forCloudPools use. This prevents conflicts that might lead to data corruption or loss.

If your organization accesses cloud provider accounts outside of OneFS CloudPoolsoperation, users must be careful not to in any way access or change data archived byCloudPools. Any such data access or modification would likely corrupt the data andcompromise data retrieval and recall from CloudPools.

To prevent this, create an account in CloudPools that is exclusively for CloudPoolsuse. Use entirely separate accounts for other cloud applications with your cloudprovider.

Managing cloud jobsYou can monitor and manage two types of cloud jobs: system jobs that are alwaysrunning in the background, and manual jobs that are created with the isi cloudjobs archive and isi cloud jobs recall commands. OneFS enables you to

CloudPools tips and troubleshooting

50 OneFS CloudPools 8.1.0 Administration Guide

Page 51: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

monitor the status of both job types, and to monitor and manage your manual archiveand recall jobs, as needed.

View a list of cloud jobs (CLI)You can list all CloudPools jobs. Both CloudPools system jobs and manual jobs arelisted.

CloudPools system jobs are always running to service caching and clean-up (garbagecollection) processes. CloudPools manual jobs include archive jobs specified in file poolpolicies, and recall jobs started from the OneFS command-line interface. Each job islisted by ID, description, state, and type.

Procedure

1. Run the isi cloud jobs list command.

Output from the command lists CloudPools job ID, description, status, and type.

View a cloud job (CLI)You can view information about a CloudPools job.

Before you begin

You need to know the ID of the job you want to view. You can run the isi cloudjobs list command to see the IDs for all cloud jobs.

Procedure

1. Run the isi cloud jobs view command.

The following command views information about a job with the ID of 63.

isi cloud jobs view 63

Pause a cloud job (CLI)You can pause a running CloudPools job. This operation is typically done only fortroubleshooting purposes.

Before you begin

To pause a job, you need to know the ID of the job. Run the isi cloud jobs listcommand to see a list of all cloud job IDs.

Procedure

1. Run the isi cloud jobs pause command.

The following command pauses a job with the ID of 63.

isi cloud jobs pause 63

This command pauses all running archive jobs:

isi cloud jobs pause archive

CloudPools tips and troubleshooting

View a list of cloud jobs (CLI) 51

Page 52: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Note

Only currently running archive jobs are paused. Any subsequent jobs that arekicked off by a file pool policy, or manually through isi cloud archive arenot paused and will run.

Resume a paused cloud job (CLI)You can resume a cloud job that has been paused.

Before you beginTo resume a job, you need to know the ID of the job. Run the isi cloud jobslist command to see a list of all cloud job IDs.Procedure

1. Run the isi cloud jobs resume command.

The following command resumes a job with the ID of 63.

isi cloud jobs resume 63

Cancel a cloud job (CLI)You can cancel a running CloudPools job.

Before you beginTo cancel a job, you need to know the ID of the job. Run the isi cloud jobs listcommand to see a list of all cloud job IDs.

Procedure

1. Run the isi cloud jobs cancel command.

The following command cancels a job with an ID of 63.

isi cloud jobs cancel 63

CloudPools troubleshootingIf you encounter problems using CloudPools, refer to the information provided in thissection before contacting customer support.

CloudPools limitations and expected behaviorsDuring normal CloudPools operation, you should be aware of the following limitationsand expected behaviors.

Rolling upgrade before CloudPools usage

If you are performing a rolling upgrade to the new OneFS version, make sure theupgrade is fully complete before activating CloudPools.

Cloud storage account deletion

Warning: Do not delete a cloud storage account that is in use by archived files.This can lead to data being lost or unavailable for the archived files that use that

CloudPools tips and troubleshooting

52 OneFS CloudPools 8.1.0 Administration Guide

Page 53: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

account. Any attempt to open SmartLink files associated with a deleted accountwill fail with I/O error messages. In addition, NDMP backup and restore andSyncIQ failover and failback will fail when a cloud storage account has beendeleted. If, through inline access, an NFS or SMB user attempts to open aSmartLink file, and receives an I/O error, this can mean that the related cloudstorage account has been deleted. We recommend trying inline access of otherSmartLink files in the same CloudPool. If the same error is generated for thosefiles, the cloud storage account has been deleted and data is lost. If the otherSmartLink files are accessible, the SmartLink file that generated the error mightbe corrupted. Either way, you should contact EMC Isilon Support for assistance.

Accessing SmartLink files

You can view and modify cloud data by accessing SmartLink files throughsupported protocols (NFS, SMB, Swift, and HDFS). If you attempt to access aSmartLink file directly on the cluster, through a command running on the cluster,or through an unsupported protocol, an I/O error is generated. Local access forSmartLink files is blocked for these commands: tar, gzip, scp, AVscan, andJob Engine jobs. Cloning of a SmartLink file is also prevented and will return anEINVAL error.

SmartLink file timestamps can change

Opening a SmartLink file through a supported protocol can change the timestampdata. When a file is first archived, and the SmartLink file is created in its place,the ctime timestamp stays the same as the original file's timestamp. However, thefirst time the SmartLink file is opened (inline access), the ctime timestampchanges as a cache component is added to the file. Also, if an archived file is fullyrecalled, its ctime and mtime timestamps change.

Inline access can appear to convert a SmartLink file to a regular file

When a user accesses a SmartLink file on the Isilon cluster from a supportedprotocol, the file opens in an application on the client computer. During thisprocess, called inline access, most applications support the creation of aCloudPools cache from which users can view and, if desired, modify archiveddata. With inline access, the SmartLink file remains intact on the cluster, and anymodifications that the user makes to file data are stored in the cache and updatedto the cloud.However, some applications do not support inline access. Instead, theseapplications create a new copy of the original file apart from the SmartLink file.The new file, containing all original file data, is given a new logical I-node (LIN)number and timestamps that differ from the file that was originally archived. Thisbehavior has been observed in only a few programs, including Microsoft Officeapplications. In these cases, since an entirely new file is created, the originalSmartLink file and its associated data in the cloud is tagged for removal (garbagecollection).

If the new file meets the criteria of the file pool policy that archived the originalfile to the cloud, the new file is archived to the cloud the next time theSmartPools job runs, and a new SmartLink file is created in its place on the localcluster. If the new file does not meet the policy criteria, the full file remains on thecluster.

For best results using CloudPools, we recommend that you avoid archiving filesthat are still being actively modified by your users.

CloudPools tips and troubleshooting

CloudPools limitations and expected behaviors 53

Page 54: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Copying a SmartLink file from one cluster to another can result in a corruptedSmartLink file

If you use scp (Secure Copy) or other copy commands to copy a SmartLink filefrom one cluster to another, the resulting SmartLink file is corrupted.

Running Antivirus on SmartLink files generates errors

We recommend that you avoid creating policies that run Antivirus on directoriescontaining SmartLink files. Running Antivirus on SmartLink files generates errormessages. If you do run Antivirus on such directories, you can safely ignore thesemessages.

Other issues accessing SmartLink files

If the node does not have quorum, or the file system is read-only, or /ifs is notavailable, then you might encounter issues accessing SmartLink files.

Client-based tools and SmartLink files

If you run an SMB or NFS client-based tool such as AVScan (anti-virus scan) or abackup application, file data in the cloud is fully cached back to the SmartLinkfiles. This can result in heavy network usage and increased service provider costs,and would also negate space saving on your cluster.

Expired SmartLink files

Expired SmartLink files are not restored using NDMP and do not synch back usingSyncIQ. A SmartLink file on an NDMP backup or on a SyncIQ secondary (target)cluster is expired when the original SmartLink file has been deleted from theprimary (source) cluster, or the original file data in the cloud has been fullyrecalled.

Recall can be interrupted

When a full cache is in process (that is, someone performed an inline access of aSmartLink file from an SMB share or NFS export), recall of the same file can fail.When this happens, the full cache is allowed to complete first, and the usershould retry the recall after caching is completed.

ADS files

CloudPools does not archive and recall ADS (alternate data stream) files.

SMB Oplock

SMB Oplock (lease/notification) does not work in cases where you create a filewith the SUPERCEDE flag, and the file already exists and is archived.

CloudPools logsYou can access CloudPools logs to view activity and troubleshoot problems.

The following logs are available in OneFS for CloudPools operation.

Type Name Path

Client cluster-side logs

Cpool daemon /var/log/isi_cpool_d.log

Job Engine /var/log/isi_job_d.log

SMB and NFS I/O /var/log/isi_cpool_io_d.logand /var/log/lwiod.log

Provisioning /var/log/isi_papi_d.log

CloudPools tips and troubleshooting

54 OneFS CloudPools 8.1.0 Administration Guide

Page 55: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Type Name Path

NDMP /var/log/isi_ndmp_d.log

SyncIQ /var/log/isi_migrate.log

Messages /var/log/messages

Platform APIcloud-side logs

Platform API (RAN) /var/log/isi_object_d.log

HTTPd apache /var/log/apache2/webui_httpd_error.logand/var/log/apache2/webui_httpd_access.log

Sessionauthentication

Messages /var/log/messages

Note

Make sure that the client cluster-side time is accurate to within 15 minutes of thecloud provider.

Troubleshooting CloudPoolsThis section describes other troubleshooting items for CloudPools administration andoperation.

Cloud storage account cannot connect to the cloud

In the OneFS, if a cloud storage account is shown in the web administrationinterface with a red Needs Attention icon, or in the CLI interface with anUnreachable state, this usually indicates that the cluster has lost Internetconnectivity or the service provider's cloud storage facility is offline. Ensure thatthe cluster has Internet connectivity. If it does, contact your service provider forhelp.

Determining if a file is a SmartLink file

To determine if a file has been archived to the cloud, you can check whether thelocal version on the cluster is a SmartLink file. Run the isi get -D command asin the following example:

isi get -D koala.jpg | grep Stubbed:

The output would be as follows if the specified file was a SmartLink (stub) file:

* Stubbed: True

If the file is not a SmartLink file, the output would be False.

CloudPools tips and troubleshooting

Troubleshooting CloudPools 55

Page 56: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CloudPools tips and troubleshooting

56 OneFS CloudPools 8.1.0 Administration Guide

Page 57: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CHAPTER 6

CloudPools CLI commands

This section provides a reference to all CloudPools commands in the OneFS commandline interface:

l CloudPools command reference........................................................................ 58

CloudPools CLI commands 57

Page 58: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

CloudPools command referenceUse CloudPools commands to manage general settings, and to create and managecloud accounts and cloud storage targets. This section provides reference informationfor each command. For any command, you can use the --help option to get a fulllisting of command options.

isi cloud access addAdds cloud write access to the cluster.

Syntax

isi cloud access add <guid> [--expiration-date]<timestamp> [--verbose]

Options

<guid>The reference number, or globally unique identifier (GUID), of the cloud account.

--expiration-date <timestamp>

The date and time at which write access to cloud data ends on this cluster. Thetimestamp format is MMDDYY:hh:mm. For example, 022016:12:00 specifies anexpiration date and time of February 20, 2016 at 12:00 PM.

--verbose

Displays more detailed information.

ExamplesThe following example adds cloud write access to a cluster by specifying the clusterGUID and an expiration date:

isi cloud access add 000556bf1e82059801563f1ad44a8c155acf--expiration-date 022016:12:00

OneFS displays a message indicating the cloud accounts and file pool policies to whichthe secondary cluster will have access, and requires confirmation. Type yes, and pressENTER to complete the process.

isi cloud access listDisplays a list of clusters on your network that have, or are eligible for, write access tocloud data. Available clusters are the primary cluster and any other clusters to whichdata has been replicated with SyncIQ or restored with NDMP.

Syntax

isi cloud access list [--limit]<integer> [--sort {name | guid | synced_from | state | accounts | policies}] [--descending]

CloudPools CLI commands

58 OneFS CloudPools 8.1.0 Administration Guide

Page 59: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

[--format {table | json | csv | list}] [--no-header] [--no-footer] [--verbose]

Options

--limit<integer>

Limits the number of eligible clusters displayed in the list.

--sortSort the list of eligible clusters according to the specified category. The followingvalues are valid:

nameguidsynced_fromstateaccountspolicies

--formatOutputs the list of eligible clusters in the specified format. The following valuesare valid:

tablejsoncsvlist

--descendingOutputs the list of eligible clusters in descending order according to the specifiedsort option.

--no-headerDisplays table and CSV output without headers.

--no-footerDisplays table output without footers.

--verboseDisplays more detailed information.

CloudPools CLI commands

isi cloud access list 59

Page 60: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

isi cloud access removeRemoves cloud write access from the specified cluster.

Syntax

isi cloud access remove <guid> [--force] [--verbose]

Options

<guid>The reference number, or globally unique identifier (GUID), of the cluster fromwhich you want to remove cloud write access.

--forceExecute the command without requiring confirmation.

--verboseDisplays more detailed information.

ExamplesThe following example removes cloud write access from a cluster identified by aspecified GUID:

isi cloud access remove 000556bf1e82059801563f1ad44a8c155acf

OneFS displays a message indicating the cloud accounts and file pool policies to whichthe cluster will no longer have access, and requires confirmation. Type yes, and pressENTER to complete the process.

isi cloud access viewView the details of a cluster with, or eligible for, write access to cloud data.

Syntax

~isi cloud access view <guid>

Options

<guid>The reference number, or globally unique identifier (GUID), of the cluster.

CloudPools CLI commands

60 OneFS CloudPools 8.1.0 Administration Guide

Page 61: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

isi cloud accounts createCreates a cloud storage account that connects CloudPools to your cloud storageprovider.

Syntax

isi cloud accounts create <name> <type> <uri> <account-username> <key> [--enabled {yes | no}] [--account-id <string>] [--telemetry-bucket <string>] [--storage-region <string>] [--skip-ssl-validation {yes | no}] [--proxy <string>] [--force] [--verbose]

Options

<name>

The name of the cloud storage account.

<type>

The type of cloud storage account: one of isilon, azure, s3, virtustream,or ecs.

<uri>

The cloud account URI. This URI must match that provided to the cloud vendor.

<account-username>

The username for the cloud account. This name must be identical to the username provided to the cloud vendor.

<key>

The cloud account access key or password. This information is provided by thecloud vendor.

--enabled {yes | no}

By default, when you create a cloud storage account, it is enabled. To disable theaccount on creation, you can use this setting with the no option.

--account-id <string>

This is a required Amazon S3-only setting. The account ID number provided byAmazon when you first establish an account with the vendor.

--telemetry-bucket <string>

This is a required Amazon S3-only setting. The telemetry bucket name that youspecified when you first established an account with the vendor.

--storage-region <string>

This is a required Amazon S3-only setting. The storage region that you specifiedwhen you first established an account with the vendor. For example, us-west-1.

--skip-ssl-validation {yes | no}

CloudPools CLI commands

isi cloud accounts create 61

Page 62: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Specifies whether to circumvent SSL certificate validation when connecting to acloud provider's storage repository. Unless you specify this setting with a yesinstruction, OneFS will attempt to perform SSL certificate validation whenconnecting. For security purposes, we recommend not enabling this setting. Ifyou are connecting to cloud provider (for example, Isilon or ECS) that is withinyour corporate network, and you are having trouble connecting, you can skip SSLvalidation.

--proxy <string>

The network proxy through which CloudPools traffic to and from a public cloudprovider should be redirected. The specified network proxy must already havebeen created with the isi cloud proxies create command.

--forceExecute the command without requiring confirmation.

--verboseDisplays more detailed information.

ExamplesThe following example creates a Microsoft Azure cloud account:

isi cloud accounts create my_azure azure https://myazure.windows.net myuser dhgXJ9OAIahXvYmL

isi cloud accounts deleteDelete a cloud storage account. Caution: deleting an account can result in loss ofaccess to cloud data.

Syntax

isi cloud accounts delete <id>[--acknowledge <string>][--verbose]

Options

<id>

The name of the cloud account. You can use the isi cloud accounts list commandto display the names of cloud accounts.

--acknowledge <string>

Enables the account deletion to proceed. This parameter is required. You mustinclude a text string with the paramater, such as yes, proceed, or other string.

--verboseDisplays more detailed information.

ExampleThe following example deletes a Microsoft Azure cloud account:

isi cloud accounts delete my_azure --acknowledge yes

CloudPools CLI commands

62 OneFS CloudPools 8.1.0 Administration Guide

Page 63: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

When you run the command, OneFS displays the following message and requiresconfirmation:

**********************************************************************WARNING: Deleting an account is extremely dangerous.Continuing with this operation will result in a permanent loss of data.Type 'confirm delete data' to proceed. Press enter to cancel:

To proceed, type confirm delete data, and press ENTER.

isi cloud accounts listList cloud accounts.

Syntax

isi cloud accounts list [--limit <integer>] [--sort {id | name | type | account_username | uri | state | bucket}] [--descending] [--format {table | json | csv | list}] [--no-header] [--no-footer] [--verbose]

Options

--limit<integer>

Limits the number of cloud accounts displayed in the list.

--sortSort the list of cloud accounts according to the specified category. The followingvalues are valid:

idnametypeaccount_usernameuristatebucket

--formatOutputs the list of cloud accounts in the specified format. The following valuesare valid:

tablejsoncsvlist

--descending

CloudPools CLI commands

isi cloud accounts list 63

Page 64: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Outputs the list of cloud accounts in descending order according to the specifiedsort option.

--no-headerDisplays table and CSV output without headers.

--no-footerDisplays table output without footers.

--verboseDisplays more detailed information.

isi cloud accounts modifyModify a cloud account.

Syntax

isi cloud accounts modify <id> [--name <string>] [--account-username <string>] [--key <string>] [--uri <string>] [--enabled {yes | no}] [--account-id <string>] [--telemetry-bucket <string>] [--storage-region <string>] [--proxy <string>] [--clear-proxy [--skip-ssl-validation {yes | no}] [--verbose]

Options

<id>

The ID of the cloud account. In this case, the ID is the same as the cloud accountname.

--name <string>

The name of the cloud account. In this case, the name is the same as the ID.

--account-username <string>

The username for the cloud account. This name must be identical to the username provided to the cloud vendor.

--key <string>

The cloud account access key or password. This information is provided by thecloud vendor.

--uri <string>

The cloud account URI. This URI must match that provided to the cloud vendor.

--enabled {yes | no}

By default, when you create a cloud storage account, it is enabled. To disable theaccount on creation, you can use this setting with the no option.

CloudPools CLI commands

64 OneFS CloudPools 8.1.0 Administration Guide

Page 65: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

--account-id <string>

This is a required Amazon S3-only setting. The account ID number provided byAmazon when you first establish an account with the vendor.

--telemetry-bucket <string>

This is a required Amazon S3-only setting. The telemetry bucket name that youspecified when you first established an account with the vendor.

--storage-region <string>

This is a required Amazon S3-only setting. The storage region that you specifiedwhen you first established an account with the vendor. For example, us-west-1.

--skip-ssl-validation {yes | no}

Specifies whether to circumvent SSL certificate validation when connecting to acloud provider's storage repository. Unless you specify this setting with a yesinstruction, OneFS will attempt to perform SSL certificate validation whenconnecting. For security purposes, we recommend not enabling this setting. Ifyou are connecting to a cloud provider (for example, RAN or ECS) that is insideyour corporate network, and you are having trouble connecting, you can skip SSLvalidation.

--proxy <string>

The network proxy through which CloudPools traffic to and from a public cloudprovider should be redirected. The specified network proxy must already havebeen created with the isi cloud proxies create command.

--clear-proxyRemoves the network proxy through which CloudPools traffic to and from apublic cloud provider had been redirected. When you remove a proxy, CloudPoolstraffic would flow directly to the cloud provider.

--skip-account-check {yes | no}

If set to yes, CloudPools skips the validation step to determine that the cloudstorage account is accessible.

--verboseDisplays more detailed information.

ExampleThe following example modifies a Microsoft Azure cloud account:

isi cloud accounts modify my_azure --uri https://myazure.windows.net --account-username myuser --key dhgXJ9OAIahXvYmL

isi cloud accounts viewView the details of a cloud account.

Syntax

isi cloud accounts view <name>

CloudPools CLI commands

isi cloud accounts view 65

Page 66: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Options

<name>

Specifies the name of the cloud account to view. You can use the isi cloudaccounts list command to display a list of the names of available cloudaccounts.

ExampleThe following example displays the details of an Amazon S3 cloud account namedmy_S3:

isi cloud accounts view my_s3

isi cloud archiveQueue one or more files to be archived to or recalled from the cloud. Specify filesindividually or by using a file matching pattern. For files to be archived, they mustmatch the specified file pool policy, or any file pool policy with a cloud target.

Syntax

isi cloud archive <files> [--recursive {yes | no}] [--policy <string>] [--verbose] [--help]

Options

<files>

Specifies the files to archive or recall. Specify --files for each additional file toprocess. Alternatively, you can specify a file matching pattern such as /ifs/data/archive/images/*.jpg.

--recursive {yes | no}

Specifies whether the operation should apply recursively to nested directories inthe file string.

--policy <string>

Specifies the file pool policy to appy to the specified files. If you specify one ormore files to be archived and do not specify a policy, OneFS will compare the fileswith each configured file pool policy.

--verboseDisplays more detailed information.

ExamplesThe following example archives multiple files to the cloud according to a specific filepool policy:

isi cloud archive /ifs/data/images/big.jpg --file /ifs/data/huge.jpg--policy my_policy

CloudPools CLI commands

66 OneFS CloudPools 8.1.0 Administration Guide

Page 67: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

The following example archives an entire directory to the cloud. The operation mustmatch an existing file pool policy to be successful.

isi cloud archive /ifs/data/images/*.* --recursive yes

The following example recalls files from the cloud:

isi cloud archive /ifs/data/images/*.* --type recall

isi cloud jobs cancelCancel a CloudPools job initiated manually with isi cloud archive or isi cloudrecall). CloudPools system jobs (such as cache-writeback) cannot be canceled.

Syntax

isi cloud jobs cancel <id> [--verbose]

Options

<id>

The ID for the cloud job. Run isi cloud jobs list to see a list of all manualand system jobs and their associated IDs.

--verboseDisplays more detailed information.

ExampleThis following example cancels a CloudPools job with the ID of 21.

isi cloud cancel 21

isi cloud jobs files listDisplays the list of files matched by the specified CloudPools job.

Syntax

isi cloud jobs files list <job-id> [--limit <integer>] [--sort {name | state}] [--descending] [--format {table | json | csv | list}] [--no-header] [--no-footer] [--verbose]

Options

<job-id>

CloudPools CLI commands

isi cloud jobs cancel 67

Page 68: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

The ID of the job. To find the list of job IDs in CloudPools, run the isi cloudjobs list command.

--limit <integer>

Display no more than the specified number of items.

--sort {name | state}

Order results by the specified field. The default value is name.

--descendingSort and present data in descending order.

--format {table | json | csv | list}

Displays output in table (default), JavaScript Object Notation (JSON), comma-separated value (CSV), or list format.

--no-headerDisplays table and CSV output without headers.

--no-footerDisplays table output without footers.

--verboseDisplays more detailed information.

ExampleThe following example displays a list of files associated with a specific cloud job:

isi cloud jobs files list 21

isi cloud jobs listView the status of CloudPools jobs, including system, archive, and recall jobs.

Syntax

isi cloud jobs list[--limit <integer>][--sort {id | job_state | operation_state |effective_state | type| state_change_time | completion_time | create_time | description}][--descending ][--format {table | json | csv | list}][--no-header][--no-footer][--verbose]

Options

--limit <integer>

Display no more than the specified number of items.

--sort {id | job_state | operation_state |effective_state | type |state_change_time | completion_time | create_time | description}

CloudPools CLI commands

68 OneFS CloudPools 8.1.0 Administration Guide

Page 69: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Order results by this field. The default value is id. Note that, to sort on otherthan ID, description, effective state, and type, use the --verbose parameterwith the command.

--descendingSort and present data in descending order.

--format {table | json | csv | list}

Display output in table (default), JavaScript Object Notation (JSON), comma-separated value (CSV), or list format.

--no-headerDisplays table and CSV output without headers.

--no-footerDisplays table output without footers.

--verboseDisplays more detailed information.

isi cloud jobs pausePause a cloud job. A paused job can be resumed with the isi cloud jobs resumecommand.

Syntax

isi cloud jobs pause <id> [--verbose]

Options

id

The ID of the cloud job to pause. Use the isi cloud jobs list command toview the IDs of all cloud jobs. Although possible, we recommend that you notpause any of the CloudPools system jobs that run in the background and arecritical for proper operation. These include cache-writeback, cache-invalidation,local-garbage-collection, and cloud-garbage-collection.

--verboseDisplays more detailed information.

ExampleThe following example pauses a cloud job with ID 19.

isi cloud jobs pause 19

CloudPools CLI commands

isi cloud jobs pause 69

Page 70: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

isi cloud jobs resumeResume a paused cloud job.

Syntax

isi cloud jobs resume <id> [--verbose]

Options

<id>

The ID for the cloud job to resume. Use the isi cloud jobs list commandto view a list of jobs and their associated IDs.

--verboseDisplays more detailed information.

ExampleThe following command resumes a paused job with an ID of 26:

isi cloud jobs resume 26

isi cloud jobs viewView the details of a cloud job.

Syntax

isi cloud jobs view <id>

Options

<id>

Specify the ID of the cloud job. Use the isi cloud jobs list command toview all jobs and their associated IDs.

ExampleThe following command views the details of a job with the ID of 27:

isi cloud jobs view 27

isi cloud pools createCreate a CloudPool, which provides the connection between OneFS and a cloudstorage account.

Syntax

isi cloud pools create <name> <type> <account> [--description <string>]

CloudPools CLI commands

70 OneFS CloudPools 8.1.0 Administration Guide

Page 71: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

[--vendor <string>] [--verbose]

Options

<name>

The name of the CloudPool.

<type>

The type of account, one of isilon, azure, s3, ecs, or virtustream.

<account>

The name of the cloud storage account to which the CloudPool connects. Thecloud storage account is required and must match the CloudPool type. Only onecloud storage account can be specified.

--description <string>

A description of the CloudPool.

--vendor <string>

The name of the vendor hosting the cloud storage account.

--verboseDisplays more detailed information.

ExampleThis following command creates a CloudPool containing a Microsoft Azure cloudstorage account:

isi cloud pools create my_cp azure http://myazure.microsoft.com --description="Financial records 2013" --vendor=Microsoft

isi cloud pools deleteDelete a CloudPool. Proceed with caution, however. If you delete a CloudPool, OneFSis no longer able to access the associated cloud storage account. If the CloudPool isreferenced by a file pool policy, OneFS does not allow the CloudPool to be deleted.

Syntax

isi cloud pools delete <id> [--force] [--verbose]

Options

<id>

The name of the CloudPool. You can use the isi cloud pools list command to listexisting CloudPools and their associated IDs.

--forceDeletes the account without asking for confirmation.

--verbose

CloudPools CLI commands

isi cloud pools delete 71

Page 72: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Displays more detailed information.

Example 1 Example

The following command specifies a CloudPool to be deleted:

isi cloud pool delete my_azure_pool

When you press ENTER to run the command, OneFS asks for confirmation. Type yes,then press ENTER.

isi cloud pools listDisplay a list of CloudPools.

Syntax

isi cloud pools list[--limit <integer>][--sort {id | name | type | state | state_details | description| vendor}][--descending ][--format {table | json | csv | list}][--no-header][--no-footer][--verbose]

Options

--limit <integer>

Displays no more than the specified number of items.

--sort {id | name | type | state | state_details | description | vendor}

Order results by this field. The default value is id, which, in this case, is the sameas name. Unless you use the --verbose option, you can only sort on name,type, or state.

--descendingSorts and presents data in descending order.

--format {table | json | csv | list}

Displays output in table (default), JavaScript Object Notation (JSON), comma-separated value (CSV), or list format.

--no-headerDisplays table and CSV output without headers.

--no-footerDisplays table output without footers.

--verbose

CloudPools CLI commands

72 OneFS CloudPools 8.1.0 Administration Guide

Page 73: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Displays more detailed information.

isi cloud pools modifyModifies a CloudPool.

Syntax

isi cloud pools modify <id> [--name <string>] [--accounts <string>] [--add-accounts <string>] [--clear-accounts] [--remove-accounts <string>] [--description <string>] [--vendor <string>] [--verbose]

Options

<id>

The ID of the CloudPool. Run isi cloud pools list to view the IDs of allCloudPools.

--name <string>

Specify a new name for the CloudPool.

--account <string>

Specify the name of the cloud account to add to the CloudPool. Only one accountper CloudPool is allowed.

--add-account <string>

Specify the name of a cloud account to add to the CloudPool. Only one accountper CloudPool is allowed.

--remove-accounts <string>

Specify the name of the cloud account to remove from the CloudPool. You canonly remove an account if you are adding a different account in the samecommand.

--descriptionSpecify the name of the cloud account to remove from the CloudPool. You canonly remove an account if you are adding a different account in the samecommand.

---vendor <string>

The name of the vendor hosting the cloud pool accounts.

--verboseDisplays more detailed information.

CloudPools CLI commands

isi cloud pools modify 73

Page 74: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

ExamplesThe following command adds a vendor name and description to an existing CloudPool:

isi cloud pools modify my_azure --vendor Microsoft --description "preferred azure account"

The following command removes one cloud account from the CloudPool, and addsanother cloud account:

isi cloud pools modify my_s3 --remove-accounts s3_acct_1 --add-accounts s3_acct_2

isi cloud pools viewView detailed information about a CloudPool.

Syntax

isi cloud pools view <id>

Options

<id>

The ID of the cloud pool. Run the isi cloud pool list command to view allCloudPools and their associated IDs.

ExampleThe following command displays information about a CloudPool namedmy_azure_pool.

isi cloud pools view my_azure_pool

isi cloud proxies createCreates a network proxy through which a cloud storage account can connect to acloud storage provider.

Syntax

isi cloud proxies create <name> <host> <type> <port> [--username <string>] [--password <string>] [--verbose]

Options

<name>

The name of the network proxy. This can be any alphanumeric string, but shouldbe a simple, recognizable name.

<host>

The DNS name or IP address of the proxy server. For example,myproxy1.example.com or 192.168.107.107.

CloudPools CLI commands

74 OneFS CloudPools 8.1.0 Administration Guide

Page 75: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

<type>

The proxy protocol type, one of socks_4, socks_5, or http.

<port>

The port number to communicate with the proxy server. The correct port numberdepends on the port opened up on the proxy server for communication withCloudPools.

--username <string>

The user name to authenticate with the SOCKS v5 or HTTP proxy server. Notethat SOCKS v4 does not support authentication.

--password <string>

The password to authenticate with the SOCKS v5 or HTTP proxy server.

--verboseDisplays more detailed information.

ExamplesThe following example creates a network proxy to use with CloudPools:

isi cloud proxies create myproxy1 myprox1.example.com socks_5 1080 --username mycloudpools --password dhgXJ9OAIahXvYmL

isi cloud proxies deleteDelete a network proxy in CloudPools. Note that CloudPools prevents deletion of aproxy that is attached to a cloud storage account.

Syntax

isi cloud proxies delete <name>[--force][--verbose]

Options

<name>

The name of the network proxy. You can use the isi cloud proxies listcommand to display the names of proxies.

--forceEnables the proxy deletion to proceed without confirmation.

--verboseDisplays more detailed information.

ExampleThe following example deletes a network proxy named myproxy1:

isi cloud accounts delete myproxy1

CloudPools CLI commands

isi cloud proxies delete 75

Page 76: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

When you run the command, OneFS displays the following message and requiresconfirmation:

Are you sure? (yes/[no]):

To proceed, type yes, and press ENTER. If the proxy is attached to a cloud storageaccount, OneFS displays the following message:

Cannot delete proxy while used by accounts

isi cloud proxies listDisplays a list of network proxies created in CloudPools.

Syntax

isi cloud proxies list [--limit <integer>] [--sort {id | name | host | type | port}] [--descending] [--format {table | json | csv | list}] [--no-header] [--no-footer] [--verbose]

Options

--limit<integer>

Limits the number of network proxies displayed in the list.

--sortSort the list of cloud proxies according to the specified category. The followingvalues are valid:

idnamehosttypeport

--formatOutputs the list of network proxies in the specified format. The following valuesare valid:

tablejsoncsvlist

--descendingOutputs the list of network proxies in descending order according to the specifiedsort option.

--no-header

CloudPools CLI commands

76 OneFS CloudPools 8.1.0 Administration Guide

Page 77: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Displays table and CSV output without headers.

--no-footerDisplays table output without footers.

--verboseDisplays more detailed information.

ExampleThe following example creates a network proxy to use with CloudPools:

isi cloud proxies create myproxy1 myprox1.example.com socks_5 1080 --username mycloudpools --password dhgXJ9OAIahXvYmL

isi cloud proxies modifyModifies the properties of a network proxy.

Syntax

isi cloud proxies create <name> [--name <string>] [--host <string>] [--type {socks_4 | socks_5 | http}] [--port <integer>] [--username <string>] [--clear-username] [--password <string>] [--clear-password] [--verbose]

Options

<name>

The current name of the network proxy.

--name <string>

The new name of the network proxy. This can be any alphanumeric string, butshould be a simple, recognizable name.

--host <string>

The DNS name or IP address of the proxy server. For example,myproxy1.example.com or 192.168.107.107.

--typeThe network proxy protocol , one of socks_4, socks_5, or http.

--portThe port number to communicate with the proxy server. The correct port numberdepends on the port opened up on the proxy server for communication withCloudPools.

--username <string>

The user name to authenticate with the SOCKS v5 or HTTP proxy server. Notethat SOCKS v4 does not support authentication.

CloudPools CLI commands

isi cloud proxies modify 77

Page 78: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

--clear-usernameClear the user name that was previously specified for proxy server authentication.

--password <string>

The password to authenticate with the SOCKS v5 or HTTP proxy server.

--clear-passwordClear the password that was previously specified for proxy server authentication.

--verboseDisplays more detailed information.

ExamplesThe following example modifies a network proxy in CloudPools:

isi cloud proxies modify myproxy1 --type socks_4 --clear-username --clear-password

isi cloud proxies viewView the details of a network proxy created for CloudPools.

Syntax

isi cloud proxies view <name>

Options

<name>

Specifies the name of the network proxy to view. You can use the isi cloudproxies list command to display a list of the available proxies.

ExampleThe following example displays the details of a network proxy named myproxy1:

isi cloud proxies view myproxy1

isi cloud recallSpecify one or more files to be recalled from the cloud. You can specify filesindividually or by using a file matching pattern. To make sure that the specified filesare present in the cloud, OneFS scans the cluster for SmartLink files prior toperforming the recall.

Syntax

isi cloud recall <files> [--recursive {yes | no}] [--verbose]

Options

<files>

CloudPools CLI commands

78 OneFS CloudPools 8.1.0 Administration Guide

Page 79: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Specifies the files to recall. Specify --files for each additional file name.

--recursive {yes | no}

Specifies whether the recall should apply recursively to nested subdirectories.

--verboseDisplays more detailed information about the operation.

ExamplesThe following example recalls all files from the cloud for a directory and itssubdirectories:

isi cloud recall /ifs/data/archives/archives2014/projects/*.* --recursive yes

The command starts a cloud job. If you use the --verbose parameter, OneFS reportsthe job number, as in the following example:

Created job [29]

You can use the isi cloud jobs view command with the job number to seeinformation about the job.

Note

When you use the isi cloud recall command to recall a file from cloud storage,the full file is restored to its original directory, and the associated SmartLink file isoverwritten. If the file pool policy that originally archived the file to the cloud is still ineffect, the next time the SmartPools job runs, the recalled file is archived to the cloudagain. If you do not want the recalled file to be re-archived, you can move the file to adifferent directory that would not be affected by the file pool policy, or you canmodify or delete the policy.

isi cloud restore_coiRestores the cloud object index (COI) for a cloud storage account on the cluster. Ifyou run the isi cloud access add command on a cluster, it automaticallyrestores the COI.

Syntax

isi cloud restore_coi [--account <string>] [--expiration-date <timestamp>] [--verbose]

Options

--account <string>

Specifies the name of the cloud storage account whose COI you intend torestore. By restoring the COI, you enable OneFS to not only read data from thecloud, but also to write data to the cloud.

--expiration-date <timestamp>

CloudPools CLI commands

isi cloud restore_coi 79

Page 80: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Specifies the expiration date for orphaned cloud data objects.

--verboseDisplays more detailed information about the operation.

ExampleThe following example restores the COI for a cloud storage account:

isi cloud restore_coi --account my_azure_acct

isi cloud settings modifyControls archiving of snapshot files. By default, archiving of snapshots is enabled.

Syntax

isi cloud settings modify [--default-accessibility {cached | no-cache}] [--default-cache-expiration <duration>] [--default-compression-enabled {yes | no}] [--default-data-retention <duration>] [--default-encryption-enabled {yes | no}] [--default-full-backup-retention <duration>] [--default-incremental-backup-retention <duration>] [--default-read-ahead <string>] [--default-writeback-frequency <duration>] [--default-archive-snapshot-files {yes | no}] [--verbose]

Options

--default-accessibility {cached | no-cache}

Specifies whether, when a SmartLink file is accessed, cloud data is incrementallydownloaded (cached) as needed, or fully downloaded (not cached).

--default-cache-expiration <duration>

Specifies the minimum amount of time until the cache expires. A number followedby a unit of time is accepted. For example, a setting of 9H would specify a nine-hour duration. Similarly, a setting of 2D would specify a two-day duration.

--default-compression-enabled {yes | no}

Specifies whether data is to be compressed when archived to the cloud.

--default-data-retention <duration>

Specifies the minimum amount of time that cloud objects associated with aSmartLink file will be retained in the cloud after the SmartLink file is deleted fromthe cluster. A number followed by a unit of time is accepted. For example, asetting of 9H would specify a nine-hour duration. Similarly, a setting of 2D wouldspecify a two-day duration.

--default-encryption-enabled {yes | no}

Specifies whether data is to be encrypted when archived to the cloud.

--default-full-backup-retention <duration>

Specifies the length of time that OneFS retains cloud data referenced by aSmartLink file that has been backed up by a full NDMP backup and issubsequently deleted. A number followed by a unit of time is accepted. For

CloudPools CLI commands

80 OneFS CloudPools 8.1.0 Administration Guide

Page 81: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

example, a setting of 9H would specify a nine-hour duration. Similarly, a setting of2D would specify a two-day duration.

--default-incremental-backup-retention <duration>

Specifies the length of time that OneFS retains cloud data referenced by aSmartLink file that has been backed up by an incremental NDMP backup, orreplicated by a SyncIQ operation, and is subsequently deleted. A number followedby a unit of time is accepted. For example, a setting of 5Y would specify a five-year duration.

--default-read-ahead {partial | full}

Specifies the cache readahead strategy when SmartLink files are accessed. Apartial strategy means that only the amount of data needed by the user is cached.A full strategy means that all file data will be cached when the user accesses aSmartLink file.

--default-writeback-frequency <duration>

Specifies the minimum amount of time to wait before OneFS updates cloud datawith local changes. A number followed by a unit of time is accepted. For example,a setting of 9H would specify a nine-hour duration. Similarly, a setting of 2Dwould specify a two-day duration.

--default-archive-snapshot-files {yes | no}

Whether or not policies should archive files with snapshots.

--verboseDisplays more information about the operation.

ExampleThe following examples modifies several of the default CloudPools settings:

isi cloud settings modify --default-writeback-frequency 12H --default-cache-expiration 9H --default-accessability no-cache--default-encryption-enabled yes

isi cloud settings regenerate-encryption-keyGenerates a new master encryption key for data to be archived to the cloud.Encrypted data already stored in the cloud requires and stores the previous encryptionkey.

Syntax

isi cloud settings regenerate-encryption-key[--verbose]

Option

--verboseDisplays more detailed information.

CloudPools CLI commands

isi cloud settings regenerate-encryption-key 81

Page 82: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

isi cloud settings viewDisplay the current default settings in CloudPools. You can use the isi cloudsettings modify command to change default settings.

Syntax

isi cloud settings view

OptionsThere are no options for this command.

CloudPools CLI commands

82 OneFS CloudPools 8.1.0 Administration Guide

Page 83: OneFS CloudPools Isilon - Dell...If you have any questions about EMC Isilon products, contact EMC Isilon Technical Support. Online Support l Live Chat l Create a Service Request Telephone

Copyright © 2015 EMC Corporation. All rights reserved. Published in USA. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. EMC², EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date regulatory document for your product line, go to EMC Online Support (https://support.emc.com).