White Paper: Managing an FLR-Enabled NAS Environment with ......retention period is set to “Infinite” unless a default retention period is configured. In FLR-E, this is a soft

White Paper

Abstract

The EMC® File-Level Retention (FLR) feature allows you to secure files on a production file system and retain the files on cost-effective network-attached storage. The FLR Toolkit helps you leverage the full capabilities of this feature in a CIFS environment. This white paper discusses how to use FLR Toolkit to manage and monitor files in an FLR-enabled file system. August 2013

Managing an FLR-Enabled NAS Environment with the EMC File-Level Retention Toolkit

2 Managing an FLR-Enabled NAS Environment with the File-Level Retention Toolkit

Copyright © 2013 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. Part Number h6310.4


Table of Contents

Executive summary.................................................................................................. 4 Audience ............................................................................................................................ 4 Terminology ....................................................................................................................... 4

Overview ................................................................................................................. 5 VNX File-Level Retention ..................................................................................................... 5

FLR states ....................................................................................................................... 6 EMC FLR Toolkit .................................................................................................................. 7

How to perform different operations ......................................................................... 8 Retention Dates .................................................................................................................. 8

Manually setting a Retention Date in Explorer ................................................................. 8 Setting a “soft” infinite Retention Date, and then a “hard” Retention Date ................... 10

Transitioning files to Append-only .................................................................................... 12 FLR Monitor Service .......................................................................................................... 15

Automatically set Retention Dates ................................................................................ 17 Automatically act on expired files ................................................................................. 23

Monitoring and report....................................................................................................... 27 Adding the FLR Retention Date and FLR State attribute column in Windows Explorer ..... 27 Querying for files with active retention periods ............................................................. 29 Running a report on FLR files by using a saved query .................................................... 30

Conclusion ............................................................................................................ 31

References ............................................................................................................ 32


Executive summary With the introduction of new regulations and compliance requirements, data has extended lifecycles that require longer maintenance periods. The rate at which files are generated is increasingly greater than the rate at which files are destroyed. Fixed-content data is consuming more than half of an average organization’s storage resources. Furthermore, safeguarding and ensuring the integrity of data is an essential business practice, if not an overarching federal requirement, in various industries such as medical, legal, and broadcasting. There is clearly an urgent need for cost-effective file retention or Write-Once Read-Many (WORM) technology. EMC® VNX File-Level Retention (FLR) is a software technology that addresses this need. With FLR, VNX offers a cost-effective file system solution that can manage files over an entire lifecycle at the compliance level.

For the VNX storage environment, FLR provides the infrastructure to protect file systems with files that require retention. However, FLR also needs an application for users to enable protection on the files. The FLR Toolkit is an application suite for Microsoft Windows platforms. The FLR Toolkit helps you manage, monitor, and enable retention on files that are stored on Common Internet File System (CIFS) shares of VNX file systems that have the FLR feature enabled.

Audience

This paper is intended to be used by EMC field personnel and customers who are familiar with EMC VNX technology, including the FLR Toolkit, but are not familiar with how to use the Toolkit in an FLR environment.

Terminology

This section covers some key terms that are used in this paper, but it does not include terms that are generic to the basic management and administration of VNX.

Append-only state — State of a file when the data in it cannot be modified, but the file can have new data appended to the end of it. In addition, the file itself cannot be deleted. Once a file in the Append-only state has been written to, changing it to the locked state, by making it read-only, locks it into that state until its Retention Date has passed.

Common Internet File System (CIFS) — File-sharing protocol based on the Microsoft Server Message Block (SMB). It allows users to share file systems over the Internet and intranets.

Expired state — State of a file when its Retention Date has passed. A file in the expired state can be reverted back to the Locked state or deleted from the FLR-enabled file system, but cannot be altered. If the expired file is empty, it can be transitioned to the Append-only state.


File-Level Retention (FLR) — FLR lets you store data on standard rewriteable magnetic disks using NFS or CIFS operations to create a permanent, unalterable set of files and directories.

FLR clock — Each file system has a non-modifiable system clock that it uses to track the Retention Date. It is initialized when an FLR-enabled file system is mounted read/write on a Data Mover. It does not advance when a file system is unmounted or mounted read-only.

Locked state — If a file system has File-Level Retention enabled, a file is in a Locked state when its permissions are changed to read-only. Files committed to the Locked (WORM) state cannot be altered or deleted until their Retention Date has passed.

Network File System (NFS) — Distributed file system that provides transparent access to remote file systems. NFS allows all network systems to share a single copy of a directory.

Not Locked state — Initial state of a file when it is created. A file that is Not Locked is treated like other files in the file system that are not enabled for FLR. This means it can be renamed, modified, or deleted.

Retention Date — Date until which a Locked file in an FLR-enabled file system will be protected. Users and applications manage a file’s Retention Date by using NFS or CIFS to set the file’s last access time to a future date and time. The retention timestamp is compared to the file system’s FLR clock to determine whether a file’s Retention Date has passed.

Overview This paper discusses the FLR operations that administrators use to manage and monitor files stored on CIFS shares of FLR-enabled file systems. It discusses some of the capabilities of the FLR Toolkit version 4.0, or later and how to use them in an FLR environment with EMC VNX. It does not discuss the design and architecture of the VNX FLR feature or the FLR Toolkit. It is important to note that the FLR Toolkit supports CIFS environments only.

This section describes the two main components that you use to manage and monitor a VNX FLR environment, FLR, and FLR Toolkit. The EMC File-Level Retention Toolkit can be found on EMC Online Support.

VNX File-Level Retention

VNX File-Level Retention (FLR) is an optional VNX software feature that protects files from modification or deletion until a specified Retention Date. FLR allows you to create a permanent, unalterable set of files and directories and ensures the integrity of the data. At the network-attached storage (NAS) level, FLR provides an option to retain necessary data to meet WORM requirements.

VNX FLR offers two options for protection: FLR-E (Enterprise) and FLR-C (Compliance). FLR-E provides a standard set of features to enforce WORM requirements and is


intended to be used by customers looking to regulate themselves. FLR-C provides a set of features that enables a system to satisfy regulatory requirements of United States Securities and Exchange Commission (SEC) Rule 17a-4 (f).

In FLR-C and FLR-E file systems, you cannot modify or delete files that are in the Locked state. Additionally, the path to a file in the Locked state is also protected from modification, which means you cannot delete or rename a directory on a FLR file system if it contains protected files.

If you try to delete a FLR-E file system that has files in the Locked state, a message will appear to warn you that you are attempting to delete a file system with locked files. For an FLR-E file system, you can confirm the message and proceed to delete the file system. To meet SEC compliance, you cannot delete a FLR-C file system with locked files.

In both FLR-E and FLR-C, when you lock a file without specifying a Retention Date the retention period is set to “Infinite” unless a default retention period is configured. In FLR-E, this is a soft infinite setting. This means that the user can change the Retention Date from infinite to a different date. However, to meet SEC compliance, the infinite setting is hard in FLR-C which means the user cannot change the retention period from infinite to another date. For this reason, it is important to exercise caution when locking a file in FLR-C.

FLR states

In a FLR-C or FLR-E file system, there are four states a file a file system can be in. Figure 1 shows the four possible states: Locked, Not Locked, Append-only, and Expired.

Figure 1 FLR state diagram

Not Locked — All files start as Not Locked. A Not Locked file is an unprotected file that is treated as a regular file in a file system. In an FLR file system, the state of an unprotected file can change to Locked or remain as Not Locked. Locked — A user cannot modify, extend, or delete a Locked file. The file remains Locked until its retention period expires. An administrator can perform two actions on a Locked file:


• Increase the file Retention Date to extend the existing retention period

• If the Locked file is initially empty, move the file to the Append-only state

Append-only — You cannot delete, rename, and modify the data in an Append-only file, but you can add data to it. The file can remain in the Append-only state forever. However, you can transition it back to the Locked state by setting the file status to Read-only with a Retention Date. Expired — When the retention period ends, the file transitions from the Locked state to the Expired state. You cannot modify or rename a file in the Expired state, but you can delete the file. An Expired file can have its retention period extended such that the file transitions back to the Locked state. An empty expired file can also transition to the Append-only state.

EMC FLR Toolkit

The FLR Toolkit helps you manage protected files that are stored on CIFS shares of VNX file systems that have FLR enabled. After a file system is created and enabled with FLR, you can apply FLR protection on each file manually, through an auto-lock policy, or through the Toolkit’s FLR Monitor Service. The Toolkit also provides a GUI tool, called FLR Explorer, for file querying, reporting, and status checks. FLR Toolkit is an application suite for Microsoft Windows platforms. FLR Toolkit provides users with GUI tools, a command line interface, and a software development kit to provide management solutions for FLR-enabled file systems. You can download the toolkit from EMC Online Support. You must install the toolkit on the client machine to use the following CLI and FLR Explorer tools.

The CLI tools are:

• CLI FLR Apply — Allows you to apply Retention Dates on files either individually or by specific directories.

• CLI FLR Query — Allows you to query the files within a CIFS share based on specified file-selection criteria and performs actions such as move, delete, and extend the Retention Date on the set of files returned by the query.

The FLR Explorer tools are:

• Dashboard — Provides an overall snapshot of a user-specified CIFS share. The dashboard generates a report with charts that show the percentage distribution of files in various FLR states, the number of files expiring within the next month on a weekly basis, and the number of files expiring in the current year on a monthly basis.

• Report generator — Builds detailed reports about files in a user-specified CIFS share. You can customize the report by FLR state, file type, Retention Date, and access and modification dates. The results are displayed in graphs and tables and are listed file by file with their corresponding path and Retention Date, if applicable.


• Query builder — Allows you to search for files in a CIFS share by specifying the path, FLR state, file type, and access and modification dates.

FLR Monitor Service is a tool that allows you to use the toolkit to actively monitor specified paths and automatically set Retention Dates on files created on specific directories. The tool can also monitor expiring files and act on them based on user-defined policies, such as deleting, moving, and extending Retention Dates on expired files. You use the FLR Monitor Service to apply different rules to different paths. This allows each path to have its own policy for retention setting and file retention period expiration.

It is important to note that you cannot use the FLR Toolkit to set Retention Dates or manage expired files if an auto-lock or auto-delete policy has been configured on the file system. If a default, minimum, and/or maximum retention date has been configured on the file system, Retention Dates set using the FLR Toolkit must comply with those rules.

How to perform different operations This section describes the steps to perform the following operations:

• Set a Retention Date:

Manually setting a Retention Date in Explorer

Setting a “soft” infinite Retention Date, and then a “hard” Retention Date

• Append-only files:

Transitioning files to Append-only

• FLR Monitor Service:

Automatically set Retention Dates

Automatically act on expired files

• Monitor and report:

Adding the FLR Date and FLR State attribute column in Windows Explorer

Querying for files with active retention periods

Running a report on FLR files by using a saved query builder

Retention Dates

Manually setting a Retention Date in Explorer

In a Windows environment, you can set a Retention Date by going to the file’s Properties window and selecting the FLR Attributes tab. The FLR Attribute tab is available on the client machine after the shell extensions component of the FLR Toolkit is installed on it. The FLR Attribute tab displays the FLR Date (or Retention Date), if applicable. The FLR Date is the date when the file expires.


You can set the Retention Date in three different ways by selecting the appropriate option in the Adjust FLR by field:

• Absolute Date and Time

• Incremental (with respect to the current date and time or the current Retention Date if it exists)

• Infinite retention period

Note that the specified retention date must comply with the minimum and maximum retention dates that are optionally configured on the file system.

To set a manual Retention Date:

1. Right-click on a file and click Properties.

2. Click FLR Attributes tab in the Properties window.

3. Select Absolute Date and Time, Incremental, or Infinite in the Adjust FLR by drop-down list box. In this example, the file is set with an incremental retention period of five days as shown in Figure 2.


Figure 2 Set the Retention Date to five days after the current date and time

4. Click OK to confirm the settings.

It is important to note that the FLR Date is derived from the current time of the client machine and not from the Data Mover. To synchronize the time reported on both the client machine and Data Mover, synchronize them to the same Network Time Protocol server. This can still represent an issue if the client machine and Data Mover are located in different time zones. For example, files may be listed as expired earlier than they should be because of a time zone difference.

Setting a “soft” infinite Retention Date, and then a “hard” Retention Date

This scenario is applicable only to files stored on an FLR-E-enabled file system. Users cannot change a Retention Date for a file that has been set with a “hard” infinite Retention Date on an FLR-C-enabled file system.


To set a “soft” infinite Retention Date:

1. Right-click on a file and click Properties.


2. Select Infinite from the Adjust FLR by drop-down list box as shown in Figure 3.

Figure 3 Setting a "soft" infinite retention period


Users can then reduce the retention period by setting an absolute or “hard” Retention Date. To set a “hard” Retention Date:

1. Right-click on the file and click Properties.


3. Select Absolute Date and Time from the Adjust FLR by drop-down list box.


4. Set the absolute Retention Date in the New FLR Value box such as 10/31/2014 at 11:59:59 PM as shown in Figure 4.

Figure 4 Setting a “hard” Retention Date with a “soft” infinite Retention Date


Transitioning files to Append-only

You can transition an empty file from the Locked state to an Append-only state. You also cannot delete files in the Append-only state. In the Append-only state, you can append data to a file but you cannot modify appended data in at a later time. You are allowed to append to the file until it is transitioned back to the Locked state.

For example, the Append-only state can be used for log files to protect previously written entries while allowing new entries to be appended to the end.

To set a file to Append-only:


1. Right-click on an empty file and click Properties.

2. Check the Read-only checkbox to set the file to a Locked State as shown in Figure 5.

Figure 5 Setting a file to the Locked state

3. Click Apply.

4. Uncheck the Read-only box to transition the file from the Locked State to the Append-only state as shown in Figure 6.


Figure 6 Transitioning an empty Locked file to Append-only


To lock the file at a later time:

1. Right-click on the file and click Properties.

2. Check the Read-only checkbox to set the file back to a Locked State as shown in Figure 5.

3. Set a Retention Date as shown in Figure 7.



Figure 7 Lock an Append-only file and set a Retention Date

FLR Monitor Service

FLR Monitor Service is a tool that allows you to use the toolkit to actively monitor specified paths and automatically set Retention Dates on files created on specific directories. The tool can also monitor expiring files and act on them based on user-defined policies, such as deleting, moving, and extending Retention Dates on expired files. You use the FLR Monitor Service to apply different rules to different paths. This allows each path to have its own policy for retention setting and file retention period expiration.

When using the FLR Monitor Service, it is important to note if a minimum and/or maximum retention date has been set on the file system. If you try to configure the FLR Monitor Service to set Retention Dates that are outside of the configured limits,


you will receive a warning and the retention date will be adjusted to fit in to the acceptable range.

To use the FLR Toolkit Monitor service:

1. Open FLR Monitor Service.

2. Click the Edit button and enter the credentials for a domain administrator account to run the service as shown in Figure 8.

Figure 8 Setting the Service Authentication account

3. Click Start to start the FLR Monitor Service as shown in Figure 9.


Figure 9 Starting the FLR Monitor Service

Automatically set Retention Dates

The FLR Monitor Service can actively monitor specific paths and automatically set Retention Dates on files in those paths if auto-lock is not enabled on the file system. To automatically set Retention Dates using FLR Monitor Service:



2. Click the FLR Connections tab. Click Add as shown in Figure 10.

Figure 10 Adding a new rule to set Retention Dates


3. Enter a path. You can also select Also monitor sub-directories to apply the rule for files in subdirectories under the path as shown in Figure 11.

Figure 11 Choosing the directory path for this rule


4. Click the Monitoring Options tab. The Monitoring Method specifies when to scan the specific path as shown in Figure 12. To use CIFS notifications to trigger a scan, select Fast. To configure a user-defined schedule, select Polling.

Figure 12 Choosing the Fast or Polling monitoring method


5. Click FLR Options. Select Absolute Date & Time, Incremental Date & Time, or Infinite and set a Retention Date as shown in Figure 13.

Figure 13 Setting a Retention Date and Time


6. If you want to exclude certain file types from this rule, click the Advanced button as shown in Figure 14.

Figure 14 Apply rule to all files or exclude certain extensions

7. Enter the extensions that you want to exclude in the format shown on Figure 14. Click OK to save the settings for the rule.

8. In the FLR Monitor Service application, click Apply to save the new rule.

9. When prompted to enable monitoring on the new connection, select Yes as shown in Figure 15. You can also select the connection and click the Enable button at a later time.


Figure 15 Enable monitoring on the new connection

Automatically act on expired files

You can manage expired files automatically through the FLR Monitor Service if auto-delete is disabled on the file system. The application provides users the ability to set a rule for a specified directory tree to handle Locked files as their retention period ends. Users can use the application to:

Delete expired files permanently

Move expired files to a specified location

Extend the retention period of expired files

To manage expired files with the FLR Monitor Service:



2. Click Expiration Rules tab and then click Add as shown in Figure 16.

Figure 16 Adding a new expiration rule

3. Enter a path. You can also select Also monitor sub-directories to apply the rule for files in subdirectories under the path.


4. Click the Execution Schedule tab. Select the frequency (daily or weekly) the rule will run on the specified directory tree as shown in Figure 17.

Figure 17 Configuring the execution schedule

5. Click Actions tab. Select the action you want to apply to expired files: Delete all files permanently, Move files to the directory specified below, or Extend retention period as specified below as shown in Figure 18. If you want to exclude certain file types from this rule, click the Advanced button.


Figure 18 Choosing the action to apply to expired files

6. Click OK to save the settings.

7. Click Apply on the FLR Monitor Service window to save the new expiration rule.

8. When prompted to enable the new expiry rules, select Yes as shown in Figure 19. You can also select the rule and click the Enable button at a later time.


Figure 19 Enabling the new expiry rule

Monitoring and report

Adding the FLR Retention Date and FLR State attribute column in Windows Explorer

You can monitor the FLR Date and FLR State for each file from Windows Explorer in Windows Server 2003 and XP. When the FLR Toolkit is installed, the FLR Date and FLR State attribute columns are available. Figure 20 on shows the FLR Date and FLR State columns in the file browser window.


Figure 20 Explorer window with the FLR Date and FLR State columns

To add the FLR Date and FLR State attribute columns:

1. In Explorer, right-click the attribute column heading as shown in Figure 2128.

Figure 21 Adding new attribute columns to Explorer

2. Click More.

3. Check the boxes for FLR Date and FLR State as shown in Figure 22.


Figure 22 Adding columns for FLR Date and FLR State to Explorer

4. Click OK.

Querying for files with active retention periods

To run a query on files stored on CIFS shares of FLR-enabled file systems, users can use the FLR Query CLI or the Query Interface in FLR Explorer. The following example shows how to build a query for files with active retention periods which expire in two years, except for files with a .ppt file extension.

To query for files with active retention periods the Query Interface:

1. Open FLR Explorer.

2. Click Build a Query.

3. In the Query Builder panel:

a. Select Files in Specific Retention State in the Search For drop-down list.

b. Enter the path to your share in the Search Path box.

c. Check the Include sub-directories checkbox to query the entire directory tree.

d. Select Exclude from the Search Pattern drop-down list and then type any files that you want to exclude in the adjacent box. Wildcards are accepted.


e. Select Will Expire Within from the Find Files That drop-down box and then enter the time frame in the Search Parameters group box.

f. Click Save and Execute.

g. When prompted, enter “Expire_Two_Years” as the file name.

Figure 23 shows the parameters entered for this example and the corresponding results.

Figure 23 Query Builder panel

Running a report on FLR files by using a saved query

This scenario describes how to build a report by using the Generate Reports option in FLR Explorer. The following example shows how to generate a report by using the query search parameters shown in Figure 23.

To run a report by using a saved query builder:

1. Open FLR Explorer.

2. Select Reports.

3. In the Report Builder panel:

a. Select Detailed from the Report Type drop-down list.


b. Select Expire_Two_Years from the Query Options drop-down list.

c. Click Generate Report.

The detailed report provides an overview of the file results segregated by file type. This report type also lists each reported file and its Retention Date. A file count is also provided. Figure 24 shows the search parameters and the detailed results.

Figure 24 Report Builder on FLR Explorer

Conclusion The EMC FLR Toolkit is an important component to manage and monitor a VNX FLR-enabled file system in a CIFS environment. The VNX FLR feature provides the software infrastructure to protect and retain mission-critical files. The toolkit provides users or administrators the ability to:

• Set Retention Dates on each file manually

• Set rules for the application to automatically set Retention Dates on multiple files

• Set policies for the application to automatically act on expiring Locked files

• Query files on CIFS shares of FLR-enabled file systems

• Build reports on files stored on CIFS shares of FLR-enabled file systems


The FLR Toolkit is an essential part of applying the FLR feature at the user level. With the FLR Monitor Service feature, an administrator can easily set policies to enable the system to automatically set Retention Dates and act on expiring files. The FLR Monitor Service allows you to set different policies on different subdirectories which enables you to precisely control the behavior of FLR files. Together, VNX FLR and the FLR Toolkit provide administrators with the ease of use and ease of manageability to strengthen the business practices in the company as file retention becomes a more important aspect of business practices.

References The following documents provide additional, relevant information about VNX FLR and FLR Toolkit. These documents can be found on EMC Online Support.

• Using VNX File-Level Retention

• File-Level Retention Toolkit Release Notes

Documents

White Paper: Managing an FLR-Enabled NAS Environment with ......retention period is set to “Infinite” unless a default retention period is configured. In FLR-E, this is a soft