Moving to Managed File Systems

September 2020

Notices

Customers are responsible for making their own independent assessment of the

information in this document. This document: (a) is for informational purposes only, (b)

represents current AWS product offerings and practices, which are subject to change

without notice, and (c) does not create any commitments or assurances from AWS and

its affiliates, suppliers or licensors. AWS products or services are provided “as is”

without warranties, representations, or conditions of any kind, whether express or

implied. The responsibilities and liabilities of AWS to its customers are controlled by

AWS agreements, and this document is not part of, nor does it modify, any agreement

between AWS and its customers.

© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved.

Contents

Introduction .......................................................................................................................... 1

Amazon EFS .................................................................................................................... 2

Amazon FSx for Windows File Server ............................................................................. 3

Use Cases ........................................................................................................................... 3

Use Case 1: Moving to fully managed SMB file shares with AWS ................................. 3

Use Case 2: Moving to EFS Hosted Home Directories for Users and Applications ...... 5

Moving your data to the Cloud using Online Data Transfer ............................................ 6

Moving your data to the Cloud using Offline Data Transfer to AWS .............................. 7

Conclusion ........................................................................................................................... 8

Contributors ......................................................................................................................... 9

Further Reading ................................................................................................................... 9

Document Revisions............................................................................................................ 9

Abstract

As enterprises and medium sized businesses take advantage of globalization, they

need to store and distribute data and information globally. For example, employees in

India may need to access data stored in London or New York, and the other way

around. The need to access data globally in a timely manner increases the total cost of

ownership (TCO) for storage. Organizations need to consider replication solutions,

additional networking WAN/LAN costs, and additional data center and infrastructure

costs. This whitepaper discusses two AWS services that can help organizations meet

these challenges by moving to a managed file system.

Amazon Web Services Moving to Managed File Systems

1

Introduction

According to IDG's research, 80% of data stored by organizations around the world is

unstructured, as reported by TechRepublic1. Unstructured data includes office

documents, department shared files, content management systems, web sites, test and

development workflows, database backups, social media feeds, digital pictures and

videos, audio transmissions, sensor data, and more. Unstructured data is most

commonly stored as files on a file system.

To meet the high demand for file storage, organizations have made significant financial

investments into traditional on-premises file storage systems and network attached

storage (NAS) systems from on-premises storage vendors. This has led to a

proliferation of file shares in corporate data centers over the last two decades.

However, the cost of owning and operating on-premises file storage systems goes

beyond the initial capital investment. There are operational costs, such as data center

real estate, power, cooling, hardware and software maintenance, regular patching and

updating, and repair or replacement of faulty hardware, as well as the specialized staff

required to manage and maintain this infrastructure. All this takes time, effort, and

money.

Also, for all storage systems, since they are hardware infrastructure, there will come a

time when the system can no longer provide enough storage, is not performant enough,

or becomes unsupported by the vendor. When this happens, a hardware refresh is

required.

While hardware refreshes typically involve the procurement and deployment of new

infrastructure, what is often overlooked during storage systems refresh is that

customers are repurchasing the capacity and performance of the retiring storage

system all over again. They are not simply investing in the capacity, capability, and

performance delta between the systems.

Apart from procuring and deploying the new infrastructure, a hardware refresh also

involves migrating the data. Whenever data is migrated from one system to another,

there is a risk of data loss or data corruption, as well as the risks associated with cutting

over to a new storage system due to the potential downtime and application

reconfiguration involved. Additionally, the man-hours required for these types of

migrations can take weeks, if not months. All of which are hidden, but nonetheless real

costs.

Amazon Web Services Moving to Managed File Systems

2

What organizations are ultimately looking for are storage solutions that meet the

following criteria:

• Enable them to procure capacity on-demand, avoid large upfront capital

investments to test business ideas, and avoid stranded capacity costs.

• Provide global access to data in a secure and compliant manner.

• Reduce overhead costs associated with managing and supporting storage

overall.

• Reduce the total cost of ownership (TCO) associated with ever-expanding

storage needs.

• Transform their organizations faster, innovate quicker, while delivering the

highest level of availability, durability, and recoverability.

With Amazon Web Services (AWS), customers don’t need to worry about the underlying

infrastructure and only need to focus on the functionality of the service they want to use,

at any capacity or performance scale.

Amazon Web Services managed file services give customers the ability to move these

workloads to the cloud. We will discuss the two most common services in the following

section.

Amazon EFS

Amazon Elastic File System (Amazon EFS) provides a simple, scalable, fully managed,

elastic cloud-native NFS file system for use with AWS Cloud services and on-premises

resources. It supports NFS v4 and is frequently used by customers in simplifying their

migrating NAS applications from on-premises to the AWS Cloud. It is built to scale on

demand to petabytes without disrupting applications, growing and shrinking

automatically as you add and remove files, eliminating the need to provision and

manage capacity.

By using Amazon EFS, customers get managed file storage infrastructure, which

removes the complexity of deploying, patching, and maintaining the underlying complex

infrastructure. Customers can also use Amazon EFS to migrate locally running

enterprise applications to the AWS Cloud.

Amazon Web Services Moving to Managed File Systems

3

Amazon FSx for Windows File Server

Amazon FSx for Windows File Server provides fully managed, highly reliable file

storage that is accessible over the industry-standard Server Message Block (SMB)

protocol. It is built on Windows Server, delivering a wide range of administrative

features such as user quotas, end-user file restores, and Microsoft Active Directory (AD)

integration. It offers single-AZ and multi-AZ deployment options, fully managed

backups, and encryption of data at rest and in transit. Amazon FSx file storage is

accessible from Windows, Linux, and macOS compute instances and devices running

on AWS or on premises. You can optimize cost and performance with SSD and HDD

storage options. Amazon FSx helps you reduce TCO with data deduplication.

Use Cases

In this white paper, we explore use cases where fully managed cloud file systems excel:

• Migrating and running managed Windows File Shares (SMB)

• Running network file system (NFS) file shares on AWS

Use Case 1: Moving to fully managed SMB file shares

with AWS

Figure 1 shows a very typical enterprise with on-premises NAS file storage with access

to/from multiple geographical locations.

Amazon Web Services Moving to Managed File Systems

4

Figure 1 - How users access data on NAS or traditional file servers

File storage is widely used by organizations within their on-premises infrastructure.

Windows file storage is used for general purpose file/print servers, home directories,

media repositories and a wide range of other uses since it is easy for end-users to

share files, regardless of where they are located. File sharing is often integrated with

users’ native environments. AWS provides a fully managed file system service that

reduces operational burdens for IT teams, helps customers to lower costs, and improve

agility, which leads to better and faster business insights. Using FSx for Windows File

Server, customers can improve file storage availability with the use of multiple

Availability Zones (AZs), which eliminates the need to deal with the complexity or cost of

setting up multi-site clustering and replication. With the data in AWS, customers can

innovate faster by taking advantage of integrations with other AWS services. Amazon

FSx integrates with AWS hosted directory services, such as AWS Managed Microsoft

AD.

Using AWS Best Practices for Windows File Services can help organizations with the

following activities:

• Consolidating file shares with DFS and FSx

• Extending Enterprise AD to FSx for Windows File Server

Amazon Web Services Moving to Managed File Systems

5

• Using Shadow copies to effectively protect data on FSx

Figure 2 shows multiple ways to access a centralized data store in the cloud that is fully

managed and secured by AWS. For SMB file shares, customers can deploy Amazon

FSx for Windows File Server.

Figure 2 - How global users access data stored in AWS cloud file services

Use Case 2: Moving to EFS Hosted Home Directories

for Users and Applications

Many business-critical enterprise applications depend on NFS file systems, other

scientific and engineering workloads like CAD and Bio-Informatics also rely on this type

of shared storage. Amazon EFS provides a fully managed file system service, which

means that customers no longer need to worry about procuring, managing, or

supporting storage system hardware or software. There is no need to plan for

infrastructure obsolescence and data migrations resulting from hardware refreshes also

go away.

AWS offers multiple fully managed cloud file services to meet the varying needs of

different workloads. The following options enable customers to unlock the true and full

potential of their data:

Amazon Web Services Moving to Managed File Systems

6

• Home directories: A common use case for shared file services. Customers with

learning management systems turn to Amazon EFS to store data that supports

their education platform. They are able to leverage the performance of Amazon

EFS with little modification to existing processes or permissions. Customers no

longer run into capacity, performance, and reliability issues as opposed to on-

premises storage.

• Application Storage: Many applications depend on file-based storage for

unstructured data, including pictures, pdfs, video, and even binary frameworks.

Developers are now able to define infrastructure as code, and engineers are now

able to reproduce environments in minutes and iteratively improve software

components in a cost-effective way.

Amazon EFS will respect the UIDs/GIDs on Linux hosts that are integrated with

enterprise identity and directory services, such as LDAP. Additionally, Amazon EFS

integrates with AWS hosted identity services, such as AWS Identity and Access

Management (IAM).

Apart from reducing management overhead, both services provide different storage

options for cost optimization, enabling enterprises to scale capacity and performance

independently as needed.

Amazon EFS provides a fully managed file system matched to the specific application

needs. No proprietary knowledge of the inner workings of the underlying storage

infrastructure, nor specifics around the access protocol are needed. Customers are able

to quickly deploy file systems based on capacity and performance requirements,

eliminating operational burden.

Moving your data to the Cloud using Online Data

Transfer

AWS DataSync makes it simple and fast to move large amounts of data online between

on-premises storage and Amazon EFS, or Amazon FSx for Windows File Server.

AWS DataSync eliminates or automatically handles many manual administrative tasks

including scripting copy jobs, scheduling and monitoring transfers, validating data, and

optimizing network utilization. DataSync uses a purpose-built protocol to transfer

hundreds of terabytes and millions of files at speeds up to 10 times faster than open-

source tools, over the internet or AWS Direct Connect links.

Amazon Web Services Moving to Managed File Systems

7

DataSync can be used to migrate active data sets or archives to AWS, transfer data to

the cloud for timely analysis and processing, or replicate data to AWS for business

continuity.

To learn how to transfer files from an existing on-premises file system to file system,

see Getting Started with AWS DataSync in the AWS DataSync User Guide.

Data can also be transferred in a more traditional manner by leveraging an existing

connection to AWS and combining that with operating system specific or open-source

tools. For example, customers could use rsync or fpsync for NFS shares, and

RoboCopy for SMB shares.

Moving your data to the Cloud using Offline Data

Transfer to AWS

The AWS Snow Family helps customers that need to run operations in austere, non-

data center environments, and in locations where there's lack of consistent network

connectivity. The Snow Family, comprised of AWS Snowcone, AWS Snowball, and

AWS Snowmobile, offers a number of physical devices and capacity points, most with

built-in computing capabilities. These services help physically transport up to exabytes

of data into and out of AWS.

• AWS Snowcone is the smallest member of the AWS Snow Family of edge

computing, edge storage, and data transfer devices, weighing in at 4.5 pounds

(2.1 kg) with 8 terabytes of usable storage. You can execute compute

applications at the edge, and you can ship the device with data to AWS for offline

data transfer, or you can transfer data online with AWS DataSync from edge

locations.

• AWS Snowball Edge is a petabyte-scale edge computing, data migration, and

edge storage device designed for local storage, large scale-data transfer,

advanced machine learning, and full motion video analysis in disconnected

environments.

• AWS Snowmobile moves up to 100 PB of data in a 45-foot long ruggedized

shipping container and is ideal for multi-petabyte or Exabyte-scale digital media

migrations and data center closures.

Amazon Web Services Moving to Managed File Systems

8

Conclusion

Regardless of the workload that a customer wishes to run; either traditional on-premises

workloads or modern application deployments, many applications require access to

shared file storage.

AWS offers several simple, scalable, and fully managed cloud-based file services to

meet the needs of varying workloads. By moving workloads to AWS, customers can

accelerate digital transformation projects, improve application performance, and reduce

cost. Additionally, by leveraging the multi-AZ capabilities of AWS, customers can often

improve the reliability of their storage architecture, without the costs associated with

traditional multi-site on-premises storage system deployments.

Customers no longer have to set up and maintain multiple datacenters, license software

to integrate the two sites, maintain the network connections between the sites, and

operate and manage the infrastructure at each site. In doing so, AWS can save

customers significant capital and operational expenses.

Depending on whether a customer is moving from an on-premises storage system to a

fully managed cloud file system, or whether a customer is implementing a modern

application deployment model for the first time in the cloud.

Customers can create file systems on Amazon EFS and Amazon FSx in a single AZ, or

across multiple AZs, using the AWS Management Console, AWS CLI, or AWS SDK.

File systems can scale storage capacity and scale throughput capacity at any time,

making it easy to respond to changing business needs.

To help customers accelerate moving on-premises data into the AWS cloud, AWS

offers several different ways of getting data into the AWS cloud. Alternatively,

customers can also leverage the AWS Migration Acceleration Program for Storage.

MAP for Storage provides prescriptive guidance, repeatable methodology, tools,

expertise, and incentives to accelerate storage migration to the cloud. As part of the

AWS Migration Services portfolio, MAP for Storage can offer AWS Professional

Services engagements, tools to understand your data and workflows, and even credits

to offset some of the costs of migration.

Amazon Web Services Moving to Managed File Systems

9

Contributors

Contributors to this document include:

• Scott Franks, Storage Specialist Solutions Architect, AWS

• Ju-Lien Lim, Sr Specialist Solutions Architect, Storage

Further Reading

For additional information, see:

• Shared File Storage on AWS

Document Revisions

Date Description

September 2020 First publication

Notes

1 TechRepublic