Moving to Managed File Systems
September 2020
Notices
Customers are responsible for making their own independent assessment of the
information in this document. This document: (a) is for informational purposes only, (b)
represents current AWS product offerings and practices, which are subject to change
without notice, and (c) does not create any commitments or assurances from AWS and
its affiliates, suppliers or licensors. AWS products or services are provided “as is”
without warranties, representations, or conditions of any kind, whether express or
implied. The responsibilities and liabilities of AWS to its customers are controlled by
AWS agreements, and this document is not part of, nor does it modify, any agreement
between AWS and its customers.
© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved.
Contents
Introduction .......................................................................................................................... 1
Amazon EFS .................................................................................................................... 2
Amazon FSx for Windows File Server ............................................................................. 3
Use Cases ........................................................................................................................... 3
Use Case 1: Moving to fully managed SMB file shares with AWS ................................. 3
Use Case 2: Moving to EFS Hosted Home Directories for Users and Applications ...... 5
Moving your data to the Cloud using Online Data Transfer ............................................ 6
Moving your data to the Cloud using Offline Data Transfer to AWS .............................. 7
Conclusion ........................................................................................................................... 8
Contributors ......................................................................................................................... 9
Further Reading ................................................................................................................... 9
Document Revisions............................................................................................................ 9
Abstract
As enterprises and medium sized businesses take advantage of globalization, they
need to store and distribute data and information globally. For example, employees in
India may need to access data stored in London or New York, and the other way
around. The need to access data globally in a timely manner increases the total cost of
ownership (TCO) for storage. Organizations need to consider replication solutions,
additional networking WAN/LAN costs, and additional data center and infrastructure
costs. This whitepaper discusses two AWS services that can help organizations meet
these challenges by moving to a managed file system.
Amazon Web Services Moving to Managed File Systems
1
Introduction
According to IDG's research, 80% of data stored by organizations around the world is
unstructured, as reported by TechRepublic1. Unstructured data includes office
documents, department shared files, content management systems, web sites, test and
development workflows, database backups, social media feeds, digital pictures and
videos, audio transmissions, sensor data, and more. Unstructured data is most
commonly stored as files on a file system.
To meet the high demand for file storage, organizations have made significant financial
investments into traditional on-premises file storage systems and network attached
storage (NAS) systems from on-premises storage vendors. This has led to a
proliferation of file shares in corporate data centers over the last two decades.
However, the cost of owning and operating on-premises file storage systems goes
beyond the initial capital investment. There are operational costs, such as data center
real estate, power, cooling, hardware and software maintenance, regular patching and
updating, and repair or replacement of faulty hardware, as well as the specialized staff
required to manage and maintain this infrastructure. All this takes time, effort, and
money.
Also, for all storage systems, since they are hardware infrastructure, there will come a
time when the system can no longer provide enough storage, is not performant enough,
or becomes unsupported by the vendor. When this happens, a hardware refresh is
required.
While hardware refreshes typically involve the procurement and deployment of new
infrastructure, what is often overlooked during storage systems refresh is that
customers are repurchasing the capacity and performance of the retiring storage
system all over again. They are not simply investing in the capacity, capability, and
performance delta between the systems.
Apart from procuring and deploying the new infrastructure, a hardware refresh also
involves migrating the data. Whenever data is migrated from one system to another,
there is a risk of data loss or data corruption, as well as the risks associated with cutting
over to a new storage system due to the potential downtime and application
reconfiguration involved. Additionally, the man-hours required for these types of
migrations can take weeks, if not months. All of which are hidden, but nonetheless real
costs.
Amazon Web Services Moving to Managed File Systems
2
What organizations are ultimately looking for are storage solutions that meet the
following criteria:
• Enable them to procure capacity on-demand, avoid large upfront capital
investments to test business ideas, and avoid stranded capacity costs.
• Provide global access to data in a secure and compliant manner.
• Reduce overhead costs associated with managing and supporting storage
overall.
• Reduce the total cost of ownership (TCO) associated with ever-expanding
storage needs.
• Transform their organizations faster, innovate quicker, while delivering the
highest level of availability, durability, and recoverability.
With Amazon Web Services (AWS), customers don’t need to worry about the underlying
infrastructure and only need to focus on the functionality of the service they want to use,
at any capacity or performance scale.
Amazon Web Services managed file services give customers the ability to move these
workloads to the cloud. We will discuss the two most common services in the following
section.
Amazon EFS
Amazon Elastic File System (Amazon EFS) provides a simple, scalable, fully managed,
elastic cloud-native NFS file system for use with AWS Cloud services and on-premises
resources. It supports NFS v4 and is frequently used by customers in simplifying their
migrating NAS applications from on-premises to the AWS Cloud. It is built to scale on
demand to petabytes without disrupting applications, growing and shrinking
automatically as you add and remove files, eliminating the need to provision and
manage capacity.
By using Amazon EFS, customers get managed file storage infrastructure, which
removes the complexity of deploying, patching, and maintaining the underlying complex
infrastructure. Customers can also use Amazon EFS to migrate locally running
enterprise applications to the AWS Cloud.
Amazon Web Services Moving to Managed File Systems
3
Amazon FSx for Windows File Server
Amazon FSx for Windows File Server provides fully managed, highly reliable file
storage that is accessible over the industry-standard Server Message Block (SMB)
protocol. It is built on Windows Server, delivering a wide range of administrative
features such as user quotas, end-user file restores, and Microsoft Active Directory (AD)
integration. It offers single-AZ and multi-AZ deployment options, fully managed
backups, and encryption of data at rest and in transit. Amazon FSx file storage is
accessible from Windows, Linux, and macOS compute instances and devices running
on AWS or on premises. You can optimize cost and performance with SSD and HDD
storage options. Amazon FSx helps you reduce TCO with data deduplication.
Use Cases
In this white paper, we explore use cases where fully managed cloud file systems excel:
• Migrating and running managed Windows File Shares (SMB)
• Running network file system (NFS) file shares on AWS
Use Case 1: Moving to fully managed SMB file shares
with AWS
Figure 1 shows a very typical enterprise with on-premises NAS file storage with access
to/from multiple geographical locations.
Amazon Web Services Moving to Managed File Systems
4
Figure 1 - How users access data on NAS or traditional file servers
File storage is widely used by organizations within their on-premises infrastructure.
Windows file storage is used for general purpose file/print servers, home directories,
media repositories and a wide range of other uses since it is easy for end-users to
share files, regardless of where they are located. File sharing is often integrated with
users’ native environments. AWS provides a fully managed file system service that
reduces operational burdens for IT teams, helps customers to lower costs, and improve
agility, which leads to better and faster business insights. Using FSx for Windows File
Server, customers can improve file storage availability with the use of multiple
Availability Zones (AZs), which eliminates the need to deal with the complexity or cost of
setting up multi-site clustering and replication. With the data in AWS, customers can
innovate faster by taking advantage of integrations with other AWS services. Amazon
FSx integrates with AWS hosted directory services, such as AWS Managed Microsoft
AD.
Using AWS Best Practices for Windows File Services can help organizations with the
following activities:
• Consolidating file shares with DFS and FSx
• Extending Enterprise AD to FSx for Windows File Server
Amazon Web Services Moving to Managed File Systems
5
• Using Shadow copies to effectively protect data on FSx
Figure 2 shows multiple ways to access a centralized data store in the cloud that is fully
managed and secured by AWS. For SMB file shares, customers can deploy Amazon
FSx for Windows File Server.
Figure 2 - How global users access data stored in AWS cloud file services
Use Case 2: Moving to EFS Hosted Home Directories
for Users and Applications
Many business-critical enterprise applications depend on NFS file systems, other
scientific and engineering workloads like CAD and Bio-Informatics also rely on this type
of shared storage. Amazon EFS provides a fully managed file system service, which
means that customers no longer need to worry about procuring, managing, or
supporting storage system hardware or software. There is no need to plan for
infrastructure obsolescence and data migrations resulting from hardware refreshes also
go away.
AWS offers multiple fully managed cloud file services to meet the varying needs of
different workloads. The following options enable customers to unlock the true and full
potential of their data:
Amazon Web Services Moving to Managed File Systems
6
• Home directories: A common use case for shared file services. Customers with
learning management systems turn to Amazon EFS to store data that supports
their education platform. They are able to leverage the performance of Amazon
EFS with little modification to existing processes or permissions. Customers no
longer run into capacity, performance, and reliability issues as opposed to on-
premises storage.
• Application Storage: Many applications depend on file-based storage for
unstructured data, including pictures, pdfs, video, and even binary frameworks.
Developers are now able to define infrastructure as code, and engineers are now
able to reproduce environments in minutes and iteratively improve software
components in a cost-effective way.
Amazon EFS will respect the UIDs/GIDs on Linux hosts that are integrated with
enterprise identity and directory services, such as LDAP. Additionally, Amazon EFS
integrates with AWS hosted identity services, such as AWS Identity and Access
Management (IAM).
Apart from reducing management overhead, both services provide different storage
options for cost optimization, enabling enterprises to scale capacity and performance
independently as needed.
Amazon EFS provides a fully managed file system matched to the specific application
needs. No proprietary knowledge of the inner workings of the underlying storage
infrastructure, nor specifics around the access protocol are needed. Customers are able
to quickly deploy file systems based on capacity and performance requirements,
eliminating operational burden.
Moving your data to the Cloud using Online Data
Transfer
AWS DataSync makes it simple and fast to move large amounts of data online between
on-premises storage and Amazon EFS, or Amazon FSx for Windows File Server.
AWS DataSync eliminates or automatically handles many manual administrative tasks
including scripting copy jobs, scheduling and monitoring transfers, validating data, and
optimizing network utilization. DataSync uses a purpose-built protocol to transfer
hundreds of terabytes and millions of files at speeds up to 10 times faster than open-
source tools, over the internet or AWS Direct Connect links.
Amazon Web Services Moving to Managed File Systems
7
DataSync can be used to migrate active data sets or archives to AWS, transfer data to
the cloud for timely analysis and processing, or replicate data to AWS for business
continuity.
To learn how to transfer files from an existing on-premises file system to file system,
see Getting Started with AWS DataSync in the AWS DataSync User Guide.
Data can also be transferred in a more traditional manner by leveraging an existing
connection to AWS and combining that with operating system specific or open-source
tools. For example, customers could use rsync or fpsync for NFS shares, and
RoboCopy for SMB shares.
Moving your data to the Cloud using Offline Data
Transfer to AWS
The AWS Snow Family helps customers that need to run operations in austere, non-
data center environments, and in locations where there's lack of consistent network
connectivity. The Snow Family, comprised of AWS Snowcone, AWS Snowball, and
AWS Snowmobile, offers a number of physical devices and capacity points, most with
built-in computing capabilities. These services help physically transport up to exabytes
of data into and out of AWS.
• AWS Snowcone is the smallest member of the AWS Snow Family of edge
computing, edge storage, and data transfer devices, weighing in at 4.5 pounds
(2.1 kg) with 8 terabytes of usable storage. You can execute compute
applications at the edge, and you can ship the device with data to AWS for offline
data transfer, or you can transfer data online with AWS DataSync from edge
locations.
• AWS Snowball Edge is a petabyte-scale edge computing, data migration, and
edge storage device designed for local storage, large scale-data transfer,
advanced machine learning, and full motion video analysis in disconnected
environments.
• AWS Snowmobile moves up to 100 PB of data in a 45-foot long ruggedized
shipping container and is ideal for multi-petabyte or Exabyte-scale digital media
migrations and data center closures.
Amazon Web Services Moving to Managed File Systems
8
Conclusion
Regardless of the workload that a customer wishes to run; either traditional on-premises
workloads or modern application deployments, many applications require access to
shared file storage.
AWS offers several simple, scalable, and fully managed cloud-based file services to
meet the needs of varying workloads. By moving workloads to AWS, customers can
accelerate digital transformation projects, improve application performance, and reduce
cost. Additionally, by leveraging the multi-AZ capabilities of AWS, customers can often
improve the reliability of their storage architecture, without the costs associated with
traditional multi-site on-premises storage system deployments.
Customers no longer have to set up and maintain multiple datacenters, license software
to integrate the two sites, maintain the network connections between the sites, and
operate and manage the infrastructure at each site. In doing so, AWS can save
customers significant capital and operational expenses.
Depending on whether a customer is moving from an on-premises storage system to a
fully managed cloud file system, or whether a customer is implementing a modern
application deployment model for the first time in the cloud.
Customers can create file systems on Amazon EFS and Amazon FSx in a single AZ, or
across multiple AZs, using the AWS Management Console, AWS CLI, or AWS SDK.
File systems can scale storage capacity and scale throughput capacity at any time,
making it easy to respond to changing business needs.
To help customers accelerate moving on-premises data into the AWS cloud, AWS
offers several different ways of getting data into the AWS cloud. Alternatively,
customers can also leverage the AWS Migration Acceleration Program for Storage.
MAP for Storage provides prescriptive guidance, repeatable methodology, tools,
expertise, and incentives to accelerate storage migration to the cloud. As part of the
AWS Migration Services portfolio, MAP for Storage can offer AWS Professional
Services engagements, tools to understand your data and workflows, and even credits
to offset some of the costs of migration.
Amazon Web Services Moving to Managed File Systems
9
Contributors
Contributors to this document include:
• Scott Franks, Storage Specialist Solutions Architect, AWS
• Ju-Lien Lim, Sr Specialist Solutions Architect, Storage
Further Reading
For additional information, see:
• Shared File Storage on AWS
Document Revisions
Date Description
September 2020 First publication
Notes
1 TechRepublic