Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
www.point.de Confidential
PoINT Software & Systems
© 2019 PoINT Software & Systems GmbH All rights reserved.
Managing storage classes for infrequently accessed data
DLR - HSM User Conference 2019
PoINT Software & Systems GmbH June 2019
© 2019 PoINT Software & Systems GmbH All rights reserved.
2
PoINT Software & Systems
www.point.de Confidential
Focused Data management software is our core competence
Sustainable Because archiving means keeping data for a long period of time
Flexible Understanding customers’ changing demands
Focused Data management software is our core competence
www.point.de Confidential
PoINT Software & Systems
© 2019 PoINT Software & Systems GmbH All rights reserved.
Who we are
Foundation of PoINT Software & Systems GmbH in 1994
Experienced ISV for data and storage management software
Software “Made in Germany”
Global customers from small businesses to large enterprises
World-wide reseller network and close cooperation with major storage system vendors
4 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
4
PoINT Storage Manager Policy-based File Tiering, Archiving and Migration
Primary Storage NAS Systems
File Tiering & Archiving Policy-based archiving of inactive files from primary to secondary/archive storage
CIFS / NFS
Storage Systems ”off premises“
LAN
Primary Storage
Client and application transparency Archived files are visible through primary
storage Various options to access archived data
PoINT Storage Manager Software Solution Windows Server based
(VMware supported) HA by Secondary Access Server
or Windows Cluster
Secondary/Archive Storage HDD Appliances (CIFS, NFS) Object / Cloud (S3, Swift) Tape (LTO, LTFS, ACSLS) Optical Storage Systems
”on premises“
Conceptual Overview
5 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
5
PoINT Storage Manager Policy-based File Tiering, Archiving and Migration
PoINT EMC FileMover Protocol
PoINT NTFS/ReFS Service
PoINT NetApp FAS API
PoINT Web Link Service
Storage Manager Services
Object Store Connectors
HDD / NAS Connectors
Integration
Primary Storage
PSM Services
PSM Storage Connectors
Secondary / Archive Storage
Tape Library Connectors
Optical Library
Connectors Po
INT
Sto
rag
e M
anag
er
Applicance Connectors
Dell EMC Unity Windows NAS Systems NetApp FAS Generic
NAS Share Virtual „WORM“ File
System
PoINT Archive File System
Software Architecture
© 2019 PoINT Software & Systems GmbH All rights reserved.
6
PoINT Software & Systems
www.point.de Confidential
Focused Data management is our core competence
Sustainable Because archiving means keeping data for a long period of time
Flexible Understanding customers’ changing demands
7 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
7
PoINT Storage Manager Policy-based File Tiering, Archiving and Migration
Stubbing File System
Web Links File System
Application Client System
Web GUI Browser
Transparent file system access
“Pass through“
Previous versions in snapshots
Primary storage support
Any NAS system
Indirect file system access
HTTP/Internet
Functionality
Versions
Search
Requires installation
Independent of operating system
HTTP/Internet
Search
Archive Access
8 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
8
PoINT Storage Manager Policy-based File Tiering, Archiving and Migration
User Controlled Archiving
Use cases Accounting department archives invoices
Development department moves closed projects
Speciality department recalls entire directories for big data analytics
…
Windows Explorer shell extension
Menu is fully customizable by administrator
Shell commands rely on PSM extensive policy framework
© 2019 PoINT Software & Systems GmbH All rights reserved.
9
PoINT Software & Systems
www.point.de Confidential
Focused Data management is our core competence
Sustainable Because archiving means keeping data for a long period of time
Flexible Understanding customers’ changing demands
10 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
10
PoINT Storage Manager Policy-based File Tiering, Archiving and Migration
Multi-Cloud Support PSM supports up to 20 cloud providers per instance
AWS, Azure, Cloudian HyperStore, Dell EMC ECS, NetApp StorageGRID, Hitachi HCP, etc.
Up to 4 cloud providers in same Storage Vault
Reduce dependence on single cloud provider
Geographical distribution
Minimize cloud data egress cost
Different cloud providers for different data sets
Comply with legal requirements
Fast background migration of Archive Volumes (containers)
No interruption of operation
Private and/or public cloud
11 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
11
PoINT Storage Manager Policy-based File Tiering, Archiving and Migration
UDF Archive Volumes Universal Disc Format is well-known (e.g. used by optical discs)
Standardised format and platform-independent
Windows, Linux and MacOS can read UDF volumes
PSM uses UDF version 2.01 (sequential)
PSM gathers files in temporary image directory and creates Archive Volumes based on UDF
Volume size is appropriately adjustable to archive device/media
Files to be archived Creation of UDF Archive Volumes in image directory
Write finalized Archive Volumes to archive device
Vendor-Lock-In
12 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
12
PoINT Storage Manager Policy-based File Tiering, Archiving and Migration
Archive Migration
Archive Devices
Write to multiple targets in parallel Technology
separation Multi-Cloud …
New Archive Device
Long-Term Retention
Archive Volumes
In a few steps
Configure new archive device Choose source and target Start migration process
(accelerated by UDF Archive Volumes)
Logging of migration Remove legacy device
www.point.de Confidential
PoINT Software & Systems
© 2019 PoINT Software & Systems GmbH All rights reserved.
Object Storage
?
What about HSM for object storage?
Workloads
Archive Backup
Sync & Share Cloud native apps
IoT Analytics
…
www.point.de Confidential
PoINT Software & Systems
© 2019 PoINT Software & Systems GmbH All rights reserved.
? Storage classes at AWS, Google etc.
AWS Google
S3 Standard Multi-Regional / Regional
S3 Standard-IA Nearline
S3 Glacier Coldline
S3 Glacier Deep Archive
"Ice cold archive storage"
Acc
ess
fre
qu
en
cy
Pri
ce p
er
GB
Disk-based? Tape-based? Or optical media?
www.point.de Confidential
PoINT Software & Systems
© 2019 PoINT Software & Systems GmbH All rights reserved.
Is it really tape vs. cloud?
https://www.itwire.com/business-it-news/storage/82332-tape-is-the-new-tape,-says-azure-cto-for-now.html
“After evaluating various technologies including Blu-ray and magnetic disk, Microsoft came to the conclusion that tape was still the way to go for Azure's archival storage tier”
www.point.de Confidential
PoINT Software & Systems
© 2019 PoINT Software & Systems GmbH All rights reserved.
Don’t tier data from object storage to file system Limited metadata, limited character set, limited file name length, limited pathname
length, limited practical number of files, limited versioning FS overhead for maintaining file and directory structure (e.g. LTFS)
Don’t put S3 service on top of file system
Requires large amount of disk space for staging Intervening FS layer (see replacement of Ceph FileStore by BlueStore) Complexity and management overhead
Tiering object data DON’T
17 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
17
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
Object Storage (S3 REST API)
Up to 8 LTO Library Devices and up to 256 LTO Drives
Software Solution
18 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
18
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
Software Solution - Key Features
High performance in terms of data and object rates
High availability and reliability
High scalability (incl. load-balancing, redundancy and failover)
Data protection (erasure coding, authentication and encryption)
Self-monitoring, reporting and alerting
User management based on domain services (AD, LDAP)
19 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
19
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
System Components
Client Network
Tape Library
Administration Network
System Network (internal and
private)
PAG-IFN1
PAG-IFN2
PAG-IFN3
PAG-IFN4
PAG-DBN1
PAG-DBN2
PAG-DBN3
Client
Client
Client
Tape Library
Interface Nodes (IFN) S3 REST API Library Drives Control Up to 8 LTO drives Multiple active nodes
Database Nodes (DBN) Databases and Logs HTML admin interface Library Robotics Control Up to 4 nodes possible Up to 100 billion objects
per S3 bucket
www.point.de Confidential
PoINT Software & Systems
© 2019 PoINT Software & Systems GmbH All rights reserved.
PAG OEM Based on optical storage
technology Object interface Up to 181 PB capacity Up to 18 GB/sec
throughput
Cooperation with Sony OAI
21 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
21
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
DBN and IFN services on one node
Easy deployment
Single Node Solution
Tape Library Client Network
S3 API
DBN & IFN
Drives Control
Robotics Control
22 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
22
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
Read/Write Performance (examples)
1) May be limited by insufficient FC or Ethernet bandwidth or by maximum performance of clients
# Libraries # LTO-8 Drives
per Library Read/Write Rates
1 8 up to 7,200 MB/s1)
4 4 up to 14,400 MB/s1)
8 32 up to 230,400 MB/s1)
(theoretically)
23 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
23
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
Native Support for connected Drives, Loaders, Libraries No additional software or appliance required HPE, IBM, Overland-Tandberg, Qualstar, Quantum, Oracle StorageTek, … Rewritable and WORM media of LTO generations 5 and higher
Enhanced Support for Drive Features Logical Block Protection Data Compression and Encryption Drive Alarms, Error Logs and Cleaning
Management Functions for Offline Media Scheduled Media Quality Check Functions
Includes automatic copying and recovery if applicable
Barcode Labels required
LTO Device and Media Support
24 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
24
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
Architecture Overview
Archival Storage Device Group of up to 8 LTO tape
loaders or library units
Archival Storage Volume - LTO medium
Protected Volume Array (PVA) - 1-4 volumes - Protected by Erasure Codes
Archival Storage Partition Space provided by PVAs Expands automatically
Object Repository (Bucket)
S3
PVA PVA
Partition
25 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
25
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
Erasure Coding (Mirroring) Code rate 1/2 (or 1+1)
Ap
pli
cati
on
Ob
ject
Str
eam
S1 S1
S1
64K Segment Size {
Write Path
LTO Drives
S1
Ob
ject
Str
eam
Ap
pli
cati
on
S1
S1
Read Path
S3 PUT
Write Segmentation Encoding Write
Tape
Read Reassambling
S3 GET
Read
26 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
26
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
Erasure Coding Code rate 3/4 (or 3+1)
Ap
pli
cati
on
Ob
ject
Str
eam
S3 PUT
EC
S1
S2
S3
S1
S2
S3
Write Segmentation Encoding Write
Tape
EC
S1
S2
S3
64K Segment Size {
Write Path
LTO Drives
Read Path
Read
S1
S2
S3
Reassambling
Ob
ject
Str
eam
Ap
pli
cati
on
S3 GET
Read
27 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
27
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
PoINT Archival Gateway
Use Cases
Hyp
erS
tore
Au
to-T
ierin
g
Sto
rage
GR
ID
ILM
HD
FS
S3
a
ba
cke
nd
S3
AP
I
Any S3
compatible App
Ge
ne
ric S
3
Co
nn
ecto
r
28 © 2019 PoINT Software & Systems GmbH All rights reserved.
www.point.de Confidential
28
PoINT Archival Gateway High Performance S3 Gateway for Tape Storage
Demo
© 2019 PoINT Software & Systems GmbH All rights reserved.
29
PoINT Software & Systems
www.point.de Confidential
www.point.de