View
212
Download
0
Category
Tags:
Preview:
Citation preview
Storage Services
Let the data flow!NorduNet 2008, .fi, 9 April 2008
Jan Meijer
2
Storage importance increasing
3
Something is brewing...
Storage as (a) utility service(s) within the
European higher ecudational and
research community
4
Spot the differences
high capacity network as a utility
compute cycles as a utility
data storage as a utility
5
Run that buzz by me again, please?
Storage as (a) utility service(s)
institutional / national / international scale
combining centralization, distribution, federation
creating a location and administrative domain agnostic ’make it so’ storage layer
6
So where does this fit in?
7
Drivers ’stuff’ as a service, the $500M data centers technology: because we can storing is cheaper then re-acquiring the
data. Sometimes reacquiring is impossible data explosion enabling new applications with old data limited pool of experts: institutional,
national, within Europe need to deal with an ever increasing
complexity and array of technologies increased reliability demands Do more with less money Storage as an enabler
8
the storage mix
Tiered storage HSM Tapes don’t spin..
9
Storage mix: Centralize, distribute, specialize
10
Revisiting that ’utility service’ buzz
’Half’ services: sites doing end-user frontends, end-user support
Block storage: iSCSI, FC (..), InfiniBand?
File systems: CIFS, (p)NFS, GPFS, ....
Archiving, data curation Full services: disaster
recovery, off site backup, storage frontends, ..?
11
Relevant tech trends
SSDs! Fast, high throughput, reliable
Improved understanding of storage reliability.
Storage middlewares, server virtualization
iSCSI Increasing demands for data
security Increasing pressure on availability
of data, services
12
Lucky us! Fast networks!
1 PetaByte = 8,000,000,000,000,000 bits
10 Gbit = 10,000,000,000 bits 1 day is 86400 sec
And this matters why? NorStore has 2x600TB netto in phase 1 1 minute of JPEG2000 video = 6GB LHC > 1 PB/year raw data production Data is kept!
13
Other challenges
proof of data ownership single name spacing, tight integration
between disparate storage elements what with storing data across borders? predictability (measurements) reliability of our storage archiving & data curation! AAI-enabling of storage access methods energy consumption
14
What are *we* doing?
Investigating what makes sense for our community
Coordinating, stimulating collaboration between sites (network!)
Learning...and putting that to use in
NorStore Experimenting
15
File Exchange
16
File exchange + EduGain + NorStore
17
Digital notary experiment
Investigating: Proof of data ownership Protecting data integrity Deployment options
18
What are folks doing?
.hu: national storage service, iSCSI
.pl: national storage service project
national research data storage NREN storage consolidation LHC PetaFlops = PetaBytes ...
19
Where is this going?
People want to share their data People want to collaborate across
organizational borders People want to have access to
their data The technology has to support this
Virtualizing storage into a service across institutional boundaries is
needed
20
To close with
http://www.terena.org/activities/tf-storage/
http://www.norstore.no/
Thank you
jan dot meijer at uninett dot no
BoF?
Recommended