Storage ServicesCollecting the pieces of

the puzzleTNC 2008, Brugge, 21 May 2008

Jan Meijer

2

Something is brewing...

Storage as (a) utility service(s) within the

European higher educational and

research community

3

What if...

4

Wait, get rid of storage on campus?

Think about it: NAS, FC, iSCSI, backup, replication – as – a – service

The network makes it possible Smaller institutions (NREN “long tail”) Moving towards a virtualized world NREN “long tail” will benefit Centralized? Decentralized?

Federated!

5

Example: Høgskole i NoenBy

6

But, why? Economies of scale Allow sites to more easily reap benefits

from new storage technology Technology enables us to consider this

option these days Mixing different financial models: pay as

you go, upfront, etc. limited pool of experts: institutional,

national, within Europe need to deal with a growing array of

ever more complex technologies increased reliability demands are more

easily met in co-operation. Think disaster recovery

Do more with less money Storage as an enabler Where do YOU want to focus your IT

efforts on?

7

Example: NorStore Storage infrastructure for

Norwegian research Main objective:

Establish and maintain a broad and sustainable infrastructure for the curation, archiving and preservation of data from computational science and the natural sciences.

integrated part of the national e-Infrastructure (HPC, national network, national AAI and national grid)

http://www.norstore.no/

8

A global storage cloud for research & education?

Some time away....

9

Spot the differences

high capacity network as a utility

compute cycles as a utility

data storage as a utility

10

Lucky us! Fast networks! 1 PetaByte = 8,000,000,000,000,000

bits 10 Gbit = 10,000,000,000 bits 1 day is 86400 sec

And this matters why? NorStore has 2x600TB netto in phase

1 1 minute of JPEG2000 video = 6GB LHC > 1 PB/year raw data production And it just keeps growing!

11

Other challenges

proof of data ownership single name spacing, tight integration

between disparate storage elements what with storing data across

borders? predictability (measurements) reliability of our storage archiving & data curation! AAI-enabling of storage access

methods energy consumption

12

Some trends Improved understanding of storage

reliability. :( Storing is cheaper than re-acquiring

data. Reacquiring isn't always possible

Enabling new applications with old data

Increased pressure on data security stemming from incidents....

Storage virtualization, server virtualization

iSCSI, pNFS SSDs! Fast, high throughput, reliable

13

Proof, pudding, tasting... .hu: national storage service, iSCSI .pl: national storage service project national research data storage NREN storage consolidation LHC PetaFlops = PetaBytes ...

14

To conclude

People want to share their data People want to collaborate across

organizational borders People want to have access to their

data The technology has to support this

Virtualizing storage into a utility service across institutional

boundaries is needed, and a great opportunity for all

15

To close

Interested in the storage challenge? Join the TF-Storage :)

http://www.terena.org/activities/tf-storage/

Next meeting 12 September, Riga, Latvia, we're inviting vendors :)

Thanksjan dot meijer at uninett dot no

+47 90177711feel free to contact me

16

File Exchange

17

File exchange + EduGain + NorStore

18

Digital notary experiment

Investigating: Proof of data ownership Protecting data integrity Deployment options

19

Services? Building block services:

sites doing end-user frontends, end-user support

Block storage: iSCSI, FC (..), InfiniBand?

File systems: CIFS, (p)NFS, GPFS, ....

Archiving, data curation Full services: disaster

recovery, off site backup, storage frontends, ..?

Different service providers in one common (federated) framework!