Storage Services Let the data flow! NorduNet 2008,.fi, 9 April 2008 Jan Meijer

2 Storage importance increasing

3 Something is brewing... Storage as (a) utility service(s) within the European higher ecudational and research community

4 Spot the differences high capacity network as a utility compute cycles as a utility data storage as a utility

5 Run that buzz by me again, please? Storage as (a) utility service(s) institutional / national / international scale combining centralization, distribution, federation creating a location and administrative domain agnostic make it so storage layer

6 So where does this fit in?

7 Drivers stuff as a service, the $500M data centers technology: because we can storing is cheaper then re-acquiring the data. Sometimes reacquiring is impossible data explosion enabling new applications with old data limited pool of experts: institutional, national, within Europe need to deal with an ever increasing complexity and array of technologies increased reliability demands Do more with less money Storage as an enabler

8 the storage mix Tiered storage HSM Tapes dont spin..

9 Storage mix: Centralize, distribute, specialize

10 Revisiting that utility service buzz Half services: sites doing end-user frontends, end- user support Block storage: iSCSI, FC (..), InfiniBand? File systems: CIFS, (p)NFS, GPFS,.... Archiving, data curation Full services: disaster recovery, off site backup, storage frontends,..?

11 Relevant tech trends SSDs! Fast, high throughput, reliable Improved understanding of storage reliability. Storage middlewares, server virtualization iSCSI Increasing demands for data security Increasing pressure on availability of data, services

12 Lucky us! Fast networks! 1 PetaByte = 8,000,000,000,000,000 bits 10 Gbit = 10,000,000,000 bits 1 day is sec And this matters why? NorStore has 2x600TB netto in phase 1 1 minute of JPEG2000 video = 6GB LHC > 1 PB/year raw data production Data is kept!

13 Other challenges proof of data ownership single name spacing, tight integration between disparate storage elements what with storing data across borders? predictability (measurements) reliability of our storage archiving & data curation! AAI-enabling of storage access methods energy consumption

14 What are *we* doing? Investigating what makes sense for our community Coordinating, stimulating collaboration between sites (network!) Learning...and putting that to use in NorStore Experimenting

15 File Exchange

16 File exchange + EduGain + NorStore

17 Digital notary experiment Investigating: Proof of data ownership Protecting data integrity Deployment options

18 What are folks doing?.hu: national storage service, iSCSI.pl: national storage service project national research data storage NREN storage consolidation LHC PetaFlops = PetaBytes...

19 Where is this going? People want to share their data People want to collaborate across organizational borders People want to have access to their data The technology has to support this Virtualizing storage into a service across institutional boundaries is needed

20 To close with Thank you jan dot meijer at uninett dot no BoF?