1. DuraCloud Pilot Program: utilizing cloud infrastructure as part of your preservation strategy
Michele Kimpton
Project Director, DuraCloud
NDIPP Partner meeting
July 21, 2010

2. DuraSpace not for profit
Follows the Apache community model, supports communities, in a unique position to look forward

3. Overview What is DuraCloud Results of survey Pilot program Use cases
Future direction

4. Future Direction, Disruptive Technologies
Interoperable
Web enabled
Distributed
collaborative

5. Cloud Infrastructure
A style of computing where massively scalable IT-related capabilities are provided “as a service” using Internet technologies to multiple external customers.
(Gartner, 6/08).

6. DuraCloud Platform
Open technology and hosted service for utilizing cloud infrastructure for preservation support and access services
Architectural Features:
Interoperable across multiple cloud providers
Web enabled
Built on highly scalable, flexible shared infrastructure
Open API’s for easy integration

7. Repository or CM system
Amazon
DuraCloud
EMC
3rd party access
Rackspace
Repository or CM system
Researcher apps
User apps
Other
Apps
Institutional clouds
If we look at DuraCloud as a component in the ecosystem of repository and cyberinfrastructure. You have userfacing and researcher applications end users might go to to get access to content within your repository ( front end). Your repository primary software for management of the content. DuraCloud an extension to your repository which enables you easily utilize cloud infrastructure for making a backup copy of your content, synchronizing your content with your repository,

8. DuraCloud High Level Interaction
API UI
Administration
Service Management
API
Storage Management
Services
Amazon S3
Rackspace CloudFiles
EMC Atmos
Other Clouds

9. Services and Capabilities
04/15/09
Services and Capabilities
Replication
Image Viewing
Image Transformation
Media Streaming
Bit Integrity Checking
…more on roadmap
Add software as service slide showing services relevant to this market- with Icons.
File format
validation
Parallel processing

10. DuraCloud Shared Services Vision
Services Registry
Verified Services
Replication
Bit Integrity Checking
Image Viewing
Public Services
Geo
Tagging
Twitter Publisher
Private Services
MyApp 1
more
Data-mining
Recollections
Drupal
Hosted
services
Submit
more
Publish
Private
services
MyApp 2
MyApp 3

11. Key Advantages Cloud completed 1/22/2010 145 participants higher ed

12. Key Challenges Cloud completed 1/22/2010 145 participants higher ed

13. Likely to use cloud services in next 12 months

14. Preservation support

15. Partners and Pilots Selected initial cloud providers
04/15/09
Partners and Pilots
Selected initial cloud providers
Selected 3 initial pilot partners
Each partner has a hosted duracloud instance that is run by us, DuraSpace on their behalf. Each has loaded roughly 10 TB of data into DuraCloud of content. Each had slightly different use cases and motivations for usind duracloud. All had interest in having a second copy of their content in a different geographic location, ideally web accessible. For purposes of todays talk, I will focus on WGBH use case.

16. Extended Pilot Partners
University
Use Case
Repository
Rice U
Preservation
DSpace, meta archive
Hamilton College
Access/international collaboration
Fedora
Northwestern U
Preservation books, audio, image
U of PEI
Image viewing/hosting
Fedora/Islandora
Cornell U
Data stream access and preservation
ICPSR
Access and Preservation
SUNY Buffalo
DSpace
IUPUI
Rhodes College
Image Access
North Carolina State U
CARL
Preservation and Services
Orbis Cascade Alliance
MIT
Preservation, OAIS compliance
Dspace

17. MIT DuraCloud use case, preservation support
-Retrieval of lost files ( admin error)
-Replacement of damaged files
DSpace
DIP
Submission files
Database
Asset stor
Ingest
AIP
File sync
Preservation services:
-Monitoring
-file retrieval
-access
-auditing
-replication
DuraCloud
Interest in coming up with the most cost effective and efficient way to recover individual files
Amazon RS
EMC

18. WGBH Access Services utilizing DuraCloud
Digital Access Management system
Fedora Repository
Open Vault
Ingest
Access file streaming
Access file
Access services:
Streaming
File format transformation
File access
collaboration
WGBH DuraCloud Instance
Amazon
EMC
RackSpace

19. Achievements during Initial Pilot
DuraCloud integrated with 3 cloud storage providers
Pilot partners loaded 30 TB into Duracloud
Integrated and deployed multiple independent services
Developed tools to overcome limitation of 5 GB file size and ease data loading

20. Lessons Learned Content transfer can be time intensive
Internet Latency is high
Minimize transactions across the wire
Data should be close to compute
Storage more mature than compute
Market still developing
EEMC
SSUN

21. Key milestones

22. DuraCloud now available open source
Open core
Open API
Open Source
Apache-style license
Architecture to create cloud networks
Public clouds
Private clouds
University consortia
Partner implementations/Integrations

23. Thank You For more information:
Come to our Pilot Partner panel at 2:45 pm today
DuraSpace organization:
Wiki: wiki.duraspace.org/display/duracloud/
DuraCloud project page: duracloud.org
DuraCloud demonstration:
DuraCloud open source: wiki.duraspace.org/display/duracloud/DuraCloud