1. Work Package 27 Orit Edelstein IBM Haifa Research Lab
APARSEN All Hands Meeting Den Haag, 3–4 December 2013

2. Work Package 27
Objective: Understand scalability gaps in long term digital preservation technologies and solutions that need to be addressed in order to attain the required levels of scalability
Task 2710: Review the scalability of storage and other techniques used and needed by partners
Task 2720: Evaluation and recommendations on scalability
Duration: August 2012-July 2013
Topic: Usability
Deliverables
D2701 Recommendations about scalability

3. Approach
Review the scalability requirements of preservation systems, obtaining information from:
A questionnaire about scalability needs for long term digital preservation
Existing surveys
Other digital preservation projects
Identify aspects that play a role in scalability
Analyze and identify gaps in technology that need to be addressed in order to attain the required levels of scalability
Summarize challenges and recommend areas that needs to be addressed

4. Information Sources Surveys
APARSEN questionnaire used for both Storage Solutions (WP23) and Scalability [2013]
KB survey for the National Coalition on Digital Preservation (NCDD) in The Netherlands [2012]
Parse.Insight survey [2009]
Scalability in selected preservation systems and Technologies
Commercial: Tessella’s SDB & Preservica, Ex Libris Rosetta
Open Source: Duraspace
EU Research Projects: ENSURE, SCAPE, SCIDIP-ES
Background: scalability and Cloud Storage

5. Scalability Analysis Observation Results on public organizations
Complex data for a long term preservation
Scalability requirements are more than the storage solution
Home grown systems vs. commercial solutions
Recommendations:
Scalability should be planned as an integral part of the tools and services development
Whenever possible, the use of commercially available systems, tools, and services should be considered instead of using in house developed solutions
In order to benefit from economies of scale, the use of cloud storage should be considered
Consider preservation as part of the overall organization’s activity

6. Recommendations from last review
N/A

7. Achievements in Year 3
Deliverable: D2701 Recommendations about scalability
Scalability should be planned as an integral part of the tools and services development
Whenever possible, the use of commercially available systems, tools, and services should be considered instead of using in house developed solutions
In order to benefit from economies of scale, the use of cloud storage should be considered
Consider preservation as part of the overall organization’s activity
Value of work done
Understanding of scalability
Direction to handle scalability

8. Common vision and VCoE Contribution to common vision and VCoE
The recommendations report can be used by the VCoE to increase awareness to scalability issues and suggest addressing scalability when developing preservation system
Common terminology

9. Plan for Year 4
N/A

10. Internal links Links to other activities in APARSEN
Related areas: storage solutions, preservation services, tools, cost models, interoperability, standards

11. External links Potential cooperation with other projects/initiatives
SCAPE?

12. Network of Excellence

13. Scalability Analysis: Parameters
Variety of data types
Size of the data
Number of digital objects and size of each object
Number of versions of the same object
Connections between different objects
Amount of metadata and connections between metadata
Retention Period
Availability
Distribution
Searchability
Degree of sharing