“Storage is Cheap” and Other Lies:

Slides:



Advertisements
Similar presentations
Lesson 3: Working with Storage Systems
Advertisements

Digital Preservation Steps 1 & 2: Identify & Select.
Digital Collections: Storage and Access Jon Dunn Assistant Director for Technology IU Digital Library Program
Evaluation of Cloud Storage for Preservation and Distribution of Polar Data. Nadirah Cogbill Mentors: Marlon Pierce, Yu (Marie) Ma, Xiaoming Gao, and Jun.
Windows XP Photo Workflow Tim Grey Imaging Strategist Microsoft Corporation.
The Challenges and Rewards of Open Source Video Digitization Identification and Discussion of Proper Methods for Videotape Preservation when no Standards.
Bringing Preservation to the Forefront: Preservation Initiatives at the UCF Libraries Michael A. Arthur Head of Acquisitions & Collection Development Lee.
Chapter 5. Slide 1 Digital Archives, Collections Digital Archives, CollectionsObjectives  To provide or improve access to the most valuable and unique.
SIO AV Archiving Presentation Introduction to the Practices and Principles of Audio-Visual Archiving International Council on Archives, Section of International.
Digital Storage in the Cloud: Amazon Web Services & DSpace Barry Davis - Coordinator of Multimedia & Digital Production Services Kevin Gilbertson - Web.
Special collections and digital libraries: a new role for consortia? Dale Flecker Harvard University Library.
SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA, SAN DIEGO Disk and Tape Storage Cost Models Richard Moore & David Minor San Diego Supercomputer.
© 2012 TeraMedica, Inc. Archiving to a Secure, Enterprise Cloud: The Business Case Karen Burton, US Healthcare Mgr Mike Martin, VP Cloud Services.
DIGITIZATION OF AUDIOVISUAL COLLECTIONS: EMPOWERING PUBLIC LIBRARIES THROUGH THE PUBLIC-PRIVATE PARTNERSHIPS Bogdan Trifunović Digital Projects Librarian.
Designing Storage Architectures for Preservation Collections Library of Congress, September 17-18, 2007 Preservation and Access Repository Storage Architecture.
Data Preservation Best Practices for preserving your research data for future reuse The goal of data preservation is to ensure that your data is in a sustainable.
Co-funded by the European Union under FP7-ICT Co-ordinated by aparsen.eu #APARSEN Storage Solutions The use case at the National Library of the.
JMU Online Video Collection Background and Technical Processes for the VIVA Multimedia Collections Task Force Jeff Clark, Grover Saunders James Madison.
Disk and Tape Square Off Again Tape Remains King of Hill with LTO-4 Presented by Heba Saadeldeen.
Comp 1001: IT & Architecture - Joe Carthy 1 Information Representation: Summary All Information is stored and transmitted in digital form in a computer.
NCSU Libraries TRLN Digital Preservation Seminar NCSU.
Computers in the real world Objectives Explain the need for secondary storage devices Understand the three main storage types – Optical – Magnetic – Solid.
“Storage is Cheap” and Other Lies Lance Stuchell, University of Michigan Library Curating and Managing Research Data for Re-Use ICPSR, July 31, 2013 “Hard.
Allison Schein.  Adobe Audition (  Recommended program, metadata creation and manipulation is easy and complete.
Digital Preservation: Store & Protect Laurie Sauer Information Technologies Librarian Knox College
George the Magnificent Prepare to Be Amazed!. On Choosing A Preservation File Format for Video: “TIFFs are too big to store”, or “We Used JPEGs and.
OCLC Online Computer Library Center Digital Preservation with OCLC Digitization Standards: Issues & Updates Taylor Surface, OCLC.
FILE FORMATS & BITS AND BYTES VOCABULARY QUIZ PER-1 ALEX YUAN, SARAH GU, YANBING WANG, HYERIN YOON.
By:Billy Nguyen Teacher: Luce. Introduction Within technological terms, storage mainly refers and relates to computer devices that keep or retains information.
Session 2.  Wake Up Call, LSTA Digitization Grant  Digital Preservation Summit, May 2008  ISU Digital Preservation Group, September 2009.
IMAX® DMR™ Technology
Overview of Physical Storage Media
The Data Capacitor and Digital Libraries at IU Jon Dunn Associate Director for Technology IU Digital Library Program February 22, 2006.
Improving Data Ingest and Preservation at The Henry Ford Brian Wilson, The Henry Ford Musueum Archives Section Symposium SAA 2014 Annual Meeting.
Public Relations Interim Image Archive Goal: Provide and INTERIM image archive solution for Public Relations 2 to 4 TB of images currently spread across.
1 NETE4631 Working with Cloud-based Storage Lecture Notes #11.
Backup. What to back up? Pictures Documents (include s if using Off Line Program) Video’s Music System Image – Programs – Registry – Operating System.
Advanced AV Production Practicum Math for The Audio Video Professional Calculating Audio and Video File Sizes Copyright © Texas Education Agency, 2015.
Desks Drawers & Trusted Repositories: Lance Stuchell, MSI Digital Preservation Librarian, University of Michigan Wayne State University, 10/23/2012 Digital.
Digital Collections Forum Doug Moncur AIATSIS September 2004.
Computer Organization. The Five Hardware Units General purpose computers use the "Von Neumann" architecture Also referred to as "stored program" architecture.
2013. Exciting Items 4K everywhere Black Magic Design 4K Camera and Equipment. BMD Pocket Cinema Camera Freefly Movi More controllable LED lights New.
Got Data? A Guide to Data Preservation in the Information Age Written by Francine Berman Presented by Akadej Udomchaiporn.
Storage Why is storage an issue? Space requirements Persistence Accessibility Needs depend on purpose of storage Capture/encoding Access/delivery Preservation.
PRESERVATION IN A DIGITAL WORLD Presented By: Darrell Garwood Imaging Lab Manager Library and Archives Division Kansas State Historical Society
Advanced AV Production Practicum Math for The Audio Video Professional Calculating Single File Sizes Copyright © Texas Education Agency, All rights.
AMS02 Data Volume, Staging and Archiving Issues AMS Computing Meeting CERN April 8, 2002 Alexei Klimentov.
Managing live digital content with DuraSpace services Bill Branan PASIG Spring 2015.
Video Formats. 1.Students will understand what the difference is between analog and digital video. 2. Students will learn the advantages of digital video.
Digital Preservation What, Why, and How? Dan Albertson’s Digital Libraries Class April 13, 2016 Jody DeRidder Head, Metadata & Digital Services University.
John Walko, Scene Savers.  “No one told me…”  What prevents projects from getting done ◦ Apathy ◦ Obsolescence (Its too difficult) ◦ Degradation ◦ Labs.
Working with Cloud-based Storage
Digital Stewardship Curriculum
Bentley Project Reel Digitization Bentley Historical Library t
Digital GVSU: lessons learned in year 1
Chapter III, Desktop Imaging Systems and Issues: Lesson II Storing Image Data
Little work is accurate
AMCOM Digital Archive Design Review - Week 3.
Data Storage In today’s lesson we will look at:
Chapter 7.
Long-Term Storage Service for Memory Institutions and Research Centers
Data Compression.
Chapter 12: Archiving Rationale to tidy up project items
Preservation at the Bentley Historical Library
Digitization Workflow
Digital Stewardship Curriculum
Local Rules Apply: Creating and Sustaining a Cost Effective Digital Preservation System on a Limited Budget Matt Ransom, Digital Assets Manager Belk Library.
The Bentley Digital Media Library
IBM Tivoli Storage Manager
Archiving and preservation services in the cloud
Presentation transcript:

“Storage is Cheap” and Other Lies: “Hard Drives” by Michael Muni CC-BY-NC-ND Moving Image Digitization at MLibrary Lance Stuchell, University of Michigan Library Mid-Michigan Digital Practitioners Meeting. August 2, 2013

A/V Digitization at MLibrary Digitization of Special Collections material Audio: Uncompressed BWF master files 1 original audio object ≈ 5 GB storage Moving Image: Uncompressed preservation master Compressed production master 1 original video object ≈ 40.4 GB Storage

A/V Storage Estimates

A/V Storage Estimates TB/Year

Initial Costs Estimates Value Storage: $250/TB per year

Initial Costs Estimates Value Storage: $250/TB per year Tape Backups: $1,825/TB per year

Initial Costs Estimates Value Storage: $250/TB per year Tape Backups: $1,825/TB per year 2013: $32,450 2018: $245,325 2023: $485,200

Reaction

Current Costs Estimates Value Storage: $250/TB per year Tape Backups (MLib): Equipment costs - Cheaper most years

Current Cost Estimates

Current Cost Estimates

Current Cost Estimates

Total 11 Year Storage Cost $569,368

Current Cost Estimates

Current Cost Estimates

The New Normal “Optimistically, for the rest of this decade the rapid decrease in cost per bit of storage that has been a constant of the last three decades will be much slower; it might even stop.” David S. H. Rosenthal, et al., “The Economics of Long-Term Digital Storage” 2012. http://www.lockss.org/locksswp/wp-content/uploads/2012/09/unesco2012.pdf

Looking Ahead Assuming steady pricing Instability in per-bit storage ITS storage costs may not fall at same rate Long-term costs of cloud are not known Amazon costs have not decreased at HD rate1 Glacier charges for storage AND transfer Tape storage for preservation copies? Economies of scale Consortial solutions? 1 Rebecca Pool, “Is cloud storage the answer to preservation?” Research Information, 2/14/2013

Ramifications Preservation Appraisal and re-appraisal Some moving image material is not retained in uncompressed formats Appraisal and re-appraisal What are we keeping and for how long? How fast are we digitizing it? Costs of starting up

Storage Takeaways HD costs are no longer predictable Budgets often focused on digitization costs Ignoring significant and ongoing storage costs Storage costs are community wide problem Question of scale Best practice may not be possible Backups can cost more than primary storage Long-term archival storage ain’t cheap! http://www.researchinformation.info/news/news_story.php?news_id=1120

Thanks!!