Australian Synchrotron Data curatorship for protein crystallography Julian Adams & Richard Farnsworth.

Slides:

Advertisements

Similar presentations

Computing Infrastructure

Advertisements

Secondary Storage Input Output Processing Internal Memory.

Storage Media. Objectives You will be able to: –Mention the most common types of storage media Magnetic Storage Optical.

Storage Devices.

Lesson 3: Working with Storage Systems

OCLC Digital Archive Overview Judith Cobb LIPA Meeting July 2006.

New SIF computer system and TLC digitisation Dan Whiter ASK Workshop June 2008, Stockholm.

Dawei Lin, Ph.D. Director, Bioinformatics Core UC Davis Genome Center July 20, 2008, SLIMS (Solexa sequencing.

SUMS Storage Requirement 250 TB fixed disk cache 130 TB annual increment for permanently on- line data 100 TB work area (not controlled by SUMS) 2 PB near-line.

1 Data Storage MICE DAQ Workshop 10 th February 2006 Malcolm Ellis & Paul Kyberd.

Data Storage Technology

Back Up and Recovery Sue Kayton February 2013.

Servers Redundant Array of Inexpensive Disks (RAID) –A group of hard disks is called a disk array FIGURE Server with redundant NICs.

3.1 Storage. Overview Compare the following storage devices in terms of storage capacity, cost, speed of data retrieval and suitability for specific purposes:

Using and Configuring Storage Devices Guide to Operating Systems Third Edition.

PORTABLE STORAGE DEVICES David Garcia i Alex Tiana.

BACKUP/MASTER: Immediate Relief with Disk Backup Presented by W. Curtis Preston VP, Service Development GlassHouse Technologies, Inc.

Distinguish between primary and secondary storage.

Costing. Life 2 Model Research (JISC Project) Application to whole operation of Library Attempts to cost common factors Does not take into account certain.

Comp 1001: IT & Architecture - Joe Carthy 1 Information Representation: Summary All Information is stored and transmitted in digital form in a computer.

Copyright © 2006 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Technology Education Copyright © 2006 by The McGraw-Hill Companies,

IMT, Inc. Proprietary & Confidential Storage The Year In Review 2008 And looking ahead for 2009 Rob Kobrin Chief Technology Officer Integrated Media Technologies,

Secondary Storage Chapter 7.

Excellence Publication Co. Ltd. Volume Volume 1.

RDX: REMOVABLE HARD DISK STORAGE SYSTEM December 2010.

INFO1 – Practical problem solving in the digital world

Computer Organisation 1 Secondary Storage Sébastien Piccand

Data Storage What is storage? Storage refers to the media and methods used to keep information available for later use. Some things will be needed right.

Your Interactive Guide to the Digital World Discovering Computers 2012.

Advanced Diploma 1 Backing Storage. Advanced Diploma 2 Aims Understand how data is stored Be able to use the binary system to represent ASCII characters.

Lecture No 11 Storage Devices

Backup Library Comparison

8 SECONDARY STORAGE CHAPTER

McLean HIGHER COMPUTER NETWORKING Lesson 15 (a) Disaster Avoidance Description of disaster avoidance: use of anti-virus software use of fault tolerance.

The Big ASC Working Lunch Agenda 1. Status: Where are we? 2. Planning: New timeline 3. RAs: Recruitment and Training 4. Schedule for recording: cities,

1618Tb SDDS storage BE-CO-IN. Proposal Upgrade operational LHC_DATA to 1.1 Tbytes with the same operational level : – New faster technology (SAS disks.

TATII ITS Network (Fiber ) Portal Server Fourth Avenue Building Database Server Dual Sparc SAN (RAID) 1.2 TB Direct Connection backup_tables raw_data_files.

Cosc 4750 Backups Why Backup? In case of failure In case of loss of files –User and system files Because you will regret it, if you don’t. –DUMB = Disasters.

Storage Devices Primary storage: is the storage provided by memory in a computer system Secondary storage: is storage provided by peripheral devices other.

PC clusters in KEK A.Manabe KEK(Japan). 22 May '01LSCC WS '012 PC clusters in KEK s Belle (in KEKB) PC clusters s Neutron Shielding Simulation cluster.

Storage Devices 1. Objectives Overview Differentiate between storage devices and storage media Describe the characteristics of an internal hard disk including.

Australian Synchrotron Mature GUI’s for protein crystallography beamlines Julian Adams.

Macromolecular Crystallography Workshop 2004 Recent developments regarding our Computer Environment, Remote Access and Backup Options.

September 23, 2010Suzanne Baker Data Storage Internal & External.

3 Storage devices and media Backing store devices.

Install, configure and test ICT Networks

Measuring and Improving Drive Performance

Secondary Storage. WHAT IS SECONDARY STORAGE  SECONDARY STORAGE IS THE STORAGE THAT IS NON- VOLATILE. RAM IS VOLATILE AND SHORT TERM AND FORGETS EVERYTHING.

01. December 2004Bernd Panzer-Steindel, CERN/IT1 Tape Storage Issues Bernd Panzer-Steindel LCG Fabric Area Manager CERN/IT.

ALICE Computing Model A pictorial guide. ALICE Computing Model External T1 CERN T0 During pp run i (7 months): P2: data taking T0: first reconstruction.

HNC COMPUTING - COMPUTER PLATFORMS 1 Computer Platforms Week 2 Backing Storage.

Basic Guide to Computer Backups Eric Moore Computer Users Group of Greeley September 13, 2008.

Basic Guide to Computer Backups

Techno Diversity in Storage Architecture Strategies Steve Tongish

Integrating Disk into Backup for Faster Restores

Part B Computer Storage

Processing Device and Storage Devices

Information Technology

Choosing the best storage method

CHAPTER 4 Memory System Organization and Architecture

Chapter Seven Types of Storage.

Distinguish between primary and secondary storage.

Hard Disk Drive Property Notes Insert Image Type of Storage

Chapter 3 – Storage Devices and Media

Overview of Mass Storage Structure

Back Up and Recovery Sue Kayton October 2015.

Tomography at Advanced Photon Source

Computer Storage.

Presentation transcript:

Australian Synchrotron Data curatorship for protein crystallography Julian Adams & Richard Farnsworth

Australian Synchrotron Current Technologies Disks SCSI IDE RAID Tapes LTO DLT Robotic tape changers Optical Media DVD DVD2

Australian Synchrotron Data quantities (worst case) 1 image every 3-5 seconds 20 images per minute 1200 images per hour images per day Each image 15 – 30 MB 20x60x24x20Mb = 576 Mb = 0.6 TB per day 3 x 12weeks x 7days x.6 TB = 250 TB per year

Australian Synchrotron Data management Two copies in as short as practical time Create a take-home copy at the end of the run for the user Retain a single copy onsite for 3 months Deposit data in a long term storage facility  This data should only be accessible by requests sent back to the beam line.

Australian Synchrotron Practicality Data collection buffer with 1.5 TB disk Server with TB raid array Users using a portable hard drives (Firewire, USB2), or optical media Long term storage, 5 years = 1.3 PB per PX beamline

Australian Synchrotron Data collection strategy

Australian Synchrotron Data rates LTO technology can stream at up to 80 MB/s given optimum conditions LTO2 typically gets in the region 8-15 MB/s Worst case sufficient for our purposes (just) Off sites limitations are: Gb/s Links to VERN/GRANGENET/AARNET. Cost prohibitive for huge data flow at the moment