San Diego Supercomputer Center

Slides:



Advertisements
Similar presentations
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Data Grids for Collection Federation Reagan W. Moore University.
Advertisements

OGF-23 iRODS Metadata Grid File System Reagan Moore San Diego Supercomputer Center.
HPSS The High Performance Storage System Developed by IBM, LANL, LLNL, ORNL, SNL, NASA Langley, NASA Lewis, Cornell, MHPCC, SDSC, UW with funding from.
NATIONAL PARTNERSHIP FOR ADVANCED COMPUTATIONAL INFRASTRUCTURE SAN DIEGO SUPERCOMPUTER CENTER Particle Physics Data Grid PPDG Data Handling System Reagan.
San Diego Supercomputer CenterNational Partnership for Advanced Computational Infrastructure1 Grid Based Solutions for Distributed Data Management Reagan.
Distributed IT Infrastructure for U.S. ATLAS Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.
SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO IEEE Symposium of Massive Storage Systems, May 3-5, 2010 Data-Intensive Solutions.
Applying Data Grids to Support Distributed Data Management Storage Resource Broker Reagan W. Moore Ian Fisk Bing Zhu University of California, San Diego.
SUMS Storage Requirement 250 TB fixed disk cache 130 TB annual increment for permanently on- line data 100 TB work area (not controlled by SUMS) 2 PB near-line.
National Partnership for Advanced Computational Infrastructure Data Intensive Computing Information Based Computing Digital Libraries / Metacomputing Services.
National Partnership for Advanced Computational Infrastructure Advanced Architectures CSE 190 Reagan W. Moore San Diego Supercomputer Center
What is it? Hierarchical storage software developed in collaboration with five US department of Energy Labs since 1992 Allows storage management of 100s.
1 Andrew Hanushevsky - HEPiX, October 6-8, 1999 Mass Storage For BaBar at SLAC Andrew Hanushevsky Stanford.
PetaByte Storage Facility at RHIC Razvan Popescu - Brookhaven National Laboratory.
Mass RHIC Computing Facility Razvan Popescu - Brookhaven National Laboratory.
UNIVERSITY of MARYLAND GLOBAL LAND COVER FACILITY High Performance Computing in Support of Geospatial Information Discovery and Mining Joseph JaJa Institute.
Distinguish between primary and secondary storage.
Enterprise Storage Our Journey Thus Far John D. Halamka MD CIO, Harvard Medical School and Beth Israel Deaconess Medical Center.
Hard Drives Non-Volatile Storage. Hard Drives Hard Drives (HD) The primary storage device in a computer system.
Data oriented job submission scheme for the PHENIX user analysis in CCJ Tomoaki Nakamura, Hideto En’yo, Takashi Ichihara, Yasushi Watanabe and Satoshi.
The Mass Storage System at JLAB - Today and Tomorrow Andy Kowalski.
National Partnership for Advanced Computational Infrastructure Digital Library Architecture Reagan Moore Chaitan Baru Amarnath Gupta George Kremenek Bertram.
San Diego Supercomputer CenterUniversity of California, San Diego Preservation Research Roadmap Reagan W. Moore San Diego Supercomputer Center
CS 111 – Aug – 1.3 –Information arranged in memory –Types of memory –Disk properties Commitment for next day: –Read pp , In other.
Production Data Grids SRB - iRODS Storage Resource Broker Reagan W. Moore
Overview of Physical Storage Media
Computer Guts and Operating Systems CSCI 101 Week Two.
National Partnership for Advanced Computational Infrastructure San Diego Supercomputer Center Persistent Management of Distributed Data Reagan W. Moore.
HPSS for Archival Storage Tom Sherwin Storage Group Leader, SDSC
Price Performance Metrics CS3353. CPU Price Performance Ratio Given – Average of 6 clock cycles per instruction – Clock rating for the cpu – Number of.
Policy Based Data Management Data-Intensive Computing Distributed Collections Grid-Enabled Storage iRODS Reagan W. Moore 1.
O AK R IDGE N ATIONAL L ABORATORY U.S. D EPARTMENT OF E NERGY Facilities and How They Are Used ORNL/Probe Randy Burris Dan Million – facility administrator.
O AK R IDGE N ATIONAL L ABORATORY U.S. D EPARTMENT OF E NERGY HPSS Features and Futures Presentation to SCICOMP4 Randy Burris ORNL’s Storage Systems Manager.
8 October 1999 BaBar Storage at CCIN2P3 p. 1 Rolf Rumler BaBar Storage at Lyon HEPIX and Mass Storage SLAC, California, U.S.A. 8 October 1999 Rolf Rumler,
CSCI-100 Introduction to Computing Hardware Part I.
ATLAS Tier 1 at BNL Overview Bruce G. Gibbard Grid Deployment Board BNL 5-6 September 2006.
PC clusters in KEK A.Manabe KEK(Japan). 22 May '01LSCC WS '012 PC clusters in KEK s Belle (in KEKB) PC clusters s Neutron Shielding Simulation cluster.
Allow computers to store programs and information for use at a later date Storage Devices.
Computers - The Journey Inside continues…
Status SC3 SARA/Nikhef 20 juli Status & results SC3 throughput phase SARA/Nikhef Mark van de Sanden.
Computational Research in the Battelle Center for Mathmatical medicine.
Computer Organization. The Five Hardware Units General purpose computers use the "Von Neumann" architecture Also referred to as "stored program" architecture.
January 30, 2016 RHIC/USATLAS Computing Facility Overview Dantong Yu Brookhaven National Lab.
Section 13.1 – Secondary storage management (Former Student’s Note)
S AN D IEGO S UPERCOMPUTER C ENTER N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE 6 TB SSA Disk StorageTek Tape Libraries 830 GB MaxStrat.
National Archives and Records Administration1 Integrated Rules Ordered Data System (“IRODS”) Technology Research: Digital Preservation Technology in a.
US ATLAS Tier 1 Facility Rich Baker Deputy Director US ATLAS Computing Facilities October 26, 2000.
SAN DIEGO SUPERCOMPUTER CENTER Replication Policies for Federated Digital Repositories Robert H. McDonald Chronopolis Project Manager
RCF Status Smooth operation over the course of last week –No reportable outages of the Mass Storage System (HPSS), the Tape Libraries and the Network PHENIX.
Collection-Based Persistent Archives Arcot Rajasekar, Richard Marciano, Reagan Moore San Diego Supercomputer Center Presented by: Preetham A Gowda.
NASA Langley Research Center’s Distributed Mass Storage System (DMSS) Juliet Z. Pao Guest Lecturing at ODU April 8, 1999.
11 October 2000Iain A Bertram - Lancaster University1 Lancaster Computing Facility zStatus yVendor for Facility Chosen: Workstations UK yPurchase Contract.
ORNL is managed by UT-Battelle for the US Department of Energy OLCF HPSS Performance Then and Now Jason Hill HPC Operations Storage Team Lead
12/19/01MODIS Science Team Meeting1 MODAPS Status and Plans Edward Masuoka, Code 922 MODIS Science Data Support Team NASA’s Goddard Space Flight Center.
© Thomas Ludwig Prof. Dr. Thomas Ludwig German Climate Computing Center (DKRZ) University of Hamburg, Department for Computer Science (UHH/FBI) Disks,
High Performance Storage System (HPSS) Jason Hick Mass Storage Group HEPiX October 26-30, 2009.
Local secondary storage (local disks)
הכרת המחשב האישי PC - Personal Computer
Unit 3—Part A Computer Memory
Distinguish between primary and secondary storage.
JDAT Production Hardware
How do computers work? Storage.
NERSC Reliability Data
Main memory and mass storage
Unit 3—Part A Computer Memory
LTO Cartridge native (uncompressed) capacity, Interchangeability and Maximum tape drive native data transfer speeds Tape Drive Operation Generation 1 cartridge.
Bits, Bytes, and Storage.
Arcot Rajasekar Michael Wan Reagan Moore (sekar, mwan,
TeraScale Supernova Initiative
Section 13.1 – Secondary storage management (Former Student’s Note)
Presentation transcript:

San Diego Supercomputer Center Data Intensive Computing Information Based Computing Digital Libraries / Metacomputing Services Reagan W. Moore San Diego Supercomputer Center moore@sdsc.edu http://www.npaci.edu/DICE

Information Based Computing Data Mining Distributed Archives Application Collection Building Information Discovery Digital Library

Characterizing Supercomputers Generators of data - numerically intensive computing Usage models for the rate at which supercomputers move data between memory, disk, and archives Usage models for capacity of the data caches (memory size, local disk, and archival storage) Analyzers of data - data intensive computing Performance models for combining data analysis with data movement (between caches, disks, archives)

Supercomputer Data Flow Model CPU Memory Local Disk Archive Disk Archive tape

HPSS Archival Storage System 108 GB SSA RAID High Performance Gateway Node High Node Disk Mover HiPPI driver Wide Node 54 GB Silver Node Storage / Purge Bitfile / Migration Nameservice/PVL Log Daemon Tape / disk mover DCE / FTP /HIS Log Client 160 GB 830 GB MaxStrat RAID 9490 Robot Four Drives 3490 Tape RS6000 Tape Mover PVR (9490) HiPPI Switch Trail- Blazer3 Switch Magstar 3590 Tape 3494 Robot Eight Tape Drives Seven

Archive Data Flow Model TeraFlops System 5-20 TB Compute Engine Local Disk 3-10 GB/sec 1 day cache 0.5-1 TB memory 30-100 MB/sec Archive Tape Archive Disk 1 week cache 20-60 MB/sec 0.5-1 PB 5-20 TB

Data Generation Metrics 7 Bytes/Flop CPU Memory 1 Byte of storage per Flop 1 Byte/60 Flops 1/7 of data persists for a day Local Disk Hold data for 1 day 1/7 of data sent to archive Archive Disk Hold data for 1 week All data sent to archive Archive tape Hold data forever

Feedback: Privacy Policy Feedback
Do Not Sell My Personal Information
About project: SlidePlayer Terms of Service

© 2025 SlidePlayer.com. Inc. All rights reserved.