What is it? Hierarchical storage software developed in collaboration with five US department of Energy Labs since 1992 Allows storage management of 100s of billions of files spanning 100s of petabytes for the HPC community. Licensed and supported by IBM

Why? Reduced cost Scalability Power usage Reliability Speed Long term storage

How? Distributed cluster architecture Metadata engine IBM DB2 Multiple storage classes Striped disks and tapes

Who Uses it? NCSA BlueWaters Argonne National Lab Indiana State University

Disk and Tape Hierarchical storage management (HSM) Frequently used data cache on disk Archival data on tape Automatic migration (Mirror offsite) Scalable, any instance of HPSS can access many tapes at the same time to provide parallel transfer rates. Pros: Lower cost No power usage Reliable Cons: High latency Pros: Low Latency Cons: Power usage Reliability Higher Cost

Standard POSIX interface Users can access files using several methods: FTP – standard FTP from mover PFTP – Parallel transfer of data from multiple movers Client API HSI – transfer files put/get files from HPSS HTAR – archive multiple files together and transfer to HPSS VFS Client XFS

Components Core Server Translation Human Readable Name -> HPSS Object Identifiers Translates virtual volumes into physical volumes Allows parallel I/O to the resources Schedules mounting/dismounting of media Migration/Purge Server Manages migration purge policies Disk Migration Purge Once files have been moved down the hierarchy they are purged from disk

Components Tape File Migration Make additional copies to multi-site setup Tape Volume Migration Move data between tapes to optimally fill up tapes Gatekeeper (GK) Account validation service Site authorization etc… Location Server (LS) Allows client to determine which location they should contact Improves speed in multi-site setups Physical Volume Library (PVL) Manages all HPSS physical volumes Mounting and dismounting ( => PVR) Atomic mounts for sets of cartridges for parallel access to data

Components Physical Volume Repository (PVR) Interface to request cartridge mounts and dismounts One to one with tape libraries Movers Servers Handles actual data transfers Communicates with Core Server to figure out source and destination Retries moves on failures

Components

Scalability Horizontally scales: Add more movers Add more tape drives

BlueWaters Software “RAIT” is being developed jointly by IBM and NCSA Add 8+2 reliability to HPSS striping 40 GbE network 100,000 tape cartridges 38.5 TB per hour

Indiana University Multi-site setup Centralized archival storage for all campus clusters