RCF Status Smooth operation over the course of last week –No reportable outages of the Mass Storage System (HPSS), the Tape Libraries and the Network PHENIX.

Slides:

Advertisements

Similar presentations

Computer-System Structures Er.Harsimran Singh

Advertisements

Storing Data: Disk Organization and I/O

Storing Data: Disks and Files: Chapter 9

Silberschatz, Galvin and Gagne  Operating System Concepts Chapter 2: Computer-System Structures Computer System Operation I/O Structure Storage.

CS4432: Database Systems II Data Storage - Lecture 2 (Sections 13.1 – 13.3) Elke A. Rundensteiner.

04/18/2007CSCI 315 Operating Systems Design1 Mass Storage Structure Notice: The slides for this lecture have been largely based on those accompanying the.

1 Storing Data: Disks and Files Yanlei Diao UMass Amherst Feb 15, 2007 Slides Courtesy of R. Ramakrishnan and J. Gehrke.

Silberschatz, Galvin and Gagne  Operating System Concepts Chapter 2: Computer-System Structures Computer System Operation I/O Structure Storage.

Manajemen Basis Data Pertemuan 2 Matakuliah: M0264/Manajemen Basis Data Tahun: 2008.

1 Storage Hierarchy Cache Main Memory Virtual Memory File System Tertiary Storage Programs DBMS Capacity & Cost Secondary Storage.

CS4432: Database Systems II Lecture 2 Timothy Sutherland.

Computer-System Structures

What is it? Hierarchical storage software developed in collaboration with five US department of Energy Labs since 1992 Allows storage management of 100s.

Introduction to Database Systems 1 The Storage Hierarchy and Magnetic Disks Storage Technology: Topic 1.

Hard Disk Assemblies. Inside a Hard Disk How a Hard Disk Works.

Mass RHIC Computing Facility Razvan Popescu - Brookhaven National Laboratory.

Layers of a DBMS Query optimization Execution engine Files and access methods Buffer management Disk space management Query Processor Query execution plan.

General System Architecture and I/O.  I/O devices and the CPU can execute concurrently.  Each device controller is in charge of a particular device.

DISKS IS421. DISK  A disk consists of Read/write head, and arm  A platter is divided into Tracks and sector  The R/W heads can R/W at the same time.

Data oriented job submission scheme for the PHENIX user analysis in CCJ Tomoaki Nakamura, Hideto En’yo, Takashi Ichihara, Yasushi Watanabe and Satoshi.

The Mass Storage System at JLAB - Today and Tomorrow Andy Kowalski.

Lecture 11: DMBS Internals

Physical Storage Susan B. Davidson University of Pennsylvania CIS330 – Database Management Systems November 20, 2007.

Introduction to Database Systems 1 Storing Data: Disks and Files Chapter 3 “Yea, from the table of my memory I’ll wipe away all trivial fond records.”

Input and Output Computer Organization and Assembly Language: Module 9.

2.1 Silberschatz, Galvin and Gagne ©2003 Operating System Concepts with Java Chapter 2: Computer-System Structures Computer System Operation I/O Structure.

CHAPTER 2: COMPUTER-SYSTEM STRUCTURES Computer system operation Computer system operation I/O structure I/O structure Storage structure Storage structure.

Chapter 2: Computer-System Structures

Thanks to Silberschatz, Galvin and Gagne  Operating System Concepts Chapter 2 Computer-System Structures n Computer System Operation n I/O Structure.

An Overview of PHENIX Computing Ju Hwan Kang (Yonsei Univ.) and Jysoo Lee (KISTI) International HEP DataGrid Workshop November 8 ~ 9, 2002 Kyungpook National.

ICS 321 Fall 2011 Overview of Storage & Indexing (i) Asst. Prof. Lipyeow Lim Information & Computer Science Department University of Hawaii at Manoa 11/9/20111Lipyeow.

Database Management Systems,Shri Prasad Sawant. 1 Storing Data: Disks and Files Unit 1 Mr.Prasad Sawant.

PHENIX and the data grid >400 collaborators Active on 3 continents + Brazil 100’s of TB of data per year Complex data with multiple disparate physics goals.

HPSS for Archival Storage Tom Sherwin Storage Group Leader, SDSC

RCF Status One issues with the Mass Storage System (HPSS) –On 1/1 at 2:40 PM the core server process stopped working –Was automatically restarted by heartbeat.

11.1Database System Concepts. 11.2Database System Concepts Now Something Different 1st part of the course: Application Oriented 2nd part of the course:

DMBS Internals I. What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently.

1 CS.217 Operating System By Ajarn..Sutapart Sappajak,METC,MSIT Chapter 2 Computer-System Structures Slide 1 Chapter 2 Computer-System Structures.

1 The PHENIX Experiment in the RHIC Run 7 Martin L. Purschke, Brookhaven National Laboratory for the PHENIX Collaboration RHIC from space Long Island,

High Energy FermiLab Two physics detectors (5 stories tall each) to understand smallest scale of matter Each experiment has ~500 people doing.

RCF Status - Introduction PHENIX and STAR Counting Houses are connected to RCF at a Network Bandwidth of 20 Gbits/sec each –Redundant (Bandwidth-wise and.

RCF Status Extended outage of the Mass Storage System (HPSS) last Wednesday –Latest transaction logs of namespace DB were erroneously deleted in the production.

Chapter 2: Computer-System Structures(Hardware) or Architecture or Organization Computer System Operation I/O Structure Storage Structure Storage Hierarchy.

DMBS Internals I. What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently.

Section 13.2 – Secondary storage management. Index 13.2 Disks Mechanics of Disks The Disk Controller Disk Access Characteristics.

ECE 456 Computer Architecture Lecture #9 – Input/Output Instructor: Dr. Honggang Wang Fall 2013.

What Should a DBMS Do? Store large amounts of data Process queries efficiently Allow multiple users to access the database concurrently and safely. Provide.

1 Lecture 16: Data Storage Wednesday, November 6, 2006.

Data Storage and Querying in Various Storage Devices.

Time Meeting May 14, 2013 Agenda Safety Schedule Machine and system reports 05/14/2013Time Meeting, Run 131.

File organization Secondary Storage Devices Lec#7 Presenter: Dr Emad Nabil.

Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke1 Disks and Files.

Chapter 2: Computer-System Structures(Hardware)

Chapter 2: Computer-System Structures

Storage and Disks.

Database Management Systems (CS 564)

Silberschatz, Galvin and Gagne  Operating System Concepts Chapter 2: Computer-System Structures Computer System Operation I/O Structure Storage.

Lecture 11: DMBS Internals

Lecture 9: Data Storage and IO Models

San Diego Supercomputer Center

CS 140 Lecture Notes: Technology and Operating Systems

Computer-System Architecture

Module 2: Computer-System Structures

Module 2: Computer-System Structures

Chapter 2: Computer-System Structures

Chapter 2: Computer-System Structures

Module 2: Computer-System Structures

Module 2: Computer-System Structures

Sarah Diesburg Operating Systems CS 3430

Expanding the PHENIX Reconstruction Universe

Presentation transcript:

RCF Status Smooth operation over the course of last week –No reportable outages of the Mass Storage System (HPSS), the Tape Libraries and the Network PHENIX & RCF experts investigating rate limitation (cap at ~2.4 Gbps) observed between Counting House and RCF –Issue understood and solved (see next 3 slides) –Data received from PHENIX & STAR Counting Houses was safely migrated to Tape 30 min interference Sun/Mon night when other applications flooded Mass Storage System w/ requests –Looking at ways to implement request prioritization

2 Gbps Steady-state rate out of buffer boxes Iperf test 5 Gbps PHENIX CH => RCF rate limit (from last Time Meeting) Back-to-back stores Causing long transfer tails Initial tests indicate that there is no limit imposed by network & HPSS data movers

Magnetic Disk Basics Arm Assembly Arm Head CylinderSpindle Platter Track Read-ahead size is a critical parameter to achieve line-rate performance In particular with sequential I/O Caching in (memory) buffer Data

PHENIX Buffer Boxes (BB) sending Data to RCF Mass Storage System 7 th additional BB does not increase the transfer rate Source: Martin Purschke 2 Gbps Last Week 5 Gbps

3150 TB 1300 TB 1250 TB 3100 TB Raw Data Volume collected & archived 02/01/ TB 240 TB Physics PHENIX Raw Data STAR Raw Data

STAR Counting House  RCF Network Hourly Data Migration Rate To Tape (GB/hour) 4 Gbps 2000 GB/h

PHENIX Counting House  RCF Hourly Data Migration Rate To Tape (GB/hour) 2 TB/h 5 Gbps 2000 GB/h