LAT HSK Data Handling from B33 Cleanroom. ISOC Software Architecture.

Slides:



Advertisements
Similar presentations
Operating System.
Advertisements

COURSE: COMPUTER PLATFORMS
GLAST LAT Project ISOC Peer Review - March 2, 2004 Document: LAT-PR Section 6.1 Ground Operations Software 1 Gamma-ray Large Area Space Telescope.
GLAST LAT ProjectAll Hands Meeting – May 11, 2005 R. Claus 1 GLAST Large Area Telescope: LAT All Hands Meeting I&T/Online May 11, 2005 Ric Claus I&T/Online.
GLAST LAT ProjectOnline Peer Review – July 21, Integration and Test J. Panetta 1 Gamma-ray Large Area Space Telescope GLAST Large Area Telescope:
Multiple Tiers in Action
Section 15-1GLAST Ground System Design Review August 18&19, 2004 ISOC Organization ISOC Manager R Cameron Commanding, H&S Timeline Planning Command Generation.
GLAST LAT ProjectNovember 18, 2004 I&T Two Tower IRR 1 GLAST Large Area Telescope: Integration and Test One and Two Tower Integration Readiness Review.
Slide 1 of 9 Presenting 24x7 Scheduler The art of computer automation Press PageDown key or click to advance.
Common Services in a network Server : provide services Type of Services (= type of servers) –file servers –print servers –application servers –domain servers.
SERVER Betül ŞAHİN What is this? Betül ŞAHİN
Grid Computing Meets the Database Chris Smith Platform Computing Session #
A Brief Overview by Aditya Dutt March 18 th ’ Aditya Inc.
NETWORK CENTRIC COMPUTING (With included EMBEDDED SYSTEMS)
The Mass Storage System at JLAB - Today and Tomorrow Andy Kowalski.
Chapter 4 Software Hardware matters little compared to software?
The Evolution of Fleet Management How do you get device information now? Manually? Configuration pages?
Analysis of SQL injection prevention using a proxy server By: David Rowe Supervisor: Barry Irwin.
Operating System. Architecture of Computer System Hardware Operating System (OS) Programming Language (e.g. PASCAL) Application Programs (e.g. WORD, EXCEL)
Chapter 2 Hardware Configuration and Considerations Database Processing Chapter 2.
LSC Segment Database Duncan Brown Caltech LIGO-G Z.
GLAST LAT ProjectDOE/NASA Baseline-Preliminary Design Review, January 8, 2002 K.Young 1 LAT Data Processing Facility Automatically process Level 0 data.
Unit Seven Database 1.Passage One. Foundation of Database.
Central Reconstruction System on the RHIC Linux Farm in Brookhaven Laboratory HEPIX - BNL October 19, 2004 Tomasz Wlodek - BNL.
Dr. Mohamed Osman Hegazi 1 Database Systems Concepts Database Systems Concepts Course Outlines: Introduction to Databases and DBMS. Database System Concepts.
Client – Server Architecture. Client Server Architecture A network architecture in which each computer or process on the network is either a client or.
How computer’s are linked together.
March 6, 2009Tofigh Azemoon1 Real-time Data Access Monitoring in Distributed, Multi Petabyte Systems Tofigh Azemoon Jacek Becla Andrew Hanushevsky Massimiliano.
Contents 1.Introduction, architecture 2.Live demonstration 3.Extensibility.
Maintaining Large Vista Installations Amy Edwards, Ezra Freelove, & George Hernandez July 12, 2007.
Overview The GLAST Data Handling Pipeline (the Pipeline) provides a uniform interface to the diverse data handling needs of the GLAST collaboration. Its.
Maintaining Large Vista Installations Amy Edwards, Ezra Freelove, & George Hernandez July 12, 2007.
INTRODUCTION TO DATABASES CS 260 Database Systems.
A Guide to Oracle9i1 Database Instance startup and shutdown.
Database Design and Management CPTG /23/2015Chapter 12 of 38 Functions of a Database Store data Store data School: student records, class schedules,
Engineering on Display: Back-End Development for Sensor Instrumentation Systems Student: Brian J Kapala Supervisor: Dr. Cavalcanti.
A Brief Documentation.  Provides basic information about connection, server, and client.
MANAGING DATA RESOURCES ~ pertemuan 7 ~ Oleh: Ir. Abdul Hayat, MTI.
Wellcome Trust Sanger Institute Informatics Systems Group Ensembl Compute Grid issues James Cuff Informatics Systems Group Wellcome Trust Sanger Institute.
GLAST Science Support CenterJuly, 2003 LAT Ground Software Workshop Status of the D1 (Event) and D2 (Spacecraft Data) Database Prototypes for DC1 Robert.
Course FAQ’s I do not have any knowledge on SQL concepts or Database Testing. Will this course helps me to get through all the concepts? What kind of.
LIMS WAREHOUSING Archiving and Migrating of LIMS Data Examples: Annual Product Review and Trending with Beckmann LIMS data Query for historical specifications.
Experiment Management System CSE 423 Aaron Kloc Jordan Harstad Robert Sorensen Robert Trevino Nicolas Tjioe Status Report Presentation Industry Mentor:
GLAST LAT Project LAT Instrument Analysis Workshop – Feb 27, 2006 Hiro Tajima, TKR Data Processing Overview 1 GLAST Large Area Telescope: TKR Data Processing.
Printing at CERN Rafal Otto, Juraj Sucik, CERN IT Department CHEP 2007 International Conference on Computing.
Eurostat November 2015 Eurostat Unit B3 – IT and standards for data and metadata exchange SDMX IT Tools Test Client Jean-Francois LEBLANC Christian SEBASTIAN.
Client – Server Architecture A Basic Introduction 1.
Cloud Computing ILAS project DONE BY:. Table of content INTRODUCTION. ◦ Cloud computing in general ◦ What are the things that worked during the implementation.
Backup Tables in SQL Server. Backup table method Cape_Codd database is used in this example 1.Righ click the database that contains the table you want.
Introduction to Utilities for New DBAs Session #332 4/19/2008 Erik Hobbs Introduction to Utilities for New DBAs.
AZ PASS User Group Azure Data Factory Overview Josh Sivey, Solution Partner October
 Computer hardware refers to the physical parts of a computer and related devices. Internal hardware devices include motherboards, hard drives,
OOS SQUARE SQUARE Lab Deployment Diagram *UML 2 and the Unified Process Second Edition 을 인용하여 작성됨.
SERIALIZED DATA STORAGE Within a Database James Devens (devensj)
Managing Data Resources File Organization and databases for business information systems.
GLAST LAT ProjectNovember 18, 2004 I&T Two Tower IRR 1 GLAST Large Area Telescope: Integration and Test Two Tower Integration Readiness Review SVAC Elliott.
Database backed DNS.
Sqoop Mr. Sriram
Existing Perl/Oracle Pipeline
TYPES OF SERVER. TYPES OF SERVER What is a server.
Cloud based Open Source Backup/Restore Tool
Database Management  .
MANAGING DATA RESOURCES
CS101 Bigger Networks.
Tiers vs. Layers.
GLAST Large Area Telescope
GLAST Large Area Telescope
GLAST Large Area Telescope: I&T Test Readiness Review
Level 1 Processing Pipeline
Final Project Geog 375 Daniel Hewitt.
Presentation transcript:

LAT HSK Data Handling from B33 Cleanroom

ISOC Software Architecture

Import Flow Diagram

Backup Slides

HSK Data Collection Flight hardware in the cleanroom is polled by the LATTE HSK Server. Sampled HSK values are stored in a MySQL database Every two hours, new data is dumped to binary files on the DMZ drive. –This shared volume is mounted by LATTE workstations in the cleanroom, and by a small number of designated hosts on the SLAC network.

HSK Data Export A cron job on the ‘glast02’ host moves the database-dump files from the DMZ volume to the U12 volume –This volume is accessible to (most) GLAST- project hosts and to the LSF batch farm The cron job also injects a new instance of the ‘b33-hsk-import’ task into the pipeline.

HSK Data Import The pipeline schedules execution of the Oracle-import program (written in Python) on the LSF batch farm. –Processing status can be tracked through the pipeline web pages. The Python program reads the binary data file, creates change records and statistics records, and posts them to Oracle.

Web Data Access The test-data processing pipeline populates the ‘ELogReport’ database table with information about each test run. –This information can be queried to provide in- test time-spans of interest for querying trending data. –It can also be used to determine the serial numbers of hardware under test (with some limitations).