SAS Overview The Big Picture

Slides:



Advertisements
Similar presentations
GLAST LAT ProjectDOE/NASA Review of the GLAST/LAT Project, Feb , 2001 IOC – S.Williams SAS - R.Dubois 1 Instrument Operations Center (IOC) and Science.
Advertisements

GLAST LAT Project I&T Meeting – Jan 20,2004 E. do Couto e Silva 1 GLAST Large Area Telescope: EM1 Data Analysis and Calibration Summary Report Eduardo.
GLAST LAT ProjectISOC CDR, 4 August 2004 Document: LAT-PR-04500Section 3.31 GLAST Large Area Telescope: Instrument Science Operations Center CDR Section.
SVAC/ISOCInstrument Analysis Workshop, February 27, 2006 Anders W. Borgland 1 Overview Of LAT Data Taking Anders W. Borgland Science Verification, Analysis.
GLAST LAT ProjectISOC Peer Review - March 2, 2004 Document: LAT-PR Section 2.1 Requirements 1 Gamma-ray Large Area Space Telescope GLAST Large.
GLAST LAT Project 1 Tracker “mini-tower” Full Size Calorimeter module Face to Face IDT meeting March 20, 2002 Convener: E. do Couto e Silva Engineering.
GLAST LAT Project ISOC Peer Review - March 2, 2004 Document: LAT-PR Section 4.3 Pipeline, Data Storage and Networking Issues1 Gamma-ray Large.
Simulation / Reconstruction Working group Toby Burnett University of Washington Jan 2000 T.
GLAST LAT Project July 19, 2005 E. do Couto e Silva 1/17 Science Verification Analysis and Calibration GLAST Large Area Telescope Eduardo do Couto e Silva.
GLAST LAT Project Analysis Meeting March 22, 2004 E. do Couto e Silva 1/8 LAT Instrument Test Analysis Eduardo do Couto e Silva March 23, 2004.
Section 15-1GLAST Ground System Design Review August 18&19, 2004 ISOC Organization ISOC Manager R Cameron Commanding, H&S Timeline Planning Command Generation.
GLAST LAT Project ISOC Peer Review - March 2, 2004 Section Gamma-ray Large Area Space Telescope GLAST Large Area Telescope ISOC Peer Review 7.2.
GLAST LAT Project IA Workshop 4 July 15, 2005 E. do Couto e Silva 1/7 Overview of Data Taking Anders W. Borgland Science Verification, Analysis and Calibrations.
GLAST LAT Project ISOC Peer Review - March 2, 2004 Document: LAT-PR Section 7.1 Pre-launch Operation and Calibration 1 Gamma-ray Large Area Space.
GLAST LAT ProjectNovember 18, 2004 I&T Two Tower IRR 1 GLAST Large Area Telescope: Integration and Test One and Two Tower Integration Readiness Review.
GLAST LAT Project 1S. Ritz Purposes of the Data Challenges “End-to-end” testing of analysis software. –define the ends –define the tests (what is success?)
GLAST LAT Project Analysis Meeting April 22, 2004 E. do Couto e Silva 1/5 SVAC Status SVAC Status Eduardo do Couto e Silva April 22, 2004.
R.Dubois Sim/Recon Overview 1/10 GLAST LAT ProjectSoftware Workshop July, SLAC Sim/Recon Overview Elements of Simulation/Reconstruction Development,
Fermi Large Area Telescope (LAT) Integration and Test (I&T) Data Experience and Lessons Learned LSST Camera Workshop Brookhaven, March 2012 Tony Johnson.
GLAST LAT ProjectDOE/NASA Baseline-Preliminary Design Review, January 8, 2002 K.Young 1 LAT Data Processing Facility Automatically process Level 0 data.
Root, GLAST, and IDL Heather Kelly NASA/GSFC Emergent Corporation.
GLAST CHEP 03 March R.Dubois1/17 GLAST Large Area Telescope: Overview of GLAST Offline Software Richard Dubois Stanford Linear Accelerator Center.
GLAST LAT ProjectI&T&C Pre PDR, October 2, 2001 E. do Couto e Silva1 I&T&C Organization Chart I&T&C Manager Elliott Bloom WBS I&T Engineer B. Grist.
GLAST LAT ProjectTracking Software Week – Pisa – 20 May 2002 R.Dubois1 SAS Overview Overall Data Flow Sim/Recon –Gleam –Recon rewrites –Calibrations Level.
T. Burnett1 GLAST LAT ProjectDOE/NASA Baseline-Preliminary Design Review, January 9, 2002 SAS Software: Sources Detector geometry model Simulation Event.
Heather Kelly PPA Scientific Computing Apps LAT was launched as part of the Fermi Gamma-ray Space Telescope on June 11 th 2008.
GLAST LAT Project LAT Instrument Analysis Workshop – Feb 27, 2006 Hiro Tajima, TKR Data Processing Overview 1 GLAST Large Area Telescope: TKR Data Processing.
1 Stepping in everyone’s toes ( but for a good cause….) Eduardo do Couto e Silva Software Meeting – January 2001.
Feb. 7, 2007First GLAST symposium1 Measuring the PSF and the energy resolution with the GLAST-LAT Calibration Unit Ph. Bruel on behalf of the beam test.
LM Feb SSD status and Plans for Year 5 Lilian Martin - SUBATECH STAR Collaboration Meeting BNL - February 2005.
GLAST LAT Offline SoftwareCore review, Jan. 17, 2001 Review of the “Core” software: Introduction Environment: THB, Thomas, Ian, Heather Geometry: Joanne.
GLAST LAT Project 1S. Ritz Data Challenge Planning Overview Purpose Boundary conditions Plan: –scope and progression of data challenges –schedule summary.
GLAST LAT ProjectI&T&C Pre PDR Presentation – Oct 2, 2001 B. Grist1 I&T&C Organization Chart I&T&C Manager Elliott Bloom WBS I&T Engineer B. Grist.
GLAST LAT ProjectSAS PDR, Aug. 17, 2001 R.Dubois1 Schedule Balloon Science Tools, Analysis Platforms Calibrations Performance Tuning, Tracking Data Production.
1 Getting Involved with GLAST Workshop, CFA, 06/21/ C. Shrader, NASA/GSFC GLAST Data & Software Release plans + Science Support Center Services C.
GLAST Large Area Telescope – ISOC Overview and StatusDOE Annual Review, 13 June GLAST Large Area Telescope Instrument Science Operations Center.
Overview of PHENIX Muon Tracker Data Analysis PHENIX Muon Tracker Muon Tracker Software Muon Tracker Database Muon Event Display Performance Muon Reconstruction.
Feb. 3, 2007IFC meeting1 Beam test report Ph. Bruel on behalf of the beam test working group Gamma-ray Large Area Space Telescope.
MAUS Status A. Dobbs CM43 29 th October Contents MAUS Overview Infrastructure Geometry and CDB Detector Updates CKOV EMR KL TOF Tracker Global Tracking.
LHCb Computing 2015 Q3 Report Stefan Roiser LHCC Referees Meeting 1 December 2015.
GLAST LAT ProjectNovember 18, 2004 I&T Two Tower IRR 1 GLAST Large Area Telescope: Integration and Test Two Tower Integration Readiness Review SVAC Elliott.
GLAST Large Area Telescope Instrument Science Operations Center
Managing the Project Lifecycle
I&T&C Organization Chart
GLAST Large Area Telescope:
Existing Perl/Oracle Pipeline
“Running Monte Carlo for the Fermi Telescope using the SLAC farm”
Overview of CLAS12 Calibration
GLAST Large Area Telescope
Integration, Test, and Calibration
GLAST Large Area Telescope:
Proposal for LAT Year 1 Data Release Plan
GLAST Large Area Telescope Instrument Science Operations Center
Science Analysis Software
GLAST Large Area Telescope
GLAST Large Area Telescope
Mark McKinnon EVLA Project Manager
Simulation and Physics
Integration and Test Organization Chart
LAT Instrument Operations Center
Introduction and Overview
Data Challenge 1 Closeout Lessons Learned Already
Status Report and highlights from workshop II
GLAST Large Area Telescope
Integration & Test Instrument Operations Coordination
GLAST Large Area Telescope Instrument Science Operations Center
GLAST Large Area Telescope:
Detector Proto-Technical board Sep 30, 2010
Overview of Beam Test Benoit, Ronaldo and Eduardo Nov 8 , 2005 thanks to Steve and Bill for helping to consolidate all the information.
Presentation transcript:

SAS Overview The Big Picture Reminder of the SAS mission, data flow, etc Simulation/Reconstruction Directions New in 2001 Who’s Doing What? Calibrations Planning Early Science Tools directions Short- and long-term schedules Management Stuff Worries

Our Mission shall perform prompt processing of Level 0 data through to Level 1 shall provide near real time monitoring information to the IOC. shall facilitate monitoring and updating instrument calibrations. shall maintain state and performance tracking. shall create high level science products from Level 1 for the PI team. shall perform reprocessing of instrument data. shall provide access to event and photon data for higher level analysis. shall perform bulk production of Monte Carlo simulations. shall interface with mirror PI team site(s) (sharing data and algorithms). shall interface with the SSC (sharing data and algorithms). shall support design of LAT instrument with simulations. Production event processing is performed in the Data Processing Facility.

Proposed Big Picture Draft 11/02/01 Mission Ops Center (MOC) Raw data from Malindi S/C, LAT and GBM Level 0 processing Spacecraft health and safety Instrument safety Commanding Alert data handling Ground station/TDRSS scheduling Acquisition data generation LAT Instrument Ops Center (IOC) Instrument monitoring Instrument ops planning Instrument calibration Instrument command load generation Instrument software generation Ground algorithms/software Instrument team product generation Level 1 processing Selected higher level processing Maintenance of calibration files and tools Transient detection Level 0 data Spacecraft data Loads Instrument Procedures Level 0 data Spacecraft data Science Support Center (SSC) High level data processing Science data distribution Data archiving Calibration archiving Software archiving Observation planning and scheduling Target of Opportunity selection Exposure maps Participation in LAT software generation Multi mission analysis tools Level 1 processing for GBM Backup Level 1 processing for LAT Loads Instrument Procedures High level products Level 1 data High level products Instrument Schedules Response Functions Processing Software Calibration Data Spacecraft data Level 1 data High level products GBM Instrument Ops Center Instrument monitoring Instrument ops planning Instrument calibration Etc, etc HEASARC Data and Software Archive

Data Flow Data recon + MC on disk. Abstract full-recon output into L1 DB for analysis DPF Italian mirror French mirror MC Recon MOC Calibs IOC L1 DB Fully automated server, with RDB for data catalogue + processing state. Uses SLAC batch CPU and disk farms. L2 DB SSC Parts of L2 processing also automated Section 7.8 SAS Overview

Instrument Simulations and Reconstruction 3 GeV gamma interaction Instrument data 3 GeV gamma recon CAL Detail

WWW Processing Pipeline Batch system Level 0 IOC HSM Level 0 Automated Tape Archive Level 1, diagnostics ~50 CPUs; ~50 TB disk by 2010 Section 7.8 SAS Overview

Sim/Recon Toolset applications Root, IDL – analysis TkrRecon, CalRecon, AcdRecon – test beam era versions; Rewrites being planned & executed gismo – simulation package GEANT4 – on its way xml – geometry, parameters Root – object I/O Gaudi – code framework VC++ – Windows IDE gnu tools - Linux vcmt – Windows gui CMT – package version management ssh – secure cvs access cvs – file version management utilities

2001 in a Nutshell New code framework – Gaudi Bulk of the software has been moved in Some useful features not moved yet eg Sawyer’s “time history” code tb_recon versions of TkrRecon and CalRecon ported and tweaked Geometries updated to match new baseline Sources updated All PDR studies run in this new environment GEANT4 just brought online – first version EM physics validation performed And, of course, PDR report, budgets, schedules, PMCS etc Using Root for object I/O system More descriptive and efficient format, suited to event data proto Recon tree & ntuples so far Code systems operational again on 2 OS – Windows + Linux Windows & Linux standard installs at UW & SLAC Data Manager prototype running Scripts produced simulation runs for PDR exercised SLAC batch farm Relational database is ready to use for tracking processing. Undergoing some tweaks Release Manager prototype could be turned on Automated code builds & limited testing Nightly runs – notify package owners of problems

SAS Organization

Institutional Responsibilities Management – SLAC Code Architect – U Washington Subsystems ACD – GSFC CAL – NRL, France TKR – SLAC, UCSC, Italy Infrastructure – GSFC, SLAC, UW GEANT4 – Italy Event Display – Italy, UW Sources – SLAC, UW, Japan DPF – SLAC, Stanford Science Tools – Stanford lead + collaboration + SSC

Who’s Doing What? Core Everything – Toby xml & geometry – Joanne detModel – Riccardo Sources - Sean Root stuff – Heather Data Manager, Release Manager – Karl CMT, librarian – soon to be Alex Calibrations – will be Joanne Event Display – led by Riccardo TKR Tracy, Leon, Bill Atwood Alignment - Hiro Digis – folks at Bari Vertexing – folks at Pisa CAL Sasha, Eric, Malcolm, Regis, Arache Calibrations – Sasha, Eric ACD Heather GEANT4 Validation – Alessandro, Francesco, Riccardo, Claudia, Tune To Gaudi – core + Monica Geometry - Riccardo Hits – Riccardo, Francesco BFEM Event Display – Nick, Gloria PDR Instrument Studies Steve, Bill, Tracy + core User Support Documentation Task Force – Heather “binaries” distributions – Alex Bug tracking - Karl

Doc & User Support Documentation Task Force Commissioned in Dec ’01 Group of 7. Heather Kelly (GSFC) chair. Charged with riding herd on all forms of doc Web, inline, Users and Developers manuals Defining procedures for maintenance “Binary” Code Distributions rpms and tarballs now available on Linux Winzip files on Windows Greatly reduce difficulty of install for non-experts Bug Tracking Currently just instituted simple majordomo mailing list Investigating use of Remedy for ‘real’ tracker. Will be a learning experience. http://www-glast.slac.stanford.edu/Software/core/documentation/

Data Structures Task Force (1) Commissioned in Dec ’01. Time is right, since TKR & CAL are rethinking their recons. Match to May/Oct ’02 major code releases. May require iteration About 10 members provide broad representation of subsystems, core and science. Leon Rochester (SLAC) chair. Charged with revisiting all transient/persistent store structures in sim + recon Content standards http://www-glast.slac.stanford.edu/Software/DataStructuresTF/

Data Structures Task Force (2) Content Add missing information Remove unneeded or duplicate information New classes Volume ID’s Event info (time, position, instrument status, etc.) Uniformity Coding rules File templates Member function names Private data names Monitor implementation Document design and implementation

Calibrations Planning Instrumental Calibrations ACD pedestals & gains CAL pedestals, gains, light tapers TKR hot/dead channel lists, alignments Schedule Drivers EM1 unit Aug ’02 Qualification Unit Nov ’03 High Level Calibrations Instrument Response Functions – resolution and efficiency parametrizations Used for astronomy Work in conjunction with Integration & Test group SAS writes algs, I&T runs them Test plans in prep for creating calibs for engineering units Test plans in prep for verification of MC against cosmics and beam tests. Current PSF, Aeff shown in Steve Ritz’s Day 1 talk Will repeat and refine this work annually

Calibrations: SVAC Data (being reviewed by subsystems) Science verification Low Level Calibration High Level Calibration ACD Detection Efficiency High Threshold detection TKR Single Hit Efficiency SSD Alignment Ladder Alignment Tray Alignment Tower Alignment Inter Tower Alignment LAT & Observatory Alignment CAL Light Attenuation Light Asymmetry Pedestals Noisy Strips Dead Strips Time-Over-Threshold Scintillation efficiency Energy range: Electronic Gain Energy range: Integral non linearity Energy range: Differential non linearity Number of reconstructed photons (Effective Area) Absolute Energy Energy Resolution Single Photon Angular Resolution Background Rejection (CAL+TKR) Monte Carlo tuning ( hit distributions, energy deposition, …) From I&T – E. do Couto e Silva SVAC = Science Verification and Calibration

Science Tools Progress At Feb review Already had list of tools and rough estimate of needed manpower ~40 MY effort estimated – to be drawn from the collaboration and SSC Seen by IPO as “Level of Effort” – after critical items are in hand. SSC did not exist Was awarded to Goddard during summer; starting to staff up Since then Negotiations with Goddard on LAT interface to SSC and deliverables Draft Project Data Management Plan Working on formalizing collaboration and internal science effort Working with SSC on requirements for Event Database used for astronomy Planning on a Science Tools workshop in Feb/March

Budget = Manpower Profile USA On project 1.25 FTE NRL 1 FTE GSFC, increasing to 2 in FY03 1 FTE Stanford, increasing to 2 in FY03 Off Project 7 FTE SLAC 1 FTE UW 1/2 FTE UCSC France 2 ½ FTE Italy 2 ½ FTE (and probably more, see TKR) Japan ½ FTE Profile shows dropoff in out years: no Science Tools work yet some lack of imagination about tasks that far out!

2002 Schedule Milestones submitted to IPO Prototype Code Release Manager - 3/02 ACD Calibration algorithm ready 4/02 Major Post-PDR Code Release (G4, flexible geometry) - 4/02 CAL Calibration algorithm ready 6/02 TKR Low level calibration alg ready 6/02 Calibration milestones are in response to I&T needs – EM1 module in 10/02 Major Code Release 5/02 First G4 ready by 1/02 Significant fraction of new TKR, CAL recons Next iteration on infrastructure Partially updated output structures Major Code Release 10/02 G4 stable New TKR, CAL recons done Fully updated output structures CDR in 8/02!

Major code releases

Management Stuff Responsibilities Scope Full WBS exists Critical areas (ie all but Science Tools) defined in Level 3 & 4 requirements. Calibrations defined in concert with I&T group SAS writes algs, I&T runs them Resource Loaded schedule ’02 and beyond loaded bottoms up. On- & off-project effort accounted for. (’01 loading is very approximate) Much use of “ongoing support” to indicate tweaking of “finished” projects Science Tools resources & schedule in as place holders from initial estimate Responsibilities All areas have clear line of responsibility Work packages defined to scope out details in combination with tasks Will be signed off by institutions, including non-US On-project folks report time per work package (started October) Off-project still to be worked out. IPO only requires reporting to top level WBS.

Manpower Projection Note: ’06 is only ½ year excludes Science Tools effort buildup

Worries Manpower Budget cuts in FY02 cost in User Support and DPF Stretching existing manpower (eg Documentation TF, and using students to help with DataManager) Situation in France is in flux Unknown effect on CAL and ripple on NRL Single code architect is a risk Toby Burnett is overloaded. Too much support work on top of design. We need another ‘architect class’ person on board to assist Toby Science Tools Collaboration not yet organized for this effort Negotiating roles with Science Center now Not ready to devote much manpower to it yet, but SSC raring to go!

Backup Slides

Manpower Estimates

Manpower – France Italy expect greater contribution to Science Tools later

Manpower - Italy excludes potential Science Tools effort continuing contribution to TKR not yet defined