Data Processing and User Software Ken Ebisawa (Astro-E2 GOF) presentation and demonstration.

Slides:



Advertisements
Similar presentations
Testing Relational Database
Advertisements

An innovative tool for the review of health and safety work practices and the implementation of effective controls of particulate exposures.
Configuration management
10 July Yohkoh Galileo: To Infinity! … and Beyond! Lockheed Martin’s Role in the Yohkoh Galileo Project 10 July 2002.
Dale E. Gary Professor, Physics, Center for Solar-Terrestrial Research New Jersey Institute of Technology 1 3/16/2012OVSA Preliminary Design Review.
Summary Role of Software (1 slide) ARCS Software Architecture (4 slides) SNS -- Caltech Interactions (3 slides)
23/04/2008VLVnT08, Toulon, FR, April 2008, M. Stavrianakou, NESTOR-NOA 1 First thoughts for KM3Net on-shore data storage and distribution Facilities VLV.
SOFIA DCS History and Overview Ian Gatley. SOFIA March DCS Preliminary Design Review2 The South Pole CARA Project: A DCS demonstration A data.
GLAST LAT ProjectISOC CDR, 4 August 2004 Document: LAT-PR-04500Section 3.11 GLAST Large Area Telescope: Instrument Science Operations Center CDR Section.
Mission Operations and Data Flow Overview Koji Mukai Astro-E2 Guest Observer Facility.
Lecture Nine Database Planning, Design, and Administration
November 2011 At A Glance GREAT is a flexible & highly portable set of mission operations analysis tools that increases the operational value of ground.
Shu Zhang (on behalf of the HXMT team) Institute of High Energy Physics, Chinese Academy of Science The current status of HXMT and its calibrations.
March 2004 At A Glance ITOS is a highly configurable low-cost control and monitoring system. Benefits Extreme low cost Database driven - ITOS software.
The Project AH Computing. Functional Requirements  What the product must do!  Examples attractive welcome screen all options available as clickable.
This chapter is extracted from Sommerville’s slides. Text book chapter
1 Shawlands Academy Higher Computing Software Development Unit.
University of Palestine software engineering department Testing of Software Systems Fundamentals of testing instructor: Tasneem Darwish.
X-ray Astronomy School 2007 Databases, software, etc. for X-ray astronomy Keith Arnaud NASA Goddard University of Maryland.
Data Management Subsystem: Data Processing, Calibration and Archive Systems for JWST with implications for HST Gretchen Greene & Perry Greenfield.
NGC 6217 in DSI mode with F658N Mosaic before new superbias why we don’t release images until after SMOV! ACS CCD monitoring and pipeline calibration review.
DCS Overview MCS/DCS Technical Interchange Meeting August, 2000.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.
IXYZ Frank Marshall NASA/GSFC 25 April 2012 April 25, 20121Implementing Portals of the Universe.
Chapter 6 : Software Metrics
06/03/06Calice TB preparation1 HCAL test beam monitoring - online plots & fast analysis - - what do we want to monitor - how do we want to store & communicate.
Data Management Subsystem Jeff Valenti (STScI). DMS Context PRDS - Project Reference Database PPS - Proposal and Planning OSS - Operations Scripts FOS.
1 STScI Pipeline Thermal-vacuum data handling I. Neill Reid for WFC3 Instrument CALibration team ICAL.
EGEE is a project funded by the European Union under contract IST Testing processes Leanne Guy Testing activity manager JRA1 All hands meeting,
1 The Software Development Process  Systems analysis  Systems design  Implementation  Testing  Documentation  Evaluation  Maintenance.
©G. Millbery 2003Systems and User Interface SoftwareSlide 1 Self Documenting Systems  Systems where no external documentation is necessary  Includes:
The DARTS database Yoshihiro Ueda (ISAS) on behalf of the PLAIN members
ALMA Software B.E. Glendenning (NRAO). 2 ALMA “High Frequency VLA” in Chile Presently a European/North American Project –Japan is almost certainly joining.
April 2004 At A Glance CAT is a highly portable exception monitoring and action agent that automates a set of ground system functions. Benefits Automates.
Consortium Meeting La Palma October ICC Status Tanya Lim 1 SPIRE Consortium Meeting La Palma, Oct. 1 – ICC Status Tanya Lim RAL.
GEONS Ground Support System Java 7, JavaFX and the NetBeans Platform supporting NASA Missions Operations.
08/30/05GDM Project Presentation Lower Storage Summary of activity on 8/30/2005.
06-1L Suzaku Suzaku UG - 13 Dec, 2007 Suzaku processing and archive Lorella Angelini/HEASARC.
Oct.10, 2007EAMA7 The next-generation Infrared astronomy mission SPICA Outline of SPICA Scientific Research Concept (SciReC) SPICA 研究コンセプト概要 17 th May.
GLAST Science Support Center May 8, 2006 GUC Meeting Demonstration of GRB Spectral Analysis with the SAE David Band (GSSC/JCA-UMBC)
Swift HUG April Swift data archive Lorella Angelini HEASARC.
P6 BTEC Level 3 Subsidiary Diploma in ICT. Automation The end user of a spreadsheet may be proficient in using the software, but the more that you automate.
The Software Development Process
1 KFPA Critical Design Review – Fri., Jan. 30, 2009 KFPA Data Pipeline Bob Garwood- NRAO-CV.
March 2004 At A Glance autoProducts is an automated flight dynamics product generation system. It provides a mission flight operations team with the capability.
All rights reserved © Altec ExoMars 2018 Rover Operations Control Centre Science instruments data pipeline G. Martucci.
06-1L ASTRO-E2 ASTRO-E2 User Group - 14 February, 2005 Astro-E2 Archive Lorella Angelini/HEASARC.
The HEASARC Coordinate data, software, and media standards with other astrophysics sites. Established December 1990 The HEASARC Charter: Maintain and disseminate.
COS PIPELINE CDR Jim Rose July 23, 2001OPUS Science Data Processing Space Telescope Science Institute 1 of 12 Science Data Processing
Topic 4 - Database Design Unit 1 – Database Analysis and Design Advanced Higher Information Systems St Kentigern’s Academy.
1 Chapter 12 Configuration management This chapter is extracted from Sommerville’s slides. Text book chapter 29 1.
GLAST Science Support Center November 17, 2006 GUC Face-to-Face Meeting GLAST GI Program (with revised schedule) David Band, GSSC.
General requirements for BES III offline & EF selection software Weidong Li.
06-1L Suzaku Suzaku UG - 3 May, 2007 Suzaku processing and archive Lorella Angelini/HEASARC.
1 Getting Involved with GLAST Workshop, CFA, 06/21/ C. Shrader, NASA/GSFC GLAST Data & Software Release plans + Science Support Center Services C.
1 SUZAKU HUG 12-13April, 2006 Suzaku archive Lorella Angelini/HEASARC.
CUPID (Bob Narron)GRITS - May 14, CUPID “Customizable User Pipeline for IRS Data” Abstract: The CUPID package will allow users to run the Spitzer.
March 2004 At A Glance ITPS is a flexible and complete trending and plotting solution which provides user access to an entire mission full-resolution spacecraft.
X-ray Astronomy School 2005 Databases, software, etc. for X-ray astronomy Keith Arnaud NASA Goddard University of Maryland.
March 2004 At A Glance The AutoFDS provides a web- based interface to acquire, generate, and distribute products, using the GMSEC Reference Architecture.
EOVSA Data and Database System J. McTiernan EOVSA Technical DesignMeeting 7-Nov-2011.
Systems and User Interface Software. Types of Operating System  Single User  Multi User  Multi-tasking  Batch Processing  Interactive  Real Time.
GROUP PresentsPresents. WEB CRAWLER A visualization of links in the World Wide Web Software Engineering C Semester Two Massey University - Palmerston.
GLAST Science Support Center May 8, 2006 GUC Meeting AI#28. SAE Release Schedule David Band (GSSC/JCA-UMBC) Julie McEnery (GSFC)
Test Executive Trades (1)
Cluster Active Archive – Wideband data BM2 mode
JWST Pipeline Overview
GLAST Release Manager Automated code compilation via the Release Manager Navid Golpayegani, GSFC/SSAI Overview The Release Manager is a program responsible.
Launch and On-orbit Checkout
Hispanic Bilingual Gifted Screening Instrument
Presentation transcript:

Data Processing and User Software Ken Ebisawa (Astro-E2 GOF) presentation and demonstration

Astro-E2 Software Policy Astro-E2 data analysis will be easy –New instrument (XRS), but standard and established software scheme (ftools, xspec etc) –Hardly new data analysis technique required Astro-E2 users do not have to worry about technicality –GOF takes care of interface between instrument/satellite team and users –Prompt release of the best software and calibartion Users can concentrate on SCIENCE –Some specific analysis may not be supported (e.g., spatial-spectral analysis of clusters of galaxies)

Astro-E2 Software Characteristics Astro-E2 is Japan-US mission There are three instruments (XRS, XIS, HXD) Several institutes in two countries involved Goals: –Same software used in US and Japan –Same software is used by different instrument teams and Guest Observers We have established a software development scheme to achieve these goals

Data Processing Outline Access to RPT is restricted All the calibration and analysis start from FFF FFF complies OGIP standard Orbit File telemetry Attitude File ISAS GSFC Raw Packet Telemetry (RPT) FITS wrap First FITS File (FFF) FITS conversion Second FITS File (SFF) Calibration Astro-E2 Users Publication Analysis GSFC (backup)

Data Processing: Stage 0 (only at ISAS) ISAS receive telemetry from the satellite –FITS wrapped “Raw Packet Telemetry (RPT)” is created from satellite database –One RPT for one pointing observation –Time assignment is made on RPT –RPT is not distributed. Access to RPT restricted Attitude is calculated from RPT Orbit is determined

Data Processing: Stage 1 (at ISAS; if necessary at GFSC) RPT is converted to a set of First FITS Files (FFF) –mk1stfits is run routinely at ISAS –mk1stfits is available only at ISAS and GSFC –In case of urgent needs, GSFC is able to run mk1stfits to create FFF (back-up purpose) From one RPT, ~50 FFFs are created –Separate for instruments and observational modes –Separate for scientific data and HK data –FFF comply OGIP standard All the analysis and calibration should start from FFF –This is the only data access route –No other data access path exist

Data Processing: Stage 2 (ISAS and GSFC; if necessary by GOs) Calibration is applied to FFF –Carried out both at ISAS and GSFC –Using Astro-E2 ftools (e.g., all the tools distributed) –Guest Observers may repeat calibration, if necessary –Second FITS Files (SFF) created –Event files may be split depending on minor modes (e.g., XIS window option) “Filter file” is created from HK, attitude and orbit –“makefilter” ftool used –Later used for data screening

Data Processing: Stage 3 (ISAS, GSFC, GOs) Data screening (using filter file) –We shall learn proper data selection criteria after launch –Earth elevation, orbital condition, particle monitor condition etc. –“Screened event files” are created Make images, spectra, light curves –Standard products will be in the archives Extract “trend products” –Earth data for background monitor, calibraiton source peak spectra etc. –Used for calibration and monitoring the satellite health condition

Astro-E2 ftools: development US-Japan collaborative development “Critical” ftools for calibration are mostly written in Japan Same codes for on-line monitoring system at ISAS and ftools for general users –XRS, XIS and HXD team adopts the standardized coding style (“ANL” system) –ANL modules are converted to FITS GSFC takes care of packaging, testing, distribution, maintenance

Astro-E2 ftools: distribution Astro-E2 ftools are distributed as a component of the “HEAdas” package HEAdas: –A new component of HEASoft –Simpler, leaner, more portable, stable, extensible than ftools –Clear separation of the mission dependent part and independent part Swift ftools is in the HEAdas package

Pipe-line processing Original script written for ASCA (in shell language) at GSFC Improved and rewritten in Perl for Astro-E1 Adopted for Swift with some modification Identical script will be running at GSFC and ISAS Data distribution package for users made Same data will be also put in archives

Ground data processing Function test data (full satellite) are taken at ISAS in 2004/12/15,16 FFFs created successfully Testing further processing –SFF (calibration), filter file, data products

XRS specific development How to make XRS ARFs? –When select particular XRS pixels –Extended sources? –Planning many ray-tracing simulations How to combine XRS RMFs –XRS RMFs are different for different chips –Need to combine different RMFs How to handle different resolution events –High and Med-p combined? –How to adjust normalization?