MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.

Slides:



Advertisements
Similar presentations
A Search for Earth-size Planets Borucki – Page 1 KEPLER; Data Validation and Follow Up Observations CoRoT Symposium W.J. Borucki & the Kepler Team 5 February.
Advertisements

STEREO Science Center Status Report William Thompson NASA Goddard Space Flight Center Phase E Kickoff Meeting February 23, 2007.
NOAO/Gemini Data workshop – Tucson,  Hosted by CADC in Victoria, Canada.  Released September 2004  Gemini North data from May 2000  Gemini.
GLAST LAT ProjectManager’s Face to Face - ISOC, 17 March GLAST Large Area Telescope WBS 4.1.B Instrument Science Operations Center Manager’s Face.
SOFIA Archiving Requirements for the SOFIA Data Cycle System Mark Morris, UCLA Joe Mazzarella & Steve Lord, IPAC John Milburn & Jochen Horn, UCLA.
GLAST LAT ProjectISOC Peer Review - March 2, 2004 Document: LAT-PR Section 2.1 Requirements 1 Gamma-ray Large Area Space Telescope GLAST Large.
Physical design. Stage 6 - Physical Design Retrieve the target physical environment Create physical data design Create function component implementation.
Implementation Review1 Moving Archive Data to the EMC Storage Array March 14, 2003 Faith Abney.
09 May 2014page 1 ROCC to PPL ICD Initial Definition ExoMars Rover Operations Control Center ROCC.
Leicester Database & Archive Service J. D. Law-Green, S. W. Poulton, J. Osborne, R. S. Warwick Dept. of Physics & Astronomy, University of Leicester LEDAS.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
STEREO Science Center SWG – Dec 2004 William Thompson NASA Goddard SFC Code Greenbelt, MD
Data Processing and User Software Ken Ebisawa (Astro-E2 GOF) presentation and demonstration.
Records Survey and Retention Schedule Recertification 2011.
Upcoming Enhancements to the HST Archive Mark Kyprianou Operations and Engineering Division Data System Branch.
GAUDI Ground-based Asteroseismology Uniform Database Interface E. Solano Bases de données en spectroscopie stellaire. Paris.
Transiting Exoplanet Survey Satellite Daryl Swade September 24, 2014 TESS DMS Management Meeting.
6e-1 Science Data Products Daryl Swade DMS Systems Engineer S&OC System Design Review #1.
Data Management Subsystem: Data Processing, Calibration and Archive Systems for JWST with implications for HST Gretchen Greene & Perry Greenfield.
AERONET Web Data Access and Relational Database David Giles Science Systems and Applications, Inc. NASA Goddard Space Flight Center.
DMS Systems Design Review 3
Dec 2, 2014 MAST Data Discovery Portal Tom Donaldson Tony Rogers.
At A Glance VOLT is a freeware, platform independent tool set that coordinates cross-mission observation planning and scheduling among one or more space.
1 Next Generation of Operational Earth Observations From the National Polar-Orbiting Operational Environmental Satellite System (NPOESS): Program Overview.
Transiting Exoplanet Survey Satellite Daryl Swade Archive Peer Review June 18, 2014.
Data Management Subsystem Jeff Valenti (STScI). DMS Context PRDS - Project Reference Database PPS - Proposal and Planning OSS - Operations Scripts FOS.
Aquarius Project Status 7 th Aquarius/SAC-D Science Team Meeting 11 April 2012.
2-1 DMS Requirements Status Daryl Swade DMS Systems Engineer Nov. 6, 2013 S&OC Data Management System Design Review 3.
Introduction to Apache OODT Yang Li Mar 9, What is OODT Object Oriented Data Technology Science data management Archiving Systems that span scientific.
SPACE TELESCOPE SCIENCE INSTITUTE Operated for NASA by AURA COS Status FUV Detector “1-bounce design” NUV Detector HST aberration fully-corrected Calibration.
A Search for Earth-size Planets Borucki – Page 1 Roger Hunter (Ames Research Center) & Kepler Team March 26, 2010.
Planetary Science Archive PSA User Group Meeting #1 PSA UG #1  July 2 - 3, 2013  ESAC PSA Archiving Standards.
ASI-Eumetsat Meeting Matera, 4-5 Feb CNM Context Matera, February 4-5, 20092ASI-Eumetsat Meeting.
COS PIPELINE PDR Daryl Swade December 7, 2000OPUS / OTFR Space Telescope Science Institute 1 of 24 Science Data Processing
MAST Users Group – July 2009 MAST will provide the archive user interface for Kepler data, primarily light curves and target pixel data. ASB Staffing for.
PDS Geosciences Node Page 1 Archiving Mars Mission Data Sets with the Planetary Data System Report to MEPAG Edward A. Guinness Dept. of Earth and Planetary.
06/30/ Data Product Service (DPS) Packaging and Context Dan Crichton Steve Hughes Ron Joyner Chris Mattman Paul Ramirez Peter Shames.
Label Design Tool Management Council F2F Washington, D.C. November 29-30, 2006
SOFTWARE TOOLS FOR DATA QUALITY ANALYSIS The Science Data Analysis Tool (SDAT) is the primary tool used by the DQA team. SDAT can access all raw and processed.
Ames Research CenterDivision 1 Information Power Grid (IPG) Overview Anthony Lisotta Computer Sciences Corporation NASA Ames May 2,
EOSDIS Status 10/16/2008 Dan Marinelli, Science Systems Development Office.
Archival Workshop on Ingest, Identification, and Certification Standards Certification (Best Practices) Checklist Does the archive have a written plan.
WGISS /09/2015 DATA PRESERVATION – CNES APPROACH B. Chausserie-Laprée.
Common Archive Observation Model (CAOM) What is it and why does JWST care?
CLASS Information Management Presented at NOAATECH Conference 2006 Presented by Pat Schafer (CLASS-WV Development Lead)
2015 GLM Annual Science Team Meeting: Cal/Val Tools Developers Forum 9-11 September, 2015 DATA MANAGEMENT For GLM Cal/Val Activities Helen Conover Information.
Nov Common Archive Observation Model What is CAOM and why should MAST use it? Brian McLean.
Swift HUG April Swift data archive Lorella Angelini HEASARC.
2007Sep06 EAC Butler - Software Overview 1 Software Overview Bryan Butler.
OAIS Rathachai Chawuthai Information Management CSIM / AIT Issued document 1.0.
Data Archives: Migration and Maintenance Douglas J. Mink Telescope Data Center Smithsonian Astrophysical Observatory NSF
Page 1JSOC Peer Review 17Mar2005 HMI & AIA JSOC Architecture Science Team Forecast Centers EPO Public Catalog Primary Archive HMI & AIA Operations House-
06-1L ASTRO-E2 ASTRO-E2 User Group - 14 February, 2005 Astro-E2 Archive Lorella Angelini/HEASARC.
STEREO Science Center Status Report William Thompson NASA Goddard Space Flight Center STEREO SWG March 27-28, 2007.
VIIRS Product Evaluation at the Ocean PEATE Frederick S. Patt Gene C. Feldman IGARSS 2010 July 27, 2010.
COS PIPELINE CDR Jim Rose July 23, 2001OPUS Science Data Processing Space Telescope Science Institute 1 of 12 Science Data Processing
1 Transiting Exoplanet Survey Satellite Daryl Swade Archive Team Meeting June 16, 2014.
Transiting Exoplanet Survey Satellite December 8, 2014 TESS Kickoff.
WFC3 PIPELINE CDR Jim Rose October 16, 2001OPUS Science Data Processing Space Telescope Science Institute 1 of 13 Science Data Processing
Implementation Review1 Archive Ingest Redesign March 14, 2003.
March 2004 At A Glance ITPS is a flexible and complete trending and plotting solution which provides user access to an entire mission full-resolution spacecraft.
GLAST LAT ProjectCDR/CD-3 Review May 12-16, 2003 Document: LAT-PR Section 5 IOC Subsystem 1 GLAST Large Area Telescope: IOC Subsystems WBS: 4.1.B.
Distributed Archives Interoperability Cynthia Y. Cheung NASA Goddard Space Flight Center IAU 2000 Commission 5 Manchester, UK August 12, 2000.
SwCDR (Peer) Review 1 UCB MAVEN Particles and Fields Flight Software Critical Design Review Peter R. Harvey.
1 (c) 2013 FabSoft. MOST Cloud Service What is a Cloud Service? A cloud service is internet-based, meaning that MOST is hosted on a server farm on the.
Chapter 1 Overview of Databases and Transaction Processing.
HST and JWST Pipelines and Reference Files
DADS Ingest and Distribution Support for WFC3 Daryl Swade
Future Data Architecture Cloud Hosting at USGS
Observing with Modern Observatories (the data flow)
Presentation transcript:

MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE SMITHSONIAN ASTROPHYSICAL OBSERVATORY MAST Daryl Swade STScl

G O D D A R D S P A C E F L I G H T C E N T E R 22 Agenda Changes Since CSR Requirements Flow down MAST Overview MAST Requirements Interfaces Schedule Risks / Open Issues

G O D D A R D S P A C E F L I G H T C E N T E R 33 Changes since CSR Full Frame Image (FFI) cadence change from 2 hours to 30 minutes will impact archive data volume.

G O D D A R D S P A C E F L I G H T C E N T E R 44 Requirements Flow Down Show the requirements flow down and where the MAST fits into the flow.

G O D D A R D S P A C E F L I G H T C E N T E R 55 MAST Overview 278 TB – Holdings size as of November 1, TB – Average Ingest Rate per month. 19 TB – average data distribution rate per month. 1,5000,000 – Average number of searches per month.

G O D D A R D S P A C E F L I G H T C E N T E R 66 TESS Data Archive Context

G O D D A R D S P A C E F L I G H T C E N T E R 77 TESS Archive Data Flow

G O D D A R D S P A C E F L I G H T C E N T E R 88 Archive Functional Overview Data Receipt –Accept data and generate archive manifest Ingest –Store and catalog all data Archive Catalog –Provide an on-line science catalog of TESS metadata for searches, data mining, and data retrieval Implemented on relational DBMS –Stage TESS Target Catalog for archival use Storage Broker –Maintain all data files on primary archive –Optionally, maintain data files on safe store and on-line cache

G O D D A R D S P A C E F L I G H T C E N T E R 99 Archive Functional Overview (cont.) Archive User Interface –Make TESS data available to the astronomical community though the MAST interface at STScI Web-based query/preview/retrieval interface –Create and maintain software tools required to search the TESS archive catalog This includes user access to target lists. –Create and maintain software tools required to access data Data Distribution –Retrieve requested data from storage broker –Ensure proper proprietary access to TESS data in the archive based on the Science Office defined TESS Data Policy Archive Operator Interface –Operator interface for monitoring and control of the system

G O D D A R D S P A C E F L I G H T C E N T E R 10 TESS Science Data Archive Functions from CSR Storage of raw, reduced, and calibrated pixel and light curve data, including centroids, for all stars and planet candidates, to be transferred to MAST for public access on a trimester basis. With the ~22 day timeline for data passage through the pipeline, this implies that data will be 2-4 months old at time of public release. Storage of image data: raw, reduced, calibrated. Storage of full-frame images acquired by TESS for validation of instrument performance. Storage of some TESS engineering data, as needed, to update science header keywords. Lists of Threshold Crossing Events from TPS in XML format will be exported to MAST (and to the Exoplanet Archive). Data Validation reports for each target with TCEs processed by DV will be exported to the MAST as PDFs and with numerical results in XML files. High Level Science Products provided to STScI from the TESS Science Operations Center. These include light curves and any compiled catalog data including a TESS Object of Interest catalog (TOI). A simple interface based on the Mission supplied FITS files for data search and retrieval.

G O D D A R D S P A C E F L I G H T C E N T E R 11 MAST Requirements Discuss the Level 3 requirements

G O D D A R D S P A C E F L I G H T C E N T E R 12 Driving Requirements Discuss the driving requirements for MAST development Data Receipt The TESS Data Archive shall receive to be specified data from the SPOC. The TESS Data Archive shall receive to be specified data from the TESS Science Office. Data Receipt shall generate a manifest for all data received. Ingest Ingest shall store all TESS data in the TESS Data Archive. Ingest shall catalog all TESS data in the TESS Data Archive. Storage Broker The Storage Broker shall store all TESS data in the primary archive. The Storage Broker shall store to be selected TESS data on safe store media. The Storage Broker shall store to be selected TESS data in the on-line cache.

G O D D A R D S P A C E F L I G H T C E N T E R 13 Driving Requirements (cont.) Discuss the driving requirements for MAST development Archive Catalog The TESS Data Archive shall provide an on-line science catalog of metadata for searches, data mining, and data retrieval. The TESS science catalog shall contain metadata extracted from the science dataset keyword values. The TESS archive database shall track each data set with a field indicating the proprietary status of the corresponding data files. Data Distribution TESS data shall be made available to the astronomical community through the MAST interface at STScI under the proprietary rights restrictions defined by the TESS Data Policy. The TESS Data Archive shall be able to distribute data to archive users via electronic mechanisms or removable media with a user specified preference, but at the discretion of the Archive Operator.

G O D D A R D S P A C E F L I G H T C E N T E R 14 Driving Requirements (cont.) Discuss the driving requirements for MAST development Archive User Interfaces MAST shall create and maintain software tools required to access data in the TESS Data Archive for at least 10 years after the end of the mission. Data Security MAST shall protect all TESS data against unauthorized access per requirements established by NPR , Security of Information Technology.

G O D D A R D S P A C E F L I G H T C E N T E R 15 MAST Interfaces Discuss the interfaces between the POC, SPOC and MAST. –Talk about the interface including the network and protocols –Talk about the data flow and the latency.

G O D D A R D S P A C E F L I G H T C E N T E R 16 TSO data to MAST Data productDelivery frequency Data volumeDelivery mechanism TESS Input Catalog Targets of Interest List Archive follow up data

G O D D A R D S P A C E F L I G H T C E N T E R 17 SPOC Data to MAST Data ProductGeneration Frequency Delivery frequency Data volumeDelivery mechanism Telemetry data Science pixels - uncalibrated Collateral pixels - uncalibrated Science pixels - calibrated Collateral pixels - calibrated Calibration reference files Focal plane geometry model Calibrated light curves Centroid time-series Full frame images - uncalibrated Full frame images - calibrated Pixel mapping reference files Optimal aperture definitions Pointing profiles Data validation products

G O D D A R D S P A C E F L I G H T C E N T E R 18 Archive User Interface

G O D D A R D S P A C E F L I G H T C E N T E R 19 Schedule Show the MAST Schedule –Include status and path to PDR

G O D D A R D S P A C E F L I G H T C E N T E R 20 Risks / Open Issues Failing to involve MAST staff closely and early in Ground Segment activities risks delayed optimization of TESS science archive usage and increased software development costs in phases D and E.