Transiting Exoplanet Survey Satellite Daryl Swade Archive Peer Review June 18, 2014

Agenda  TESS Ground System Architecture  MAST Operations Overview  Interfaces  MAST design  Archive System  Archive User Interface  Schedule  Testing plan  Deliverables  IT Security  Document Status  Forward Work 2

TESS Ground Segment Architecture 3

MAST Overview  300 TB – Current size archive holdings.  3 TB – Average Ingest Rate per month.  19 TB – average data distribution rate per month.  1,5000,000 – Average number of searches per month. 4

TESS Raw Data  Data types  Kepler: long cadence target – 150,000 targets, 32 pixels/target, 30 minutes cadence short cadence target – 300 targets, 85 pixels/target, 1 minute cadence full frame image – one per month  TESS: Target – targets/month, 100 pixels/target, 1 minute cadence Full Frame Image – 30 minute cadence  TESS raw data volume: 6600 GB/ year (uncompressed)  Target data: 3500 GB/year  Full Frame Image data: 3100 GB/year  Kepler raw data volume: 380 GB/year (uncompressed)  Long cadence: 293 GB/year  Short cadence: 84 GB/year  FFI: 2.3 GB/year 5

Archive Operations  TESS monthly processing (Kepler quarterly)  Transferred from SPOC via hard drive  TESS archive data volume – 50 TB/year (to be refined)  Total Kepler archive data volume – 19 TB (Q0-Q17) Does not consider reprocessed data  No proprietary period on data  Availability depends primarily on SPOC processing time (<1 month)  Primary TESS web site hosted by MIT  Links to MAST web site 6

MAST Interfaces 7

TESS Archive Data Products from SPOC  Full Frame Image  Target Pixel Files  All pixels for a single target  Raw and calibrated values  Light Curves  Output of photometric analysis and cotrending  Collateral Pixel Files  Calibration data collected on-board  Ground calibration data  Cotrending Basis Vectors Systematic trends present in the ensemble of flux data for each CCD  Pixel Response Function Point spread function and pointing and electronic systematics  Flat Field Pixel response non-uniformity map 8

TESS Data Products from TSO  TESS Input Catalog (TIC)  Threshold Crossing Events (TCE) summary report  Identified from light curves  Data Validation (DV) results  Characterize and validate TCEs  TESS Objects of Interest (TOI) Catalog  Follow-up observations 9

Archive System Overview 10

Archive Functional Overview (1/3)  Data Receipt  Receives data from the SPOC  Receives data from the TESS Science Office  Accepts data and generate an archive manifest  Ingest  Stores TESS data in the Storage Broker  Catalogs TESS metadata the Archive Database  Storage Broker  Provides a storage service manager for the file I/O on the primary, secondary safestore, and public data stores 11

Archive Functional Overview (2/3)  Archive Catalog  Provides an on-line science catalog of TESS metadata for searches, data mining, and data retrieval Implemented on relational DBMS Contains metadata extracted from the science data keyword values  Stages TESS catalogs for archival use  Data Distribution  Retrieves requested data from storage broker  Distributes data to archive users via electronic mechanisms or removable media  Archive Operator Interface  Maintains an operator interface for monitoring and control of the archive system 12

Archive Functional Overview (3/3)  Archive User Interface  Makes TESS data available to the astronomical community Web-based query/preview/retrieval interface  Hosts software tools required to search the TESS archive catalog This includes user access to target lists.  Hosts software tools required to access data  Provides data mining, visualization, and correlation with other major mission datasets International Virtual Observatory Alliance (IVOA) standards compliant 13

Archive Catalog  Three main types of database tables  File tracking tables – What datasets and files exist, the file/dataset status, and the relationship between them.  Instrument and data specific tables  Common Archive Observation Model (CAOM) Database 14

Common Archive Observation Model (CAOM)  Unified model for metadata about observation (position, time, wavelength) implemented in a database enabling easy uniform cross-mission searches  Extensible for mission-specific data  May require additional static data not in FITS file  Enables easy creation of Virtual Observatory (VO) services  VO allows s/w access to data holdings  Current version enabled as part of the MAST Data Discovery or Portal Interface 15

Data Distribution Options  Electronic mechanisms  Streamed through browser  Data delivered to MAST staging area for user down load via browser or FTP  Data delivered directly to user disk via SFTP  Wget and cURL scripts created and delivered to user  Removable media: Hard drive, DVD  Archive Operator has the ability to override specified user preference. 16

Archive Distribution Performance  MAST has a requirement to be capable of distributing 2.5 TBytes of archived science data products externally and 9.4 TBytes internally per month. 17  MAST is implementing scalable hardware and networking technologies, with high-bandwidth ISP to the internet to handle increased loads.  Historic data distribution metric includes internal transfers and uncompressed volumes. Internal data transfers do not use ISP bandwidth. JWST and TESS estimates include internal transfers.

Archive User Interface: Overview  Archive User Interface (AUI) provides ability to:  Search for data products by: Data file name Target name, position or list of positions Other observation parameters (instrument, waveband, program ID, etc.)  Explore and discover data through the Data Portal  Request retrieval of those products 18

Kepler Data Search and Retrieval Form 19

Results of Kepler Data Search 20

Data Retrieval Request 21

Kepler Target Search Form 22

Results of Target Search 23

MAST Portal: Internal Observations 24

MAST Portal: VO 25

Archive Development Schedule  Phase B ( April 1, 2014 – October 1, 2014 )  Continue developing TESS Archive Operations Concept  Continue developing TESS-MAST ICD Work with SPOC to develop SPOC-MAST ICD section Work with TSO to develop TSO-MAST ICD section  Continue to participate in Ground Segment Monthly Technical Reviews  Participate in TESS Mission PDR (September 2014)  Phase C ( October 2, 2014 – November 29, 2016)  Develop TESS specific archive components based on Archive Operations Concept and TESS-MAST ICD  Participate in TESS Mission CDR  Participate in Ground Segment testing  Phase D ( November 30, 2016 – July 17, 2017)  Develop TESS Archive User’s Guide 26

Archive System Enhancements for TESS 27 TESS mission specific s/w

Archive Development Effort 28 StaffPhase(s)Total FTEAvg. FTE Program ManagerB-D Project ManagerB-D Systems EngineerB-D Archive Database EngineerC-D Archive Software EngineerC-D Archive ScientistD Archive System Admin.C0.01 Phase B: April 1, 2014 – October 1, 2014 Phase C: October 2, 2014 – November 29, 2016 Phase D: November 30, 2016 – July 17, 2017

Testing Plan  MAST will work with the SPOC and TSO to develop test data.  MAST will participate in Ground Segment end-to-end tests. 29

MAST Deliverables 30 DeliverableResponsibilityPhaseDate of Delivery TESS Archive Operations ConceptSystems EngineerBPDR TESS-Archive ICD SPOC-Archive TSO-Archive Systems EngineerB/CPDR/CDR Archive Catalog Design SpecificationSystems EngineerB/CPDR/CDR Archive ArchitectureSystems EngineerB/CPDR/CDR Archive Software DesignSystems EngineerCCDR TESS Archive User InterfaceSoftware EngineersC/DGS Test Archive Test ProceduresSystems EngineerDGS Test Archive Operations ProceduresArchive OperatorDLaunch TESS Archive User’s GuideArchive ScientistELaunch

IT Security  MAST will protect all TESS data against unauthorized access per requirements established by NPR , Security of Information Technology.  STScI is moving towards becoming a NASA external system  STScI will have a single Moderate system Security Plan  Compliance with FISMA and NPR A still required 31

Document Status  TESS Archive Operations Concept  Initial draft available dated June 12, 2014  TESS-MAST ICD  SPOC-MAST ICD Initial draft available  TSO-MAST ICD Initial draft available by PDR 32

Forward Work  Systems Engineering  Complete TESS Archive Operations Concept  Work with SPOC to complete SPOC-MAST section of TESS- MAST ICD  Work with TSO to complete TSO-MAST section of TESS-MAST ICD  Prepare for PDR  Archive Software Development  Data Receipt and Ingest  Archive Catalog  Archive User Interface 33