Aquarius Algorithm Workshop March 9-11, 2010 Santa Rosa, CA Aquarius Data Processing System (ADPS) Overview and Status Fred Patt.

Slides:



Advertisements
Similar presentations
Network II.5 simulator ..
Advertisements

During spacecraft-level environmental testing, after all instruments were integrated, the government assembled a data clerk team to support test data collection.
Page 1GlobColour CDR Meeting – July 10-11, 2006, ESRIN All rights reserved © 2006, ACRI-ST Resulting Technical Specification.
OVERVIEW TEAM5 SOFTWARE The TEAM5 software manages personnel and test data for personal ESD grounding devices. Test and personnel data may be viewed/reported.
STEREO Science Center Status Report William Thompson NASA Goddard Space Flight Center Phase E Kickoff Meeting February 23, 2007.
GLAST LAT ProjectISOC CDR, 4 August 2004 Document: LAT-PR-04500Section 3.11 GLAST Large Area Telescope: Instrument Science Operations Center CDR Section.
GLAST LAT ProjectISOC Peer Review - March 2, 2004 Document: LAT-PR Section 2.1 Requirements 1 Gamma-ray Large Area Space Telescope GLAST Large.
Aquarius/SAC-D Mission Mission Simulators - Gary Lagerloef 6 th Science Meeting; Seattle, WA, USA July 2010.
Components and Architecture CS 543 – Data Warehousing.
Section 15-1GLAST Ground System Design Review August 18&19, 2004 ISOC Organization ISOC Manager R Cameron Commanding, H&S Timeline Planning Command Generation.
Aquarius/SAC-D Science Team Meeting July 19, 2010 Seattle, wA Aquarius Data Processing System (ADPS) and Data Distribution Fred Patt.
Aquarius/SAC-D Mission Error Validation and Early Orbit Corrections Gary Lagerloef 6 th Science Meeting; Seattle, WA, USA July 2010.
MWR Algorithms (Wentz): Provide and validate wind, rain and sea ice [TBD] retrieval algorithms for MWR data Between now and launch (April 2011) 1. In-orbit.
March 2004 At A Glance ITOS is a highly configurable low-cost control and monitoring system. Benefits Extreme low cost Database driven - ITOS software.
System Design/Implementation and Support for Build 2 PDS Management Council Face-to-Face Mountain View, CA Nov 30 - Dec 1, 2011 Sean Hardman.
STEREO Science Center SWG – Dec 2004 William Thompson NASA Goddard SFC Code Greenbelt, MD
2013 GOES-R AWG 2 nd Validation Workshop January 2014 Status of Activities and Plans for the GOES-R Core Ground System 1 Satya Kalluri Ph.D. GOES-R.
1ESDIS HDF-EOS Workshop IV Landover, Maryland, September 20, 2000 The Landsat 7 Processing System ( LPS ) Level Zero-R Science Products Michael R. Reid.
AERONET Web Data Access and Relational Database David Giles Science Systems and Applications, Inc. NASA Goddard Space Flight Center.
Web Trnsport – Beta Testing and Implementation TUG Roundtable Discussion Elizabeth Rodgers Info Tech, Inc. October 9, 2007.
MASSACHUSETTS INSTITUTE OF TECHNOLOGY NASA GODDARD SPACE FLIGHT CENTER ORBITAL SCIENCES CORPORATION NASA AMES RESEARCH CENTER SPACE TELESCOPE SCIENCE INSTITUTE.
Data Management Subsystem Jeff Valenti (STScI). DMS Context PRDS - Project Reference Database PPS - Proposal and Planning OSS - Operations Scripts FOS.
Aquarius Project Status 7 th Aquarius/SAC-D Science Team Meeting 11 April 2012.
Certification and Accreditation CS Phase-1: Definition Atif Sultanuddin Raja Chawat Raja Chawat.
John T. Gunn, AVDS Manager Earth & Space Research 1910 Fairview Ave. E., Suite 210 Seattle, WA Ph: ; 1 st Joint GOSUD/SAMOS.
Final Version Micro-Arcsecond Imaging Mission, Pathfinder (MAXIM-PF) Mission Operations Tim Rykowski Jeffrey Hosler May 13-17, 2002.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger Life Cycle Architecture Review.
® Kick off meeting. February 17th, 2011 QUAlity aware VIsualisation for the Global Earth Observation system of systems GEOVIQUA workshop February, the.
SCIENCE PROCESSING OVERVIEW David Le Vine Aquarius Deputy PI 07 July 2009.
Planetary Science Archive PSA User Group Meeting #1 PSA UG #1  July 2 - 3, 2013  ESAC PSA Archiving Standards.
1 Mexico Regional AMDAR Workshop November 2011 Data Quality Monitoring and Control (QM / QC) Axel Hoff Convenor of WMO AMDAR Panel‘s Science and Technical.
PDS Geosciences Node Page 1 Archiving Mars Mission Data Sets with the Planetary Data System Report to MEPAG Edward A. Guinness Dept. of Earth and Planetary.
September 4, 2003MODIS Ocean Data Products Workshop, Oregon State University1 Goddard Earth Sciences (GES) Distributed Active Archive Center (DAAC) MODIS.
VIPER Quality Assessment Overview Presenter: Sathyadev Ramachandran, SPS Inc.
GLAST Science Support CenterAugust 9, 2004 Users’ Committee Meeting GSSC USER SUPPORT David Band – GSSC.
CLASS Information Management Presented at NOAATECH Conference 2006 Presented by Pat Schafer (CLASS-WV Development Lead)
Swift HUG April Swift data archive Lorella Angelini HEASARC.
AMSR-E SIPS Processing Status Kathryn Regner Information Technology and Systems Center at the University of Alabama in Huntsville
22 March 2007 Science Simulator Overview and Status ICDS Susie Kennison.
Workshop Agenda: Day One 9:30 IntroductionLagerloef / Le Vine 9:45 Workshop objectivesG. Feldman 10:00 Overview of the Aquarius Data Processing System:G.
March 2004 At A Glance autoProducts is an automated flight dynamics product generation system. It provides a mission flight operations team with the capability.
06-1L ASTRO-E2 ASTRO-E2 User Group - 14 February, 2005 Astro-E2 Archive Lorella Angelini/HEASARC.
STEREO Science Center Status Report William Thompson NASA Goddard Space Flight Center STEREO SWG March 27-28, 2007.
VIIRS Product Evaluation at the Ocean PEATE Frederick S. Patt Gene C. Feldman IGARSS 2010 July 27, 2010.
Mission Operations Review February 8-10, 2010 Cordoba, ARGENTINA SECTION 16.x Aquarius Science Commissioning and Acceptance Draft 2 Prepared by: Gary Lagerloef,
August 2003 At A Glance The IRC is a platform independent, extensible, and adaptive framework that provides robust, interactive, and distributed control.
National Aeronautics and Space Administration Aquarius Validation Data System Overview and Status Algorithm Workshop March 21-22, 2007 John Gunn Earth.
Aquarius Mission Simulation A realistic simulation is essential for mission readiness preparations This requires the ability to produce realistic data,
18 th GIST Meeting 14 th –16 th May 2003 Ground Segment (GGSPS) Report 1 GERB Ground Segment B.C.Stewart RAL.
Science Operation Readiness Review ICESat SIPS (I-SIPS) Status N0v 30, 2001 David Hancock ICESat Science Ground System Manager
Welcome to the PRECIS training workshop
STEREO Science Center Status Report William Thompson NASA Goddard Space Flight Center STEREO SWG November 2007 Pasadena, California.
ADPS Science Software Development Bryan Franz NASA Ocean Biology Processing Group Aquarius Data Processing Workshop, NASA/GSFC, March 2007.
Cal/Val for physics MED-MFC internal meeting CMCC-INGV-SOCIB Lecce E. Clementi, INGV.
MODIS SDST, STTG and SDDT MODIS Science Team Meeting (Land Discipline Breakout Session) July 13, 2004 Robert Wolfe Raytheon NASA GSFC Code 922.
1 SUZAKU HUG 12-13April, 2006 Suzaku archive Lorella Angelini/HEASARC.
March 2004 At A Glance ITPS is a flexible and complete trending and plotting solution which provides user access to an entire mission full-resolution spacecraft.
GLAST LAT ProjectCDR/CD-3 Review May 12-16, 2003 Document: LAT-PR Section 5 IOC Subsystem 1 GLAST Large Area Telescope: IOC Subsystems WBS: 4.1.B.
Zou Ziming 1 Ma Wenzhen Li Lei Zhao Hua Wang Chi 1: Center for Space Science and Applied Research Chinese Academy of Sciences Moscow ·
Aquarius Level 0-to-1A Processing Rule #1: save everything from the Level 0 data. Rule #2: never forget Rule #1! The objective is to ensure that the Level.
From Missions to Measurements: an Ocean Discipline Experience.
Mission Operations Review February 08-10, 2010 Falda del Carmen, Córdoba, ARGENTINA DAY 2 SECTION 08 Aquarius Data Processing System (ADPS) Gene Carl Feldman.
5-7 May 2003 SCD Exec_Retr 1 Research Data, May Archive Content New Archive Developments Archive Access and Provision.
Simulation Production System Science Advisory Committee Meeting UW-Madison March 1 st -2 nd 2007 Juan Carlos Díaz Vélez.
NOAA Near Real-Time AIRS Processing and Distribution System Walter Wolf Mitch Goldberg Lihang Zhou NESDIS/ORA/CRAD.
Page 1 June 26, 2016 NPOESS Preparatory Project (NPP) Science Data Segment (SDS) Ocean PEATE Status and Plans January 27, 2010 Ocean PEATE Team.
A Solution for Maintaining File Integrity within an Online Data Archive Dan Scholes PDS Geosciences Node Washington University 1.
MODIS SST Processing and Support for GHRSST at OBPG
Data Discovery Tools and Services Part B
Presentation transcript:

Aquarius Algorithm Workshop March 9-11, 2010 Santa Rosa, CA Aquarius Data Processing System (ADPS) Overview and Status Fred Patt

2 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Overview and Status Data Acquisition and Ingest Science Data Processing Level-2 Science Software Implementation Level-2 Data Product Format (separate presentation) Data Archive and Distribution Mission Simulation Aquarius Web Site What We Need Agenda

3 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Data Processing System Overview The Aquarius Ground System at NASA/GSFC consists of the Aquarius Data Processing System (ADPS) and the Aquarius Command and Control System. The ADPS will support: –Acquisition and ingest of Aquarius/SAC-D data from CONAE. –Acquisition of dynamic ancillary required to support the science processing. –Processing of Aquarius data to standard product levels using algorithms provided by the Science Team. –Quality control of data products using methods specified by the Science Team. –Match-up of Aquarius data with in situ salinity measurements. –Archive and distribution of all data products during the active life of the mission to the science team and general science community. –Delivery of source data (Level-1a) and “validated” Level-2 and Level-3 derived data products to the PO.DAAC. The ADPS will be supported within the existing infrastructure of the Ocean Biology Processing Group (OBPG).

4 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Data Acquisition and Ingest Aquarius/SAC-D data downlink strategy comprises multiple, overlapping downlinks each day of Aquarius science high-rate data (HRD) and Aquarius and SAC-D housekeeping telemetry (HKT). –3 to 5 passes per 24 hours over the Cordoba ground station. –1 pass per week over Matera ground station, which is transferred to Cordoba after pass. – minutes per pass. –Downlink full science memory of 110 MB at 4Mbps (just under 4 minutes), repeating throughout pass. ADPS automatically acquires downlinks from the CUSS (separate files for Science, SAC-D/HKT) after each pass. Orbit Products including Ephemeris files are generated daily by the CODS and acquired by ADPS. All files are ingested and cataloged automatically after acquisition, and made available for processing.

5 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Overview and Status Data Acquisition and Ingest Science Data Processing Level-2 Science Software Implementation Level-2 Data Product Format (separate presentation) Data Archive and Distribution Mission Simulation Aquarius Web Site What We Need Agenda

6 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Data Processing Overview Level-0 Preprocessor Level-0 to 1A Level-1A to 2 Level-1A Merge Raw Level-0 Downlink File Level-1A orbit Level-2 Level-3 Map SAC-D HKT Data Level-2 to 3 Binning Level-2 to 3 Smoothing Level-3 Binning Level-3 Mapping Dynamic Ancillary Data Static Ancillary Data Level-1A temp Level-0 Level-3 Bin Level-0 Report Level-1a Report Automated Telemetry Analysis Tool Standard Data Processing Web-Based Telemetry Monitoring Tools Telemetry Report Predicted Downlink Times In-Situ Match-up Process Level-2 Match-up AVDS Bias Adjustment Web-Based Browse, Download and Order Tools CONAE-provided Interactive Telemetry Analysis Tool SAC-D HKT Data

7 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Science Data Processing The Aquarius science data processing consists of the following steps: Level-0 Preprocessor – process each HRD downlink file to produce a single time-ordered set of Aquarius science blocks and removes duplicates– developed by Joel Gales - ADPS. Level-0-to-1A – separate Level-0 files into orbits, unpack science data, incorporate overlapping ephemeris and SAC-D HKT and convert Aquarius HKT – developed by Joel Gales - ADPS. Telemetry Analysis – HKT data from Level-1A files analyzed for web display on the Aquarius web site – developed by Susie Kennison, Norman Kuring and Liang Hong - ADPS/ACCS. Interactive Telemetry Analysis - HKT files analyze using CONAE-provided telemetry viewer. Level-1A Merge – consolidate Level-1A files from overlapping downlinks into a single, best- quality full orbit product – Joel Gales - ADPS. Level-1A-to-2 – perform calibration, atmospheric correction and salinity retrieval for Aquarius science data – radiometer and scatterometer code delivered by RSS and JPL, respectively; integration and input/output code developed by Joel Gales and Liang Hong - ADPS. Level-2-to-3 Binning – Level-2 salinity retrievals for one day geographically projected and collected into equal-area bins – existing OBPG software modified by Joel Gales- ADPS. Level-2-to-3 Smoothing – Level-2 salinity retrievals for one day optimally interpolated into equal-area bins – algorithm by Jonathan Lilly/ESR implemented by Joel Gales - ADPS. Level-3 Binning and Mapping – Level-3 binned files aggregated to longer time periods (weekly, monthly, etc.) and projected onto map grid – existing OBPG software. All processing steps are run automatically when all input data sets are available. All software exists in at least preliminary prelaunch form at this time.

8 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop From now until launch we need to: –Integrate latest Radiometer and Scatterometer code into data processing system (Jan/Feb/Mar 2010) –Develop ingest procedures for additional ancillary data required by new code (Feb/Mar 2010) –Verify output products with algorithm providers (Feb/Mar 2010) –Process one year simulated data set and make available to science community (March 2010) –Begin real-time mission simulation (April/May > Launch) –Incorporate additional prelaunch updates into the science software (as needed) –Participate in mission rehearsals as scheduled Status - Science Code TBD

9 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Overview and Status Data Acquisition and Ingest Science Data Processing Level-2 Science Software Implementation Level-2 Data Product Format (separate presentation) Data Archive and Distribution Mission Simulation Aquarius Web Site What We Need Agenda

10 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level-2 Software Implementation Implemented using software delivered by RSS and JPL with I/O by Joel Gales Current Capabilities: Processes a Level-1A input product. Performs radiometer Level-1B calibration and Level-2 processing using latest code from RSS. Performs scatterometer Level-1B calibration and Level-2 processing using latest code from JPL. Reads SST, wind, atmospheric, sea ice and solar flux data from dynamic ancillary files. Computes geolocation. Generates HDF5 output product.

11 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level 2 Processing Flow Look-up Tables Level-1A Product L2gen_aquarius Level-2 Product QMET Ancillary Files atm_tb_aquarius L2gen_scatter Radiometer Ancillary Files Scatterometer ASCII Files mk_aquarius_ ancillary_data Solar Flux Ancillary Data SST Ancillary Files Static Ancillary Files ATM Ancillary Files Sea Ice Ancillary Files Look-up Tables

12 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Dynamic Ancillary Data Data TypeSourceInterface Geopotential height, surface Temperature at surface Atmospheric pressure at surface Relative Humidity, entire atmosphere Precipitable water Cloud water U/V wind 10 m Geopotential height (profile) Temperature (profile) Relative Humidity (profile) Cloud Water Mixing Ratio (profile) NCEP Primary: tads1u.ecs.nasa.gov Secondary: dss.ucar.edu Sea Surface TemperatureReynolds (NCEP)Primary: eclipse.ncdc.noaa.gov Solar Flux and Flares at 1.4 GHzUSAF (NGDC) Primary: ftpdir/latest/dayind.txt -- production Sea Ice Concentration/ Extent/AgeNSIDC Primary: polar.ncep.noaa.gov/pub/cdas/archive -- for 2007 simulation polar.ncep.noaa.gov/pub/cdas/ -- for production

13 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Today’s Objectives Agree on our understanding of the current software deliveries and what they are capable of. Approve the current implementation strategy. Agree on the ancillary data list and sources for the 2007 and mission simulations. (Product content objectives will be addresses in the next presentation)

14 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Overview and Status Data Acquisition and Ingest Science Data Processing Level-2 Science Software Implementation Level-2 Data Product Format (separate presentation) Data Archive and Distribution Mission Simulation Aquarius Web Site What We Need Agenda

15 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Overview and Status Data Acquisition and Ingest Science Data Processing Level-2 Science Software Implementation Level-2 Data Product Format (separate presentation) Data Archive and Distribution Mission Simulation Aquarius Web Site What We Need Agenda

16 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Data Archive and Distribution Overview of the currently operational web-based browse, order and distribution tool that is available to support the distribution of Aquarius data within the Aquarius Data Processing System that also serves as a data quality monitoring tool for both internal and external use.

17 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Level-1-2 Data Browser

18 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Level-1-2 Data Browser

19 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop All Beams - daily Aquarius Level-3 Data Browser

20 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Overview and Status Data Acquisition and Ingest Science Data Processing Level-2 Science Software Implementation Level-2 Data Product Format (separate presentation) Data Archive and Distribution Simulations Aquarius Web Site What We Need Agenda

21 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Year-Long Simulation RSS and JPL have provided one-year (2007) simulated data sets for the radiometer and scatterometer. Aquarius GS has combined these to generate one year of simulated downlink files. One month of this data has already been processed to level-2 files: –ftp://samoa.gsfc.nasa.gov/aquarius-web/simdata/L2/2007/ Once the content is approved, the entire year will be processed and made available to the Science Team.

22 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Mission Simulation The Aquarius Mission Simulation will be a running, day-by-day exercise of the Aquarius/CONAE MOC interface and the ADPS. The simulation will follow this daily sequence of events: –The Radiometer (RSS) and Scatterometer (JPL) algorithm teams will deliver a day’s worth of simulated instrument data to the Aquarius GS. –The GS will generate simulated downlink (HRD and HKT) files for a typical pass schedule and a simulated daily ephemeris file. –The simulated files will be staged on the CONAE servers. –The ADPS will automatically detect, acquire and ingest the files, and process them through all product levels. –The products will be archived, cataloged, and made available for distribution through the Browse and Order interface and to the PO.DAAC. –The Aquarius Science Team will use this simulation to test their tools, data quality monitoring processes and anomaly detection. The Mission Simulation will commence ~May 2010 and will continue until the Aquarius/SAC-D launch.

23 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Simulated Data Set The Mission Simulation starts with simulated sensor data generated by members of the Aquarius Science Team, using assumed salinity fields and spacecraft orbit data, actual ancillary data and models of the instrument response

24 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Simulated Data Set Once the simulated sensor files are created, these are formatted into Aquarius science blocks and packaged into realistic downlink files based on predicted contact times and durations which are as close to the actual downlink data format as possible given the currently available information and sample telemetry

25 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Overview and Status Data Acquisition and Ingest Science Data Processing Level-2 Science Software Implementation Level-2 Data Product Format (separate presentation) Data Archive and Distribution Mission Simulation Aquarius Web Site What We Need Agenda

26 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Website Consolidated Data access, Information, Services and Community Feedback

27 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Wiki Online Documentation and Software Descriptions available online at: Under Data and Documents Link labeled “Aquarius Ground System and Mission Operations Wiki Documentation” Major WIKI documentation update planned in March/April 2010

28 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Overview and Status Data Acquisition and Ingest Science Data Processing Level-2 Science Software Implementation Level-2 Data Product Format (separate presentation) Data Archive and Distribution Mission Simulation Aquarius Web Site What We Need Agenda

29 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop What We Need Approval of the current Level-1A to 2 software implementation to support the year-long and mission simulations. Agreement on the time frame to start the mission simulation. Agreement on the ancillary data sets and sources. Agreement on the Level-2 parameters for the simulations. Schedule and responsibility for algorithm and software updates between now and launch.

30 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Backup Slides

31 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Software Status For each software module, we will show: High-level flow (inputs and outputs). Current status and capabilities. Remaining work to be done. Input File Intermediate Data Output File Processing Module

32 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level-0 Preprocessor Level-0 Downlink File l0gen Level-0 File Implemented by Joel Gales Current Capabilities: 1.Processes a Level-0 downlink file at CONAE to remove duplicate blocks and sort by time. To be done: 1.Final data quality checks.

33 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level-0 to 1A SAC-D Telemetry SAC-D Ephemeris Data Level-0 File l1agen Level-1A Files Implemented by Joel Gales Current Capabilities: 1.Processes a Level-0 multi-orbit file to multiple Level-1A orbit files with overlap. 2.Inputs SAC-D ephemeris and telemetry from separate files. 3.Unpacks Aquarius science data. 4.Converts Aquarius telemetry to engineering units. 5.Generates HDF5 L1A files. To be done: 1.Implement actual SAC-D telemetry input file formats when these are known.

34 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level-1A Merge l1amerge Level-1A Product Implemented by Joel Gales Current Capabilities: 1.Merges multiple Level-1A files for the same orbit from successive downlinks. To be done: 1.Data quality checks to keep only “best” data. Level-1A Files

35 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level-2 to 3 Binning Level-2 Products l2bin Level-3 Binned Beam Product Implemented by Joel Gales based on current OBPG binning software Current Capabilities: 1.Processes multiple Level-2 products (e.g., for 1 week). 2.Average samples for each beam in 1 degree bins. 3.Outputs a Level-3 binned product per beam. To be done: 1.Add flag exclusion rules.

36 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level-2 to 3 Smoothing Level-2 Products l2smooth Implemented by Joel Gales based on algorithm from Jonathan Lilly Current Capabilities: 1.Processes multiple Level-2 products (e.g., for 1 week). 2.Generates smoothed values in 1 degree bins. 3.Outputs a Level-3 smoothed product. To be done: 1.Additional smoothing algorithms or options. 2.Add flag exclusion rules. Level-3 Smoothed Product

37 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level-3 Binning Level-3 Binned Products l3bin Level-3 Binned Product Implemented by Joel Gales based on current OBPG binning software Current Capabilities: 1.Processes multiple Level-3 binned products. 2.Averages samples in common bins. 3.Outputs a single Level-3 binned product.

38 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Level-3 Mapping Level-3 Binned Product l3bin Level-3 Mapped Product Implemented by Joel Gales based on current OBPG binning software Current Capabilities: 1.Reads a Level-3 binned product. 2.Outputs a Level-3 mapped (lat- lon gridded) product.

39 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Ground System Interfaces ADPSACCS Aquarius DataCommand & Processing ControlSystem CONAE DataMission RF Archive Operations Services Command Center Aquarius Command Schedules And Reports S-Band X-Band Wallops Svalbard Alaska McMurdo NASA Ground Network Science Team and Algorithm Providers S-Band Algorithms & Science Code Evaluation Products Ancillary Data Sources Commands Real-Time Telemetry & Coordination Schedule Confirmation And Reports AVDS Aquarius Validation Data System PO.DAAC Level 1, 2, 3 Aquarius Data Data Users and Science Team Aquarius L1, 2, 3 Data Ancillary Data Feedback SeaDAS Software X-Band ASI Matera Cordoba Principal Investigator and Science Planning Team Calibration & Instrument Configuration Change Requests Instrument Engineering Team Aquarius Instrument Testbed Flight Software Updates ASI Malindi 1- SAC-D Ground System to Aquarius Ground System ICD 2- SAC-D / NASA PSLA, Network Requirements Document & Scheduling Interface 3- SD - Observatory Ground IF Specification Document 4- Aquarius Science Team to Aquarius Ground System ICD 5- AQ - Aquarius Ground System – Aquarius Instrument Team Anomaly ICD and Aquarius Test bed User Manual 6- Aquarius Data Product User Guide (PO.DAAC) & Format Specification (ADPS) 7- Ocean Data Processing System Data and Software Management Plan, ATBD’s and Software User’s Guides 8- A V D S to A D P S ICD 9- Aquarius Ground System to PO.DAAC ICD Insitu Data Bias Adjustments Aquarius Match-up Data Aquarius Data, SAC-D Telemetry and Orbit Products CUSS CODS

40 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop OBPG Computing Facility NASA Goddard Space Flight Center Greenbelt, Maryland Server location: Building 28 Room W square feet dedicated to the project - Raised computer floor with under-floor A/C - Primary and backup A/C units - 80 KVA Powerware UPS unit with dual 50 KVA PDUs, additional 5 KVA rack-mounted UPS units - Connections to GSFC high speed networks: SEN

41 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop OBPG System Hardware Processing Cluster 46 processing nodes 15 TB 4 storage nodes 10.9 TB Ingest Servers 2 SeaSpace workstations 78 GB 1 storage node 5.1 TB Distribution Servers (ftp) 8 storage nodes 49.3 TB 1 server 250GB Distribution and Storage Servers (web) Production: 3 processing nodes 75 storage nodes 1.03 PB Testing Cluster Network Support Servers Backup Servers 2 storage nodes 19.2 TB Extreme Networks Black Diamond 8810 Gigabit Ethernet switch 480 ports Cal/Val & QC Systems Mission Operations Systems Development Servers 2 superstore nodes 10 TB User Desktops Aquarius Telemetry Viewer 2 Windows PC Aquarius Command Planning Tool Servers 2 processing nodes 586 GB Subversion CM Servers 2 superstore nodes 586 GB Forum Servers (web) Production: 2 processing nodes Database Servers 2 superstore nodes 16.4 TB AQOPS Servers (web) Production: 2 processing nodes

42 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Aquarius Backup Strategy Data On-site Downlinks, L0, L1a data files primary copy on RAID6 storage node mirror copy on separate RAID6 storage node, at file creation Science Products primary copy on RAID6 storage node mirror copy if desired, or regenerated from L1a Off-site Downlinks: stored at CONAE L1A archived at PO.DAAC Source Code On-site primary copy on subversion server, RAID6 mirror copy on backup Subversion server, RAID6, nightly mirror Off-site Subversion data copied to external disk, taken off-site every 1 month Multiple copies, given to Project Manager, SE, Processing Manager Other On-site AQOPS, Forum: database mirrored to backup server nightly Telemetry Viewer, other: image backups periodically, manual Off-site AQOPS, Forum: databases included in source code off-line backups Telemetry Viewer, other: image backups stored off-line like source code

43 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop IT Security Plan ADPS and ACCS: covered under the same plan as other Ocean Biology Processing Group (OBPG) systems. NASA IT Security Plan Identification –System name: Sciences and Exploration Directorate (SED) Multi- Program/Project (MPP) IT Science Systems –System Abbreviation: SED MPP –System Unique Identifier: CD-014-L-GSF-6004 –Information System Type: General Support System –Information System Security Categorization: Low (transitioning to medium within the year) –Information System Operational Status: Operational Effective 5/31/2007. Passed Certification & Accreditation in Summer in process of being upgraded Plan designated Sensitive But Unclassified by NASA Implementation questions: addressed off-line

44 of 29 March 9-11, 2010 ADPS Overview and Status Aquarius Algorithm Workshop Ground System Staffing NASA Management - Gene Feldman: Oversight of the overall effort, communication/coordination with other NASA organizations and centers and CONAE; oversight of science support, algorithm integration, QC, validation and in situ data support. Contractor Management - Fred Patt: Oversight of contract support staff, including hiring, tasking, review of deliverables and schedules; ADPS interface definition and product format specification. Mission Operations - Susie Kennison: Technical lead for instrument operations and commanding support; interface to instrument engineers and CONAE mission operations. Data Systems Engineering - John Wilding: Technical lead for the data system integration effort; overall system design, implementation and testing. Software Engineering – Joel Gales: Technical lead for the level conversion / product generation software, including development of the Level 0-to-1 software and integration/packaging of the science (Level 2 and 3) processing software to be provided by the algorithm developers. Data Systems Support - Dan Knowles: Integrate the product generation software into the existing automated processing system, develop ingest modules, develop database tables for archive products, develop schedules and recipes for data acquisition, production and migration. Systems Administration - Paul Smith: Specify and acquire new processing and data storage hardware, integrate into existing data processing facility, verify/upgrade network capability for external data transfers. Quality Control and Validation - Liang Hong: Working with the Science Team, develop methodologies, tools, procedures and reports for operational QC; specify data and develop algorithms and tools for data and science algorithm validation. In Situ Data Support - Jeremy Werdell: Work closely with the Aquarius Validation Data System (AVDS) to develop methods and tools for data analysis and quality control, building upon the existing SeaWiFS Bio-optical Archive and Storage System (SeaBASS). Archive and Distribution - Norman Kuring: Incorporate Aquarius product browse, search, order and distribution into existing web- based Ocean Color capability and work with JPL PO.DAAC regarding “institutional” archive support requirements.