1. Edward Armstrong, Jorge Vazquez, Andrew Bingham, Thomas Huang, Chris Finch, Charles Thompson, Tim McKnight, and Cynthia Chen JPL PO.DAAC / GDAC California Institute of Technology 11 th GHRSST Science Team Meeting Lima, Peru 21 June 2010

2. Organization Introduction PO.DAAC Data Management and Archiving System (DMAS) Operational issues New products Usage statistics Metadata and discovery Tools and Services development Summary

3. Introduction The Global Data Assembly Center (GDAC) Ingest and maintain interfaces to GHRSST L2P/L3P/L4 datastreams from 12 RDACS. ~30 G/day > 6000 files/day Realtime quality checking for data and metadata granules Distribution via FTP/OPeNDAP/POET Create FGDC metadata for daily collection granules Maintain interfaces to the LTSRF for GHRSST data and metadata exchange GDAC web portal Vet new GHRSST products Maintain database for GHRSST metadata User and application support MODIS L2P production in collaboration with the OBPG and RSMAS Ancillary L2P filling

4. PO.DAAC Data Management and Archive System (DMAS) New PO.DAAC wide data management system. Introduced last science team meeting All GHRSST datastreams are “passing” through the system transparent to the data producer Improvements in scalability, throughput, load balancing, discovery, database storage, product validation, archive capabilities, metadata ingestion, and metrics capturing A “parallel” GHRSST data directory exists on PO.DAAC FTP server (GHRSST2/) Some end-to-end testing and configuration needs to be finalized with the Longterm Stewardship and Reanalysis Facility

5. 22 June 2010 DMAS Top Level Architecture Service-oriented system Common metadata data model Metadata extraction and translation Data discovery Parallel and load-balanced data ingestion Dynamic deployment of ingest engines Storage management Validation State-driven Secure Portable Scalable

6. 22 June 2010 DMAS data handler for GHRSST data Adaptation MMR validation and translation Data file validation Scans local/remote locations for new data Integration with back-end RDAC cluster Inventory Full migration from existing MySQL database to Oracle Port to use the new data model FGDC and Index generators Website

7. 22 June 2010 Additional improvements Ancillary L2P filling and MODIS L2P production transitioning from Linux cluster to high performance servers Better performance (improved I/O) Additional capability for more L2P datastreams (e.g., MODIS) Aerosol Optical Depth ancillary field is again being filled with FNMOC model data

8. Operations GDAC uptime No significant outages Our systems are on backup power and backup battery Latency Most customer are happy Coordination with LTSRF Close communication maintained between operational centers Multiple versions of same L2P files from REMSS

9. New GHRSST products Level 4 MUR NCAMERICA UHfnd RTO NCAMERICA UHfnd G1SST GLOB UHfnd Level 2P MetOP AVHRR MTSAT-1R Level 3 MetOP AVHRR Product table URL http://ghrsst.jpl.nasa.gov/GHRSST_product_table.html

10. GDAC Distribution Statistics Number of distinct monthly users through May 2010 FTP/OPeNDAP/POET

11. Distribution Statistics Number of files distributed

12. Distribution Statistics Volume distribution

13. Usage

14. Product distribution – L2P Product NameUsers (hosts)GBFiles GHRSST Level 2P USA NASA MODIS Aqua 11 micron SST657400.02212395 GHRSST Level 2P USA NASA MODIS Terra 11 micron SST445969.42045778 GHRSST Level 2P USA NAVOCEANO AVHRR18 Local SST785742.3565162 GHRSST Level 2P USA NOAA GOES11 SST312389.5212762 GHRSST Level 2P USA NAVOCEANO AVHRR18 Global SST822137.1202610 GHRSST Level 2P USA NAVOCEANO AVHRR17 Local SST221899.2151993 GHRSST Level 2P USA NAVOCEANO AVHRR17 Global SST931741.2188103 GHRSST Level 2P USA NOAA GOES12 SST22943.1102840 GHRSST Level 2P USA NAVOCEANO METOP-A Global SST41694.753103 GHRSST Level 2P USA Remote Sensing Systems AMSRE SST154475.3267678 GHRSST Level 2P RAL AATSR SST52400.2125931 GHRSST Level 2P NEODAAS AVHRR17 Local SST10132.47652 GHRSST Level 2P NEODAAS AVHRR18 Local SST1781.07210 GHRSST Level 2P USA Remote Sensing Systems TMI SST8375.2228468 GHRSST Level 2P European Medspiration NAR18 SST1136.65023 GHRSST Level 2P European Medspiration NAR17 SST817.52720 GHRSST Level 2P USA Remote Sensing Systems AMSRE SST5113.527012 GHRSST Level 2P European Medspiration SEVIRI SST3710.719602 GHRSST Level 2P USA Remote Sensing Systems TMI SST319.122929 GHRSST Level 2P European Medspiration AMSRE SST130.813190 GHRSST Level 2P European Medspiration AVHRR17 Global SST30.12629 GHRSST Level 2P European Medspiration AVHRR17 Local SST10.0482 GHRSST Level 2P European Medspiration TMI SST10.01

15. Product distribution – L4 Product NameUsers (hosts)GBFiles GHRSST Level 4 UKMO OSTIA Global SST466121.929967 GHRSST Level 4 USA NCDC AVHRR_OI SST127260.53214692 GHRSST Level 4 USA AVHRR_AMSR_OI SST9128.8114943 GHRSST Level 4 USA NAVOCEANO K10_SST Global SST706.87834 GHRSST Level 4 European Medspiration ODYSSEA Global SST6016.39375 GHRSST Level 4 USA Remote Sensing Systems mw_ir_OI Global SST4220.621885 GHRSST Level 4 Australian Bureau of Meteorology RAMSSA_09km Australia SST422.16368 GHRSST Level 4 Denmark DMI North Sea Baltic SST401.613060 GHRSST Level 4 Australian BOM Global SST270.72969 GHRSST Level 4 European Medspiration ODYSSEA Mediterranean SST181.84354 GHRSST Level 4 European Medspiration ODYSSEA NW Europe SST180.52731 GHRSST Level 4 European Medspiration ODYSSEA Galapagos SST1132455

16. Metadata – XML TAG XML TAG activities: Met for 4 telecons to discuss the adaption of ISO 19115 to GHRSST metadata Produced the GDS 2.0 metadata specification with at least two major revisions since the last meeting Primarily driven by JPL, NOAA NGDC and NODC Presented on file format and metadata issues (Climate Forecast and ISO 19115) at 2010 NASA DAAC managers. Has now been assimilated into the DAS-TAG

17. Metadata – Discovery As part of DMAS capabilities: GHRSST granules (L2P/L3/L4) now exported to NASA ECHO (EOS ClearingHOuse). Discoverable via WIST tool: the ECHO search interface. GHRSST has its own topic keyword. Currently 300K+ GHRSST granules in ECHO; 32 GHRSST datasets DIF records to be exported to NASA Global Change Master Directory (GCMD): in progress Metric harvesting of usage statistics to be implemented on a routine basis

18. New PO.DAAC webpages

19. Tools and Services NAIAD and Data Miner for L2P subsetting and visualization Collaboration with IFREMER Replaces the HITIDE concept introduced last year Public since May 2010

20. Tools and Services Google Earth interface to subsetting, viewing and data exploration

21. Summary GDAC meeting all its fundamental operational and distribution requirements All GHRSST datastreams are now handled by DMAS DMAS is scalable, portable and has made it easier to ingest, track and manage an expanding GHRSST catalog New infrastructure committed to L2P ancillary filling and MODIS L2P production Improved tools, services, discovery L2 subsetting and visualization with Data Miner and Google Earth interface New PO.DAAC web pages include all GHRSST products Discovery via the NASA ECHO and GCMD DAS TAG ISO metadata development GDS implementation New UHR L4 products !