Page 1 Land PEATE support for CERES processing Ed Masuoka Gang Ye August 26, 2008 CERES Delta Design Review

Page 2 Element Overview Facilitate VIIRS Science Team assessment of Land EDRs for use as Climate Data Records Facilitate evaluation of improved Land EDR algorithms provided by Science Team Support NICSE Cal/Val activities through processing SDRs as requested Provide access to data products and results of product quality assessment to VIIRS Science Team Support CERES Instrument Team by providing CERES RDRs and VIIRS SDR subsets and VIIRS Aerosol EDRs in MODIS format (5 minutes granules in HDF-EOS)

Page 3 Overview: SDR/EDR evaluation Land PEATE is sized to produce VIIRS SDR and EDRs at 3x post-launch for two versions of code (operational algorithms and improved algorithms) and to produce global gridded products from VIIRS EDRs for comparison with MODIS. Products run with specific versions of software are maintained in separate archive sets Product retention is governed by online storage Improvements to algorithms running in IDPS will be provided by the VIIRS science team. Land Q/A team will post results of their analyses of product quality at:

Page 4 Science and Calibration Team Science TeamProduct R. WolfeGeolocation J. Xiong (NICSE), E. VermoteCalibrated Radiances E. Vermote, A. LyapustinLand Surface Reflectance C. SchaffAlbedo Z. Wan, S. HookLand Surface Temperature D. Hall, J. MaslanikSnow Cover M. FriedlSurface Type I. Csiszar, C. JusticeActive Fires, (Burned Area) D. Hall, J. MaslanikIce Surface Temperature A. Huete, R. MyneniVegetation Indices, (LAI)

Page 5 Land PEATE Test Result Results from evaluation of each test by LDOPE are provided Links to following are included: –Global browse –Issues uncovered in the data products –Time series plots

Page 6 Land PEATE – VIIRS systems Shared resource with MODIS

Page 7 CERES resources in Land PEATE 12 Dual quad-core compute servers for product generation 1 Dual core server and 19 TB of disk for product storage Supports 1x forward processing plus reprocessing for SDR generation and subsetting and Aerosol EDR generation

Page 8 Overview: CERES support Land PEATE provides CERES RDRs and VIIRS inputs to CERES processing in MODIS format on PDR server at GSFC using the SIPS Interface Land PEATE runs VIIRS SDR subsetting software developed by CERES to produce subsets with desired bands Land PEATE runs Aerosol PGE (IDPS or Science Team improved) to produce VIIRS Aerosol in MODIS format Land PEATE will reprocess the VIIRS input products as requested by CERES to support of the generation of a long-term climate record using hardware acquired for CERES support.

Page 9 Operations Concept for CERES support IDPS creates CERES Raw Data Records (RDRs) and provides RDRs to the SDS SD3E and to ADS/CLASS Land PEATE pulls CERES RDRs and VIIRS xDRs from SD3E Land PEATE places CERES RDRs on GSFC PDR server Land PEATE runs sub-sampler on SDR and places output on GSFC PDR server Land PEATE places Aerosol EDR in MODIS format (5 minute granules HDF-EDOS) on GSFC PDR server Land PEATE handles clean-up of files in CERES directory on PDR server based on PANs provided by ASDC as defined in the SIPS Interface Land PEATE will reorder products for CERES from CLASS or remake products upon request

Page 10 Requirements for CERES support Land PEATE shall ingest CERES RDRs from the SD3E Land PEATE shall integrate, test and run in production VIIRS SDR subsetting software developed by CERES IT Land PEATE shall produce VIIRS Aerosol product in MODIS format (HDF-EOS, 5 minute granules) Land PEATE shall transfer CERES RDRs, VIIRS SDR subsets and VIIRS Aerosol EDR to ASDC via PDR server Land PEATE shall meet latency requirements in CERES Science Data Products Working Agreement for product delivery Land PEATE shall reorder or remake CERES RDRs, VIIRS SDR subsets or Aerosol EDR upon request by CERES Land PEATE shall reprocess VIIRS inputs for CERES Climate Data Record upon request by CERES IT with versions of VIIRS software selected by CERES IT.

Page 11 CERES Operational Data Flow SD3EL-PEATE ADS IDPS SDS All RDRs All xDRs CERES RDRs, VIIRS SDR sub- sampled, VIIRS Aerosol EDR in MODIS format C3S Commands, Loads and requests RDRs SDRs EDRs IPs CERES Instrument Ops Team Svaalbard Cmd & Tlm Mission Data CERES Data Data Users Mission Notices and Data requests RDRs for LEO&A CERES PEATE Data and Science Operations Mission Notices and Data requests Ancillary data Data Providers NPOESS Systems NPP Systems NOAA Systems CERES Systems GSFC PDR server MODIS System

Page 12 MODAPS/LPEATE-ASDC Interface SIPS Interface –ASDC will implement EOSDIS SIPS PDR-PAN interface in their new system (ANGe). –Products will be placed in a disk partition reserved for NPP CERES on the MODIS PDR server located at GSFC –ASDC will transfer products from this directory using SIPS protocol.

Page 13 Operations Concept for CERES: Reprocessing for Climate-quality During 2 years of VIIRS post-launch product evaluation, algorithm improvements will be developed by the VIIRS Science Team members and evaluated by NVCSE (VIIRS Calibration Support Element), LDOPE (Land Q/A team) and VIIRS Science Team. Results of these evaluations will be available at LDOPE, NVCSE and ST web sites. CERES Science Team will determine versions of SDR and Aerosol software and calibration tables* to use in reprocessing on Land PEATE resources dedicated for CERES support. –Reprocessing will proceed from earliest well calibrated scene to present using best available calibration. * NVCSE is tasked only with evaluation of SDR, so complete calibration tables to ensure a climate-quality SDR may not be available.

Page 14 Status of PEATE development Proxy data generator (extended SDRgen) running in Land PEATE produces VIIRS SDR (MODIS bands with VIIRS geometry).* 16-days of proxy SDRs has been produced to test code integrated from IDPS Build –Cloud Mask, Active Fires, Aerosol Optical Thickness, Surface Reflectance and Vegetation Indices –Geolocation problem in Build fixed in current delivery –Aerosol calculations use 6x6 elements not 8x8 Additional days of proxy SDRs will be generated for CERES in October * CERES requested simulated day/night band be added and that modification will complete testing by 10/1/08

Page 15 CERES FM-5 Delta Design Review 15 Schedule for CERES support 8/20/08: CERES compute servers received - COMPLETED 9/30/08: CERES disk storage received 10/1/08: SDRGEN modified to add simulated day/night band 10/1/08: VIIRS sub-sampling software complete and tested at LaRC 10/3/08: VIIRS sub-sampling software delivered to Land PEATE for integration by CERES 11/5/08: VIIRS sub-sampling software completes PEATE integration tests 11/27/08: FMD connects new power for SDS computing facility 12/1/08: VIIRS sub-sampling software running at SDS Land PEATE, expected sub-sampled output has been verified by CERES DMT 1/2/09: CERES processors and disk storage installed at Land PEATE. Dependency on new power in November. 1/15/09: Large set of VIIRS sub-sampled output produced at SDS Land PEATE and verified at LaRC by CERES DMT

Page 16 Open Items Sample of CERES RDRs are needed to work ingest of these files into Land PEATE for delivery to CERES. When? Aerosol using 8x8 calculation (in Build 1.5.?). SIPS-interface/PDR server tests with ASDC does not involve Land PEATE delivery to ASDC system but rather ASDC pulling files from GSFC PDR server per working agreement. Define how product re-order requests will come from CERES ( message with or w/o an attached file, or a file pushed to an ftp directory). Define content of message or file. NVCSE is not responsible for ensuring the calibration tables are adjusted to maintain a climate quality NPP VIIRS record

Page 17 CERES FM-5 Delta Design Review 17 SDS/Land PEATE Assumptions Subscribe to SD3E for all CERES RDRs (Science, Diagnostic, and Housekeeping/Telemetry) Acquire CERES RDRs and forward to ASDC via existing Land PEATE interface Support CERES reordering of missing data Produce aggregated VIIRS SDRs (MODIS HDF-EOS format) in 5 minute granules (VIAE, VMAE, VDAE). Sub-sample and ship 5 minute granules (VIMD) to ASDC Produce VIIRS aerosol in 5 minute granules (VAOT) and ship to ASDC Post-launch: When VIIRS calibration and/or algorithm changes, reprocess VIIRS from covers- open forward and ship to ASDC to support CDRs