1. GOES-R AWG Product Validation Tool Development Downward SW Radiation at Surface and Reflected SW Radiation at TOA Hongqing Liu (Dell) Istvan Laszlo (STAR) Hye-Yun Kim (IMSG) Rachel Pinker (UMD) Ells Dutton & John Augustine (ESRL) 1

2. 2 OUTLINE Products Validation Strategies Examples Ideas for Further Enhancement and Utility of Validation Tools Summary

3. Products Shortwave Radiation Products: –Downward Shortwave Radiation at Surface (DSR) CONUS: 25km/60min Full Disk: 50km/60min Mesoscale: 5km/60min –Reflected Shortwave Radiation at TOA (RSR) CONUS: 25km/60min Full Disk: 25km/60min Only daytime 3

4. Monitoring & Validation Background Functions of tools: –routine monitoring (may not need reference data) –routine validation (reference data, matchup procedure) –deep-dive validation (reference data, other correlative data, matchup) Basic elements: –data acquisition (ABI, ground, other sat products) (Fortran 90) –spatial and temporal matching (lots of possibilities) (Fortran 90) –analysis (computing statistics) (IDL) Metadata Accuracy/Precision RMSE Minimum/Maximum Error –present results (display maps, scatter plots, tables) (IDL) 4

5. Validation Strategies 5 Satellite Measurements –Clouds and the Earth’s Radiant Energy System (CERES) Cloud and Radiative Swath (CRS) dataset: (1) measured TOA upward SW flux, (2) calculated Surface and Atmospheric Radiation Budget (SARB). http://eosweb.larc.nasa.gov/PRODOCS/ceres /level2_crs_table.html http://eosweb.larc.nasa.gov/PRODOCS/ceres /level2_crs_table.html Reference (“truth”) data Collocation of ABI retrievals and reference data is performed at the instantaneous time scale. Matching: ABI retrievals averaged spatially; ground measurements averaged temporally. Averaging window size is flexible. Independent satellite retrieval (CERES) Collocation: CERES data are averaged to the ABI retrieval grid on a daily basis. Matching: current retrievals use MODIS data as input; CERES is on same platform; no need for temporal matching. Reference Dataset Collocation/Match-up Ground Measurements –High-quality routine ground radiation measurements over Western Hemisphere from 20 stations from SURFRAD (ftp://ftp.srrb.noaa.gov/pub/data/surfrad/) and BSRN (ftp://ftp.bsrn.awi.de/) networks.ftp://ftp.srrb.noaa.gov/pub/data/surfrad/ftp://ftp.bsrn.awi.de/

6. Routine Validation Tools Instantaneous Monitoring Present retrieval results –Specify date & load data –Selection from ‘Variable’ menu Primary Outputs (image) –DSR –RSR Diagnostic Outputs (image) –Surface diffuse flux –Surface albedo –Clear-sky composite albedo –Clear-sky aerosol optical depth –Water cloud optical depth –Ice cloud optical depth Quality Flags (image) –66 flags (inputs, retrieval, diagnostics) Metadata (ascii file output) Independent of validation truth; can be executed automatically by scripts once retrievals are available. 6

7. Routine Validation Tools Validation with Ground “Truth” Validates DSR&RSR for a period of time –Specify time period & load data –‘Validation’ menu Generate scatter plot of retrievals against measurements Generate validation statistics and output to ascii file –‘TimeSeries’ menu Generate time series plots of retrieval and measurements over ground stations 7

8. 8 ”Deep-Dive” Validation Tools Validation with CERES An expansion of routine validation with CERES including cross validation against NASA SARB satellite products –Options: Scene types –all; snow ; clear ; water cloud; ice cloud Retrieval path –Hybrid path –Direct path only –Indirect path only –TOA matching (all; succeed; failed) –Surface albedo (all; succeed; failed) –Specify date & load data –Selection from ‘Validation’ menu Reflected SW Radiation at TOA (RSR) Retrieval; Retrieval-CERES; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics (Scatter plot; Statistics in ascii file) Downward SW Radiation at Surface (DSR) Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics Absorbed SW Radiation at Surface (ASR) Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics Absorbed SW Radiation in Atmosphere (ABS) Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics Surface SW Albedo (ALB) Retrieval; Retrieval-SARB Tuned; Retrieval-SARB Untuned; Statistics

9. Calculate and display –additional statistics (histograms) –temporal averages on different scales (daily, weekly, monthly) Identify signatures by which even non-experts can identify potential problems – needed for routine operational monitoring Implement automatic detection of possible systematic drift or continuous abnormal retrieval in routine validation. –establish “reference” (expected) statistics from good data –compare time series of actual statistics with reference stats –trigger action (e.g., sending warning email) when actual stats exceed reference stats + x std. Combine SW validation with LW radiation retrievals –check consistency e.g., high RSR low OLR is expected for cloudy scenes –additional diagnostic information for deep-dive validation (LW radiation) Current tool uses retrievals from MODIS proxy data. Adjustment to tools for retrievals from geostationary orbit will be needed (data preparation). 9 Ideas for Further Enhancement and Utility of Validation Tools

10. 10 Summary Current tools perform three functions: –routine monitoring of product –routine validation with reference data –deep-dive validation with reference and intermediate data Validation truth data have been identified and processed Planned enhancements include: –more stats –automatic detection of problems –checking consistency with LW

11. GOES-R AWG Product Validation Tool Development Upward LW Radiation at TOA Upward and Downward LW Radiation at Surface Hai-Tien Lee (CICS/UMD) Istvan Laszlo (STAR/NESDIS) Ells Dutton & John Augustine (ESRL) 11 Acknowledgments : NOAA SURFRAD, NASA CERES, BSRN, DOE ARM, Eumetsat GERB & LSA-SAF GOESR AWG Annual Meeting, June 14-16, 2011, Fort Collins, CO

12. Products Longwave Radiation Products: –Upward LW Radiation at TOA (OLR) CONUS: 25km/60min Full Disk: 25km/60min –Downward LW Radiation at Surface (DLR): Clear sky only CONUS: 25km/60min Full Disk: 25km/60min –Upward LW Radiation at Surface (ULR): Clear sky only CONUS: 25km/60min Full Disk: 25km/60min 12 GOES12 Imager OLR

13. Validation Strategies Reference Dataset (Ground) Ground Measurements –High-quality routine ground radiation measurements over Western Hemisphere used for validating ABI Longwave Radiation retrievals are collected from 7 stations from SURFRAD network. –Selected stations of BSRN and Eumetsat LSA SAF that provide surface upward and downward longwave radiation measurements can be used for offline/framework algorithm evaluation. 13 StationNetworkLongitudeLatitudeElevation[m]Measurements Used fpk SURFRAD -105.1048.31634 surface LW downward, upward fluxes; clear sky index; interpolated meteorological profiles sxf-96.6243.73473 psu-77.9340.72376 tbl-105.2440.131689 bon-88.3740.05213 dra-116.0236.631007 gwn-89.8734.2598

14. Validation Strategies Reference Dataset (Satellite) Satellite Measurements –OLR product from Clouds and the Earth’s Radiant Energy System (CERES) Single Scanner Footprint (SSF) datasets are used as algorithm validation reference. –Future NPP and JPSS OLR (from CERES FM5/6) can be used for routine monitoring and evaluation (possibly with lag). –Operational HIRS OLR from NOAA and MetOp polar orbiters will be used as a backup for routine monitoring purpose. 14

15. Tools: –IDL (primarily) Data Collocation Instantaneous Monitoring Validation over Ground Stations Validation with CERES Deep-dive Validation over Ground Stations Deep-dive Validation with CERES Statistics: –Metadata (ATBD), plus Mean/StDev for Global, zonal and selected domains of interests –Mean, StdDev, RMS, Min and Max of Errors Visualization: –IDL, GrADs –Figures rendered in PNG format 15 Validation Strategies Tools, Statistics & Visualization

16. 16 Validation Strategies Example of Deep-Dive FM1 FM2 FM3 FM4 OLR Error vs LZA OLR Error vs SEVIRI Ch 5 radiance (UTH) OLR Error vs SEVIRI Ch 9 radiance (window) OLR Error as a function of Ch 7 and Ch 9 radiances Extended OLR Validation (March 2004)

17. 17 Summary Validation truth data have been identified and being acquired Validation tools are designed to perform: –Routine monitoring of product –Routine validation with reference data –Deep-dive validation with reference and auxiliary data Planned enhancements include: –Temporal tracking of stats –Define level of alarms (for routine monitoring) –Explore possible sources of more ground truth –Clear-sky identification with auxiliary data.

18. END 18

19. Monitoring & Validation Background Functions of tools: –Routine monitoring (may not need reference data) –Routine validation (reference data, matchup procedure) –Deep-dive validation (reference data, other correlative data, matchup) Basic elements: –Data acquisition (ABI, ground & satellite products) (Unix Script, IDL) –Spatial and temporal matching (closed pixel vs area average) (IDL) –Analysis (computing statistics) (IDL, Datadesk) –Present results (display maps, scatter plots, tables) (IDL, GrADs, Kaleidagraph) Special considerations: –Degradation flag 19