1. 1 GOES-R AWG Product Validation Tool Development Land Baseline Products Land Surface Temperature and Fire Detection and Characterization Land Team Chair: Yunyue (Bob) Yu NOAA/NESDIS/STAR

2. 2 GOES-R AWG Product Validation Tool Development Land Surface Temperature Application Team Yunyue (Bob) Yu (STAR) Dan Tarpley (Short & Associates) Hui Xu, Xiao-long Wang (IMSG) Rob Hale (CIRA) Kostya Vinnikov (CICS)

3. 3 LST Products ●The ABI Land Surface Temperature (LST) algorithm generates the baseline products of land surface skin temperatures in three ABI scan modes: Full Disk, CONUS, Mesoscale.; ●Meets the GOES-R mission requirements specified for the LST product; ●Has a good heritage, will add to the LST climate data record; ●Simplicity for implementation/ease of maintenance, operational robustness, and potential for improvement. Full Disk CONUS

4. 4 Products ProductAccuracyPrecisionRangeRefresh RateResolution LST (CONUS)2.5 K2.3 K213 ~ 330 K60 min 2 km LST (Full Disk)2.5 K2.3 K213 ~ 333 K60 min 10 km* LST (Mesoscale)2.5 K2.3 K213 ~ 330 K60 min 2 km Product Temporal Coverage Product Extent Cloud Cover Conditions Product Statistics LST (CONUS)Day and NightLZA < 70 Clear Conditions associated with threshold accuracy Over specified geographic area LST (Full Disk)Day and NightLZA < 70 Clear Conditions associated with threshold accuracy Over specified geographic area LST (Mesoscale)Day and NightLZA < 70 Clear Conditions associated with threshold accuracy Over specified geographic area Specifications Qualifiers *Requirement Change Requested: to be 2 km.

5. Validation Strategies 5  Utilize existing ground station data »Stations under GOES-R Imager coverage »Stations under MSG/SEVIRI coverage  Ground site characterization  Stringent cloud filtering  Multiple comparisons: satellite vs satellite, satellite vs ground station.  Direct and indirect comparisons  International cooperation SURFRAD Sites

6. Validation Strategies  Development for routine validation tools »Characterizations of SURFRAD and CRN ground sites »Routinely acquired matchup data sets of satellite and ground Data »Ground LST estimation »Procedures for converting point ground LST to “pixel” ground LST »Direct comparisons and statistics for each ground LST vs satellite LST for last x months »Time series plots of selected coincident LST and ground LST for last x months 6  Development for deep dive validation tools »All of the routine validation tools + »Data sets consisting of multiple years of clear radiances coincident with ground LSTs »Indirect comparison and statistics for each ground LST vs ground LST climatology for last x years »Comparisons and statistics for GOES-R LST vs other satellite LST »Routines for calibrating LST algorithm coeffs using the validation results “Routine” Validation Tools Bulk/overview analysis Executed soon after product generation Run routinely Run within OSPO and STAR Automated “Deep Dive” Validation Tools Detailed/point analysis Not executed in real-time. May need to wait for other datasets Run when more detailed analysis of product performance is needed Run within STAR Automated and/or Interactive components

7. Validation Tools 7 Satellite Data Match-up Datasets Time Match-up Geolocation Match-up Ground Data Mask Satellite Cloud Mask Manual Cloud Control Ground LST Estimation/Extraction Satellite LST Calculation/Extractio n Synthetic Analysis and Correction Ground Data Ground Data Reader Satellite Data Reader Indirect Comparison Direct Comparison Statistical Analysis Outputs (Plots, Tables, etc.) Outputs (Plots, Tables, etc.) Components of Validation Tools

8. Routine Validation Tools -- SURFRAD data results 8 Comparison results of GOES-8 LST using six SURFRAD ground station data, in 2001.. Month Site 1Site 2Site 3Site 4Site 5Site 6 DayNightDayNightDayNightDayNightDayNightDayNight 148175564107 58481006710051 2433038 7357553441546448 300517194629379416312334 49418398081346261815713935 571222759127657583824516875 650268210282516550866418760 76491774082738404317374 84330129113413882137564510752 9115579510812449791101168518998 10103386295184399079745811561 1111434431291467064531165313188 12403867661247173701077411356 Total727314779100212236518238429407081609732 Numbers (Table, left) and scatter plots (right) of the match-up LSTs derived from GOES-8 Imager data vs. LSTs estimated from SURFRAD stations in year 2001. Data sets in plots are stratified for daytime (red) and night time (blue) atmospheric conditions

9. Routine Validation Tools -- A visualization interface 9

10. 10 T(x,y,t) T(x 0,y 0,t 0 ) ASTER pixelThe site pixelMODIS pixel  Quantitatively characterize the sub-pixel heterogeneity and evaluate whether a ground site is adequately representative for the satellite pixel. The sub-pixels may be generated from pixels of a higher-resolution satellite.  For pixel that is relatively homogeneous, analyze statistical relationship of the ground-site sub-pixel with the surrounding sub-pixels: {T(x,y) } ~ T(x 0,y 0 )  Establish relationship between the objective pixel and its sub-pixels (i.e., up-scaling model), e.g., T pixel = T(x,y) +  T (time dependent?) Site characterization analysis using ASTER data— an integrated approach for understanding site representativeness and for site- to-pixel model development Surface heterogeneity is shown in a 4km x 4km Google map (1km x 1km, in the center box) around the Bondville station area The Synthetic pixel/sub-pixel model Site MODIS Pixel- SURFRAD Synthetic Pixel- SURFRAD Nearest Aster Pixel – Synthetic pixel MeanStdDevMeanStdDevMeanStdDev Desert Rock, NV-0.441.842.091.690.040.44 Boulder,CO-0.492.081.492.90-0.090.67 Fort Peck, MT0.352.520.781.950.381.15 Bondville, IL-0.171.511.041.480.030.90 Penn State, PA-1.531.910.611.960.041.07 Site-to-Pixel Statistical Relationship for 5 SURFRAD sites ”Deep-Dive” Validation Tools

11. Others ? 11 ”Deep-Dive” Validation Tools Comparison of SEVIRI-Retrieved LST and station LST at Evora Apparent diurnal patterns are shown in the 10 days’ LST comparison profiles for selected months. Comparison of LST diurnal profiles revealed higher station LST than SEVIRI LST around mid-days (i.e. maximum daily LST) and slightly higher SEVIRI LST than station LST at night (with low LST). The diurnal differences are larger in warm months. We need to understand and fix this problem

12. EOGC 2009, May 25-29, 2009 12 Goodwin Creek, MS, observation pairs are about 510. View Zenith of GOES-8/-10: 42.68 0 /61.89 0 ”Deep-Dive” Validation Tools -- Directional effect study Due to the satellite LST directional properties (surface components, topography, shadowing etc.), the satellite LST can be significantly different from different view angles. Deep dive validation tools may be used for case studies and improved algorithms.

13. 13 Complementary Posters Validation of Land Surface Temperature Algorithm for U.S. GOES-R Mission Y.Yu, H. Xu, X-L. Wang, D. Tarpley, R. Hale Development of a Multi-Satellite Validation System for NOAA/GOES-R ABI Land Surface Products K. Gallo, G. Stensaas,G. Chander, Y. Yu, M. Goldberg,R. Hale, and D. Tarpley Impacts of Emissivities in the Retrieval of SEVIRI LST and the Calculation of LST from Surface Measurements H. Xu and Y. Yu Developing Tools for LST Validation and Deep-Dive Analysis X-L. Wang, Y. Yu, H. Xu, D. Tarpley, R. Hale Land Surface and Air Temperature (LST & SAT) at Clear and Overcast skies K. Y. Vinnikov, Y. Yu, M. D. Goldberg, D. Tarpley, M. Chen, C. N. Long

14. 14 GOES-R AWG Product Validation Tool Development Fire Detection and Characterization Application Team Christopher Schmidt (CIMSS) Ivan Csiszar (STAR) Wilfrid Schroeder (CICS/UMD)

15. Products Fire detection and characterization algorithm properties:  Refresh rate: 5 minute CONUS, 15 minute full disk  Resolution: 2 km  Coverage: CONUS, full disk  ABI version of the current GOES Wildfire Automated Biomass Burning Algorithm (WF_ABBA)  Product outputs: »Fire location »Fire instantaneous size, temperature, and radiative power »Metadata mask including information about opaque clouds, solar reflection block-out zones, unusable ecosystem types. 15

16. Products 16

17. Validation Strategies 17 FDCA Routine Validation Current practice for GOES WF_ABBA: No automated realtime method is available. Ground-based fire reports are incomplete and typically not available in realtime. At the Hazard Mapping System Human operators look at fire detections from various satellites and at satellite imagery to remove potential false alarms. This method is labor intensive and actual fire pixels are often removed.

18. Validation Strategies 18 FDCA Routine Validation ABI near realtime validation: Co-locate ABI fire pixels with other satellite data Ground-based datasets tend to be incomplete and not available in realtime Fire detections from other satellites (polar orbiting) can be used in near realtime Perfect agreement is not expected. Due to resolution, viewing angle, and sensor property differences a substantial number of valid fires will be seen by only one platform Other fire properties (instantaneous fire size, temperature, and radiative power) have no available near realtime validation source (see Deep-Dive tools) Important note: the product requirement does not align with user expectations. The requirement states: “2.0 K brightness temperature within dynamic range (275 K to 400 K)” This applies to a pixel brightness temperature, and the algorithm achieves it for 100% of the fires where fire characteristics are calculated. When used to recalculate the input brightness temperature the fire characteristics match the input data to better than 0.0001 K.

19. Validation Strategies 19 FDCA Validation Tools Routine validation tools: Perform co-locations for individual fires and for clusters of fires Provide statistics on matches Table on following slide shows example of routine statistics from model- generated proxy data cases. 75 MW of fire radiative power is the estimated threshold for fire detectability. Deep-Dive validation tools: Allow for validation of fire location and properties Utilize high-resolution data from satellite or aircraft to provide fire locations and enable estimates of fire size, temperature, and radiative power Can be partially automated, availability of high resolution data is limiting factor

20. Validation Strategies 20 CIRA Model Simulated Case Studies^ CIRA TruthABI WF_ABBA Total # of fire clust ers* Total # of ABI fire pixels* Total # of ABI fire pixels > FRP of 75 MW* Total # of detected clusters % Fire clusters detected* Total # of fire pixels detected > FRP of 75 MW* % Fire pixels detected > FRP of 75 MW* % False postives (compared to model truth, will not be available for routine validation) Kansas CFNOCLD 97206328852234964899.3%4748290.9%<1% Kansas VFNOCLD 57233691926600569599.5%55180.6%<1% Kansas CFCLD 91405655346446876895.9%3938084.8%<1% Cent. Amer. VFCLD 8492859166980895.2%142485.3%<1% Oct 23, 2007 California VFCLD 9904710238898999.9%209087.5%<1% Oct, 26 2007 California VFCLD 120522252120100%21183.7%<1% CFNOCLDConstant Fire No Cloud ^ Limit to ~ 400K minimum fire temperature VFNOCLDVariable Fire No Cloud CFCLDConstant Fire with Cloud * In clear sky regions, eliminating block-out zones VFCLDVariable Fire with Cloud

21.  Deep-dive fire detection and characterization validation tool builds on methods originally developed for MODIS and GOES Imager »Use of near-coincident (<15min) Landsat-class and airborne data to generate sub-pixel summary statistics of fire activity – Landsat-class data are used to assess fire detection performance l History of successful applications using ASTER, Landsat TM and ETM+ to estimate MODIS and GOES fire detection probabilities and commission error rates (false alarms). Methods published in seven peer reviewed journal articles l Limited fire characterization assessment (approximate fire size only). Frequent pixel saturation and lack of middle infrared band prevent assessment of ABI’s fire characterization parameters – Airborne sensors are used to assess fire characterization accuracy l High quality middle-infrared bands provide fine resolution data (<10m) with minimum saturation allowing full assessment of ABI’s fire characterization parameters (size, temperature, Fire Radiative Power) l Sampling is limited compared to Landsat-class data »Regional × hemispheric/global coverage »Targeting case-study analyses 21 ”Deep-Dive” Validation Tools

22.  Several national and international assets will be used to support ABI fire validation »USGS Landsat Data Continuity Mission (2013) »ESA Sentinel-2 (2013) »DLR BIROS (2013) »NASA HysPIRI (TBD ~2020) »Airborne platforms (NASA/Ames Autonomous Modular Sensor-Wildfire; USFS FireMapper)  Will perform continuous assimilation, processing and archival of reference fire data sets »Daily alerts targeting false alarms, omission of large fires – Main output: Quick looks (PNG) for visual inspection of problem areas showing ABI pixels overlaid on high resolution reference imagery »Probability of detection curves and commission error rates derived from several weeks/months of accumulated validation data – Main output: Tabular (ASCII) data containing pixel-based validation summary (graphic output optional) 22 ”Deep-Dive” Validation Tools

23. 23 ”Deep-Dive” Validation Tools Sample visual output of simulated ABI fire product (grid  2km ABI pixel footprints) overlaid on ASTER 30m resolution RGB (bands 8-3-1). Red grid cells indicate ABI fire detection pixels; green on background image corresponds to vegetation; bright red is indicative of surface fire ASTER binary (fire – no fire) active fire mask indicating 494 (30m resolution) active fire pixels coincident with GOES-R ABI simulated fire product Using Landsat-class imagery to validate ABI fire detection data

24. 24 ”Deep-Dive” Validation Tools For more visit: Deep-Dive Validation of GOES-R Active Fire Detection and Characterization Product at the poster session