1. GOES-R Field Campaign Workshop, Apr 8-9, 2015 GOES-R Algorithm Working Group (AWG) Planning and Participation in GOES-R Field Campaign Activities Jaime Daniels Center for Satellite Applications and Research National Environmental Satellite Data and Information Surface National Oceanic and Atmospheric Administration 1 Contributions from AWG Team Leads and Team Members

2. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Outline The Algorithm Working Group (AWG): A Brief Introduction AWG Level-2 Product Validation Methods, Reference Data, and Activities AWG Team Participation in Targeted Field Campaigns 2

3. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Outline The Algorithm Working Group (AWG): A Brief Introduction AWG Level-2 Product Validation Methods, Reference Data, and Activities AWG Team Participation in Targeted Field Campaigns 3

4. GOES-R Field Campaign Workshop, Apr 8-9, 2015 GOES-R Algorithm Working Group End-to-End Capabilities –Instrument Trade Studies –Proxy Dataset Development –Algorithm Development and Testing –Product Demonstration Systems –Development of Cal/Val Tools –Integrated Cal/Val Enterprise System –Radiance and Product Validation –Algorithm and application improvements –User Readiness and Education Mission:Mission:  To select, develop, test, validate, and demonstrate Level-2+ algorithms that meet the GOES-R F&PS requirements and provide them to the GOES-R Ground Segment.  Provide sustained life cycle validation and Level-2 product enhancement s 4

5. GOES-R Field Campaign Workshop, Apr 8-9, 2015 STAR GOES-R AWG PM Jaime Daniels STAR GOES-R AWG PM Jaime Daniels Management Support Tess Valenzuela (Budget/Schedule) Management Support Tess Valenzuela (Budget/Schedule) Algorithm Integration Walter Wolf Algorithm Integration Walter Wolf Proxy Fuhzong Weng Proxy Fuhzong Weng Radiation Budget Istvan Laszlo Radiation Budget Istvan Laszlo Land Bob Yu Land Bob Yu Cryosphere Jeff Key Cryosphere Jeff Key Ocean Dynamics Eileen Maturi Ocean Dynamics Eileen Maturi SST Alexander Ignatov SST Alexander Ignatov Lightning Steve Goodman Lightning Steve Goodman Hydrology Bob Kuligowski Hydrology Bob Kuligowski Aerosols Shobha Kondragunta Aerosols Shobha Kondragunta Winds Jaime Daniels Winds Jaime Daniels Imagery Tim Schmit Imagery Tim Schmit Soundings Tim Schmit Soundings Tim Schmit Aviation Ken Pryor Mike Pavolonis Aviation Ken Pryor Mike Pavolonis Clouds Andy Heidinger Clouds Andy Heidinger Cal/Val (Sensor) Fred Wu Cal/Val (Sensor) Fred Wu Algorithm Working Group Team leads are STAR Scientists 5

6. GOES-R Field Campaign Workshop, Apr 8-9, 2015 GOES-R Products Advanced Baseline Imager (ABI) Aerosol Detection (Including Smoke and Dust) Aerosol Optical Depth (AOD) Clear Sky Masks Cloud and Moisture Imagery Cloud Optical Depth Cloud Particle Size Distribution Cloud Top Height Cloud Top Phase Cloud Top Pressure Cloud Top Temperature Derived Motion Winds Derived Stability Indices Downward Shortwave Radiation: Surface Fire/Hot Spot Characterization Hurricane Intensity Estimation Land Surface Temperature (Skin) Legacy Vertical Moisture Profile Legacy Vertical Temperature Profile Radiances Rainfall Rate/QPE Reflected Shortwave Radiation: TOA Sea Surface Temperature (Skin) Snow Cover Total Precipitable Water Volcanic Ash: Detection and Height Geostationary Lightning Mapper (GLM) Lightning Detection: Events, Groups & Flashes Space Environment In-Situ Suite (SEISS) Energetic Heavy Ions Magnetospheric Electrons & Protons: Low Energy Magnetospheric Electrons: Med & High Energy Magnetospheric Protons: Med & High Energy Solar and Galactic Protons Magnetometer (MAG) Geomagnetic Field Extreme Ultraviolet and X-ray Irradiance Suite (EXIS) Solar Flux: EUV Solar Flux: X-ray Irradiance Solar Ultraviolet Imager (SUVI) Solar EUV Imagery Baseline Products Level-2 products outlined in red 6

7. GOES-R Field Campaign Workshop, Apr 8-9, 2015 7 http://www.goes-r.gov/products/baseline.html 7

8. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Outline The Algorithm Working Group (AWG): A Brief Introduction AWG Level-2 Product Validation Methods, Reference Data, and Activities AWG Team Participation in Targeted Field Campaigns 8

9. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Overarching Product Science Validation Methods Product inspection  Visualization of product and/or any output fields in the product, intermediate, and/or diagnostic files using in-house software tools  Provides a quick qualitative assessment of product performance Comparison to reference/correlative/ground truth data  Collocate product and applicable reference/correlative/ground truth datasets and compute quantitative statistics (accuracy, precision, etc)  Visualization of product and/or any output fields in the product, intermediate, and/or diagnostic files together with reference/correlative/ground truth data using in-house software tools  Provides a quantitative assessment of product performance. Focuses on assessing and characterizing product quality (ie., accuracy and precision) that needs to be conveyed to the user community. 9

10. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Overarching Product Science Validation Process Diverse earth (atmospheric & surface) and space weather conditions are represented over space, time and measurement range. Compute estimates of on-orbit product performance Characterize errors Goal: Determine if product meets success criteria Product Generation Validate (Inspect and/or Compare with Reference/ Ground Truth) Algorithm Approach for validating performance of products derived from GOES-R data and correlative data sources… Validation Tools 10

11. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Reference/”Ground Truth” Data Sources Aeronet Stations Aerosol Optical Depth Radiosondes Winds,Temperature, Moisture, Stability CALIPSO, CLOUDSAT Clouds, Icing NWP Analyses Winds, Temperature, Moisture Bouys, Ships SST SURFRAD, ARM LST, Radiation Rain Guages Precipitation Sfc Snow Reports, NESDIS IMS Snow National Lightning Detection Network (NLDN) Lightning Pilot, aircraft Reports Icing,Turbulence, winds Ground-based Ozone Ozone Validation teams use a wide variety of Reference/“Ground Truth” datasets to assess and validate product performance. GOES EPS and HEPAD Protons, Alpha Particles and Electrons NASA SDO Solar Imagery 11

12. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Types – Primarily a mix of in-house developed software & COTS software – In-house developed software includes: visualization, collocation, and analysis tools – Types of COTS software include: McIDAS-X, McIDAS-V, IDL, GrADS Current State – Mature. – AWG teams have been working to develop these tools since 2006. – Nearly all tools are in a working state with updated versions being developed. Status of AWG L2 Product Validation Tools 12

13. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Product Validation Tools AWG teams have developed a cadre of product validation tools as part of their ongoing product validation activities Significance: The tools will enable the routine monitoring of L2 product performance and for “Deep- dive” assessments and analysis of products to resolve any issues/anomalies that may arise Clouds LST Aerosols 13

14. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Outline The Algorithm Working Group (AWG): A Brief Introduction AWG Level-2 Product Validation Methods, Reference Data, and Activities AWG Team Participation in Targeted Field Campaigns 14

15. GOES-R Field Campaign Workshop, Apr 8-9, 2015 AWG product application teams asked to identify where gap filling measurements would enable more complete validation for L2 products they are responsible for The Unmanned Aircraft Systems (UAS) capability has had strong support from a number of the AWG product teams (Radiation Budget, Land, Aerosol) AWG has also provided critical support since early discussions with the NOAA UAS team through numerous TIMs and support in the development of a draft GOES-R UAS Near Surface Science Requirements document as well as an attendant CONOPS plan AWG Team Participation in Targeted Field Campaigns 15

16. GOES-R Field Campaign Workshop, Apr 8-9, 2015 AWG Team Participation in Targeted Field Campaigns Land Team (Bob Yu) – Land surface temperature – Fire Radiation Budget Team (Istvan Laszlo) – Shortwave radiation Cloud Team (Andy Heidinger) – Cloud-top type, cloud-top height, cloud optical depth, cloud particle size, liquid water path, ice water path Aerosols/Air Quality/Atmospheric Chemistry Team (Shobha Kondragunta) – Aerosol optical depth (AOD), but more importantly, PM2.5 concentrations derived from retrieved AOD 16 Lightning Team (Steve Goodman) Lightning Detection In collaboration with external partners…

17. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Land Surface Temperature Slides courtesy of Bob Yu 17

18. GOES-R Field Campaign Workshop, Apr 8-9, 2015 18 Components of LST Validation  In-situ measurement comparisons and analyses  Cross-satellite comparisons and analyses  Successful applications– users promotion Strategy of In-situ measurement comparisons and analyses  Existing ground station observations (e.g. SURFRAD Network), serve as long-term validation data source  Field campaign data plays three important roles  High quality observations for direct comparison and analysis  Calibrating co-site ground station observations  Characterizing the heterogeneity of the ground station site  Towards the field campaign readiness  Platform: Low altitude, small Unmanned Aircraft Systems (UAS)  Instrument readiness: Accurate infrared radiometers cover ABI bands  Site selection: Better to cover SURFRAD/CRN station  Data processing and algorithms: noise filtering, spatial characterization, calibration to station data, etc.  Coordination with the Field Campaign Team Towards Field Campaign for LST Validation

19. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Fire Slides courtesy of Ivan Csiszar 19

20. GOES-R Field Campaign Workshop, Apr 8-9, 2015 In general, for proper validation we need the mapping of the thermal conditions at high resolution within the entire pixel footprint. This needs to be done with sensors that have the proper bands (ideally ~ 4 microns, but there is some flexibility here), and simultaneously with the satellite overpass due to the very dynamic nature of fires. These data help determine detection probabilities as a function of sub-pixel fire characteristics. As a minimum, statistically we can determine the probability of detection as a function of sub-pixel fire size or fraction. If we can also measure unsaturated thermal radiance we can also do this in a two-dimensional space - detection probability as a function of fire size and temperature (or intensity). Getting these data is very hard and opportunistic. Realistically, we cannot sample the entire size/temperature space, but rather, use such reference data as anchor points to confirm simulated performance statistics. Benefits to Fire Product Algorithm VIIRS 375m Fire data 20

21. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Shortwave Radiation Slides courtesy of Istvan Laszlo 21

22. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Benefits to AOD and SRB Algorithms Better characterization of surface and atmospheric state – Can test/diagnose algorithm* under optimal conditions (many normally remote-sensed, modeled or climatological parameters are “measured”) – Can get information on spatial variability of surface and atmosphere conditions, and on L2 parameters on a scale approaching/comparable to that of an ABI pixel Measurements* of AOD and SRB at locations (e.g., open ocean) where reference data are not readily available. *NOTE: data are limited in space and time; it is understood that it will not provide true measure of algorithm performance and product quality Surface? Cloud? Aerosol? Gas? SURFRAD+ sites 22

23. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Plan for Using Field-Campaign Data Use field-campaign AOD and DSR to evaluate ABI retrievals of like quantities Use field-campaign AOD, surface reflectance (directional and spectral), water vapor profile, sky condition (cloudiness), etc., to identify sources of errors. – Retrievals will be performed by replacing derived, modeled or climatological data with field-campaign data – one at a time, and – Results will be compared to those from “normal” retrievals. 23

24. GOES-R Field Campaign Workshop, Apr 8-9, 2015 AOD Example UAV measurements can be used to check components of the ocean model values UAV measures sum of Terms 1 and 2 (approximately). “Integration” of directional UAV measurements gives the sum of all four Terms. Dividing the “integrated” value by the separately measured downward irradiance gives the albedo needed in AOD retrieval over land (and in SRB retrieval). Term 2 diffuse in direct out Ocean surface reflectance in ABI AOD algorithm is the sum of 1.water-leaving reflectance 2.whitecap reflectance 3.bi-directional reflectance Bi-directional reflectance is modeled as the sum of 4 components. Term 1 direct in direct out Term 3 direct in diffuse out Term 4 diffuse in diffuse out +++ 24

25. GOES-R Field Campaign Workshop, Apr 8-9, 2015 SRB Example Large negative downward shortwave radiation (DSR) bias at Cape Cod during an ARM field campaign in 2012. Deep-dive analysis: On average, retrieved AOD > ground-measured AOD; retrieved spectral reflectance albedo retrieved DSR < ground-measured DSR. DSR is severely underestimated on this easy, dominantly clear-sky day. Why? Too small surface albedo? Too large AOD? 25

26. GOES-R Field Campaign Workshop, Apr 8-9, 2015 UAV Measurements Quantify Spatial Representativeness of Station Data Illustration of problem UAV data can help Satellite “sees” larger area than downward looking instrument does. Upward looking instrument “sees” radiation from an entire hemisphere. This represents inconsistency unless atmosphere and surface are homogeneous (uniform) Deployment of UAV at several different locations within the satellite footprint can characterize degree of uniformity within footprint. Ideally, this should be done for all reference (e.g. SURFRAD) sites in different seasons. Satellite footprint on ground Box size of satellite footprint station UAV 26

27. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Cloud Products Slides courtesy of Andy Heidinger 27

28. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Use of LIDAR for Cloud Validation Airborne lidars provide higher spatial resolution data than spaceborne. Our CALIPSO CALIOP tools have been applied to ground-based lidar during DC3 Provides almost ideal validation source of cloud vertical profiles when viewed from above. Depolarization also provide cloud-top phase. Multi-layer detection component of cloud type and height algorithms can also be validated. Example Comparison of CALIPSO and ACHA for one AQUA/MODIS Granule. 28

29. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Use of Surface Based Radiometers for Daytime Cloud Optical and Microphysical Properties Validation (DCOMP) The Solar Spectral Flux Radiometer (SSFR) is a shortwave spectrometer operated by U. Colorado / LASP. During CALNEX 2010 it was operated on a ship looking up. It provides retrievals of DCOMP cloud properties using radiation travelling through the cloud (not reflected off the top like DCOMP). This provides a more independent validation and can be accomplished with any upward looking well-calibrated radiometers. AVIRIS also is useful for this, if available. Red lines are the DCOMP Error Bars Liquid Water Path (lwp) is an option 2 DCOMP product 29

30. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Aerosols Slides courtesy of Shohba Kondragunta and Xinrong Ren (UMD) 30

31. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Benefits to Aerosol Algorithms The field campaign focus is NOT to validate Aerosol Optical Depth (AOD) as this can be done sufficiently with AERONET observations. Rather, the field campaign focus is to evaluate surface PM2.5 derived from AOD AWG Aerosol Team has partnered with the University of Maryland (UMD) to do the flights and get aerosol vertical profile and Single Scattering Albedo (SSA) information. 31

32. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Evaluating Surface PM2.5 Derived from Retrieved Aerosol Optical Depth 32

33. GOES-R Field Campaign Workshop, Apr 8-9, 2015 UMD Research Aircraft Capabilities for GOES-R Validation Results from the summer 2013 flights over the Eastern Shore (see the poster Ren et al.) 33

34. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Extinction = Absorption + Scattering and can be measured at 3 wavelengths Extinction measured during 3 spirals Results from a FLAGG-MD Winter 2015 flight on 02/20/2015 UMD Research Aircraft for Measuring Urban Emissions 34

35. GOES-R Field Campaign Workshop, Apr 8-9, 2015 GLM Lightning Product Slide information courtesy of Steve Goodman 35

36. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Benefits to GLM Lightning Detection Algorithm Validation of GLM flash detection efficiency: collect coincident and collocated high altitude data over thunderstorms with the Fly’s Eye GLM Simulator (FEGS*):  Minimum Collection Set: Over-flights of thunderstorms over well characterized total lightning super sites. Emphasis on large scale convection such as Mesoscale Convective Systems (MCSs) from pre-storm through entire evolution (include all times of day & other storm types)  Secondary Collection Set: Over-flights of thunderstorms at other locations (day / dawn or dusk/ night; high / low latitudes; land / ocean; various storm types / regimes) Validation of GLM flash location & time-stamp accuracy, and INR Validation of optical energy calibration for the GLM product (lightning, which consists of events, groups, and flashes) SURFRAD+ sites 36

37. GOES-R Field Campaign Workshop, Apr 8-9, 2015 Summary AWG team participation identified along with their associated Level-2 products whose validation can benefit from GOES-R Field Campaign measurements AWG teams will perform data analysis at their respective local compute facilities – Leverage many of the validation tools already developed as part of AWG program – Product reprocessing capability exists and can be exercised AWG teams involved in this will perform data analysis and document their analysis results (Expect some extra cost beyond current funding) For Discussion – Who/where/how will data archival and management be done? 37

38. GOES-R Field Campaign Workshop, Apr 8-9, 2015 38 BACKUP

39. GOES-R Field Campaign Workshop, Apr 8-9, 2015 39 Sensor characterization - Radiometric calibration - Geolocation/navigation Post-Launch Tests (PLT) and engineering tests (compliance) Established sensor stabilityQC/QA processes in place Proxy data generationCalibration Processing; Analysis of L1b products Sensor characterization - Radiometric calibration - Geolocation/navigation Continuous assessment & monitoring, trend analysis of product quality Algorithm assessment and verification Quick look analysis of L2 products; comparisons to NWP model/analyses Finalize L2 algorithm tuning and testing; Establish L2 product stability Algorithm improvements Determination of validation strategies, including identification and acquisition of “ground-truth”/reference datasets Work to establish sensor stability; Work to establish L1b and L2 product stability; L1b and L2 algorithm testing and tuning L1b/L2 product validation processes in place; L1b and L2 product validations Full and continuous data release to the user community Cal/Val tool developmentEstablish routine validation processes Increasing data release to the user community Cal/Val tool improvements Development of L1b & L2 Cal/Val Plans Data released to users, but data is understood to be non-operational Cal/Val tool improvementsData Archival Early Orbit Check-out Long Term Monitoring & Operations Intensive Cal/Val Pre-launch Cal/Val Pre-LaunchPost-LaunchLaunch Timeline ~6 months Calibration/Validation Phases Phases 39