1. Preparing for GOES-R: Post Launch Test Planning Wayne MacKenzie GOES-R Product Area Lead GOES-R Product Readiness and Operations (PRO) Team Deputy GOES-R/JPSS OCONUS Satellite Proving Ground Meeting June 28, 2016

2. Agenda What does it take to make Imagery? Pre-Launch Activities PLT Validation Maturity Stages Algorithm Change Process L2+ Algorithm Updates 2

3. WHAT DOES IT TAKE TO MAKE A SATELLITE IMAGE? 3

4. Sectorized CMIL1b ImageL1β Data L0 Data 01010100110 11110001010 10101000111 1001011010… Calibrate L1α DataL1a Data Scale to Ref/BT Resample Data Processing 4 Image credit: Dave Pogorzala

5. PRE-LAUNCH ACTIVITIES 5

6. Data Operations Exercises DOEs are mission rehearsals executed by the Data Operations Support Team to support readiness Conducted in a “rehearse like we fly” manner DOE-4, July 18-August 26, 2016, will include participation by the NWS TOWR-S (Total Operational Weather Readiness - Satellites) project which is incorporating distribution to elements of the AWIPS community NWS sends data via the Satellite Broadcast Network (SBN) to select WFOs and NCEP Centers to ensure preparedness of their distribution systems. This test data is accessible to anyone with a SBN receiver. 6

7. DOE-4 Planning Supporting end-to-end LUT updates, and Algorithm Changes Cal/Val Teams will have access to data via PDA to evaluate the products in real-time Using near real-time simulated imagery Supporting anomaly resolution NWS sends data via the Satellite Broadcast Network (SBN) to select WFOs and NCEP Centers to ensure preparedness of their distribution systems. This test data is accessible to anyone with a SBN receiver. Will perform complete end-to-end L2 algorithm update 7

8. Simulated Cloud & Moisture Imagery (CMI) from the Ground Readiness Exercise GRE-DO1 (AAWDS) 8 GOES-R Inspection Program (GRIP) 10/21/15 NWS AWIPS 4-Panel Display 10/15/15 The GOES-R AWG Proxy Team at NESDIS/STAR’s Cooperative Institute for Meteorological Satellite Studies (CIMSS) creates 16 bands of real-time modeled data using WRF and GFS models, called AAWDS (ABI AWG/UW-Madison Data Set) RaFTR software formats the data and interpolates to the GOES-R real-time cadence for sampling Earth then the data are injected to the GOES-R GS data fabric Finally, PD sends the NetCDF files to NWS which broadcasts the file over the SBN to AWIPS-II terminals at select WFOs Forecasters then look at these simulated “GOES-R products” and can see the spatial/spectral/temporal benefits and make side-by-side comparisons against other tools

9. POST-LAUNCH ACTIVITIES: VALIDATION MATURITY 9

10. Product Maturity Levels What do the Product Maturity Levels mean? There is a PS-PVR at each stage as a method of informing the user community of the following readiness for use: Beta: Products are made available to users to gain familiarity with data formats and parameters. It has been minimally validated and may still contain significant errors and is not optimized for operational use. Provisional: Product ready for operational use but has documented known issues. Product analyses are sufficient to communicate product performance to users relative to expectations. Full: Product is operational. All known product anomalies are resolved and/or documented and shared with the user community.

11. Product Maturity Level Success Criteria

12. How much work is it to achieve these maturity levels?

13. Beta Validation Tests 13 L2 AlgorithmNumber of Tests Clear Sky Mask8 Aerosol Detection11 Aerosol Optical Depth6 Cloud Optical Properties8 Cloud Top Parameters8 Imagery7 DMW56 Shortwave Radiation8 Soundings (including TPW and DSI)32 Fire10 Hurricane Intensity5 LST8 Rainrate3 Snow Cover8 SST1 Volcanic Ash5

14. Provisional Validation Tests 14 L2 AlgorithmNumber of Tests Clear Sky Mask3 Aerosol Detection3 Aerosol Optical Depth2 Cloud Optical Properties3 Cloud Top Parameters3 Imagery3 DMW21 Shortwave Radiation3 Soundings (including TPW and DSI)12 Fire2 Hurricane Intensity1 LST3 Rainrate1 Snow Cover3 SST1 Volcanic Ash1

15. Full Validation Tests 15 L2 AlgorithmNumber of Tests Clear Sky Mask3 Aerosol Detection3 Aerosol Optical Depth2 Cloud Optical Properties3 Cloud Top Parameters3 Imagery3 DMW21 Shortwave Radiation3 Soundings (including TPW and DSI)12 Fire2 Hurricane Intensity1 LST3 Rainrate1 Snow Cover3 SST1 Volcanic Ash1

16. DATA RELEASE INFORMATION 16

17. Detailed Data Release Strategy 17 Internal Flow L+15 Days External Distribution L+88 Days Handover Readiness Review (HRR) & Operations Handover L+6 Months Internal Flow (ABI) for Cal/Val Teams L+30 Days 17

18. Detailed Data Release Strategy 18 nternal Flow L+15 Days External Distribution L+88 Days Handover Readiness Review (HRR) & Operations Handover L+6 Months Beta – ABI L1b, CMI GRB ABI Feed turned on, CMI over PDA L+88 Days 18

19. Detailed Data Release Strategy 19 nternal Flow L+15 Days External Distribution L+88 Days Handover Readiness Review (HRR) & Operations Handover L+6 Months 19 Beta – GLM GRB GLM Feed turned on L+148 Days

20. Detailed Data Release Strategy 20 nternal Flow L+15 Days External Distribution L+88 Days Handover Readiness Review (HRR) & Operations Handover L+6 Months Provisional – ABI L1b, CMI, GLM Beta – L2 Products L+6 Months 20

21. Detailed Data Release Strategy 21 nternal Flow L+15 Days External Distribution L+88 Days Handover Readiness Review (HRR) & Operations Handover L+6 Months East/West Assignment Provisional – L2 Products L+12 Months 21

22. Detailed Data Release Strategy 22 nternal Flow L+15 Days External Distribution L+88 Days Handover Readiness Review (HRR) & Operations Handover L+6 Months Full Validation – L1 Products L+18 Months 22

23. Detailed Data Release Strategy 23 nternal Flow L+15 Days External Distribution L+88 Days Handover Readiness Review (HRR) & Operations Handover L+6 Months Full Validation – L2 Products L+21 Months 23

24. ALGORITHM CHANGE PROCESS 24

26. L2 Algorithm Change Process Within the ground system, L2 algorithms are individual RPMs This enables changing individual algorithms without re- compiling the entire ground system and a fast deployment During PLT, we will be implementing a three week build cycle for algorithms Each build will contain 5-10 fixes depending on the level of effort Each build will involve system testing and working with the AWG to ensure algorithm output is correct. Promotion to ops can occur quickly after the DE testing is complete 26

27. Engineering Patch An engineering patch is multiple DAPs and/or changes that that require chain testing and coordination by multiple groups. – L1b changes that affect multiple algorithms – L2+ changes – Non-science changes that are needed quickly requested by the DCRB. This process will occur on 2 week schedule every 3 weeks where it does not interfere with an ITCO. The process involves 3 day chain testing and all changes will be included in a single OCCR.

28. 2 Week Engineer Patch Overview CM Merge/DE Prep Test on Live Data Stop Test/ Send Data Test Data Review/ Write WR ARB Review/OCCR Review/Deployment 1.5 Days2.5 Days 3 Days 1 Days7 Days

29. 2 Week Engineering Patch Overview con’t CM Merge/DE Prep – Thursday – Friday -The changes are merged into the PRO INT branch and back-out procedures are compiled. Test on Live Data - Tested on live data over the weekend. Stop Test/Send Data – On Monday the test is stopped. The changes are backed out of the DE and the test data is sent to the cal/val teams or appropriate party. CM Merge/DE Prep Test on Live Data Stop Test/ Send Data Test Data Review/ Write WRs ARB Review/OCCR Review/Deployment 1.5 Days2.5 Days 3 Days 1 Days7 Days

30. 2 Week Engineering Patch Overview con’t Test Data Review/Write WRs – Tuesday – Thursday - The test data is reviewed by the cal/val teams. This is a regression test so not only are the cal/val teams looking for the changes to functioning properly but also that there are not adverse consequences. WRs are also written. On Friday if the test is successful the OCCR is written and goes out for review. CM Merge/DE Prep Test on Live Data Stop Test/ Send Data Test Data Review/ Write WRs ARB Review/OCCR Review/Deployment 1.5 Days2.5 Days 3 Days 1 Days7 Days

31. 2 Week Engineering Patch Overview con’t Monday – The ARB approves the change. Tuesday – The WR is moved into work at WAM. Wednesday – The OCCR is approved and changes are deployed to the DE via Satellite Server. Thursday – The changes are deployed to the ITE and OE. CM Merge/DE Prep Test on Live Data Stop Test/ Send Data Test Data Review/ Write WRs ARB Review/OCCR Review/Deployment 1.5 Days2.5 Days 3 Days 1 Days7 Days

32. STATUS OF L2 ALGORITHMS 32

33. L2+ Algorithm Status 6/2/2016Operations Status Review (OSR) AlgorithmStatus ADPMatches AWG Test Data Results AODMatches AWG Test Data Results CMIMatches AWG Test Data Results Vol AshMatches AWG Test Data Results Clear Sky MaskMatches AWG Test Data Results CTP/CTT/CTHMatches AWG Test Data Results SoundingsMatches AWG Test Data Results Cloud PhaseMatches AWG Test Data Results DSR/RSRMatches AWG Test Data Results Snow Cover- Deferred. Degraded performance due to lack of Albedo product. Albedo was removed from Baseline in 2012, but is being developed by AWG. To accomplish delivery in 2016 Q4 & implementation in 2017 Q4. - User impact is to NIC which uses for comparisons in development of IMS. Four downstream products depend on Snow Cover, but all 4 have switched to IMS polar product (2km resolution matches GOES-R). DMWRemaining issues are being worked. Held a TIM with AWG/Harris/AER on 4/28 and 6/16. Several issues were identified and are being examined with AWG. Should be resolved and deployed in early PLT. LSTMatches AWG Test Data Results FireMatches AWG Test Data Results Rain RateMatches AWG Test Data Results – However, operational implementation is missing parallax correction which will impact operational implementation cal/val activities. Mitigation – implement parallax correction as one of the wrapped algorithm trade studies ASAP, which is likely after PLT. Hurricane IntensityMatches AWG Test Data Results SSTMatches AWG Test Data Results 33

34. Deltas to the Baseline Versions of the L2 Product Algorithms (Baseline Enhancements) Deltas to the Baseline Versions of the L2 Product Algorithms (Baseline Enhancements) L2 Product Algorithm Deltas? Change Type(s) Data Interface Changes? Aerosol Detection YTN Aerosol Optical Depth YI, LN Clear Sky Mask YT (baseline), M (new)N Cloud and Moisture Imagery (KPP) N--- Cloud Optical Depth YL,T--- Cloud Particle Size Distribution YL,T--- Cloud Top Height YI,TN Cloud Top Phase YTN Cloud Top Pressure YI,TN Cloud Top Temperature YI,TN Derived Motion Winds YI, C, TN Derived Stability Parameters N--- Downward Shortwave Radiation: SFC YLN 34 Change Types: Incremental (I) Major (M) Look-up Table (L) Threshold (T) Config (C)

35. Deltas to the Baseline Versions of the L2 Product Algorithms (Baseline Enhancements) Deltas to the Baseline Versions of the L2 Product Algorithms (Baseline Enhancements) L2 Product Algorithm Deltas? Change Type(s) Data Interface Changes? Fire/Hot Spot Characterization YT,LN Hurricane Intensity Estimation YI, TY Land Surface Temperature (Skin) YI, LY Legacy Vertical Moisture Profile YTN Legacy Vertical temperature Profile YTN Radiances N--- Rainfall Rate/QPE YIY Reflected Shortwave Radiation: TOA YLN Sea Surface Temperature (Skin) YI,T (Baseline), M (new)N Snow Cover YLY Total Precipitable Water N--- Volcanic Ash: Detection and Height YM (new)Y 35 Change Types: Incremental (I) Major (M) Look-up Table (L) Threshold (T) Config (C)

36. Deltas to the Baseline Versions of the L2 Product Algorithms (Baseline Enhancements) Deltas to the Baseline Versions of the L2 Product Algorithms (Baseline Enhancements) L2 Product Algorithm Deltas? Change Type(s) Data Interface Changes? Fire/Hot Spot Characterization YT,LN Hurricane Intensity Estimation YI, TY Land Surface Temperature (Skin) YI, LY Legacy Vertical Moisture Profile YTN Legacy Vertical temperature Profile YTN Radiances N--- Rainfall Rate/QPE YIY Reflected Shortwave Radiation: TOA YLN Sea Surface Temperature (Skin) YI,T (Baseline), M (new)N Snow Cover YLY Total Precipitable Water N--- Volcanic Ash: Detection and Height YM (new)Y 36 Change Types: Incremental (I) Major (M) Look-up Table (L) Threshold (T) Config (C) BOTTOM LINE: Algorithms that require a LUT or Threshold Change will be implemented during PLT. Incremental and configuration changes will be addressed on a per algorithm basis during PLT. BOTTOM LINE: Algorithms that require a LUT or Threshold Change will be implemented during PLT. Incremental and configuration changes will be addressed on a per algorithm basis during PLT.

37. Questions? For questions or general information, please contact me at: – wayne.mackenzie@noaa.gov 37