1 NUOPC National Unified Operational Prediction Capability Update to COPC 27 – 28 May 2015 Dave McCarren, NUOPC DPM

2 NUOPC National Unified Operational Prediction Capability 2 Agenda  National ESPC Update  NUOPC UEO Committee Update  NUOPC CMA Committee Update  Questions and Discussion

3 NUOPC National Unified Operational Prediction Capability. The National Earth System Prediction Capability (National ESPC) 3

The National Earth System Prediction Capability ESPC 4 National ESPC Update National ESPC Strategy 2-pager created for appropriate distribution to get support for the project; sent to Liaisons for review and taken to AMS meeting in Phoenix National ESPC staff drafting Strategy paper for submission to BAMS ESG meeting 26 Jan 2015 ESG Principals +1 meeting hosted by Navy 5 May 2015

The National Earth System Prediction Capability ESPC 5 National ESPC Update ESG meeting 26 Jan 2015 Action Items: Modify the proposed National ESPC Management Structure to show reporting relationships and communications. Project office will provide proposal with fully coordinated recommendation, options, with pros and cons. Coordinate a topical ESG meeting within 90 days for various topics, N2/6E to host. Semi-closed session: just staff and Principals + 1. Verify that the ESMF paper contains proper mention of National ESPC. Provide the ESG principals background on ESMF management, funding, and requirements process. Coordinate specific sessions for National ESPC at upcoming conferences. Coordinate a response to the Presidential Executive Order on Coordination of National Efforts in the Arctic on the role of ESPC in Arctic prediction, including a potential National ESPC brief by the principals to OSTP. This response may include the possible development of an integrated NAVY/NOAA operational Arctic Modeling Strategy.

The National Earth System Prediction Capability ESPC 6 National ESPC Update ESG meeting 5 May 2015 draft Action Items: Revise the ESPC and NUOPC charters and create a new National ESPC Charter. Include recommendations from Decision Brief on Management Structure/Plan, and Personnel Issues. Work with climate community to help define what is operational for products to support strategic decision making, Define components that will be part of National ESPC - spiral 1 Coordinate AF and Navy collaboration to determine the DOD way ahead and converge on a common core

National Unified Operational Prediction Capability NUOPC 7 UEO Committee Update

National Unified Operational Prediction Capability NUOPC 8 Unified Ensemble Operations NUOPC metrics from NCEP, FNMOC, and AFWA briefed at 26 Jan ESG meeting ½ Degree Data Exchange Upgrade on schedule for Summer 2015 implementation at NCEP, CMC, and FNMOC NUOPC/NAEFS mini workshop on Feb –Face to face meeting of the Co-Chairs of the UEO Committee –Agenda focused on the development of NUOPC that includes data exchange (of global and wave ensemble data, post process, ensemble week 3&4 forecast and future plans. –Research following a concern by FNMOC over the timing of the CMC raw ensemble data resulted in NCEP starting raw data processing 90 minutes earlier, much closer to real time and bias corrected data. AF moved up operational GEPS processing to take advantage of this; GEPS suite had 5,000 unique users and about 4M product hits in the last 12 months

½ Degree Data Exchange Implementation Schedule NCEPCMCFNMOC Test hi-res dataAvailable Pre-production hi-res dataJan 2015Sep-Oct 2014 Production hi-res dataApr 2015Nov-Dec 2014Oct 2014 Ready to receive hi-res production dataJan 2015 Test hi-res NAEFS productsMar 2015Apr 2015 (SFC-3HR) Pre-production hi-res NAEFS productsApr 2015 Production hi-res NAEFS productsMay 2015Jul 2015 (SFC-3HR) Earliest practical date for each stage at each center

5-day forecast for surface wind (U) 10-day forecast for surface wind (U)

NH T2m

NA T2m

Northern Hemisphere Wind Speed 13 The spread ratio for wind speed was greater than one for the entire period (under-dispersive) with NAVGEM and GFS members both being over-dispersive and GEM members also under-dispersive. There was a spike in May when NOGAPS was replaced by NAVGEM. GEM members are under-dispersive while NAVGEM and GFS members are over-dispersive

Ensemble Mean Northern Hemi -2 Meter Temps 14 Ensemble mean in solid red. GFS members in dotted green, NAVGEM members in dotted blue, and GEM members in dotted purple. The solid green is the GFS control member and the solid purple is the CMC control member. We do not currently receive a NAVGEM control member. Ensemble mean has nearly no bias. GFS members have a cold bias, and NAVGEM members have a warm bias

National Unified Operational Prediction Capability NUOPC 15 CMA Committee Update

National Unified Operational Prediction Capability NUOPC NUOPC Common Model Architecture Draft manuscript for ESPS paper submitted to BAMS; CMA Committee revising first draft for 1 June deadline CMA Committee drafting new whitepaper on Using ESPS PI Group building a working Physics Driver prototype with a common physics interface to GFS physics, by June 2015, to address NGGPS requirement to begin testing how it works and use in the selection of a new Dynamc Core, using a common set of physics. PI Group presented modernized Kalnay Rules paper at AMS

National Unified Operational Prediction Capability NUOPC 17 Questions & Discussion

NUOPC Verification Metrics EMC/NCEP May Acknowledgement: Dr. Yan Luo For all three individual bias corrected ensemble forecast (NCEP/GEFS, CMC/GEFS and FNMOC/GEFS) and combined (NUOPC) ensemble (equal weights) against UKMet analysis Period: April 1 st 2011 – May

Ratio of RMS error over spread Northern Hemisphere 500hPa height: 30-day running mean scores of day-5 CRP score RMS error and ratio of RMS error / spread Anomaly correlation All other regions could be seen from: aefs/VRFY_STATS/NUOPC_bc_COMB_spr201 5_ts.html aefs/VRFY_STATS/NUOPC_bc_COMB_spr201 5_ts.html NH 500hPa anomaly correlation 5-day forecast NH 500hPa CRP scores Under-dispersion Over-dispersion NH 500hPa RMS errors

Northern Hemisphere 500hPa height: 30-day running mean scores of day-10 CRP score RMS error and ratio of RMS error / spread Anomaly correlation All other regions could be seen from: fs/VRFY_STATS/NUOPC_bc_COMB_spr2015_ts. html fs/VRFY_STATS/NUOPC_bc_COMB_spr2015_ts. html 10-day forecast NH 500hPa CRP scores NH 500hPa anomaly correlation NH 500hPa RMS errors

NH CRP scores NH RMS errors NA CRP scores NA RMS errors 5-day forecast for surface temperature

10-day forecast for surface temperature

5-day forecast for surface wind (U) 10-day forecast for surface wind (U)

5-day forecast for surface wind (V) 10-day forecast for surface wind (V)

Last winter statistic scores Dec. 1 st 2014 – Feb. 28 th 2015

Northern Hemisphere 500hPa height: Latest 3-month winter scores: CRPS skill score RMS error and ratio of RMS error / spread Anomaly correlation All other regions/scores could be seen from: UOPC/NUOPC_bc_win1415.html UOPC/NUOPC_bc_win1415.html NH CRPS skill scores NH anomaly correlation 500hPa Height NH RMS errors

NH CRPS skill scoresNH RMS errors NA CRPS skill scoresNA RMS errors Surface temperature

NH CRPS skill scores NH RMS/Spread scores For surface wind (U) NH CRPS skill scores NH RMS/Spread scores For surface wind (V)

Comparing of NCEP/GEFS, NAEFS and NUOPC for all bias corrected ensemble forecasts, against UKMet analysis Period: April 1 st 2011 – May 15 th 015

Day-5 NH 500hPa height Day-10 NH 500hPa height

NH T2m

NA T2m

Aim High…Fly, Fight, Win NUOPC Ensemble Performance 1 Jan 2015 – 1 Apr 2015 Mr. Bob Craig 16 WS/WXN

Aim High…Fly, Fight, Win Overview Method Brier Skill Scores Spread Ensemble Mean 34

Aim High…Fly, Fight, Win Method All models verified against 25km UKMO analysis except for total cloud cover that uses WWMCA Data shown is mainly for the northern hemisphere domain containing Navy - NAVGEM CMC - GEM NCEP - GFS On ensemble mean slides, ALL refers to the GEPS ensemble For skill scores, climatology was used as reference forecast 35

Aim High…Fly, Fight, Win Brier Skill Score Brier Score (BS) for ensembles is equivalent to Root Mean Square error for deterministic models. Model probabilities for an event are compared to the relative frequency of the event To get the skill score, the BS is compared to climatology Brier skill scores greater than 0 indicate skill 36

Aim High…Fly, Fight, Win Brier Skill Score Total Cloud > 80% 37 Total Cloud > 80% - looking at the CONUS domain, significant skill is indicated out to about 108hrs. For global and hemisphere domains, little significant skill is indicated. Some reasons for this can be seen on the next slide.

Aim High…Fly, Fight, Win Typical Total Cloud Forecast Total Cloud > 80% 38 Black squares are non- zero probabilities and green “1” are observation hits. Over high latitudes there tends to be less hits compared to lower latitudes. Cloud coverage tends to be over- forecasted

Aim High…Fly, Fight, Win Brier Skill Score Wind Speed > 25kts 39 For wind speeds >= 25kts, global coverage and northern hemisphere have significant skill out to 240 hours. Higher winds speeds (>35, and >50) also have skill, but an insignificant number of events resulted in much larger error bars. 25Kts

Aim High…Fly, Fight, Win Ensemble Spread Spread measures the difference between the ensemble mean and ensemble forecasts Typically the RMSE of the ensemble mean is compared to the standard deviation of the members from the mean (RMSE/Stdev) An ideal ensemble will have the same size of ensemble member deviation from the mean as Root Mean Square error of the ensemble mean An ensemble spread greater than one means the ensemble does not contain enough spread in the members to account for the errors in the ensemble system 40

Aim High…Fly, Fight, Win Northern Hemisphere 500mb Heights 41 This chart depicts the ensemble spread ratio of the ensemble members compared to the ensemble mean over the last year for the 120 hour forecast. Green represents the spread ratio of the combined ensemble (ALL) compared to the spread for GFS (purple), Navy (red), and CMC (blue). Ideally, the spread ratio should be around one. A spread ratio greater than one indicates the ensemble is under-dispersive and the opposite is true for spread ratio less than one. This chart depicts the ensemble spread by forecast hour. The ensemble starts off under-dispersive in early forecast hour getting better in later forecasts, green line approaching one

Aim High…Fly, Fight, Win Northern Hemisphere Wind Speed 42 The spread ratio for wind speed was greater than one for the entire period (under-dispersive) with NAVGEM and GFS members both being over-dispersive and GEM members also under-dispersive. There was a spike in May when NOGAPS was replaced by NAVGEM. GEM members are under-dispersive while NAVGEM and GFS members are over-dispersive

Aim High…Fly, Fight, Win Northern Hemisphere Temperature 43 The spread ratio was above 1 (under_dispersive) for most of the winter but approaching 1 as Spring arrives Over the last several months, the spread starts under-dispersive with early forecast hours but becomes neutral by the end of the period

Aim High…Fly, Fight, Win Ensemble Mean Ensemble mean is the average value of the ensemble members The ensemble mean error should be less than the error of all the ensemble members for a good ensemble One of the ensemble members from each center is the control and is configured like the deterministic model from the center though its resolution is degraded We do not currently get a control member from NAVGEM, but that will be added when available 44

Aim High…Fly, Fight, Win Ensemble Mean Northern Hemi -2 Meter Temps 45 Ensemble mean in solid red. GFS members in dotted green, NAVGEM members in dotted blue, and GEM members in dotted purple. The solid green is the GFS control member and the solid purple is the CMC control member. We do not currently receive a NAVGEM control member. Ensemble mean has nearly no bias. GFS members have a cold bias, and NAVGEM members have a warm bias

Aim High…Fly, Fight, Win Ensemble Mean Northern Hemi -10 Meter Winds 46 GFS members have the lowest error and NAVGEM members have the highest The ensemble mean has a positive bias and the NAVGEM members seem to be pulling it that way

Aim High…Fly, Fight, Win Ensemble Mean Northern Hemi – 500mb Hgts 47 NAVGEM members have the highest error out to 168 hrs, then all members are similar Ensemble mean bias is near 0 out to about 168 hours then starts to rise as GEM and NVGEM members pull it positive

Aim High…Fly, Fight, Win Ensemble Mean Northern Hemi – 250mb Winds 48 A three model members are close for this field Ensemble mean has a negative bias with all model members contributing. GFS members show a discontinuity after 192 hrs in their bias

Aim High…Fly, Fight, Win Ensemble Mean Northern Hemi – 850mb Temps 49 GFS members slightly lower error in early forecast hours Ensemble mean has a cold bias with GEM and GFS members pulling it into down

Aim High…Fly, Fight, Win Summary The GEPS ensemble system is performing as expected Brier skill scores for wind speed indicated the GEPS ensemble has skill out to 240hrs Precipitation forecasts weren’t included here due to recent problem with UM GRIB files The GEPS ensemble spread score tends to be greater than 1 with the mean RMSE > member standard deviation, indicating the ensemble members are under- dispersed which is a common ensemble issue 50

Aim High…Fly, Fight, Win Summary Ensemble mean forecasts have lower errors than the members which is expected for correctly designed ensembles. 51