© Crown copyright Met Office WAFC turbulence and Cb hazard verification Recent results and future plans Dr Philip G Gill WAFSOPSG 7/14, 30 th April 2013

© Crown copyright Met Office Contents This presentation covers the following areas Introduction Turbulence verification Cb verification Future plans Questions and answers

© Crown copyright Met Office Introduction Turbulence and Cb are a major cause of aviation incidents Turbulence and Cb hazard forecasting are part of the World Area Forecast Centre (WAFC) service provided by WAFC London and WAFC Washington Forecasts currently produced both by forecasters – Significant Weather Chart (SIGWX) and from gridded model output Objective verification was set up in 2009 Forecasts are global but verification is currently limited Photos © P Gill

© Crown copyright Met Office Objective turbulence verification

© Crown copyright Met Office Verification Objective verification of gridded clear air turbulence (CAT) forecasts Verification by severity of turbulence (eg. moderate or greater) Verify against automated aircraft observations from the Global Aircraft Data Set (GADS) Restricted to cruise level (above 28, 000 ft) to limit unwanted effects on observations of manoeuvring at lower levels

© Crown copyright Met Office Global Aircraft Data Set Fleet of Boeing aircraft Global coverage, but flights mainly over northern hemisphere Automated aircraft observations available every 4 seconds January 2009 Good coverage of N Atlantic, US and Europe Poor coverage of E Asia/Pacific region and southern hemisphere – need more observations

© Crown copyright Met Office Turbulence observations Pilot Reports (PIREPS) are issued by pilots when turbulence is encountered – useful additional information but are subjective, aircraft specific, temporal and spatial errors, no routine null reports AMDAR are automated reports from aircraft but are not regular and can differ between aircraft Derived Equivalent Vertical Gust (DEVG) - Indicator of turbulence derived from automated in-situ flight data: vertical acceleration, aircraft mass, altitude and airspeed Eddy Dissipation Rate (EDR) also a measurement of turbulence derived from automated in-situ data – ICAO standard At present access to EDR data limited to US – DEVG only option for wider coverage

© Crown copyright Met Office Verification methodology Aircraft track within time window Turbulent event Turbulence forecast field Ellrod TI1

© Crown copyright Met Office Forecast assessment Turbulent/non turbulent event defined on 10min aircraft track ~120km - approx grid size Forecast turbulent event – CAT potential >= Threshold Observed (moderate or greater) turbulent event - DEVG>=4.5m/s Construct 2x2 contingency tables for each threshold Sum entries in contingency tables over the verification period Turbulence observed No turbulence observed Turbulence forecast HitFalse alarm No turbulence forecast MissCorrect rejection 2x2 contingency table

Forecast skill Contingency tables can be used for various probability thresholds to produce a reliability table From the reliability table various scores can be calculated The skill of the forecast at discriminating between events and non-events can be measured using the Receiver Operating Characteristic (ROC) by plotting hit rate against false alarm rate The curve from a probabilistic forecast and single point from a deterministic forecast can be compared. Line of no skillPerfect forecast Area under curve measures skill

© Crown copyright Met Office WAFC CAT verification Robert Coulson

Latest results – mean CAT Acceptable hit rates? Acceptable false alarm rates? Area of interest could be measured by partial area under ROC curve

Latest results – max CAT

Latitudinal variation 50N to 90N 20S to 20N 20N to 50N 50S to 20S Good performance from above 20N

Summary of CAT verification Blending process continues to produce both consistent and skilful turbulence forecasts. Some variation in skill by latitude with best performance in the mid latitudes. Rolling 12-month results produced monthly since June 2012.

© Crown copyright Met Office WAFC Cb verification Teresa Hughes

Cb verification Verification against lightning strikes (SFERICS) from Met Office ATDnet Gridded SFERICS field produced using observations at +/- 30 minutes from forecast validity time T+24 forecast verified from all four model runs

ATDnet Domain

Latest results

Latitudinal variation 50N to 80N20N to 50N 20S to 20N40S to 20S

Summary of Cb verification Blending process continues to produce both consistent and skilful Cb forecasts. Some variation in skill by latitude Rolling 12-month results produced quarterly since June 2012.

© Crown copyright Met Office Future verification plans

Improving turbulence verification Switch from DEVG to EDR ICAO standard Forecasting and observing EDR Source more EDR observations from other airlines to increase the area of coverage Increase coverage to fill gaps in current GADS database

Improving Cb verification Extend verification against lightning reports by outside of the ATDnet domain Source additional lightning reports outside [80N, 40S, 100W, 80E] Use geostationary satellite data in addition to lightning data MSG [60N, 60S] Overshooting convection product (work underway) New satellites over America (GOES-R) and East Asia (Himawari-8) Verify Cb cloud top height and Cb extent

MSG - the European Geostationary Imager Scans Africa, Europe, Middle East and Atlantic every 15 minutes 3/1 km spatial resolution at 0° N, 0° E 12 channels (different wavelengths) Meteosat-10 operational, Meteosat-8,-9 backup

Severe convection product Data from Meteosat Second Generation Calculated from the visible and infrared channels. Generated every 15 mins. Current product over Europe identifies area where severe convection is occurring (day-time only) New overshooting convection product will identify intense convective areas with overshooting cloud tops. Could be merged with lightning data from ATDnet Could be merged later with Meteosat third generation lightning data

Cloud top height product Data from Meteosat Second Generation Calculated from the infrared channels at wavelengths of 10.8, 12.0 and 13.4 microns. Generated every 15 mins Covers Europe, Atlantic, Africa Extend to global coverage following GOES-R and Himawari-8 satellite launches (~ 2015) Mask with Cb product

Verification – all forecasts Verify probabilistic forecasts using suitable metrics Forecast skill – ROC (partial area under curve?) Reliability diagram Value – relative economic value measure Publish verification results – plots and contingency tables Breakdown into WMO regions Forecast ranges T+12, T+24, T+36 Rolling 12-month scores Geostationary satellite data limited to 60N, 60S. Polar orbiter data could supplement this if necessary.

Timescales 2013/14 Preliminary work on satellite data specifications Routine CAT and Cb verification published by WMO region. 2014/15 Include satellite data in Cb verification Develop Cb cloud top height verification Move to using EDR in turbulence verification 2015/16 Expand coverage of Cb verification using new satellite data over East Asia (Himawari-8) 2016/17 Expand coverage of Cb verification using new satellite data over America (GOES-R)

© Crown copyright Met Office Questions & answers