Evaluation of CONCEPTS Global and Regional Sea Ice Forecasts Greg Smith and the CONCEPTS Team Meteorological Research Division, RPN-E AOMIP Workshop, Woods Hole, USA October 23-26, 2012

Hierarchy of CONCEPTS Systems Coupled Gulf of St. Lawrence Atmosphere-Ice- Ocean Forecasting System Daily coupled 48hr forecasts Resolution of 15km (atm) and 5km (ice-ocean) Global Ice-Ocean Prediction System (GIOPS) Weekly 1/4° resolution global ice-ocean analyses and 10 day forecasts Regional Ice Prediction System (RIPS) 3DVAR ice analysis system providing 4 analyses/day North American domain on 5km grid 48hr forecasts using CICE sea ice model

Gulf of St. Lawrence Coupled Atm-Ice-Ocean Forecasting System Coupled – Uncoupled differences Operational since June 2011 48 forecast daily at 00Z Coupled system: Atm: GEM (10km) Ice: CICE (5km) Ocean: MoGSL (5km) Mar. 10, 2012 Sea Ice 2m Temperature Jul. 10, 2012 10m Winds 2m Temperature 15km, 58 levels

CONCEPTS Global Ice-Ocean Prediction System Mercator Ocean Assimilation System (SAM2-PSY3V2): Sea surface temperature CMCSST Sub-surface T&S from Argo, CTDs, XBTs, Moorings Sea level anomaly from satellite altimeters J1,J2,EN with Rio05 MDT CMC Ice analysis: SSM/I, CIS charts, Radarsat Blended ice-ocean analysis and NEMO 10day forecast produced every Wednesday since Dec. 2010. Model configuration: ORCA025 (~1/4°), <15km in Arctic NEMOv3.1, LIM2-EVP

Regional Ice Prediction System Daily 48hr forecasts: CICEv4.1 forced by CMC RDPS 5km North American grid 3DVar-FGAT Sea Ice Analysis: Four analyses per day of ice concentration at 5 km resolution Currently assimilates: SSM/I, AMSR-E, CIS daily charts, Radarsat image analyses Work in progress to add: SSMIS, scatterometer, visible-infrared, SAR and ice thickness satellite-based observations SSM/I AMSR-E CIS Chart Radarsat

Evaluation against ice analyses Compare forecasts to analyses as function of lead time(t) Apply two criteria: Remove points of low confidence Mask out regions where “Days Since Last Obs” > 0.5 Focus on ice edge Only include points where: | Conc(t) – Conc(0) | > 0.1 Compare forecast to persistence of initial analysis equivalent to a forecast of “no change” Perform evaluation for GIOPS and RIPS for : North American domain (RIPS analysis as “truth”) NH domain (CMC analysis as “truth”)

Evaluation against RIPS: lead 24h North American domain RIPS beats persistence throughout 2010 GIOPS beats CMC-pers throughout year Large differences in spring melt season RMS Bias

Evaluation against RIPS: lead 48h North American domain RIPS beats persistence throughout 2010 GIOPS beats CMC-pers throughout year Large differences in spring melt season RMS Bias

Evaluation against RIPS: lead 120h North American domain GIOPS shows much reduced RMS throughout year GIOPS bias also smaller than CMC persistence Esp. during fall Large differences in spring melt season RMS Bias

Evaluation against RIPS: lead 240h North American domain GIOPS shows much reduced RMS throughout year GIOPS bias also smaller than CMC persistence Esp. during fall Large differences in spring melt season RMS Bias

RMS errors vs CMC : Lead 120hr Weekly forecasts from 2010 GIOPS shows smaller errors than persistence over most of the NH Largest errors in : E. Greenland Current Barents Sea Labrador Sea GIOPS Persistence

Evaluation against IMS analyses Interactive Multisensor Snow and Ice Mapping System (NOAA-NIC) Daily Northern Hemisphere ice analyses on 4km grid (ice/water) Assimilates : AVHRR, GOES, SSM/I Evaluation Methodology: Interpolate model forecasts to IMS grid Calculate contingency table values using 0.4 ice concentration cutoff Bin results on 1° lat-lon grid Compare to persistence IMS Ice IMS No ice Forecast Ice Hit ice False Alarm Forecast No ice Miss Hit water

Accumulated statistics for 5 day lead Jan-Dec, 2011 (52 forecasts) Ice? YIMS NIMS YFcst Hit ice False Alarm NFcst Miss Hit water Persistence of ice analysis Model forecasts Misses Here we compare of 5 day model forecast with the persistence of CMC ice analyses Note that we use CMC analysis as initial condition, so difference between CMC and model is due to evolution of ice fields in model Images show accumulated statistics of Wednesday forecasts from Jan-Oct, 2011 (43 forecasts) Colour scale shows number of occurences. Values not particularly important, but warm colours show where errors occur most often. Note also that the colour scales are the same for persistence and model forecasts. Values near zero are shown in white. Misses: Model able to forecast ice edge (formation and/or advection) with some skill. In particular: Lab Sea, GIN seas, Barents, Bering. False Alarms: Model shows increased false alarms in many or the places where misses were reduced. Likely due to biases in atmospheric forcing as well as over-formation (weak SST constraint in SAM2 near ice edge and SST not affected by ice assimilation) and excessive offshore advection of ice (due perhaps to lack of land-fasting? Or excessive wind stress? TBD) False Alarms

Accumulated statistics for 5 day lead Jan-Dec, 2011 (52 forecasts) Ice? YIMS NIMS YFcst Hit ice False Alarm NFcst Miss Hit water ΔPCI ( Fcst – Pers) Proportion Correct Ice PCI= Hits ice / (Hits + Misses) Skillful forecasts along ice edge Some errors near coast Proportion Correct Water PCW= Hits water / (Hits water + False Alarms) Small errors along ice edge Correct lead formation near coast ΔPCW ( Fcst – Pers) PCI: Proportion Correct Ice = Correct Ice / ( Correct Ice + Misses) Varies from 0-1 (1=perfect) PCW: Proportion Correct Water = Correct No ice / ( Correct no ice + False Alarms ) Red: Forecast better, Blue: Forecast worse PCI: Can see that model forecasts are better for most ice edges, with errors mostly along coastlines (error from landfast ice or wind stress?) PCW: Again, errors along ice edge (over formation or over advection?)

10 day forecast for 2011-02-05 Persistence of ice analysis YIMS NIMS YFcst Hit ice False Alarm NFcst Miss Hit water 10 day forecast for 2011-02-05 Persistence of ice analysis Model Forecast Misses Colours show number of Misses (top) and False Alarms (bottom) for forecast valid Feb 5, 2011. Red contours show rough position of ice edge (ie “Hit Ice”) Persistance misses large formation event along ice edge in Lab Sea and Baffin Bay right down to Newfoundland Forecast captures well the formation event Forecast overestimates formation in Baffin Bay and near Hudson Strait (and E. Greenland) Interesting detail: Along Lab Coast forecast shows misses and false alarms. Misses mostly right at coast, whereas false alarms are slightly offshore: this points to either excessive wind stress or lack of landfasting as a possible issue.

10 day forecast for 2011-02-05 Persistence of ice analysis YIMS NIMS YFcst Hit ice False Alarm NFcst Miss Hit water 10 day forecast for 2011-02-05 Persistence of ice analysis Model Forecast Misses Colours show number of Misses (top) and False Alarms (bottom) for forecast valid Feb 5, 2011. Red contours show rough position of ice edge (ie “Hit Ice”) Persistance misses large formation event along ice edge in Lab Sea and Baffin Bay right down to Newfoundland Forecast captures well the formation event Forecast overestimates formation in Baffin Bay and near Hudson Strait (and E. Greenland) Interesting detail: Along Lab Coast forecast shows misses and false alarms. Misses mostly right at coast, whereas false alarms are slightly offshore: this points to either excessive wind stress or lack of landfasting as a possible issue. False Alarms

Accumulated statistics for 5 day lead Jan-Dec, 2011 (52 forecasts) Ice? YIMS NIMS YFcst Hit ice False Alarm NFcst Miss Hit water ΔPCI ( Fcst – Pers) Proportion Correct Ice PCI= Hits ice / (Hits + Misses) Skillful forecasts along ice edge Some errors near coast Proportion Correct Water PCW= Hits water / (Hits water + False Alarms) Small errors along ice edge Correct lead formation near coast ΔPCW ( Fcst – Pers) PCI: Proportion Correct Ice = Correct Ice / ( Correct Ice + Misses) Varies from 0-1 (1=perfect) PCW: Proportion Correct Water = Correct No ice / ( Correct no ice + False Alarms ) Red: Forecast better, Blue: Forecast worse PCI: Can see that model forecasts are better for most ice edges, with errors mostly along coastlines (error from landfast ice or wind stress?) PCW: Again, errors along ice edge (over formation or over advection?)

Summary Two models, two analyses, two evaluation methods : Results highlight similarities and differences between systems Cause of ice edge errors : Sensitivity to P* Wave-ice interactions Landfast ice Atm-ice and ice-ocean drag Need for atmosphere coupling Model is slow to melt in spring and slow to form ice in fall Could a short-range forecasting intercomparison be a useful addition to AOMIP? E.g. Philips (2004), Rodwell (2007)

Thank you! Main contributors & collaborators Francois Roy, CMDN Matteusz Reszka, CMDA Jean-Marc Belanger, RPN-E Frederic Dupont, CMDN Gilles Garric, Mercator-Ocean Jean-Francois Lemieux, RPN-E Christiane Beaudoin, RPN-E Fraser Davidson, DFO Youyu Lu, DFO Hal Ritchie, RPN-E Mark Buehner, ARMA Alain Caya, ARMA Tom Carrieres, CIS Zhongjie He, RPN-E Simon Higginson, DFO … Model error for 2010 – 120hr lead

Extras…

Summary Gulf of St. Lawrence coupled atmosphere-ice-ocean forecasting system has demonstrated potential benefits of coupling to polar prediction Through CONCEPTS EC-DFO-DND collaboration, global and regional coupled atmosphere-ice-ocean forecasting systems under development Over the next 4 years, EC will be developing an integrated marine prediction system (atmosphere, snow, ice, ocean, waves) to provide high-resolution analyses and forecasts of marine conditions over these new METAREAS.

Plans for Global Ice-Ocean Prediction System Continue routine analysis production and evaluate analyses and forecasts With bias correction to SAT and SW/LW radiation Possible transfer of system to operations in 2012 Further Development: Combine SAM2 with 3DVAR ice analysis system Improve feedback between ice and ocean analyses (e.g. SST in MIZ) Daily analyses Coupled experiments

EC’s involvement in METAREAs Development of an integrated marine Arctic prediction system in support of METAREA monitoring and warnings. Produce short-term marine forecasts using a regional high-resolution coupled multi-component modelling and data assimilation system Atm, sea ice, ocean, snow, wave Improved Arctic monitoring

Summary and Plans for Regional Ice Prediction Demonstrated skill of ice-only forecasting model Verification remains a challenge… Strong sensitivity to: Analyzed SST in MIZ Lack of fast ice Ongoing efforts: Coupling with ocean NEMO, 3-8km configuration Coupling with atmosphere GEM, 10km GEM RDPS 10km

Case study: Oct. 7, 2010 (33h lead) Ice formation in Beaufort Sea Forecast with dynamics only shows some improvement over persistence but misses formation Forecast with full model captures about half of ice formation Accurate initial SST near ice edge critical 33h Forecast Fcst(dyn)-Radarsat Fcst(full)-Radarsat Pers-Radarsat

METAREA Ice Forecasting System C. Beaudoin, J.-F. Lemieux Results show significant forecast skill in ice concentration Equivalent contribution from currents and wind forcing on Labrador Coast Evaluating separate contribution from dynamics and thermodynamics Planned operational implementation for 2012 Development of coupled atmosphere-ice-ocean configuration underway Verification against Radarsat (24hr lead) Lab Coast – May10 Persistence Forecast (no currents) Forecast (wt currents) East Arctic – Jan10 Persistence Forecast (dyn only) Forecast (wt thermo) Pers Fcst-dyn only Fcst wt thermo