Harnessing butterflies for seasonal and interannual Predictions Stochastic Seasonal to Interannual Prediction System Harnessing butterflies for seasonal and interannual Predictions CAP, 13 June, 2016 S. Lovejoy, L. Del Rio Amador, McGill, Montreal

StocSIPS* with SLIMM** Forecasts from months to decades: The unsuspected Elephantine (“long range”) memory StocSIPS* with SLIMM** 10% of the information needed for global seasonal temperature forecasts comes from fluctuations more than 300 years old… But we can (almost) do it! *StocSIPS= Stochastic Seasonal and Interannual Prediction System **SLIMM= ScaLIng Macroweather Model

Statistical Mechanics Low level (fundamental) deterministic stochastic Vortices in strongly turbulent fluid (M. Wiczek, numerical simulation, 2010) Continuum mechanics Thermodynamics Higher level Continuum limit Fluctuations ≈ (turbulent flux) x (scale)H Higher level Laws of turbulence High Reynolds number limit Richardson, Kolmogorov, Corrsin, Obukhov, Bolgiano Long time limit >10days Space-time factorization Macroweather laws

Temperature Change (oC) Power laws H=0.4 H=-0.4 H=0.5 H = -0.7 Temperature Change (oC) Montreal Weather EPICA (Antarctic) ±2K ±3K Ice ages 2ox2o (75oN) 5oC Macroweather macroclimate 2oC 10 days 1yr 2 hours 10,000 yrs 10 yrs 100 yrs 1 oC 1000 yrs 106 note the low freq. climate slope is about -0.8 100,000 yrs 0.5oC climate

Trichotomy: Weather – macroweather - climate H ≈ 0.4: Fluctuations Growing H ≈ -0.4: Fluctuations Decreasing = constant Fluctuation 1 Century, Vostok, 20-92kyr BP Climate (30-100 yrs to 50,000 yrs) 14 Temperature (oC) 12 20 days, 75oN,100oW, 1967 - 2008 Macroweather (10 days to 30 -100 yrs) 2 4 6 8 10 1 hour, Landers, 10 Feb.-12 March, 2005 0.067 s, Rutherford Physics, 5:05 pm Nov. 4, 2004 Weather (up 10 days) The series are nomlized and then displaced by 4 for clairty bottom is Landers (dertended yearly and daily), middle is 75N, long=100o 20 day averages, top is Vostok at 1 centruy res 20kysBP to 92kyrsBP Sd’s: sdVostok= 1.39498, sdlowfreq = 2.59006, sdLanders = 4.48689 orange is 1967 - present with 20 day average Blue is hourly, Feb 10 - March 12, 2005, The bottom (new is from the 3.8m thermisotor at 15Hz; the s.d. was 0.35 K H=0.4 also Ponk is 20kyrBP-92kyrBP - 100 200 300 400 500 600 700 t Lovejoy, 2013, EOS

Beyond their deterministic limit: GCMs stochastic “Brute force” Weather systems (<10 days) generated by GCMs = random weather noise (statistics)… but not fully realistic Averages: slow convergence to Model climate Scaling laws generate realistic (empirically based) statistics (noise) Our climate Potential advantages of direct stochastic macroweather (>1 month) forecasting: More realistic weather “noise” (statistics: based on empirical data, not constrained by model). Ability to use empirical data to force convergence to the real climate.

Statistical characteristics of Macroweather Temporal domain - Low intermittency Gaussian theory (except for extremes). - Scale symmetry ≈ 1 month- >100 years (anthropocene ->30 years) Fluctuations tend to cancel: H<0. - Theoretical stochastic limits to forecast skill: theoretical “benchmark” - Scale symmetry: huge memory Spatial domain - Scale symmetry up to ≈ 4000 km. - Strong (multifractal) intermittency: climatic zones Space-time Statistical space-time Factorization. Strong spatial correlations do not give useful information for forecasting: single grid or single station forecasts close to the theoretical maximum.

Statistical space-time Factorization (Ex: factorization of second order statistics) Spectral densities: Weather, no factorization Space-time Scale function Typical form Macroweather: factorization Structure Functions: Implies size – lifetime relation Macroweather: factorization No relation between size and lifetime

GISS E2R temperatures (historical run since 1850) Factorization: GCM’s GISS E2R temperatures (historical run since 1850) Log10S(Δq) Factorization: Scaling: Factorization implies parallel, identical curves Δt 0.5 1 months 2 Log10Δq 4 8 16 1.0 1.5 2.0 32 64 128 - 0.5 Fluctuations decrease as the time-averaged interval increases. 256 512 - 1024 1.0 1872 80o

Deterministic predictability limits for Weather forecasts Sensitive dependence on initial conditions leads to limits (“butterfly effect”). Fundamental limit = error doubling time. The doubling time increases with the lifetime (hence size). For planetary structures ≈ 10 days. Weather forecast skill can be judged by how close the doubling time is to the 10 day limit. This is the basic benchmark for deterministic weather forecasts

Stochastic predictability limits for Macroweather forecasts Temporal scaling and statistical space-time factorization imply stochastic predictability limits. Skill = 1 - (forecast error variance)/(temperature variance) Theoretical skill = F (lead time, H(x)) the exponent H(x) and hence skill varies with position x. This is the basic benchmark for macroweather forecasts -For 5ox5o resolution monthly and seasonal StocSIPS forecast skill =25%, 18%: theoretical limits = 29%, 21%, respectively. -StocSIPS forecasts are on average 86% of the theoretical limit. Comments: -The skill has only a weak dependence on spatial resolution… hence we can avoid downscaling. Can GCM’s improve on the theoretical stochastic limits? Not obvious since the GCM’s appear to satisfy space-time factorization and temporal scaling

Hasselman (1976) type stochastic Macroweather processes b=2 log10w log10E(w) (10 days)-1 b=0 macroweather weather Low frequency (b=0) White noise (no predictability) (Orenstein Uhlenbeck processes, Linear Inverse Modelling, Auto Regressive processes) The Hasselman (1976) type stochastic approach: White noise E(w)≈w-b

Scaling stochastic Macroweather processes log10E(w) red noise (potentially huge predictability) White noise Scaling, stochastic approach (Ignoring intermittency) -1/2<H<0 (fluctuation exponent) b≈1+2H b≈1.8 Empirical, turbulence Empirical, macroweather E(w)≈w-b b=0 Hasselman type process b=2 (10 days)-1 log10w

Gaussian white noise: H= -0.5 Skill Skill as a function of forecast lead time 1.0 S k Skill = 1- (Error variance)/(temperature variance) 0.8 0.6 58% (seasonal, hindcasts) 61% (monthly, hindcasts) 68% (theory analytical) 64% (theory numeric) Global H = -0.085 Low frequency LIM: H = -1/2 0.4 Dimensionless Forecast horizon/resolution l=t/t 1 2 4 0.2 Gaussian white noise: H= -0.5 64 - 0.5 - 0.4 - 0.3 - 0.2 - 0.1 H H = 0: Pure “1/f” noise Land Ocean Skill for horizons 1, 2, 4, 8, ... 64 units away GOOD!!!!! Finf is good one

Fluctuations contributing to 90% of the theoretical maximum skill - 0.5 0.4 0.3 0.2 0.1 H 1 2 3 4 Log 10 l mem The unsuspected Elephantine (“long range”) memory Fluctuations contributing to 90% of the theoretical maximum skill Global 1200 Ex: with H = -0.085, fluctuations over the last 1200 seasons (300 yrs) are responsible for 90% of the seasonal variability Ocean: PDO 600 50 15 the Log10 memory needed to give 90 % of the maximum skill at horizon = 1 forecast 9 White noise Pure “1/f” noise

Preprocessing of the data: Ref: (NCEP/NCAR) Example for the grid point (-72.5, 47.5), Montreal Raw Data and Annual Cycle Raw Data Trend Annual Cycle Anomalies

𝛽=1+2H Spectrum and Fluctuation Analysis Spectra pre and after processing annual peak Montreal 𝛽=1+2H Global spectra pre and after processing annual peak Global (36x72 pixels ensemble) 𝛽=0.46±0.02 Fluctuation Analysis Global (36x72 pixels ensemble) 𝐻=0.27±0.01

“The ‘closest witnesses’ to the unobserved past have special weight” Scaling LInear Macroweather model (SLIMM) Prediction of fGn Gaussian noise Power law correlation. Vast memory that can be exploited. Predictor for -0.5 < H < 0 based on past data. Kernel for H = -0,1. kernel Weight of present predictor data Weight of the distant past “The ‘closest witnesses’ to the unobserved past have special weight”

Theoretical and numerical Skills. Monthly resolution. Period Sep, 1980 - Dec, 2015. Reference: ERA-Interim Reanalysis Theory MSSS, lead time = 1 month <r> = 0.95 1-r Numerical MSSS, lead time = 1 month

StocSIPS relative advantage increases with lead time A = Percentage of the globe where MSSSStocSIPS.MSSSCanSIPS Diff=MSSSStocSIPS-MSSSCanSIPS lead time = 1 month, A=26% lead time = 2 months, A=55% lead time = 6 months, A=65% lead time = 9 months, A=69%

Skills StocSIPS and CanSIPS: Comparison Annual resolution, lead time = 2 years for StocSIPS. Monthly resolution, lead time = 6 months for CanSIPS MSSS StocSIPS, annual, lead time = 2 years MSSSStocSIPS (2 years) - MSSSCanSIPS(6 months) 𝐴 =69% MSSS CanSIPS, monthly, lead time = 6 months <Diff>=0.13 sdiff=0.22

Relative Skill of StocSIPS increases with lead time MSSSStocSIPS>MSSSCanSIPS for more than one month MSSSglobal = 1-<MSE>/Varanom Mean Square Error 𝑀𝑆𝐸 - Varanom = Variance of the anomalies

Global Producing Centers: Actuals GPC- Beijing, JFM GPC-Exeter Anthropogenic Natural variability Actuals For actuals (negative skill) GPC-Melbourne GPC-Toulouse GPC-Montreal GPC-Washington 𝑴𝑺𝑬=𝟓𝟎𝟎 % 𝐨𝐟 𝑽𝒂𝒓 𝒂𝒏𝒐𝒎

Skills StocSIPS and CanSIPS: Actuals MSSS StocSIPS, monthly, lead time = 1 month MSSS StocSIPS, annual, lead time = 1 year MSSS StocSIPS, seasonal, lead time = 1 season MSSS CanSIPS, monthly, lead time = 1 month

Hindcasts Skill: Actuals Diff=MSSSStocSIPS--MSSSCanSIPS 𝐴 =93% MSSSglobal = 1-<MSE>/Varanom <Diff>=3.7 Anthropogenic Natural variability Actuals For actuals MSE>Var (anom) (negative skill)

Probabilistic Forecast Forecast probability Climatological probability If all probabilities < 40% then grey

Probabilistic Forecast: verification and validation Global average PC = 48,5% (Same as CanSIPS for Canada) A = 95% of the globe with PC > 33%

Visit our site http://www.physics.mcgill.ca/StocSIPS

Conclusions Theoretical basis of StocSIPS StocSIPS performance -Long term memory (scaling, one parameter, H) -Space-time factorization (space-time prediction decoupling) -Stochastic predictability limits StocSIPS performance -Anomalies: StocSIPS has higher skill than CanSIPS for two months and longer. -Actuals: Higher skill at all lead times due to direct forecasting of climatology. -StocSIPS relative advantage: increases with lead time and is higher over land than oceans. StocSIPS’ advantages include -No data assimilation -No ad hoc post processing -No need for downscaling -Speed: (factor 105- 106) Future developments -Prediction of other fields (precipitation, wind, solar insolation, degree-days, forest fire indices, drought indices). -Find co-predictors such as El Niño indices. -Prediction of Extremes http://www.physics.mcgill.ca/StocSIPS