Presentation is loading. Please wait.

Presentation is loading. Please wait.

Do different machines simulate different climates? François Massonnet M. Asif, O. Bellprat, E. Exarchou, M. Ménégoz, C. Prodhomme, F. J. Doblas-Reyes EC-Earth.

Published byMaude Williamson Modified over 9 years ago

Similar presentations

Presentation on theme: "Do different machines simulate different climates? François Massonnet M. Asif, O. Bellprat, E. Exarchou, M. Ménégoz, C. Prodhomme, F. J. Doblas-Reyes EC-Earth."— Presentation transcript:

1 Do different machines simulate different climates? François Massonnet M. Asif, O. Bellprat, E. Exarchou, M. Ménégoz, C. Prodhomme, F. J. Doblas-Reyes EC-Earth meeting 5-6 May 2015

2 OBS MOD Identifying sources of model error

3 Initial conditions are wrong OBS MOD Identifying sources of model error

4 Initial conditions are wrong Boundary conditions are wrong OBS MOD Identifying sources of model error

5 Initial conditions are wrong Boundary conditions are wrong Model physics is wrong OBS MOD Identifying sources of model error

6 Initial conditions are wrong Boundary conditions are wrong Model physics is wrong Discretization errors OBS MOD Identifying sources of model error

7 Initial conditions are wrong Boundary conditions are wrong Model physics is wrong Discretization errors Softwares / hardwares are wrong OBS MOD Identifying sources of model error

8 Hardware/software as sources of model error Bit-reproducibility of EC-Earth Clim-reproducibility of EC-Earth

9 Hardware/software as sources of model error Bit-reproducibility of EC-Earth Clim-reproducibility of EC-Earth

10 Software / harwdare is a multiple and underestimated source of model error Round-off errors and floating-point representation The order matters: associativity is no longer valid 12.077112.0777 ≠

11 Software / harwdare is a multiple and underestimated source of model error Round-off errors and floating-point representation The order matters: associativity is no longer valid Agressive optimization Can degrade accuracy [Thomas et al. Wea. And Forecast., 2002] 12.077112.0777 ≠

12 Software / harwdare is a multiple and underestimated source of model error Round-off errors and floating-point representation The order matters: associativity is no longer valid Agressive optimization Can degrade accuracy [Thomas et al. Wea. And Forecast., 2002] Number of processors and their distribution Processor topology defines order of operations [Thomas et al., Wea. And Forecast., 2002; Senoner et al., AIAA, 2008] 12.077112.0777 ≠

13 Software / harwdare is a multiple and underestimated source of model error Round-off errors and floating-point representation The order matters: associativity is no longer valid Agressive optimization Can degrade accuracy [Thomas et al. Wea. And Forecast., 2002] Number of processors and their distribution Processor topology defines order of operations [Thomas et al., Wea. And Forecast., 2002; Senoner et al., AIAA, 2008] Compiler version Different FORTRAN compilers can produce different outcomes [Lawrence et al., EOS, 1999] 12.077112.0777 ≠

14 Software / harwdare is a multiple and underestimated source of model error Round-off errors and floating-point representation The order matters: associativity is no longer valid Agressive optimization Can degrade accuracy [Thomas et al. Wea. And Forecast., 2002] Number of processors and their distribution Processor topology defines order of operations [Thomas et al., Wea. And Forecast., 2002; Senoner et al., AIAA, 2008] Compiler version Different FORTRAN compilers can produce different outcomes [Lawrence et al., EOS, 1999] Unpredictable hardware failures [Düben and Palmer, Mon. Wea. Rev., 2014] 12.077112.0777 ≠

15 Hardware/software as sources of model error This aspect has been overlooked but is of non-negligible importance Bit-reproducibility of EC-Earth Clim-reproducibility of EC-Earth

16 Hardware/software as sources of model error This aspect has been overlooked but is of non-negligible importance Bit-reproducibility of EC-Earth Clim-reproducibility of EC-Earth

17 All things being equal, EC-Earth is reproducible bitwise Near-surface global temperature K 287 286 285 284 JFMAMJJASOND 22 +384+96 # of processors Mare Nostrum

18 All things being equal, EC-Earth is sensitive to processor distribution Near-surface global temperature K 287 286 285 284 JFMAMJJASOND 22 +384+96 22 +96 +16 22 +352+96 22 +384+32 # of processors Mare Nostrum

19 Hardware/software as sources of model error This aspect has been overlooked but is of non-negligible importance Bit-reproducibility of EC-Earth -Two identical runs give exactly matching output -No reproducibility for different processor (IFS or NEMO) distribution Clim-reproducibility of EC-Earth

20 Hardware/software as sources of model error This aspect has been overlooked but is of non-negligible importance Bit-reproducibility of EC-Earth -Two identical runs give exactly matching output -No reproducibility for different processor (IFS or NEMO) distribution Clim-reproducibility of EC-Earth

21 Assessing climate-reproducibility is not straightforward Three necessary conditions to validate ports of global atmospheric GCMs [Rosinski and Williamson, 1995] 1. During the first few time steps, difference between two runs should be much less than rounding error, 2. During the first few days, difference should not exceed error growth induced by bit-change on a machine, 3. The statistics of a long simulation must be unchanged.

22 Restart from EC-Earth v3.1b 500 yr-spinup (CNR: tome) + white noise SST (σ=10 -4 K) 18501870 5 members 1860 Forcing: pre-industrial

23 Machine 1 (Mare Nostrum, BSC) Machine 2 (ECMWF) Machine 3 (Ithaca, CFU) Motherboard Operating system LINUX environment Compilation flags Identical NetCDF, GRIB, HDF5 libraries Different # of processors 22+32+1622+480+9622+384+96 Autosubmit ensures identical configurations

24 The simulations are evaluated against the same, static data set ECMWF Ithaca, IC3 Mare Nostrum, BSC Reference [Reichler and Kim, BAMS, 2008] [ECMean: Paolo Davini, CNR]

25 Statistically significant differences are found in the performance indices ECMWF Mare Nostrum Red = difference according to Kolmogorov-Smirnov test (alpha=5%; overestimates true rejection rate for small samples)

26 Difference (ECMWF − Mare Nostrum) near-surface temperature °C

27 10 8 6 4 2 0 10 6 km² 18 12 0 15 9 6 3 ECMWF Mare Nostrum ECMWF Mare Nostrum Differences originate in winter. Associated to deep oceanic convection and parameterizations?

28 Hardware/software as sources of model error This aspect has been overlooked but is of non-negligible importance Bit-reproducibility of EC-Earth -Two identical runs give exactly matching output -No reproducibility for different processor (IFS or NEMO) distribution Clim-reproducibility of EC-Earth EC-Earth is not climate-reproducible on different platforms

29 Take home messages and implications 1.Machines introduce an additional, non negligible source of error in climate simulations The results are supported by several published studies, unpublished documents, colloquial discussions, good practices 2.Following the precautionary principle, CMIP6 simulations should be centralized… … unless a benchmark experiment is produced to re-assess reproducibility with the next EC-Earth version 3.Model evaluation should account for dependency of results on software/hardware Machines sample uncertainty just as members sample internal variability 4.Control and sensitivity experiments must be designed on the same machine

30 Thank you! francois.massonnet@uclouvain.be www.climate.be/u/fmasson @FMassonnet

Download ppt "Do different machines simulate different climates? François Massonnet M. Asif, O. Bellprat, E. Exarchou, M. Ménégoz, C. Prodhomme, F. J. Doblas-Reyes EC-Earth."

Similar presentations

Introduction of Version 2.0b Significant changes to sediment model –Box volumes are no longer static. –Net sedimentation is constrained by box geometry.

Introduction of Version 2.0b Significant changes to sediment model –Box volumes are no longer static. –Net sedimentation is constrained by box geometry.
Mercator Ocean activity

Mercator Ocean activity
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Improving COSMO-LEPS forecasts of extreme events with.

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Improving COSMO-LEPS forecasts of extreme events with.
Geophysical Fluid Dynamics Laboratory Review June 30 - July 2, 2009 Geophysical Fluid Dynamics Laboratory Review June 30 - July 2, 2009.

Geophysical Fluid Dynamics Laboratory Review June 30 - July 2, 2009 Geophysical Fluid Dynamics Laboratory Review June 30 - July 2, 2009.
ENSEMBLES RT4/RT5 Joint Meeting Paris, 10-11 February 2005 Overview of Seasonal-to-Decadal (s2d) Activities during the Initial 18 Months Francisco J. Doblas.

ENSEMBLES RT4/RT5 Joint Meeting Paris, February 2005 Overview of Seasonal-to-Decadal (s2d) Activities during the Initial 18 Months Francisco J. Doblas.
Seasonal-to-decadal forecasting with EC-Earth at IC3/BSC F.J. Doblas-Reyes ICREA, BSC and IC3, Barcelona, Spain.

Seasonal-to-decadal forecasting with EC-Earth at IC3/BSC F.J. Doblas-Reyes ICREA, BSC and IC3, Barcelona, Spain.
Climate Forecasting Unit Ocean and sea ice initialization and prediction activities at Climate Forecasting Unit (CFU), Catalan Institute of Climate Sciences.

Climate Forecasting Unit Ocean and sea ice initialization and prediction activities at Climate Forecasting Unit (CFU), Catalan Institute of Climate Sciences.
Chapter 8 Introduction to Hypothesis Testing

Chapter 8 Introduction to Hypothesis Testing
Grid for Coupled Ensemble Prediction (GCEP) Keith Haines, William Connolley, Rowan Sutton, Alan Iwi University of Reading, British Antarctic Survey, CCLRC.

Grid for Coupled Ensemble Prediction (GCEP) Keith Haines, William Connolley, Rowan Sutton, Alan Iwi University of Reading, British Antarctic Survey, CCLRC.
Barcelona, 2015 Climate Prediction and Climate Services Virginie Guemas and the Climate Forecasting Unit 9 February 2015.

Barcelona, 2015 Climate Prediction and Climate Services Virginie Guemas and the Climate Forecasting Unit 9 February 2015.
Observers-Modelers Observations-Models Same struggle? François Massonnet Barrow workshop 29 Apr – 31 May 2015.

Observers-Modelers Observations-Models Same struggle? François Massonnet Barrow workshop 29 Apr – 31 May 2015.
Using CCI data in the SPECS research project

Using CCI data in the SPECS research project
Barcelona, 2015 Ocean prediction activites at BSC-IC3 Virginie Guemas and the Climate Forecasting Unit 9 February 2015.

Barcelona, 2015 Ocean prediction activites at BSC-IC3 Virginie Guemas and the Climate Forecasting Unit 9 February 2015.
Observing Strategy and Observation Targeting for Tropical Cyclones Using Ensemble-Based Sensitivity Analysis and Data Assimilation Chen, Deng-Shun 3 Dec,

Observing Strategy and Observation Targeting for Tropical Cyclones Using Ensemble-Based Sensitivity Analysis and Data Assimilation Chen, Deng-Shun 3 Dec,
The high resolution in seasonal climate predictions with EC-Earth François Massonnet C. Prodhomme, L. Batté, V. Guemas, F. J. Doblas-Reyes Polar Predictability.

The high resolution in seasonal climate predictions with EC-Earth François Massonnet C. Prodhomme, L. Batté, V. Guemas, F. J. Doblas-Reyes Polar Predictability.
Climate Forecasting Unit Arctic Sea Ice Predictability and Prediction on Seasonal-to- Decadal Timescale Virginie Guemas, Edward Blanchard-Wrigglesworth,

Climate Forecasting Unit Arctic Sea Ice Predictability and Prediction on Seasonal-to- Decadal Timescale Virginie Guemas, Edward Blanchard-Wrigglesworth,
ISDA 2014, Feb 24 – 28, Munich 1 Impact of ensemble perturbations provided by convective-scale ensemble data assimilation in the COSMO-DE model Florian.

ISDA 2014, Feb 24 – 28, Munich 1 Impact of ensemble perturbations provided by convective-scale ensemble data assimilation in the COSMO-DE model Florian.
Atlantic Multidecadal Variability and Its Climate Impacts in CMIP3 Models and Observations Mingfang Ting With Yochanan Kushnir, Richard Seager, Cuihua.

Atlantic Multidecadal Variability and Its Climate Impacts in CMIP3 Models and Observations Mingfang Ting With Yochanan Kushnir, Richard Seager, Cuihua.
Systems Development Lifecycle Testing and Documentation.

Systems Development Lifecycle Testing and Documentation.
EUROBRISA Workshop – Beyond seasonal forecastingBarcelona, 14 December 2010 INSTITUT CATALÀ DE CIÈNCIES DEL CLIMA Beyond seasonal forecasting F. J. Doblas-Reyes,

EUROBRISA Workshop – Beyond seasonal forecastingBarcelona, 14 December 2010 INSTITUT CATALÀ DE CIÈNCIES DEL CLIMA Beyond seasonal forecasting F. J. Doblas-Reyes,

Similar presentations

Ads by Google