What does a quantitative and “intelligent” model-data comparison mean? André Paul, Stefan Mulitza and Michal Kucera

“Until now, most paleoclimate model-data comparisons have been limited to simple statistical evaluation and simple map comparisons.”

Joel Guiot et al. (1999) “Until now, most paleoclimate model-data comparisons have been limited to simple statistical evaluation and simple map comparisons.”

Paleoclimate data is generally –sparse –irregularly spaced –characterized by different spatial scales ➔ Should aim for model-data comparison without interpolation to common grid

Example Sensitivity of simulated (present-day) sea- surface δ 13 C DIC distribution to air-sea gas exchange formulation Implementing carbon and oxygen isotopes in the MITgcm in collaboration with: Stephanie Dutkiewicz, Jake Gebbie, Annegret Krandick, Takasumi Kurahashi-Nakamura, Martin Losch, Olivier Marchal, Stefan Mulitza, Thejna Tharammal, Rike Völpel

Air-sea gas exchange formulation Wanninkhoff (1992): Krakauer et al. (2006): quadratic square-root

Simulated δ 13 C DIC compared to observations Data from Gruber and Keeling (2001), mean ~1.6 permil subtracted Model mean ~1.6 permil subtracted Surface values for 1990, Krakauer et al. (2006) gas exchange

Simulated δ 13 C DIC compared to observations Data from Schmittner et al. (2013), mean ~1.5 permil subtracted Model mean ~1.5 permil subtracted Surface values for , Krakauer et al. (2006) gas exchange

PMIP-MARUM-PAGES Workshop “Comparing Ocean Models with Paleo-Archives” (Bremen, Germany, 18–22 March 2012) - participants: Ballarotta M, Braconnot P, Brady E, Karami P, Chen MT, Crucifix M, Fischer N, Dail H, de Garidel- Thoron T, Gebbie J, Groeneveld J, Harrison S, Hertzberg J, Jungclaus J, Kageyama M, Kucera M, Kurahashi T, Laepple T, Lea D, Mariotti V, Merkel U, Milker Y, Mix A, Mulitza S, Paul A, Prange M, Rosell-Melé A, Roy T, Schneider B, Shumilovskikh L, Tharammal T, Waelbroek C, Zhang X  PAGES News 20(2):102, 2012

Conclusion of COMPARE workshop: “Future model-data comparisons need to be quantitative and ‘intelligent’ (that is, diagnostic and process-oriented)”

1.How to quantify model-data misfit? 2.How to quantify model error? 3.What “metrics” can paleo-ocean data provide?

How to quantify model-data misfit? Data assimilation: minimize single measure of model-data misfit (“cost”)

How to quantify model-data misfit? Benchmarking: allows for multiple “metrics” –More than one/less than 20 may be useful

Model mean Data mean BiasDifferences in means Model standard deviation Data standard deviation Ratio of standard deviationsVariability Correlation coefficientSpatial patterns Pattern error or “centered RMSE” Root mean square error (RMSE)General deviation How to quantify model-data misfit?

OCMIP2Krakauer et al. (2006) GK2001Sch2013GK2001Sch2013 Model mean Data mean Bias Model standard deviation Data standard deviation Ratio of standard deviations Correlation coefficient Pattern error/centered RMSE Root mean square error (RMSE) How to quantify model-data misfit?

Beyond point-by-point: comparing similarity of patterns even if shifted Hagaman distance between two “left-right fuzzy numbers” Guiot et al. (1999) BD Bhattacharyya distance between probability density functions Bhattacharyya (1943), Ilyana et al. (2013)

How to quantify model-data misfit? Calculating the Hagaman distance between two maps (Guiot et al., 1999, Fig. 5) left-right fuzzy number ModelData

How to quantify model error? Model error, structural error or bias: due to errors in “physics” or parameterizations –Identify discrepancy that remains after optimizing model parameters (  δ 13 C DIC bias in South Atlantic Ocean) –In theory perfect observations are required

What “metrics” can paleo-ocean data provide? IPCC WG1 AR5, Fig Hagaman distance (Guiot et al., 1999) and normalized mean-square error (P. Gleckler et al., 2008) just for annual MARGO (2009) SST

What “metrics” can paleo-ocean data provide? IPCC WG1 AR5, Fig Bottom-water temperature and salinity at just two ODP sites

What “metrics” can paleo-ocean data provide? Further temperature metrics: tropical cooling and east-west gradients Map of gridded (5°x5°) MARGO (2009) LGM SST annual mean anomalies for the 30° S-30° N tropical band, including location of data points

What “metrics” can paleo-ocean data provide? Property and density gradients: –Glacial North Atlantic chemocline (in terms of benthic δ 13 C DIC and δ 18 O c ) –Density gradients that drive AMOC Atlantic north-south density gradient ρ Atl between southern tip of Africa and regions of deep water formation (Schmittner et al., 2002) Density contrast ρ SN between AABW and NADW (e.g. Weber et al., 2008) – 231 Pa/ 230 Th (Lippold et al., 2012)

Conclusions Design metrics that –go beyond point-by-point comparison –test for entire processes Include quantities other than temperature –E.g., sea ice, density, nutrient or productivity proxies Average paleo-ocean data over regions or water masses Extend model-data distance calculation to variable horizontal and vertical resolution –Develop easy-to-use software package

Conclusions Many “metrics” are available that may be adapted to paleoclimate (and in particular paleocean) model-data comparions. Paleo-metrics seem indeed to be conclusive in spite of uncertainties.

Cf. Dutkiewicz et al. (2005) Implementing carbon isotopes atmosphere gas exchange production of organic carbon and calcium carbonate remineralization Organic production and remineralization at every depth level MITgcm carbon cycle component (“DIC package”):

Additional state variables DO 13 C, DO 14 C, DI 13 C, DI 14 C, p 13 CO 2 and p 14 CO 2 Fractionation of carbon isotopes during photosynthesis and air-sea gas exchange Closed carbon cycle –using “real freshwater flux” boundary condition, nonlinear free surface and balanced precipitation and evaporation (no “virtual fluxes” or “salinity restoring”) Cf. Marchal et al. (1998), Schulz (1998)

1.Fractionation during air-sea gas exchange: where: Furthermore, and Zhang et al. (1995) Depends on water temperature and carbonate chemistry ([CO 3 2- ]) at surface

Jasper et al. (1994), Zhang et al. (1995) 2.Fractionation during photosynthesis: Depends on water temperature and carbonate chemistry ([CO 3 2- ], [CO 2(aq) ]) in photic zone

2.Fractionation during photosynthesis:

Testing sensitivity to air-sea gas exchange formulation Wanninkhoff (1992): Krakauer et al. (2006):

Preliminary results Pre-industrial equilibrium experiments forced by atmospheric concentrations: –p 13 CO 2 = 278 μatm – δ 13 C atm = -6.5 permil –δ 13 C atm = 0 permil

Outlook (1) Study sensitivity to –wind-speed field (SSM/I vs. ECMWF) –maximum value of organic production –fractionation coefficients –… Your suggestion(s)?

Implementing water isotopes Added state variables (mass ratios) R H 2 16 O, R H 2 18 O, R HD 16 O Isotopic content of precipitation and water vapor obtained from NCAR Community Atmospheric Model with isotopes (IsoCAM) Kinetic fractionation during evaporation treated explicitly in ocean model

Data from NASA GISS Global Seawater Oxygen-18 Database Model results by Annegret Krandick

Outlook (2) Force with LGM boundary conditions –From NCAR IsoCAM (by Thejna Tharammal, David Noone) –Compare to LGM δ 18 O c (e.g., by the MARGO project)