1. A protocol for model validation ACCENT/GLOREAM Workshop Paris, France, 11-13 October 2006 Peter Builtjes, TNO-the Netherlands and FU-Berlin

2. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation2 I)Introduction II)A first lay-out of a protocol III)Testing of the protocol IV)Discussion

3. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation3 I)Introduction Model validation/evaluation/testing: comparison between calculated and observed concentrations/depositions Observed concentrations should be accurate (the instrument) and spatial representative (in balance with the model grid)

4. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation4 Spatial representative Ask the expert, rural, Educated guess Analysis of concentration patterns in time More stations in one grid Field study with for example passive samplers Determination by modelling-data assimilation For regional scale modelling, using rural stations, +/- 20 % ?

5. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation5 Recent model intercomparison and validation studies: EUROTRAC, EMEP-review, EURO-DELTA Recommendation: No model validation for just one model Combine model intercomparison with model validation

6. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation6 EURODELTA PM2.5 concentrations over Europe, preliminary/confidential first results

7. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation7 Carlos Borrego, AIR4EU Total Model Uncertainty = Model uncertainty + Input data uncertainty + Variability Recommended Quality Indicators: Correlation Coefficient Fractional Bias RMSE (RPE) NMSE

8. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation8 Model intercomparison and validation City Delta and Euro Delta: JRC-graphical tool EMEP model intercomparison: TNO-tool

9. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation9 Taylor diagram for mean Summer ozone Fine-scale and coarse-scale models

10. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation10 Compared to EMEP-review O 3 DAY TIME SUMMER 1999obs. meanmod. meanresidueRMSEcorr.σ-ratio* EMEP_v1.083.3685.3919.1524.780.450.66 MATCH_v1.083.3695.7820.4925.610.570.56 LOTOS_v1.083.3678.2920.9026.280.470.87 LOTOS-EUROS83.3680.1618.9124.360.570.98 SO 4 Year 2001obs. meanmod. meanresidueRMSEcorr.σ-ratio* EMEP_v1.02.302.171.231.870.571.31 MATCH_v1.02.342.781.291.840.621.26 LOTOS_v1.02.343.331.922.960.461.77 LOTOS-EUROS2.341.871.231.810.500.97 NO 3 Year 2001obs. meanmod. meanresidueRMSEcorr.σ-ratio* EMEP_v1.03.333.742.032.980.611.42 MATCH_v1.03.332.881.462.040.610.83 LOTOS_v1.03.333.261.942.710.360.89 LOTOS-EUROS3.333.631.842.520.571.11

11. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation11 In TNO: Quick Scan based on EMEP review tool to test different model versions of the LOTOS-EUROS model + input: Required QA/QC

12. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation12 II)A first lay-out of a protocol Based on discussions at the ACCENT Workshop on Model Benchmarking and Quality Assurance Thessaloniki, 29/30 May 2006 Items of a protocol: a)Define the purpose of the model and of the validation Which output should be validated? Example: Hourly ozone or annual averaged Benzene

13. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation13 b)Identify the processes required in the model Is aerosol chemistry required or not? c)Define the horizontal and vertical resolution, and the time scale of the output d)Concerning the input data, decide which data should be fixed Example: are emissions taken as they are given?

14. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation14 e)Concerning observations, decide about QA/QC and spatial representativity d)Quality indicators should be defined, including a threshold below which the model performance will be considered as inadequate For daily max O 3, the correlation coefficient between calculated and observed should be more than 0.5, based on previous studies e)Sensitivity runs should be defined for key processes, or key input data

15. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation15 III)Testing of the protocol An attempt a)Purpose of the validation for O 3 daily max Summer, over Europe b)Processes required: Only gasphase chemistry, like EMEP, RADM, CBM4 (box model validation needed, see Poppe 1996)

16. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation16 c)Horizontal resolution : 25 x 25 km 2, or 50 x 50 km 2 (for discussion!!) Vertical resolution: 20 layers upto the lower stratosphere, or 5 layers upto 5 km (for discussion!!) Time resolution: hourly, not for discussion d)Fixed input data: Anthropogenic and biogenic emissions Meteorology, prognostic and diagnostic Boundary conditions, MOZART/TM5, or Logan Landuse data base

17. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation17 e)Observations: EMEP only, rural stations Uncertainty +/- 20 % ??? f)Quality indicators: Based on experience in Euro-Delta/EMEP review g)Sensitivity runs: for example: Biogenic emissions Reactivity of anthropogenic VOC-emissions Cloud cover Dry deposition, also over sea

18. Paris, France, 11-13 October 2006ACCENT/GLOREAM Workshop - A protocol for model validation18 IV)Discussion/Statements Always combine model validation with model intercomparison Make ensemble approach and data assimilation an integral part of model validation Work towards toolkit : JRC + TNO-EMEP review The proposed protocol - seems to work for daymax O 3 - should be tested in several projects and by several groups, and based on experience improved upon And finally being “accepted”