1. FAIRMODE WG2 – SG4 activity

2. Outcome of the November 2009 meeting What happened in 2010 up to now
Outline
Outcome of the November 2009 meeting
What happened in 2010 up to now
Current situation: Procedure & developments
Agenda for SG4

3. Outcome of the 2009 WG2 meeting
Development of a prototype tool for benchmarking models used under the AQD directive
Focus on urban agglomeration
Open to Lagrangian and Eulerian models, first
Prototype flexibility requirements
Emission disagragation to allow local scale models contribution (line sources)
Open source code
Allows multiscale applications - regional first
Links to MACC, TFMM, EURODELTA, MEGAPOLI, CITYZEN, AQMEII

4. Special session at Harmo13 (June)
2010
Review of current methodologies (evaluation tools, inter-comparison projects…)
AMET, ENSEMBLE, CityDelta, Eurodelta, BOOT…
EPA, AEA, AIR4EU, COST…
New approaches (Jolliff, Boylan…)
Document “ Procedure for AQ models Benchmarking” sent to SG4 participants (May)
Special session at Harmo13 (June)
Development of the benchmarking prototype is on-going

5. Benchmarking procedure: objectives
Develop a procedure for the benchmarking of AQ models in order to evaluate their performances.
Applications: Assessment & planning
Scales: Regional, urban and local
Support both model users & model developers in the implementation of the AQD
Provide technical/scientific support to policy on the analysis of the quality of model results
Identify a common and permanent scale for model evaluation
Update the procedure and assess model quality improvement based on periodic joint exercises (= every x years)

6. Benchmarking procedure: Key elements
USER
JRC
Data Extraction Facility
Model results
DELTA
BENCHMARKING
service

7. Benchmarking procedure: Key elements
The BENCHMARKING service:
Selection of a core set of statistical indicators and diagrams for a given model application in the frame of the AQD
Provide indicative bounds related to model performances (goals & criterias)
Organised around different testing levels
Production of summary performance reports based on a common model evaluation scale

8. Benchmarking procedure: Key elements
USER
JRC
Data Extraction Facility
Model results
DELTA
BENCHMARKING
service
Model performance evaluation reports

9. Benchmarking procedure: work plan
Discussion and consensus on overall methodology (FAIRMODE meeting 09/2010)
Development of the DELTA and benchmarking service prototypes (Dec 2010)
Testing of the prototypes on existing datasets (2011)
Development of the JRC Web facilities (data extraction, links ENSEMBLE-Benchmarking service…)
Set-up of a joint exercise for testing of the whole system (2012)

10. Links to other SGs (station representativeness, emissions…)
Points for discussion
Discussion and definition of the benchmarking service elements (species, statistics, bounds…) for model performance reporting.
Links to other SGs (station representativeness, emissions…)
Definition of and participation to the joint activities
Practical: organisation & communication

11. SG4 participants - EoI Person Institute Marco BEDOGNI AMAT (IT)
Jana MATEJOVICOVA
HMI (SK)
Bertrand BESSAGNET
INERIS (FR)
Anna MIRANDA
University of Aveiro (PT)
David CARRUTHERS
CERC (UK)
Dietmar OETTL
Government of Styria (AT)
Koen DE RIDDER
VITO (BE)
Christakis PAPADOPOULOS
DLI (CY)
Dick DERWENT
AEA (UK)
Guido PIROVANO
ERSE (IT)
Hilde FAGERLI
Met Inst (NO)
Peter ROBERTS
CONCAWE (BE)
Alenka FRITZEL
Ministry of Environment (SI)
Martijn SCHAAP
TNO (NL)
Stefano GALMARINI
EC-JRC, Ispra
Camillo SILIBELLO
ARIANET (IT)
Emilia GEORGIEVA
Alexandros SIRAKOS
UOWM (GR)
Maria GONCALVES
BSC (ES)
Philippe THUNIS
Pernige GRENNFELT
IVL (SE)
Keith VINCENT
Ari Karppinen
FMI (FI)
Rahela ZABKAR
University of Ljubljana (SI)
Fernando MARTIN
CIEMAT (ES)
John BARTZIS
Jose BALDASANO
Nicolas MOUSSIOPOULOS
AUT (GR)
John DOUROS
Leonor TARRASON
NILU (NO)
Bruce DENBY
H. BECKSTROEM
SMHI (SE)
Joanna STRUZEWSKA
WOT (PL)
Nutthida KITWIROON
HERTS (UK)
Marcus Hirtl
ZAMG(AT)
Ruben Beijk
RIVM (NL)
Joost Wesseling