2. Data Extraction Facility
USER
JRC
Data Extraction Facility
Model info
Model results
DELTA
Benchmark
Case
&
Report DB
DELTA
BENCHMARKING
service
Model performance
Evaluation reports
Deadline: end 2012

3. What has been done since Oslo meeting
The DELTA tool has been developed and released (about 20 users) together with the document “Delta concepts and user’s guide” (March 2011).
Within this document a template for reporting model performances has been proposed.
A questionnaire has been sent to SG4 participants to get feedback (10 replies) on:
DELTA
Template for reporting performance

4. Objectives of SG4 meeting
Get feedback and discuss the template for reporting model performances:
Content of the performance report
Links between Target and more “traditional” indicators (an analysis based on 3 datasets)
A “first-guess” for criteria/goals
Comparison with RDE
Observation uncertainty
Future plans
Deliverables for TSAQ review

5. About the template Agreed: Template format
Use of target as main indicator complemented by additional statistic indicators
The relation between target indicator and other indicators has been clarified
Use of quality bounds for the indicators is fine but more datasets are required to fix values for these bounds.
Performance criteria should not be geographically dependent
RDE is not providing much insight, is not sufficient and can be misleading.

6. About the template Agreed:
Exploration mode is extremely important before producing the report
Selection of the stations to be used is crucial (consistence between resolution and station type  SG1)
To do
To be extended to annual averages for NO2 and PM10
Add info on AOT, SOMO, exceedances
Assess sensitivity of criteria/goals to scale, time averaging…
Assess sensitivity of target indicator to normalisation by standard deviation

7. About the template To do as SG4 participants
Use DELTA on own dataset and provide it to the JRC.
Main concern:
Use of the quality objectives for policy

8. Templates for annual averages Medium Term (2012)
Future plans
Short term (2011)
Delta updates
Templates for annual averages
Medium Term (2012)
Development of the benchmarking service (completion of the procedure)
SG4 report on the interpretation of the performance report and fixing of criteria and goals, example on different datasets.
Extension of the analysis to other datasets

9. Deliverables for TSAP review
Benchmarking procedure
DELTA  Benchmarking service  ENSEMBLE (+ data prep. Facility)
Recommendations on:
Common template for reporting model performances as complement/substitute to the current RDE indicator.
Quality objectives (to be periodically revised)