Data Extraction Facility USER JRC Data Extraction Facility Model info Model results DELTA Benchmark Case & Report DB DELTA BENCHMARKING service Model performance Evaluation reports Deadline: end 2012
What has been done since Oslo meeting The DELTA tool has been developed and released (about 20 users) together with the document “Delta concepts and user’s guide” (March 2011). Within this document a template for reporting model performances has been proposed. A questionnaire has been sent to SG4 participants to get feedback (10 replies) on: DELTA Template for reporting performance
Objectives of SG4 meeting Get feedback and discuss the template for reporting model performances: Content of the performance report Links between Target and more “traditional” indicators (an analysis based on 3 datasets) A “first-guess” for criteria/goals Comparison with RDE Observation uncertainty Future plans Deliverables for TSAQ review
About the template Agreed: Template format Use of target as main indicator complemented by additional statistic indicators The relation between target indicator and other indicators has been clarified Use of quality bounds for the indicators is fine but more datasets are required to fix values for these bounds. Performance criteria should not be geographically dependent RDE is not providing much insight, is not sufficient and can be misleading.
About the template Agreed: Exploration mode is extremely important before producing the report Selection of the stations to be used is crucial (consistence between resolution and station type SG1) To do To be extended to annual averages for NO2 and PM10 Add info on AOT, SOMO, exceedances Assess sensitivity of criteria/goals to scale, time averaging… Assess sensitivity of target indicator to normalisation by standard deviation
About the template To do as SG4 participants Use DELTA on own dataset and provide it to the JRC. Main concern: Use of the quality objectives for policy
Templates for annual averages Medium Term (2012) Future plans Short term (2011) Delta updates Templates for annual averages Medium Term (2012) Development of the benchmarking service (completion of the procedure) SG4 report on the interpretation of the performance report and fixing of criteria and goals, example on different datasets. Extension of the analysis to other datasets
Deliverables for TSAP review Benchmarking procedure DELTA Benchmarking service ENSEMBLE (+ data prep. Facility) Recommendations on: Common template for reporting model performances as complement/substitute to the current RDE indicator. Quality objectives (to be periodically revised)