Quality Assurance at the European Monitoring Centre for Drugs and Drug Addiction Sandrine Sleiman European Conference on Quality in Official Statistics, Rome 8-11 July 2008

Contents Scope of EMCDDA activities Quality assurance: operating framework Challenges and results

Who we are A decentralised EU agency Formally established in 1993 Based in Lisbon, Portugal (operating since 1995) The hub of drug-related information in the EU

Guiding principles and priorities: Principles Scientific excellence Partnership Good governance and efficiency Priorities Consolidating monitoring and reporting activities Enhancing data analysis Communicating more effectively

Collecting and analysing existing data Areas: Epidemiology — drug situation, monitoring new trends, five key indicators Interventions — health, social and criminal justice responses, best practice in demand and supply reduction Legislation — national and Community strategies and policies Policy — political and institutional framework, coordination

Five Key indicators Drug use in the general population (students and young adults): prevalence and patterns of drug use in the general population Drug users accessing treatment (TDI) Drug related infectious diseases (HIV, HCV, HBV) Drug related deaths and mortality of drug users Problem drug use: prevalence and patterns of problem drug use Requires commitment from MS to invest in data collection on a routine basis

Type of data collected Annual registry and epidemiological data Descriptive and qualitative data (non-annual) Data from research studies Institutional data Database cases Coverage and availability of prevention, treatment and harm reduction Statistics from law enforcement sources

Collecting and analysing existing data Reitox national focal points (NFPs), national experts Annual reporting to EMCDDA (national reports, standard tables, questionnaires) Expert groups for developing indicators (nominated by NFPs) Ad hoc working groups

Reporting system: standard templates National reports and selected issues (based on detailed guidelines) Standard Tables (ST) for quantitative information Sturctured Questionnaires (SQ) for qualitative information supplementing ST and NR

Quality Management at the EMCDDA Data collection instruments Quality overviews Quality Control by Scientific Units Quality assurance at EMCDDA level Data Validation Data collection instruments, protocols and guidelines National Focal Points & National data providers Data submission in FONTE, including quality control at national level National data providers Quality Assurance by QA team

Main quality related outputs & reports l Annual Quality reports Implementation Needs Profiles on the 5 key indicators EU experts groups ad-hoc documents Regular reporting to EC on implementation of 5 KI and on quality of reporting

Annual Quality reports

Criteria for National reports Completeness: the report contains all the necessary and existent information in order to provide an overview of the situation Insight: the report includes complete and significant information, giving an interpretation to the reported information, according to social and political contexts. Reliability: the extent to which the information in the report allows comparisons (between different time periods). Usefulness: information oriented to the targets; acceptable and pertinent to the report objectives; no redundant information is presented. Internal consistency: the extent to have coherent information in the report or to describe the reasons for a lack of internal consistency.

Criteria for data checking in STs CNDA: Not clear whether all data in the ST are new or not CQUA: Quantitative information needed clarification CQUAL: Qualitative information needed clarification CREF: References needed clarification DNR: Discrepancy between NR and ST MDA: Quantitative information missing MME: Methodological information missing MRE: References missing OTH: Others

Criteria for SQ Level of details Insufficient, sufficient, good Conciseness Insufficient, sufficient, good Discrepencies with NR Yes - No General comment Any other important feedback

Implementation Needs Profiles Concerns 5 key epidemiological indicators Focuses on activities related to indicators and partially on content Is a general analysis, helps to identify problematic areas, status quo or improvement in data collection

Criteria used Most recent data Reference Description Weaknesses Activities implemented or ongoing/planned Actions needed

Quality assurance: work in progress Quality Assurance… is mentioned as a priority in the EU action plan is mentioned in the external evaluation report is part of the EMCDDA’s 3-year work programme Quality documents should be … considered as real tools for the EMCDDA and the REITOX network (as well as for external actors) aimed at scientific excellence and not administrative purposes an overview of strengths and weaknesses of data or collection processes …

Quality assurance: work in progress Production of 5 KI methodological packages Definition of precise assessment criteria for 5 KI based on : minimum implementation standards specific to each indicator sensitive to national contexts somewhere between pragmatism and perfection Redefinition of templates and guidelines with adaptation of quality criteria

Quality assurance : work in progress Understand needs and capacity to deliver from the Reitox network Priorities for value delivery: revision of reporting guidelines Promise what can be delivered Change capability to match promise Deliveries matching standards and priorities Change capability to match future needs: - Quality feedbacks - Capacity development activities

Case study: national reports Success Recommendations for improvement are generally applied as regards adherence to guidelines: -Document layout -References to standard tables and structured questionnaires -Sources and use of Harvard for bibliography -Methodological information is more precise -Efforts in writing the National report

Case study: national reports Gaps Deadlines missed Conciseness Lack of insight on trends and interpretation of results In some countries, it is still too early to have information on trends Work on quality assurance depends on external factors, such as Member States’ involvement in quality issues

THANK YOU FOR YOUR ATTENTION