1. BMH4-98-3669 CLINICAL GUIDELINES IN EUROPE

4. OUTLINE Background to the project Objectives The AGREE Instrument: validation process and results Outcomes and impact of AGREE Research & policy implications Future developments

5. BACKGROUND to AGREE PROJECT  Clinical guidelines play a key role in healthcare improvement in most countries  Ensuring their quality is a shared concern  There is a need for consistent international guidance on guideline development and assessment  ‘The whole is greater than the sum of its parts’

6. OBJECTIVES Develop compatible approaches for the creation of clinical guidelines Establish a structure for the appraisal and monitoring of clinical guidelines Define relevant quality criteria Promote and encourage the diffusion of these criteria through international exchanges and collaborative links

8. PURPOSE OF THE AGREE INSTRUMENT  To provide a systematic framework for appraising the quality of clinical guidelines  To help guideline developers follow a structured and rigorous methodology  To help policymakers decide which guideline to recommend for use in practice  To help health care providers assess guidelines before adopting recommendations in practice

9. WHAT IS A GOOD GUIDELINE? ‘Quality of clinical guidelines’ is the confidence that: the potential biases of guideline development have been addressed adequately the recommendations are both internally and externally valid, and are feasible for practice

10. STRUCTURE  23 items  4-point Likert Scale Six domains 1. Scope & purpose (3) 2. Stakeholder involvement (4) 3. Rigour of development (7) 4. Clarity & presentation (4) 5. Applicability (3) 6. Editorial independence (2) User guide Overall assessment

11. DEVELOPMENT PROCESS Subgroup reviews literature First draft circulated for comments First workshop, instrument modified First validation study Second workshop, instrument revised Final validation study

12. VALIDATION PROCESS (1) Phase 1  The instrument was applied to 100 guidelines from 11 countries  All countries followed the same protocol  Each guideline was independently assessed by 4 appraisers (194 altogether)  Instrument was used in English

13. VALIDATION PROCESS (2) Phase 2 Refined instrument tested on 33 guidelines randomly selected from 1st study Same countries participated New set of appraisers recruited (70) Each country assessed 3 guidelines

14. RESULTS  Over 90% of appraisers found the instrument and the user guide useful  Reliability was satisfactory for each domain  - Cronbach’s Alpha 0.64 - 0.88  - ICC 0.57 -0.91  There was suggestion of criterion validity

15. OUTCOMES The AGREE Instrument has been translated into 7 European languages, more in preparation Six papers have been published All EU-funded projects are requested to assess guidelines with the AGREE Instrument WHO has endorsed the AGREE Instrument The Council of Europe has formally recommended its use

16. IMPACT OF AGREE Guideline Assessment Europe South East Asia Middle East Latin America Australia Canada Education and training Postgraduate courses (Mc Master University, New Zealand, UK, France) Guideline development European Society Cardiology National guidelines programmes Research USA UK Canada

17. “Both NICE and SIGN aim to meet the international standard set by the AGREE guideline appraisal instrument” Joint NICE/SIGN statement on working together to improve patient care NICE website 18 April 2002

18. RESEARCH & POLICY IMPLICATIONS n The AGREE Instrument creates a common framework for assessing guidelines that can be applied internationally. n Can be used to compare quality of guidelines across countries and disease areas. n Provides standards for reporting of clinical guidelines published in medical journals. n Governmental agencies can use AGREE as basis for making policy decisions about guidelines

19. FURTHER DEVELOPMENTS: AM Project Structured visits to 7 central & East European countries Dissemination workshops in 15 European countries AGREE Training manual & Evaluation form Research programme EU 6 th Framework

20. G.I.N. The Guidelines International Network The Association for Guideline REsearch and Evaluation

21. http://www.agreecollaboration.org Website address