1. Implementing the GRADE Method in Guideline Development: Real- World Experiences Contemplation Stage: To GRADE or Not to GRADE? Sheila A. Agyeman, MHA Director of Evidence Based Medicine American Gastroenterological Association Institute September 15, 2009

2. AGA Institute Guideline Development Process Prior to 2007 Searching for and appraising the evidence Authors performed literature searches Table below was provided to aide authors in organizing findings but not a requirement Evidence summarized in Technical Review document Technical review (intended for academics) summarized into Medical Position Statement (intended for practicing clinicians) ReferenceYearStudy Design/type No. of Subjects Length of Study InterventionOutcome Measurement Statistical Analysis ConclusionsComments

3. AGA Institute Guideline Development Process Prior to 2007 Inconsistencies identified in Pre-2007 Process Technical review and medical position statement not adequately evidence based Disconnect between technical review and medical position statement  Not guiding readers in interpreting the data  Not giving clear clinical recommendations Concept of quality of evidence and strength of recommendations not well emphasized

4. AGA Institute Guideline Development Current Process Modifications made to the process Construction of specific clinical questions to be addressed in technical review Implementation of U.S. Preventive Services Task Force (USPSTF) Ratings – Management of GERD guideline Composition of Medical Position Panel o Patient/patient advocate o Payer o Gastroenterologist in community practice o Gastroenterologist with knowledge in health services research o Primary care physician, surgeon or pathologist (included as needed based on topic)

5. AGA Institute Guideline Development Current Process Challenges of current process USPSTF grading system primarily designed for preventive services USPSTF grading system changed after first guideline was already in the final stages (mid 2007) USPSTF grading system caused confusion amongst guideline authors (Dysplasia in IBD guideline) how to grade evidence and recommendations (e.g., diagnostic tests)

6. AGA Institute Guideline Development Current Process What made the AGA Institute decide to consider switching from USPSTF to using GRADE? Invitations to collaborate with other GI and non-GI societies Establish uniformity across all GI societies  Further invitations for collaboration led to the discovery that more societies are using or also considering GRADE  Considered using GRADE to reduce the confusion in different grading schemes used in various guidelines  Joining effort to establish an international standard

7. AGA Institute Guideline Development Current Process Benefits of using GRADE we considered: A very rigorous and formalized process that will ensure evidence-based guidelines necessary for value-based health care initiatives (e.g., P4P) Already closely resembles AGA Institute’s current process Provides increased transparency of judgments of quality of evidence and strength of recommendations Investment into GRADE evidence profiles in the long- term could be future asset at the time of guidelines update Availability of GRADE methodologists who could assist in the process

8. AGA Institute Guideline Development Current Process Initial reservations about using GRADE Higher costs?  Need to hire full time methodologist?  Answer: GRADE not necessarily more complex than other grading systems.  Increased training of staff and authors?  Answer: Any grading system would require some form of training Requires more time and lengthens the development process? Answer: No. The most time consuming and costly phase of the guideline development process is the systematic review of the evidence. This is independent from the grading system used.

9. AGA Institute Guideline Development Current Process Next Steps Develop an implementation plan  Train staff in GRADE process  Train committee members in GRADE process  Include GRADE experts on committee Include in budget cycle for 2010-2011