Making Clinical Practice Guidelines “Better” Katrin Uhlig, MD MS Director, Guideline Development Tufts Center for Kidney Disease Guideline Development and Implementation Tufts Medical Center Boston, MA Presentation to IOM Committee on Standards for Developing Trustworthy Clinical Practice Guidelines 1/11/2010 Washington DC

2 Guidelines Program for Kidney Disease Improving Global Outcomes (KDIGO) Goal –3 guidelines ongoing simultaneously –18 months duration, 3 meetings KDIGO: 2 Co-Chairs, 50 Board Members, National Kidney Foundation (NKF): provides institutional and administrative support ERT: standing team of systematic review /guideline methodologists –MD Co-Directors (neph/IM) –MD Research Associates (junior neph/IM) –Fellows, Research Assistants Work Group: 2 co-chairs; ~15-20 experts Funding: fundraising from industry through NKF Cost per guideline: $$$ Published as supplements to Kidney Int (KDOQI: Am J Kidney Dis) –~ pages Care of the Kidney Transplant Recipient –~45 topics –~275 individual recommendations –~150 RCTs; ~20 SRs; ~100 other studies So far 13 guidelines completed

3 Methods Milestones in Kidney Disease Guideline Development DOQI Guidelines. Supported by one lead methodologist. Systematic searches, narrative reviews. Binary grading: Evidence or Opinion KDOQI replaces DOQI. Methods support by Tufts EPC staff. Systematic reviews for some topics. Data extraction by work group experts KDIGO launched. Contract with designated methods staff at Tufts Medical Center. Methods staff takes on data extraction Pilot of GRADE approach and development of evidence profiles ERT provides input on wording and grading. Co-authorship.

4 Challenges in Guideline Development Challenges in general (expanded on in the following) –Philosophy –Participants –Process –Format Other challenges (not expanded on in the following) –Grading –Low quality of evidence in the field of kidney diseases Small number of RCTs Usually limited data on clinical outcomes –“International guidelines” - no specific reference setting Evidence predominantly from resource rich settings Work Group membership mostly from NA and Europe

5 Challenge: Small Number of RCTs in Nephrology Strippoli et al. JASN 2004

6 Challenges: ‘Philosophy’ What topics to pick for a guideline? What are alternatives to guidelines? Should there be recommendations when the scientific evidence is inconclusive (absent, poor, conflicting)?

7 Suggestions: Philosophy Pick topics related to treatment and diagnosis. –This requires a body of evidence of sufficient breadth and depth –For other topics, use other formats: Consensus, controversies, “state of the science” conferences Clearly transmit organization’s “philosophy” –Guideline panel Can or Can Not issue recommendations when evidence is inconclusive –If panel may issue recommendations when evidence not definitive, be explicit about values and preference applied –Allow for alternatives (for example un-graded statements) –If only “evidence based” recommendations, define minimum standards for evidence to support recommendations

8 Challenges: Participants What should the composition be of a CPG development panel? –Topical experts –Methodologists –Consumers How should they collaborate and interact? How can one contain conflicts of interest COI?

9 Suggestions: Participants Need both methods and topical experts Topical experts –Multi-disciplinary, representing different relevant specialties and provider perspectives Have adequate methods experts –Dedicated methods staff –Methods expertise in panel members –Methodologically trained chair Unsure how best to incorporate consumers –Reviewers of scope and questions –Reviewers of drafts –? Panel members Collaboration/Interaction –Different models can work depending on skills and capacity in Work Group

10 Suggestions: Participants Contain potential COI 1.Formally discuss potential COI for particular guideline topics 2.Appoint Work Group chairs free of potential COI 3.Avoid appointing Work Group members with strong COI 4.Where appointing Work Group members with strong COI is unavoidable, ensure that other Work Group members adequately represent alternative perspectives 5.Periodically take inventory of potential COI of Work Group members and disseminate this information to the Work Group 6.Document adherence to procedures and make this information available for review upon request Tufts Center for Kidney Disease Guideline Development and Implementation 2008

11 Challenges: Process How to refine scope? In evidence review, what is the optimal balance between quality, speed and resources? –De novo systematic reviews versus using existing? –Exhaustive searches versus adding relevant papers by experts? –Minimum standard in evidence review or adjustment of inclusion criteria based on expected or actual yield of evidence? –Qualitative summary versus meta-analysis with quantitative synthesis?

12 Suggestions: Process Focus scope before inviting experts Use existing systematic reviews to focus questions. Use instead of new reviews, if possible (although this is often not a time saving step) Do comprehensive, but not exhaustive systematic searches. Ask experts to add relevant missed papers For treatment questions, use restriction in study design (usually RCT) and sample size Supplement with observational studies identified by experts if sparse RCT evidence on clinical outcomes and for harms Little value added in lowering criteria in response to expected sparse or low quality evidence. Adhere to general concepts in evidence grading: no “up grading” for “best available”, but still flawed evidence. MA would be informative, but often not feasible

13 Challenges: Format How to write, edit, rewrite? How to format? How to contain length? How to orchestrate review and respond to comments?

14 Suggestions: Format Plan for this to be a resource-intense and time consuming step which requires –Guidance and templates –Adequate time for arriving at consensus and revising drafts –Administrative and logistic support

15 Acknowledgment of Participants in KDIGO Guideline Development Tufts Evidence Review Team Joseph Lau Andrew Levey Ethan Balk Katrin Uhlig Nephrology fellows Research coordinator, assistants National Kidney Foundation Kerry Willis Guideline support staff KDIGO Co-Chairs Kai-Uwe Eckardt Bert Kasiske Gary Eknoyan (Emeritus) Nathan Levin (Emeritus) KDIGO Board Work Group Chairs Work Groups

16 End