Download presentation
Presentation is loading. Please wait.
1
The GRADE approach: an introductory workshop
Holger Schünemann, MD, PhD Professor and Chair, Dept. of Clinical Epidemiology & Biostatistics Professor of Medicine Michael Gent Chair in Healthcare Research McMaster University, Hamilton, Canada NTP, Raleigh June 22, 2011
2
2
3
The Department of Clinical Epidemiology & Biostatistics at McMaster
History – Founded by David Sackett - 6 chairs since - Instrumental in specialty of Clinical Epidemiology, origin of “Evidence-Based Medicine” People 45 full time and joint faculty ~ 120 associate & part time faculty; 19 emeritus ~ 180 staff ~ 200 PhD and Master students
4
Time Session title 8.30 – 10.00 Overview of GRADE 10.00 – 10.15 BREAK 10.15 – Noon Details of quality assessment -small group sessions Noon – 1.00 LUNCH 1.00 – 3.00 Framework for devleoping recommendations Small group sessions 3.00 – 5.00 OHAT assessment tool
5
Content Guidelines and GRADE Quality of evidence
Background about GRADE Quality of evidence Going from evidence to recommendations
6
What is a guideline? "Guidelines are recommendations intended to assist providers and recipients of health care and other stakeholders to make informed decisions. Recommendations may relate to clinical interventions, public health activities, or government policies." WHO 2003, 2007
7
Evidence based healthcare decisions
(Clinical) state and circumstances Population values and preferences Expertise Research evidence Haynes et al. 2002
8
Confidence in evidence
There always is evidence “When there is a question there is evidence” Better research greater confidence in the evidence and decisions
9
Hierarchy of evidence based on quality
BIAS STUDY DESIGN Randomized Controlled Trials Cohort Studies and Case Control Studies Case Reports and Case Series, Non-systematic observations Expert Opinion
10
“Everything should be made as simple as possible but not simpler.”
Explain the following? Confounding, effect modification & ext. validity Concealment of randomization Blinding (who is blinded in a double blinded study?) Intention to treat analysis and its correct application P-values and confidence intervals
11
BMJ 2003 BMJ, 2003
12
Relative risk reduction: ….> 99.9 % (1/100,000)
U.S. Parachute Association reported 821 injuries and 18 deaths out of 2.2 million jumps in 2007 BMJ 2003
13
Simple hierarchies are (too) simplistic
STUDY DESIGN Randomized Controlled Trials Cohort Studies and Case Control Studies Case Reports and Case Series, Non-systematic observations BIAS Expert Opinion Expert Opinion Schünemann & Bone, 2003
14
Which hierarchy? Recommendation for use of oral anticoagulation in patients with atrial fibrillation and rheumatic mitral valve disease Evidence Recommendation B Class I A 1 IV C Organization AHA ACCP SIGN
15
Oxford Centre for Evidence Based Medicine
This is a more sophisticate development of Sackett’s initial approach by the Oxford Centre of Evidence-Based Medicine. This approach, however, is quite different from what we have used for the ACCP Consensus Conference and is too complicated.
16
USPSTF - Grade Definitions After May 2007: Certainty
Level of Certainty Description High The available evidence usually includes consistent results from well-designed, well-conducted studies in representative primary care populations. These studies assess the effects of the preventive service on health outcomes. This conclusion is therefore unlikely to be strongly affected by the results of future studies. Moderate The available evidence is sufficient to determine the effects of the preventive service on health outcomes, but confidence in the estimate is constrained by such factors as: The number, size, or quality of individual studies. Inconsistency of findings across individual studies. Limited generalizability of findings to routine primary care practice. Lack of coherence in the chain of evidence. As more information becomes available, the magnitude or direction of the observed effect could change, and this change may be large enough to alter the conclusion. Low The available evidence is insufficient to assess effects on health outcomes. Evidence is insufficient because of: The limited number or size of studies. Important flaws in study design or methods. Gaps in the chain of evidence. Findings not generalizable to routine primary care practice. Lack of information on important health outcomes. More information may allow estimation of effects on health outcomes. The USPSTF defines certainty as "likelihood that the USPSTF assessment of the net benefit of a preventive service is correct."
18
Recommendations for prognosis
Use prognostic information to determine baseline risk for healthcare decisions
21
Center for Disease Control and Prevention (CDC)
This is the approach used by the CDC. On of the main criticisms with this approach is the lack of standardization with this approach and the probably lack of reproducibility.
22
“Long term perspective”
“Healthy people” “Herd immunity” “Long term perspective” “Disease perception” “Lots of other things” Healthcare problem recommendation
23
Grades of Recommendation Assessment, Development and Evaluation
GRADE Working Group Grades of Recommendation Assessment, Development and Evaluation Aim: to develop a common, transparent and sensible system for grading the quality of evidence and the strength of recommendations International group of guideline developers, methodologists & clinicians from around the world (>250 contributors) – since 2000 International group: ACCP, AHRQ, Australian NMRC, BMJ Clinical Evidence, Cochrane Collaboration, CDC, McMaster, NICE, Oxford CEBM, SIGN, UpToDate, USPSTF, WHO CMAJ 2003, BMJ 2004, BMC 2004, BMC 2005, AJRCCM 2006, Chest 2006, BMJ 2008
24
GRADE Uptake World Health Organization CDC-ACIP
Allergic Rhinitis in Asthma Guidelines (ARIA) American Thoracic Society American College of Physicians European Respiratory Society European Society of Thoracic Surgeons British Medical Journal Infectious Disease Society of America American College of Chest Physicians UpToDate® National Institutes of Health and Clinical Excellence (NICE) Scottish Intercollegiate Guideline Network (SIGN) Cochrane Collaboration Clinical Evidence Agency for Health Care Research and Quality (AHRQ) Partner of GIN Over 40 major organizations
25
Guideline development Process
26
Case scenario A 13 year old girl who lives in rural Indonesia presented with flu symptoms and developed severe respiratory distress over the course of the last 2 days. She required intubation. The history reveals that she shares her living quarters with her parents and her three siblings. At night the family’s chicken stock shares this room too and several chicken had died unexpectedly a few days before the girl fell sick. Potential interventions: antivirals, such as neuraminidase inhibitors oseltamivir and zanamivir
27
Types of questions Background Questions Definition: What is Avian Influenza? Mechanism: What is the mechanism of action of oseltamivir? Foreground Questions Benefit > harm: In patients with avian influenza, does oseltamivir therapy improve survival, …?
28
Framing a foreground question
Population: Avian Flu/influenza A (H5N1) patients Intervention: Oseltamivir Comparison: No pharmacological intervention Outcomes: Mortality, hospitalizations, resource use, adverse outcomes, antimicrobial resistance Schunemann, et al., The Lancet ID, 2007
29
Choosing outcomes Desirable outcomes Undesirable outcomes
lower mortality reduced hospital stay reduced duration of disease reduced resource expenditure Undesirable outcomes adverse reactions the development of resistance costs of treatment Every decision comes with desirable and undesirable consequences Developing recommendations must include a consideration of desirable and undesirable outcomes
30
Relative importance of outcomes
Decision makers (and guideline authors) need to consider the relative importance of outcomes when balancing these outcomes to make a recommendation Relative importance vary across populations Relative importance may vary across patient groups within the same population When considered critical - evaluate
31
GRADE: recommendation – quality of evidence
Clear separation: 1) Recommendation: 2 grades – weak/conditional/optional or strong (for or against an intervention)? Balance of benefits and downsides, values and preferences, resource use and quality of evidence 2) 4 categories of quality of evidence: (High), (Moderate), (Low), (Very low)? methodological quality of evidence likelihood of bias by outcome and across outcomes *
32
GRADE Quality of Evidence
In the context of a systematic review The quality of evidence reflects the extent to which we are confident that an estimate of effect is correct. In the context of making recommendations The quality of evidence reflects the extent to which our confidence in estimates of the effects is adequate to support a particular recommendation.
33
Likelihood of and confidence in an outcome
34
Definition of grades of evidence Research
/A/High: Further research is very unlikely to change confidence in the estimate of effect. /B/Moderate: Further research is likely to have an important impact on confidence in the estimate of effect and may change the estimate. /C/Low: Further research is very likely to have an important impact on confidence in the estimate of effect and is likely to change the estimate. /D/Very low: Any estimate of effect is very uncertain.
35
Confidence in evidence
/A/High: We are very confident that the true effect lies close to that of the estimate of the effect. /B/Moderate: : We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. /C/Low : Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect. /D/Very low : We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect.
36
Determinants of quality
RCTs observational studies 5 factors that can lower quality limitations in detailed design and execution (risk of bias criteria) Inconsistency (or heterogeneity) Indirectness (PICO and applicability) Imprecision (number of events and confidence intervals) Publication bias 3 factors can increase quality large magnitude of effect all plausible residual confounding may be working to reduce the demonstrated effect or increase the effect if no effect was observed dose-response gradient
37
1. Design and Execution/Risk of Bias
Examples: Inappropriate selection of exposed and unexposed groups Failure to adequately measure/control for confounding Selective outcome reporting Failure to blind (e.g. outcome assessors) High loss to follow-up Lack of concealment in RCTs Intention to treat principle violated
38
Design and Execution/RoB
From Cates , CDSR 2008
39
Design and Execution/RoB
Overall judgment required
40
Design and Execution/RoB
Cancer, anticoagulation and mortality Akl E, Barba M, Rohilla S, Terrenato I, Sperati F, Schünemann HJ. “Anticoagulation for the long term treatment of venous thromboembolism in patients with cancer”. Cochrane Database Syst Rev Apr 16;(2):CD
41
2. Inconsistency of results (Heterogeneity)
if inconsistency, look for explanation patients, intervention, comparator, outcome if unexplained inconsistency lower quality
42
Reminders for immunization uptake
43
Indoor air polution: ALRI
44
Non-steroidal drug use and risk of pancreatic cancer
Capurso G, Schünemann HJ, Terrenato I, Moretti A, Koch M, Muti P, Capurso L, Delle Fave G. Meta-analysis: the use of non-steroidal anti-inflammatory drugs and pancreatic cancer risk for different exposure categories. Aliment Pharmacol Ther Oct 15;26(8):
45
3. Directness of Evidence
differences in populations/patients (children – neonates, women in general – pregnant women) interventions (all vaccines, new - old) comparator appropriate (new policy – old or no policy) outcomes (important – surrogate: cases prevented – seroconversion) indirect comparisons interested in A versus B have A versus C and B versus C Vaccine A versus Placebo versus Vaccine B
46
Possibly. The “high” dose effects of bisphenol A in laboratory animals that provide clear evidence for adverse effects on development, i.e., reduced survival, birth weight, and growth of offspring early in life, and delayed puberty in female rats and male rats and mice, are observed at levels of exposure that far exceed those encountered by humans. However, estimated exposures in pregnant women and fetuses, infants, and children are similar to levels of bisphenol A associated with several “low” dose laboratory animal findings of effects on the brain and behavior, prostate and mammary gland development, and early onset of puberty in females. When considered together, these laboratory animal findings provide limited evidence that bisphenol A has adverse effects on development.
47
Myocardial infarction
Hierarchy of outcomes according to their importance to assess the effect of phosphate lowering drugs in patients with renal failure and hyperphosphatemia Flatulence Importance of outcomes 2 5 6 7 8 9 3 4 1 Mortality Myocardial infarction Fractures Pain due to soft tissue calcification / function Critical for decision making Important, but not critical for Low importance for Coronary calcification Ca2+/P- product Bone density Soft tissue calcification Surrogates: relation to important outcomes increasingly uncertain
48
4. Publication Bias Should always be suspected
Only small “positive” studies (hypothesis confirming) For profit interest Various methods to evaluate – none perfect, but clearly a problem
49
I.V. Mg in acute myocardial infarction
ISIS-4 Lancet 1995 Meta-analysis Yusuf S.Circulation 1993 Publication bias Egger M, Smith DS. BMJ 1995;310:752-54
50
Funnel plot Symmetrical: No publication bias Standard Error Odds ratio
Symmetrical: No publication bias 1 Standard Error 2 3 0.1 0.3 0.6 1 3 10 Odds ratio
51
Funnel plot File drawer problem
File drawer problem No interest in publishing or being published 1 0.4 Standard Error Asymmetrical: Publication bias? 2 3 0.1 0.3 0.6 1 3 10 Odds ratio
52
Indoor air polution: ALRI
53
5. Imprecision Small sample size Wide confidence intervals
small number of events Wide confidence intervals uncertainty about magnitude of effect Extent to which confidence in estimate of effect adequate to support decision
54
Example: Immunization in children
56
What can raise quality? 1. large magnitude can upgrade (RRR 50%/RR 2)
very large two levels (RRR 80%/RR 5) criteria everyone used to do badly almost everyone does well parachutes to prevent death when jumping from airplanes
57
Reminders for immunization uptake
58
What can raise quality? 2. dose response relation
(higher INR – increased bleeding) childhood lymphoblastic leukemia risk for CNS malignancies 15 years after cranial irradiation no radiation: 1% (95% CI 0% to 2.1%) 12 Gy: 1.6% (95% CI 0% to 3.4%) 18 Gy: 3.3% (95% CI 0.9% to 5.6%) 3. all plausible confounding may be working to reduce the demonstrated effect or increase the effect if no effect was observed
59
All plausible residual confounding would result in an overestimate of effect
Hypoglycaemic drug phenformin causes lactic acidosis The related agent metformin is under suspicion for the same toxicity. Large observational studies have failed to demonstrate an association Clinicians would be more alert to lactic acidosis in the presence of the agent Vaccine – adverse effects Meningococcal Conjugate Vaccine (Menactra) – Guillain Barre
60
Quality assessment criteria
61
Evidence Profiles/Summaries
62
Evidence Profiles/Summaries
63
Evidence Profiles/Summaries
64
Evidence Profiles/Summaries
65
Content Background Quality of evidence
Moving from evidence to recommendations
66
Strength of recommendation
“The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.” Strong or weak/conditional
67
Determinants of the strength of recommendation
68
Developing recommendations
69
Case scenario A 13 year old girl who lives in rural Indonesia presented with flu symptoms and developed severe respiratory distress over the course of the last 2 days. She required intubation. The history reveals that she shares her living quarters with her parents and her three siblings. At night the family’s chicken stock shares this room too and several chicken had died unexpectedly a few days before the girl fell sick.
70
Methods – WHO Rapid Advice Guidelines for management of Avian Flu
Applied findings of a recent systematic evaluation of guideline development for WHO/ACHR Group composition (including panel of 13 voting members): clinicians who treated influenza A(H5N1) patients infectious disease experts basic scientists public health officers methodologists Independent scientific reviewers: Identified systematic reviews, recent RCTs, case series, animal studies related to H5N1 infection
71
Oseltamivir for Avian Flu
Summary of findings: No clinical trial of oseltamivir for treatment of H5N1 patients. 4 systematic reviews and health technology assessments (HTA) reporting on 5 studies of oseltamivir in seasonal influenza. Hospitalization: OR 0.22 (0.02 – 2.16) Pneumonia: OR 0.15 ( ) 3 published case series. Many in vitro and animal studies. No alternative that is more promising at present. Cost: 40$ per treatment course
72
From evidence to recommendation
73
Example: Oseltamivir for Avian Flu
Recommendation: In patients with confirmed or strongly suspected infection with avian influenza A (H5N1) virus, clinicians should administer oseltamivir treatment as soon as possible (????? recommendation, very low quality evidence). Schunemann et al. The Lancet ID, 2007
74
Example: Oseltamivir for Avian Flu
Recommendation: In patients with confirmed or strongly suspected infection with avian influenza A (H5N1) virus, clinicians should administer oseltamivir treatment as soon as possible (strong recommendation, very low quality evidence). Values and Preferences Remarks: This recommendation places a high value on the prevention of death in an illness with a high case fatality. It places relatively low values on adverse reactions, the development of resistance and costs of treatment. Schunemann et al. The Lancet ID, 2007
75
Implications of a strong recommendation
Patients: Most people in this situation would want the recommended course of action and only a small proportion would not Clinicians: Most patients should receive the recommended course of action Policy makers: The recommendation can be adapted as a policy in most situations
76
Implications of a conditional/weak recommendation
Patients: The majority of people in this situation would want the recommended course of action, but many would not Clinicians: Be more prepared to help patients to make a decision that is consistent with their own values/decision aids and shared decision making Policy makers: There is a need for substantial debate and involvement of stakeholders
77
Panel P I C O Systematic review evidence profile with GRADEpro Create
Summary of findings & estimate of effect for each outcome Outcomes across studies Rate quality of evidence for each outcome Formulate question Randomization increases initial quality Select outcomes Rate importance Risk of bias Inconsistency Indirectness Imprecision Publication bias P I C O Outcome Critical Grade overall quality of evidence across outcomes based on lowest quality of critical outcomes High Outcome Critical Moderate Grade down Low Outcome Important Very low Outcome Not important Large effect Dose response Confounders Grade up Panel Guideline development Formulate recommendations: For or against (direction) Strong or weak (strength) By considering: Quality of evidence Balance benefits/harms Values and preferences Revise if necessary by considering: Resource use (cost) “We recommend using…” “We suggest using…” “We recommend against using…” “We suggest against using…”
78
Issues in guideline development in Public Health
Causation versus effects of intervention Causation not equivalent to efficacy of interventions Bradford Hill Nearly half a century old – tablet from the mountain? Harms caused by medications Assumption is that removal of exposure leads to NO adverse effects How confident can one be that removal of the exposure is effective in preventing disease? Whether drugs or environmental factors it will depend on the intervention to remove exposure
79
GRADE and immunizations
Can herd immunity following immunisation and indirect effects on the co-circulation of other pathogens typically be ascertained only through the use of observational epidemiological methods? Innovative randomized controlled trials (RCTs) using cluster-randomization increasingly being conducted to provide this evidence A 94% protective effect of a live, monovalent vaccine against measles is classified as “moderate level of scientific evidence.” GRADE’s strength of association criteria maybe applied to increase the grade by 2 levels – from “low” to “high” - possible in this situation GRADE ratings do not give credit to “gradient of effects with scale of population level impact compatible with degree of coverage.” GRADE’s dose-response criterion would apply to such gradients May anti-vaccination lobby groups abuse the ratings. Abuse of any system is possible: equally likely that increased transparency provided by the GRADE framework can strengthen, rather than undermine, the trust in vaccines and other interventions
80
Schünemann et al. JECH 2010
81
Bradford Hill Criteria
Would agree with modifying if these criteria were not already considered Emphasis is on strength of recommendation, quality of evidence is only one factor
82
Conclusions Clinical practice guidelines should be based on the best available evidence to be evidence based GRADE combines what is known in health research methodology and provides a structured approach to improve communication Criteria for evidence assessment across questions and outcomes Criteria for moving from evidence to recommendations Transparent, systematic four categories of quality of evidence two grades for strength of recommendations Transparency in decision making and judgments is key
84
Formulating Questions and Choosing Outcomes
85
Outline Type of questions Framing a foreground question
Choosing outcomes Relative importance of outcomes
86
Guidelines and questions
Guidelines are a way of answering questions about clinical, communication, organisational or policy interventions, in the hope of improving health care or health policy. It is therefore helpful to structure a guideline in terms of answerable questions. WHO Guideline Handbook, 2008
87
Types of questions Background Questions Definition: What is COPD? Mechanism: What is the mechanism of action of mucolytic therapy? Foreground Questions Efficacy: In patients with COPD, does mucolytic therapy improve survival?
88
Framing a foreground question
P I C O
89
Framing a foreground question
Population: Intervention: Comparison: Outcomes:
90
Case scenario A 13 year old girl who lives in rural Indonesia presented with flu symptoms and developed severe respiratory distress over the course of the last 2 days. She required intubation. The history reveals that she shares her living quarters with her parents and her three siblings. At night the family’s chicken stock shares this room too and several chicken had died unexpectedly a few days before the girl fell sick. Potential interventions: antivirals, such as neuraminidase inhibitors oseltamivir and zanamivir
91
What are examples of: Background questions Foreground questions
Population: Intervention: Comparison: Outcomes:
92
Framing a foreground question
Population: Avian Flu/influenza A (H5N1) patients Intervention: Oseltamivir (or Zanamivir) Comparison: No pharmacological intervention Outcomes: Mortality, hospitalizations, resource use, adverse outcomes, antimicrobial resistance Schunemann, Hill et al., The Lancet ID, 2007
93
Choosing outcomes Every decision comes with desirable and undesirable consequences Developing recommendations must include a consideration of desirable and undesirable outcomes Outcomes should be patient important outcomes.
94
Choosing outcomes lower mortality reduced hospital stay
desirable outcomes lower mortality reduced hospital stay reduced duration of disease reduced resource expenditure undesirable outcomes adverse reactions the development of resistance costs of treatment
95
Choosing outcomes What if what is important is not measured?
What if what is measured is not important? How do we make sure we’ve covered all important outcomes?
96
Relative importance of outcomes
Decision makers (and guideline authors) need to consider the relative importance of outcomes when balancing these outcomes to make a recommendation Relative importance vary across populations Relative importance may vary across patient groups within the same population When considered critical - evaluate
97
Relative importance of outcomes
9 Critical for decision making Important, but not critical for decision making Of low importance 8 7 6 5 4 3 2 1
98
Using GRADEpro
99
Creating a new GRADEpro file
101
Profile groups Profiles
112
Managing outcomes
119
Content Quality of evidence Going from evidence to recommendations
120
Healthcare problem recommendation
121
Strength of recommendation
“The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.” Strong or conditional
122
Strength of recommendation
The degree of confidence that the desirable effects of adherence to a recommendation outweigh the undesirable effects. Desirable effects health benefits less burden savings Undesirable effects harms more burden costs
123
Determinants of the strength of recommendation
124
Balancing benefits and downsides
↑ Allergic reactions ↑ Local skin reactions ↑ Nausea ↑ Resources ↑ QoL ↓ Death ↓ Morbidity ↑ herd immunity Conditional Strong For Against
125
Balancing benefits and downsides
↑ Allergic reactions ↑ Local skin reactions ↑ Nausea ↑ Resources ↑ QoL ↓ Death ↓ Morbidity ↑ herd immunity Conditional Strong For Against
126
Balancing benefits and downsides
↑ Allergic reactions ↑ Local skin reactions ↑ Nausea ↑ Resources ↑ QoL ↓ Death ↓ Morbidity ↑ herd immunity Conditional Strong For Against
127
Balancing benefits and downsides
↑ Allergic reactions ↑ Local skin reactions ↑ Nausea ↑ Resources ↑ QoL ↓ Death ↓ Morbidity ↑ herd immunity Conditional Strong For Against
128
Balancing benefits and downsides
↑ Allergic reactions ↑ Local skin reactions ↑ Nausea ↑ Resources ↑ QoL ↓ Death ↓ Morbidity ↑ herd immunity Conditional Strong For Against
129
Implications of a strong recommendation
Policy makers: The recommendation can be adapted as a policy in most situations Patients: Most people in this situation would want the recommended course of action and only a small proportion would not Clinicians: Most patients should receive the recommended course of action
130
Implications of a conditional recommendation
Policy makers: There is a need for substantial debate and involvement of stakeholders Patients: The majority of people in this situation would want the recommended course of action, but many would not Clinicians: Be more prepared to help patients to make a decision that is consistent with their own values/decision aids and shared decision making
131
Case scenario A 13 year old girl who lives in rural Indonesia presented with flu symptoms and developed severe respiratory distress over the course of the last 2 days. She required intubation. The history reveals that she shares her living quarters with her parents and her three siblings. At night the family’s chicken stock shares this room too and several chicken had died unexpectedly a few days before the girl fell sick.
132
Methods – WHO Rapid Advice Guidelines for Avian Flu
Applied findings of a recent systematic evaluation of guideline development for WHO/ACHR Group composition (including panel of 13 voting members): clinicians who treated influenza A(H5N1) patients infectious disease experts basic scientists public health officers methodologists Independent scientific reviewers: Identified systematic reviews, recent RCTs, case series, animal studies related to H5N1 infection
133
Oseltamivir for Avian Flu
Summary of findings: No clinical trial of oseltamivir for treatment of H5N1 patients. 4 systematic reviews and health technology assessments (HTA) reporting on 5 studies of oseltamivir in seasonal influenza. Hospitalization: OR 0.22 (0.02 – 2.16) Pneumonia: OR 0.15 ( ) 3 published case series. Many in vitro and animal studies. No alternative that was more promising at present. Cost: 40$ per treatment course
134
From evidence to recommendation
135
Complex data & decisions: yes/no?
136
Recommendation The Guidelines Group recommends that fluoroquinolones are / not used in the treatment of all patients with MDR (Strong(conditional) recommendation/ low(moderate, high) grade of evidence)
137
Recommendation: In women with histologically confirmed CIN, the expert panel recommends/suggests cryotherapy/LEEP over cryotherapy/LEEP. Population: Women with histologically confirmed CIN Intervention: Cryotherapy versus LEEP Factor Decision Explanation High or moderate evidence (is there high or moderate quality evidence?) The higher the quality of evidence, the more likely is a strong recommendation. Yes N0 ÅÅOO There is moderate quality evidence from both randomised and observational controlled studies for recurrence rates. However, there is low quality evidence for other outcomes which were considered critical and important for decision making (e.g., severe adverse events, cervical cancer). There is uncertainty for fertility and other obstetrical outcomes, and HIV acquisition/transmission was not measured. Certainty about the balance of benefits versus harms and burdens (is there certainty?) The larger the difference between the desirable and undesirable consequences and the certainty around that difference, the more likely a strong recommendation. The smaller the net benefit and the lower the certainty for that benefit, the more likely is a conditional/ weak recommendation. No Benefits of LEEP were greater, and harms were fewer or similar Recurrence rates of CIN I, CIN II-III and all CINs are probably greater with cryotherapy CIN II-III, OR 3.3 (1.04 to 10.46) CIN I, OR 2.74 (0.62 to 12.07) All CIN, OR 2.14 (1.05 to 4.33) Cryotherapy may be less acceptable to patients than LEEP There may be little difference in serious adverse events between cryotherapy and LEEP, but there may be fewer minor adverse events (such as pain) with cryotherapy It is unclear whether there is a difference in fertility/obstetric outcomes Certainty in or similar values (is there certainty or similarity?) The more certainty or similarity in values and preferences, the more likely a strong recommendation. YES Similar values across women High value was placed on CIN recurrence, serious adverse events and acceptability to the patient Low value was placed on minor adverse events Resource implications (are the resources worth the intervention?) The lower the cost of an intervention compared to the alternative that is considered and other costs related to the decision – that is, the less resources consumed – the more likely is a strong recommendation. More resources required for LEEP Need for more skilled providers to perform LEEP Need for more or expensive equipment/supplies for LEEP; electrical supply for LEEP Need for local anaesthesia with LEEP Overall strength of recommendation Conditional
138
Example: Oseltamivir for Avian Flu
Recommendation: In patients with confirmed or strongly suspected infection with avian influenza A (H5N1) virus, clinicians should administer oseltamivir treatment as soon as possible (strong recommendation, very low quality evidence). Remarks: This recommendation places a high value on the prevention of death in an illness with a high case fatality. It places relatively low values on adverse reactions, the development of resistance and costs of treatment. Schunemann et al. The Lancet ID, 2007
139
Issues in guideline development for immunization
Causation versus effects of intervention Causation not equivalent to efficacy of interventions Bradford Hill Nearly half a century old – tablet from the mountain? Harms caused by interventions Assumption is that removal of vaccine (or no exposure) leads to NO adverse effects How confident can one be that removal of the exposure is effective in preventing disease? Whether immunization or environmental factors: will depend on the intervention to remove exposure
140
Current state of recommendations
141
Current state of recommendations
Reviewed 7527 recommendations 1275 randomly selected Inconsistency across/within 31.6% did not recommendations clearly Most of them not written as executable actions 52.7% did not indicated strength
142
Recommendation The Guideline Group recommends rapid DST testing for resistance to INH and RIF or RIF alone over conventional testing or no testing at the time of diagnosis of TB (conditional, /low quality evidence). Values and preferences: A high value was placed on outcomes such as preventing death and transmission of MDR as a result of delayed diagnosis as well as avoiding spending resources.
143
Group composition Group composition might affect recommendation
Common principle: include all affected by the recommendations ( multi-disciplinary groups incl. patients/carers) – Industry? Keep a manageable size
144
The Process: How to make it constructive?
Group members are heterogeneous and might have different objectives Chair facilitates rather than leads the group Common understanding of goal, tasks and ground rules Similar level of required knowhow and skills Sufficient technical support
145
Balanced participation and formal agreement
Key task of chair Formal consensus processes Delphi Method Nominal group process Voting
146
Group processes
147
How to present controversies
Lay out the controversies Describe the evidence Ask members to focus on the agreed upon evidence and the factors leading to a decision Ask whether there still is disagreement Vote Make voting explicit and transparent (ways of doing this to come tomorrow)
148
Conclusions - Process Success depends on strong chair(s), training of group, good facilitation and technical support Clinical and methods co-chairs Formal consensus developing methods might support agreement on recommendations Voting represents forced consensus Guideline development will require sufficient resources.
149
GRADE Grid
152
Panel P I C O Systematic review evidence profile with GRADEpro Create
Summary of findings & estimate of effect for each outcome Outcomes across studies Rate quality of evidence for each outcome Formulate question Randomization increases initial quality Select outcomes Rate importance Risk of bias Inconsistency Indirectness Imprecision Publication bias P I C O Outcome Critical Grade overall quality of evidence across outcomes based on lowest quality of critical outcomes High Outcome Critical Moderate Grade down Low Outcome Important Very low Outcome Not important Large effect Dose response Confounders Grade up Panel Guideline development Formulate recommendations: For or against (direction) Strong or conditional (strength) By considering: Quality of evidence Balance benefits/harms Values and preferences (Revise by considering:) Resource use (cost) “We recommend using…/should” “We suggest using…/might” “We recommend against using…/might not” “We suggest against using…/should not”
153
Conclusions WHO guidelines should be based on the best available evidence to be evidence based GRADE is the approach used by WHO and gaining acceptance internationally combines what is known in health research methodology and provides a structured approach to improve communication Does not avoid judgments but provides framework Criteria for evidence assessment across questions and outcomes Criteria for moving from evidence to recommendations Transparent, systematic four categories of quality of evidence two grades for strength of recommendations Transparency in decision making and judgments is key
154
Format Mix of seminars/interactive lectures, self directed learning and simulation Large group and smaller group discussion Computer work Simulate guideline panel work Select rapporteur (both for large group and any small group work)
155
Format Mix of seminars/interactive lectures, self directed learning and simulation Large group and smaller group discussion Computer work? Simulate guideline panel work Select rapporteur (for any small group work)
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.