Plan GRADE backgroundGRADE background confidence in estimates (quality of evidence)confidence in estimates (quality of evidence) evidence profilesevidence profiles confidence in estimates and recommendationsconfidence in estimates and recommendations

Dilemma: proliferation of systems Solution: common international grading system? GRADE (Grades of recommendation, assessment, development and evaluation)GRADE (Grades of recommendation, assessment, development and evaluation) international groupinternational group –Australian NMRC, SIGN, USPSTF, WHO, NICE, Oxford CEBM, CDC, CC ~ 30 meetings over last twelve years~ 30 meetings over last twelve years (~10 – 50 attendants)(~10 – 50 attendants)

60+ Organizations

GRADE uptake

Confidence in estimate (quality of evidence) no confidence totally confident High Moderate Low Very Low Randomized trials start high confidence Observational studies start low confidence

Determinants of confidence risk of bias –concealment – blinding – loss to follow-up imprecision –wide confidence intervals publication bias

Relative Risk with 95% CI for Vitamin D Non-vertebral Fractures

25% 50% 75% No worries Some concern Serious concern Why are we Pooling?

Relative Risk with 95% CI for Vitamin D Non-vertebral Fractures

Quality judgments: Directness populationspopulations –older, sicker or more co-morbidity interventionsinterventions –warfarin in trials and community outcomesoutcomes –important versus surrogate outcomes –glucose control versus CV events

What can raise confidence? large magnitude can upgrade one levellarge magnitude can upgrade one level –very large two levels common criteriacommon criteria –everyone used to do badly –almost everyone does well –quick action hip replacement for hip osteoarthritiship replacement for hip osteoarthritis mechanical ventilation in respiratory failuremechanical ventilation in respiratory failure

Quality assessment criteria

Nonfatal MI – Fixed Effects

Quality Assessment Summary of Findings Quality Relative Effect (95% CI) Absolute risk difference Outcome Number of participants (studies) Risk of Bias ConsistencyDirectnessPrecision Publication Bias Myocardial infarction 10,125 (9) No serious limitations No serious imitations No serious limitations Not detected High 0.71 (0.57 to 0.86) 1.5% fewer (0.7% fewer to 2.1% fewer) ) Beta blockers in non-cardiac surgery

Mortality – Fixed Effects

Quality Assessment Summary of Findings Quality Relative Effect (95% CI) Absolute risk difference Outcome Number of participants (studies) Risk of Bias ConsistencyDirectnessPrecision Publication Bias Myocardial infarction 10,125 (9) No serious limitations No serious imitations No serious limitations Not detected High 0.71 (0.57 to 0.86) 1.5% fewer (0.7% fewer to 2.1% fewer) Mortality 10,205 (7) No serious limitations Possiblly inconsistent No serious limitations Imprecise Not detected Moderate or low 1.23 (0.98 – 1.55) 0.5% more (0.1% fewer to 1.3% more) Beta blockers in non-cardiac surgery

Stroke – Fixed Effects Total events 75/10,290

Quality Assessment Summary of Findings Quality Relative Effect (95% CI) Absolute risk difference Outcome Number of participants (studies) Risk of Bias ConsistencyDirectnessPrecision Publication Bias Myocardial infarction 10,125 (9) No serious limitations No serious imitations No serious limitations Not detected High 0.71 (0.57 to 0.86) 1.5% fewer (0.7% fewer to 2.1% fewer) Mortality 10,205 (7) No serious limitations Possiblly inconsistent No serious limitations Imprecise Not detected Moderate or low 1.23 (0.98 – 1.55) 0.5% more (0.1% fewer to 1.3% more) Stroke 10,889 (5) No serious limitaions No serious limitations Not detected High 2.21 (1.37 – 3.55) 0.5% more (0.2% more to 1.3% more0 Beta blockers in non-cardiac surgery

Significance of strong vs weak variability in patient preferencevariability in patient preference –strong, almost all same choice (> 90%) –weak, choice varies appreciably interaction with patientinteraction with patient –strong, just inform patient –weak, ensure choice reflects values use of decision aiduse of decision aid –strong, don’t bother –weak, use the aid quality of care criterionquality of care criterion –strong, consider –weak, don’t consider

Strength of Recommendation strong recommendation –benefits clearly outweigh risks/hassle/cost –risk/hassle/cost clearly outweighs benefit what can downgrade strength? low confidence in estimates close balance between up and downsides

Conclusion clinicians, policy makers need summariesclinicians, policy makers need summaries –quality of evidence explicit rulesexplicit rules –transparent, informative GRADEGRADE –simple, transparent, systematic