Why Grade Recommendations? strong recommendationsstrong recommendations –strong methods –large precise effect –few down sides of therapy weak recommendationsweak recommendations –weak methods – imprecise estimate – small effect – substantial down sides

Why Grade Recommendations? focus on the evidencefocus on the evidence alternate practitioner behavioralternate practitioner behavior –apply uniformly or –examine evidence themselves –consider patient circumstances –explore with the patient

What Are We Grading? methodological quality of evidencemethodological quality of evidence –likelihood of bias strength of recommendationsstrength of recommendations –must do to might do

Grade of Evidence for Specific Question too vague: alendronate in osteoporosistoo vague: alendronate in osteoporosis patients – post-menopausal womenpatients – post-menopausal women interventionintervention –daily alendronate, dose 10 to 20 mg. outcome – non-vertebral fracturesoutcome – non-vertebral fractures

What Influences Grade? study designstudy design –basic –detailed design and execution consistencyconsistency directnessdirectness reporting biasreporting bias

Methodological Quality study designstudy design –randomization –quasi-randomization –observational study detailed design and executiondetailed design and execution –concealment –balance in known prognostic factors –intention to treat principle observed –blinding –completeness of follow-up

Methodologic Quality consistency of resultsconsistency of results if inconsistency, look for explanationif inconsistency, look for explanation –patients, intervention, outcome, methods no clear thresholdno clear threshold –size of effect, confidence intervals, statistical significance

Relative Risk of Conversion to Sinus Rhythm Amiodarone vs Placebo or Digoxin or CCB Favours Control Favours Amiodarone ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' Cowan (0.78 to 1.58) Noc (1.17 to 276) Capucci (0.37 to 1.62) Cochrane (0.91 to 1.44) Donovan (0.69 to 1.60) Hou (0.97 to 1.72) Kondili (0.71 to 2.47) Galve (0.84 to 1.52) Kontoyannis (1.08 to 1.85) Bellandi (1.15 to 1.72) Cotter (1.15 to 1.80) Kochiadakis (1.19 to 1.78) Bianconi (0.19 to 22.00) Galperin (2.08 to 546) Hohnloser (1.50 to 6.70) Joseph (0.95 to 1.80) Natale (2.60 to 10.00) Peukurinen (1.49 to 4.02) Vardas (1.55 to 2.60) Villani (1.60 to 14.00) Cybulski (1.37 to 2.55) n = 83 n = 95 n = 203 n = 85 n = 120 n = 34 n = 24 n = 40 n = 30 n = 64 n = 39 n = 42 n = 100 n = 42 n = 120 n = 100 n = 204 n = 75 n = 62 n = 208 n = 160

Relative Risk of Conversion to Sinus Rhythm Amiodarone vsPlacebo orDigoxinor CCB FavoursControlFavours Amiodarone ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' Bianconi (0.19 to 22.00) Galperin (2.08 to 546) Hohnloser (1.50 to 6.70) Natale (2.60 to 10.00) Villani (1.60 to 14.00) Pooled Estimate 4.33 (2.76 to 6.77) Cowan (0.78 to 1.58) Noc (1.17 to 276) Capucci (0.37 to 1.62) Cochrane (0.91 to 1.44) Donovan (0.69 to 1.60) Hou (0.97 to 1.72) Kondili (0.71 to 2.47) Galve (0.84 to 1.52) Kontoyannis (1.08 to 1.85) Bellandi (1.15 to 1.72) Cotter (1.15 to 1.80) Kochiadakis (1.19 to 1.78) Joseph (0.95 to 1.80) Peukurinen (1.49 to 4.02) Vardas (1.55 to 2.60) Cybulski (1.37 to 2.55) Pooled Estimate 1.40 (1.25 to 1.57) AF Duration > 48 hrs AF Duration =/< 48 hrs n = 83 n = 95 n = 203 n = 85 n = 120 n = 34 n = 24 n = 40 n = 30 n = 64 n = 39 n = 42 n = 100 n = 42 n = 120 n = 100 n = 204 n = 75 n = 62 n = 208 n = 160 '

Directness of Evidence indirect treatment comparisonsindirect treatment comparisons –interested in A versus B –have A versus C and B versus C alendronate vs risedronatealendronate vs risedronate –both versus placebo, no head-to-head

Four Levels of “Directness” patients meet trials’ eligibility criteriapatients meet trials’ eligibility criteria not included, but no reason to questionnot included, but no reason to question –slight age difference, comorbidity, race some question, bottom line applicablesome question, bottom line applicable –valvular atrial fibrillation serious question about biologyserious question about biology –heart failure trials applicability to aortic stenosis

Levels of Directness interventionsinterventions –same drugs and doses –similar drugs and doses –same class and biology –questionable class and biology outcomesoutcomes –same outcomes –similar (duration, quality of life) –less breathlessness for role function –laboratory exercise capacity for q of life

Magnitude, Precision, Reporting Bias magnitude not generally part of qualitymagnitude not generally part of quality –but very large magnitude can upgrade precision not generally part of qualityprecision not generally part of quality –but sparse data can lower quality reporting biasreporting bias –high likelihood can lower quality

Grading System high qualitywell done RCThigh qualitywell done RCT intermediatequasi-RCTintermediatequasi-RCT lowwell done observationallowwell done observational insufficient anything elseinsufficient anything else

Moving Down study execution – –serious flaws can lower by one level – –fatal flaws can lower by two levels consistency – –important inconsistency can lower by one level directness of evidence – –some uncertainty re relevance lower by one level – –major uncertainty re relevance lower by two levels selection bias – –strong suspicion lower by 1 level

Moving Up very strong association, up 2 levels – –insulin in diabetic ketoacidosis strong, consistent association with no plausible confounders, up 2 levels – –fluoride for preventing cavities strong association can move up 1 level – –? HRT ?

Risk/Benefit tradeoff aspirin after myocardial infarctionaspirin after myocardial infarction –meta-analysis of high quality RCTs –clear positive treatment effect –unequivocal recommendation to treat, uniform practice accelerated TPA vs streptokinase after MIaccelerated TPA vs streptokinase after MI –very large single trial –clear positive effect –uncertainty whether to treat, varying practice

Risk/Benefit tradeoff Aspirin after myocardial infarctionAspirin after myocardial infarction –25% reduction in relative risk –side effects minimal, cost minimal –benefit obviously much greater than risk/cost TPA vs streptokinase after MITPA vs streptokinase after MI –12% reduction in relative risk –increased rate of intracranial hemorrhage –large increase in cost –benefit vs risk/cost a judgment call

Strength of Recommendations Aspirin after MI – do it TPA rather than SK in MI -- probably do it -- probably don’t do it -- probably don’t do it

Grade of Recommendations do itdo it probably do itprobably do it toss-uptoss-up probably don’t do itprobably don’t do it don’t do itdon’t do it

Judgment: Benefits vs Risks/Costs seriousness of outcomeseriousness of outcome magnitude of effectmagnitude of effect precision of treatment effectprecision of treatment effect risk of target eventrisk of target event risk of adverse eventsrisk of adverse events cost of therapycost of therapy valuesvalues

Grades Translations do it - 90% would choose itdo it - 90% would choose it possibly do it – 60% to 90% would choosepossibly do it – 60% to 90% would choose toss-up – 40% to 60% would choosetoss-up – 40% to 60% would choose possibly don’t – 60% to 90% would declinepossibly don’t – 60% to 90% would decline don’t - 90% would declinedon’t - 90% would decline

Conclusion challenges in gradingchallenges in grading –balancing simplicity and complexity –judgement always required consistent grading system requiredconsistent grading system required must consider study design, execution, consistency, directness, reporting biasmust consider study design, execution, consistency, directness, reporting bias –magnitude, precision, only when extreme balance of benefits and risks/costbalance of benefits and risks/cost –magnitude of effects; precision of effects; values and preferences