1. Mark Pletcher 6/10/2011 Quantifying Treatment Effects

2. Rationale Any treatment involves tradeoffs Weigh benefits against risks/costs Benefit $$ Harm

3. Rationale Sometimes the decision is difficult! Benefit $$ Harm

4. Rationale Benefit $$ Harm How big is this box? And this one?

5. Rationale Tests can help us understand who is most likely to benefit from a treatment Benefit $$ Harm How big is this box? And this one?

6. Rationale Tests can help us understand who is most likely to benefit from a treatment Rapid strep to decide who will benefit from penicillin BNP to decide who will benefit from furosemide CRP to decide who will benefit from statins

7. Rationale The utility of a test depends on: How beneficial the treatment is How harmful the treatment is How much the test tells us about these benefits and harms in a given individual Risk of harm from the test itself

8. Rationale The utility of a test depends on: How beneficial the treatment is How harmful the treatment is How much the test tells us about these benefits and harms in a given individual Risk of harm from the test itself The topic for this lecture

9. Outline Is an intervention really beneficial? How beneficial is it? Pitfalls Examples

10. Is the intervention beneficial? Randomized trials compare an outcome in treated to untreated persons MI in 10% vs. 15% Duration of flu symptoms 3 vs. 5 days

11. Is the intervention beneficial? Randomized trials compare an outcome in treated to untreated persons MI in 10% vs. 15% Duration of flu symptoms 3 vs. 5 days *Statistics* are used to decide if should reject the null hypothesis and accept that the intervention is beneficial

12. Is the intervention beneficial? But statistics cannot help us interpret effect size

13. Quantifying the Benefit Effect size How do we summarize and communicate this? What is really important for clinicians and policymakers?

14. Quantifying the Benefit Effect size How do we summarize and communicate this? What is really important for clinicians and policymakers? Example: MI in 10% vs. 15% Q: What do we do with these two numbers?

15. Quantifying the Benefit Two simple possibilities: 10% / 15% = 0.66 15% - 10% = 5%

16. Quantifying the Benefit Two simple possibilities: 10% / 15% = 0.66 15% - 10% = 5% Relative Risk (RR) Absolute Risk Reduction (ARR)

17. Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small?

18. Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime

19. Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime Medium Big Small

20. Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime RR is NOT the best measure of effect size

21. Quantifying the Benefit Absolute risk reduction (ARR) is better ARR = Risk difference = Risk2 – Risk1

22. Quantifying the Benefit Absolute risk reduction (ARR) is better RRARR MI: 10% vs. 15% in 10 years.665% Death: 50% vs. 75% in 3 years.6625% Basal Cell CA: 2% vs. 3% in lifetime.661%

23. Q: What does the 34% reduction mean?

24. Nimotop® Ad Graph 22% 33% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% =.66 ARR = 33% - 22% = 11%

25. Nimotop® Ad Graph 22% 33% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% =.66 ARR = 33% - 22% = 11% What is 34%?

26. Nimotop® Ad Graph 22% 33% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% =.66 ARR = 33% - 22% = 11% Relative risk reduction (RRR) = 1 – RR = 1-.66 =.34 or 34%

27. Quantifying the Benefit RRR is no better than RR RRRRR MI: 10% vs. 15% in 10 years.6634% Death: 50% vs. 75% in 3 years.6634% Basal Cell CA: 2% vs. 3% in lifetime.6634%

28. Quantifying the Benefit RRR is ALWAYS bigger than ARR (unless untreated risk is 100%)

29. Quantifying the Benefit BEWARE of risk reduction language!! ARR or RRR? We reduced risk by 34% Risk was 34% lower

30. Quantifying the Benefit BEWARE of risk reduction language!! ARR or RRR? We reduced risk by 34% cant tell Risk was 34% lowercant tell Very hard to be unambiguous!

31. Quantifying the Benefit Another reason that ARR is better: Translate it into Number Needed to Treat NNT = 1/ARR

32. Why is NNT = 1/ARR? 67 no stroke anyway 22 strokes with Nimotop® 11 strokes prevented 22 strokes with with treatment 33 strokes with no treatment 100 SAH patients treated R2 R1

33. Why is NNT 1/ARR? Treat 100 SAH patients prevent 11 strokes Ratio manipulation: 100 treated1 treated9.1 treated 11 prevented.11 prevented1 prevented ==

34. Why is NNT 1/ARR? Treat 100 SAH patients prevent 11 strokes Ratio manipulation: 100 treated1 treated9.1 treated 11 prevented.11 prevented1 prevented == 1/ARR= NNT

35. Why is NNT 1/ARR? NNT best expressed in a sentence: Need to treat 9.1 persons with SAH using nimodipine to prevent 1 cerebral infarction

36. Quantifying the Benefit NNT calculation practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% Death: 50% vs. 75% in 3 years.6625% Basal Cell CA: 2% vs. 3% in lifetime.661%

37. Quantifying the Benefit NNT calculation practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% 20 Death: 50% vs. 75% in 3 years.6625% Basal Cell CA: 2% vs. 3% in lifetime.661%

38. Quantifying the Benefit NNT calculation practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% 20 Death: 50% vs. 75% in 3 years.6625% 4 Basal Cell CA: 2% vs. 3% in lifetime.661%

39. Quantifying the Benefit NNT calculation practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% 20 Death: 50% vs. 75% in 3 years.6625% 4 Basal Cell CA: 2% vs. 3% in lifetime.661% 100

40. Quantifying the Benefit NNT expression practice RRARR NNT? MI: 10% vs. 15% in 10 years.665% 20 Death: 50% vs. 75% in 3 years.6625% 4 Basal Cell CA: 2% vs. 3% in lifetime.661% 100 Statins Chemo Sunscreen every day

41. Quantifying the Benefit NNT expression practice Need to treat 20 patients with statins for 10 years to prevent 1 MI Need to treat 4 patients with chemo for 3 years to prevent 1 death Need to treat 100 patients with sunscreen every day for their whole life to prevent 1 basal cell

42. Example 1 Randomized controlled trial of the effects of hip replacement vs. screws on re-operation in elderly patients with displaced hip fractures. Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

43. Example 1 Re-operationNo Re-operation Hip Replacement12217229 Internal Fixation with Screws90136226 Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

44. Example 1 Re-operationNo Re-operationRisk Hip Replacement1221722912/229 =5.2% Internal Fixation with Screws9013622690/226 =39.8%

45. Example 1 Re-operationNo Re-operationRisk Hip Replacement1221722912/229 =5.2% Internal Fixation with Screws9013622690/226 =39.8% RR= R1/R2= 5.2% / 39.8%=.13 RRR= 1-RR= 1-.13= 87% ARR= R2 – R1= 39.8% - 5.2%= 34.6% NNT= 1/ARR= 1/.346= 3 Need to treat 3 patients with hip replacement instead of screws to prevent 1 from needing a re-do operation

46. Example 2 JUPITER: Randomized controlled trial of high dose rosuvastatin in patients with LDL 2.0 Ridker et al. NEJM 2008; 359:2195-207

47. Example 2 Ridker et al. NEJM 2008; 359:2195-207

48. Example 2 Ridker et al. NEJM 2008; 359:2195-207

49. Example 2 HR= (R1/R2)(from regression)=.56 RRR= 1-HR= 1-.56= 44% ARR= R2 – R1= 1.36 - 0.77=.59 / 100py* =.0059 / py NNT= 1/ARR= 1/.0059 = 100/.59= 169 pys Need to treat 169 patients for a year to prevent 1 CVD event Or better: Need to treat 85 patients for 2 years to prevent 1 CVD event (average treatment duration in trial was 1.9 years) * py = person-years

50. Example 4 Warfarin vs. placebo for atrial fibrillation WarfarinPlacebo Risk of major bleed (/yr)1.2%0.7% Ann Intern Med 1999; 131:492-501

51. Example 4 Warfarin vs. placebo for atrial fibrillation RR= R1/R2= 1.2% /.7%= 1.7 RR (flipped) = R2/R1=.7% / 1.2%=.59 RRR (flipped) = 1-RR= 1 -.59= 41% ARR= R2 – R1=.7% - 1.2%= -.5% ARI – Absolute risk increase = 0.5% NNT= 1/ARR= 1/-.5%= -200 NNH – Number needed to harm = -NNT = 1/ARI = 200 If you treat 200 Afib patients with warfarin, you will cause 1 major bleed

52. Circling back to test utility… Tests help determine: If the RR applies Treatment for a disease doesnt help if you dont have the disease! Interactions (RR is higher or lower than average) Statins more effective if CRP is high? Patients with gene XYZ more likely to have a side effect Baseline risk The higher the risk, the larger the ARR, the smaller the NNT

53. Key Concepts Test utility depends on how good the treatment is RR and p-values good for hypothesis testing/statistics ARR and NNT (and NNH) better for interpreting clinical importance ARR = risk difference NNT = 1/NNT Beware RRR and ambiguous language