VSM CHAPTER 6: HARM Evidence-Based Medicine How to Practice and Teach EMB

VSM Harm Concerns about potentially harmful interventions occur daily We must make judgments about whether our patients may be at risk Evaluate evidence about causation –Validity –Importance –Relevance to our patients Goal is to avoid false + and false -

VSM Types of Studies on Harm/Etiology Systematic reviews –Best evidence about effects of therapy RCT –Seldom large enough to detect rare adverse events –Ill suited in size, duration and ethics Cohort, case-control studies, and cross-sectional studies

VSM Is this evidence about harm valid? 1.Were there clearly defined groups of patients, similar in all important ways other than exposure to treatment or other cause? 2.Were treatments/exposures and clinical outcomes measured in the same ways in both groups? 3.Was the follow-up appropriate? 4.Do the results of the harm study fulfill some of the diagnostic tests for causation?

VSM Is this evidence about harm valid? 1.Were there clearly defined groups of patients, similar in all impt ways other than exposure to treatment or cause? 2.Were treatments/exposures and clinical outcomes measured in the same ways in both groups? 3.Was the follow-up long enough? 4.Do the results of the harm study fulfill some of the diagnostic tests for causation?

VSM Is it Valid? Best  Systematic review or RCT –Randomization would make groups identical for all other causes –Ill suited (in size, duration and ethics) The validity of the study designs used to detect harm is inversely proportional to their feasibility. Must often look at other types of studies

VSM Cohort study Observational study Group of pts who are exposed and group of pts not exposed are followed for development of outcome Not randomized—exposure based on patient’s/MD’s preferences –Yields more confounders –Multivariable analysis useful, but adjustments can only be made for known confounders –Still may not be useful if outcome rare

VSM Case-Control Studies “cases” pts with outcome of interest “controls” those without Proportion of each groups exposure assessed retrospectively Study design easy for exploring possible relationships b/t many exposures and outcome of interest –Even greater potential for confounders –If a large number of associations are explored, a statistically significant finding may be due to chance alone.

VSM Cross-Sectional Studies Most common for looking at etiology Look at group with outcome and group without, and compare exposure Exposures and outcomes measured at same time –which came first? –Lots of confounders

VSM Case Series/Case Reports Look at few pts with adverse outcome while receiving suspected treatment Useful in rare and dramatic outcomes Lack comparison groups Usually only sufficient for hypothesis generation

VSM Is this evidence about harm valid? 1.Were there clearly defined groups of patients, similar in all impt ways other than exposure to treatment or cause? 2.Were treatments/exposures and clinical outcomes measured in the same ways in both groups? 3.Was the follow-up long enough? 4.Do the results of the harm study fulfill some of the diagnostic tests for causation?

VSM Are outcomes and exposures measured the same way? RCT’s –Best if exposures and outcomes measured in same way in both groups Cohort studies –Want outcome assessors blinded to exposure Case-control studies –Want pts and assessors blinded to outcome and study hypothesis

VSM Is this evidence about harm valid? 1.Were there clearly defined groups of patients, similar in all impt ways other than exposure to treatment or cause? 2.Were treatments/exposures and clinical outcomes measured in the same ways in both groups? 3.Was the follow-up appropriate? 4.Do the results of the harm study fulfill some of the diagnostic tests for causation? 1.Is it clear the exposure preceded the onset of the outcome? 2.Is there a dose-response gradient? 3.Is the association consistent from study to study? 4.Does the association make biological sense?

VSM Was follow-up appropriate? Complete –Ideally want no pts lost to follow-up –20% loss to follow-up cut off Long enough –Want sufficient length of time to ensure all outcomes included

VSM Is this evidence about harm valid? 1.Were there clearly defined groups of patients, similar in all impt ways other than exposure to treatment or cause? 2.Were treatments/exposures and clinical outcomes measured in the same ways in both groups? 3.Was the follow-up long enough? 4.Do the results of the harm study fulfill some of the diagnostic tests for causation?

VSM Do the results of the harm study fulfill some of the diagnostic tests for causation? 1.Is it clear the exposure preceded the onset of the outcome? 2.Is there a dose-response gradient? 3.Is the association consistent from study to study? 4.Does the association make biological sense?

VSM Are the results important? If study passes validity test, we must decide if association is strong enough for us to act upon Magnitude of association –Use relative risk or odds ratio Precision of estimate of association –Use confidence interval

VSM Relative Risk RCT and cohort studies Relative Risk (RR) RR= proportion of those with outcome in exposed and not exposed groups

VSM Relative Risk Example: Exposure group –1000 pts receive Tx –20 develop outcome –a/(a+b) –20/1000= 2% Not exposed group –1000 pts no Tx –2 experience outcome –c/(c+d) –2/1000=0.2% RR=(a/(a+b))/(c/c+d) RR= 2%/0.2%=10 PresentcaseAbsentcontrols exposeda20B980a+b1000 Not exposed C2D998c+d1000 Adverse outcome Patients receiving treatment are 10x more likely to experience the outcome.

VSM Odds Ratio Case-control studies –Can’t calculate RR in studies because the investigator selects the people with the outcomes (rather than the exposure) –Can’t calculate “incidence” –Use “odds ratio” or relative odds OR=ad/bc

VSM Odds Ratio Ie: urge incontinence –100pt with urge incontinence 90 with hx of caffeine exp 10 without caffeine exp –100pts without urge incontinence 45 with exposure to caffeine 55 without caffeine use OR=ad/bc –OR=(90x55)/(45x10) –OR=11 casecontrols exposeda90B45 Not exposed C10D55 Odds of experiencing urge incontinence for people with caffeine use is 11 times that of those who did not use caffeine

VSM OR and RR OR and RR >1 –Increased risk of adverse event OR and RR=1 –Adverse event is no more likely to occur OR and RR <1 –Adverse event is less likely to occur OR and RR approximate each other when event rate and treatment effect are small

VSM How big should the RR or OR be for us to be impressed? Strength of study design/validity determine Professor Irwig –Suggests we compare the unadjusted measure of association with one in which at least one known confounder has been adjusted out –If adjustment produces a large decline in RR or OR, be suspicious of spurious association –If adjustment yields stable OR or RR, or if it rises, our confidence in the validity should be greater

VSM Number Needed to Harm (NNH) OR and RR tell about strength of association NNH=number of pts who need to be exposed to the causal agent to produce one additional harmful event

VSM RCT and Cohort Studies Can be calculated directly from trials and cohort studies Reciprocal of difference in adverse events NNH= 1/(a/(a+b))-(c/(c+d))

VSM Case Control Studies Can’t determine incidence Complex If OR<1 –1-(PEER(1-OR))/PEER(1-PEER)(1-OR) If OR>1 –1+(PEER(OR-1))/PEER(1-PEER)(OR-1) PEER= patient expected event rate Different PEERs with same OR can lead to different NNH

VSM Confidence interval In addition to magnitude of OR or RR must look at its precision Credibility is highest when entire CI is narrow and remains within a clinically importantly increased risk

VSM Can this valid and important evidence about harm be applied to our patient? 1.Is our patient so different from those included in the study that its results cannot apply? 2.What is our patient’s risk of benefit and harm from the agent? 3.What are our patient’s preferences, concerns, and expectations? 4.What alternatives are available?