Teaching Evidence-Based Medicine Gary S Gronseth, MD, FAAN Professor of Neurology University of Kansas

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table Emphasize Evidence never enough –Apply to your patient –Incorporate patient values

A case… A 58 year-old right-handed man suddenly developed problems speaking, right lower facial weakness and right hand clumsiness. His symptoms slowly resolved over a week. He had a history of controlled HTN and no other risk factors. Head MRI: small left frontal infarct. EKG: sinus rhythm. MRA: no cranial artery stenosis. Echocardiogram: PFO

The Physician’s Dilemma To Close or Not to Close Even if the answer is unknown, a decision must be made!

Clinical Reasoning Close PFO? “Where I trained”

To Teach EBM… Explicitly Reason –Exclude the unreasonable Clinical Reasoning Close PFO?

Deceitful

“Closure of PFO in patients with cryptogenic ischemic stroke is the standard of care in the community.” “The consequences of a second stroke are potentially devastating. PFO closure is mandatory.”

Fallacious Irrelevant Rhetoric Psychological appeal Emotion-Driven Persuasion

Patient Intervention Co-intervention Outcome Determining relevance: Define the question

For patients with cryptogenic stroke and PFO does PFO closure vs no PFO closure reduce the risk of the next stroke Determining relevance: Define the question

Popular Appeal “Closure of PFO in patients with cryptogenic ischemic stroke is the standard of care in the community.”

Begging the Question “The consequences of a second stroke are potentially devastating. PFO closure is mandatory.”

Irrelevant Outcomes I’ll be sued. I’ll be reimbursed

Deceitful Fallacious

To Teach EBM… Explicitly Reason –Exclude the unreasonable Deceit Close PFO? Fallacy

Deceitful Fallacious Reasoned

Relevant Logical appeal Data-Driven Truth

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Principles Close PFO? Evidence Judgment

Principles Decision

Deductive Inference From Principles The left side of the brain controls the right side of the body My patient can’t control the right side of his body My patient has a problem with the left side of his brain

Principles Use a Parachute?

Principles Close PFO?

PFO Fibrous adhesions fail to seal the atrial septum after birth Persistence of a potential shunt between the right and left atria of the heart

PFO might allow paradoxical embolism Small emboli normally filtered by lung without clinically important consequence In patients with PFO, emboli can travel to the brain and cause ischemic stroke Closing the PFO will prevent future strokes

Principles Close PFO?

Reasoned Relevant Reason Logical appeal Data-Driven Truth Deduction (Principles)

Principles Close PFO? Evidence

Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke Jane has a stroke and PFO. We should treat her with closure to prevent another stroke. John had a stroke and PFO and was treated with closure, he didn’t have another stroke. Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. Analogy and Inductive Inference

Evidence What happened to patients?

Principles Close PFO? Evidence

Reasoned Relevant Reason Logical appeal Data-Driven Truth Induction (Evidence) Deduction (Principles)

Principles Close PFO? Evidence Judgment Best Guess Opinion Hypothesis

Reasoned Relevant Reason Logical appeal Data-Driven Truth Intuition (Judgment) Induction (Evidence) Deduction (Principles)

Distinguishing Opinion from Principles Is there equipoise? –Do reasonable people disagree? –Would an IRB approve a trial? –Is there an ongoing trial? Evidence separates judgment from principles Principles Close PFO? Evidence Judgment

Theory Scientific Method Experiment Hypothesis

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Principles Close PFO? Evidence Judgment

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy Strong Weak

Jane has a stroke and PFO. We should treat her with closure to prevent another stroke. John had a stroke and PFO and was treated with closure, he didn’t have another stroke. Strong Weak Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke

Inferences from Evidence… Are not valid or invalid Are never certain Strong Weak

Jane has a stroke and PFO. We should treat her with closure to prevent another stroke. John had a stroke and PFO and was treated with closure, he didn’t have another stroke. Strong Weak Sue had a stroke and PFO and was treated with closure, she didn’t have another stroke. Bob had a stroke and PFO and wasn’t treated with closure, he had another stroke Informally recalled cases Why is this a weak inference?

Inferences from informally recalled cases can mislead Too few cases Selective recall: remember those –That are more recent –With extreme results –That support our pre-conceptions Experts are not immune to these limitations

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error

Often too few cases Selective recall: remember those –That are more recent –With extreme results –That support our pre- conceptions Two Sources of Error Systematic Bias Random Chance

Find More Cases Retrospective Observational Sudy 2002 to 2010 Of all Stroke and PFO Cases: 319

Rats… I’m going to have to start counting these cases

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table

Relationships between variables PFO Closure and Stroke

2 X 2 Table Outcome TreatmentNo stroke StrokeAll Closure167 No Closure152 Total

Expected if No Relationship Outcome TreatmentNo stroke StrokeAll Closure No Closure Total

Expected if no Realtionship Outcome TreatmentNo stroke StrokeAll Closure75%25%100% No Closure75%25%100% Total75%25%100%

“Actual” Outcome TreatmentNo stroke StrokeAll Closure No Closure Total

“Actual” Outcome TreatmentNo stroke StrokeAll Closure90%10%100% No Closure59%41%100% Total75%25%100%

2 X 2 Table Outcome TreatmentNo stroke StrokeAll Closureab167 No Closurecd152 Total

Relative Risk stroke = b/(a+b) d/(c+d) Risk difference stroke = b/(a+b) - d/(c+d) Measures of Association Outcome TreatmentNo stroke StrokeAll Closureab167 No Closurecd152 Total

Measure of Association Relative Risk Stroke Outcome Outcome TreatmentNo stroke Stroke Closure90%10% No Closure59%41% RR Stroke 10/41 = 0.24

Cryptogenic stroke patients receiving Closure were 0.24 times less likely to have stroke. Therefore, I should offer my patients with stroke and PFO Closure.

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table

Random (Sampling) Error --Incorrect result from bad luck Equally likely to be too high or too low Statistical power/precision --Measured by: P-values (p < 0.001) Confidence intervals RR 0.24: (95% confidence intervals 0.15 to 0.40)

Systematic Error Incorrect results from poor study design or execution More likely to be too high or too low Risk of Bias Measured: Semi-quantitatively Class of Evidence Truth Measured

Our Study +Cl -Cl Stroke No Stroke Patients not receiving Closure were more often older, diabetic and hypertensive Sometimes had to “guess” the outcome from the record.

Major Sources of Bias +Cl -Cl Poor Good Confounding Misclassification

Lower Risk of Bias The Randomized Masked Trial +Cl -Cl Poor Good R

Randomized Masked Trial Single Case Report What is the risk of Bias? Strong Weak

Find the best evidence Search online databases MEDLINE

There is insufficient evidence to support or refute the benefit or lack of harm of PFO closure. Conclusion Strong Weak

Despite the weak evidence, a decision must be made. Decide Strong Weak

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table Emphasize Evidence never enough –Apply to your patient –Incorporate patient values Principles Close PFO? Evidence Judgment

Induction is never certain Often the evidence is weak Even when strong, the Evidence never perfectly applies to your patient Explicitly consider how well the evidence applies to your patient

Incorporating patient values BenefitsRisks Uncertainty

Know what is not Known If you fail to acknowledge the uncertainty and tell the patient we know that the PFO should or should not be closed… You have failed to distinguish opinion from principles. Principles Close PFO Evidence Judgment

To Teach EBM… Explicitly Reason –Exclude the unreasonable –Distinguish opinion from principles Rate Evidence on a Hierarchy –Understand two sources of error –Love the 2 x 2 table Emphasize Evidence never enough –Apply to your patient –Incorporate patient values Principles Decision Evidence Judgment