1. Evidence Based Medicine for the Athletic Trainer: What is It?
July 11, 2015
BOC Approved EBP CEUs
Evidence-Informed Clinical Practice for the Athletic Trainer:
Evidence Based Medicine for the Athletic Trainer: What is It?
Jennifer M. Medina McKeon, PhD, ATC, CSCS
…while others wandered around…and dined with the rich and powerful …we on the other hand labored diligently…we first learned all the things that were creditably discovered by the ancients; then through deeds we both tested and practiced them. -Galen ( CE [disputed]) Method of Medicine 9.4, K

2. “EBP is just a fad.”
…while others wandered around…and dined with the rich and powerful …we on the other hand labored diligently…we first learned all the things that were creditably discovered by the ancients; then through deeds we both tested and practiced them. -Galen ( CE [disputed]) Method of Medicine 9.4, K
EBP is a just a fad [Galen slide]
During the last Screen Cast, we discussed that EBP is not a new thing. Further, in most cases, and without even knowing it, we have been practicing EBP all along. Hopefully, at the end of this course, you will have some new tools with which to actively recognize how you practice EBP, and also the recognition that there are many EBP concepts that you are already utilizing in everyday practice or in teaching your students.

3. What EBP is not…
“All this EBP stuff just means that they think that clinicians are all supposed to become researchers now”
“All this EBP stuff just means that they think that clinicians are all supposed to become researchers now”
The role of EBP is not to make everyone a research scientist. What we do want to foster is the creation of the clinician-scientist – critical thinkers who bring a questioning mind and engage in life-long learning.

4. “EBP means that we all need to be doing stats on everything.”
What EBP is not…
“EBP means that we all need to be doing stats on everything.”
“EBP means that we all need to be doing stats on everything.”
It’s true that doing EBP means that at times we use numerical values to quantify the effect of a treatment, the risk of an injury, how important key features are in a diagnosis…but this does not mean that we run a statistical test, such as an ANOVA or regression on everything that we do. There is also a critical element of selecting which values are logical to document – those that are most important to you in your own clinical practice.
We all know that you can prove anything with statistics…
…I recently proved that nobody likes statistics, except a few professors
- Chottiner (1990)

5. What EBP is not…
“EBP means that someone should read all the research that’s out there on a topic, and tell me what to do.”
EBP means that someone should read all the research that’s out there on a topic, and tell me what to do.”
While other researchers can help to summarize the literature, it is up to the individual clinician to identify what is most critically important to their own practice and to look for factors that may alter the way that they interpret the research study that they are reading.

6. What EBP is not…
“Ideally, we would like to see a clinician encounter an injury, and right there, pull out a laptop or tablet, and look up the most recent article on that injury, and incorporate what the study says right there on the spot.”
Ideally, we would like to see a clinician encounter an injury or clinical problem, and right there, pull out a laptop or tablet, and look up the most recent article on that injury, and incorporate what the study says right there on the spot.”
Who has time for that? Incorporating the best available evidence does include reading and consuming the scientific literature. But it also includes critique and reflection, not just blind application of the recommendations from one study.

7. What is the EBP?
So now that we’ve gone over some of the most common misconceptions, we will discuss what EBP is.

8. Evidence-Based Practice
Best Available
Evidence
Patient
Values
Clinical
Experience
“Integration of the best available research evidence with clinical experience
and patient values to make clinical decisions” – Sackett et al, BMJ 1996.

9. BETTER RESEARCH EVIDENCE
Hierarchy of Clinical Research Evidence
BETTER RESEARCH EVIDENCE
Meta-analyses
Systematic Reviews
Randomized Clinical Trials
Cohort Studies
(Prospective)
Case-Control Studies
(Retrospective)
Case Series
Case Studies
Unpublished Clinical Observations
HARDER TO DO
More Control
Less Risk of Bias
More Subjects Involved 9 9

10. Critical Appraisal
Next, we will review concepts related to Critical Appraisal of the literature.

11. Critical Appraisal
Critical appraisal is the process of determining a study’s worth.

12. Critical Appraisal More than Peer Review
Vital Step to making INFORMED clinical decisions
Combination of External Evidence (the stuff you find) & the Internal Evidence (the stuff you know)
It goes beyond the peer review process that a study goes through on the way to being published.

13. Critical Appraisal More than Peer Review
Vital Step to making INFORMED clinical decisions
Combination of External Evidence (the stuff you find) & the Internal Evidence (the stuff you know)
Critical appraisal is a vital step in consuming the literature and making a judgement of its truthfulness and value to you and your clinical practice.

14. Critical Appraisal More than Peer Review
Vital Step to making INFORMED clinical decisions
Combination of External Evidence (the stuff you find) & the Internal Evidence (the stuff you know)
We will now take a look at WHAT to look for as we go through the process of appraising the evidence that we find.

15. Critical Appraisal More than Peer Review
Vital Step to making INFORMED clinical decisions
Combination of External Evidence (the stuff you find) & the Internal Evidence (the stuff you know)
We will now take a look at WHAT to look for as we go through the process of appraising the evidence that we find.

16. YUCK – that sounds like research
RESEARCH EVIDENCE
What is the BEST SPECIAL TEST???
What is the BEST TREATMENT???
What makes a test good?
Accurate (Valid)
Consistent results (Reliable)
YUCK – that sounds like research

17. YUCK – that sounds like research
RESEARCH EVIDENCE: The Best Tests
What makes a test good?
Accurate (Valid)
Consistent results (Reliable)
YUCK – that sounds like research

18. RESEARCH EVIDENCE: The Best Tests
Accurate (Valid)
Consistent results (Reliable)
What makes a test good?
SENSITIVITY
SPECIFICTY
Definition
Proportion of patients WITH the injury who test POSITIVE
Proportion of patients WITHOUT the injury who test NEGATIVE
100% Sensitivity
(Perfect Score)
Test correctly identifies every person who HAS the injury
Test correctly identifies every person who DOES NOT HAVE the injury
Outcome
True Positive Rate
True Negative Rate
“Ideal” Test Result
Negative
Positive
“Ideal” Test Interpretation
(What to tell the Patient)
Definitely NEGATIVE
(Patient is OK)
Definitely POSITIVE
(Patient is Injured)
“The RULE”
RULE OUT (SnOUT)
If Test is Negative, You can Rule Out injury (Patient is OK)
RULE IN (SpIN)
If Test is Positive, You can Rule In injury (Patient is Injured)
Poor Test
Lots of FALSE NEGATIVES
Lots of FALSE POSITIVES
Adapted from
Accessed January 23, 2015

19. RESEARCH EVIDENCE: The Best Tests
Accurate (Valid)
Consistent results (Reliable)
What makes a test good?
SENSITIVITY
SPECIFICTY
Definition
Proportion of patients WITH the injury who test POSITIVE
Proportion of patients WITHOUT the injury who test NEGATIVE
100% Sensitivity
(Perfect Score)
Test correctly identifies every person who HAS the injury
Test correctly identifies every person who DOES NOT HAVE the injury
Outcome
True Positive Rate
True Negative Rate
“Ideal” Test Result
Negative
Positive
“Ideal” Test Interpretation
(What to tell the Patient)
Definitely NEGATIVE
(Patient is OK)
Definitely POSITIVE
(Patient is Injured)
“The RULE”
RULE OUT (SnOUT)
If Test is Negative, You can Rule Out injury (Patient is OK)
RULE IN (SpIN)
If Test is Positive, You can Rule In injury (Patient is Injured)
Poor Test
Lots of FALSE NEGATIVES
Lots of FALSE POSITIVES
Adapted from
Accessed January 23, 2015

20. RESEARCH EVIDENCE: The Best Tests
Accurate (Valid)
Consistent results (Reliable)
What makes a test good?
SENSITIVITY
SPECIFICTY
Definition
Proportion of patients WITH the injury who test POSITIVE
Proportion of patients WITHOUT the injury who test NEGATIVE
100% Sensitivity
(Perfect Score)
Test correctly identifies every person who HAS the injury
Test correctly identifies every person who DOES NOT HAVE the injury
Outcome
True Positive Rate
True Negative Rate
“Ideal” Test Result
Negative
Positive
“Ideal” Test Interpretation
(What to tell the Patient)
Definitely NEGATIVE
(Patient is OK)
Definitely POSITIVE
(Patient is Injured)
“The RULE”
RULE OUT (SnOUT)
If Test is Negative, You can Rule Out injury (Patient is OK)
RULE IN (SpIN)
If Test is Positive, You can Rule In injury (Patient is Injured)
Poor Test
Lots of FALSE NEGATIVES
Lots of FALSE POSITIVES
“New Rule”
High SNNN
High SeNsitivity, Negative test, Negative pt.
High SPPP
High SPecificity, Positive test, Positive pt.
Adapted from
Accessed January 23, 2015

21. RESEARCH EVIDENCE: The Best Tests
QUESTIONS TO ASK
Bias
Comparison
Generalizability

22. Generalizability RESEARCH EVIDENCE: The Best Tests Bias Comparison
QUESTIONS TO ASK
Is the Test Relevant to your practice?
Who was the Test performed on?

23. RESEARCH EVIDENCE: The Best Tests
Bias
Comparison
Generalizability

24. RESEARCH EVIDENCE: The Best Tests
Bias
Comparison
Generalizability
QUESTIONS TO ASK
Who did the Test?
Was the Test compared to a Gold Standard? Was the Test tried in a group of Patients who could conceivably have the Injury?

25. RESEARCH EVIDENCE: The Best Tests
Bias
Comparison
Generalizability

26. RESEARCH EVIDENCE: The Best Tests
Bias
Comparison
Generalizability
QUESTIONS TO ASK
Did the testers “know” if the patient had the Injury?
Did ALL patients tested also get tested with the Gold Standard?

27. RESEARCH EVIDENCE: The Best Tests
QUESTIONS TO ASK
Is the Test Relevant to your practice?
Who was the Test performed on?
Who did the Test?
Was the Test compared to a Gold Standard?
Was the Test tried in a group of Patients who could conceivably have the Injury?
Did the testers “know” if the patient had the Injury?
Did ALL patients tested also get tested with the Gold Standard?
Bias
Comparison
Generalizability
Improve your ability to
Critically Analyze Diagnostic Test research
IMPROVE YOUR OWN PRACTICE

28. PREDICTION
BEST PREDICTORS =
Get rid of these…
they don’t add much

29. Clinical Prediction Rules (the other CPR)
Application, not Development
Combination of 3 or more specific patient characteristics that provides a quantifiable probability for diagnosis (or prognosis or benefit for a procedure)
Best combination of S/S to predict probability of Injury
Calculated for Maximized Prediction
Support the decisions that you make
Considerations
Must be validated
Do not guarantee n outcome (a probability)
Used fore making predictions, not recommendations (what is likely to happen, not what to do next)
“Suggest” vs. “Recommend” next course of action Clinical Decision Rule

30. Clinical Prediction Rules (the other CPR)
Application, not Development
Combination of 3 or more specific patient characteristics that provides a quantifiable probability for diagnosis (or prognosis or benefit for a procedure)
Best combination of S/S to predict probability of Injury
Calculated for Maximized Prediction
Support the decisions that you make
Considerations
Must be validated
Do not guarantee an outcome (they’re a probability)
Used fore making predictions, not recommendations (what is likely to happen, not what to do next)
“Suggest” vs. “Recommend” next course of action (vs. Clinical Decision Rule)

31. RESEARCH EVIDENCE: The Best Treatments
What makes a treatment good?

32. Posing a Clinical Question
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question

33. P I C O Posing a Clinical Question
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question P I C O

34. P I C O Posing a Clinical Question
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question P
Patient or Problem or Population
What are the most important Characteristics of your Patient? I C O

35. P I C O Posing a Clinical Question
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question P
Patient or Problem or Population
What are the most important Characteristics of your Patient? I
Independent Variable
(Intervention or Assessment or Exposure)
What is the main Intervention, Prognostic factor, Diagnostic factor, or Exposure you are considering for your patient? C O

36. P I C O Posing a Clinical Question
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question P
Patient or Problem or Population
What are the most important Characteristics of your Patient? I
Independent Variable
(Intervention or Assessment or Exposure)
What is the main Intervention, Prognostic factor, Diagnostic factor, or Exposure you are considering for your patient? C
Comparison or Control
What is the main alternative to the Intervention / Assessment / Exposure? O

37. P I C O Posing a Clinical Question
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question P
Patient or Problem or Population
What are the most important Characteristics of your Patient? I
Independent Variable
(Intervention or Assessment or Exposure)
What is the main Intervention, Prognostic factor, Diagnostic factor, or Exposure you are considering for your patient? C
Comparison or Control
What is the main alternative to the Intervention / Assessment / Exposure? O
Outcome of Interest
What is considered “success”

38. P I C O Posing a Clinical Question
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question P
Patient or Problem or Population
What are the most important Characteristics of your Patient? I
Independent Variable
(Intervention or Assessment or Exposure)
What is the main Intervention, Prognostic factor, Diagnostic factor, or Exposure you are considering for your patient? C
Comparison or Control
What is the main alternative to the Intervention / Assessment / Exposure? O
Outcome of Interest
What is considered “success”

39. RESEARCH EVIDENCE: The Best Treatments
QUESTIONS TO ASK
The PEDro Scale
Accessed January 23, 2015

40. RESEARCH EVIDENCE: The Best Treatments
Bias
Comparison
Generalizability

41. RESEARCH EVIDENCE: The Best Treatments
Bias
Comparison
Generalizability
QUESTIONS TO ASK
Who was the Treatment performed on?
Did all of the patients present similarly?

42. RESEARCH EVIDENCE: The Best Treatments
Bias
Comparison
Generalizability

43. RESEARCH EVIDENCE: The Best Treatments
Bias
Comparison
Generalizability
QUESTIONS TO ASK
Did they compare the Groups?
Were they meaningfully different?
(In other words, were the patients who received the treatment Really better than the controls?)

44. RESEARCH EVIDENCE: The Best Treatments
Bias
Comparison
Generalizability

45. RESEARCH EVIDENCE: The Best Treatments
Bias
Comparison
Generalizability
QUESTIONS TO ASK
Did the individuals involved already have an “idea” of what should happen?
In other words: No Blinding of Patients
- Therapists
- Investigators

46. RESEARCH EVIDENCE: The Best Treatments
Bias
Comparison
Generalizability

47. RESEARCH EVIDENCE: The Best Treatments
QUESTIONS TO ASK
Who was the Treatment performed on?
Who Did all the patients present similarly?
Did they compare the groups?
Where they meaningfully different?
Did the individuals involved have an “idea” of what should happen
Bias
Comparison
Generalizability
Improve your ability to
Critically Analyze Intervention research
IMPROVE YOUR OWN PRACTICE

48. We focused on Foundational concepts of EBP
RESEARCH EVIDENCE: Summary
We focused on Foundational concepts of EBP
Specifically, we covered:
Sackett’s 3-pronged EBP model
Levels of Evidence
Critical appraisal - Diagnosis (Diagnostic accuracy & CPRs)
Critical appraisal - Intervention (PICO, Therapeutic studies)

49. Evidence-Based Practice
Best Available
Evidence
Clinical
Experience
Patient
Values
“Integration of the best available research evidence with clinical experience
and patient values to make clinical decisions” – Sackett et al, BMJ 1996.

50. Read not to contradict and confute, nor to believe and take for granted…but to weigh and consider Francis Bacon ( )
The goals of this lecture are to first, Review the hierarchy of clinical research evidence. We will follow that with an Introduction to Critical Appraisal. Lastly, we will cover WHAT to look for and judge when critically appraising the Best Available Evidence.

51. “Commitment to Excellence”
“Commitment to Excellence”