1. Part 4 appreciate appraise apply
Welcome back. In part 4 of Evidence Based Practice, I’ll introduce a few more selected parameters related to the diagnostic accuracy of tests. In this presentation, I’ll introduce positive and negative likelihood ratios, Confidence Intervals, and the use of a Nomogram.
Part 4
Brenda Boucher PT, PhD, CHT, OCS, FAAOMPT

2. Positive & Negative Predictive Values “In the sample population tested, what is the probability of a positive or negative test result in giving me the correct answer?”
At the end of the last presentation, we looked at positive and negative predictive values. As mentioned then, positive and negative predictive values help answer the question: “In the sample population tested, what is the probability of a positive or negative test result in giving me the correct answer?”

3. Positive & Negative Predictive Values Predictive values are limited in use because their calculations are based on the prevalence of the condition in the sample population that was tested
Predictive values are designed to give helpful information, but are limited in use because their calculations are based on the prevalence of the condition in the sample population that was tested. So depending on the prevalence of the condition, values can potentially be inflated or deflated.
Test Result

4. Likelihood Ratios Positive LR = sensitivity / (1 – specificity)+
True Positive +
False Positive -
False Negative −
True Negative
Positive LR = sensitivity / (1 – specificity)
Negative LR = (1 – sensitivity) / specificity
Unlike the limitations associated with Predictive Values, Likelihood Ratios are free of such limitations and are considered the best statistic to summarize the usefulness of a diagnostic test. This is because Likelihood Ratios combine information from all cells of the contingency table to quantify a shift in the probability of a patient having a condition once the results of a diagnostic test are known.

5. Likelihood Ratios
“Based on the result I get when I test my patient, what is the likelihood that my patient has the condition?”
Likelihood ratios help answer one of the most important clinical questions: “Based on the result I get when I test my patient, what is the probability that my patient has the condition?” None of the parameters I’ve talked about so far are designed to answer that particular question.

6. Likelihood Ratios
Likelihood ratios are calculated using combined information contained in sensitivity and specificity values into a ratio used to quantify shifts in probability once the test results are known
Likelihood ratios can answer that question, because they are able to overcome the limitations other parameters have, because they are calculated using combined information contained in sensitivity and specificity values into a ratio that can be used to quantify shifts in probability once the test results are known.

7. Positive & Negative Likelihood Ratios
Positive Likelihood Ratio: indicates the increase
In odds favoring the condition given a positive
test result.
Negative Likelihood Ratio: indicates the change
In odds favoring the condition given a negative
Ljjljl
Positive likelihood ratios are used when the test result is positive, and negative likelihood ratios are used when the test result is negative.
In other words,
A Positive Likelihood Ratio indicates the increase in odds favoring the condition given a positive test result.
A Negative Likelihood Ratio indicates the change in odds favoring the condition given a negative test result.

8. Positive & Negative Likelihood Ratios
Ljjljl
Positive likelihood ratios have values of 1 or greater: the higher the value, the greater the certainty that a positive test result indicates the presence of the condition.
Negative likelihood ratios have values between 1 and 0: the lower the value, the greater the certainty that a negative test result indicates the absence of the condition.
Positive likelihood ratios have values of 1 or greater: the higher the value, the greater the certainty that a positive test result indicates the presence of the condition.
Negative likelihood ratios have values between 1 and 0: the lower the value, the greater the certainty that a negative test result indicates the absence of the condition.

9. Likelihood Ratio Values
Interpretation of Likelihood Ratio (LR) Values
Positive Negative
LR LR Interpretation
__________________________________________________________________________
>10 < Generate large and often conclusive shifts in probability
Generate moderate shifts in probability
2 – – 0.5 Generate small, but sometimes important, shifts in probability
1 – – 1 Alter probability to a small, and rarely important, degree
Said a different way, a positive likelihood ratio indicates the shift in odds in favor of a condition when a test result is positive. Therefore, large positive likelihood values are desirable. Negative likelihood ratios indicate a shift in odds in favor of the absence of the condition. This chart represents the relative size of likelihood ratios and their impact on shifting probabilities.
Taken from Fritz & Wainner (adapted from Jaeschke et al)

10. Likelihood Ratio Values
Interpretation of Likelihood Ratio (LR) Values
Positive Negative
LR LR Interpretation
__________________________________________________________________________
>10 < Generate large and often conclusive shifts in probability
Generate moderate shifts in probability
2 – – 0.5 Generate small, but sometimes important, shifts in probability
1 – – 1 Alter probability to a small, and rarely important, degree
Positive likelihood ratios that range in size from 5 to greater than 10 will generate moderate to large shifts in probability of the patient having the condition. Tests with large Positive likelihood ratios generally have high Specificity, because both values reflect the usefulness of a positive test result.
Negative likelihood ratios that range in size from 0.2 to less than 0.1 generate moderate to large shifts in probability of the absence of the condition.
Small negative likelihood values are useful to help rule out a condition. Small negative LR correspond to high Sensitivity, as both of these measures attest to the usefulness of a negative test result.
Taken from Fritz & Wainner (adapted from Jaeschke et al)

11. Likelihood Ratio (LR) Drop Arm Test Hawkins Test
Sensitivity = 8% Sensitivity = 92%4
Specificity = 97% Specificity = 25%4
+ LR = LR = 1.225
- LR = LR = 0.325
Let’s look at the Hawkins Test and the Drop Arm test again for their use to diagnose Subacromial Impingement. Based on the Sensitivity of each test, it’s easy to see that the Hawkins test is useful during the screening process to help rule-out impingement. Conversely, if subacromial impingement happens to be the clinician’s leading hypothesis, then the Drop Arm test may provide more useful information to help rule-in the condition. The likelihood values support this, but perhaps not as high as one might expect. It’s probably no surprise that clinicians perform a cluster of tests to rule-in subacromial impingement before they reach a threshold for diagnosis. Let’s look at a couple of tests with either high sensitivity or high specificity values that also have corresponding strong likelihood ratios.

12. Likelihood Ratio (LR) Ottawa Ankle Rules
The Ottawa Ankle Rules are used to define a need for radiographic assessment of the ankle to assess for fractures following an injury. The Ottawa Ankle Rule exhibits high Sensitivity with a corresponding low negative likelihood ratio value. This makes it an excellent test to screen out an ankle fracture.
Sensitivity =98%8 - LR = 0.078
Specificity = 32%8 + LR = 1.48

13. Joint Line Tenderness for Torn Tibial Meniscus
Likelihood Ratio (LR)
Joint Line Tenderness for Torn Tibial Meniscus
Karachalios et al.7
(Medial) Sensitivity = 71% - LR = .33
(Lateral) Sensitivity = 78% - LR = .24
(Medial) Specificity = 87% + LR = 5.5
(Lateral) Specificity = 90% + LR = 7.8
Abdon et al.6
(Medial) Sensitivity = 78% - LR = .41
(Lateral) Sensitivity = 78% - LR = .24
(Medial) Specificity = 54% + LR = 1.7
(Lateral) Specificity = 92% + LR = 9.8
The Joint Line Tenderness Test for a torn meniscus has interesting values reported by a couple of groups of colleagues. The highest parameters are associated with the Specificity of the tests, which have corresponding strong positive LR values. Interestingly, both groups of researchers found the test to be most useful for the lateral meniscus, and the best utility of the test appears to assist the clinician to rule-in a torn lateral meniscus if the test result is positive.

14. Confidence Interval
This is probably a good time to insert a reminder that all statistical parameters represent findings from a sample population. In this way, they represent an estimate of the true value that could be found if the entire population was tested. Different sample populations may yield different test results. So how do we put confidence in a statistical parameter? The answer to that question is the Confidence Interval.

15. Confidence Interval (CI)
The confidence interval functions to attest to the precision of an estimate finding. A 95% confidence interval is most commonly used and indicates a range of values within which the population value would lie with 95% certainty, if, indeed, the entire population was studied. The more narrow the confidence interval, the greater certainty we have that the statistic of interest represents Truth. Conversely, if the CI is wide, the usefulness of the measure may be questionable. In other words, a wide confidence interval suggests the point estimate is not very precise in its ability to identify the true test result.
The more narrow the confidence interval, the greater certainty we have that the statistic of interest represents Truth.
A wide confidence interval suggests the point estimate is not very precise in its ability to identify the true test result.

16. Confidence Interval (CI)+
+LR = (0.35, 21.7)4
(0.35, 21.7)4
Positive Negative
LR LR Interpretation
__________________________________________________________________________
>10 < Generate large and often conclusive shifts in probability
Generate moderate shifts in probability
2 – – 0.5 Generate small, but sometimes important, shifts in probability
1 – – 1 Alter probability to a small, and rarely important, degree
For example, in the Calis study, the drop arm test had a decent size positive liklihood ratio of This was, in fact, one of the largest positive likelihood ratios of all the tests reported in this study to assess the presence of subacromial impingement. However, the 95% confidence interval was very wide (0.35 to 21.7). This wide CI indicates the true positive likelihood ratio could be anywhere from 0.35, which is below the threshold for altering the probability to an important extent, to 21.7, which indicates a very large shift in the probability of the condition being present. In short, this wide confidence interval is not very precise and should prompt questioning of the usefulness of this particular statistical finding.

17. Applying the Evidence diagnosis: an exercise in probability revision
Remember when I made a statement in an earlier presentation about the diagnostic process being one of probability revision? Let’s look at that again with knowledge we know have about the statistical parameters we’ve discussed.
diagnosis: an exercise in probability revision

18. highly unlikely . . . . . . . highly likely
Pre-test Probability:
Mechanism of injury
Patient’s age
Epidemiological prevalence of the condition
Clinical experience of the therapist
Other factors
Hypothesis
Prior to performing an examination, a clinician has an idea of the likelihood that the patient has a particular condition. This condition forms the initial hypothesis. Most likely, the clinician has formed an idea of how probable it is that the patient has the condition based on a number of factors. We call this the pre-test probability. The pre-test probability is derived from various factors, including the mechanism of injury, the patient’s age, the epidemiological prevalence of the condition, the clinical experience of the therapist, as well as other factors.
highly unlikely highly likely

19. Clinical Reasoning Sensitivity Specificity Likelihood Ratios
Using clinical reasoning to formulate an initial hypothesis is a very important step in the examination. It is the first step to ensure an effective and efficient examination, because it whittles away less likely, competing diagnoses and allows the clinician to focus on the more likely diagnosis. Selecting the best tests and measures based on Sensitivity, Specificity and Likelihood ratios is the second step in an effective and efficient examination. Tests that most closely match findings to the findings of a reference or gold standard are best, which is why we look to tests with high Sensitivity, Specificity, and strong Likelihood Ratiios.
Sensitivity Specificity Likelihood Ratios

20. Probability Revision Likelihood Ratio
Likelihood ratios quantify the direction and magnitude of change in the pretest probability based on the test result
Pre-test
probability
The information gained by the results of selected tests or measures will alter the pre-test probability, resulting in a revised post-test probability that the hypothesis is correct. The degree to which the post-test probability changes from the pre-test probability can be quantified when likelihood ratios are known. Likelihood ratios quantify the direction and magnitude of change in the pretest probability based on the test result.
Post-test
probability
Likelihood
Ratio

21. Nomogram Pre-test probability Likelihood Ratio Post-test probability
A Nomogram can be used to help identify the post-test probability. A clinician uses information from the pre-test probability and the likelihood ratio to identify the post-test probability. Let’s use an example to illustrate.

22. Pretest -> Posttest Probability
Let’s say we have a 22 y/o patient with pain and symptoms of instability of the left knee following a cut and turn injury sustained while playing soccer. A clinician probably considers whether the patient’s symptoms are related to a cruciate ligament injury. What is a reasonable pretest probability of either a torn ACL or PCL for this patient? Based on the age, symptoms, epidemiological data and the clinical experience of the therapist, the probability is likely to be high. We’ll quantify it at 50%. Our next question is “what tests can be performed to rule- in or rule-out these two conditions?
“What tests should be performed to rule-in or rule-out these conditions?”

23. Pretest -> Posttest Probability
Nomogram
Pretest -> Posttest Probability
ACL > PCL
Rule-out PCL first
Test with high sensitivity and a corresponding low negative likelihood ratio is most helpful to rule-out the condition
Let’s say the clinician thinks the probability is greater that the patient has an ACL injury rather than a PCL injury. He or she may want to rule out the PCL first. In this case, a test with high sensitivity and a corresponding low negative likelihood ratio is most helpful to help rule-out the condition.
The clinician selects the posterior drawer test, seeing that it has a negative likelihood ratio of .10, which with a negative test result, should generate a moderate shift in the post-test probability that the patient is free from having a significant PCL injury.
But how much is that posttest probability likely to be shifted with this particular test? A nomogram can be used to help quantify the shift.
Posterior Drawer Test:
Sensitivity = .909
-LR = .109

24. Posttest Probability Nomogram Posterior Drawer Test:
If a line is anchored at the pretest value of 50% probability and connects at the point that represents .1% on the Likelihood ratio line, then the resulting post-test probability can be identified. In this case, the post-test probability of the patient having a significant PCL injury is just less than 10%. This information can help the clinician consider ruling-out a significant PCL injury and proceed to the next test, which is to see if an ACL injury can be ruled-in.
Posterior Drawer Test:
Sensitivity = .909
-LR = .109

25. Pretest -> Posttest Probability
Nomogram
Pretest -> Posttest Probability
Let’s say the clinician selects the pivot shift test help rule-in an ACL injury. The pivot shift test has a positive likelihood ratio of 41. If the test is positive, this should generate a large shift in the post-test probability. Let’s see.
If a line is anchored at the pretest value of 50% probability and connects at the point that represents 41% on the Likelihood ratio line, then the resulting post-test probability is >95%. This finding is note-worthy and can help the clinician rule-in a significant ACL injury. The clinician can then proceed to the next step in the clinical decision-making process on how to best manage the patient.
Pivot Shift Test:
Specificity =
+LR = 4110

26. Hopefully, the puzzle pieces are starting to fit together
Hopefully, the puzzle pieces are starting to fit together. The shift in our profession towards evidence based practice has generated new and useful information about the accuracy of tests and measures we use to make clinical decisions regarding diagnoses, prognoses and treatment selections. Knowing the utility of the tests and measures allows clinicians to select those with known diagnostic values capable of facilitating an efficient and effective examination. And that’s what it’s all about, right? Making practice effective, efficient, and successful.

27. References
Haynes RB, Sackett RB, Gray JMA, Cook DC, Guyatt GH. Transferring evidence from research into
practice, 1: the role of clinical care research evidence in clinical decisions. ACP Journal Club. Nov-Dec 1996;125:A
Guyatt GH, Rennie D. User’s Guide to the Medical Literature. AMA press. 2002, Chicago, IL
Simoneau, GG, Allison SC. Physical therapists as evidence-based diagnosticians.
JOSPT. 2010;40(10):
Calis M, Akgun K, Birtane M, et al. Diagnostic values of clinical diagnostic tests in subacromial
impingement syndrome. Ann Rheum Dis. 2000;59:44-47.
Fritz JM, Wainner RW. Examining diagnostic tests: an evidence-based perspective. Physical
Therapy. 2001;81(9):
Abdon P, Lindstrand A, Thorngren KG. Statistical evaluation of the diagnostic criteria for meniscal
tears. Int Orthop. 1990;14:
Karachalios T, Hantes M, Zibis AH, Zachos V, Karantanas AH, Malizos KN.
Diagnositc accuracy of a new clinical test for early detection of meniscal tears. J Bone Joint Surg Am. 2005;87:
Bachmann LM, Kolb E, Koller MT, Steurer J, ter Riet G. Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review. BMJ. 2003;326(7386):417
9. Rubinstein RA, Jr., Shelbourne KD, McCarroll JR, et al. The accuracy of the clinical examination in the setting of posterior cruciate ligament injuries. Am J Sports Med 1994; 22(4):
10. Katz J, Fingeroth R. The diagnostic accuracy of ruptures of the anterior cruciate ligament comparing the Lachman test, the anterior drawer sign, and the pivot shift test in acute and chronic knee injuries. Am J Sports Med 1986; 14:88-91.
Stay tuned for the next presentation, which will focus on hints to help with critical appraisal of the literature.
My primary references are provided on this slide for your convenience. Until next time, I wish you well.