Part 4 appreciate appraise apply Welcome back. In part 4 of Evidence Based Practice, I’ll introduce a few more selected parameters related to the diagnostic accuracy of tests. In this presentation, I’ll introduce positive and negative likelihood ratios, Confidence Intervals, and the use of a Nomogram. Part 4 Brenda Boucher PT, PhD, CHT, OCS, FAAOMPT

Positive & Negative Predictive Values “In the sample population tested, what is the probability of a positive or negative test result in giving me the correct answer?” At the end of the last presentation, we looked at positive and negative predictive values. As mentioned then, positive and negative predictive values help answer the question: “In the sample population tested, what is the probability of a positive or negative test result in giving me the correct answer?”

Positive & Negative Predictive Values Predictive values are limited in use because their calculations are based on the prevalence of the condition in the sample population that was tested Predictive values are designed to give helpful information, but are limited in use because their calculations are based on the prevalence of the condition in the sample population that was tested. So depending on the prevalence of the condition, values can potentially be inflated or deflated. Test Result

Likelihood Ratios Positive LR = sensitivity / (1 – specificity) + True Positive + False Positive - False Negative − True Negative Positive LR = sensitivity / (1 – specificity) Negative LR = (1 – sensitivity) / specificity Unlike the limitations associated with Predictive Values, Likelihood Ratios are free of such limitations and are considered the best statistic to summarize the usefulness of a diagnostic test. This is because Likelihood Ratios combine information from all cells of the contingency table to quantify a shift in the probability of a patient having a condition once the results of a diagnostic test are known.

Likelihood Ratios “Based on the result I get when I test my patient, what is the likelihood that my patient has the condition?” Likelihood ratios help answer one of the most important clinical questions: “Based on the result I get when I test my patient, what is the probability that my patient has the condition?” None of the parameters I’ve talked about so far are designed to answer that particular question.

Likelihood Ratios Likelihood ratios are calculated using combined information contained in sensitivity and specificity values into a ratio used to quantify shifts in probability once the test results are known Likelihood ratios can answer that question, because they are able to overcome the limitations other parameters have, because they are calculated using combined information contained in sensitivity and specificity values into a ratio that can be used to quantify shifts in probability once the test results are known.

Positive & Negative Likelihood Ratios Positive Likelihood Ratio: indicates the increase In odds favoring the condition given a positive test result. Negative Likelihood Ratio: indicates the change In odds favoring the condition given a negative Ljjljl Positive likelihood ratios are used when the test result is positive, and negative likelihood ratios are used when the test result is negative. In other words, A Positive Likelihood Ratio indicates the increase in odds favoring the condition given a positive test result. A Negative Likelihood Ratio indicates the change in odds favoring the condition given a negative test result.

Positive & Negative Likelihood Ratios Ljjljl Positive likelihood ratios have values of 1 or greater: the higher the value, the greater the certainty that a positive test result indicates the presence of the condition. Negative likelihood ratios have values between 1 and 0: the lower the value, the greater the certainty that a negative test result indicates the absence of the condition. Positive likelihood ratios have values of 1 or greater: the higher the value, the greater the certainty that a positive test result indicates the presence of the condition. Negative likelihood ratios have values between 1 and 0: the lower the value, the greater the certainty that a negative test result indicates the absence of the condition.

Likelihood Ratio Values Interpretation of Likelihood Ratio (LR) Values Positive Negative LR LR Interpretation __________________________________________________________________________ >10 < 0.1 Generate large and often conclusive shifts in probability 5 - 10 0.1 - 0.2 Generate moderate shifts in probability 2 – 5 0.2 – 0.5 Generate small, but sometimes important, shifts in probability 1 – 2 0.5 – 1 Alter probability to a small, and rarely important, degree Said a different way, a positive likelihood ratio indicates the shift in odds in favor of a condition when a test result is positive. Therefore, large positive likelihood values are desirable. Negative likelihood ratios indicate a shift in odds in favor of the absence of the condition. This chart represents the relative size of likelihood ratios and their impact on shifting probabilities. Taken from Fritz & Wainner (adapted from Jaeschke et al)

Likelihood Ratio Values Interpretation of Likelihood Ratio (LR) Values Positive Negative LR LR Interpretation __________________________________________________________________________ >10 < 0.1 Generate large and often conclusive shifts in probability 5 - 10 0.1 - 0.2 Generate moderate shifts in probability 2 – 5 0.2 – 0.5 Generate small, but sometimes important, shifts in probability 1 – 2 0.5 – 1 Alter probability to a small, and rarely important, degree Positive likelihood ratios that range in size from 5 to greater than 10 will generate moderate to large shifts in probability of the patient having the condition. Tests with large Positive likelihood ratios generally have high Specificity, because both values reflect the usefulness of a positive test result. Negative likelihood ratios that range in size from 0.2 to less than 0.1 generate moderate to large shifts in probability of the absence of the condition. Small negative likelihood values are useful to help rule out a condition. Small negative LR correspond to high Sensitivity, as both of these measures attest to the usefulness of a negative test result. Taken from Fritz & Wainner (adapted from Jaeschke et al)

Likelihood Ratio (LR) Drop Arm Test Hawkins Test Sensitivity = 8%4 Sensitivity = 92%4 Specificity = 97%4 Specificity = 25%4 + LR = 2.665 + LR = 1.225 - LR = .955 - LR = 0.325 Let’s look at the Hawkins Test and the Drop Arm test again for their use to diagnose Subacromial Impingement. Based on the Sensitivity of each test, it’s easy to see that the Hawkins test is useful during the screening process to help rule-out impingement. Conversely, if subacromial impingement happens to be the clinician’s leading hypothesis, then the Drop Arm test may provide more useful information to help rule-in the condition. The likelihood values support this, but perhaps not as high as one might expect. It’s probably no surprise that clinicians perform a cluster of tests to rule-in subacromial impingement before they reach a threshold for diagnosis. Let’s look at a couple of tests with either high sensitivity or high specificity values that also have corresponding strong likelihood ratios.

Likelihood Ratio (LR) Ottawa Ankle Rules The Ottawa Ankle Rules are used to define a need for radiographic assessment of the ankle to assess for fractures following an injury. The Ottawa Ankle Rule exhibits high Sensitivity with a corresponding low negative likelihood ratio value. This makes it an excellent test to screen out an ankle fracture. Sensitivity =98%8 - LR = 0.078 Specificity = 32%8 + LR = 1.48

Joint Line Tenderness for Torn Tibial Meniscus Likelihood Ratio (LR) Joint Line Tenderness for Torn Tibial Meniscus Karachalios et al.7 (Medial) Sensitivity = 71% - LR = .33 (Lateral) Sensitivity = 78% - LR = .24 (Medial) Specificity = 87% + LR = 5.5 (Lateral) Specificity = 90% + LR = 7.8 Abdon et al.6 (Medial) Sensitivity = 78% - LR = .41 (Lateral) Sensitivity = 78% - LR = .24 (Medial) Specificity = 54% + LR = 1.7 (Lateral) Specificity = 92% + LR = 9.8 The Joint Line Tenderness Test for a torn meniscus has interesting values reported by a couple of groups of colleagues. The highest parameters are associated with the Specificity of the tests, which have corresponding strong positive LR values. Interestingly, both groups of researchers found the test to be most useful for the lateral meniscus, and the best utility of the test appears to assist the clinician to rule-in a torn lateral meniscus if the test result is positive.

Confidence Interval This is probably a good time to insert a reminder that all statistical parameters represent findings from a sample population. In this way, they represent an estimate of the true value that could be found if the entire population was tested. Different sample populations may yield different test results. So how do we put confidence in a statistical parameter? The answer to that question is the Confidence Interval.

Confidence Interval (CI) The confidence interval functions to attest to the precision of an estimate finding. A 95% confidence interval is most commonly used and indicates a range of values within which the population value would lie with 95% certainty, if, indeed, the entire population was studied. The more narrow the confidence interval, the greater certainty we have that the statistic of interest represents Truth. Conversely, if the CI is wide, the usefulness of the measure may be questionable. In other words, a wide confidence interval suggests the point estimate is not very precise in its ability to identify the true test result. The more narrow the confidence interval, the greater certainty we have that the statistic of interest represents Truth. A wide confidence interval suggests the point estimate is not very precise in its ability to identify the true test result.

Confidence Interval (CI) + +LR = 2.664 (0.35, 21.7)4 (0.35, 21.7)4 Positive Negative LR LR Interpretation __________________________________________________________________________ >10 < 0.1 Generate large and often conclusive shifts in probability 5 - 10 0.1 - 0.2 Generate moderate shifts in probability 2 – 5 0.2 – 0.5 Generate small, but sometimes important, shifts in probability 1 – 2 0.5 – 1 Alter probability to a small, and rarely important, degree For example, in the Calis study, the drop arm test had a decent size positive liklihood ratio of 2.66. This was, in fact, one of the largest positive likelihood ratios of all the tests reported in this study to assess the presence of subacromial impingement. However, the 95% confidence interval was very wide (0.35 to 21.7). This wide CI indicates the true positive likelihood ratio could be anywhere from 0.35, which is below the threshold for altering the probability to an important extent, to 21.7, which indicates a very large shift in the probability of the condition being present. In short, this wide confidence interval is not very precise and should prompt questioning of the usefulness of this particular statistical finding.

Applying the Evidence diagnosis: an exercise in probability revision Remember when I made a statement in an earlier presentation about the diagnostic process being one of probability revision? Let’s look at that again with knowledge we know have about the statistical parameters we’ve discussed. diagnosis: an exercise in probability revision

highly unlikely . . . . . . . highly likely Pre-test Probability: Mechanism of injury Patient’s age Epidemiological prevalence of the condition Clinical experience of the therapist Other factors Hypothesis Prior to performing an examination, a clinician has an idea of the likelihood that the patient has a particular condition. This condition forms the initial hypothesis. Most likely, the clinician has formed an idea of how probable it is that the patient has the condition based on a number of factors. We call this the pre-test probability. The pre-test probability is derived from various factors, including the mechanism of injury, the patient’s age, the epidemiological prevalence of the condition, the clinical experience of the therapist, as well as other factors. highly unlikely . . . . . . . highly likely

Clinical Reasoning Sensitivity Specificity Likelihood Ratios Using clinical reasoning to formulate an initial hypothesis is a very important step in the examination. It is the first step to ensure an effective and efficient examination, because it whittles away less likely, competing diagnoses and allows the clinician to focus on the more likely diagnosis. Selecting the best tests and measures based on Sensitivity, Specificity and Likelihood ratios is the second step in an effective and efficient examination. Tests that most closely match findings to the findings of a reference or gold standard are best, which is why we look to tests with high Sensitivity, Specificity, and strong Likelihood Ratiios. Sensitivity Specificity Likelihood Ratios

Probability Revision Likelihood Ratio Likelihood ratios quantify the direction and magnitude of change in the pretest probability based on the test result Pre-test probability The information gained by the results of selected tests or measures will alter the pre-test probability, resulting in a revised post-test probability that the hypothesis is correct. The degree to which the post-test probability changes from the pre-test probability can be quantified when likelihood ratios are known. Likelihood ratios quantify the direction and magnitude of change in the pretest probability based on the test result. Post-test probability Likelihood Ratio

Nomogram Pre-test probability Likelihood Ratio Post-test probability A Nomogram can be used to help identify the post-test probability. A clinician uses information from the pre-test probability and the likelihood ratio to identify the post-test probability. Let’s use an example to illustrate.

Pretest -> Posttest Probability Let’s say we have a 22 y/o patient with pain and symptoms of instability of the left knee following a cut and turn injury sustained while playing soccer. A clinician probably considers whether the patient’s symptoms are related to a cruciate ligament injury. What is a reasonable pretest probability of either a torn ACL or PCL for this patient? Based on the age, symptoms, epidemiological data and the clinical experience of the therapist, the probability is likely to be high. We’ll quantify it at 50%. Our next question is “what tests can be performed to rule- in or rule-out these two conditions? “What tests should be performed to rule-in or rule-out these conditions?”

Pretest -> Posttest Probability Nomogram Pretest -> Posttest Probability ACL > PCL Rule-out PCL first Test with high sensitivity and a corresponding low negative likelihood ratio is most helpful to rule-out the condition Let’s say the clinician thinks the probability is greater that the patient has an ACL injury rather than a PCL injury. He or she may want to rule out the PCL first. In this case, a test with high sensitivity and a corresponding low negative likelihood ratio is most helpful to help rule-out the condition. The clinician selects the posterior drawer test, seeing that it has a negative likelihood ratio of .10, which with a negative test result, should generate a moderate shift in the post-test probability that the patient is free from having a significant PCL injury. But how much is that posttest probability likely to be shifted with this particular test? A nomogram can be used to help quantify the shift. Posterior Drawer Test: Sensitivity = .909 -LR = .109

Posttest Probability Nomogram Posterior Drawer Test: If a line is anchored at the pretest value of 50% probability and connects at the point that represents .1% on the Likelihood ratio line, then the resulting post-test probability can be identified. In this case, the post-test probability of the patient having a significant PCL injury is just less than 10%. This information can help the clinician consider ruling-out a significant PCL injury and proceed to the next test, which is to see if an ACL injury can be ruled-in. Posterior Drawer Test: Sensitivity = .909 -LR = .109

Pretest -> Posttest Probability Nomogram Pretest -> Posttest Probability Let’s say the clinician selects the pivot shift test help rule-in an ACL injury. The pivot shift test has a positive likelihood ratio of 41. If the test is positive, this should generate a large shift in the post-test probability. Let’s see. If a line is anchored at the pretest value of 50% probability and connects at the point that represents 41% on the Likelihood ratio line, then the resulting post-test probability is >95%. This finding is note-worthy and can help the clinician rule-in a significant ACL injury. The clinician can then proceed to the next step in the clinical decision-making process on how to best manage the patient. Pivot Shift Test: Specificity = .97-.9810 +LR = 4110

Hopefully, the puzzle pieces are starting to fit together Hopefully, the puzzle pieces are starting to fit together. The shift in our profession towards evidence based practice has generated new and useful information about the accuracy of tests and measures we use to make clinical decisions regarding diagnoses, prognoses and treatment selections. Knowing the utility of the tests and measures allows clinicians to select those with known diagnostic values capable of facilitating an efficient and effective examination. And that’s what it’s all about, right? Making practice effective, efficient, and successful.

References Haynes RB, Sackett RB, Gray JMA, Cook DC, Guyatt GH. Transferring evidence from research into practice, 1: the role of clinical care research evidence in clinical decisions. ACP Journal Club. Nov-Dec 1996;125:A-14-15. 2. Guyatt GH, Rennie D. User’s Guide to the Medical Literature. AMA press. 2002, Chicago, IL Simoneau, GG, Allison SC. Physical therapists as evidence-based diagnosticians. JOSPT. 2010;40(10):603-605 Calis M, Akgun K, Birtane M, et al. Diagnostic values of clinical diagnostic tests in subacromial impingement syndrome. Ann Rheum Dis. 2000;59:44-47. Fritz JM, Wainner RW. Examining diagnostic tests: an evidence-based perspective. Physical Therapy. 2001;81(9):1546-1564. 6. Abdon P, Lindstrand A, Thorngren KG. Statistical evaluation of the diagnostic criteria for meniscal tears. Int Orthop. 1990;14:341-345. 7. Karachalios T, Hantes M, Zibis AH, Zachos V, Karantanas AH, Malizos KN. Diagnositc accuracy of a new clinical test for early detection of meniscal tears. J Bone Joint Surg Am. 2005;87:955-962. Bachmann LM, Kolb E, Koller MT, Steurer J, ter Riet G. Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review. BMJ. 2003;326(7386):417 9. Rubinstein RA, Jr., Shelbourne KD, McCarroll JR, et al. The accuracy of the clinical examination in the setting of posterior cruciate ligament injuries. Am J Sports Med 1994; 22(4):550-557. 10. Katz J, Fingeroth R. The diagnostic accuracy of ruptures of the anterior cruciate ligament comparing the Lachman test, the anterior drawer sign, and the pivot shift test in acute and chronic knee injuries. Am J Sports Med 1986; 14:88-91. Stay tuned for the next presentation, which will focus on hints to help with critical appraisal of the literature. My primary references are provided on this slide for your convenience. Until next time, I wish you well.