1. The Impact on Clinical, Research, and Disability Findings
Performance Validity Test Failure Rates in Military and Veteran Populations:
The Impact on Clinical, Research, and Disability Findings
John H. Denning, Ph.D.
Ralph H. Johnson Veterans Affairs Medical Center, Charleston, SC
Abstract
Due to the high rates of traumatic brain injury in military samples, there has been an increasing need for neuropsychological assessment of Active Duty soldiers and Veterans. However, high rates of performance validity test (PVT) failure in these populations has been suspected which confounds accurate assessment. The current study reviewed freestanding PVT failure rates in Military/Veteran samples since approximately 2008 to determine the extent of the problem.
There were 43 studies encompassing approximately 8,000 individuals where freestanding PVT failure rates could be calculated. The most common PVTs administered were the Test of Memory Malingering (22), Word Memory Test (17), Medical Symptom Validity Test (11), Victoria Symptom Validity Test (3), and Nonverbal-Medical Symptom Validity Test (1). Despite the significant variability in sample composition, context (research, clinical, disability), and PVT administered, an overall failure rate of 29% was found in the published literature. Average PVT failure rates ranged from 15%-41%. The lowest failure rates were often in research-only samples (8%-38%) with disability samples typically showing the highest failure rates (over 40%). Results reiterate the importance of not only accurately identifying invalid testing in these samples, but also suggest the need to explore possible interventions to reduce non-credible testing behaviors.
Methods
Non-exhaustive review of select studies:
Identified PVT failure rates from published research
Supplemental data provided by study authors
Calculated overall PVT failure rate across all studies
Weighted mean used to account for sample sizes
Results
43 studies were included (n = 7,959).
Number of samples with PVTs:
TOMM1 = 12, TOMM = 10, WMT = 17, MSVT = 11
VSVT = 3, NV-MSVT = 1
29% overall failure rate averaged across all samples
Individual PVTs averaged 15%-41% failure rate.
Research samples had lowest failure rates (8%-38%)
Disability samples had highest failure rates (> 40%).
WMT had highest failure rate (39%).
TOMM1 & MSVT were similar (26% & 28% failure)
Over half the samples included those with mTBI
Discussion
Non-exhaustive review of select studies:
High PVT failure rates found (average = 29%).
Clinical diagnoses/treatments may be inappropriate
if based on invalid cognitive test data.
Disability determinations may result in over-
diagnosis/payment if testing is not screened.
Unreliable research data may lead to inaccurate
conclusions about cognitive impairment.
Clinical trials/interventions may not show accurate
outcomes due to non-credible cognitive data.
Novel interventions to reduce invalid cognitive test
performance should be explored in clinical,
research, and disability settings.
First Author N
Study Sample
Test
Failure Rate
Armistead-Jehle (2010) 45
Veterans-Clinical (mTBI)
MSVT
58%
Armistead-Jehle (2011) 85
Active Duty-Clinical Evaluation
20%
NV-MSVT
15%
TOMM
11%
Armistead-Jehle (2012)
335
Active Duty-Clinical/Disability Evaluation
WMT
42%
Axelrod (2010)
286
Veterans-Mixed Clinical (includes dementia)
MSVT no GMIP
24%
Axelrod (2014)
215
Veterans-Mixed Clinical (no suspected dementia)
17%
202 6%
Benitez (2011)
308
Veterans-Clinical (Non-demented elderly)
12%
Bolzenius (2014) 16
Veterans-Research (blast mTBI)
13%
Campbell (2009) 86
Veterans-Research (PTSD and/or mild-moderate TBI)
19%
Cifu (2013) 61
Active Duty-Research (mTBI with PCS)
TOMM1 < 40
25%
Clark (2014)
187
Veterans-Research (40% TBI, 60% PTSD) 9%
Denning (2012)
497
Veterans-Mixed Clinical (no dementia)
33% 75
Veterans-C&P Evaluation (mTBI)
43%
Denning (2013)
151
34%
32%
Dretsch (2014) 71
Active Duty-Deployed (blast and non-blast mTBI) 3%
Franke (2014)
196
Active Duty/Veterans-Research (blast mTBI)
Gfeller (2013) 74
Veterans-Research (67% mTBI)
18%
Gordon (2011) 95
Veterans-Mixed Clinical (mTBI with/without PTSD)
14%
Hetherington (2014) 21
38% 20
35%
Hilsabeck (2011)
229
Veterans-Mixed Clinical (5% dementia)
27%
Ioachimescu (2014)
Veterans-Research (mTBI)
Jones (2012)
350
Active Duty Military (50% mTBI)
36%
313
VSVT
55%
122
45%
Kulas (2014)
126
Veterans-Mixed Clinical (60% TBI, 90% mTBI)
Lange (2013)
129
Active Duty Military (mild TBI)
125
Active Duty Military (All severity TBI)
Lippa (2014)
131
McCormick (2013)
214
Veterans/Active Duty-Research (64%), Clinical (36%)
Morel (2008) 37
Veterans-(40% C&P Evaluation, 60% mixed clinical)
59%
Neipert (2014)
115
Veterans (mTBI and controls)
Nelson (2010) 44
Veterans-Research (50% mTBI) 8%
Proto (2014)
178
Veterans-Clinical (positive screen for mTBI)
62%
28%
Russo (2012) 38
68%
Schutte (2011)
124
Veterans-Mixed Clinical
Shura (2014)
212
Veterans-Research (40% mTBI)
21%
Spencer (2013)
138
Veterans-Mixed Clinical (mostly mTBI)
Troyanskaya (2015)
139
Veterans and Active Duty (50% blast mTBI)
30%
Verfaellie (2013)
127
Waldron-Perrine (2014)
246
Veterans-Research (80% mTBI)
Whitney (2008) 46
Veterans-Clinical (26% mTBI, suspect effort)
48%
Whitney (2009) 23
Veterans and Active Duty-Clinical
Whitney (2010)
Veterans-Clinical (suspect effort)
Whitney (2013)
194
Veterans-Mixed Clinical (30% mostly mTBI)
23%
Wisdom (2012)
213
Veterans-Clinical (Epilepsy Unit)
16%
Young (2012)
259
Veterans-Clinical (74%), C&P Evaluation (22%)
44%
Weighted Average
7,959
29%
Introduction
Traumatic brain injury has been viewed as the “signature injury” of the recent military conflicts.
Many Active Duty soldiers and Veterans have been screened for mTBI and referred for cognitive testing.
Recent studies have found high rates of invalid cognitive testing in these populations.
Exaggerated deficits on testing confound accurate interpretation of actual cognitive functioning.
Inadequate assessment of test validity may lead to over-diagnosing cognitive impairment.
Performance Validity Tests (PVTs) are measures that help to identify invalid cognitive test performance.
Freestanding PVTs are specifically designed to identify those underperforming on cognitive testing.
The current study reviews (freestanding) PVT failure rates to highlight the extent of suspected invalid cognitive testing in Active Duty and Veteran samples.
* The views expressed in this poster are those of the author and does not reflect the position of the DVA or United States Government. This work was supported by the resources at the Ralph H. Johnson VA Medical Center, Charleston, SC