1. Wechsler Test-Retest Combinationsd
Base Rates of Reliable Cognitive Decline (RCD) on Wechsler Variables in an Elderly Clinical Sample
A-12
Center for Alzheimer’s Care,
Imaging and Research (CACIR)
Gordon Chelune,1 James Holdnack,2 John Sheehan,1 Kevin Duff,1 Otto Pedraza,3 John Crawford4
1University of Utah School of Medicine, 2Pearson, 3Mayo Clinic, 4University of Aberdeen
Department of Neurology
BACKGROUND
Serial testing is increasingly used to evaluate cognitive change.
The determination of reliable cognitive decline (RCD) plays an important role in the evaluation of elderly patients to establish a diagnosis of dementia.
Interpretation of simple test-retest difference scores can be problematic due to measurement error, regression to the mean and expected practice effects. Various reliable change methods have been developed to take these issues into account.
Further, issues about comparability arise when different editions of the same test are used (e.g., the 3rd and 4th editions of the Wechsler Adult Intelligence [WAIS] and Memory Scales [WMS]).
The predicted-difference method of reliable change was developed to assess whether a person’s observed retest score deviates from their regression-based predicted retest score in a clinically meaningful and statistically reliable manner.
Multivariate regression-based equations have been developed for the WAIS/WMS-3, and they have also been developed and incorporated into the Advanced Clinical Solutions software for the WAIS/WMS-4.
More recently, Holdnack et al (in press) have expanded these and also generated equations for predicting WAIS/WMS-4 retest scores for people previously given the WAIS/WMS-3 at baseline.
To examine the validity and comparability of these equations for evaluating cognitive change, we applied the respective regression equations to patients seen serially in a cognitive disorders clinic. Because these patients are at high risk for cognitive decline, we hypothesized that: a) they would show a significantly higher than chance incidence of RCD on the Wechsler measures compared to the standardization sample, and b) the incidence of decline would be comparable across different versions of the Wechsler tests.
Table 1. Demographic characteristics of the three Wechsler test-retest cohorts.
Table 2. Incidence of Reliable Cognitive Decline (RCD) on the Wechsler variables by test-retest cohort.
Wechsler
3 to 3
3 to 4
4 to 4
Total
Sample
Sample Size N
115 68 54
237
Age M
75.9
73.9
75.0
75.1
(SD)
5.7
5.6
5.9
5.8
Education
14.9
14.8
15.3
2.7
2.6
Geriatric Depress.Scale
7.2
6.5
6.8
6.9
4.8
5.5
5.3
Oral Reading (WTAR/TOPF)
104.7
104.8
104.0
104.6
12.6
11.1
10.9
11.7
Mini Mental State (MMSE)
26.0
26.4
25.5
2.5
Sex (Male) %
48.7
54.4
50.0
50.6
Race (Caucasian)
93.9
95.6
98.1
95.4
Handedness (Right)
93.0
92.6
93.7
Variable
Wechsler Test-Retest Combinations
Total
3 to 3
3 to 4
4 to 4
p <
GAI %
32.2
32.7
11.4
53.1
.001 N
180
113 35 32
VCI
30.2
37.7
11.8
40.6
.024
179 34
POI/PRI
31.1
27.2
20.5
47.2
.010
206
114 39 53
PSI
22.6
23.9
26.3
15.4 ns
186
109 38
LM-Immed.
47.7
43.1
55.1
51.4
195 49 37
LM-Delayed
64.1
54.1
89.8
59.3
.0001
Comparison Measures
MMSE
53.3
50.4
54.4
53.7
237
115 68 54
Mattis DRS
64.6
52.9
71.2
76.9
.020
161 70 52
RESULTS
CONCLUSIONS
Examination of reliable change using the predicted-difference method was much more sensitive in detecting reliable cognitive decline than consideration of the simple test-retest difference scores, underscoring the importance of taking into account sources of statistical error and expected practice effects in serial assessments.
As expected, the incidence RCD on the WAIS and WMS was quite high among older adults seen in a cognitive disorders clinic. Decrements in verbal memory were particularly common and comparable to more global measures of cognitive dysfunction.
Predicted-difference equations using scores from the 3rd edition of the Wechsler tests to predict performance on the Wechsler 4th edition measures proved to be clinically sensitive to cognitive decline among high risk patients. While there were some differences in sensitivity on specific measures between retest combinations, the overall findings support the clinical use of different versions of a test to detect cognitive change over time when appropriate methods are used.
Test-Retest Change
Simple-Difference Method: While observed retest scores were lower for all variables, repeated measures ANOVAs failed to reveal a significant Test-Retest effect for any variable. There was a significant Version effect for LM Delayed; patients in the W44 group obtained significantly lower scores suggesting that the WMS-4 task may be more difficult than in the WMS-3. This was also reflected in a significant interaction effect.
Predicted-Difference Method: Repeated measures ANOVAs revealed significant lower Observed versus Predicted retest scores on all measures except VCI, indicating the increased sensitivity of using this reliable change method. There were no differences between groups effects.
Comparison of Base rates of RCD (z-scores < )
Table 2 presents the base rates of RCD for each variable for each Wechsler combination. Sample sizes varied as a function of missing test-retest data due to test selection difference between clinicians. Table 2 also presents the results of chi-square analyses, which revealed a number of differences in incident RCD between different Wechsler test-retest combinations. Patients in the W34 group tended to have lower rates of RCD on the WAIS variables except PSI, whereas they showed much higher base rates of decline on LM Delayed.
Patients: The sample consisted of 237 patients seen for repeat testing in a cognitive disorders clinic with baseline MMSE scores > 21. They were divided into 3 cohorts based on the test-retest combination of Wechsler editions administered: patients given some aspect of the Wechsler 3rd edition twice (W33, n=155) or 4th edition (W44, n=54) twice, and those given edition 3 at baseline and edition 4 at retest (W34, n=68). The demographics of the 3 cohorts were comparable, and their descriptive data are presented in Table 1.
Variables of Interest: The Wechsler variables of interest were:
WAIS: General Ability Index (GAI)
Verbal Comprehension Index (VCI)
Perceptual Organization/Perceptual Reasoning Index
(POI/PRI)
Processing Speed Index (PSI)
WMS: Logical Memory Immediate Scaled Score (LM-I)
Logical Memory Delayed Scaled Score (LM-D)
Procedures: Reliable change regression equations were generated for the variables of interest for each Wechsler test-retest combination using the standardization data for the Wechsler 3rd and 4th editions (Holdnack et al, in press; Lineweaver & Chelune, 2003; Chelune, 2003). These equations were then applied to the respective cohorts using the predicted difference method to obtain z-scores of reliable change, with RCD defined as a predicted difference z-score of <
METHODS
REFERENCES
ACKNOWLEDGEMENTS
Supported by the University of Utah Center for Alzheimer’s Care, Imaging and Research (Salt Lake City, Utah)
Wechsler data, Copyright © 1997, 2008, 2009 NCS Pearson, Inc. All rights reserved. Used with Permission.
Chelune GJ. (2003). Assessing reliable neuropsychological change. In R Franklin (Ed.),  Prediction in forensic and neuropsychology: Sound statistical practices (pp ). Mahwah, NJ: Lawrence Erlbaum.
Holdnack, J.A, Zhou, X., Drozdick, L.W. & Chelune, G. (in press).  Serial Assessment. In J. Holdnack, L. Drozdick, L.Weiss & G. Iverson. WAIS-IV, WMS-IV, & ACS: Advanced Clinical Use and Interpretation. San Diego, CA: Elsevier Science
Lineweaver TT,Chelune GJ. (2003). Use of the WAIS-III and WMS-III in the context of serial assessments: Interpreting reliable and meaningful change. In DS Tulsky, DH Saklofske, GJ Chelune, RK Heaton, RJ Ivnik, RA Bornstein, A Prifitera, MF Ledbetter (Eds.),  Clinical Interpretation of the WAISIII and WMS-III (pp ). New York: Academic Press.