Intelligence testing: A matter of life and death? Chad W. Buckendahl Buros Center for Testing University of Nebraska, Lincoln

Buros Center for Testing Oscar K. Buros, 1935; UNL, 1979 Improve the science and practice of testing Buros Institute of Mental Measurements (BIMM): commercially available tests –Mental Measurements Yearbook (MMY) –Test in Print (TIP) Buros Institute for Assessment Consultation and Outreach (BIACO): proprietary tests –Psychometric services for educational licensure, certification, admissions, and employment testing

Case Background Defendant (Erick Vela) participated in a bank robbery in Norfolk, NE in September, 2002 Five people were killed during the robbery One of four defendants convicted Motion asserting mental retardation

Legal Background Atkins v. Virginia (2002) –Prohibited execution of persons with mental retardation Additional Nebraska statute –“An intelligence quotient of 70 or below on a reliably administered intelligence quotient test shall be presumptive evidence of mental retardation.”

Diagnosing Mental Retardation Relies on multiple criteria American Psychiatric Association and American Association of Mental Retardation: –Sub-average intellectual functioning, –Significant limitations in adaptive skills such as communications, self-care, and self-direction, and –Manifest before age of 18

Defining primary criteria Sub-average intelligence: –IQ 70 or below (2 std. dev. below mean) –Considers measurement error Adaptive functioning: –Coping with common life demands –Standards of personal independence –Socio-cultural background –Community setting

Intelligence tests Tests administered by three different defense experts: –Wechsler Abbreviated Scale of Intelligence (WASI): July 2003 –Wechsler Adult Intelligence Scales (3 rd ed.) (WAIS-III): November 2003 –Stanford-Binet (5 th ed.) (SB5): July 2004

Adaptive Behavior Tests July 2003: none November/December 2003: none : Vineland Adaptive Behavior Scales –Defense: defendant’s sister –State: two friends of the defendant

Malingering tests Administered when there are concerns about motivation/effort in performance –July, 2003: none –November, 2003: Test of Memory Malingering (TOMM) Victoria Symptom Validity Test (VSVT) Validity Indicator Profile (VIP) 21-item test –July, 2004: none

Professional Standards Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999) –Validity –Reliability –Administration –Scoring –Score Interpretation

Validity Argument “Ultimately, the validity of an intended interpretation of test scores relies on all the available evidence relevant to the technical quality of a testing system. This includes...appropriate test administration and scoring...” (Standards, p. 17)

Administration and Scoring “The usefulness and interpretability of test scores require that a test be administered and scored according to the developer’s instructions... Without such standardization, the accuracy and comparability of score interpretations would be reduced.” (Standards, p. 61)

Scores WASI: 87 (Full Scale IQ) –82 (Verbal IQ), 94 (Performance IQ) WAIS-III: 75 (FSIQ) –75 (VIQ), 78 (PIQ) SB5: 66 (FSIQ) –56 (VIQ), 79 (Nonverbal IQ)

Interpreting scale scores Full scale score relative to average (mean) score for population (i.e. 100) Full scale score relative to the statutory cut score (i.e. 70) Expected range of full scale and sub- scale scores given defendant’s observed scores

WASI caution “[The WASI] should not be used alone to make diagnosis or educational placement decisions. It is not meant to replace more comprehensive measures of intelligence, such as the WISC-III and the WAIS-III." (Technical Manual, p.8)

“...a low score on the WAIS-III does not necessarily reflect a low level of intellectual functioning... other factors such as the following may be implicated: cultural and linguistic discrepancy from the test’s standardization sample, disabling distractibility or anxiety, severe psychopathology, deafness, poor motivation or inadequate persistence, and extremely oppositional behavior or very poor rapport with the examiner. Before diagnosing low intellectual functioning or mental retardation, the examiner must rule out these factors.” (emphasis added) Administration and Scoring Manual (pp.7-8) Interpreting WAIS-III Scores

SB5 VIQ/NVIQ Differences The defendant had a… Full scale IQ (FSIQ) score of 66 Verbal IQ (VIQ) score of 56 Nonverbal IQ (NVIQ) score of point difference between VIQ and NVIQ Only 1.9% of the standardization sample had a difference of 23 or more between VIQ and NVIQ

“… when significant differences exist between the NVIQ and the VIQ, examiners should be cautious about evaluating FSIQ as a summary of an individual’s general ability level...When the context of the assessment and examinee’s background is influenced by such factors as communication disorders, learning disabilities, autism, or non-English background, the NVIQ may be a better indicator of global cognitive potential.” (emphasis added) Examiner’s Manual (p. 134) Interpreting SB5 Scores

Full Scale Scores Over Time

Verbal Scale Scores Over Time

Nonverbal Scale Scores Over Time

Summary of Intelligence Tests

Adaptive Functioning Information Vineland Adaptive Behavior Scales School records –Course grades, remedial programs Observations –Family members, teachers, inmates –Employment record, corrections officials

Defense approach Strict interpretation of the statute (i.e. reliably administered intelligence test) Suggested that accuracy (validity) could be obtained without precision (reliability) Consider the standard error in determining the range of possible true IQ scores Suggested that observed scores across tests were different due to the Flynn effect (i.e. intergenerational influence on norm sample)

Judge’s Ruling Presumption of MR rejected Validity of presumptive score: –3 rd intelligence test administered by defense –Probability of 66 on SB5 was remote –No malingering tests administered on it –Examiner did not follow interpretation directions in test publisher’s manual

Judge’s Ruling (cont.) Sub-average intelligence –Considered standard error –WAIS-III score met the criterion Adaptive functioning –Preponderance of evidence did not support limitations in this area Age of onset –NE’s statute does not specify age Defense motion denied

Future Research Questions Measurement –Application of standard setting methodology for cut score –Validity of score interpretations given intended uses –Instrument development (e.g., content, norm sample) Legal –Implications for future cases/legislation –Court’s application of current validity theory Psychological –Criterion definition of mental retardation (rather than norm) –Interpretation of intelligence as unique from achievement –Stability of intelligence as a construct