Dr. Lisa Woodcock-Burroughs, Ph.D. Assessing English Language Learners Using the Culture-Language Interpretive Matrix Dr. Lisa Woodcock-Burroughs, Ph.D. Materials are adapted from the work of Dr. Samuel Ortiz. For more information please refer to the following: Essentials of Cross Battery Assessment, Third Edition and www.crossbattery.com

How are we currently assessing ell students? Modified and Altered Testing Nonverbal Testing Native Language Testing Consecutive Monolingual Testing

In order to interpret test results we need to establish that they are valid

Modified and altered testing Fundamentally altering the test eliminating or not administering certain test items with presumed culturally biased content mediating culturally based task concepts prior to administration repeating verbal instructions to ensure full comprehension accepting responses in either the native language or the language of the test administering only the subtests that do not rely on oral expression eliminating or modifying time constraints—violate standardization, introducing error and results are suspect and effectively preclude subsequent valid and defensible interpretation Use of interpreter These methods all violate standardization, introduce error, and effectively preclude subsequent valid and defensible interpretation

Modified and altered testing Tests were developed on a norm group different from the student being assessed While race and ethnicity are used to stratify the norm group, culture and cultural differences are not taken into account; neither are difference in experience that affect language or acculturative knowledge development CANNOT use C-LIM if you have altered or modified the test in any way

Modified and Altered Testing If, you want to see how the student would perform under the altered conditions, first administer the assessment in the standardized way and then re-administer it with the modifications This may give you valuable qualitative information about student functioning However, you cannot use C-LIM to interpret the scores obtained with the modified/altered testing

‘Nonverbal Tests’ All tests require communication between examinee and examinee in some fashion Reduce but do not eliminate language barrier Not necessarily any more culture fair as the pictures used tend to remain culturally bound Only give a global IQ score Problematic in evaluations for SLD for 4 reasons

4 Reasons ‘nonverbal tests’ are problematic for evaluation of SLD Nonverbal estimates may be no more fair or valid than those that include verbal abilities Range of abilities measured by nonverbal composite is lower than that measured by verbal Most SLD referrals are for problems in ELA (reading) Need to test cognitive areas related to reading such as Ga and Gc which cannot be easily measured, or at all with nonverbal tests Nonverbal tests continue to have the norm sample representation problem Don’t control for cultural experiences Predictive validity of nonverbal tests is dubious

Language reduced assessments Only one component of XBA for ELL students ELL students should also be evaluated using C- LIM

Native Language Testing Not standardized to be used bilingually (using the 2 languages simultaneously) Normed on monolingual speakers of the language, residing in a country other than the United States, who also speak the language Not representative of the type of students we are assessing Little research exists regarding the performance of bilingual individuals on monolingual tests, administered in their native language What is known is complicated by age, level and type of prior education, current language of instruction and type of instructional program Neuropsychological differences cognitive functioning in bilingual versus monolingual speakers detectable as early as 7 months of age and lasting throughout the lifespan

Native Language Testing In order to use, psychologist must speak the language of the test Consecutive monolingual testing is recommended; Administer the English version first, followed by the version in the student’s primary language WJ III Cognitive followed by Batteria WISC IV followed by WISC IV Spanish May require 2 psychologists, one to administer the English version and another who speaks the students native language, to administer the native language version of the test

Consecutive Monolingual Testing Give English version first, Use CLIM for validity If valid give native language version to follow up, specifically on deficit areas If ‘practice effect’ when given in Spanish that would qualitatively speak to respond to referral concerns i.e. more of a language/culture issue

Culture-Language Interpretive Matrix

English Language Testing There is a lot of research on the performance of ELL students on English tests C-LIM evaluates whether or not score profile follows or does not follow pattern in the literature “Testing in English provides perhaps the most practical as well as the most defensible approach among them all, especially in relation to being evidence based.” (p. 308)

Culture-Language Interpretive Matrix The single purpose of the XBA C-LIM v2.0 is to evaluate the extent to which differences in developmental language proficiency and acculturative learning opportunity may have affected the validity of scores obtained from standardized tests. It is not a diagnostic tool Helps answer the difference verses disorder question which is fundamental to ELL evaluations

C-LIM Use of the XBA C-LIM v2.0 to evaluate the validity of test results is not sufficient to establish fairness and equity in evaluation in the absence of other data and information. The entire assessment must follow a comprehensive framework for nondiscriminatory evaluation.

C-LIM Tests arranged by degree of cultural loading versus language demands Low Moderate High Tests are NOT organized based on intended measurement of specific cognitive abilities It is possible to use the C-LIM to identify an assessment with lower cultural and language requirements, but you must still use C-LIM to assess validity after administration

C-LIM Except in special cases use CLIM to look at combined effect of culture and language and not culture OR language in isolation

C-LIM C-LIM requires that the degree of difference be established for the examinee before evaluation of the validity of test results. Difference may be understood in terms of developmental language proficiency in English coupled with knowledge of the individual’s acculturative and educational learning experiences

Slightly Different Includes individuals with high levels of English language proficiency (e.g., advanced BICS/emerging CALP) and high acculturation, but still not entirely comparable to mainstream U.S. English speakers. Examples include individuals who have resided in the U.S. for more than 7 years or who have parents with at least a high school education, and who demonstrate native-like proficiency in English language conversation and solid literacy skills.  

‘Different’ Includes individuals with moderate levels of English language proficiency (e.g., intermediate to advanced BICS) and moderate levels of acculturation. Examples include individuals who have resided in the U.S. for 3-7 years and who have learned English well enough to communicate, but whose parents are limited English speakers with only some formal schooling, and improving but below grade level literacy skills.

Markedly Different Includes individuals with low to very low levels of English language proficiency (e.g., early BICS) and low or very low levels of acculturation. Examples include individuals who recently arrived in the U.S. or who may have been in the U.S. 3 years or less, with little or no prior formal education, who are just beginning to develop conversational abilities and whose literacy skills are also just emerging.

C-LIM use and interpretation Proper use of the C-LIM begins by looking for the highest score in the upper left-hand corner, the lowest score in the bottom right- hand corner, and other scores falling in between these anchor points.

Invalid Results If pattern follows the expected pattern, results are invalid and reflect primary contribution of culture and language; if don’t follow expected pattern (magnitude of scores is lower than expected range or there is an absence of a systematic decline as linguistic and cultural demands rise), results are valid and culture/language differences are not primary  Expected Pattern: All bars hit or exceed the line in the C-LIM program If results are invalid, you CANNOT INTERPRET THEM However, if pattern follows that which is expected for ELL’s you can assume that individual is of at least average cognitive ability and this would likely rule out possibility of SLD or other cognitive disorder.

Valid Results If results are ‘valid’ still need to exclude other variables that may have affected test performance such as lack of motivation, emotional disturbance or incorrect scoring or administration. Once it has been established via the C-LIM that test scores have not been invalidated by linguistic proficiency and acculturative learning factors, scores should be evaluated within the context of XBA principles and procedures or their original framework (as specified by the publisher).

Valid test results and interpreting Gc   After establishing the validity of test results, evaluation of any composite score that measures Gc must be done relative to performance as seen within the right- handmost portion of the C-LIM graph (high/high cell). Scores for Gc that fall within the shaded are or above should be interpreted as being average or higher; even if they fall below or outside normal limits. The influence of culture and language cannot be separated from the direct measurement of culture and language as occurs with Gc. Therefore, once validity is established, Gc must be interpreted relative to performance of other diverse individuals as indicated in the shaded area of the C-LIM.

Valid test results and interpreting Gc Average Gc Range for ELL students in the context of valid and interpretable C-LIM results Slightly Different: 15-20 points Gc Cluster of 80 to 85 Different: 20 to 30 points Gc Cluster of 70 to 80 Markedly Different: 25 to 35 points Gc Cluster of 65 to 75

XBA Sample Cases

Does not follow expected pattern for English Language Learners Does not follow expected pattern for English Language Learners. Your results are considered valid and can be interpreted.

Follows expected pattern for English Language Learners Follows expected pattern for English Language Learners. Your testing results are considered to be invalid and cannot be interpreted. Current profile indicates the student is of at least Average cognitive ability given English language skills and level of acculturation.

Questions