English Language Arts Performance Assessment: Its Fairness and Predictive Validity Jia Wang, David Niemi, Pete Goldschmidt, and Haiwen Wang UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing (CRESST) American Educational Research Association –Applying Research-Based Performance Assessment Models in Routine Practice in a Large Urban School District: The Pleasure-Pain Principle San Diego, CA April 15, 2004

Purpose This study is to examine the English language arts performance assessment in the following two aspects: 1. Relative fairness 2. Predictive validity

Research Questions 1. Is the performance assessment a fair test? 2. Is the performance assessment predictive of the California High School Exit Exam (CAHSEE)?

Data Description 50 schools 5,427 students 9 th graders in Spring 2001 and 10 th graders in Spring 2002

School Description

Student Description 21% of the students were English proficient; 81% of the students were Hispanic; 26% of the students were immigrant; 75% of the students spoke Spanish at home; 75% of the students were in Title 1; 73% of the students were in lunch program; 3% of the students were gifted; and 1% of the students were in special education program.

Hierarchical Linear Model (school level) School level variables: Average class size Percent of students in lunch program School enrollment (in 1,000s) API rank in 2001

Hierarchical Linear Model (student level) Student level variables: Gender Ethnicity English language proficiency Home language Immigrant status SES (lunch program and Title 1) Special education program Gifted status

Results on Fairness (Variance Partition)

Results on Fairness (summary 1)

Results on Fairness (summary 2) School level: None of the 4 variables were significant Student level: Ethnicity and SES were not significant Positive female effect Positive gifted and former ELL effects Negative ELL, Spanish (HL), immigrant, and special education effects

Result on Fairness (school level)

Result on Fairness (student level 1)

Results on Fairness (Student level 2)

Results on Predictive Validity (cross-tabulation)

Results on Predictive Validity (school level)

Results on Predictive Validity (student level)

Results on Predictive Validity (reading)

Results on Predictive Validity (mathematics)

Conclusions 1. The performance assessment a fair test. 2. The performance assessment is predictive of students’ California High School Exit Exam results.