Amelia Courts, WESTELL Putting it All Together WV Department of Education

Amelia Courts, Agenda Overview Standards-Based Assessment Design Language Domains Validity Issues Challenges

Amelia Courts, Overview Collaborative Initiative funded by $1.3 million dollar USDE grant Chief Council of School State Officers (CCSSO), LEP SCASS (State Collaborative on Standards and Assessment), American Institutes for Research (AIR) Census Operational / Field test 2004 Goal of balance: Language proficiency development Psychometric requirements Practical feasibility needs Cost

Amelia Courts, Standards-based Title III requirements Performance Standards Steering Committee collected and analyzed all ELP standards from member states Rigorous definition of FEP Realistic expectations for beginning LEPs Informative levels between the two Content Standards

Amelia Courts, Assessment Design Align with Content and Performance Standards Four Content Areas English Language Arts Math / Science / Technology Social Studies Social / School Interactions Four Grade Clusters K-2 * not yet developed

Amelia Courts, Assessment Design Report Performance Reading Listening Writing Speaking Comprehension RLWS and C scores

Amelia Courts, Language Domains Listening Focus on comprehension of academic English Authentic, self-contained events 6 Standards (e.g., follow spoken directions, making inferences) 5 Text Types (phrases, extended presentations)

Amelia Courts, Language Domains Listening Number of Items in final form Item Types Multiple choice*Multiple choice Approx. Administration times 60 min. 75 min. Administration type Group

Amelia Courts, Language Domains Reading 8 Broad Categories (e.g., early reading skills, making inferences) 3 Text Types ( word passages with 5-8 comprehension questions)

Amelia Courts, Language Domains Reading Specifications Number of Items in final form Item Types Multiple choice Approx. Administration times 60 min. 75 min. Administration type Group

Amelia Courts, Language Domains Writing Constructed Response Comprehensibility and discourse features Short (3-5 per form) Extended (1-2 per form) Multiple Choice Editing and Revising Planning and Organizing

Amelia Courts, Language Domains Writing Specifications Number of Items in final form 1 Extended CR 3 SCR 15 Multiple Choice 1 Extended CR 3 SCR 15 Multiple Choice 1 Extended CR 4 SCR 15 Multiple Choice Item Types ECR SCR Multiple choice ECR SCR Multiple choice ECR SCR Multiple choice Approx. Administration times 60 min. 65 min. Administration type Group

Amelia Courts, Language Domains Speaking Task Structure Input Prompt Scaffold Prompt Repeated

Amelia Courts, Language Domains Speaking Specifications Number of Items in final form 16 Item Types Prompted oral response** Prompted oral response* Approx. Administration times 15 (Plus field test items and instructions) 15 (Plus field test items and instructions) 20 (Plus field test items and instructions) Administration type Group

Amelia Courts, Psychometrics Score reporting scales Pilot test Spring states, 31 schools, 100 students per cluster Encouraging results Score reliabilities: , mostly mid-80s Improve item clarity

Amelia Courts, Validity Research Agenda Alignment of items with standards Item-level analysis (scoring, difficulty) Test-level analysis (reliability, scaling/equating) Internal structure/construct validity Correlations with external measures

Amelia Courts, Challenges Spring 2004 Field test Screen and select items Produce 1 operational form Establish item bank Produce student scores for Title III reporting Speaking Component Design to meet intent of law / administrative feasibility Cost

You are the key…to putting it all together! WESTELL