Assessing Student Growth to Foster School Excellence Coalition for Psychology in Schools and Education Education Leadership Conference American Psychological Association September 18, 2006

Overview Goal: To understand current uses of assessment in NCLB and to clarify consensus on how APA might influence NCLB reauthorization.Goal: To understand current uses of assessment in NCLB and to clarify consensus on how APA might influence NCLB reauthorization. PresentationsPresentations –Status, improvement, and growth models of school accountability (Jeff Braden) –Educational evaluation of English Learners: Issues and answers for measuring progress fairly (Sam Ortiz) –What is school excellence? Is it the same for everyone? (Gary Stoner) Discussion: How should we work to maintain/change NCLB? (Steve Rollin)Discussion: How should we work to maintain/change NCLB? (Steve Rollin)

Status, Improvement, and Growth Models of School Accountability Jeffery P. Braden, PhD Committee on Psychological Tests & Assessments North Carolina State University

Overview Identify adequate yearly progress (AYP) definitions under No Child Left BehindIdentify adequate yearly progress (AYP) definitions under No Child Left Behind AYP modelsAYP models –Status –Improvement (aka “Safe Harbor”) –Growth Promises and Problems with Growth ModelsPromises and Problems with Growth Models

No Child Left Behind: Adequate Yearly Progress AYP currently defined by three types of targetsAYP currently defined by three types of targets –Participation in annual state tests –Performance on state tests (% proficient) –One non-test indicator (attendance, graduation rate) NCLB mandates testing for…NCLB mandates testing for… –Reading & Mathematics (and in 2007, Science) –Grades 3-8, once in HS (but different for Science) –Specific populations w/in schools

AYP Targets Defined by… Test outcomes = 4 dimensionsTest outcomes = 4 dimensions –Participation (> 95%) & Performance (% proficient) –For two subject areas (Reading & Math) –2 subject areas X 2 types of goals = 4 dimensions Student groups = 9Student groups = 9 –All students –Ethnicity (American Indian/Alaska Native, Asian/PI, black, Hispanic, white) –Economically disadvantaged –Limited English Proficient –Students with Disabilities

AYP Targets For A School Pool data from all eligible gradesPool data from all eligible grades Determine if minimum sample size is reached for the 9 target groupsDetermine if minimum sample size is reached for the 9 target groups –If yes, target applies –If no, target does not apply and data “roll up” to district, state Compare test data to participation & performance goals for all eligible groupsCompare test data to participation & performance goals for all eligible groups Failure to miss any target for any group = failure to meet AYP for schoolFailure to miss any target for any group = failure to meet AYP for school

ReadingReading Math AYP is determined by making it over all 18 hurdles (9 hurdles for reading and 9 for math) by disaggregation of data. Composite American Indian American Indian Asian Black White Hispanic Students with Disabilities Students with Disabilities Low Income Low Income LEP

AYP = Status Model Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

AYP Safe Harbor: Improvement Model Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

Comparison of Models Promise of status modelPromise of status model –Easy to understand –Same goals for all (equity) –Leads to 100% proficiency by 2014 Problems of status modelProblems of status model –Fails to accommodate differences in student populations between schools –Fails to identify/reward progress toward proficiency –Increasingly difficult targets yield high rates of school failure

Growth Model Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do accountability models differ? Washington, DC: Council of Chief State School Officers.

Comparing Models: A Hypothetical Example North Carolina AYP Performance TargetsNorth Carolina AYP Performance Targets –76.7% annual target for Reading –95% participation goals Failing AYPFailing AYP –Not meeting AYP in three consecutive years –No meaningful improvement; fails to meet safe harbor provisions –Yet strong growth for students entering in 3 rd grade over three consecutive years

M = 216 M = 251 State Proficiency Level for 5 th Grade = 247

Additional Issues for Growth Models Value addedValue added –Implies causality by attempting to factor out other influences (e.g., demographics, typical school gains) What constitutes appropriate growth?What constitutes appropriate growth? –Progress toward proficiency standard –Improvement generally Can growth be measured longitudinally?Can growth be measured longitudinally? –Qualitative vs. quantitative change across academic domains, scaling issues

Using Growth Models for AYP PromisesPromises –Accommodates differences between schools in student populations –Could reward progress –Potential linkage to progress monitoring ProblemsProblems –Capacity to measure & predict change over time –Mobility –Equity –Practicality

Educational evaluation of English Learners: Issues and answers for measuring progress fairly (Sam Ortiz)Educational evaluation of English Learners: Issues and answers for measuring progress fairly (Sam Ortiz)Educational evaluation of English Learners: Issues and answers for measuring progress fairly (Sam Ortiz)Educational evaluation of English Learners: Issues and answers for measuring progress fairly (Sam Ortiz) What is school excellence? Is it the same for everyone? (Gary Stoner)What is school excellence? Is it the same for everyone? (Gary Stoner)What is school excellence? Is it the same for everyone? (Gary Stoner)What is school excellence? Is it the same for everyone? (Gary Stoner)

Discussion

Proposed Changes to NCLB Drop 100% proficiency goal by 2014Drop 100% proficiency goal by 2014 –Replace with “goals that are ambitious, scientifically achievable, and increase regularly…” Drop safe harbor (improvement) provisionsDrop safe harbor (improvement) provisions –Replace with “growth concurrent with achieving proficiency on State annual assessments or on scientifically based measures for student progress monitoring (as defined by the National Center on Student Progress Monitoring)”

Proposed Changes to NCLB Require states to align proficiency definitions to NAEP (prevents low/varying standards)Require states to align proficiency definitions to NAEP (prevents low/varying standards) Increase test inclusion requirement for students in US schools from three to six years.Increase test inclusion requirement for students in US schools from three to six years. Alternate assessmentsAlternate assessments –Change “significant [cognitive] disability” –Increase cap counted toward proficiency from 1% to 2% Modified achievement standards cap from 2% to 3%Modified achievement standards cap from 2% to 3%