School-level Correlates of Achievement: Linking NAEP, State Assessments, and SASS NAEP State Analysis Project Sami Kitmitto CCSSO National Conference on Large- Scale Assessment June 2006

Create a valuable data set for policy analysis by adding achievement scores to a comprehensive school survey  School and Staffing Survey (SASS)  Extensive information from a national survey of schools, but no achievement scores  National Assessment of Educational Progress (NAEP)  Nationally representative scores comparable between states  State Assessment Database (NLSLSASD)  Collection of all available school-level state assessment data  Scores comparable within states Overview of the Study

 What are the important school characteristics that correlate with achievement?  Do the results of Don McLaughlin and Gili Drori (2000) compare to the results from a larger and more recent set of data?  2000 SASS vs SASS  states vs. 20 states Research Questions

NAEP 1998, 2000 and 2002  Used 2000 Math Grades 4 & 8 and 1998 & 2002 Reading scores for Grades 4 & 8  Used full population estimates  Mean and standard deviation at the school level  Mean and standard deviation at the state level  Replicate weights used Data Assembly NAEP Data

NLSLSASD 2000  Selected two scores for each grade/subject:  Grade 4 Math, Grade 4 Reading  Grade 8 Math, Grade 8 Reading  Remove between state variation  Create standard score within each state: Data Assembly NLSLSASD 2000 Data

NAEP and NLSLSASD Correlation  Using only schools in both NAEP and NLSLSASD:  Calculated correlation between NAEP and NLSLSASD scores at the state level for matched schools Data Assembly NAEP and NLSLSASD School-Level

 Used NAEP to introduce between state differences and variation to standardized scores   Rescaled to mean of 50 and standard deviation of 10  Data Assembly NAEP State-Level and NLSLSASD

School Level Information  From school, principal, teacher and district surveys  Social Background  Organizational Characteristics  School Behavioral Climate  Teacher Characteristics Data Preparation Step 2 SASS 2000

Analysis Sample  Dropped schools with less than 50 students  Did not include schools that were combinations of elementary, middles and or high schools  Missing values: list-wise deletion of observations Teacher Qualifications Dropped  Teacher sample is not random or representative at the school level  High percent of variation was within schools not between schools  Results indicated that these measures were mostly noise Data Set Used for Analysis

Number of Schools With Two Valid Scores Number of Schools in Analysis Sample Data Numbers

Structural Equation Modeling  Similar to multiple regression analysis  Allows for multiple measures of concepts  Models measurement error  Observed variables = Measures  Conceptual factors = Latent Variables Analysis Methodology

Path Model Relating Latent VariablesModel

Measurement ModelModel

Fit Statistics Replication Results

Estimated Coefficients for Achievement Equation Replication Results (cont)

 Latent variables are scaled to one of their measures  ‘Class Size’ is scaled to student/teacher ratio  Coefficients are standardized  A one standard deviation increase in ‘Class Size’ is correlated with a -.23 standard deviation difference in math achievement in elementary schools  Standard deviation of student/teacher ratio in the sample is ~ 4 students/teacher  Mean is 15.5 students/teacher Interpretation of Coefficients

Reported Estimated Effects of Student/Teacher Ratio and Class Size Literature on ‘Class Size’

 Add principal responses to school climate questions  Add additional controls: urbanicity, % IEP, magnet school indicator  ‘Principal Leadership’  ‘Resources’  Per pupil expenditures (district level)  Number of computers  ‘Parent Involvement’  Teacher and principal reports of parent involvement being a problem  School programs to involve parents Avenues for Future Research

 Linking NAEP, NLSLSASD and SASS provides a powerful national sample of schools matched to achievement scores  SASS provide multiple measures of key conceptual factors  SEM provides a methodology to take advantage of the depth of SASS information  Class size found to be correlated with achievement  In middle schools, more important for reading than math  Results on achievement are similar to McLaughlin and Drori 2000 with improved fitConclusions