CALIFORNIA DEPARTMENT OF EDUCATION Jack O’Connell, State Superintendent of Public Instruction Results of the 2005 National Assessment of Educational Progress for California and the Nation Presented to the California State Board of Education November, 2005

JACK O’CONNELL State Superintendent of Public Instruction 2 NAEP Design NAEP collects data from a sample of schools in each jurisdiction. In many schools, only a random sample of students in a given grade may be assessed. Samples are stratified by geographic location, ethnic mix, school size, and STAR scores. Parents may opt their students out of NAEP. Learning disabled and English learner students may be excluded from the assessment if NAEP does not offer the appropriate accommodation or if the student does not normally take state tests. The NAEP assessments are matrix tests: No student answers every question. Scores are only reported at the state and national level.

JACK O’CONNELL State Superintendent of Public Instruction 3 Interpreting NAEP Results Comparisons should be focused on specific sub-groups of interest. –Whole state population comparisons are problematic. –For example, English learners have substantial impacts on California results. NAEP scores contain variability due to sampling and measurement error. –Statistical tests must be conducted to determine if observed differences are larger than would occur simply by chance. Changes in populations over time can impact statewide trends. –For example, the change in the proportion of economically disadvantaged students between 2003 and 2005.

JACK O’CONNELL State Superintendent of Public Instruction 4 California Participation in NAEP Grade 4 and 8 Reading

* * * *Change from 1992 is statistically significant 5

*Change from 1990 is statistically significant * * * * 6

Changes in California NAEP Grade 4 Math and Reading Scores 1992 to Not significantly different from

California NAEP Grade 4 Reading Average Scale Scores and Sample Population Percentages for Major Ethnic Groups 1992 to 2005 All Changes from 1992 are statistically significant Bubbles are centered on group average, area of bubble is proportional to population proportion 8

California NAEP Grade 4 Math Average Scale Scores and Sample Population Percentages for Major Ethnic Groups 1992 to 2005 Bubbles are centered on group average, area of bubble is proportional to population proportion 9

JACK O’CONNELL State Superintendent of Public Instruction 10 Notes on Comparing Performance Among States Because of sampling and measurement error in NAEP, small differences in scores may not be significantly different. –Therefore, a ranking of states by average scale score is inappropriate. Differences in population composition can greatly affect scores. Differences in exclusion rates for various groups can potentially confound results.

English Learner Proportions and Exclusion Rates on NAEP 2005 Grade 4 Reading for Selected States. StatePercent of All Students Identified as EL Percent of All Students Excluded for EL Percent of English Learners Excluded California % Texas % New York7228.6% Florida8225.0% Illinois % Source: NAEP 2005 Reading Report for California. Appendix A: Overview of Procedures Used for the NAEP 2005 Reading Assessment, page 27. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP). 11

Grade 4 Math - Overall 12

Grade 4 Math - White 13

Grade 4 Math - Black 14

NAEP Grade 8 Reading 2005 Results for Selected States – All Students * Average scale score significantly different from California + Average scale score of the 90 th percentile is significantly different from California

NAEP Grade 8 Reading 2005 Results for Selected States - White Students * Average scale score significantly different from California + Average scale score of the 90 th percentile is significantly different from California 16

NAEP Grade 8 Reading 2005 Results for Selected States - Black Students * Average scale score significantly different from California + Average scale score of the 90 th percentile is significantly different from California 17

JACK O’CONNELL State Superintendent of Public Instruction 18 Summary California student’s NAEP scores in reading and mathematics parallel the national trend. Progress in reading is slower than in mathematics. Hispanic students that are not English learners have made the greatest gains on NAEP in recent years.