1 Leanna Stiefel and Amy Ellen Schwartz Faculty, Wagner Graduate School and Colin Chellman Research Associate, Institute for Education and Social Policy New York University Education Finance Research Consortium September 26, 2003 Test Score Gaps in New York State Schools: What Do Fourth and Eighth Grade Results Show?

2 Introduction Test scores differ by: race gender income Many hypotheses about why: academic preparation bias in test questions testing environment teacher expectations Research constrained by data

3 “No Child Left Behind” New data reporting requirements NYS ahead of most, releasing ’00-’01 data We use NYS 4 th and 8 th grade ELA and Math We analyze school level gaps in performance between groups Introduction, cont’d.

4 Outline Statewide Test Score Gaps The Race Gap Racial Makeup of State Schools Gaps and School Characteristics by Racial Makeup Gaps in Schools That Are Integrated Relationships between Gaps and School/District Characteristics The Income Gap The Gender Gap

5 The Necessary, the Required and the Boring: Data and Number of Schools New York State Education Department (SED) for academic year by School: School Report Card data (SRC) data Institutional Master File (IMF) data State of Learning/Chapter 655 and Fiscal Profile data

6 The Necessary, the Required and the Boring: Data and Number of Schools, cont’d. Number of Schools in Analyses 2262 “Elementary Schools” – schools with 4 th grades 1074 “Middle Schools” – schools with 8 th grades 207 schools are in both groups i.e., they have both 4 th and 8 th grades Schools must have more than 5 students tested cannot have > 40% IEP

Higher % fourth graders pass White students outperform nonwhite students Advantaged students outperform disadvantaged Females outperform males in English; Math, mixed results

Whites and Asians outperform Hispanics, blacks and American Indians.

9 School Gaps by Race/Ethnicity New York State has segregated schools Predominantly white schools: more than 5 white students and 5 or fewer non-white students Mixed: more than 5 white and 5 non-white students Predominantly non-white schools: 5 or fewer white students and more than 5 non-white students Statewide test score gaps reflect differences in scores across segregated schools and differences within integrated ones

Elementary Schools: 34% are predominantly white (26% of students) 25% are predominantly non-white (23% of students) Middle Schools: 34% are white (19% of students) 17% are nonwhite (14% of students) Racial Mix of New York State Schools

Difference in non-white pass rates between segregated and mixed schools White performance similar in segregated and mixed schools Non-white performance is lower in segregated schools Elementary School Pass Rates and Gaps by Racial Mix of Schools

12 Elementary School Pass Rates and Gaps by Racial Mix of Schools, cont’d. Thus: Larger disparity exists between segregated schools than within mixed schools Segregation poses some challenges for reducing statewide gaps

District Characteristics by Racial Mix of Elementary Schools Non-white schools in NYC White schools in rural and upstate suburbs Mixed schools in Big 4, upstate small cities, and downstate suburbs

District Characteristics by Racial Mix of Elementary Schools, cont’d. White schools are in districts with 95% students white Nonwhite schools in districts with 16% students white

District Characteristics by Racial Mix of Elementary Schools, cont’d. Biggest differences between segregated white schools and segregated non-white schools Segregated white schools mostly “advantaged” districts, except lowest per pupil income and lowest per pupil spending

School Characteristics by Racial Mix of Elementary Schools Predominantly non-white schools are: Largest Highest percentages of students in poverty Highest percentages of English Language Learners

17 Are Any Schools “Beating the Odds”? Yes Let’s look at the racially mixed schools 1034 Elementary Schools 527 Middle Schools Look at both ELA and math exam results “Small:” Differences in white/non-white pass rates + or – 5 percentage points (inclusive) from zero.

Schools That “Beat the Odds” Around 18% of elementary and 14% of middle schools are “beating the odds” with non-traditional or small gaps

Location of Schools by Gap Category Elementary schools that beat the odds are disproportionately in New York City and downstate suburbs (over 85% of the 77 small gap schools) Middle schools (not shown) that beat the odds are disproportionately in New York City, downstate suburbs and rural areas (over 90% of the 32 small gap middle schools)

Elementary School Characteristics by Gap Category Non-traditional gap schools: are disproportionately poor; educate a larger share of children in special education; educate a larger share of children in ELL programs; and, Are disproportionately non-white, with high shares of Asians and Hispanics. Traditional gap schools are disproportionately poor and non-white (particularly black students).

Fourth Grade Racial/Ethnic ELA Pass Rates by Racial Mix of School Pass rate in predominantly non-white schools is lower than in mixed schools In mixed schools, gaps are: largest for black students; large for Hispanic and American Indian students; and, nonexistent for Asian students.

22 When a school is “beating the odds”, do all race groups do better, or just one or two? In general, schools with white/nonwhite gaps that are non-traditional or small are unlikely to be associated with a subgroup rate that is traditional.

Hispanic example: ( ) * 100 = 8.5% (1034 – 55) Comparable numbers for the other subgroups: blacks 10.1%, Asians 3.8%, and American Indians 12.7%.

24 Is the size of the gap related to any school or district characteristics? To start to analyze this question, we use linear relationships between gap size and characteristics. The following graphs do not prove or disprove causation – but they are informative.

How are ELA and Math Gaps Related? Pearson r 0.59

How is ELA Gap Related to Overall Pass Rates? Pearson r -0.17

How is ELA Gap Related to the White Pass Rate? Pearson r 0.21

How is ELA Gap Related to the Non-white Pass Rate? Pearson r -0.59

How is ELA Gap Related to the Black Pass Rate? Pearson r -0.38

How is ELA Gap Related to the Hispanic Pass Rate? Pearson r -0.37

How is ELA Gap Related to the Asian Pass Rate? Pearson r -0.28

How is ELA Gap Related to School Size (Enrollment)? Pearson r -0.06

How is ELA Gap Related to School Poverty? Pearson r 0.05

How is ELA Gap Related to Limited English Proficiency? Pearson r -0.04

How is ELA Gap Related to District’s Expenditure Per Pupil? Pearson r -0.02

36 Conclusions on Race/Ethnicity Gaps School segregation presents challenges for undoing statewide gaps – some schools can’t report subgroup scores due to segregation There is a sizable number of schools that have small or non-traditional gaps (from 14 to 18%)

37 Conclusions on Race/Ethnicity Gaps, cont’d. Schools with small or non-traditional gaps are not equally distributed across the state 85% of small gap elementary schools are in New York City and downstate suburbs 90% of small gap middle schools are in New York City, downstate suburbs and rural areas The only real “simple” relationship between the size of the gap and school or district characteristics is that: Higher pass rates by minorities are associated with lower gaps

38 The Income Gap The advantaged perform better than the disadvantaged in both ELA and Math, and across grades.

39 The Income Gap, cont’d. New York State schools show some income segregation, but it is not as profound as racial segregation. Predominantly Advantaged: more than 5 advantaged students and 5 or fewer disadvantaged students. Mixed: more than 5 advantaged and 5 disadvantaged students. Predominantly Disadvantaged: 5 or fewer advantaged students and more than 5 disadvantaged students.

Elementary Schools: 16% are predominantly advantaged (13% of students) 12% are predominantly disadvantaged (11% of students) Middle Schools: 13% are advantaged (10% of students) 8% are disadvantaged (4% of students) Income Mix of New York State Schools

Pass rates differ between income segregated and mixed schools Segregated schools have higher pass rates for advantaged students and lower for disadvantaged students. Elementary School Pass Rates and Gaps by Income Mix of Schools

42 Elementary School Pass Rates and Gaps by Income Mix of Schools, cont’d. Thus, as with race: Larger disparity exists between income segregated schools than within mixed schools

District Characteristics by Income Mix of Elementary Schools Disadvantaged schools in NYC and Big 4 Advantaged schools in suburbs

District Characteristics by Income Mix of Elementary Schools, cont’d. Advantaged schools in districts average 88% students white Disadvantaged schools in districts average 20% students white

District Characteristics by Income Mix of Elementary Schools, cont’d. Biggest differences between income segregated schools: Predominantly advantaged schools have highest district per pupil spending Predominantly disadvantaged schools do not have lowest per pupil spending.

School Characteristics by Income Mix of Elementary Schools Predominantly disadvantaged schools are, on average: Largest Highest percentages of special education and English Language Learners Highest percentages of students in poverty

47 Are Any Schools “Beating the Odds”? Yes Let’s look at the mixed schools by income 1629 Elementary Schools 846 Middle Schools Look separately at ELA and math exam results “Small:” Differences in advantaged/ disadvantaged pass rates + or – 5 percentage points (inclusive) from zero.

Schools That “Beat the Odds” 4 th grade: Substantial fraction have small or non-traditional gaps: 20% for ELA and 29% for Math. Lower % for 8 th grade. Majority, between 70% and 80% of schools, have traditional gaps

Location of Schools by Income Gap Category, ELA Schools that “beat the odds:” disproportionately in New York City and downstate small cities. Schools with traditional gaps: disproportionately in rural areas and upstate small cities.

Elementary School Characteristics by Income Gap Category, ELA Traditional gap schools: are smaller; have lower poverty rates; and, have lower ELL rates. These schools generally educate a less challenged group of students do very well by their advantaged students do least well by their disadvantaged students

51 Is the size of the gap related to any school or district characteristics? As with race, we use linear relationships between gap size and characteristics. The following graphs do not prove or disprove causation – but they are informative.

How are ELA and Math Gaps Related? Pearson r 0.57

How is ELA Gap Related to Performance of the Advantaged? Pearson r 0.44

How is ELA Gap Related to Performance of the Disadvantaged? Pearson r -0.45

How is ELA Gap Related to School Size (Enrollment)? Pearson r -0.12

How is ELA Gap Related to School Poverty? Pearson r -0.13

How is ELA Gap Related to Limited English Proficient? Pearson r -0.09

58 Conclusions on Income Gaps On average, in New York State, advantaged students do better than disadvantaged students. Again, larger disparity between segregated schools than integrated. Income segregation is not as profound as racial segregation. Segregated schools have higher pass rates for the advantaged but lower for the disadvantaged. Disadvantaged schools are larger and have higher percentage of students in special education and ELL.

59 Conclusions on Income Gaps, cont’d. A significant number of schools have comparable performance between income groups 20% for ELA and 29% for Math. Lower % for 8 th grade. Schools that “beat the odds” are not equally distributed across the state: Disproportionately more in NYC, downstate small cities. Traditional gap schools are smaller, educate a less challenged group of students, and do least well by their disadvantaged students. Several variables related to income gaps: subgroup pass rates; school poverty; and, ELL percentages.

60 School Gaps by Gender Females outperform males in English; Math, mixed results across grades.

Gender Mix of New York State Schools New York State has very few single-sex schools. Predominantly Female: more than 5 female students and 5 or fewer male students. Mixed: more than 5 female and 5 male students. Predominantly Male: fewer than 5 female students and more than 5 male students.

Different gender gap between 4 th and 8 th grades: Math: Almost no gap in 4 th grade; boys do better than girls in 8 th. ELA: Girls do better in both grades; gap is larger in 8 th grade. For consistency: Negative gaps will be called “Non-traditional” and positive gaps “Traditional.” Elementary School Pass Rates and Gaps

Distribution of Schools by Gender Gap Category Many schools have small gaps.

School Characteristics by Gender Gap Category, Elementary Schools Note lower rates of poverty, special education, and ELL. Schools with small gaps show above average performance for both groups.

School Characteristics by Gender Gap Category, Middle Schools Note higher rates of poverty, special education, and ELL. Schools with small gaps show below average performance for both groups.

66 Conclusions on Gender Gaps Segregation by gender is negligible. Many schools have small gaps. No Math gap in 4 th grade larger and positive in 8 th grade. ELA gender gap in favor of females is clearly evident in 4 th grade even larger in 8 th grade. Schools with small gaps… in 4 th grade show above average performance for both males and females. in 8 th grade, show below average performance for both.

67 Wrap-up Racial segregation is considerable; income segregation less profound, but still considerable. A considerable number of schools cannot report subgroup performance due to segregation. Income segregated schools have higher pass rates for the advantaged but lower for the disadvantaged. Disadvantaged schools are larger and have higher percentages of special education and ELL students. Traditional income gap schools are smaller and educate a less challenged group of students, and do least well by their disadvantaged students.

68 Wrap-up, cont’d. A sizeable number of schools have small or non- traditional gaps, especially between genders. Schools with small or non-traditional gaps are not distributed equally across the state. disproportionately in NYC and downstate districts (for income, race, and gender) The wide ELA gender gap against boys in 8 th grade is of particular concern. Also of concern: schools small gaps in 8 th grade show below average performance for both females and males.