Center for Educational Performance and Information The Tortured Path of Data Through Policy and Politics January 22, 2007 Margaret Merlyn Ropp, Ph.D. Director

The birth of a data set…it’s more painful than you think

In MI, a compliance reporting need starts the process Required (penalties) vs. “nice-to-have for research” Headlee Amendment - requires the state to reimburse local governmental units for any new state-mandated programs (potentially even data collections) Grant programs allow the collection of much more data because there is an expectation of using data to monitor and audit in exchange for the accepting grant $$.

The birth of a data set…it’s more painful than you think Step 1) CEPI facilitates a “data governance” process with state agency data managers to identify and define definitions of mandated data elements and the manner in which they should be collected from districts. Step 2) CEPI publishes the data definitions six months in advance of a new collection to allow software vendors to make changes to district systems. Step 3) Districts enter data into their systems of record and then submit data to CEPI at specified times of the year Step 4) CEPI receives, cleanses, structures and stores the data Step 5) Analysts use analysis and reporting software to make federal compliance reports (now through Education Data Exchange Network (EDEN) as well as public reports and data sets.

On the surface, educational data are neutral… but the needs are not. Politics The reauthorization of ESEA (No Child Left Behind Act of 2001) formalized federal accountability in what used to be state-only turf in local education. Policy Each state responded with policies to meet the federal requirements and defined the data elements needed to do so.

Graduation rate calculations: A case study of the tortured path Politics: NCLB, Title I Regulations Graduation Rate: The graduation rate for public high schools, which means--(A) The percentage of students, measured from the beginning of high school, who graduate from high school with a regular diploma (not including an alternative degree that is not fully aligned with the State's academic standards, such as a certificate or a GED) in the standard number of years;

Graduation rate calculations: A case study of the tortured path Politics: National Governor’s Association “Graduation Counts Compact” standard, four-year adjusted cohort graduation rate Divide the number of on-time graduates in a given year by the number of first-time entering ninth graders four years earlier. Only regular diplomas count - no GEDs or certificates Response to nation-wide concern over the variety of states’ definitions and calculation methodologies – hot news for education publishers and media for three years.

Technical impact - Tracking individual students over time Challenges include: Implementing and maintaining a stable student identifier - CEPI’s Unique Identification Code (UIC) Connecting multiple years of data (e.g., 1.7 million students reported 3 times a year = 16 total cycles needed - technically very difficult and expensive Using a system designed to capture student FTE pre-2001, not student mobility Districts are responsible for data entry. Data quality is a shared responsibility between LEAs and the state.

Political impact – No child transferred out In Michigan, big changes include Tracking individual students over time instead of an “estimated four-year graduation rate” All public school students should be included in the calculation population - no more transfer outs for “completers” Confirming transfers And…students must graduate in four years in order to “count” for accountability

What do we do with the bad news of the transition to a true cohort? Understand that the new rates will be more accurate and give us a solid baseline of data Provide reporting tools to present data so that districts can take action CEPI reports (beginning fall 2006) are the first proactive uses of state compliance data by The state identifying students who may be at risk of not graduating “on time” and districts finding interventions to get them there tracking students in their original cohort, regardless of how many times they move over four years.

Cool things about working at CEPI (Meg’s view) CEPI data sets represent the entire “population” under study An opportunity to put data visualization through the reality test – can we use predictive data mining to impact the future in a positive direction instead of merely reporting what happened? All our efforts should focus on improving student learning and achievement

Questions?