1. Evaluating a PreK-3 rd Grade Approach

2. Evaluation is not a one-size-fits-all undertaking. Because PreK-3 rd reforms look different in different places, the type of evaluation needed depends on where the PreK-3 rd effort is in its development and what types of reforms are being implemented. results

3. PreK-3 rd reforms have four developmental stages. Stage 1: Developing a PreK-3 rd Approach Initial development and implementation of PreK-3 rd efforts Stage 2: Examining Implementation Results Determining whether PreK-3 rd reforms produce changes for those who work and function within the PreK-3 rd system Stage 3: Establishing Child-Level Success Rigorously testing for evidence that PreK-3 rd reforms produce impacts for children. Stage 4: Scaling Up Spreading PreK-3 rd reforms across a school district, region, state, or multiple states.

4. Stage 1 evaluation focuses on early implementation and continuous improvement. Stage 1: Developing a PreK-3 rd Approach Stage 2: Examining System- Level Changes Stage 3: Establishin g Child- Level Success Stage 4: Scaling Up QUESTIONS: How are parts or aspects of the PreK-3 rd effort changing? What can be done to improve PreK-3 rd implementation? WHAT DATA TO COLLECT: Some baseline data on child status and performance Implementation and process data on specific, discrete strategies or efforts “Pipeline” data (e.g., feeder patterns, mobility)

5. Stage 2 evaluation tracks changes to actors in the system. Stage 3: Establishin g Child- Level Success Stage 4: Scaling Up WHAT DATA TO COLLECT: Results measures or indicators for the groups of interest (e.g., teachers, superintendents, or parents) Some progress data on child status and performance Stage 1: Developing a PreK-3 rd Approach QUESTIONS: Are PreK-3 rd reforms producing behavior changes for the groups (of adults) involved in implementing those reforms? Stage 2: Examining Implementation Results

6. Stage 3 evaluation tests for evidence that reforms produce impacts for children. Stage 1: Developing a PreK-3 rd Approach Stage 4: Scaling Up QUESTIONS: Do PreK-3 rd reforms produce impacts for children? WHAT DATA TO COLLECT: Child-level impacts at every stage along the PreK-3 rd grade continuum and at the end of 3 rd grade Stage 3: Establishing Child-Level Success Stage 2: Examining System- Level Changes

7. Stage 4 evaluation tracks implementation and impacts at scale. Stage 1: Developing a PreK-3 rd Approach QUESTIONS: Are reforms being implemented with high fidelity? Do reforms produce results across places or populations? WHAT DATA TO COLLECT: Implementation results and child- level impacts across all sites in a certain geographic location, such as a city, county, region, or state Stage 2: Examining System- Level Changes Stage 3: Establishin g Child- Level Success Stage 4: Scaling Up

8. Identify which stage of development your PreK-3 rd grade effort is in. Stage 1: Developing a PreK-3 rd Approach Stage 2: Examining Implementation Results Stage 3: Establishing Child-Level Success Stage 4: Scaling Up

9. What are the implications for your evaluation efforts?  What are the questions you need to be asking yourselves?  What kind(s) of data are you collecting?  How are you evaluating your efforts? What methods are you using?  Who is using the data you collect? For what purpose?