Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Literacy Coach’s Focus In Data Analysis Program Evaluation Regrouping Form needs-based groups for classroom instruction Assign children to interventions To what extent is my program keeping Benchmark children at benchmark? Choose instructional emphasis To what extent is small-group work moving strategic children to benchmark? To what extent is my program moving Intensive children to benchmark? To what extent are classroom effects apparent?

Literacy Coach’s Focus In Data Analysis Regrouping Form needs-based groups for classroom instruction Assign children to interventions Choose instructional emphasis Which DIBELS reports should I use? Do you have curriculum materials to accomplish this?

Literacy Coach’s Focus In Data Analysis Program Evaluation To what extent is my program keeping Benchmark children at benchmark? To what extent is small-group work moving strategic children to benchmark? To what extent is my program moving Intensive children to benchmark? To what extent are classroom effects apparent?

Evidence to Consider DIBELS Summary of Effectiveness Reports, by project and by school DIBELS Summary of Effectiveness Reports, by Classroom Information about curriculum and instruction that you gather by school (Implementation Survey) and LCs gather by classroom (observations)

Summary of Effectiveness Reports General question: How are the groups moving? “Instructional Recommendation” is a combined, weighted score used as a starting point Specific subtests are targeted at middle and end of year Two reports each grade level each year: Beginning to Middle, Middle to End

How do you find the data? We asked Wireless to generate state-level reports from last year When the bridge is finished to the DIBELS site, you can get these reports from the site  In January, you can start to evaluate changes from last year’s trends

What are they tracking? BeginningMiddleEnd KInstructional Recommendation ISFPSF 1Instructional Recommendation NWFORF 2Instructional Recommendation ORF 3

And who are they tracking? Benchmark Children Strategic Children Intensive Children

What are your expectations? What percentage of benchmark children should be benchmark at the next assessment point? What percentage of strategic children should be benchmark at the next assessment point? What percentage of intensive children should be benchmark at the next assessment point?

Let’s get some evidence KN = N = N = N = 7309

Kindergarten What are the goals? How realistic are they? What obstacles do we face? What strategies are we using?

Kindergarten: Fall to Winter

Kindergarten: Winter to Spring

Kindergarten: Fall to Winter

Kindergarten: Winter to Spring

Kindergarten: Fall to Winter

Kindergarten: Winter to Spring

What does that mean?

First Grade What are the goals? How realistic are they? What obstacles do we face? What strategies are we using?

First Grade: Fall to Winter

First Grade: Winter to Spring

First Grade: Fall to Winter

First Grade: Winter to Spring

First Grade: Fall to Winter

First Grade: Winter to Spring

What does that mean?

Second Grade What are the goals? How realistic are they? What obstacles do we face? What strategies are we using?

Second Grade: Fall to Winter

Second Grade: Winter to Spring

Second Grade: Fall to Winter

Second Grade: Winter to Spring

Second Grade: Fall to Winter

Second Grade: Winter to Spring

What does that mean?

Third Grade What are the goals? How realistic are they? What obstacles do we face? What strategies are we using?

Third Grade: Fall to Winter

Third Grade: Winter to Spring

Third Grade: Fall to Winter

Third Grade: Winter to Spring

Third Grade: Fall to Winter

Third Grade: Winter to Spring

Curriculum Design If you are not yet getting the results you want... Your curriculum design is not yet sufficient OR Your professional development system is not yet sufficient How do you find out which? What do you do about it?

Whole-Class Instruction Why are the benchmark groups not entirely stable? Is the whole-class instruction explicit? Balance? Systematic? Interactive?

Needs-based Instruction Is this time directed by a “guided reading” model or is it truly needs-based? –Do intensive children get support at the word level? Is it organized and repetitive? –Do we differentiate between fluency problems and word-level problems? Do fluency groups get true fluency work? –Do teachers manage instruction so that needs-based time is truly interactive and intensive?

Intervention Are intervention programs matched to children’s needs in the cognitive model? Should we help coaches to select informal assessments or to use their diagnostic assessments?

Next Steps: School Level Find positive outliers in each region... schools where there is significant positive difference between their achievement and the state mean Move to the implementation interviews to see what might explain these differences Steal great ideas for other, less successful schools

Next Steps: Classroom Level Find positive outliers in each school... Classrooms where there is significant positive difference between their achievement and the school mean Move to the LC observations to see what might explain these differences Steal great ideas for other, less successful classrooms