Oregon Reading First Cohort B Regional Coaches’ Meeting October, 2005 Progress Monitoring Oregon Reading First Cohort B Regional Coaches’ Meeting October, 2005 © 2005 by the Oregon Reading First Center Center on Teaching and Learning

Cohort B, Regional Meeting Content Development Content developed by: Carrie Thomas Beck Wayne Callender Rachell Katz Jeanie Mercier Smith Additional support: Katie Tate

Acknowledgments Oregon Department of Education Center for Teaching and Learning, College of Education, University of Oregon Early Childhood Research Institute on Measuring Growth and Development Select slides from this presentation were modified from the presentation entitled: Using DIBELS in an Outcomes Driven Model (Good, Kaminski, ReMillard, Ebmeyer, VanLoo, 2003) U.S. Department of Education, Office of Special Education Programs

Copyright All materials are protected by copyright and should not be reproduced or used without expressed permission of Dr. Carrie Thomas Beck, Oregon Reading First Center. Selected slides were reproduced from other sources and original references cited.

Reading First Initiative has Outlined Four Purposes for Assessment: Outcome - Assessments that provide a bottom-line evaluation of the effectiveness of the reading program. Screening - Assessments that are administered to determine which children are at risk for reading difficulty and who will need additional intervention. Diagnosis - Assessments that help teachers plan instruction by providing in-depth information about students’ skills and instructional needs. Progress Monitoring - Assessments that determine if students are making adequate progress or need more intervention to achieve grade level reading outcomes.

DIBELS and the Purposes of Assessment Diagnosis Not an objective of DIBELS May give direction for further assessment Progress Monitoring Sensitivity to growth and alternate forms allow frequent, ongoing monitoring Outcome DIBELS benchmark goals provide overall outcome criteria Screening Able to assess all children three times a year to identify at-risk students ** DIBELS is not the only assessment you need in reading. Schools/teachers should examine their current assessments and ask: What information do each of these assessments give us? How do we use the information? Is collecting the information easy and efficient? Can we modify it? Is the information redundant with another assessment? What information do we still need to know?

Progress Monitoring Repeated, formative assessment to evaluate progress toward important goals for the purpose of modifying instruction or intervention. Frequency of Progress Monitoring 3 times per year for students at low risk (All Students) Benchmark 1 per month for students with some risk Strategic 1 per every two weeks for students at risk Intensive

Effects of Progress Monitoring: Fuchs and Fuchs (1986) Reading achievement was improved by… monitoring progress graphing progress adding decision rules setting ambitious goals

DIBELS Progress Monitoring Data Base Logistics

DIBELS Progress Monitoring Logistics Progress Monitoring Materials Progress Monitoring Data System Selecting Students Selecting Measures (e.g., 1st Grade ORF for 3rd grade Student) Entering Scores Progress Monitoring Graphs/ Reports

Materials Materials for: 20 Alternate Forms per measure ISF PSF NWF ORF (1st, 2nd, 3rd) (not LNF) 20 Alternate Forms per measure One booklet per measure One Student Materials per measure

Progress Monitoring Data Base

Progress Monitoring Menu

Selecting Students

Selecting Students

Selecting Students

Entering Scores

Entering Scores

Selecting Out-of-Grade Measures Two Methods of Entering “Out-of Grade” Measures When Selecting Class/Grade When Entering Data

Selecting Out-of-Grade Measures Select Measure For ORF only: Choice of Grade Level

Viewing Graphs/ Reports

Viewing Graphs/ Reports Select School, Year, Grade, and Type of Graph (Color or Black & White)

Progress Monitoring Graphs

DIBELS Progress Monitoring: Guidelines Administer 2 measures at each time: 1. The measure for the goal that the student is currently working on, and 2. The measure for the next most difficult goal.

Guidelines: It is not necessary to progress monitor students on the Letter Naming Fluency (LNF) measure. LNF is primarily a measure of risk. Do not progress monitor first, second, or third graders on the Initial Sounds Fluency (ISF) measure unless the student bottoms out on the Phoneme Segmentation Fluency measure (e.g., scores less than 5).

Guidelines: Continue monitoring on ISF, PSF, and NWF until the student has met the benchmark goal for two consecutive data points. Once a student is progress monitored on ORF at his or her grade level, continue to monitor on ORF even after the student reaches the benchmark range.

Guidelines: When progress monitoring on ORF, select the grade level passages that will be sensitive to the instruction provided. It is only necessary to administer one passage each time a student is progress monitored on ORF. Once a student meets the benchmark range on a particular grade level passage (e.g., 40-60 cwpm on first grade passages), then move the student up to the next grade level passages (e.g., second grade passages) and work on the corresponding benchmark goal (90 cwpm).

Guidelines: Caution: If a student is being progress monitored using ORF passages below grade level, it is critical to plan ahead to provide a schedule that will allow the student to close the gap as quickly as possible.

Guidelines: First Grade Example First grade benchmark assessment in Fall measures LNF, PSF, and NWF (ORF will start in the Winter) On the Fall data collection, Fred scored 18 on PSF and 7 on NWF. Monitor on: PSF (goal he is currently working on) NWF (the next most difficult goal) As soon as Fred meets the goal for PSF (35 or more) on two consecutive data points, then monitor on NWF and ORF. Once Fred meets the NWF goal, then continue monitoring on ORF for the remainder of the year.

Guidelines: Second Grade Example In the Fall, Sarah scored 20 on NWF and 10 on ORF. Monitor on: 1. NWF (goal she is currently working on) 2. ORF* (next most difficult goal) *Monitor with first grade ORF passages so as to be sensitive to intervention. Note: It is probably a good idea to go back and check to see if your second graders made the benchmark for PSF. Look at Spring of first grade data. If student is not fluent on segmentation, you’ll want to provide some instruction on segmentation and thus monitor on PSF and NWF.

Guidelines: Third Grade Example In the Fall DIBELS data collection, Kelsey scored 12 on ORF. Administer PSF and NWF to determine if Kelsey has met the benchmarks. Based on performance on PSF and NWF, select the two measures to monitor Kelsey on. Remember to begin with the first grade ORF passages if that is one of the goals you are monitoring her on. Place Kelsey in an intervention program that allows for acceleration and provide Kelsey with enough instructional time to make good lesson progress.

Guidelines: Third Grade Example In the Fall DIBELS data collection, Les scored 50 on ORF. Administer NWF to determine if Les has met the benchmark. Progress monitor on NWF if necessary. If Les is placed in a second grade level of a decoding program, begin progress monitoring using second grade ORF passages.

Guidelines: Kindergarten Example Kindergarten benchmark assessment in Fall measures LNF and ISF (PSF and NWF will start in the Winter) On the Fall data collection, Ginger scored 5 on ISF and has an intensive instructional recommendation. Monitor on: ISF (goal she is currently working on) PSF (the next most difficult goal) As soon as Ginger meets the goal for ISF (25 or more) on two consecutive data points, then monitor on PSF and NWF. Once Ginger meets the PSF goal, then continue monitoring on NWF for the remainder of the year.

Let’s Try Some . . . Using the student data and grade level information on the DIBELS Progress Monitoring handout, determine which measures you would monitor each student on.

DIBELS Progress Monitoring Activity

“Easing In” to Progress Monitoring With your ERT, develop a plan to begin progress monitoring students in October. It is not necessary to have a “full blown” progress monitoring system in place these first few months. Depending on the number of intensive students in your school, you may decide to monitor intensive students once a month to begin with. Similarly, you may decide to begin by monitoring strategic students once every 6 weeks. Depending on the numbers of intensive and strategic students in your school, you may decide to progress monitor on just one measure to begin with. Goal is to have a full progress monitoring system in place after the January DIBELS data collection.

Bringing Classroom Teachers and Specialists On Board ORFC will be offering DIBELS administration training in the upcoming months (October, November, or December) to train classroom teachers and specialists to progress monitor. All teachers and specialists should be trained on DIBELS administration by January of 2006. Each classroom teacher should be progress monitoring at least 2-3 of the students in their reading group(s) to gain experience with the measures.

How to Use Graphs to Summarize and Interpret Progress Monitoring Data Why not use a Graph??! Wayne

Progress Monitoring Booklets

Benefits of Using a Graph Creates a learning picture. Allows for decision making. Helps predict learning. Provides documentation. Makes data easier to interpret.

“What are major graphing conventions?” vertical lines (dots) are week lines (Week 1, Week 2, Week 3, Week 4) thick vertical lines indicate a change in month horizontal lines (dots) are counting/data lines you must plot the junction of the week and the data on one dot shaded area indicates benchmark range baseline depicts performance before an intervention aimline tells you the expected rate of learning use a phase line when you make a change in the intervention of a goal The Y (vertical) axis depicts performance The X (horizontal) axis depicts time/categories/nominal data

Data Should Be: Stable Representative Three data points or more Collected by a reliable assessment procedure Collected in a relatively short period of time Representative Teacher says it is “typical” Accurately reflects the student’s performance

Graphing the Baseline: Before Intervention

Validating Need for Support Verify need for instructional support by retesting with alternate forms until we are reasonably confident. Phoneme Segmentation Fluency January cutoff Good, Kaminski, ReMillard, Ebmeyer & Van Loo, 2003

Exploring Support - What is an Aimline? The aimline connects where you are to where you want to get to, and shows the course to follow to get there. Phoneme Segmentation Fluency Aimline Good, Kaminski, ReMillard, Ebmeyer & Van Loo, 2003

“What is a data decision rule?” A decision rule is the systematic procedure by which patterns of data are analyzed. This data analysis assists in making a decision about the effectiveness of an intervention.

Why Decision Rules? How do you know when to continue or modify an intervention? Do you have unlimited time to continue with interventions that are not working? Are students growing at the necessary rate required to reach the DIBELS benchmark? Would you like to know which things work and which things don’t work for your students?

Applying the Decision Rule Three data points are plotted and the number of data points above or below the aimline are used to determine if the individual is achieving as predicted. Administer 1 probe each week for 3 weeks or every other week until three data points are obtained. Each data collection period, plot your student’s newest score. This is a quick and easy method which reduces variability and eliminates the need for a trendline.

Data Decision Rules If three (3) consecutive data points are above the aimline, raise the criteria

Data Decision Rules If three (3) consecutive data points are below the aimline, change the intervention

Data Decision Rules If neither of the above rules apply, make no change

Things to consider in decision-making: Focus on the question: “Will the individual reach his/her goal by the end of the goal period?” Decide to change an intervention whenever the rate of progress falls below the expectation. Think of changes to the intervention as fine tuning (alterable variables) rather than major reconstruction of interventions. Notes

Changes in the intervention Draw a phase line A phase line is drawn vertically on the graph to identify the beginning point where the intervention change occurs. Notes

Evaluating Support – Modify Instruction? Whoops! Time to make a change! Phoneme Segmentation Fluency Aimline Good, Kaminski, ReMillard, Ebmeyer & Van Loo, 2003

Evaluating Support - Is Instructional Support Sufficient Now? Phoneme Segmentation Fluency Aimline Good, Kaminski, ReMillard, Ebmeyer & Van Loo, 2003

DIBELS are the GPS for educators! Phoneme Segmentation Fluency Aimline Good, Kaminski, ReMillard, Ebmeyer & Van Loo, 2003

Good, Kaminski, ReMillard, Ebmeyer & Van Loo, 2003 How am I doing? What course should I follow? Where is the port? Where are we? GPS On Port Desired Course You are Here Actual Course Good, Kaminski, ReMillard, Ebmeyer & Van Loo, 2003

Let’s Try One: Using the following information, set up a progress monitoring graph for Tony (a second grader) on NWF: • Sept. Week 3 (DIBELS Baseline) = 12 • Draw in aimline. • Oct. Week 1 = 14 • Oct. Week 3 = 16 • Nov. Week 1 = 16 • Change Intervention? Add a phase line. • Nov. Week 3 = 20 • Dec. Week 1 = 28 • Dec. Week 3 = 40 • Is Tony on track now?

Examples from Cohort A Coaches What are some ways I can organize the progress monitoring data for my small groups? Examples from Cohort A Coaches

Third grade intensive progress monitoring All in 1st grade HM last year. Sept to Jan this year in Read Well, now in 2nd grade HM. John being tested for sped.

Progress Monitoring - ORF and RTF Oral Reading Fluency Progress Monitoring (with Retell Fluency) – Grades 1-3 Teacher: Progress Monitoring - ORF and RTF   ORF RTF Name Who Monitor M 1 M 2 A 1 A 2

Kindergarten Progress Monitoring Example Teacher: Progress Monitoring - ISF and PSF   ISF PSF Name Who Monitor S 1 O 1 O 2 N 1 N 2 D 1 J 1 F 1 F 2 M 1 M 2 A 1 A 2