Progress Monitoring Goal Setting Overview of Measures Keith Drieberg, Director of Psychological Services John Oliveri, School Psychologist Cathleen Geraghty, School Psychologist

Overview Comparison of Terms Between Systems TermAIMSWEBDistrictGrade Level Frequency I/II/IIILevelsTiers III (Red Zone)Progress Monitor IntensiveInstructionalWeekly II (Yellow Zone)StrategicTargetedAssigned1 x Monthly I (Green Zone)BenchmarkSchool- wide Screenings Assigned3 Times a Year

General Education Screening and Progress Monitoring in General Education

Screening Screening allows for quick identification of students that are at-risk for academic difficulties Different screening measurements When to do screenings Who is screened?

Screening When and Why? 1st Screening - identification 2nd Screening - prediction and identification 3rd Screening - goal attainment

Which Measures Kindergarten –Fall - Letter Naming Fluency (LNF) –Winter - LNF & Phoneme Segmentation Fluency (PSF) –Spring - LNF & PSF 1st Grade –Fall - Nonsense Word Fluency (NWF) & Oral Reading Fluency (ORF) –Winter - NWF & ORF –Spring - NWF & ORF 2nd Grade –Fall - ORF –Winter - ORF Spring - ORF

Administration for Early Literacy

Letter Naming Fluency

Example: Letter Naming Fluency

Phoneme Segmentation Fluency

Example: Phoneme Segmentation Fluency

Nonsense Word Fluency

Example: Nonsense Word Fluency

Oral Reading Fluency

ORF (R-CBM) Administration Student’s read the passage aloud for 1 minute Number of words read correct and number of errors are counted –WRC/errors

Example: Oral Reading Fluency

Curriculum Based Measurement Reading Maze AreaTimingTest ArrangementsWhat is Scored? CBM Maze Reading 3 minutes Individual, Small Group, or Classroom Group # of Correct Answers CBM Maze is designed to provide educators a more complete picture of students’ reading skills, especially when comprehension problems are suspected.

An example of CBM Maze

Who is Screened? Ideally, all students are screened in the Fall, Winter, and Spring Students scoring below the 25th percentile are considered at-risk and should be progress monitored

What Is & Why Progress Monitor What is Progress Monitoring –Formative assessment tool Why Progress Monitor –Tell us whether students are profiting from the curriculum, and whether or not an intervention is effective for that particular student Some Everyday Examples of How We Use Informal Progress Monitoring

What is Progress Monitoring Technically adequate –reliability and validity Capacity to model growth –able to represent student achievement growth within and across academic years Treatment sensitivity –scores should change when students are learning Independence from specific instructional techniques –instructionally eclectic so the system can be used with any type of instruction or curriculum Capacity to inform teaching –should provide information to help teachers improve instruction Feasibility –must be doable

Why Progress Monitor Screening is not enough for some students because they may be in ineffective programs for too long. Progress monitoring allows for individualized goals to be written and determination of a feasible amount of time for the goal to be reached. Allows for an analysis of student need and resources for determining progress monitoring frequency. –Programs that are more intensive (e.g., special education), should monitor student outcomes more frequently that 3x per year.

More Frequent Evaluation

Progress Monitoring Plan –No Learning Center Monitor at-risk students, once a month at grade level –Learning Center Monitor at-risk students –once a month at grade level –every week at instructional level Collect data on Wednesday or Thursday

Goal Setting - General Each student should have a year-long goal (you want the student to be performing at the 50th percentile on grade level material) Every goal should have: –Time frame (when the goal should be reached) –Behavior (what the desired level of performance is) –Condition (which measure and where you obtained it from) –Criterion (which grade level passage you are using - if applicable)

Sample ORF Goal Oral Reading Fluency (Fluency) –In (#) of weeks (student name) will read (#) word correct in 1 minute as measured by a (grade ___ ) (AIMSWeb Oral Reading Fluency Measure).

Norms and Growth Rates

Year-Long Goal Expected weekly growth –If the student is not at-risk use the average rate of growth –If the student is at-risk use the ambitious rates of growth (e.g., below the 25th percentile) Multiply the growth rate by 36 (weeks of school) and add it to the baseline level of performance

Year-Long Goal Example In the fall, a 4th grade student’s instructional level is 2nd grade and the student reads 22 words correct per minute. Ambitious growth for 2nd grade is 2 words per week –22 + (2 x 36) = 94 –The year-long goal would be 94 words correct per minute

Short-Term Goal All students being progress monitored need shorter-term goals These goals should use ambitious rates of growth to help the students ‘catch-up’

Short-Term Goals Determine what level of performance is needed to move up a percentile rank (e.g., 10th to 25th; 25th to 50th) Take the difference in performance (e.g., how many words correct the student will need to move from the 10th to 25th percentile), and divide by the ambitious rate of growth –This product will be the approximate number of weeks needed for the student to reach the next percentile rank

Short-Term Goals If using early literacy measures (LNF, PSF, NWF) move up instructional level until the student has mastered that skill, then move up to the next skill

Short-Term Goals In the Winter, a 3rd grade student’s instructional level is 2nd grade and the student reads 29 words correct per minute. –The student is currently performing at the 10th percentile. The short term goal should be at the 25th percentile. Difference between the 25th and 10th percentile at the 2nd grade level is 24. Divide 24 by the ambitious growth rate (2.0) and you get 12. That is the number of weeks it should take the student to move from the 10th to the 25th percentile.

Charting Progress Once you set the goal, you need to chart progress to see if the student is responding to the change in instruction

Evaluating Goal Attainment Process of assessing student achievement during instruction to determine whether an instructional program is effective for individual students.Process of assessing student achievement during instruction to determine whether an instructional program is effective for individual students. –When students are progressing, keep using your instructional programs. –When tests show that students are not progressing, you can change your instructional programs in meaningful ways. –Has been linked to important gains in student achievement (Fuchs, 1986) with effect sizes of.7 and greater.

Trendline Goal Line

Trendline

Goal line Trend Line

Special Education How can Progress Monitoring be used in Special Education?

Educational Benefit Progress monitor at both instructional and grade level –Grade Level Goals How is the student performing relative to his/her peers? –Instructional Level Goals How is the student’s performance changing as a function of the current instruction / intervention? –Instructional level progress monitoring is sensitive to growth.

General Goal Setting Long-Term Goal –The long-term goal should be either to move the student: Up a full grade level (Ex. 50 th percentile on a 1 st grade probe to 50 th percentile on a 2 nd grade probe) Up to a higher skill (Ex. Letter Naming Fluency to Letter Sound Fluency)

General Goal Setting (continued) Short-Term Goals (Benchmark 1 & 2) –The benchmark goals should be to move the student up a percentile level within their instructional level (could be the same as grade level). (Ex. Move from the 25 th to the 50 th percentile).

Review/Overview Comparison of Terms Between Systems Term AIMSWEB District Grade Level Frequency I/II/IIILevelsTiers III (Red Zone) Progress Monitor Intensive Instructional Weekly II (Yellow Zone) StrategicTargetedAssigned 1 x Monthly I (Green Zone) BenchmarkSchool- wide Screenings Assigned 3 Times a Year

First Example (Topic Two ) The Movie “City Slickers” The Cattle Drive----

That Great Line: “Don’t Know Where We Are At; Don’t Know Where We Are Going; But We Are Sure Are Making A Lot Of Progress”

That Great Line: A Case For Level I Bench Mark Screenings At Assigned Grade Levels For All Students Don’t Know Where We Are At No Base Line Point Don’t Know Where We Are Going No Goal Point Not Sure Of Progress No Goal Line or Aimline

What We Already Know : A Case For Level II Strategic/ Targeted Assigned Grade Level Progress Monitoring On a Regular Basis The Earlier We Start Interventions; The Less Behind the Student is in Comparison to Their Grade Level Peers; and The Sooner A Student Starts to Makes Progress Their Peer Growth Rate Is High And Student “Catches Up Quickly” (within about a year) Because They are Not Too Far Behind (Assigned Grade Level Progress Monitoring / Strategic/Targeted – ideal 1 x Monthly)

What We Need to Think More About: A Case For Level III Progress Monitor/Intensive Instructional Level Progress Monitoring Higher Grade Levels (Especially 4 th Grade +) Student Significantly Farther Behind Assigned Grade Level Peers Need Long Term Goals (Takes Longer Than One Year to “Catch Up”) Kids Give Up On Themselves (Highest Drop Out Rate 9 th Grade) We Expect Less of Students at Higher Grade Levels and Blame Others (Instructional Grade Level Progress Monitoring To Know If Intervention is Workng; Appreciate Progress Sooner; and Receive Frequent Feedback )

What We Need to Think More About: Instructional Grade Level Progress Monitoring Higher Grade Levels (Especially 4 th Grade +) Student Significantly Farther Behind Assigned Grade Level Peers Need Long Term Goals (Takes Longer Than One Year to “Catch Up”) In Addition, Peer Growth Rate Is Slower at Higher Grade Levels If We and the Student Continue Ambitious Goals Each Year The Student Will Make Significant Gains and Eventually “Catch Up”- (Instructional Grade Level Progress Monitoring)

Why We Need Both Assigned Grade Level and Instructional Grade Level Goals and Progress Monitoring 6 th Grade90 WC 7 th Grade120 WC 8 th Grade150 WC