1. Development of a Math Screening Assessment on a Districtwide Basis Washington Educational Research Association Annual Conference December 5-7 2007 Mike Jacobsen-Assessment and Curriculum Director Andy McGrath-Principal Glacier Middle School White River School District 360-829-3820 mjacobse@whiteriver.wednet.edu

2. By The End of This Presentation You Will: Understand how the district implements a K-10 CBM reading assessment system Understand how the WRSD developed a math screener –District-wide focus –Establish a committee –Pilot process –Full implementation –Fall, winter & spring data 06-07 –Next steps

3. Basic Definitions CBM=Curriculum Based Measurement –Developed Initially at University of Minnesota Institute for Research on Learning Disabilities –Measures students progress in basic skills using existing curriculum –Psychometrically sound ORF=Oral Reading Fluency –What is measured is students’ ability to read out loud, accurately and fluidly DIBELS=Dynamic Indicators of Basic Early Literacy Skills –Researchers from the University of Oregon coined the phrase

4. What is CBM? Standard, simple, short duration fluency measures of reading, spelling, written expression and mathematics computation –WRSD Reading CBM is very similar to DIBELS with one exception –WRSD Math Screener is different than DIBELS in math In reading CBM is oral reading fluency Measures “vital signs” of student achievement Academic thermometer

5. Big Ideas About CBM Extensive data supporting validity of use as a measure of basic skills Principle use is in formative evaluation Sensitive to changes in performance due to instruction Easy to use within classrooms Brief Repeatable

7. ORF and Other Reading Tests 1999- 3rd Grade Qualitative Reading Inventory to 3rd Grade ORF=.89 1999-3r Grade ITBS to 3rd Grade ORF=.64 1999-2th Grade Gates-MacGinitie to 2nd Grade ORF=.84 1999-3rd Grade Gates-MacGinitie to 3r Grade ORF=.77 1999-4th Grade Gates-MacGinitie to 4th Grade ORF=.64 1999-5th Grade Gates-MacGinitie to 5th Grade ORF=.86

8. ORF and WASL Relationships 1998- 4th Grade WASL to 5th Grade ORF=.70 1999-4th Grade WASL to 4th Grade ORF=.51 2003-6th Grade ORF to 7th Grade WASL=.68 2000-4th Grade ORF to 4th Grade WASL=.66 2002-4th Grade ORF to 4th Grade WASL=.65

9. ORF and WASL Relationships

10. CBM ORF/WASL

12. Why Assess Computational Fluency? “Many of the difficulties children have in arithmetic result from not understanding number ideas supposedly learning at an earlier time” –Engelhart, Ashlock & Wiebe, 1984 “In most cases the precision and fluency in the execution of the skills are the requisite vehicles to convey the conceptual understanding.” –H. Wu, 1999

13. White River School District Assessment Process Implemented during the 98-99 school year for K-6 Reading –6th-8th grade added 2002 –9th/10th grade added 2005 Implemented during the 2006-2007 school year for 1-10 Math screener Kindergarten students, initial sound fluency, letter names and segmenting phonemes Grades 1-10 orally read passages from appropriate grade level material Conducted three times per year during September, January and May

20. Background of Development of the Math Screener: District Learning Improvement Planning Established Fall of 2005 Approximately 30 members, teachers, building administrators, central office administrators, parents and school board members –Each building had a stipend position for a teacher who served as DLIP coordinator Met monthly during the 05/06 school year The first meeting was on structure and goals, research on effective schools and role of the district

21. Background: District Learning Improvement Planning Established Fall of 2005 Approximately 30 members, teachers, building administrators, central office administrators, parents and school board members –Each building had a stipend position for a teacher who served as DLIP coordinator Met monthly during the 05/06 school year The first meeting was on structure and goals, research on effective schools and role of the district

22. Background: District Learning Improvement Planning The second meeting focused on district-wide information using the data carousel format –WASL trend data-desegregated –ITBS –CBM –Demographics –Safe and Civil Surveys –Nine Characteristics –Healthy Youth Survey –Sports and Arts program participation –Curriculum alignment –Professional development

26. Background: District Learning Improvement Planning Used data from the carousel process to identify three major focus areas: –Professional development –Curriculum alignment-math –Math Each focus area had co-chairs Every member of the district learning improvement team was on one of the focus area committees Outcome oriented

27. Math Committee –District Math TOSA Kathie Ross and Andy McGrath Co-chaired the Math Committee – Goal: To produce a math assessment that will reliably predict a student’s success on the WASL (not diagnostic) To produce an assessment that can be given in 20-30 minutes and can be graded in a timely manner without added cost

28. Math Committee To Achieve This Goal: –We added teachers to the committee from each level primary, intermediate, middle and high school –Committee Makeup »3 Administrators »7 Teachers »1 Central Office »2 Parents

29. Math Committee ASSESSMENT DEVELOPMENT –Committee met for about 2 months discussing the makeup of the assessment: –Assessment Structure: 20 Total Questions 12 Computation 8 Applied Problems Single Number Answer –Reading fluency assessment already established in district –Reviewed Fuchs and Fuchs- Monitoring Basic Skills Progress - 2nd Ed. –Reviewed Ken Howell’s et all- Multilevel Academic Skills Inventory-Revised –Next step split subcommittee into three groups: Elementary Middle High School

30. Math Committee-Assessment Cont. The Groups using the GLEs as a guide developed a draft assessment for each grade level Assessments were brought back to full committee to be discussed and edited Developed assessments for grades 2 – 10 Assessment give three times a year in conjunction with reading assessment

31. Sample Page 4th Grade Computation

32. Sample Page 4th Grade Applied Problems

33. Pilot Process IMPLEMENTATION An assessment for each grade level completed by April 2005 Piloted last May with volunteer classrooms at least two per grade level Pilot results to Assessment Office analyze math assessment and reading fluency to see if this would be a good predictor of WASL success If the assessment proved to be an accurate predictor of WASL success then implement district wide Fall 06

34. Pilot Process Manila envelope provided to each pilot teacher –Directions for Administration –Instructions for Scoring –Student Response Sheets –Test Key Copies of student response forms provided to each teacher 624 students grades 1-8 participated Statistically strong relationships with WASL math and spring oral reading fluency demonstrated

35. Pilot Results