For Each Student Instruction GoalsAssessment For All Students OR Reading First: Review of Comprehensive Programs

2 Objective of Reading First (1) “To provide assistance to State educational agencies and local educational agencies in establishing reading programs for students in kindergarten through grade 3 that are based on scientifically based reading research to ensure that every student can read at grade level or above not later than the end of grade 3.”  NCLB, 2001, Part B, Sec

3 Why Focus on Reading Programs Now  Unprecedented convergence about what children need to be successful readers  National syntheses provide scientific evidence on which to base practice  Much classroom practice is shaped by textbooks  State standards are embracing the science  Publishers respond to the marketplace and need  Window of opportunity to align what we know, what we use, and how we teach to attain critical results

4 Comprehensive Reading Programs  Purpose:  to provide sufficient instruction in the core components of reading (enough of the “right stuff” in a systematic design)  to provide instruction that enables the majority of students to meet or exceed grade-level standards on all the key Reading First elements  to serve as the primary reading program for a school within and between grades (K-3)

5 Why Adopt A Comprehensive Reading Program?  Increases continuity, coherence, and community of effort within and between grades (all teachers are aware and working toward the same goals)  Creates more “buying power” regarding professional development  Affords greater differentiation of instruction for children (can share children within and between grades)  Reading instruction is rocket science (Moats, 1999) and expecting teachers to construct and instruct is unreasonable and too important to leave to chance.

6 Oregon Reading First Curriculum Review  Focus: Comprehensive reading programs  Purpose: To determine alignment with SBRR & provide consumers guidance  Curriculum Review Panel  Tool/Criteria: Consumer’s Guide (Simmons & Kame’enui)  Review Process

7 Curriculum Review Panel: Selection of Members  Former State Superintendent of Public Instruction, Stan Bunn, invited 25 educators across the state to serve on the Curriculum Review Panel.  5 additional members from the U of O, Eugene Research Institute, and Oregon Research Institute and 3 members from ODE were identified in Oregon’s Reading First grant.

8 Curriculum Review Panel: 17 final members  2 district administrators  2 district curriculum specialists  1 classroom teacher  4 university faculty from three state universities (U of O, Pacific, Univ. of Portland)  2 doctoral students from the U of O  4 researchers from ERI  1 researcher from ORI  1 educational specialist from ODE

9 Oregon Reading First Curriculum Review Panel Members  Julie Anderson Educational Specialist/English Language Arts, ODE  Scott Baker, Ph.D Researcher, Eugene Research Institute  Lynette Doht Reading Specialist, Portland Public Schools  Hank Fien Research Assistant, University of Oregon  Barbara Gunn, Ph.D Researcher, Oregon Research Institute  Arlene Hett, Ph.D Director of Teacher Education, University of Portland  Sara Johnson Principal, Henry L. Slater Elementary School  Edward J. Kame’enui, Ph.D Professor, University of Oregon  Kristen MacConnell Research Assistant, University of Oregon  Anita McClain, Ph. D Professor, Pacific University  Janet Otterstedt Research Assistant, Eugene Research Institute  Sandra Pellens, Ph.D Director of Instruction, Molalla River SD 35  Michael Rebar, Ph.D Researcher, Eugene Research Instiitute  Deborah C. Simmons, Ph.D Professor, University of Oregon  Sylvia Smith, Ph.D Researcher, Eugene Research Institute  Carrie Thomas-Beck, Ph.D Curriculum Specialist, Springfield SD 19  Deborah White Teacher of the Year, Lyle Elementary School

10 TOOL: Consumer’s Guide to Evaluating a Core Reading Program  Developers: Drs. Deborah C. Simmons and Edward J. Kame’enui, University of Oregon  Why Developed: To assist states, districts and schools in the selection of research-based tools  When Developed: As part of National Center to Improve the Tools of Educators’ scope of work ( )  Purpose: To document and quantify the design and delivery features of core reading programs.

11 Consumer’s Guide: Organization  Programs Evaluated by Grade  Within Grade by Essential Component:  Phonemic Awareness  Phonics  Fluency  Vocabulary  Comprehension

12 Essential Components by Grade K123 PANA Phonics FluencyNA Vocab. Compre- hension

13 Consumer’s Guide: Organization (cont.)  For each Essential Component:  High Priority Items  Discretionary Items  Overarching Design Items for Each Grade

14 Consumer’s Guide: Sample Items  KINDERGARTEN  Phonemic Awareness  High Priority Items:  #1 Progresses from the easier phonemic awareness activities to more difficult (e.g., isolation, blending, segmentation, and manipulation) (ss)  #2 Teaches skills explicitly and systematically (w)  #4 Integrates letter-sound correspondence instruction to phonological awareness (w)  #5 Focuses on segmentation or the combination of blending and segmenting for greatest transfer (ss)

15 Consumer’s Guide: Sample Items  KINDERGARTEN  Phonemic Awareness  Discretionary Items:  #1 Focuses beginning instruction on the phonemic level of phonological units with short words (two to three phonemes; e.g., at, mud, run)  #3 Focuses first on the initial sound (sat), then on the final sound (sat), and lastly on the medial sound (sat) in words.

16 Consumer’s Guide: Sample Items  KINDERGARTEN  Design Features  1. Coordinates and integrates phonemic awareness and phonics instruction and student materials.  2. Provides ample practice on high-priority skills.  3. Provides explicit and systematic instruction.  4. Includes systematic and cumulative review of high priority skills.  5. Demonstrates and builds relationships between fundamental skills leading to higher order skills.

17 Scoring Criteria Use the following criteria to score each item: = Program consistently meets/exceeds criterion = Program partially meets/exceeds criterion = Program does not satisfy the criterion When evaluating individual elements, slash ( / ) the respective circle that represents your rating (e.g., ).

18 Analysis of High Priority Items

19 Within a Sequence of Lessons Analysis (W) High Priority Items - Phonemic Awareness Instruction - K Rating Criterion Evidence InitialWeek __ (Unit 4) Week __ (Unit 11) 2.Teaches skills explicitly and systematically. (w) Unit 2, Day 2: Introduce /m/ + 1st sound isolation Day 3 /m/ pictures and objects - 1st sound Unit 4, Day 3: Auditory Blending Also see Days 4 and 5. Unit 11, Day 2: isolate final phoneme. Day 3: final phoneme Day 4: Final phoneme

20 Scope and Sequence Analysis (SS) High Priority Items - Phonemic Awareness Instruction - K RatingCriterion Evidence Initial Instruction Week __ (Unit 4) Week __ (Unit 8/9) 1.Progresses from the easier phonemic awareness activities to the more difficul t. (ss) Unit 2, Day 2 & Day 3 Isolate initial /m/. Day 4 Isolate initial /a/. Day 3: Intro auditory blending Day 5: Practice auditory blending Day 7: auditory blending Day 4: Segmentation Day 9: 1st sound manipulation + segmentation Unit 9: Day 4: segmentation

21 Skills Trace of Letter-Sound Review (ST) - K Sequence, Instruction, Review 1 - Lesson/Day 2 - Lesson/Day 3 - Lesson/Day 4 - Lesson/Day 5 - Lesson/Day Day or Lesson Unit 5, Day 2Unit 5, Day 3Unit 5, Day 4Unit 5, Day 5Unit 5, Day 6 Instructionjyqznone Review Cycle Ss review all sounds. j (X2) + Ss review all sounds. j, y d, j, n, s, y all sounds q / k, p, r, v/ q / all sounds k, p, r, v, g z q and z Sequence, Instruction, Review 6 - Lesson/Day 7 - Lesson/Day 8 - Lesson/Day 9 - Lesson/Day 10 - Lesson/Day Day or Lesson Unit 5, Day 7Unit 5, Day 8Unit 5, Day 9Unit 5, Day 10 Unit 6, Day 1 Instructionvxnone Review Cycle all soundsj / o, b, v / v / all sounds x all sounds g, h, w, c, p, l

22 Review Process  Announcement posted on Oregon Reading First and WOATRA’s websites inviting publishers to submit core reading programs for review. (February 14, 2003 deadline for submissions)

23 Call for Comprehensive Programs Comprehensive Programs:  (a) include materials for all grades K-3,  (b) comprehensively address the “five essential components” of the Reading first legislation in scientifically based beginning reading instruction: phonemic awareness, phonics, reading fluency, vocabulary, and reading comprehension.

24 Comprehensive Programs Reviewed:  15 programs submitted / 9 reviewed as comprehensive programs:  Harcourt  Houghton-Mifflin  Macmillan/McGraw-Hill  Open Court  Reading Mastery  Rigby  Scott Foresman  Success For All  Wright Group

25 Review Process: Program Assignment  OR CRP members provided with 6 hours of training by OR RF staff (February 21, 2003)  CRP members randomly assigned to programs to review.  Each reviewer assigned to either a K/1 or 2/3 grade. No reviewer evaluated an entire program K-3.  Each member reviewed 1 to 4 programs based on availability.  CRP members were not permitted to review any program for which they were an author, consultant, or advisor. (Reviewers signed statements of disclosure.)  Initial review completed March to May, 2003.

26 Review Process:  Each section of a program (K/1 or 2/3) was reviewed by 2 independent reviewers.  The same two reviewers were never paired more than once.  Thus, each program was reviewed by 4 different members of the review panel.  Members spent from 8 to 30 hours to complete a program assignment (K/1 or 2/3)  All review work was completed at the OR Reading First Center.

27 Review Process: Reliability Between Reviewers  Upon completion of the review, scores between the two reviewers were compared.  Each rating was assigned a point value:  = 2 points  = 1 point  = 0 points  Items that were off by one were averaged (e.g. full circle and partial circle = 1.5)  For items that were off by more than one (e.g. full circle and empty circle), a third reviewer was asked to reconcile the items.

28 Review Process: Third Reviewer  Third reviewers assigned to “reconcile” did not conduct the initial review of the program.  Third reviewers examined the documented evidence from the 2 initial reviewers and the program materials to determine which of the two scores best represented the selected item as the final score.

29 Review Process: Summarizing Results  Result of the review process is one averaged/ rectified score for each item for a program.  Final Report includes a completed Consumer’s Guide for each program (item by item).  Final Report summarizes results by program, by essential components, and by grade.  For each program, results are summarized by high priority, discretionary, and design items.

30 Final Report: Sample Consumer’s Guide High Priority Items - Phonemic Awareness Instruction RatingCriterion 1. Progresses from the easier phonemic awareness activities to the more difficult (e.g., isolation, blending, segmentation,and manipulation).(ss) + 2. Teaches skills explicitly and systematically. (w) 3. Models phonemic awareness tasks and responses orally and follows with students’ production of the task. (w) 4. Integrates letter-sound correspondence instruction to phonological awareness. (w) [NRP, pg. 2-41] + 5. Focuses on segmentation or the combination of blending and segmenting for greatest transfer. (ss) [NRP, pg 2-41]

31 Final Report: Sample Consumer’s Guide (cont.)  Kindergarten Phonemic Awareness Instruction - High Priority  Tally the number of elements with each rating.  __ + __  (2 pts) (1.5 pts) (1 pt) (.5 pts) (0 pts)  Total Points/Total Possible Points 8/10 = 80%

32 Sample: Summary of Kindergarten Ratings High Priority Items Phonemic Awareness Instruction (5) 80% Phonics Instruction (9) Letter-Sound Association Instruction(3) Decoding Instruction (5) Irregular Words Instructions (1) 75% Vocabulary Instruction (3) 0% Comprehension Instruction (4) 38% Kindergarten High Priority Totals 58%

33 Next Steps  Final report distributed to districts, schools, and publishers for guidance in program selection at IBR I in June  Final report discussed in detail on Day 3 of IBR I.  Programs will be on display throughout the four days of IBR I for school staff to examine.  Schools will need to allot their professional development funds to pay a team of educators to examine the programs more closely during the last week of June.

34 Next Steps (cont.)  Schools will need to notify ODE of their program selection by early July to secure approval then order materials.  Schools will select those programs that were reviewed favorably by Oregon’s RF Curriculum Review Panel.  Schools will then need to arrange for professional development on their new program early in the fall.