Reading First Supplemental Review June 1-4, 2004 Dr. Robin G. Jarvis, Director Division of School Standards, Accountability, and Assistance

Objective of Reading First “To provide assistance to state educational agencies and local educational agencies in establishing reading programs for students in kindergarten through grade 3 that are based on scientifically based reading research to ensure that every student can read at grade level or above not later than the end of grade 3.” NCLB, 2001, Part B, Sec. 1201

Reading First Professional Development Must Teach Consumers How to Evaluate the Sufficiency of Programs 3 Core Reading Programs Supplemental Reading Programs Intervention Reading Programs

Instructional Needs You Might Expect in Schools Core - 80% Intervention - 5% Supplemental - 15%

Knowledge of How To Evaluate Evidence-Based Curriculum Materials  A core instructional program of validated efficacy adopted and implemented school wide  Programs and materials that teach enough of the critical elements  Selection of research-based supplemental and intervention programs  Programs implemented with high fidelity Adoption and Implementation of Research-Based Reading Programs that Support the Full Range of Learners

Understanding the Purpose of Different Programs Programs are tools that are implemented by teachers to ensure that children learn enough on time. Classifying Reading Programs: What is the purpose of the program? 1. Core 2. Supplemental 3. Intervention Core Reading Program Meeting the needs for most Supplemental Reading Program Core Supplemental Intervention Reading Program Supporting the CoreMeeting the needs for each (Vaughn et al. 2001)

A Core Program Is… …the “base” reading program designed to provide instruction on the essential areas of reading for the majority of students schoolwide. In general, the core program should enable 80% or more of students to attain schoolwide reading goals.

Evaluating Core Programs: Identifying Gaps One size does not fit all! We may need to supplement or provide intervention, but we must do so based on what research tells us.

Supplemental and Intervention Reading Programs: Support and extend the critical elements of core reading program. Provide additional instruction in one or two areas (e.g., phonological awareness, fluency). Provide more instruction or practice in particular area(s) of need.

 Designed for children who demonstrate reading difficulty and are performing below grade level.  Provide more explicit, systematic instruction to accelerate learning and bring the learner to grade-level performance.  Specialized, intense, and typically delivered in small group settings. Supplemental and Intervention Reading Programs: (Continued)

 May focus on more than one area (e.g., phonics, fluency, and comprehension). May focus exclusively on one essential reading area. Allow teachers to meet the needs of individual students who are struggling in their classrooms. Supplemental and Intervention Reading Programs: (Continued)

A Consumer’s Guide to Evaluating Supplemental and Intervention Reading Programs Grades K-3: A Critical Elements Analysis Deborah C. Simmons, Ph.D., Edward J. Kame’enui, Ph.D., Carrie Thomas Beck, Ph.D., Nicole Sherman Brewer, and Hank Fien Oregon Reading First Center, College of Education, University of Oregon

A Consumer’s Guide to Evaluating Supplemental and Intervention Reading Programs Grades K-3: A Critical Elements Analysis (October 2003) ORGANIZATION: by essential component items generally ordered from K to 3 appropriate grades marked with an X two columns for documentation Initial Instruction Additional Evidence summary and comment pages following each essential component

Breakdown by Essential Component: Phonics - 10 items Phonemic Awareness - Decoding - 13 items Irregular Words - 6 items Fluency - 12 items Vocabulary - 10 items Comprehension - 10 items

Sample Page:

Scoring Criteria: Reviewers will use the following criteria to score each item: = Program consistently meets/exceeds criterion = Program partially meets/exceeds criterion = Program does not satisfy the criterion NOTE: When evaluating individual elements, a slash ( / ) will be made over the respective circle that represents the reviewer rating.

Scoring Guidelines for Reviewers: 1. Identify targeted essential components and grade levels. 2. Complete only those items that evaluate the targeted components and grades.

3. Mark one program rating per relevant item for multi-grade programs that instruct students across grade levels (e.g., a K-1 program). Scoring Guidelines for Reviewers: (Continued)

4. Review and assign a separate score for each grade for each item for grade-specific programs. Scoring Guidelines for Reviewers: (Continued)

Program Assignment: n Each reviewer will evaluate all levels of a program (e.g., K, 1, 2, and 3) based on literacy component. n Each reviewer will work as a team to reach consensus on final scores on the Consumer’s Guide to Evaluating Supplemental and Intervention Reading Programs Grades K-3: A Critical Elements Analysis (October 2003).

Upon completion of the review program, scores will be assigned a point value: = 2 points = 1 point = 0 points Scoring Procedures for Reviewers:

Summarizing Results Result of the review process is one averaged/ rectified score for each item for a program. Final Report includes a completed Consumer’s Guide for each program (item by item). Final Report summarizes results by program and by essential components.

Program Summary KindergartenFirst GradeSecond GradeThird Grade Phonemic Awareness 35% Phonics22% Fluency

RatingCriterionGrade K Teaches skills explicitly. (w)X 2. Models phonemic awareness tasks and responses orally and follows with students production of the task. (w) X +3. Progresses from the easier phonemic awareness activities to the more difficult (e.g., isolation, blending, segmentation, and manipulation). X 4. Incorporates letters into phonemic awareness activities [NRP, pg 2-41]. (w)X 5. Makes students’ cognitive manipulations of sounds overt by using auditory cues or manipulatives that signal the movement of one sound to the next. (w) X +6. Analyzes words at the phoneme level (e.g., working with individual sounds within words). (ss) X 7. In K, focus is on first the initial sound, then on final sound, and lastly on the medial sound in words. In grade 1, focus is on phonemes in all positions. X 8. Focuses beginning phonemic level instruction on short words (two to three phonemes). (ss) X 9. Works with increasingly longer words and expands beyond consonant-vowel- consonant words to more complex phonemic structures (consonant blends). (ss) X +10. Focuses appropriate amount of daily time on blending, segmenting, and manipulation tasks until proficient [NRP, pg. 2-41]. (w) X Program Summary: Phonemic Awareness Sample

Percentage Summaries for Vocabulary and Comprehension?  Louisiana Reading First will follow recommendations from The University of Oregon and NOT report percentage scores for vocabulary and comprehension.  Total raw scores will be used to make recommendations from reviewers.

“Lack of convergence in the NRP report on vocabulary and reading comprehension (i.e., NRP not able to conduct a meta- analysis because of limited studies) makes it very difficult to argue convincingly and to justify a quantification using percentage scores that ostensibly results in a rank order of program design and quality.” - University of Oregon

Supplemental Reading Program Review June 1- June 4, 2004 Baton Rouge Sheraton Hotel and Convention Center 15 Reviewers 3 Teams –Phonological Awareness & Phonics –Vocabulary & Comprehension –Fluency

Meeting Details:  Three rooms will be set up for each team.  Reviewers and tables will be grouped by teams.  A Consumer’s Guide… and other recording materials will be provided at each group table.

Meeting Details: (Continued)  Vendor materials will be placed on tables by teams.  Vendor seating will be in back of the room.

Meeting Details: (Continued)  Public minutes will be posted daily.  Monitoring of reviewers and vendors will be conducted by LDE staff.

Preparations for the Review All reviewers have knowledge of :  Reading First  Scientifically based reading research  K-3 core, supplemental, and intervention reading programs

All Reviewers Will:  Represent various positions in education.  Not be connected to vendors.

 Receive training on how to use A Consumer’s Guide to Evaluating Supplemental and Intervention Reading Programs Grades K-3: A Critical Elements Analysis (October 2003). All Reviewers Will: (Continued)

Review Assurances: Each reviewer must sign assurances that they:  Do not stand to gain financially from any vendor as a result of service on the review team.  Are not an author of a commercial core, supplemental, or intervention program.

 Agree not to communicate with any reading vendors prior to and during the week of review, and ensuing times until schools/districts have had an opportunity to review the reading programs. Reviewer Assurances: (Continued)

Reviewer Pool:  National Reading Consultants  Louisiana Reading Leadership Team members  Louisiana Higher Education

 In-state school level personnel  Harvard Literacy Task Force members  In-state district level personnel Reviewer Pool: (Continued)

“The List”:  The list will be grouped based on component, rather than by grade level.  A list for all five essential elements of effective reading instruction will be decided upon by the supplemental review committee.

“The List”: (Continued)  Evaluation will be based on raw scores from the consumer’s guide.  List will be used by Reading First districts to purchase supplemental and intervention reading programs