Program Evaluation MAASE: Leading By Convening PREZI and ”Worksheet” on PLAAFP is available

MAASE – Four Focus Areas High Quality Student Evaluations Specially Designed Instruction (SDI) Skilled Staff Program Evaluation is the use of research-based methods to systematically investigate the effectiveness of programs/services, to inform decision-making, and to ultimately improve educational outcomes.  Program evaluation entails asking questions in one or more of four different domains: (1) program/service need and design; (2) program implementation; (3) program impact; (4) and program efficiency.

BLOG – Don’t Believe Everything on the Internet Why is it Failing? Schools are using RTI “as a kind of general education substitution for special education.” This was the concern we highlighted in our earlier article on special education gatekeeping. Schools are not adequately evaluating students for learning disabilities before initiating an RTI program. Many schools don’t perform any evaluations prior to RTI and therefore don’t know if the interventions they are using are even suitable for the students they are attempting to help. Schools implementing RTI are not clearly separating the broader goals of general education instruction and the more narrowly focused goals of RTI instruction, implying a confusion as to what the program is actually trying to achieve. The RTI instruction in the study was found to be rigid and standardized for all students. In looking at RTI for reading, for example, the study found that the instruction focused on foundational skills like phonics and not reading comprehension, regardless of the individual student’s needs. In short, RTI, for all its good intentions, is a only a theory without empirical validation.

Evaluation of Response to Intervention Practices for Elementary School Reading 308 Pages

How to Interpret the Impact Findings and How This Study Differs from Prior Literature  The study uses a Regression Discontinuity (RD) design for its impact estimation. While this design demonstrates a causal relationship between assignment to receive intervention services and reading test outcomes in the impact sample, it also requires caution when interpreting the impact findings. In particular, the RD design estimates the impact of assignment to intervention by comparing outcomes of students just above or just below the cut point. Findings based on this design, therefore, cannot be generalized to all students receiving intervention services.  …..In this sense, these results are relevant for decisions about expanding or reducing the scope of intervention but not, necessarily, for decisions about offering or not offering intervention. It would be misleading to conclude from these findings that providing increasing intensity of services to the students most at risk (for example, students whose screening test scores are far below the cut point) is inappropriate or ineffective.  In addition, this study is unique in the sense that it examines the RtI system as it operated in multiple states in a large sample of experienced schools that had implemented RtI on their own, without monitoring or support from researchers.  Unexplored but plausible factors that may be related to negative impacts of assignment to intervention on some Grade 1 students include (1) false or in-correct identification of students for intervention, (2) mismatch between reading intervention and the instructional needs of students near the cut point, and (3) poor alignment between reading intervention and core reading instruction. 

Mike Mattos – email response Solution Tree (Simplifying RtI) When research and evidence as the criteria, we have a professional obligation to take a comprehensive look at the research on the topic, and not make final decisions on a single study.  Professional practice and common sense would conclude that a meta-analysis of the research on RTI would be more dependable information to make a professional decision.  In his book Visible Learning for Teachers (2012), John Hattie’s  completed such a study.  In his finding—which are based on over eighty thousand studies and one hundred million students worldwide—he found that RTI ranks second on his list of the most effective influences (inside or outside of school) that can increase student achievement. When implemented well, RTI has the power to help students improve multiple grade levels in a year, with an exceptional 1.07 standard-deviation growth rate in a year.   This is why a vast majority of our nation’s professional educational organizations endorse RTI as best practice.

MAASE - Program Evaluation program/service need and design; WHY - Mission/Values/Key Principles program implementation; How will you determine your program is using research based best practices and that they are implemented with fidelity? program impact; Outcome Data? What are your indicators? and program efficiency. Effective use of available resources

START Effective Practices Assessment Tool (EPAT) for Young Children with Autism Spectrum Disorders

Early On® MISSION AND KEY PRINCIPLES FOR PROVIDING EARLY INTERVENTION SERVICES IN NATURAL ENVIRONMENTS Infants and toddlers learn best through everyday experiences and interactions with familiar people in familiar contexts. All families, with the necessary supports and resources, can enhance their children’s learning and development. The primary role of a service provider in early intervention is to work with and support family members and caregivers in children’s lives. The early intervention process, from initial contacts through transition, must be dynamic and individualized to reflect the child’s and family members’ preferences, learning styles and cultural beliefs. IFSP outcomes must be functional and based on children’s and families’ needs and family- identified priorities. The family’s priorities, needs and interests are addressed most appropriately by a primary provider who represents and receives team and community support. Interventions with young children and family members must be based on explicit principles, validated practices, best available research, and relevant laws and regulations.

Discussion What tools/rubrics are using for Program Evaluation in: Resource Room ECSE Center Based Programs Transition Programs RtI / MTSS What are the overlapping items in every Program? Effective Instruction – good instruction is good instruction Outcome? Resource: A collection of Program Evaluation Tools

Contact Information ”Notecards” Paul Dymowski Allegan Area Educational Service Agency pdymowski@alleganaesa.org