1. RTI Professional Learning Webinar Series: Using Research to Select and Design Effective Interventions. 1 Session #7 11/19/09 3-4:30 pm Putting Research to Use: Reviewing Interventions Session #7 11/19/09 3-4:30 pm Putting Research to Use: Reviewing Interventions

2. Counting ALL Participants After you enter your profile information to join the webinar as usual: 1.Go to the Chat Window 2.Type your district name and school name or organization name or “parent” 3.Type the names of every person in attendance with you 4.Send it to “This Room” 2

3. Who are we?  Teachers  Administrators  Counselors or Social Workers  County Office or DOE  Parents  PK-5, 6-8, 9-12, postsecondary  Statewide Organizations  Other? 3

4. Welcome and Introductions  John Wight, GaDOE  Dr. Kimberly Anderson, REL-SE at SERVE Center at UNCG 4

5. 5 REGIONAL EDUCATION LABORATORY- SOUTHEAST (REL-SE) Executive Director Dr. Ludwig van Broekhuizen 800-755-3277 www.serve.orgwww.serve.org Operated by the SERVE Center at UNCG Serving Alabama, Florida, Georgia, Mississippi, North Carolina, and South Carolina

6. 6 The Regional Educational Laboratory (REL) System

7. REL-SE MISSION To serve the educational needs of the Southeast using applied research, development, dissemination, and training and technical assistance, to bring the latest and best research and proven practices into school improvement efforts. 7

8. REL-SE Services 1.Outreach and dissemination of research, evaluation, and policy information 2.Technical assistance to SEAs and LEAs 3.“Issues & Answers” publications 4.Randomized Controlled Trial experimental studies on interventions of relevance to our region AMSTI/Alabama and K-PAVE/Mississippi 8

9. Series Framework: EBDM Cycle 9

10. 10 WEBINAR SERIES OVERVIEW SESSION 1 SESSION 2 SESSION 3 SESSION 4 Series Introduction RTI Overview INTRODUCTION Understand Scientifically Based Research and the Evidence- Based Decision- Making Cycle SBR & EBDM Understand methodologies used in ed. research, and their strengths and weaknesses for determining “what works” TYPES OF RESEARCH METHODS Understand how to assess the rigor and findings of studies Practice using the Critical Reading Protocol CRITICAL READING PROTOCOL FOR STUDIES ABOUT INTERVENTIONS

11. IES Practice Guide RTI Math IES Practice Guide RTI Reading STATE OF RESEARCH ON RTI Determine the desired outcomes, program features, implementation issues, and extent of evidence on an intervention. INTERVENTION REVIEW PROTOCOL Learn what fidelity of implementation is and how to monitor it effectively. FIDELITY OF RTI IMPLEMENTATION Learn from colleagues in GA how they put the concepts and tools from this series into action. LOOKING FORWARD: PROMISING PROGRESS Webinar Series Overview sessions 5 & 6 session 7 session 8 session 9 11

12. TODAY’S OBJECTIVES 1. Reach back to key information from Sessions 3-6 2. Understand how to determine if an intervention meets your students’ and school’s needs. + Practice using the Protocol with an intervention relevant to GA 3. Debrief the Intervention Review Protocol process and discuss the relevance of this type of process for RTI decision making 4. Where to find expert reviews of effective interventions and strategies 5. Preview Session 8 12 Types of Research Methods & Critical Reading Protocol for Studies About Interventions posted on GaDOE website Intervention Review Protocol & 5 Cognitive Tutor documents posted on GaDOE website

13. Objective #1 Reach back to key information from Sessions 3-6  Types of Research Methods  Critical Reading Protocol for Studies About Interventions Both are posted on GaDOE website 13

14. Session 3: We discussed 5 types of research methods. What stood out, & what questions remain? 1.Descriptive 2.Correlational 3.Quasi-Experimental 4.Experimental 5.Meta-Analysis 2 min. Chat Facilitators lead their groups Individuals utilize chat box 14

15. Quick Poll 1: Types of Research Methods Studies using which method can make claims that an educational intervention caused changes in student achievement? A.Descriptive B.Correlational C.Quasi-Experimental D.Experimental E.Meta-Analysis 15

16. Quick Poll 2: Types of Research Methods Studies using which method can make claims that relationships were found between an intervention and changes in student achievement? A.Descriptive B.Correlational C.Quasi-Experimental D.Experimental E.Meta-Analysis 16

17. Quick Poll 3: Types of Research Methods What is the key difference between quasi-experimental (Q-E) and experimental (E) studies? A.E utilizes random assignment into “treatment” vs. “control” groups; Q-E does not B.Q-E has threat of selection bias; E does not C.Q-E allows you to study an “intervention” and a “comparison” group already in process; E requires groups to be set up prior to study D. E can claim causality; Q-E cannot E. All of the above 17

18. Session 4: Critical Reading Protocol for Studies about Interventions 1.Which of the 5 key questions in the protocol was most informative for you? 2.What was a “takeaway” for your RTI decision making about interventions that you had from our training on this protocol? 2 min. Chat Facilitators lead their groups Individuals utilize chat box 18

19. Sessions 5 & 6: IES Practice Guides on RTI math & reading 1.What were some things that stood out to you from those Practice Guide presentations? 2.What questions remain? 2 min. Chat Facilitators lead their groups Individuals utilize chat box 19

20. Objective #2 puts together all the knowledge and skills we have built so far Intervention Review Protocol: Understand how to d etermine the desired outcomes, program features, implementation issues, and extent of evidence of an intervention, to determine if it meets your students’ and school’s needs. + Practice using the Protocol with an intervention relevant to GA 20

21. Intervention Review 1.Gather all available info on an intervention’s desired outcomes, features, implementation issues, and evidence of effectiveness>>>make a “portfolio.” 2. Decision-making team utilizes the Intervention Review Protocol (IRP) to review the information in the portfolio. 3. This will help you determine if you should utilize an intervention or strategy for RTI. *the IRP is in the GaDOE RTI manual 21

22. Intervention Review Protocol Desired Outcomes: What are the goals of this intervention? How well do those match with your students’ needs? Program Features: What are the core features of the intervention? Implementation Issues: As you reviewed the information, did any implementation challenges emerge? Extent of the evidence: Are there any studies that used a strong design to determine the intervention’s impact? (*use Critical Reading Protocol For Studies on Interventions, & Types of Research Methods handouts for this) + side 2 of IRP 22

23. IRP Process: Practice With Cognitive Tutor documents posted on GaDOE website  Sample intervention review portfolio: 1.Cognitive Tutor “info. chart” 2.Morgan study 3.Sarkis study 4.Carnegie Learning/Colony High School study 5.WWC Topic Report--Middle School Math 23

24. . 1.Team splits up & reviews resources  Which section/s in the IRP does your resource address?  What do you find regarding the questions in that section/s? 2.Team regroups to discuss  What were the answers to the questions in each IRP section?  So, is this intervention right for us? Is there any additional info we need? 24

25. Objective #3 Debrief the Intervention Review Protocol process and discuss the relevance of this type of process for RTI decision making 25

26. Debrief the IRP Process and Relevance for RTI 1.How did this IRP process compare to the way you normally review interventions? 2.Think of a time when you could have used the IRP to select an intervention/strategy 3.What are the opportunities and challenges in using this process in RTI across GA? 3 min. Chat Facilitators lead their groups Individuals utilize chat box 26

27. Objective #4 As we prepare intervention review portfolios, where can we find expert reviews of effective interventions and strategies? 27

28. Good Sources What Works Clearinghouse: Practice Guides, Topic and Intervention reports, Doing What Works http://ies.ed.gov/ncee/wwc/ Best Evidence Encyclopedia http://www.bestevidence.org/ Center on Instruction http://www.centeroninstruction.org/ Library Search Engines--Galileo http://www.galileo.usg.edu Regional Educational Laboratories: “Ask a REL” and Issues & Answers briefs http://ies.ed.gov/ncee/edlabs/ 28

29. Objective #5 Preview Session #8 December 3, 2009 3-4:30pm Fidelity of implementation: what is it; how can we do it, and how can we monitor it effectively? Dr. Joe Witt, LSU— video +Call for presenters for session 9 29

30. IES Practice Guide RTI Math IES Practice Guide RTI Reading STATE OF RESEARCH ON RTI Determine the desired outcomes, program features, implementation issues, and extent of evidence on an intervention. INTERVENTION REVIEW PROTOCOL Learn what Fidelity of Implementation is, how to do it, and how to monitor it effectively. FIDELITY OF RTI IMPLEMENTATION Learn from colleagues in GA how they put the concepts and tools from this series into action. PROMISING PROGRESS: LOOKING FORWARD Webinar Series Overview sessions 5 & 6 session 7 session 8 session 9 30

31. Accessing session #8: John Wight  Video  Participant materials  Elluminate Live! webinar 31

32. Thank you for being here today! Dr. Kim Anderson – kanderson@serve.org – 404-657-6174 The SERVE Center/REL-SE – 800-755-3277 – www.serve.org www.serve.org John Wight, GaDOE  jwight@doe.k12.ga.us jwight@doe.k12.ga.us  404-656-0478 32