Data-Based Decision Making College and Career Readiness and Success Center Jenny Scala Senior Researcher, American Institutes for Research January 2017

Agenda Introductions Data-based decision making overview Selecting appropriate interventions

Overview of Data-Based Decision Making

Institute of Education Sciences (IES) Practice Guide: Using Student Achievement Data The recommended practices for effective data use are as follows: Make data part of an ongoing cycle of instructional improvement. Teach students to examine their own data and set learning goals. Establish a clear vision for schoolwide data use. Provide supports that foster a data-driven culture within the school. Develop and maintain a districtwide data system. Source: Hamilton et al., 2009.

Examples of Recommendations Analyze data at all levels (i.e., state, school, Tier I, Tier II, Tier III). Establish routines and procedures for making decisions. Set explicit decision rules for assessing student progress (e.g., division benchmarks). Use data to compare and contrast the adequacy of the core curriculum and the effectiveness of different instructional and behavioral strategies. In a comprehensive RTI framework, data analysis occurs at all levels of RTI implementation, not just at the student level. For example, States may use RTI data to establish policy and guidance and allocate resources. Districts may use data to evaluate the effectiveness of RTI, establish policies and procedures, and allocate resources. Schools may use data to evaluate the effectiveness of their overall framework and the essential components, assess alignment among grade levels, and allocate resources. Grade-level teams may use data to evaluate core curriculum and instruction, identify students for secondary and tertiary instruction, and allocate resources. Data analysis and decision making occur in all levels of the prevention system. For example, in primary prevention, the interest is the effectiveness of the core curriculum and instruction. With secondary and tertiary prevention, the interest is in student-level decisions, but also how well particular interventions work for the majority of students in the secondary and tertiary levels. Districts and schools should have established routines and procedures, ideally in writing, for making decisions. Written procedures increase fidelity of the data-based decision making process; ensure equity of resources among students, classes, and schools; and help train new teachers more efficiently. Teams should follow pre-established routines and procedures for making decisions. For example, data teams should meet at regularly scheduled intervals, such as monthly or bi-monthly, to systematically review data. Districts and schools should also establish explicit decision rules for assessing student progress. This includes goal-setting procedures, changing instruction/interventions, referring students to special programs, and moving students to more or less intensive levels. Schools can also use data to compare and contrast the adequacy of the core curriculum and the effectiveness of different instructional and behavioral strategies at all levels of the prevention system.

Tiered Approach Academic Focus Behavior Focus Students with disabilities Receive services at all levels, depending on need Tier III Specialized, individualized systems for students with intensive needs ~5% ~15% Tier I Schoolwide instruction for all students, including differentiated instruction Tier II Supplemental group systems for students with at-risk response to primary level Although the design of the RTI model implemented is up to the state education agency (SEA) or LEA, it is recommended that special education staff and students with disabilities are included in the development and the implementation of the multi-level system. (CLICK to show animation). In this model, students with disabilities continue to received access to the core curriculum through accommodations, differentiated instruction, and other support. Then, depending on the services outlined in the IEP, students may receive ongoing secondary (unlike short-term secondary) support or intensive, individualized services delivered by the special education teacher. Thus, complete removal from the core curriculum is not typically recommended because students are assessed against grade-level standards. ~80% Academic Focus Behavior Focus

Types of Decisions Instruction Evaluate effectiveness How effective is the instruction? What instructional changes need to be made? Evaluate effectiveness Is the core curriculum effective for most students? Is one intervention more effective than another? Movement between supports and interventions How do we know when a student no longer needs additional supports? These are the more common types of decisions that schools make. Instruction – How effective is the instruction? What instructional changes need to be made? Evaluate Effectiveness – Is the core curriculum effective for most students? Is one intervention more effective than another intervention? Movement within the multi-level prevention system – How do we know when a student no longer needs secondary prevention or should move from secondary prevention to tertiary? These decision rules should be outlined prior to the implementation of your RTI framework. Ask if there are any questions about data-based decision making.

Middle School Examples From the Field Prescreening questionnaire is given to all incoming sixth graders. District-provided cut scores are used to determine which students are in need of interventions. School counselors organize all the data. Leadership team meets every four weeks and discusses all students receiving intervention as well as those students who have been referred to the team by content-area teachers. Pre-screening questionnaire asks for a brief report from the feeder elementary schools on how each student is doing in reading/math. These data gives staff starting data point prior to the beginning of the year screener for incoming students. They use all the screening data when making decisions about what the screening information means. In addition, the results of diagnostic assessments, grades, and other sources of "soft data” are used to inform data discussions. The school principal credits the leadership data teams for gaining buy-in for the RTI initiative. The team has core members (e.g., Administration, counselors, school psychologist, special education teachers) and has the general education teachers rotate onto the team to experience the data-based decision making process, as well as to lend a breadth and depth of expertise to the needs of the students. Leadership teams meet once a month to discuss students receiving interventions – to see if the interventions are working, does the student still need the intervention, etc. Teachers are also able to refer students to the team if a student starts demonstrating need (that was not “caught” by the screener) Prewett MS is starting to explore the idea of involving the students in the team data-based decision making process so they can more fully participate in their learning.

Middle School Examples From the Field Intervention teachers meet every two weeks with primary- level teachers to discuss students’ progress in both the core curriculum and in the intervention. Data are also used as a “report card” for instruction.

High School Examples From the Field The student information system contains screening and progress monitoring data. The early warning system tool identifies which students are at risk for not graduating high school. Data are reviewed during department, small learning community, or monthly data meetings Data inform which students are placed in interventions. Student progress in interventions is reviewed during meetings. The school establishes exit and entrance criteria for interventions.

High School—Examples From the Field Data are shared with entire faculty during “data days” (half days of professional development are held three times a year). Students receiving Tier II or Tier III instruction are given the opportunity every other week to view their progress monitoring data and goals. Parents are notified of students participation in secondary and/or tertiary levels of support with two weeks of placement.

Process for Analyzing Data Review big picture data and predictions. What patterns emerge? Big Picture What students or groups of students most concerned? What initial theories may explain why the student is at risk? Define Target What additional information could you collect to better understand underlying causes of risk? Are there gaps in data you have available? Other Data What have you learned from this new data or evidence? What do you now believe is the likely cause(s) of risk? What do student(s) need (define the problem to be solved)? Confirm Cause What steps or tasks need to be implemented to address the underlying cause of concern? How will these changes be monitored to determine student progress? How will fidelity be monitored? Action Planning