Assessment within a Response to Intervention Model: Tools and Ideas Karen Karp Amy Lingo

Topics Overview of RtI Model, a multi-tiered intervention approach Universal Screeners: What do the data tell us and not tell us? Designing and implementing strategies for tiers of intervention

3-Tiered Support Model - UNIVERSAL Tertiary Prevention: Specialized & individualized strategies for students who don’t respond to Tier 2 supplementary support - INTENSIVE ~5% Secondary Prevention: Supplementary strategies for students who do not respond to core instruction - TARGETED ~15% Primary Prevention: High quality engaging core instruction - UNIVERSAL ~80% of Students

Has RtI implementation closed the gap? Note here on the bottom line (the 10th percentile that would be inclusive of some of the special education population) that since 2004 When rti support models and the use of supplemental interventions were mandated that the line is relatively flat. Why is that? Let’s look at eighth grade

Are we doing any better at 8th grade?

Mathematics Performance on NAEP 2013 4th grade At or Above Proficient: 45% (+5%) of all students and 18% (+1%) of students with disabilities Below Basic: 18% (-4%) of all students and 45% (=) of students with disabilities 8th grade At or Above Proficient: 39% (+4%) of all students and 9% (=) of students with disabilities Below Basic: 21% (-6%) of all students and 65% (=) of students with disabilities So to meet proficiency on the NAEP – 83% of students with disabilities need to move up in 4th grade (with as you see more than half of that group at the below basic level) and 91% of the students in 8th grade need to move up to proficiency (with approximately ¾ of that group at the below basic level) You have to ask – do we think worksheets or a generic computer program can make that happen? (I am not talking about delivering high quality instruction via technology – but the generic programs I see in use are not instructive) National Center for Education Statistics, 2013

What about RtI? Why isn’t it Helping? A recent study revealed that teachers providing Tier 2 mathematics instruction to elementary and middle grade students largely used worksheets (Foegen & Dougherty, 2010; Swanson, Solis, Ciullo & McKenna, 2012) In our travels to classrooms many others use a one-size- fits-all generic computer program. Why would we think that students who don’t function well in a class of 25 students can work independently to learn the mathematics? Unfortunately this goes back to the attempt to buy one program for all students who struggle

Components of A Strong RtI Model Incorporates a regular screening process Includes evidenced based practices Instructs with preventative methodology Integrates progress monitoring Uses diagnostic assessment to align intervention Newman-Gonchar, R., Clarke, B., & Gersten, R. (2009). A summary of nine key studies:Multi-tier intervention and response to interventions for students struggling in mathematics. Portsmouth, NH: RMC Research Corporation, Center on Instruction.

Tier 1 - Universal Mathematics Instruction Implementation of core mathematics instruction Instruction with methodology addressing both conceptual and procedural understanding. Implementation of instruction with fidelity

Identification of Students Through Screening Universal Screener Building level team to facilitate the implementation of the screening and progress monitoring Use benchmarks or growth rates to identify students at low, moderate, or high risk for developing mathematics difficulties.

is given a Universal Screening Student Body is given a Universal Screening X X X X X X X X X X X X X X X X X X X X X X X X X

(What we hear schools are using) Universal Screenings (What we hear schools are using) AIMSWeb Think Link G-MADE MAP Universal Screening: Determines which students may have possible mathematics difficulties. This is not an endorsement of the products, but a listing of those described by schools.

MAP: Adaptable Computer Based Assessment NWEA Measures of Academic Progress® (MAP®) tests present students with engaging, age-appropriate content. As a student responds to questions, the test responds to the student, adjusting up or down in difficulty. With each assessment a student will get approximately 50% of the questions correct to get an accurate RIT score. The result provides detailed information for teachers, parents and administrators to use in the classroom to drive instruction, interventions, and enrichment.

What is RIT? RIT or “Rasch Units” are units of measurement involving the difficulty of individual items. They estimate a particular student’s achievement. The RIT scale measures achievement growth over time. As a student goes to the next grade level, RIT scores “grow” with them. For example if a grade 2 student had a 200 point RIT score and then 210 at grade three, it means that his/her academic performance increased by 10 RITs. RIT scores should not be used to compare one student with another. Instead, RIT scores assess if a student has “grown” academically. The scores represent individual performance on individual items so similar scores as other students doesn’t necessarily mean similar achievement.

NWEA Norms

Normative Data

MAP Data for Instruction Tier 1 RIT Bands for instructional flexible grouping

Individual Student Report Summary (over time)

Student Graph District average graph Student graph Norm group graph

Sample Math Question from MAP Work the problem on scratch paper. Click on the best answer. Click the Go on button. Talking Points: This shows what a math question might look like. You will have scratch paper at your computer if you need to use it to help you solve the problem. Once you have worked the problem, click on the best answer from the list. Then click the Go on button at the bottom of the screen. Remember, you may change your answer as many times as you want, but once you click Go on you cannot go back to this question. MAP® Student Presentation Revised 7/2012 © 2012 Northwest Evaluation Association™

Individual Student Report

In-Depth Assessment determines: Students Identified by the Universal Screening are given more in-depth mathematics assessment. X X X X X X X X X X X X X X X X In-Depth Assessment determines: Tier of Support Specific Areas of Need Plan of Intervention

In-Depth Assessment CCSS- Math Domains Counting and Cardinality Operations and Algebraic Thinking Number and Operations in Base 10 Number and Operations- Fractions Geometry Data and Measurement

Focus on Skills

Focus on Skill

Focus on Sense Making

The Two Worlds Collide: Sense Making Meets Skill Petit, Laird,& Marsden, 2010

Diagnostic Interview Gathers in-depth information about an individual student’s knowledge and mental strategies. Provides evidence of prior knowledge, naïve understandings and students’ ways of thinking about concepts. Focuses on a task or problem where students are asked to either verbalize their thinking or demonstrate ideas through models or drawings Emphasizes the collection of evidence Is not a teaching opportunity Uses errors to identify barriers to understanding, to inform instructional decisions

What might a student’s brain look like? What if one student had a good understanding of a concept and the other student had just memorized it (or lacked the ability to efficiently memorize – like a student with disabilities)?

Relational understanding: Focus on how and why Build on prior knowledge Discuss relationships between strategies and ideas Instrumental understanding: Focus on how Develop isolated skills, concepts, rules, and symbols Doing math problems without understanding From Teaching Student Centered Mathematics (2014) (Volumes I - III)

Link Sheet Equation Story problem or Situation Model, Picture, Illustration My Explanation of the Operation Van de Walle, J., Karp, K., & Bay Williams, J. (2013). Elementary and Middle School Mathematics: Teaching developmentally. New York: Pearson.

Student work 5 + 8 = 13 Equation Word Problem Model/Illustration There were 5 birds on the feeder. Then 8 more birds flew to the feeder. How many birds are now on the feeder? Model/Illustration Explanation First I had 5 birds. Then I needed to combine them with 8 more birds. I added so all together I have 13 birds. 5 + 8 = 13 Van de Walle, J., Karp, K., & Bay Williams, J. (2013). Elementary and Middle School Mathematics: Teaching developmentally. New York: Pearson.

Middle School Level Algebraically Graphically Numerically in Tables Verbal Description Van de Walle, J., Karp, K., & Bay Williams, J. (2013). Elementary and Middle School Mathematics: Teaching developmentally. New York: Pearson.

How to design a diagnostic assessment Identify what students need to know CCSS Give a task Record and/or make notes Have materials available for students to use The difference between a regular task and a diagnostic interview is that you are sitting and listening to the student talk about their thinking and how they are solving the task. Questions are used to probe and materials and other representations are used to see the depth of understanding. You never walk away wondering why.

Diagnostic Interviews – Knowing students’ thinking Select tasks that are close to what you expect the children to be able to do – start easy and ramp up if you are not sure Ask questions to prompt why they are doing something – or to ask them to explain their thinking or use models or drawings to demonstrate their ideas Be neutral – watch your body language - avoid clues or leading questions Wait silently - Do not interrupt – Do not teach Take notes, keep student work, take photos Say – show me, tell me, do…, try…, can you do that another way? Karen

Diagnostic Interviews Collect in-depth information about an individual student’s knowledge and mental strategies. Provide evidence of students’ prior knowledge, naïve understandings and ways of thinking Focus on a task/problem where students are asked to verbalize their thinking and/or demonstrate ideas through multiple representations Is not a teaching opportunity Use errors to identify barriers to understanding and to inform instructional decisions Van de Walle, J., Karp, K., & Bay Williams, J. (2010). Elementary and Middle School Mathematics: Teaching developmentally. New York: Pearson.

Diagnostic Interview Task Ask the student to write the number that shows: 3 ones, 1 hundred and 5 tens Karen

Student Work Karen

Progress Monitoring: Assessment Procedural measures: skills, algorithms Typically multiple-choice format

Assessment If assessments only measure skill, it is difficult to determine what a student knows. If you cannot determine what a student knows, it’s difficult to plan an instructional sequence. Barb

Progress Monitoring Conceptual measures May not require computation Focuses on reasoning Emphasizes Big Ideas May be multiple choice but the multiple choice options are linked to thinking, not just the answer May open-response items, scored with a metric/rubric

Interventions Following Diagnostics Use of screening information Use of diagnostic interview Create a plan for intervention

Recommendations for identifying and supporting students struggling in mathematics Recommendations are based on strong and moderate levels of evidence resulting from comprehensive reviews of current research Gersten, R., Beckmann, S., Clarke, B., Foegen, A., Marsh, L., Star, J. R., & Witzel, B. (2009). Assisting students struggling with mathematics: Response to Intervention (RtI) for elementary and middle schools (NCEE 2009-4060). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies. ed.gov/ncee/wwc/publications/practiceguides/.

General Screening and Intervention Recommendations Screen all students Choose appropriate instructional materials Intervention with explicit instruction Modeling Talk aloud (verbalization) Guided practice Feedback (correction of errors) Frequent review of progress Problem solving instruction based on common underlying structures Visual representations Strong Moderate Low

Screening and Intervention Recommendations 10 minutes per session devoted to fluency building of basic mathematics facts Progress monitoring Integration of motivational strategies Strong Moderate Low

Recommendations for students identified as low-achieving On a regular basis; and For the purpose of building computation and problem solving proficiency; Explicit instruction including opportunities for asking and answering questions Think aloud opportunities regarding decisions during problem solving Dedicated time to foundational skills necessary for grade level mathematics learning National Mathematics Advisory Panel. Foundations for Success: The Final Report of the National Mathematics Advisory Panel, U.S. Department of Education: Washington, DC, 2008.

Recommendations from research involving small group interventions Explicit instruction Concrete--Semi-concrete--Abstract approach Modeling Underlying mathematical structures Examples (consideration of range and sequence) Independent work with immediate corrective feedback Visuals (drawings & diagrams) Note: The interventions may be effective for other student groupings. This listing specifically targets small groups. Newman-Gonchar, R., Clarke, B., & Gersten, R. (2009). A summary of nine key studies: Multi-tier intervention and response to interventions for students struggling in mathematics. Portsmouth, NH: RMC Research Corporation, Center on Instruction.

CSA – Concrete Semi-Concrete Abstract Van de Walle, J., Karp, K., & Bay Williams, J. (2013). Elementary and Middle School Mathematics: Teaching developmentally. New York: Pearson.

Effective Practices for Teachers Explicit Instruction A range of instructional examples, a sequence from concrete-semi-concrete-abstract Verbalization by the students and the teacher Use of visual representation Multiple heuristic strategies Formative assessment information provided to teachers Peer-assisted learning (1:1 tutoring) Cross age (more effective) Within classroom same grade, role exchange Performance based Jayanthi, M., Gersten, R., Baker, S. (2008). Mathematics instruction for students with learning disabilities or difficulty learning mathematics: A guide for teachers. Portsmouth, NH: RMC Research Corporation, Center on Instruction.

Thank you Contact information. Karen Karp karen@louisville.edu Amy Lingo amy.lingo@louisville.edu