Everything You Ever Wanted to Know about Curriculum-Based Measurement Scott Methe, Ph.D. University of Massachusetts at Boston Assistant Professor of School Psychology scott.methe@umb.edu 617-287-3167

Five Key Parts for Today Curriculum-based measurement (CBM) is a set of procedures used to help make better, and more accurate instructional decisions. We will go over key terms (read the “hosp_2012” article in the wiki to help with terms). Most of today’s talk will center around basic examples of CBM, how they assist with decisions, and discuss how to locate measures that are already available. We will then discuss how to build a scope and sequence of tools based on the different decisions to be made in a RTI model using NCRTI and go over the handout that is available on the wiki. A process for creating your own mathematics measures will be shared and the benefits discussed. Time permitting, we’ll discuss some important related issues such as: assessment schedules, how to examine the “assessment literacy” of your staff, how to assess your school’s readiness to use CBM, and the benefits of using CBM in psychoeducational evaluations.

What is Testing? What is Assessment? Refer to the “hosp_2012” article as a refresher of important terms and an overview of CBM. Assessment simply means a process for gathering information. A test is a means used to gather that information. A standardized test is given in the same way to all students so that scores can be compared and decisions about instructional allocation can be made. And more…

The Most Important but Often Most Neglected Point CBM is not something additional, or another test to layer on to the 10,543,395 tests you already use that take you away from quality instructional time. CBM is being presented as an ALTERNATIVE to these tests and a means by which the other tests used for the “big three” decisions SHOULD BE REPLACED. This “replacement clause” is based upon the PURPOSE of the assessment / test being used. Not all tests need to be replaced (but most of them do).

What is CBM? CBMs are “General Outcome Measures” CBM key features are: Read “Big Ideas About GOM” in Wiki. CBM key features are: Short duration to save time for teaching Routine and standardized administration instructions Ability to use alternative forms for progress monitoring (repeated measurement) Sensitive to small improvements in student learning Example that illustrates the importance of using a CBM approach: standard tests and conventional teacher grading that considers the word “elefant” as incorrect and assigns zero points fails to give credit to “letter sequences” that can indicate that the student is knowledgeable about spelling, with slight problems. Talk about Deno’s origin – that they were literally taken from the curriculum and now they are not, but work across different curriculum.

Oral Reading Fluency is a General Outcome Measure Book and Print Awareness Letter (Orthographic) Knowledge Oral Language Phoneme Awareness CBM of Oral Reading Fluency Sight Word Identification Phonological Awareness Alphabetic Principle Rich Vocabulary Why? Because you NEED everything in the box to read fluently, or, having everything in the box enables oral reading fluency, and indicates that it is ready!

Math Computation is a General Outcome Measure Number Names Relative Size Knowledge One to One Correspondence Ordinal Number Knowledge CBM of Math Computation Cardinal Number Knowledge Unitizing and Grouping Subitizing: Conceptual and Perceptual Partitioning and Part Whole Knowledge Why? Because you NEED everything in the box to do speedy, accurate, and flexible computations (and to explain how you did it).

So many different types of tests, CBMs, and measures available So many different types of tests, CBMs, and measures available. What the heck is going on??? See next slide!

What are the differences between DIBELS, AIMSweb, easyCBM, STEEP What are the differences between DIBELS, AIMSweb, easyCBM, STEEP? I’m SO CONFUSED! Help me, Dr. Methe! These are just companies that develop and sell the tests. Use a medical analogy; hospitals use different manufacturers and brands of devices built to sample a person’s blood oxygen level, for example, but the blood oxygen level is just a number that characterizes a person’s vital signs. As an educational professional, you use different brands to measure things like oral reading fluency, which, like blood oxygen level, is just a number that characterizes a student’s academic vital signs.

What to Look for In CBM Reliability – The consistency and trustworthiness of a test is important, and the degree of trustworthiness is related to the decision that needs to be made. Low-stakes decisions do not require measures like CBM, although CBM can be helpful for a variety of purposes. High-stakes decisions require reliable tests. Predictive Validity – Predictive validity is what sets CBM apart from other tests, and it means that when the test is given to a group of children, the rank-ordering of that same group is very similar at a later time point and on a different and more comprehensive test (i.e., state end-of-grade tests). Diagnostic Accuracy – This looks at whether or not a cutoff score can be used reliably to form groups that are stable over time. On the next slide, we talk about 4 groups and CBM’s ability to differentiate among these groups. Sensitivity to Student Growth – This is whether or not the test score changes over time, and changes predictably among a group of students (i.e., faster for students with more skills and slower for students with less skill). Sensitivity to Intervention – This pertains to whether or not a test score grows more rapidly when students are given an instructional approach that is known to work (and less rapidly when the approach is not being given).

What are Some Examples of CBMs?

Letter Sound Fluency (www.aimsweb.com) Specific Materials: Student copy of Letter Sound Fluency Examiner copy of Letter Sound Fluency Clipboard Stopwatch Administration and Scoring of Early Literacy Measures for Use with AIMSweb Training Workbook p. 20

Phoneme Segmentation Fluency (www.aimsweb.com) Have Materials Arranged: Examiner copy of Phonemic Segmentation Fluency (numbered for easy scoring) Clipboard to provide a hard surface for recording student answers Stopwatch Administration and Scoring of Early Literacy Measures for Use with AIMSweb Training Workbook p. 25

An Example of Oral Reading Fluency - Illustration 1 (from www. aimsweb An Example of Oral Reading Fluency - Illustration 1 (from www.aimsweb.com)

How Many Correct Word Sequences are Written? As I walked down the path, I saw a lost puppy in the field, crying for his mother… The puppy waz little and had white fir with floppie ears he was walking towerd me and I wanted to pick him up and hold him. I called the puppy and he came and sat down net to me and I think he is my friend. I wonder if mom will be let me have him.

Example of Standardized WE-CBM Scoring: CWS Rule 1. Pairs of Words Must Be Spelled Correctly (CWS marked with caret): All of the kids started to laugh. CWS = 8 All of the kids started to laghf. CWS = 6 Rule 2. Words Must Be Capitalized and Punctuated Correctly with the Exception of Commas. 2a. Correct punctuation must be present at the end of the sentence. 2b. The first word of the next sentence must be capitalized and be spelled correctly for a correct writing sequence to be scored. The sky was blue. It was pretty. CWS = 9 The sky was blue. it was pretty CWS = 6 ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 2a 2b ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^

M-CBM Math Facts: Addition Page 1 of 2 Page 1 of 2 Grades 1-3 (fewer problems) Grades 4-6 (more problems)

M-CBM Math Facts: Addition Page 1 of 2 Page 1 of 2 Grades 1-3 (fewer problems) Grades 4-6 (more problems)

Right Tools for the Right Purposes Screening: CBM Question: which children need different intensity of academic support? Analyzing and Diagnosing: All available tools in teacher’s arsenal. Question: which subskill seems to be missing and causing most problems for the child? Target instruction and intervention here. Progress Monitoring: CBM Question: is the intervention effective enough to produce a noticeable response?

How Does CBM Help in Decision Making? CBMs are FORMATIVE TESTS and are used to help children learn. They are used before and during instruction Although they can also be used after instruction, they were not designed like test batteries or state tests, which are designed as accountability or program eligibility measures. Before instruction, teachers care about the extent of resources to allocate to students or groups of students. During instruction, teachers want to know if students are progressing. When students do not progress, teachers want to know why. Using this same order, we have the following three classes of decisions: Screening – how much instruction is necessary? Progress – how much growth are students making in response to instruction? Diagnostic – what specific skills are students lacking and thus holding them back from progressing?

What More Should I Know About these Three Key Decisions? Schools (i.e., teachers and teams) must decide how much instruction (i.e., intensity, duration, materials, etc.) is necessary for groups and for individuals. This is called a screening decision, and CBM was built to help with this. All teachers want to know whether or not students are progressing and if instruction is hitting its mark. This is called a progress decision, and CBM was built to help with this. When students fail to progress, one of the most fundamental reasons is that they are lacking prerequisite skills. CBM can help point out which skills are lacking. This is called a diagnostic decision. However, CBMs were not originally built for these decisions. Many “CBM” tests can be used but look for the term “subskill mastery measures” or “skill-based measures.”

Where are CBMs Available? Before using any CBM, visit the National Center for Response to Intervention (NCRTI) at www.rti4success.org. The center has “tools charts” for measures that fit into screening, progress monitoring, and into diagnostic assessment. Some screening tools are comprehensive and diagnostic, and some “progress monitoring mastery measures” tools are diagnostic.

How Should all These CBM Tools Be Organized in a School? Just like a curriculum is a scope and sequence of instructional goals and objectives, you must identify the scope and sequence of tests that you use for each decision, at three time points, and across grades. In addition to identifying the tests by name, the technical properties of each of the tools used should be closely examined (i.e., reliability, diagnostic accuracy, etc.). If you are lacking these, you do not have the foundations for an effective and efficient RTI model to function.

What do I need to know if I wanted to create my own math computation probes?

Can we save a TON of time and replace costly test batteries such as the WIAT and WJ-III with CBM? Yes, by the way!

CBM sounds great. I want to use it CBM sounds great. I want to use it. However, I don’t think everyone in my school will want to. How do I start getting buy in and support?