The Many Meanings of “Multiple Measures” Susan Brookhart Volume 2009, Volume 67:3 ASCD, November 2009, pp. 6-12

Would you choose a house using one measure alone?

Why use multiple measures for decisions in education? Construct validity –The degree to which a score can convey meaningful information about an attribute it measures Decision validity –The degree to which several relevant types of information can inform decision- making

Multiple Measures Measures of different constructs Different measures of the same construct Multiple opportunities to pass the same test

Using Multiple Measures for Educational Decisions Conjunctive Approach (All measures count) Compensatory Approach (High performance on one measure can compensate for lower performance on another measure) Complementary Approach (High performance on any measure counts) Measures of different constructs School accreditation ratings based upon student achievement meeting identified targets in Reading, Math, Science, and Social Studies An outside agency identifies the “best schools identified by computing an index of weighted scores AYP “Safe Harbor” by having a percentage of students who scored below proficiency decreasing by ten percentage points from the previous year Different measurers of the same construct Students have to pass a reading comprehension test on two stories at the same reading level before the student is allowed to read stories at the next higher reading level Teachers determine standards-based grades in a course using scores on multiple assessments measuring the same GLCE or HSCE Teachers allow student choice on assessment tasks to demonstrate their understanding of the learning targets for a unit Multiple opportunities to pass the same test Students meeting all requirements will graduate after passing an exit exam, no matter how many opportunities Teachers allow students to retake a unit test to demonstrate mastery of the unit’s outcomes Students must pass one mathematics test in order to graduate; students can choose the state test or an end-of-course exam in either Algebra I or Geometry

Other Examples NCLB accountability is conjunctive (i.e., aggregate and subgroups must reach threshold to make AYP) Most classroom grading policies are compensatory (i.e., average, percentage) Getting a driver’s license is complementary (i.e., passing one of the requirements when you want)

Guiding Principle for Multiple Measures Know your purpose! –What do you need to know? –Why do you need to know it?

Suggestions for Using Multiple Measures for Decision Making Classroom assessments linked to the same construct to determine mastery Granting credit for graduation requirements Evaluating school programs