Developing Comprehensive Evidence-based Assessment in Reading Comprehension: Alison L. Bailey & Margaret Heritage CRESST/UCLA CRESST Conference - September 9th, 2004 A View of the Construct and Assessment Model

Overview 1. Introduction 2. Context 3. Constructs of Reading Comprehension 4. Role of Formative Assessment 5. Activity: Applying the Model 6. Specter of Needed Data Management Effort 7. Web Solutions: QSP System and NSF TBALL study

Background: NAEP 2003 – only one third of 8th graders at the proficient level or above Mandate: Federally-funded National Reading Panel (NRP, 2000) & the Rand Report (Snow, 2002) identify the increasing need for assessment (and instruction) of Reading Comprehension (RC) Needs Analysis: Tests and teachers require improved (deeper and broader) construct definition of RC for assessment development Prior work: The Literacy Development Checklist (LDC) (Bailey, Gallimore, Heritage, et al. (2000), an assessment tool conceived of as a lens for teachers Introduction

The Black Box Black & Wiliam, 1998

Inside the Black Box

Using Assessment Information: Reading Comp. Checklist (RCC) Model

Phoneme awareness Alphabetic principle Decoding Adams, 1990; Lyon, 1998; National Reading Panel, 2000 Construction Definition: Word Recognition

Construction Definition: Fluency Rapid word recognition Syntactic sensitivity Rapid use of punctuation Adams, 1990; Pinell et al., 1995; Snow, Burns, & Griffin, 1998

Construction Definition: Text Presentation and Structure Conventions Features Organization Patterns Pearson & Fielding, 1991; Seidenberg, 1989

Construction Definition: Oral Language and Pragmatics Vocabulary Syntax Discourse Pragmatics Nagy & Herman, 1987; Roth, Speece, Cooper, & DeLaPaz, 1996; Senechal & LeFevre, 2002; Tunmer, Nesdale, & Wright, 1987; Whitehurst & Lonigan, 1998; 2001

Construction Definition: Metacognition Metalinguistics Motivation Knowledge of strategies Gillet & Temple, 2000; Menyuk & Chesnick, 1997; Palinscar, David, Winn, & Stevens, 1991; Schunk & Rice, 1992

Event Representation Background Knowledge Memory Home/School Connections Hudson & Nelson; 1986; Long, Oppy & Seely, 1997; Gallimore & Goldenberg, 1993; Heath, 1983; Wells, 1985 Additional Construct Definitions

Using Assessment Information

Integrating Assessment and Instruction

1.Select a RC construct (or another content area) 2.Nominate a formative assessment to measure a 5 th -grade student’s performance on the construct 3.Nominate an intervention/teaching strategy to target development of the construct 4.Imagine no student growth/improvement detected in a repeat formative assessment: H 1 - Specify and nominate new intervention H 2 - Specify and nominate new construct Activity

1.What kinds of formative assessment did you nominate? 2.What other kinds of formative assessment would teachers need to be aware of in this context? 3.When would you know to shift from H 1 to H 2 ? 4.How much content knowledge would you need to operate successfully within this assessment model? Guiding Questions

Integrating Assessment and Instruction

Problem: RCC Model raises specter of a colossal data management effort Solution: Data driven website and software tools for data storage, query and report generation Applications: QSP System & NSF TBALL study Web Technology as a Vehicle for Data Management

Decision Support: QSP

Technology Support