1. Developing Comprehensive Evidence-based Assessment in Reading Comprehension: Alison L. Bailey & Margaret Heritage CRESST/UCLA CRESST Conference - September 9th, 2004 A View of the Construct and Assessment Model

2. Overview 1. Introduction 2. Context 3. Constructs of Reading Comprehension 4. Role of Formative Assessment 5. Activity: Applying the Model 6. Specter of Needed Data Management Effort 7. Web Solutions: QSP System and NSF TBALL study

3. Background: NAEP 2003 – only one third of 8th graders at the proficient level or above Mandate: Federally-funded National Reading Panel (NRP, 2000) & the Rand Report (Snow, 2002) identify the increasing need for assessment (and instruction) of Reading Comprehension (RC) Needs Analysis: Tests and teachers require improved (deeper and broader) construct definition of RC for assessment development Prior work: The Literacy Development Checklist (LDC) (Bailey, Gallimore, Heritage, et al. (2000), an assessment tool conceived of as a lens for teachers Introduction

4. The Black Box Black & Wiliam, 1998

5. Inside the Black Box

6. Using Assessment Information: Reading Comp. Checklist (RCC) Model

7. Phoneme awareness Alphabetic principle Decoding Adams, 1990; Lyon, 1998; National Reading Panel, 2000 Construction Definition: Word Recognition

8. Construction Definition: Fluency Rapid word recognition Syntactic sensitivity Rapid use of punctuation Adams, 1990; Pinell et al., 1995; Snow, Burns, & Griffin, 1998

9. Construction Definition: Text Presentation and Structure Conventions Features Organization Patterns Pearson & Fielding, 1991; Seidenberg, 1989

10. Construction Definition: Oral Language and Pragmatics Vocabulary Syntax Discourse Pragmatics Nagy & Herman, 1987; Roth, Speece, Cooper, & DeLaPaz, 1996; Senechal & LeFevre, 2002; Tunmer, Nesdale, & Wright, 1987; Whitehurst & Lonigan, 1998; 2001

11. Construction Definition: Metacognition Metalinguistics Motivation Knowledge of strategies Gillet & Temple, 2000; Menyuk & Chesnick, 1997; Palinscar, David, Winn, & Stevens, 1991; Schunk & Rice, 1992

12. Event Representation Background Knowledge Memory Home/School Connections Hudson & Nelson; 1986; Long, Oppy & Seely, 1997; Gallimore & Goldenberg, 1993; Heath, 1983; Wells, 1985 Additional Construct Definitions

13. Using Assessment Information

14. Integrating Assessment and Instruction

15. 1.Select a RC construct (or another content area) 2.Nominate a formative assessment to measure a 5 th -grade student’s performance on the construct 3.Nominate an intervention/teaching strategy to target development of the construct 4.Imagine no student growth/improvement detected in a repeat formative assessment: H 1 - Specify and nominate new intervention H 2 - Specify and nominate new construct Activity

16. 1.What kinds of formative assessment did you nominate? 2.What other kinds of formative assessment would teachers need to be aware of in this context? 3.When would you know to shift from H 1 to H 2 ? 4.How much content knowledge would you need to operate successfully within this assessment model? Guiding Questions

17. Integrating Assessment and Instruction

18. Problem: RCC Model raises specter of a colossal data management effort Solution: Data driven website and software tools for data storage, query and report generation Applications: QSP System & NSF TBALL study Web Technology as a Vehicle for Data Management

19. Decision Support: QSP

20. Technology Support