Assessment for Instruction in Reading in Grades 3-12 Joseph Torgesen and Yaacov Petscher Florida Center for Reading Research International Dyslexia Association, November, 2009

To describe Florida’s new 3-12 assessments for instruction in the context larger issues about formative assessments Purpose of this presentation: Three questions/issues will be discussed How does the Fair system fit within the larger context of a comprehensive formative assessment plan to improve outcomes in adolescent literacy? What is the nature of each of the assessments? What are the purposes of each of the assessments within the FAIR system?

Torgesen, J. K., & Miller, D. H. (2009). Assessments to guide adolescent literacy instruction. Portsmouth, NH: RMC Research Corporation, Center on Instruction. www. centeroninstruction.org centeroninstruction.org Click on Reading Section

Three important purposes for assessment in reading with adolescents 1. How successful are schools in helping all students meet grade level standards in literacy? 2. Monitor student growth during the year to help guide instructional adjustments within classrooms or instructional groups 3. Identify students who may need special interventions to accelerate growth toward grade level standards

Three important purposes for assessment in reading with adolescents 1. How successful are schools in helping all students meet grade level standards in literacy? 2. Monitor student growth during the year to help guide instructional adjustments within classrooms or instructional groups 3. Identify students who may need special interventions to accelerate growth toward grade level standards

The FAIR system for grades 3-12 Broad Screen/Progress Monitoring Tool Reading Comprehension Task 3 times a year Targeted Diagnostic Inventory Maze and Word Analysis Tasks OngoingProgressMonitoring Diagnostic Toolkit If Necessary As needed

Brief Description of tests in the FAIR system Broad Screen/Progress monitoring tool. Is a computer- adaptive, multiple choice test of reading comprehension designed to assess grade level literacy skills in a manner similar to the FCAT. Maze test – a computer administered maze test to assess reading efficiency and low level comprehension Word analysis test – a computer-adaptive measure of student knowledge of the phonemic, orthographic, and morphological knowledge necessary for reading words accurately in text Informal assessment toolkit- Informal diagnostic instruments

Instructional questions the FAIR can answer Which students may require additional reading instruction (i.e. some type of intervention) because they currently have a low probability of success on the FCAT at their grade level? – Reading comprehension test Of students needing some type of intervention, which have relatively serious or pervasive problems with fundamental reading skills? Maze and WA test Of students needing some type of intervention, which might profit from relatively less intensive interventions that focus primarily on comprehension skills (strategies, knowledge, vocabulary) Maze and WA test

Instructional questions the FAIR can answer Are students who receive intensive interventions in basic reading skills making acceptable progress in those skills in response to the instruction they are receiving? Maze and WA tests Are interventions that focus exclusively on improving higher level reading skills effective in increasing students’ reading skills and their probability of success on the FCAT at the end of the year? Reading comprehension test Are students who do not receive interventions making acceptable progress toward grade level standards in reading throughout the year? Reading comprehension test What specific aspects of basic reading skills or comprehension are students struggling with? Informal assessment toolkit

The FAIR system for grades 3-12 Broad Screen/Progress Monitoring Tool Reading Comprehension Task 3 times a year Targeted Diagnostic Inventory Maze and Word Analysis Tasks If Necessary OngoingProgressMonitoring As needed Diagnostic Toolkit As needed

The Broad Screen/Progress Monitoring Tool: The RC Task minute computer-adaptive task of reading comprehension10-30 minute computer-adaptive task of reading comprehension A student receives a minimum of one passage and a maximum of three passages-7 to 9 questions perA student receives a minimum of one passage and a maximum of three passages-7 to 9 questions per Administered to 3 rd grade students identified with a reading deficiency by their districtsAdministered to 3 rd grade students identified with a reading deficiency by their districts Other students may take the assessment at the discretion of their districtsOther students may take the assessment at the discretion of their districts Predicts student’s FCAT success probability (FSP)Predicts student’s FCAT success probability (FSP) In addition to FSP, provides standard scores based on Florida normsIn addition to FSP, provides standard scores based on Florida norms Administered to students performing below grade level on FCAT in previous year (levels 1 and 2)Administered to students performing below grade level on FCAT in previous year (levels 1 and 2)

The Maze Task Two 3-minute (grade-level) passagesTwo 3-minute (grade-level) passages Assesses text reading efficiency (i.e., reading accuracy and speed, and gist - level comprehension)Assesses text reading efficiency (i.e., reading accuracy and speed, and gist - level comprehension) Administered 3 times a yearAdministered 3 times a year Student responds to embedded cloze items within textStudent responds to embedded cloze items within text Provides raw scores and standard scores based on Florida normsProvides raw scores and standard scores based on Florida norms

TDI: The Word Analysis Task A 5-15 minute computer-adaptive spelling task that assesses a student’s understanding of letter/sound correspondence, orthographic conventions, and morphologyA 5-15 minute computer-adaptive spelling task that assesses a student’s understanding of letter/sound correspondence, orthographic conventions, and morphology Results indicate whether probability of success in reading is hampered by difficulties with word- level skills.Results indicate whether probability of success in reading is hampered by difficulties with word- level skills. A student receives a minimum of 5 words and a maximum of 30 wordsA student receives a minimum of 5 words and a maximum of 30 words Provides raw score and standard scores based on Florida normsProvides raw score and standard scores based on Florida norms

All FAIR assessments are intended to be formative (help guide instructional decisions), rather than summative (evaluate instructional and learning success) A definition from Black and William (2007) “An assessment is formative to the extent that information from the assessment is fed back within the system and actually used to improve the performance of the system in some way.” (p. 31)

There is a lot of confusion in the literature about what types of assessments can be called formative, because there are many different kinds of instructional decisions to be made What is assessment for instruction? Does this student need to be in an intervention class? What kind of intervention class should be provided? Does this student need more instruction/practice in using the prediction strategy to improve comprehension? What specific aspects of writing does this study need more instruction in? Is this student’s current intervention placement strong enough to help him close the gap in reading?

There is a lot of confusion in the literature about what types of assessments can be called formative, because there are many different kinds of instructional decisions to be made What is assessment for instruction? Does this student need to be in an intervention class? What kind of intervention class should be provided? Does this student need more instruction/practice in using the prediction strategy to improve comprehension? What specific aspects of writing does this study need more instruction in? Is this student’s current intervention placement strong enough to help him close the gap in reading?

Screening tests– what are initial student abilities in critical areas related to the outcome of interest– intervention or no intervention, initial placement, etc. RC, Maze, WA What types of formative assessments are currently in use? General outcome progress monitoring/benchmark – is student making acceptable progress in critical outcomes? – change class, change teacher, more time, etc. – typically no details about why student is having problems, RC, Maze, WA Formal diagnostic assessments –in depth and reliable information about skill and knowledge levels in critical areas – similar to screening tests in types of decisions they inform

Classroom based formative assessments – what strategies is the student using? What are the problems with making correct inferences? What parts of current objectives has the student mastered? – information to guide ongoing lesson planning and instructional adjustments by the teacher What types of formative assessments are currently in use? Mastery assessments within CBM are a more formalized subset of classroom based assessments. Typically provided within well specified curriculum, and assess mastery of steps in complex tasks Informal diagnostic assessments – what specific phonics skills has the student mastered, what kind of questions can they answer from grade level text? – can be used for initial lesson planning. Informal assessment toolkit

Evidence/rationale for investment in formative assessments “The need for accurate assessments arises because of the enorrmous diversity in the rate of learning and level of literacy skills among adolescents.” (Torgesen & Miller, 2009) When there is great diversity among students in their rate of learning and level of literacy… “little variation in teaching will always result in great variation in student learning.” Formative assessments are necessary to help teachers provide instruction that is targeted and paced appropriately

Evidence/rationale for investment in formative assessments – how can you tell when you have a good system of formative assessment? Traditional ways of evaluating tests include examinations of their reliability and validity Can you have a reliable and valid test of reading comprehension that does not provide information useful for guiding instruction? It depends on the instructional question you want to answer–not all tests provide information relevant to all important instructional decisions

Evidence/rationale for investment in formative assessments The critical questions for “formative” tests are: 1) what kind of information do they provide, and 2) what is that information useful for? The most important outcome from formative assessments is improvement in student performance – thus, the ultimate test of their utility can only be made as they are integrated within a system of instructional response.

Evidence/rationale for investment in formative assessments “It is impossible to disentangle the impact of formative assessment from the instruction that follows it. When improved student outcomes are used as the criterion for effective practices, evaluations of classroom-based formative assessment are as much an evaluation of the instructional adjustments resulting from the assessments as they are of the assessments themselves.” (Torgesen & Miller, 2009)

Evidence/rationale for investment in formative assessments Two research summaries of interest Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Educational Assessment: Principals, Policy and Practice, 5, 7–74. A comprehensive review of classroom-based formative assessment. Effect sizes from.4 to.7 Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208. A look at very frequent (2-5 times a week) progress monitoring. ES of.92 when teachers required to follow up in specific ways,.42 when follow-up actions left to teacher judgement

The FAIR system in this context: Screening/PM tool (reading comprehension test)– screening and general outcome progress monitoring no more frequent than once a month Maze test – screening and general outcome progress monitoring no more frequent than once a month Word analysis test – screening and general outcome progress monitoring no more frequent than three times a year– also can be used diagnostically through error analysis Informal assessment toolkit- Informal diagnostic instruments

Beginning of year Screening and informal diagnostics 1. General Screening 2. Targeted Screening 3. informal diagnostics Throughout the year Progress Assessments 1.Classroom Based Formative Assessment including CBA Mastery assessment 2. General Outcome CBM End of year Standards based accountability measures End of year “high stakes” test At this time, the only evidence to support formative assessments with older students concern classroom based formative assessments and very frequent general outcome progress monitoring with special needs students.

Questions/ Discussion