Text Dependent Analysis: Building teacher capacity to instruct for a new item type Susan Lyons, Ph.D. CCSSO 2016 National Conference on Student Assessment Philadelphia, PA June 2016

Text-Dependent Analysis 2015-2016 Exploration: If teachers understand the instructional strategies necessary to teach the elements underlying a TDA (e.g., close reading strategies, scaffolded essay writing, instruction of scoring guidelines, teaching students to ask questions, etc.), student achievement will improve.

Text-Dependent Analysis Multi-Group Study (grades 4, 6, 8): Group 1 – multiple-year group of teachers receiving PD Group 2 – first year group of teachers receiving PD Group 3 – first year group of teachers receiving no PD

Data Collection Efforts Three TDA prompts completed independently by students throughout the year Scored by teacher Back-scored by expert Classroom observations with observer protocol Focus group meeting with each group of teachers State assessment scores on TDA item

Independent TDA prompts Key questions to answer: Does student performance on TDA items vary by research group? Does student performance improve throughout the year? Does achievement improve differentially for teachers in the different research groups? Does teacher scoring agree with expert scoring? Does teacher use of the rubric improve with additional professional development? Do patterns in student performance on the TDA prompts for the research study hold when analyzing the PSSA data?

Classroom Observations Strong Clear focus and intended outcome Purposeful lesson plan that provides pathway to achieving goal Teacher has deep understanding of text and prompt Nuanced feedback and response to student interactions Students are highly engaged Moderate Elements of strength with execution lacking in some way Fast-paced questioning Underestimation of student ability –surface-level questions or low-level text Teacher-centered Weak Activity-driven rather than objective-driven Flow or logic of lesson structure difficult to follow Weak connection to TDA prompt Questioning is unrelated to moving students forward in their thinking Classroom management is disruptive to student learning

Focus Groups Groups 1 & 2 Group 3 What worked well and what didn’t? How had their instruction changed? How have they shared this knowledge? Groups 1 & 2 What do they know about TDAs? What PD experiences have they had if any? What types of resources would be helpful? Group 3

Standardized Test Criterion Do the patterns in the classroom-administered TDAs hold for the state standardized assessment? How has student achievement on the PSSA TDA item changed since last year for the students of the teachers in the study? What factors can explain differences in student achievement? Does the PD have an impact on student assessment scores?

Resource Modules Grade-level specific learning progressions Annotated and scored student work samples Exemplar lesson plans

slyons@nciea.org Page 10 • Thompson • CCSSO-TDA • 2016 Page 10 •