The Evolution of Standards-Based Data Driven Instruction Summer DATAG July 2019

Bill Haltermann Pam Roberge

Agenda The importance of state tests as sources of standards-based diagnostic data What have we seen as the impact of (ELA) Data Driven Instruction over the past 5 years

Essential Question Why is it so important to analyze state test information/data to measure students’ critical thinking skills?

Some Ingredients For DDI Success Standards based Focus on foundational skills Diagnostic testing Use appropriate benchmark analysis Must reflect appropriate level of instructional rigor Deep understanding of state assessments Combine analysis and teacher professional development

DDI is Complicated

Common Core Misalignments Affecting DDI Changing standards and assessments put a focus on remedial teaching and scaffolding that elementary teachers were unprepared for Disconnect in many classrooms between instructional rigor and the level of rigor students find on state tests Skills versus scores data (are state tests accountability measures or instructional aids?)

NY State ELA Assessments Mapping is critical for DDI

Notice the design differences in the 2018 tests versus the 2017 tests Foundational skills are embedded in standards 1-3 (Power Indicators) Notice the change in standards distribution from year to year

The understanding of foundational skills is critical when it comes to DDI

ELA Standards Mapping Is Not Enough – BAP Annotations Identify Sub-Skills

Focuses on sub-skills and testing formats These details will become more important with the Next Gen Standards Notice that many questions require multiple skills

Do the standards define the level of instructional rigor?

No It’s the NYS Assessments that define the level of rigor

Without the state assessments to define rigor, skills instruction could vary widely from district to district, building to building or classroom to classroom

Is there diagnostic state assessment data we can use to target skill gaps? Skills versus Scores Data

Scores Data

Measuring skills performance is done using benchmark analysis Why is benchmarking so important for data analysis?

BAP released questions annotations Performance benchmark gap analysis Notice the S2 skills deficits: Grades 3-6: summarization Grade 6: theme/theme development

How can state assessments helps us instructionally?

What State Assessments Provide What skills are being tested, how the skills are being tested and the grade-to-grade progression of skills Diagnostic data – District, building, grade and individual students P-Values to determine question difficulty Benchmarks Targeted Skill Deficits

Why we like the state assessments The BAP Blog page at www.bapanalytics.com

we can use state tests to help measure critical thinking skills THE GOOD NEWS we can use state tests to help measure critical thinking skills THE BAD NEWS we can only measure grade level skills

BAP SANSI (Strengths And Needs Skills Inventory) Assessments How do we know what key skills students need to backfill?

BAP SANSI Assessments Mirrors the rigor of the state tests Customized standards-based assessments Focus on below-grade level skills

SANSI + DataMate: A complete standards-based DDI system

DataMate lists all the SANSI reports under one menu item DataMate SANSI Reporting DataMate lists all the SANSI reports under one menu item

(sorted by total score) SANSI Score Report (sorted by total score) Overview of student reading/writing performance Notice 61% of students scored over 60

SANSI Targeted Skills Gap Report The largest skill deficit: inferencing character traits Skills sorted by standard and performance gap

List of students grouped by skill missed

What have we learned over the past 5 years?

Common Instructional Issues Understanding and using benchmarked state test skills data Rigor Gap between state tests and instruction Writing (both Short and Extended CR) Creative versus academic writing

Outcomes Understanding and consistent use of key definitions Topic versus main idea versus central idea Message versus theme Improved whole group instruction Using graphic organizers to model thinking skills processes Better short response instruction Use of graphic organizers/mnemonic devices

Evolving DDI Trends Extended response writing Interaction of standards Short response writing seems to have improved Better understanding of scoring Interaction of standards Multi-step/multi-skill questions Incorporating standards 4 and 5 Small group (differentiated) instruction Use of DDI in RTI Transition to Next Generation standards

Best Practices DDI Connects Data Analysis To Classroom Instruction

Integrated DDI Level of rigor on assessments need to align to classroom instruction Professional development needs to be targeted based on data analysis Professional development should include strategies and scaffolding to address specific instructional needs Analysis and professional development need to happen right after testing to get feedback to teachers as soon as possible

DDI Outcomes Increased Student Achievement Teacher Engagement Deep analysis of released questions Having student state test data for teachers’ current students Ability to target below grade level skills Whole class instruction, Small groupings, Individual AIS and special education students Standards-based strategies and scaffolding Increased Student Achievement

DDI in the NextGen Environment Impact of new ELA standards is small Understanding sub-skills and the progression of skills will remain important It’s an opportune time to review curriculum to align to standards Appropriate focus of time and resources on foundational critical thinking skills Assessments will continue to have multi-step, multi-skill questions Align RTI to standards-based DDI

DDI Research

University Collaborators Dr. Katherine Schiller – SUNY School of Education Dr. Francesca Durand – The Sage Colleges School of Education

Research Objectives Based on DDI - why and how do successful teachers change their classroom instruction Using psychometrics to determine the efficacy of an intervention or educational practice (“effect size” of DDI)

Designing a Diagnostic Assessment of Accumulated ELA Skill Deficits to Inform Instruction: A Pilot Project AERA 2017- San Antonio Francesca Durand, The Sage Colleges, Albany, NY Kathryn Schiller, University at Albany, SUNY Pam Roberge, Capital Area School Development Association, Albany, NY Bill Haltermann, CASDA, Albany, NY

Some districts are looking for quick fix approaches to boosting student scores “We’ve found by applying just the tiniest bit of electric shock, test scores have soared”

There Are No Quick Fixes For School Improvement Only Standards-Based Data Driven Instruction

QUESTIONS www.bapanalytics.com

For More Information Contact: Bill Haltermann bill.haltermann@bapanalytics.com Pam Roberge pam.roberge@bapanalytics.com