Data Analysis: Literacy NC Content Literacy Continuum Fall Symposium High Point University October 15-16, 2014

Objectives Understand the components of a comprehensive assessment system for literacy at the secondary level Identify and inventory the data you currently have available to guide decision making Learn ways to analyze student data to ensure responsive instruction

"Assessment... should be the servant, not the master, of the curriculum." Department of Education and Science in the UK, 1988

Assessment Measurement of student growth; assessment tool choice is dependent on the purpose and use of measurement results- RtI Action Network

Assessment Prior achievement is a predictor of outcomes – effect size 0.67 “our job as teachers is to mess this up, by planning ways in which to accelerate the growth of those that start behind….so before the lesson is planned, the teacher must know what students already know and can do” Hattie (2012). Visible Learning for Teachers

S e = Teacher Developmental Reverse 1.2 Negative Low High Medium Feedback d = Formative evaluation Teaching Desired Effects

S e = Teacher Developmental Reverse 1.2 Negative Low High Medium Feedback d = Feedback Teaching Desired Effects

Goals for Adolescent Literacy Instruction Increase overall levels of reading proficiency Help students who have achieved grade-level reading by the end of third grade continue to meet increasingly difficult standards in middle and high school Assist students who are reading below grade-level standards acquire skills necessary to meet those standards

Purposes of Assessment Instructional program effectiveness Ongoing monitoring of student progress Identify students needing additional instruction and intervention

Comprehensive Assessment System a 21st Century assessment system must include both formative and summative assessment includes classroom assessments, interim/benchmark assessments, and statewide assessments that are aligned to state standards.

Universal Screening Diagnostic Assessments/Pro cesses Progress- Monitoring of Interventions Progress- Monitoring of Core Instruction Outcome Assessments Interim Assessments Comprehensive Balanced Assessment System

What are we measuring? It is as easy as ABC… A- Attendance B- Behavior C- Course /Classroom Performance Early Warning Indicators

Types of Assessments TypeCharacteristicsPurpose Outcome AssessmentsAfter Instruction Measures students against standards/expectation s One x per year/course Evaluate student performance and program effectiveness Academic Examples EOG Testing EOC Testing NC Final Exams Behavior Examples Office Discipline Referrals (ODR)- End of Year/Course Attendance Reports (End of year) Retention reports

Outcome Assessments- What do you do now?

Types of Assessments TypeCharacteristicsPurpose Interim AssessmentsAdministered throughout the year after sections of instruction Administered 3-4 times per year/course Designed to predict performance on outcome assessments Academic Examples: District created or purchased benchmark assessments

Types of Assessment TypeCharacteristicsPurpose Progress Monitoring of Core Instruction Tied to content standards or building expectations To help teachers know if students have learned concepts taught To adjust instruction, reteach concepts or provide additional practice Academic Examples: Common Formative Assessments; Informal Formative Assessments; Classroom tests Behavior Examples: ODR’s per month; suspensions (OSS/ISS); attendance rates;

Interim and Formative Assessment: What do you do now?

Types of Assessment TypeCharacteristicsPurpose Universal ScreeningAll students 2-3 x per year Critical skills typically measured by curriculum- based measures of accuracy and fluency Standardized administration Quick administration Predictive of larger outcomes Designed to identify students at risk Evaluate program effectiveness and growth throughout a school year Academic Examples: K-8 typically Curriculum-Based Measures in reading and math; 9-12 historical data/ongoing monitoring of risk factors for drop-out (may also include 8 th grade CBM) Behavior Examples: Office Discipline Referral (ODR) data analyzed at preset intervals for behavior type, student, location, time, referral source; Possible formal screening for internalizing behaviors

Early Warning System Classroom/Course Performance: –Elementary: off grade level in reading or math –An inability to read on grade level by 3 rd grade –Failure in English or Math in 6 th -9 th grade –GPA of less that 2.0 –2 or more 9 th grade course failures –Not on time for promotion to 10 th grade

Middle School Indicators Academic Performance Indicators Demonstrates successful credit earning behavior Successfully passes math and ELA GPA of at or above 2.0 Course of study that places student on track for accelerated learning programs and/or college and career ready courses in HS Scores proficient or highly proficient on EOG/ EOC exams* *On time to graduate College and Career Ready

High School Indicators Academic Performance Indicators Earns sufficient credits for on-time progression and graduation GPA at or above 2.0 Participates in accelerated learning programs and/or college- and career- ready courses Successfully completes Algebra I by 9th grade Scores proficient/ highly proficient on EOC Achieves College- or Career-Ready score on ACT* *On time to graduate College and Career Ready

At Risk Report

Moving from Reaction to Prevention Schools do not have to be surprised every year by the fact that some students require support beyond initial instruction to achieve grade-level goals and expectations –The best predictor of future failure is past failure and disengagement –Disengaged students are likely to be disengaged at the next grade/school level in the absence of prevention and intervention support

Students with a history of academic underachievement or failure are likely to continue to fail without prevention and intervention support Addressing academic and engagement issues earlier rather than later is more successful and more cost effective schools/early-warning-systems-moving-from-reaction-to- prevention Moving from Reaction to Prevention

Universal Screening- What do you do now?

Types of Assessment TypeCharacteristicsPurpose Diagnostic Assessments/Processe s Used for individual students for problem analysis (why is the problem occurring) Used to plan effective interventions that target specific skills Used to determine instructional focus of intervention Academic Examples: Formal Diagnostic assessments (based on essential components of reading and/or math); Informal surveys; error analysis of student work; student interview Behavior Examples: Functional Behavioral Assessment (FBA)- including interviews and structured observations; attendance patterns; anecdotal teacher reporting

Assessments- Diagnostic What do you do now?

Types of Assessment TypeCharacteristicsPurpose Progress-Monitoring of Interventions Brief Repeatable Sensitive Alternate equivalent forms Reliable and Valid Measure Accuracy and Fluency To ensure effectiveness of interventions Measure growth over time Inform instructional decisions Measure student response Academic Examples: Curriculum- Based Measures; Computer Adaptive Testing (if standard metric is available) Behavior Examples: Specific to behavior intervention; ODR data and individual student monitoring data (point sheets, repeated structured observations, reinforcement earned, etc.)

Curriculum Based Measurement (CBM) Standardized Sensitive to improvement over time Multiple equivalent versions administered throughout the year Content reflects performance desired at end of the year

Progress Monitoring What do you do now?

So now we have all of this data, what do we do with it?

Now What? Review the data sources you have collected. Do you notice any voids or redundancies? Can you think of any barriers to implementation of a comprehensive assessment plan for literacy? What are your next steps?

Outcome Assessments/Universal Screening Is our program for all students effective? Who needs intervention? Who needs enrichment? Key questions

Interim and Formative Assessments Are all of our students progressing towards standards? Are my strategies working? Was my instruction effective? Key questions

Diagnostic and Progress-Monitoring Why is there a gap? Are we closing the gap? Key questions

Reading Comprehension Screening –All Students (Historical Data?) Assess Oral Reading Fluency Students Below Grade Level Students At Grade Level- Vocab & Comprehension in content classroom Weak Phonics Skills- Vocab/Comp in content and intensive reading intervention in all components Students at or above fluency benchmark- Vocab/Comp in content classes and possibly vocab/comp intervention Students below fluency benchmark Monitor Progress for Student Response Assess Phonics Skills Intact phonics- Vocab/Comp in content classes, possibly vocab/comprehension intervention, and fluency intervention

Next Steps Do we have the information we need to make decisions that address the literacy needs of the students in the building? How can you efficiently manage the multiple data sources? Do your team structures support ongoing analysis of student literacy data?

Questions? Lynne Loeser- NC DPI Specific Learning Disability Consultant Amy Miller- NC DPI Regional MTSS Consultant