Using Student Data as a Basis for Feedback to Teachers Ronnie Detrich Wing Institute Cal-ABA, 2011

Everybodys Talking Teacher accountability often offered as solution for education troubles. Numerous proposals to pay and evaluate teachers based on student performance. o Proposals usually linked to performance on high stakes tests. Very weak means of influencing teacher behavior.

Difficulties with High Stakes Tests as Basis for Feedback Low frequency o Once per year. Long delay between behavior and feedback. o Test in spring, feedback following fall. No direct link between teacher behavior and feedback. o Feedback is about student performance. o Does not specify what teachers do to influence student scores.

Alternatives to High Stakes Tests Formative assessment o Curriculum-based measurement o Direct measures of behavior. Characteristics o Frequent measurement o Easy and brief o Reliable and valid.

Why Feedback to Teachers is Important Hattie (2009)

Function of Feedback Systems Assure that all students are benefitting from educational services. Student performance primary source of feedback to educators. o There are constraints.

What Should be Targets for Feedback Logistically difficult to provide feedback on all aspects of teacher behavior. Student performance influenced by variables not under teacher control: o Curriculum-usually selected at another level. If curriculum is ineffective then inappropriate to hold teacher accountable. Effectiveness of curriculum can be measured by student performance but only implemented with integrity. How well educators implement an intervention is primary target for feedback.

Mortenson & Witt, 1998

Importance of Treatment of Integrity Students cannot benefit from interventions they do not experience. Unless educator knows what they are supposed to be doing they cannot do more of it.

Dimensions of Treatment Integrity (Dane & Schneider, 1998) Exposure (Dosage): the extent to which participants are exposed to the intervention as prescribed. o Curricula usually prescribe frequency and duration of exposure that is necessary for benefit. Ex: 3/week for 30 minutes/session. o Failing to satisfy either can impact student benefit. Ex: 1/week for 30 minutes.

Dimensions of Treatment Integrity (Dane & Schneider, 1998) Adherence: the extent to which the components of an intervention are delivered as prescribed. o Most common measured dimension. o It is necessary but not sufficient to produce benefits. Adherence with low dosage not likely to produce positive outcomes.

Dimensions of Treatment Integrity (Dane & Schneider, 1998) Quality of delivery: qualitative measure of how well the intervention is implemented. o Applied behavior analysts acknowledge importance. Have not developed good measures. o Possible measures through social validity methods: Enthusiasm Sincerity Variations in inflection and content of speech.

The Role of Student Data? Student data provides feedback about progress. If we know about adequacy of treatment integrity then can make decisions: o Adequacy of intervention o Adequacy of implementation If implementation is inadequate then focus should be on improving educator behavior. If implementation is adequate then focus should be on changing intervention so student can succeed. Decisions can be made about increasing or decreasing intensity of intervention.

Grade Level Standard Aim Line Trend Line Visual Aids Can Enhance Data Interpretation

Taking Treatment Integrity Measurement to Scale Insufficient resources to frequently observe all educators working with students to determine adequacy of implementation. One solution: Measure student behavior and analyze at different units of analysis: o School o Classroom o Individual student Any level there is gap between obtained and expected performance assess levels of treatment integrity.

Taking Treatment Integrity Measurement to Scale Requires that data are routinely collected across all levels. o RtI: CBM on all students 3/year. o PBS Continuous measurement of Office Discipline Referrals School-wide Evaluation Tool (SET) measures treatment integrity at the school level. Schools with scores higher than 80% greater impact on social behavior than schools that do not exceed 80%.

Implications of Providing Feedback to Educators Moves sole responsibility for student outcomes from teacher to broader system. If teachers are to effectively implement: o System selects effective interventions for teachers to implement. o Assure that implementers are actually trained. o System must monitor teachers implementation of intervention. o Assure that resources are to present to implement effectively.

Thank you Copies of all presentations may be downloaded at winginsitute.org