1. Teacher Performance Assessment (TPA): Scorer Training Overview
An Overview…
by Donna Hanby

2. Setting the Context State Local National
Traditional vs. AlternativeTeacher PreparationPrograms
National
House Bill 1
State
PIII Transition to Resident Educator
Local
Waiting for Superman is gaining public knowledge and conversation about education. Higher Ed is very much part of that conversation…
House Bill 1 said that Teacher Education Institutions would be measured on a set of metrics to become public knowledge about our teacher preparation programs… (Metric Handout)

3. House Bill 1
Transfers responsibility for approving teacher preparation programs from the State Board to the Chancellor of the Board of Regents
Directs the Chancellor, jointly with the State Superintendent, to: (1) establish metrics and educator preparation programs for the preparation of educators and other school personnel, and (2) provide for inspection of the institutions.
Through HB1, Ohio is first in the nation to require a four-year induction program (Resident Educator)

5. Career Ladder Preservice Preparation Resident Teacher License
Professional License
Senior Professional License
Lead Professional License
So what do we have at the end of our five years. We will end with a model that provides a coherent apprenticeship and seamless transition throughout the four-year Ohio Residency Model, and a program that can be sustained over time.
Teacher Performance Assessment Model

6. Ohio Comprehensive System of Educator Accountability
Metrics
Performance
Outcome
More coursework or enter different area of study
Pre-Service
Not Effective
Content Knowledge: Praxis II
Performance Assessment: TPA
Effective
Recommended for resident educator license
Not Effective
Employment terminated
Teacher Residency
PAR Program
Not Effective
Formative assessment coupled with goal setting and coaching
Annual summative assessment based on multiple measures of educator effectiveness including student growth
Effective
Effective
Continue with Residency
Recommended for Five Year Professional License
Not Effective
Employment terminated
Annual Teacher Evaluation
Not Effective
PAR Program
Formative assessments that inform PD and coaching support
Annual summative assessment based on multiple measures of educator effectiveness including student growth
Effective
Informs decisions: retention, dismissal, tenure, promotion, compensation
Effective
Continue as Teacher

7. TPA Background
Three-year grant to create a National Teacher Performance Assessment
Based upon the Performance Assessment for Teacher Candidates (PACT) from California (
Co-PIs ~ Linda Darling-Hammond & Ray Pecheone
So… what about TPA … is it our Superman … here to save the day?

8. Project Partners
AACTE (American Association of Colleges for Teacher Education)
CCSSO (Council of Chief State School Officers)
Stanford University
Lead IHE in Each State (Wright State, for Ohio)
IHEs in Each State (University of Dayton, University of Cincinnati & The Ohio State University)
SEAs (ODE & OBR)

10. National Progress to Date
Commitments from 20+ participating pilot states, including State Ed Agencies (OBR/ODE) and Institutions of Higher Education IHEs
Field-based state and IHE review of the California PACT assessment methodology, upon which this initiative is based
Conducted an alignment study with INTASC teaching standards, as well as state developed teaching standards
Implemented tasks last spring in accelerated IHEs and provided feedback
Convened design team of leading measurement experts and researchers to inform the development of the TPA
It is building momentum since it’s inception in early 2009…

11. Ohio’s TPA Timeline
Ohio became one of 20+ States engaged in TPA in spring of 2009.
Ohio became one of five accelerated states in late spring of 2010.
Others are Massachusetts, Minnesota, Tennessee and Washington.
Summer 2010, Four Year timeline completed with the TPAC team at Stanford University (3 IHE reps and 2 OBR reps attended).
Nov. 2010, State TPA meeting with 47 IHEs attending.
Dec. 2010, MOUs distributed to all 51 IHEs by Chancellor (4 yr timeline) 70% returned to date.
Jan. 2011, Ohio trained 12 Trainers-of-Training.
April 2011, Scorer Training to be held (300+ assessments).

12. April – May 2011 April ~ State scorer training
April/May – calibrate scorers & assess portfolios
May 30 - Submit Results to Stanford
Fall 2011 – All Ohio IHEs will complete the full assessment cycle
Becomes high stakes in 2013 as one of House Bill 1’s Metrics for Teacher Education Institutions

13. A MULTIPLE MEASUREMENT ASSESSMENT
Embedded Signature Assessments
— examples —
The Capstone Teaching Event
Teaching Event
Demonstrates :
Planning
Instruction
Assessing
Reflecting
Academic Language
Child Case Studies
Analyses of Student Learning
Curriculum /Teaching Analyses
The Left side of the chart are the typical aspects of a student teaching experience.
Some programs do child case studies, analysis of student learning etc. and are observed,
evaluated by a supervisor and cooperating teacher and given feedback. That will stay the same.
The right hand side of the chart, (Capstone Teaching Even) is the TPA portfolio, the new part of the student teaching experience.
Observation/Supervisory Evaluation & Feedback RP 13 13 13

14. Content Assessments
There are six content assessments currently being utilized nationwide:
Elementary Literacy
Elementary Math
Secondary Math
Secondary English/Lang. Arts
Secondary Science
Secondary History/Social Science
Next Section …
Detailed information regarding the three tasks and specifics as to what is submitted by the teacher candidates, if needed

15. What do Candidates Do for TPA?
A Content-Specific Teacher Work Sample that includes:
Planning Instruction and Assessment – Task 1
Instructing & Engaging Students In Learning – Task 2
Assessing Student Learning – Task 3
Retrospective Reflection – Task 4
Academic Language
Theory into Practice application

16. What is submitted and by what means?
An electronic portfolio with the three to five days of planning, instructing ( including a minute video), assessment, and reflection
Unique IDs will be assigned to each T. candidate and calibrated scorer for each portfolio to be scored
Portfolio artifacts will be kept on an IHEs home assessment system (Chalk & Wire, Taskstream etc.) for accreditation needs AND portfolio constructed/submitted to a separate server to protect identity and to be assessed
Results returned to candidate and program end of term
ID doc and blue timeline

17. How is it completed?
During the student teaching experience in 3-5 lessons the teacher candidate will be teaching
It includes planning, instructing, assessing, and reflecting over 3-5 consecutive lessons to be taught (such as within a unit of study)
A faculty/supervisor may assist to determine a timeline to complete the portfolio
It will then be completed electronically (like other key assessments) and scored by a trained/calibrated assessor

18. Task 1
Purpose
The Planning Instruction & Assessment task asks the candidate to:
Describe plans for the learning segment and explain how they are appropriate for the students and the content being taught;
Demonstrate the ability to organize curriculum, instruction, and assessment to help diverse students meet standards for the content; and,
Develop academic language related to the content.
Evidence ~ to select, adapt, or design learning tasks and materials that offer students equitable access to the content.

19. TPA TASK # 1: Planning Instruction and Assessment
Describe school and classroom context. 
Describe students’ academic development, language abilities, social and emotional development, and family and community assets.
Describe modifications planned for students with specific learning needs.
Identify what will be taught in the content area.
Describe plans to teach content in ways to support student learning. 
Describe how student learning will be monitored while teaching. 
Identify the language needed to process and express the content.
For task 1 – Planning Instruction and Assessment… here are the areas the candidate provides evidence

20. Here is the first set of directions for Task One, within the Elementary Literacy Handbook. It is similar for all three tasks and all content areas. The left column denote the task, the center describes what you’ll do and the right side tells you what you’ll submit within the TPA portfolio, which will be done through Tk20.

21. Commentary 7-9 single-spaced pages
Summarize content focus (big idea/essential question)
Describe how knowing Ss influences choices of instructional strategies for learning
How do plans support student learning of content and academic language
Monitoring student learning during the learning segment
Reflection (research/theory guiding strategies)

22. Evidence provided: Context for Learning form
Context for Learning commentary
Lesson plans
Copies of handouts, assessments, and other materials needed to understand the learning and assessment tasks
Planning commentary
These are the forms which assist the candidate to provide evidence for the aforementioned questions.

23. Task 2: Instructing and Engaging Students in Learning
Purpose
The Instructing and Engaging Students in Learning task asks to demonstrate how one facilitates students’ developing understanding of skills and strategies to comprehend or compose text.
Evidence of engaging students in meaningful tasks, monitor understanding, and use responses to students to guide their learning.

24. Task 2 Instructing & Engaging Students in Learning
Identify lessons where students are engaged in using relevant skills and strategies to comprehend and/or compose text. One lesson is selected for filming.
Collect permission forms from parents and prepare for filming.
Video the lesson.
Review the video to identify one or two video clips that meet requirements.
Respond to commentary prompts to analyze your teaching and your students’ learning in the video clip(s).
To Submit:
 Video Clip(s) ~ Video Label Form ~ Instruction Commentary

25. Video Taping Specs Content Maximum Minutes # of Clips
Elementary Literacy 15
1-2
Elementary Math
Secondary Math 20
Secondary Science
2/10 minutes each
History/Social Science
English/Lang. Arts

26. Video Guidelines
A video clip should be continuous and unedited, with no interruption in the events. If two clips are used, the two clips must come from the same lesson.
The clip(s) can feature either the whole class or a targeted group of students.
Both the teacher candidate and their students should be visible and clearly heard on the video submitted.
Tips for recording a class on video should be available from one’s program.
Before videotaping, gain the appropriate permission from parents/guardians of the teacher candidate’s students and from adults that appear on the video.

27. Instruction Commentary
2-4 single-spaced pages
Describe strategies used to engage Ss in learning tasks to develop skills & strategies to build their understandings of the specific content.
Cite language supports seen in the clip to help Ss understand content and/or build their academic language skills.
Describe strategies for eliciting student thinking and how ongoing responses further their learning.
Reflection – student learning of concepts and academic lang. If done over, what might have been done to take advantage of missed opportunities or to improve the learning of Ss with diverse learning needs and characteristics.

28. Task 3: Assessing Student Learning
Purpose
The Assessment of Student Learning task asks to assess student achievement, diagnose student learning strengths and needs, and inform instruction.
Provide evidence to:
1) develop evaluation criteria aligned with big idea or essential question, standards, and learning objectives;
2) analyze student performance on an assessment in relation to student needs and the identified learning objectives;
3) provide feedback to students; and
4) use the analysis to identify next steps in instruction for the whole class and individual students.

29. Task #3: Assessing Student Learning
Describe the variation in student learning based on whole class and individual student performance.
Describe how your feedback specifically addressed your students’ learning needs.
Analyze what students learned based on an assessment that is aligned with the content learning goals.

30. Task 3 Commentary
Refer to lesson plans & cite specific standards/objectives that are measured by the assessment chosen for analysis
Create a summary of student learning for the whole class relative to the evaluation criteria
Discuss what most students appear to understand well, and, if relevant, any misunderstandings, confusions, or needs for greater challenge apparent for some or most. Cite evidence to support analysis from the three student work samples you selected.
For two specific Ss – strengths/challenges, conclusions from the work sample, feedback influencing their Individual needs and opportunities for them to apply the feedback given
Reflection on student performance on assessment, next steps, individual next steps for the two students

31. Task 3 Assessing Student Learning
Analyze student performance across the class from one assessment completed during the learning segment.
Identify three student work samples that illustrate class trends in student understanding.
Select two focus students from the class whose learning you will analyze in more depth, and for whom you will document feedback on their work.
Respond to commentary prompts to analyze the extent to which the whole class met the standards/objectives, analyze the individual learning of two focus students and describe your feedback to them, and identify next steps in instruction based on your analysis.
What to submit: 
Evaluation Criteria Student Work Samples
Evidence of Feedback Assessment Commentary

32. Retrospective Reflection
Reflection is incorporated throughout TPA
During Planning, Instruction and Assessing
There is also a culminating Retrospective Reflection (Task 4) incorporated at the end of the total TPA portfolio

33. Academic Language
Academic language is the oral and written language needed by students to understand and communicate in the academic disciplines for specific purposes and audiences. Academic language genres include the specialized vocabulary, linguistic features, and textual resources associated with genres within a field (e.g., literary criticism, explanations of historical phenomena, lab reports). It also includes instructional language needed to participate in learning and assessment tasks. (See Appendix A for Content-Specific Academic Language).
Details found in Appendix A

34. Scoring Scoring in accelerated states should include:
Independent scoring by scorers who have not supervised the candidate
Random selection of 10-15% of portfolios submitted for double-scoring
Double scoring of any portfolio receiving more than 4 of 11 scores at rubric level “1”
The portfolio can be a requirement to be passed during student teaching (like BCI, FBI etc.) but should not be “graded” as part of the student teaching course… Passing is determined by the calibrated scorer(s) not the instructor of the student teaching course.

35. Ohio Spring TPAC Timeline
SPRING TPAC ASSESSMENT COMPLETED IN STUDENT TEACHING (SPRING TERM) DUE TO IHE REP MAY 2
APRIL SCORER TRAINING
APRIL 29 SCORER CALIBRATION DUE DATE
APRIL 30- MAY 6 ASSESSMENT ASSIGNMENTS MADE REGIONALLY
MAY 20 RETURN COMPLETED ASSESSMENTS TO DH FROM TOT TRAINERS
MAY 21 DOUBLE SCORING SAMPLE AS NEEDED, RETURNED TO DH BY MAY 25
OHIO ASSESMENTS SENT TO STANFORD, ELECTRONICALLY AND BY DISC/FLASHDRIVE BY MAY 26

36. Thanks!
For Participating In This National Initiative!!!

37. Next Steps… Room Assignments: Check-in & meals The Capitol Room (#6)
Elementary Literacy The Worthington Room (#7)
Elementary Mathematics The Ohio Room (#10)
Secondary English-Language Arts The Westerville Room (#2)
Secondary History-Social Science The Powell Room (#1)
Secondary Mathematics The Board Room (#8)
Secondary Science The Dublin Room (#9)