1. Evaluation of Teacher Preparation Programs: Purposes, Methods, and Policy Options Robert E. Floden, Michigan State University & Jeanne Burns, Louisiana Board of Regents CCSSO Webinar December 10, 2013

2. National Academy of Education report Feuer, M. J., Floden, R. E., Chudowsky, N., & Ahn, J. (2013). Evaluation of Teacher Preparation Programs: Purposes, Methods, and Policy Options. Washington, DC: National Academy of Education. Supported by NSF Award No. 1153848 Full report: http://bit.ly/193kNof

3. Purpose of Report Foster thoughtful discussion of approaches to evaluation of teacher preparation programs (TPP) Describe dimensions of variability in current systems of TPP evaluation: – Purposes – Systems – Sources of evidence Describe examples from other fields and nations Offer guidance for TPP evaluation system design and revision

4. Focus on Preparation Programs Evaluation of Programs, not Individual Teachers Most programs are part of higher education system Approximately 1,400 higher education institutions prepare teachers in the US Some teacher preparation done by other organizations, such as Teach For America

5. Purposes for Evaluating Teacher Preparation Programs Hold programs accountable Provide consumer information to – prospective TPP students – their potential future employers Support program self-improvement

6. Systems for TPP Evaluation Federal (Title II) reporting requirements National non-governmental (e.g., CAEP) State program approval Media and independent organizations (e.g., US News & NCTQ) Teacher preparation programs – individually and in networks

7. Principles Guiding our Analysis Importance of validity (i.e., defensible conclusions and desired consequences) Program evaluation is not sufficient for improvement Multiple players with varying purposes and interests Weigh limits and benefits of system Consider differential effects on diverse populations Systems should be adaptable Systems should be transparent and held accountable

8. Examples of Types of Evidence “Input” measures – Selectivity – Faculty qualifications – Quality and substance of instruction – Student teaching experience “Output” measures – Teacher tests and performance assessments – Hiring and retention – Surveys of employers – Impact on student learning

9. Typical Input Measure Strengths and Weaknesses Strengths – Some evidence relatively easy to gather – Face validity – e.g., faculty should be experts, teachers should study learning theories Weaknesses – Little empirical evidence of connection to teacher quality – Difficult to gather evidence on quality of instruction – May produce superficial compliance

10. Typical Output Measure Strengths and Weaknesses Strengths – Focuses directly on desired outcomes – Familiar to key constituencies – funders, citizens, employers Weaknesses – Many important outcomes difficult to measure – May ignore variation in institutional mission – Challenging to account for differences in incoming students

11. Strengths and Weaknesses: An output example Average Value Added of TPP graduates: – Strengths Connection to outcome valued by policy makers and employers May be used in system by which graduates are evaluated – Weaknesses Confounded with school placements Little information about change in TTP students due to program Grain size too coarse for program improvement

12. Questions to Ask When Designing or Revising TPP Evaluation Systems 1: What is the primary purpose of the TPP evaluation system? 2: Which aspects of teacher preparation matter most? 3: What sources of evidence will provide the most accurate and useful information about those aspects? 4: How will the measures be analyzed and combined? 5: What are the intended and potentially unintended consequences of the evaluation system? 6: How will transparency be achieved? 7: How will the evaluation system be monitored?

13. Teacher Preparation Transformation Guide ElementsDescriptions Teacher Preparation Concerns/Needs: Primary Purpose of Teacher Preparation Transformation: Primary Stakeholders to be Engaged in the Teacher Preparation Transformation: Aspects of Teacher Preparation Programs that Matter the Most: Characteristics of “Learner Ready” New Teachers: Initiatives that Support the Teacher Preparation Transformation:. Types of Evidence for Outcomes: Web Site Links: Reports to Guide Teacher Preparation Transformation Discussions: Tool to Guide NTEP & NAE Report Discussions

14. Screens/Assessments/Evidence for Teacher Preparation Programs Teacher Preparation Stages Types of Screens/Assessments/Evidence Current University Requirements Current State Law/Policy/Procedures Recommendations of Other Stakeholders Admission to Universities Entry Into Teacher Preparation Programs (e.g., Professional Coursework) Teacher Preparation Progress & Entry into Student Teaching Completion of Teacher Preparation Program Teacher Licensure Post-Graduate Assessment of Teacher Preparation Completers Evaluation of Teacher Preparation Programs by State and National Agencies Tool to Guide NTEP & NAE Report Discussions

15. http://www.naeducation.org/xpedio/groups/naed site/documents/webpage/naed_085581.pdf