Measuring Differences in the Quality of Professional Development Symposium - Evaluating the Quality of Professional Development: Implications for Districts and States Measuring Differences in the Quality of Professional Development Michael S. Garet Kwang Suk Yoon American Institutes for Research Andrew C. Porter Vanderbilt University

Focus How to conceptualize and measure the full set of professional development experiences districts offer and teachers participate in

Levels of analysis District portfolio of Professional Development (PD) Specific PD activities Specific time-periods during which PD takes place (months, semesters) Teacher collection of PD experiences

Levels of analysis (continued) District Portfolio Other PD A1 A2 A3 A4 tp1 tp2 tp1 tp2 tp1 tp1 tp2 tp3 Teacher 1 PD Teacher 2 PD

Features of PD activities Structural features Type Duration (contact hours and span) Collective participation Core features Content focus Active learning Coherence

Design of MSP evaluation 4 MSP projects Within each project, 2 groups of teachers Treatment group: teachers targeted for MSP participation Comparison group: matched teachers or schools

Design of MSP evaluation (cont) Data on professional development collected through Professional Development Activity Log (PDAL) teachers provide 1 log on each activity each month data collected for 15 months (July 2003 - October 2004)

Professional Development Activity Log (PDAL)                                                                           Professional Development Activity Log (PDAL) The PDAL is a web-based, self-administered, longitudinal data collection tool for teachers to record their professional development experiences in detail with the assistance of a series of structured prompts Teachers log on to their password-protected web account and fill out their PDAL at regular intervals Visit www.PDAL.net for more information

PDAL data structure (teacher by activity by month)

Contact Hours (teacher by activity by month)

% of months with PD (math) 0=none, 1=little (10% or less of instructional time), 2=some (11-25%), 3=moderate (26-50%), 4=considerable (51-75%), and 5=almost all (more than 75%).

Contact hours by calendar month 0=none, 1=little (10% or less of instructional time), 2=some (11-25%), 3=moderate (26-50%), 4=considerable (51-75%), and 5=almost all (more than 75%).

Contact hours per month (math) 0=none, 1=little (10% or less of instructional time), 2=some (11-25%), 3=moderate (26-50%), 4=considerable (51-75%), and 5=almost all (more than 75%).

Topic intensity (science) 0=none, 1=little (10% or less of instructional time), 2=some (11-25%), 3=moderate (26-50%), 4=considerable (51-75%), and 5=almost all (more than 75%).

Next steps 1. Exploring teachers’ PD experiences Is there a relationship between PD participation and change in instructional practice and achievement? Which teachers receive high-quality PD?

Next steps (cont.) 2. Using the PDAL to examine specific PD activities and portfolios. Do MSP activities have the intended features? Are some activities of particularly high quality? 3. Examining the reliability and validity of the PDAL

Contact Information Michael S. Garet mgaret@air.org Kwang Suk Yoon ksyoon@air.org Andrew C. Porter andy.porter@vanderbilt.edu Visit us www.PDAL.net