Professional Development Activity Log: A New Approach to Design, Measurement, Data Collection, and Analysis AERA Annual Meeting San Diego April 13, 2004.

Slides:



Advertisements
Similar presentations
Co-Teaching as a Model of Student Teaching: Common Trends and Levels of Student Engagement Co-Teaching as a Model of Student Teaching: Common Trends and.
Advertisements

Title I Directors Conference Sept 2007 Carol Diedrichsen Gwen Pollock Surveys of the Enacted Curriculum for English.
Performance Assessment
Systems of Large-Scale Professional Development Sean Smith Horizon Research, Inc.
PAYS FOR: Literacy Coach, Power Hour Aides, LTM's, Literacy Trainings, Kindergarten Teacher Training, Materials.
© 2004 CCSSO ALL RIGHTS RESERVED. Findings on Alignment of Instruction using Enacted Curriculum Data Rolf K. Blank Director, Education Indicators
April 6, 2011 DRAFT Educator Evaluation Project. Teacher Education and Licensure DRAFT The ultimate goal of all educator evaluation should be… TO IMPROVE.
Barbara Miller Education Development Center Implementing a Teacher Leadership Program with Sustainability as a Goal: Lessons from Math/Science Partnerships.
Professional Learning in the Learning Profession: A Status Report on Teacher Development in the United States and Abroad Stephanie Hirsh Executive Director.
Instrument Development for a Study Comparing Two Versions of Inquiry Science Professional Development Paul R. Brandon Alice K. H. Taum University of Hawai‘i.
Fred Gross Education Development Center, Inc.
John Smithson, WCER Carolyn K. Eastman, CCSSO Building Knowledge of Instructional Alignment using SEC.
Introduction to Surveys of Enacted Curriculum Tools for School Improvement Oregon SEC Collaborative SEC 101a.
What Can Districts and Schools Do to Make Professional Development Work? Andy Porter Vanderbilt University June, 2004.
What is Effective Professional Development? Dr. Robert Mayes Science and Mathematics Teaching Center University of Wyoming.
Professional Development Activity Log: Comparing Teacher Log and Survey Approaches to Evaluating Professional Development AERA Annual Meeting Montreal,
ENHANCE Update Research Underway on the Validity of the Child Outcomes Summary (COS) Process ECO Center Advisory Board Meeting March 8, 2012 Arlington,
Experiences and requirements in teacher professional development: Understanding teacher change Sylvia Linan-Thompson, Ph.D. The University of Texas at.
Longitudinal Study to Measure Effects of MSP Professional Development on Improving Math and Science Instruction: Year 3 Update Report MSP-RETA Project.
Developing teachers’ mathematics knowledge for teaching Challenges in the implementation and sustainability of a new MSP Dr. Tara Stevens Department of.
Content Analysis of Common Core State Standards: Initial Findings Webex: September 20, 2010.
Illinois MSP Program Goals  To increase the content expertise of mathematics and science teachers; 4 To increase teaching skills through access to the.
GTEP Resource Manual Training 2 The Education Trust Study (1998) Katie Haycock “However important demographic variables may appear in their association.
HOW TO DO A STATE LONGITUDINAL EVALUATION MATH AND SCIENCE PARTNERSHIPS PROGRAM FEBRUARY 2011.
Reaching for Excellence in Middle and High School Science Teaching Partnership Cooperative Partners Tennessee Department of Education College of Arts and.
Establishing a Culture of Mathematics Learning in Urban Schools Syracuse City School District / Syracuse University Partnership Beyond Access to Math Achievement.
The Science Content, Conceptual Change, and Collaboration (SC4) Mathematics and Science Partnership - Maine U.S. Department of Education MSP Regional Meeting.
An Orientation of the Surveys of the Enacted Curriculum Ohio ELL Project Carolyn Karatzas Technical Assistance Provider SEC ELL Grant.
Longitudinal Study to Measure Effects of MSP Professional Development on Improving Math and Science Instruction.
Research Indicators for Sustaining and Institutionalizing Change CaMSP Network Meeting April 4 & 5, 2011 Sacramento, CA Mikala L. Rahn, PhD Public Works,
SEC Survey Administrators’ Training and Teacher Orientation Presenter Name Presenter Title Month and Year.
Teacher Algebra Network: Our Model for Professional Development in Three Rural North Carolina Counties Presented by Katie J. Mawhinney and Tracie McLemore.
SEC Training and Teacher Orientation for Taking the Surveys Online Put date here Oregon SEC Collaborative SEC 201.
2008 TEACHER ORIENTATION. Minnesota Department of Education Minnesota Service Cooperatives Council of Chief.
Empirical Study of the Effects of Professional Development on Improving Mathematics and Science Instruction MSP Goal 3 NSF/RETA Study Website:
Math and Science Partnership Program Approaches to State Longitudinal Evaluation March 21, 2011 San Francisco MSP Regional Meeting Patty O’Driscoll Public.
AERA Annual Meeting 2004, San Diego April, Optimizing Evaluation Quality & Cost Effectiveness Evaluating the University of Texas Master Teacher.
PRIMES Partnerships and Research Investigations with Mathematicians, Engineers, and Scientists Professional Development Model MSP Regional Meeting February.
THINKING ABOUT MATHEMATICS INSTRUCTION (TMI) A Study of Elementary and Middle School Principals’ Leadership Content Knowledge Education Development Center.
Exploring Relationships between Professional Development, Teacher Practice, and State Content Standards By John L. Smithson, Ph.D. Director, Measures of.
Tim Brower Professor & Chair Manufacturing & Mechanical Engr. Oregon Institute of Technology MSP Regional Meeting, San Francisco, February 14 & 15, 2008.
Integrated Document Analysis Workshop Peter Worth, Assessment Standards & Development Services, WestEd, CA John L. Smithson, Director, Measures of the.
11 Report on Professional Development for and Update Developed for the Providence School Board March 28, 2011 Presented by: Marco Andrade.
November 15, Regional Educational Laboratory - Southwest The Effects of Teacher Professional Development on Student Achievement: Finding from a Systematic.
Professional Development for High-Poverty Schools Joseph F. Johnson, Jr., Ph.D. MSP Conference January 10, 2007 Phoenix, AZ.
Goals for Webinar: Applications of SEC Alignment Analysis
Introduction to Surveys of Enacted Curriculum Presentation: Introduce SEC to Educators [Enter place and date]
National Science Foundation Mathematics & Science Partnerships Program (MSP)
© 2005 CCSSO ALL RIGHTS RESERVED. Tools for Aligning Instruction, Standards, & Assessments SEC Project Planning & Leader Training February 2005 SEC State.
Survey of Enacted Curriculum— © 2004 CCSSO All rights reserved. Surveys of Enacted Curriculum: An Introduction to Accessing and.
School Accreditation School Improvement Planning.
Passport to Science MSP Science Program Indianapolis Public Schools.
STEM MI Champions SURVEYS OF ENACTED CURRICULUM Teacher Orientation Gary Money August 2 and 3, 2011.
Mathematics Performance Tasks Applying a Program Logic Model to a Professional Development Series California Educational Research Association December.
AIM: K–8 Science Iris Weiss Eric Banilower Horizon Research, Inc.
Statewide Evaluation Cohort 7 Overview of Evaluation March 23, 2010 Mikala L. Rahn, Ph.D.
Learning in Physical Science (LeaPs) Administrator Session Kim Zeidler-Watters – UK PIMSER 1/21/2010.
Content Analysis Workshop John L. Smithson, Director, Measures of the Enacted Curriculum Alissa Minor, Projects Manager, Measures of the Enacted Curriculum.
North Carolina MSP Data Collection Center. Primary Purposes To create a database application containing common information for all MSP’s across NC To.
June 25, Regional Educational Laboratory - Southwest Review of Evidence on the Effects of Teacher Professional Development on Student Achievement:
What if… You could use data to guide professional development? You could have consistency across grade levels? You could know how well aligned your state.
Southern Regional Education Board Annual Leadership Forum
Measuring Differences in the Quality of Professional Development
Claudia Flowers, Diane Browder, & Shawnee Wakeman UNC Charlotte
Introduction to the SEC
K–8 Session 1: Exploring the Critical Areas
Exploring Relationships between Professional Development,
Common Core State Standards May 2011
Information July 15, 2015.
Claudia Flowers, Diane Browder, & Shawnee Wakeman UNC Charlotte
Presentation transcript:

Professional Development Activity Log: A New Approach to Design, Measurement, Data Collection, and Analysis AERA Annual Meeting San Diego April 13, 2004

Longitudinal Study to Measure Effects of MSP Professional Development on Improving Math and Science Instruction Math and Science Partnership

A collaborative study conducted by: Council of Chief State School Officers (CCSSO) American Institutes for Research (AIR) Wisconsin Center for Educational Research (WCER)

Authors Kwang Suk Yoon, AIR Reuben Jacobson, AIR Mike Garet, AIR Bea Birman, AIR Meredith Ludwig, AIR

Research Questions To what extent is the quality of the professional development supported by MSP activities consistent with research-based definitions of quality (e.g., content focus, active learning, coherence, collective participation, and sustained efforts) (Garet et al., 2001)? What effects do teachers' professional development experiences have on instructional practices and content taught in math and science classes? Are high-quality professional development activities more likely than lower-quality activities to increase the alignment of instructional content with state standards and assessments?

Logic Model Implementation of Professional Development Content Focus Collective Participation; Active Learning; Coherence; Sustained Effort Pre-PD: Alignment of Instruction with Content Standards; Instructional Practice Teacher Characteristics: Background Variables, Prior PD Experiences Target Class Students: Diversity School Culture: Trust Post-PD: Alignment of Instruction with Content Standards; Instructional Practice Year 1Year 2Year 3Year 0 PDAL Survey of Enacted Curriculum wave 1 Survey of Enacted Curriculum wave 2

Participants Four MSP projects were selected for the study. In each project, we are collecting data with teachers mostly in middle schools or middle grades about their professional development in mathematics and science education. N=472 teachers

Survey of Enacted Curriculum (SEC) Instructional practice (e.g., instructional time in target class) Content coverage and alignment: –Instructional time on topics and subtopics –Expectation for students (e.g., memorize facts, perform procedure, or solve non-routine problems) Past experiences in professional development Teacher characteristics (e.g., gender, teaching experience)

Why PDAL? Gathers accurate, time-sensitive information; Minimizes recall problem with retrospective reports Collects disaggregate information about specific PD activities – Reduces bias introduced by gross data aggregation Generates context sensitive questions Is able to tailor technical assistance to teachers based on their response patterns Allows teachers to review their own logs – Teachers can reflect on their own PD experiences

Professional Development Activity Log (PDAL) Teachers create an ongoing monthly log of any professional learning activity in which they participate Longitudinal data collected over 15 months Web-based, self-administered log Aligned with SEC (e.g., content coverage) Inclusive approach to professional development –Includes MSP-sponsored and non-MSP-sponsored activities –Documents one-time and recurring activities –Captures both formal and informal activities

PDAL Entries Name of activity Number of hours spent on each activity and its duration Whether the activity is a one-time or continuous event (e.g., recurring over a number of months) Type of activity (e.g., workshop, summer institute, study group) Purpose of activity (e.g., strengthening subject matter knowledge) PD quality features (e.g., active learning, coherence, collective participation) Content focus (e.g., algebraic concepts: absolute values, use of variables, etc.) Instructional practice – instructional topics covered in each activity (e.g., use of calculators, computers, or other educational technology)

Analysis of PDAL Data Implementation analysis –Patterns of responses to monthly logs –Response rates; sample attrition; extent of missing data Descriptive analysis –Patterns of teachers’ PD experiences –Correlates of high-quality PD activities –Latent classes of teachers based on their PD experiences Impact analysis –Assess the impact of PD on math & science instruction

Table 1: PDAL Data Structure: Disaggregated log data: Teacher by activity by time

Table 2: PDAL Data Structure: Activity-level data aggregated across teachers

Implementation Analysis # of Activities (over 9 months) –Range: –Mean: 3.1 # of Logs (over 9 months) –Range: –Mean: 4.9 Sessions –Per session: mean 12 min. –Per teacher over 9 months: mean of 50 min. 83% finished in one session; 13% finished in two sessions.

Implementation Analysis Timing of log entries: –Weekends are least popular. Tues. - Thurs. are most popular –Spike at the beginning of each month User understanding of the instrument –95% of logs created were fully completed Communication: –Info Packets –Phone calls, s, letter reminders –Helpline

Table 3: Duration and Contact Hours of PD Activities

Table 4: PD Quality Construct Active Learning

Table 5: PD Quality Construct Coherence

Table 6: PD Quality Construct Collective Participation

Conclusion: Revisiting the Logic Model Implementation of Professional Development Content Focus Collective Participation; Active Learning; Coherence; Sustained Effort Pre-PD: Alignment of Instruction with Content Standards; Instructional Practice Teacher Characteristics: Background Variables, Prior PD Experiences Target Class Students: Diversity School Culture: Trust Post-PD: Alignment of Instruction with Content Standards; Instructional Practice Year 1Year 2Year 3Year 0 PDAL Survey of Enacted Curriculum wave 1 Survey of Enacted Curriculum wave 2

Contact Information Kwang Suk Yoon (202) Reuben Jacobson (202) Visit us in the future