RM-MSMSP Evaluation Presentation for Leadership Team Meeting Presented by Bonnie Walters Organized by Barbara P. Heath September 12, 2006

Evaluation and Research Collaboration Provides opportunity to Extend the evaluation process into research Share data Increase efficiency Create project-level database

Timeline: Year 3 Data Collection Combines the evaluation and research sources (denoted by shading) Includes description of: Instruments Data collection process Random sample cycle Summer camp outline Refer to timeline handout

Summer 2006 Data Collection TCI Course Packs Participant Post-Course Survey Development Team Survey Instructional Team Survey RTOP Observations Camp Surveys and Observations

Analysis In process Individual reports for each instructional team Case studies for each summer camp Summary report for Management Team

Preliminary TCI Results, SCI (results are statistically significant at the 0.001 level) Course Max Score Normalized Gain Effect Size BIO I 40 0.48 1.41 BIO II 44 0.35 1.25 ESC I 75 0.58 1.89 ESC II 43 0.91 3.8 PHY I 25 0.43 1.5 PHY II 20 0.66 2.23

Preliminary TCI Results, MATH (results are statistically significant at the 0.001 level) Course Max Score Normalized Gain Effect Size Discrete 12 0.49 1.69 Modeling 139 1.68 History 45 0.74 3.35 Geometry 100 0.45 1.18 Stats 40 0.42 0.7

Participant Post-Course Survey: Response Rates Physics I 63.64 Biology I 52.63 Earth Science I 65.00 Geometry 68.42 History of Math 61.54 Chemistry I 58.33 Physics 2 46.67 Biology 2 59.09 Algebra 83.33 Math Modeling 63.16 Chemistry 2 60.87 Statistics 66.67 Discrete Math 70.59 Math of Change 75.00 WY Earth Science 52.00

Participant Post-Course Survey Refer to handout

Participant Post-Course Survey The overall quality of this course was… 0=Poor --> 5=Excellent

Participant Post-Course Survey: % of Respondents who would recommend the course to others BIO I 100% BIO II 85% PHY I PHY 2 86% CHM I CHM II ESC I ESC II ALG 95% Math Chng 100% Discrete GEOM Hist. Math 88% Modeling Stats 71%

Participant Post-Course Survey The course material was presented at an appropriate level. 0=Strongly Disagree --> 5=Strongly Agree

Participant Post-Course Survey The course stimulated my thinking about the content of (subject). 0=Strongly Disagree --> 5=Strongly Agree

Participant Post-Course Survey Interaction with classmates assisted with my learning. 0=Strongly Disagree --> 5=Strongly Agree

Participant Post-Course Survey Refer to handout

Preliminary Conclusions All courses demonstrate increase in participant content knowledge on the individual Teacher Content Inventories (TCI). Participants report high level of satisfaction with the individual courses they completed Higher percentage of participants report they enroll in the courses to improve their teaching, lower percentage to earn stipend and graduate credit (compared to 2005 data)

Acknowledgements Dr. Aruna Lakshmanan, Statistician Aaron Perlmutter, EMEC Project Coordinator Alex Cramond, EMEC Summer Staff Brandi Whaley, EMEC Staff

Contact Information Barbara P. Heath, Ph.D. East Main Educational Consulting, LLC Wilmington, NC bpheath@bizec.rr.com (910) 620-4089