Issues Surrounding the Evaluation of Teacher Internship Programs Donna Barrett Center for Education Integrating Science, Mathematics & Computing Georgia Institute of Technology Atlanta, Georgia

…. a collaborative effort of corporations, universities and school districts designed to enhance mathematics and science experiences of Georgia teachers and their students. Since 1991, teachers have been placed in over 1100 GIFT internships throughout the state.

Goals of Increasing the teacher’s personal knowledge base of science, mathematics and technology Enhancing the teacher’s enthusiasm for teaching Providing access to new techniques and procedures Encouraging partnerships with business and education

The GIFT Experience - Commonalities Common application Internship in STEM-based work setting Emphasis on “Real World” problem solving and reasoning Development of Inquiry Based Unit – Action Plan

The GIFT Experience- Variables Location of placement  University  Corporate  Informal science/public education Goal of placement  ”Traditional”  Student research  International  Curriculum development Length of placement:  4-8 weeks

The GIFT Experience - Variables Geography  Placements across Georgia Teaching Level  High school vs. middle school Teaching Field  Science, math, technology, social science Type of school  affluent vs. low income, stable vs. changing, minority vs. majority

GIFT Evaluation Plan An Advisory Board composed of 25 members from corporations, universities, and school districts Utilizes the strategy of subcommittee phone conferences to provide guidance on program issues such as evaluation Evaluation subcommittee has provided input on program evaluation Worked with an internal evaluator to develop a GIFT Logic Model and develop assessments

Assessing Impact on Teachers Participant Reactions  Numbers  Are participants satisfied?  Teacher and Mentor Surveys Participant Learning  Action Plan Analysis  What have teachers learned about inquiry based learning? GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

Assessing Impact on Teachers Participant Actions  How have they applied inquiry based learning in their classroom?  Surveys – Pre-Summer, Post-Summer & School Year Follow-up (Post-GIFT)  Alumni Survey  Classroom Observation GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

Using inquiry and hand-on activities Pre-GIFT N=47 Post-GIFT N=24 Pre and Post Surveys Based on surveys from:

Assessing Impact on Teachers Organizational Change  Change in school infrastructure  Teachers as agents of change  Alumni Survey  Focus Group Discussions  Classroom Observation GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

GIFT Alumni Survey Searching for 668 former participants GIFT database/Districts/Websites/Searches 123 Fellows – No contact information 545 Contact Attempts: 382 & 163 mailed; 36 and/or mail returned 509 Total Sample Size Surveys returned Return Rate: 25%

To what extent, if any, do you feel that you experienced each of the following types of learning as a result of your participation in GIFT? Strongly Agree or Agree I gained a greater understanding of the applications science, mathematics, and/or technology in every day life % I gained greater understanding of fundamental concepts in science, mathematics or technology % I increased my knowledge of current issues in scientific or mathematical research % I increased my knowledge of careers that utilize science, mathematics, and/or technology % I gained an appreciation of the difficulties some students encounter when learning new material % It increased my comfort level with inquiry-based learning strategies % It increased my ability to incorporate "real life" examples of the subjects I teach %

Leadership Roles  16% reported they became a department or grade level chair person AFTER their GIFT experience  21% reported they became a school district leader (Principal, AP or Coordinator) AFTER their GIFT experience

Limitations of Data The data is self reported, although anonymous Return rate of Alumni Survey – 25% Sample sizes often vary on the pre and post surveys Focus Groups have provided more specific information

Challenges in Correlating Student Achievement with Teacher Professional Development Designing a scientifically valid study with appropriate controls Appropriate measures of student achievement across grade levels, subjects Access to student achievement data Teachers changing schools, subjects and grade levels

Assessing the Impact on Students Student Achievement  Connecting the dots through a literature review  Encouraging the use Research based best practices (Inquiry, Problem Based Learning, etc.)  Impacting teacher content knowledge GIFT Logic Model adapted from Guskey’s 5 Level Model for Assessing Professional Development

Future Evaluation Goals Collect student achievement data of GIFT participants – proposed study with a local school district to collect preliminary data  Criterion-Referenced Competency Tests (CRCT) – Middle Grades  End of Course Test (EOCT )– High School Biology, Physical Science, Algebra, Geometry  Aggregate data from specific class and compare to state average  Pre GIFT data, Post GIFT data, YR 1, YR 2 Post

Evaluation Tools/Resources Columbia Program: Industry Initiatives for Science and Math Education (IISME): RET Network Surveys: SWEPT study: TRE Conference on Research Experiences. Visiting a High School Inquiry Classroom: How to Prepare and Observe:

For more information: Donna Barrett Program Director (404)