Closing the Loop: A case study in the use of formative program evaluation to direct the successful growth of an intensive summer research program 5-WEEK SUMMER RESEARCH PROGRAM Research Components Small groups conduct laboratory research with a faculty member (40 hrs/wk) Two program-wide research meetings End of program professional-style conference in which students make PowerPoint presentations Social Components Residence in apartment suites with access to all campus facilities including a gym and an art museum. Field trips involving hiking, canoeing, and visits to an art museum and a sculpture garden.. Karen Singer-Freeman, Ronnie Halperin & Joseph Skrivanek ANNUAL FORMATIVE EVALUATION METHODOLOGY Individual interviews with each student Focus group meeting with faculty End-of-the-summer student survey Direct observation of key program events Reports include findings, conclusions, and recommendations THREE PROBLEMS IDENTIFIED Problem #1:Academic Readiness Students enter program with limited coursework in science Faculty expect students to be more prepared Corrective Actions Faculty review of student applications leads to appropriate lab assignments and the development of appropriate projects Spring orientation program begins to prepare students for summer research program Faculty presentations during first week of program provide conceptual introduction to all projects for all students Outcomes Faculty report greater satisfaction with the students in their lab groups Students report that they feel more prepared for the program Problem #2: Student Engagement in Science: Students lacked vision of selves as serious students as manifested by: Attendance and behavior problems Engagement at research meetings Corrective Actions Contractual agreements clearly describing obligations Proactive instructions about desired group behaviors Integration of program alumni to serve as role models Outcomes Virtual elimination of attendance and behavioral problems More active participation by students in research meetings Increased student productivity as evidenced by conference participation Problem #3: Navigating Route through College Students lack roadmap to becoming college graduates and professionals Uninformed about the admissions and financial aid systems Unaware timeline for successful transfer Unaware of career opportunities Corrective Actions Creation of a new faculty position dedicated to advising and mentoring Each student is provided with: Individual advising Workshops on selecting appropriate 4-year-institutions, applying and transferring, selecting majors, financial aid, and careers in STEM fields Outcomes The majority of students reported the workshops were enriching and enjoyable. CONCLUSIONS The use of formative evaluations, not simply to document process, but to guide program modification, has the as yet unrealized potential to effect meaningful change. This presentation demonstrates how closing the loop through formative evaluation contributes to the development of a successful program. Baccalaureate and Beyond Summer Research Program 4-Year Liberal Arts College Purchase College, SUNY 6 Community College Partners Outcomes over nine years The program has enrolled more than 200 students from surrounding community colleges. 42% of program participants transfer to Purchase 33% of participants are currently pursuing post-baccalaureate work Rates of degree completion for all participants are significantly higher than national averages for community college students Supported in part by NIH grant #06210 and NSF grant #