1. What’s So Great About Undergraduate Research? Recent Assessment Results Linda Rueckert, Northeastern Illinois University Kathy Morgan, Wheaton College, Norton MA Presented to the Council on Undergraduate Research, June, 2006

2. What do we want (or need) to know about the impact of undergraduate research? Why?

3. Formative Vs. Summative Evaluation Robert Stakes: "When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative."

4. Approaches to Assessment Authentic Assessment Often more qualitative than quantitative Portfolio Assessment Often more qualitative than quantitative Exit Interviews/surveys Can be both qualitative and quantitative

5. Approaches to Assessment Using standardized or other tests Pretty much quantitative Using existing data (GRE scores, GPA, grad school application success, retention rates, number of sciences courses taken, etc.) Pretty much quantitative

6. When we do assessment, what do we want to find out? Often that will influence HOW we assess….Often that will influence HOW we assess….

7. What We Know Now…. Impact on Grad School Application and Success: In general, undergraduate research experience is associated with increased interest in or pursuit of graduate or professional school (although there are some exceptions….) Ex……  Lopatto, 2004: in a sample of over 1000 undergrads participating in research at 41 universities and colleges, >90% reported that participation in research sustained or increased their interest in postgraduate work  Ishiyama, 2002: political science students who participated in collaborative research with faculty were more likely than others with similar grades to attend graduate/professional school  Nnadozie et al, 2000: For one sample of low-income, first-generation, or under- represented minority students who had been participants in a research internship, only a weak relationship was found between that participation and later graduate school success. The authors postulate that 3 key elements of grad school success—preparation, presentation, and publication—are often missing from research internship experiences, and that these elements may be essential for this population.

8. Impact on student skills: Participation in undergraduate research is associated with improvements in a number of different kinds of skills (although mentors differ from students in their opinions of gains, and female students differ from male students) Ex…  Ishhiyama, 2002: Political science students who participated in collaborative research with faculty scored higher on a nationally normed discipline-specific exit exam than did peers in the same discipline.  Lopatto, 2004: Students who participated in research reported gains in research-specific learning outcomes (ex. Learning lab techniques, learning to work independently, understanding the research process) What We Know Now….

9. Impact on student skills: Participation in undergraduate research is associated with improvements in a number of different kinds of skills (although mentors differ from students in their opinions of gains, and female students differ from male students) Ex…  Bauer, 2001: Student participation in research was correlated with increased scores on a standardized test of critical thinking, but only for science/math/nursing/chemistry/engineering majors who participated in 800 or more hours of undergraduate research  Kardash, 2000: Both faculty mentors and undergraduate research interns reported improvements in some kinds of research skills (although students reported more and larger gains than did their faculty mentors). Female students rated gains in their ability to understand concepts in their field lower than did male students (similar finding for ability to formulate research hypotheses) What We Know Now….

10. Impact on student skills: Participation in undergraduate research is associated with improvements in a number of different kinds of skills (although mentors differ from students in their opinions of gains, and female students differ from male students) Ex…  Bauer & Bennett, 2003: Alums who had participated in a resource-providing URP reported greater benefits from that experience than did other research students (ex. In skills such as speaking effectively, finding information on one’s own, etc.) What We Know Now….

11. Impact on retention: Participants in undergraduate research are more likely to remain in college Jonides et al., 1992: Minority sophomores who had participated in research as first year students were more likely to still be enrolled in upper division coursework than minority or white peers, or than other minority students who had applied but not been placed in research internshipsJonides et al., 1992: Minority sophomores who had participated in research as first year students were more likely to still be enrolled in upper division coursework than minority or white peers, or than other minority students who had applied but not been placed in research internships What We Know Now….

12. Asking Students About their Experiences….Asking Students About their Experiences…. What We Know Now….

13. Undergraduate Research Survey http://www.neiu.edu/~lruecker/research_surveyhttp://www.neiu.edu/~lruecker/research_survey Learning statements from Lopatto et al.Learning statements from Lopatto et al. Project info: field, duration, advisor, etc.Project info: field, duration, advisor, etc. Data collected Aug. 2005 – May, 2006Data collected Aug. 2005 – May, 2006 N = 726N = 726

14. Demographics Students: 61% female, 39% maleStudents: 61% female, 39% male Advisors: 36% female, 64% maleAdvisors: 36% female, 64% male 25% required projects, 75% optional25% required projects, 75% optional 61% received a stipend or payment61% received a stipend or payment

15. Race / ethnic group

16. Class

17. Field

18. Project was done as…

19. The idea for the project was…

20. Source of the idea made no difference for: student gender.student gender. overall ratings of learning.overall ratings of learning. overall satisfaction ratings.overall satisfaction ratings. satisfaction with advisor.satisfaction with advisor. duration of the project.duration of the project. publication.publication. plans to go to grad. school.plans to go to grad. school.

21. Students who came up with the idea on their own were more likely to: be a senior or have graduated.be a senior or have graduated. be in the social sciences.be in the social sciences. have a female advisor.have a female advisor. not get paid.not get paid. give an oral presentation.give an oral presentation. write a paper.write a paper. be working on a required project.be working on a required project.

23. Ratings of learning

25. Source of the idea made no difference in ratings for: understanding the research process.understanding the research process. understanding how scientists think.understanding how scientists think. learning to work independently.learning to work independently. learning to tolerate obstacles.learning to tolerate obstacles. learning ethical conduct.learning ethical conduct. learning to analyze data.learning to analyze data. learning to interpret results.learning to interpret results. feeling prepared for more research.feeling prepared for more research. career clarification.career clarification.

26. So what is left? What remains to be known, or do we need to know more about?

27. How should we best try to find the answers to our questions? (Where to from here?)