Evaluating the Impact of Peer Programmes on Student Learning and Academic Attainment – A Cross-Discipline Approach Presenters: Amanda Pocklington, Bella Magner & Ferdinand Boucher Project research team: Tim Fawcett, David MacDonald, Ellis Langley & Jonny Gurr

Project Background - Literature Participation in peer-learning programmes is associated with: higher mean grades; lower failure and withdrawal rates and higher retention and graduation rates. However, the methodology used in many studies has been heterogenous (Dawson et al., 2014) Many studies reviewed use a quasi-experimental design and have required qualitative synthesis due to a lack of definition in qualifying what the ‘treatment’ of peer-learning is (Dawson et al., 2014; Keenan, 2014; Stigmar, 2016; Asgari & Carter, 2016) Some quantitative research has been conducted, however this has mainly been on small samples with inadequate methodology e.g., no randomization of groups (Asgari & Carter, 2016) or has only been on specific subjects with no comparison groups in other disciplines (Kim et al., 2013)

Methodology – Study Design Two Peer-Programmes at The University of Exeter were evaluated, which ran in the Colleges of Medicine & Health and Life & Environmental Sciences during Term 2 In addition, a pilot study in the Medical school was conducted in Term 1 to test the study design All studies followed the same design, consisting of four different protocols, which participant groups were randomly assigned to.

Methodology – Study Design Weekly pre- and post-session quizzes to evaluate participant learning gain Completed by all participants 10 MCQ questions delivered online using Qualtrics software 1min to attempt each question Confidence rating included with each question Answers provided following completion of study David screengrab showing MCQ quiz and confidence bar.

Methodology - Recruitment Recruitment of Peer Impact Mentors: Existing Peer Mentors were eligible to join the project as Peer Impact Mentors Underwent project-specific training Received remuneration Recruitment of student participants: Open to all Year 1 students in respective programmes Project information sessions timetabled where participant sign-up forms were circulated £10 voucher offered to participants completing the study

* * Results: MCQ performance Attending the introductory session increased the proportion of correct answers, but this did not reach significance (p = 0.079) * * Attending the review session led to a significant improvement in the proportion of correct answers (p = 0.023*)

* ** ** Results: confidence Attending the introductory session led to a significant increase in self-reported confidence (p = 0.047*) * ** ** Attending the review session led to a significant increase in self-reported confidence (p = 0.007**)

* Results: response speed Attending the introductory session led to significantly quicker responses (p = 0.015*) * Attending the review session had no effect on response speed

“They could share common mistakes made by themselves and their peers” Evaluation Survey Statements “They could share common mistakes made by themselves and their peers” “Comfortable environment where it feels easier to ask questions to clarify misunderstandings, PALS are very encouraging and approachable ” Ferdinand & Bella “explanations were made for/by students so they know how to effectively convey information to their audience”

Review sessions led to significant improvement in MCQ performance Conclusions Review sessions led to significant improvement in MCQ performance Introductory and review sessions led to a significant increase in self reported confidence – individually & combined Introductory session led to significantly faster response times Ferdinand and Bella

Comments Students that participate in peer programmes may have different qualities to those who do not; e.g. higher motivation and engagement in their studies Participant attrition was high across all studies. Conflicting demands on student time (e.g. coursework submissions during final weeks of Term 2) may have contributed to this The study design of delivering student-led introductory and review sessions each week was time intensive, but was valued by participating students Could this peer support programme operate without financial incentives for Peer Mentors? Ferdinand and Bella

Any questions?

References (1) Dawson P, van der Meer J, Skalicky J, Cowley K. On the effectiveness of supplemental instruction: A systematic review of supplemental instruction and peer-assisted study sessions literature between 2001 and 2010. Review of Educational Research. 2014 Dec;84(4):609-39. (2) Keenan C. Mapping student-led peer learning in the UK. York: Higher Education Academy. 2014 Nov. (3) Martin Stigmar. Peer-to-peer Teaching in Higher Education: A Critical Literature Review, Mentoring & Tutoring: Partnership in Learning. 2014; 24:2, 124-136, DOI: 10.1080/13611267.2016.1178963 (4) Asgari S and Carter Jr F. Peer mentors can improve academic performance: A quasi-experimental study of peer mentorship in introductory courses. Teaching of Psychology. 2016 Apr;43(2):131-5. (5) Kim SC, Oliveri D, Riingen M, Taylor B, Rankin L. Randomized controlled trial of graduate-to- undergraduate student mentoring program. Journal of Professional Nursing. 2013 Nov 1;29(6):e43-9. (6) Ody, M. and Carey, W. Peer Education. In: E. Dunne and D. Owens (eds). The Student Engagement Handbook: Practice in Higher Education. Bingley. Emerald Publishing. 2013.

gave more correct answers Results—Psychology study In the post-MCQ test, students who had attended the introductory session … gave more correct answers (P = 0.041) answered more quickly (P < 0.001) had higher confidence (P = 0.006) Attending the review session also led to higher confidence (P = 0.008), but did not affect answering speed or correctness

Results—Medical Sciences study Attending the introductory session did not affect confidence, answering speed or correctness Attending the review session led to higher confidence (P = 0.027), but did not affect answering speed or correctness