Associate Professor Matthew Stubbs and Dr Carolyn Semmler, with the members of the Community of Practice on the Small Group Discovery Experience A Preliminary Evaluation of the Small Group Discovery Experience in Large First-Year Courses

Outline of Presentation Evaluation The SGDE: Why Evaluate? Evaluation Design Key Questions Results Overall Satisfaction Course by Course Responses Aggregate Assessment of Individual Features Demographic Factors Qualitative Results Best Practice Considerations for SGDE Implementation University of Adelaide

Evaluation

The SGDE: Why Evaluate? A new signature pedagogy of the University of Adelaide Emphasizes the link between the University’s teaching and its world-class research, in the context of equipping graduates of the University of Adelaide to be the creators of new knowledge throughout their lives Aims of the Project Generate representative quantitative and qualitative data about the student experience of the SGDE in 2014/15 Form the basis for more broad-based qualitative and quantitative assessments of the SGDE in the future Provide an evidence-based list of considerations for best practice implementation of the SGDE It was introduced as a requirement for first-year courses in 2014, and from 2016 is offered to each student at least once in each year of undergraduate study. University of Adelaide

Evaluation Design Methodology Courses Evaluated Online survey emailed to students through the LMS Deliberative Discussion Forum School of Psychology Human Research Ethics Sub-Committee approval 15/98 Courses Evaluated Large (300+), First-Year courses Law Computer Science Public Health Science Arts Biology Psychology Accounting 25-69 responses per course; 330 overall responses University of Adelaide

Key Questions Q 6: In general, how would you rate the quality of learning you received in the SGDE for this course? Q 7: The SGDE supported my learning experience for this topic Q 8: The SGDE was well organized Q 9: I had adequate contact time with the lecturer/academic during the SGDE Q 10: The SGDE for this course improved my knowledge of research Q 11: The SGDE for this course increased my appreciation of the research process University of Adelaide

Results

Overall Satisfaction University of Adelaide

Course by Course Responses G reason for high 7 (‘The SGDE supported my learning experience for this topic’): runs for most of the course and has specific SGDE assessment (but so does F, which is modest on this scale) D reason for low 9 (‘I had adequate contact time with the lecturer/academic during the SGDE’): D had only two SGDE sessions; score similar to B which was the other course with this structure Mean of questions 6-11 by course University of Adelaide

Aggregate Assessment of Individual Features Preference Order: Q9: Contact with academic mentors Q8: Organisation Q10: Improving knowledge of the research process Q11: Appreciating the research process Q7: SGDE supported learning Q6: Quality of learning in SGDE University of Adelaide

Demographic Factors Age Gender Groups No significant differences 18-24 years 25-34 years > 35 years No significant differences Gender University of Adelaide

Qualitative Results Positives Mixed Areas for Improvement Discussions with Staff and Peers Contact with Academics Mixed Group Work Areas for Improvement Integration with Courses Organisation University of Adelaide

Summary of Results Students generally like the SGDE Key Beacon of Enlightenment aims for the SGDE are being achieved, including increasing student contact with academic mentors, which is highly valued developing student understanding and appreciation of the research process Group work experiences vary positives include students getting to know their peers and developing a sense of cohort amongst the student body negatives focus on difficulties working within groups Integration with course structure and link to course learning outcomes important University of Adelaide

Best Practice Considerations for SGDE Implementation

Best Practice Considerations for SGDE Implementation (part 1) SGDE content should be closely linked to the content in the rest of the course, and students need to be given a clear understanding of how the SGDE is integrated with the overall course structure and linked to course learning outcomes. Where multiple groups of students are involved in simultaneous interaction with mentors, each group should be limited to 3-4 students, in classes of no more than 25-30 students in total, supported by at least two academic mentors. Students should be assisted to learn about expectations for group work, and processes should be in place to manage conflict in group assessment settings. If there is group assessment attached to the SGDE (which should be seriously considered), it should be weighted at 10%- 20% of the entire course grade. University of Adelaide

Best Practice Considerations for SGDE Implementation (part 2) Each SGDE should have at least 3 mentoring sessions (even if only one hour each, and even if mentors are shared amongst multiple groups) (fewer sessions may be possible where mentors are unique to each group of students). Ongoing feedback should be offered to students as they engage in the discovery process. SGDEs should be scaffolded across each program, with the content being appropriate to each year level. Mid- and early- career researchers, and even postgraduate students, should be considered for opportunities to academically mentor students in the SGDE (at least at the first-year level). University of Adelaide

Our thanks to the fantastic teams RAs: Dr Peta Callaghan; Heidi Long Course Coordinators (anonymous) CoP on Teaching Large Classes (2014-15) Dr Carolyn Semmler (co-facilitator) Assoc Prof Matthew Stubbs (co-facilitator) Prof Nicholas Burns Dr Shona Crabb Dr Michelle Coulson Dr Georgina Drew Dr Matthew Dry Dr Adrian Koerber Dr Virginie Masson Dr Adriana Milazzo Dr Lynn Rogers Dr Adam Webster Dr Natalie Williamson CoP on the SGDE (2016) Dr Carolyn Semmler (co-facilitator) Assoc Prof Matthew Stubbs (co-facilitator) Prof Amanda Able  Prof Nicholas Burns Jason Chan Dr Michelle Coulson Dr Matthew Dry Nel Duffield Dr Chad Habel‎ Margaret Hosking ‎Dr Kim Hynes Dr Cate Jerram Joel Lisk (Student) Dr Beth Loveys Dr Andrew MacKinnon‎ Dr Virginie Masson Dr Joy McEntee‎ Dr Danijela Menicanin‎ Dr Dee Michell Dr Adriana Milazzo Devon Milton-Hutchison (Student) Dorothy Missingham Dr Edward Palmer‎ Assoc Prof Lucy Potter Assoc Prof Catherine Snelling‎ Dr Adam Webster Dr John Willison‎ University of Adelaide