1. Maximising student engagement in learning gain measurement
Dr Liz Austen, Nathaniel Pickering and Alan Donnelly
Sheffield Hallam University

2. www.blogs.shu.ac.uk/steer @SHU_StEER
Institutional research and evaluation
Student experience data
Student engagement
Student communications
Brief introduction to STEER
@SHU_StEER

3. NMMLG (Office for Students)
Interim evaluation highlighted a number of issues with the data collection process and a low response rate for the longitudinal sample.
The evaluation of the NMMLG project, led by Sheffield Hallam University, will continue in the autumn, working with providers and others involved in the project and focusing on improving our understanding of students’ perceptions of learning gain.
Brief overview of our role in LG (only use text from OFS announcement)
Discuss need to fully understand student engagement/motivation/support/impact (wellbeing)
@SHU_StEER

4. www.blogs.shu.ac.uk/steer @SHU_StEER
Student engagement
LEGACY analysis indicates that to be effective and ensure engagement, learning gain needs to be embedded in the curriculum or as part of developing a set of capabilities and skills that students recognise as necessary.
Define student engagement in relation to LG measures
@SHU_StEER

5. Survey Design Checklist
1. Access to survey
2. Level of student support
3. Incentives
4. Optimal length of survey
5. Timing and possible information fatigue
6. Question wording and ordering
7. Survey question formats
8. Reporting and debriefing
Do we really need
another survey?
institutional approaches to surveys (using surveys as description of quantitative LG measurement) - referred to as exercise, test etc
- VOLUME
- REPETITION (of methods and theme)
- EXPLOITATION
- some HEIs have a blanket ban without a clear business case
brief overview of SDC - developed by STEER and used as a basis for evaluation of NMMLG (CAN WE SAY THIS)
can be used more widely than LG - handout to follow at the end of the session - contact us for more info and if you want to use please feedback to tell us how
Click to animate - all discussion require the questions "do we really need another survey?" to be discussed and debated

6. Survey Design Checklist
Criterion
Sector Evidence
LG Evidence
Incentives
- response rates improve when providing a clear narrative for how scheme incentives are administered at the outset
- immediate and guaranteed reward alongside longer-term 'lottery' opportunities is optimum
Fan and Yan (2010), Cobanoglu and Cobanoglu (2003), Porter and Whitcomb (2003), Singer and Ye (2013)
- Retail voucher payment
- Charitable donation
- Prize draw (two prizes of £500 and five prizes of - £100)
- Printing credit
- Drink vouchers
- Results/personal development
- None
SDC - each criterion was reviewed to draw on sector evidence to cite best practice. All areas can be applied to LG methodologies.
literature review to build checklist - linked to evidence - linked to context (LG - but could be satisfaction, experience, engagement, emplyability)

7. How can we maximise student engagement in learning gain measurement?
Apply your knowledge of LG best practice to the criteria on your handout.
Discuss with the person next to you and design the best approach to maximise student engagement (one criterion per pair/group).
Provide handouts each with one of the eight criteria for effective survey design (if you really have to do one).
In pairs, discuss how you would operationalise this aspect of methodological design to maximise student engagement.
@SHU_StEER

8. Reflections and next steps
Thoughts on best practice?
Handout print version of SDC - Creative Commons license attribute to SHU
Publication to follow
ACCESS - Response rates for students are usually much higher if surveys are embedded within their own programmes of study, rather than seen as additional or institutional.
LENGTH - Studies show that an optimal time for students to engage in any survey is approximately 13 minutes.
Handout print version of SDC - Creative Commons license attribute to SHU
take reflections from the audience? (Time)
@SHU_StEER

9. www.blogs.shu.ac.uk/steer @SHU_StEER