2. The Impact of Peer Learning on Assessment Literacy and Feedback Orientation
Neil Ford
Senior Teaching Fellow (Education Development)
Southampton Business School
6th June 2019 International Peer Learning Forum, Brighton

3. Context Rationale Aims Approx. 250 students across 3 programmes
Peer Learning since – based on PAL/PASS
Evolved from previous buddy scheme
Good attendance and student feedback
Good engagement from Peer Leaders
High first year failure
BTEC vs A-Level performance
Modular curriculum + Large cohorts = Poor cohort identity
Learning behaviours/ Student engagement
Student satisfaction
Adapt quickly to Uni life
Meet and work with students on the same programme
Get a clear view of expectations
Develop independent learning for HE
Develop an understanding of assessment

4. OfS Catalyst B Project – Maximising Student Success in Assessment Through a Focus on Self-Regulation
2 year Office for students Catalyst B project
Multi-institutional project researching interventions to develop student self- regulation in relation to assessment and feedback
Standard set of validated research instruments to measure assessment literacy and feedback orientation
Our sub project is looking at the impact of attending peer learning sessions on the development of assessment literacy and feedback orientation

5. Research methodology 2017-18 cohort 2018-19 cohort
October: Post-its, Demographic data, Pre ALQ survey,
November: Pre FOS Survey
February: Focus group 1, Post-its, Post ALQ survey, Post FOS survey
May: Focus group 2
Data analysis/ develop peer learning and induction
October: Post-its, Demographic data, Pre ALQ survey, Pre FOS Survey
February: Focus group 1, Post-its, Post ALQ survey, Post FOS survey
May: Focus group 2, Peer Leader Focus Group
Data analysis/ present findings

6. Lessons Learned Collecting data is hard work!
Data collection (use paper and be careful about formatting!)
Pick lectures that are going to be attended
Data is inherently skewed to students who engage
Post-it note surveys are easy and SO useful!
Numbers required for quantitative analysis present real challenges

7. All this resulted in A LOT OF DATA!

8. Survey instruments Assessment Literacy Questionnaire (ALQ) alqAL
Assessment for Learning
e.g. I use assessment to work out how well I am doing
alqMEO
Minimum Effort Orientation
e.g. My aim is to pass the course with as little work as possible
alqAU
Assessment Understanding
e.g. I understand the criteria
alqAJ
Assessment Judgement
e.g. I judge my own work using my knowledge of the criteria
Feedback Orientation Scale (FOS)
fosUT
Feedback Utility
e.g. Feedback is critical for improving performance
fosACC
Feedback Accountability
e.g. I don’t feel a sense of closure until I respond to feedback
fosSA
Social Awareness
e.g. Feedback lets me know how I am perceived by others
fosSE
Feedback Self-Efficacy
e.g. I know that I can handle the feedback I receive
Smith, C.D. et al., (2013) Assessment literacy and student learning: the case for explicitly developing students ‘assessment literacy’, Assessment & Evaluation in Higher Education, 38:1, 44-60
Linderbaum, B.A., and Levy, P.E. The Development and Validation of the Feedback Orientation Scale (FOS) Journal of Management, 36:6,

9. Findings: Assessment Literacy
Students showed significant improvements in assessment understanding after semester 1 for both years ( : M=3.53->M=3.77, p=0.002, : M=3.40->M=3.86, p<0.001)
In year 2, students also showed significant improvement in assessment judgement (M=3.00->M=3.20, p=0.028)
Corresponding increase in Minimum Effort Orientation in year 2 (M=2.09->M=2.34, p=0.003)

10. Findings: Assessment Literacy and Peer Learning
Negative correlation between pre_MEO and attendance in year 1 (i.e. students with a high minimum effort orientation attended fewer sessions).
Positive correlation between attendance and assessment understanding in year 1
Aggregating both years data:
High attenders significantly improved assessment understanding, judgement and minimum effort orientation – is this a good thing?
Low attenders also significantly improved assessment understanding but not by as much as high attenders (gap of 0.45 vs 0.24)
Differences in gap scores between high and low attenders for AU, AJ and MEO are approaching or at significance

11. Assessment Literacy and Peer Learning: Qualitative data
Students in focus groups and peer leaders identified ways that assessment literacy had been improved:
“Some of them weren’t aware of the criteria, the marking criteria being available on Blackboard and I showed them and they’re very confused because the lecturer didn’t show them” (Peer Leader)
“They said they can advise on it, but, obviously, they can’t give us a grade, or whatever. I don’t think they’re able to do that (1st year student)
“I can say they helped me with understanding how I am supposed to structure my essays, basically, which is good” (1st year student)
“They tell you what to expect from assessments, and how to do them. It’s good overall” (1st year student)

12. Findings: Feedback orientation
In general, feedback utility and feedback accountability start high and decline after the first semester!

13. Findings: Feedback orientation
No significant findings between attendance and feedback orientation
In white students had significantly higher feedback utility that BME students after semester 1
In students who’s parents attended uni had significantly higher feedback utility after semester 1
In these differences disappeared

14. Conclusions… so far!
There is some evidence that structured peer learning can help to develop assessment literacy (understanding and judgement)
Improving understanding and effort also results in a reduction of the amount of effort students perceive they need to put in!
The relationship between peer learning and feedback orientation is less clear
Attendance is significantly related to grades (but this is well known and could relate to motivation to study)

15. The benefits of Peer Learning go far beyond assessment and feedback!
“They help you with actual student life, and not just your modules, and assessments, and things like that. People going to university, for most people it’s a really big change to their lifestyle, because they’re learning to live by themselves, and everything. I think, from someone who’s already had previous experience of it, like peer leaders, they have a lot of knowledge and things that they can actually teach us.”
(1st year student)

16. Neil Ford Senior Teaching Fellow (Education Development) Southampton Business School