1. Peer assessment of group work using WebPA Neil Gordon Symposium on the Benefits of eLearning Technologies University of Manchester, in conjunction with the Higher Education Academy Subject Centres in Engineering, Physical Sciences, and Mathematics, Statistics and Operational Research Wednesday 13 April 2011

2. Introduction – why include group work? It is perceived as a key skill QAA expect it Employers expect it Accrediting bodies require ““working with others” It is a useful skill for our students – one which they will need after their studies

3. Practical Issues a key problem with group work is assessment. Students may consider the process unfair if individual levels of contribution are not recognised. But the tutor rarely has full knowledge of the team interactions to allow realistic allocation of marks. Gathering data to do this from students is not viable with large cohorts. WebPA provides a tool to allow this approach – providing a web interface to gather data, and implement algorithms to allow the allocation of individual marks based on a team’s overall mark.

4. Setting Groups Many different approaches Self-selection: students are given the specifications (e.g. group size, any special requirements) and self organise Groups organised by the teacher: – Random assignment (perhaps by surname); – Assigned according to degree type, with a mix of students from particular degrees to ensure a mix of knowledge and skills within the group; – Grouped according to ability as measured by some previous assessment

5. Assessment Considered a major issue for students Many fear some group members not contributing fully and damaging their marks Clear L.O.s can avoid this Need to consider whether to assign a group mark to all, a weighted group mark, or individual marks (or a combination of these).

6. Challenges for peer and self assessment Suitability of the assessment Fairness in the marking process Transparency in marking criteria Practical management issues in handling multiple marks for individuals

7. Technology and Group Work Use of technology to support team activities is standard in industry In the context of teaching, use of these technologies in education helps develop “real world” skills, alongside its educational value Many technologies now in common use (especially with Web 2.0) : VLEs, GroupWare and Intranet, Wikis, etc.

8. Peer (and self) Assessment Many potential benefits – especially in ensuring that students become aware of marking criteria It can reduce the marking burden Different approaches e.g. – group agrees a weighting for each group member – Each student assesses each other student – Groups of students assess other groups Can be problems with the volume of data gathered, as well as mediation issues

9. Transparency in marking criteria By supplying clear marking criteria to students they can understand the assessment process Particularly important when marking each other Explaining this helps make students aware of assessment processes;

10. Example Peer marking criteria Students asked to assess each other in terms of: Contribute to meetings/online discussion Contribute to meetings/online discussion Offer constructive input to discussions Offer constructive input to discussions Contribution to production of report Contribution to production of report Overall contribution to assignment Overall contribution to assignment Marked in the range 0 (no input) to 10 (excellent input)

11. WebPA: a tool for peer assessment WebPA provides a tool to automate much of the above with regards to peer (and self) assessment Allows groups to be set up, and for students to assess their other team members Uses a simple algorithm to allocate a weighted mark Provides for students to enter marks in a faceless environment

12. Issues in peer and self assessment Tools such as WebPA allow students to mark each other in a safe environment - through a web interface which protects the anonymity of individual students’ marks of their peers from other students; WebPA also applies a suitable algorithm to apportion contribution based on the student marks; One major issue when considering the use of peer assessment is the volume of data and its management Arranging to collect and then use the data becomes a problem with large classes (e.g. >100): WebPA can manage this process in an automated way.

13. Case Study: Computer Science @ Hull Regularly use team/group projects I have used peer assessment across several modules, throughout the range of teaching (level 3 to level 6 and from cohort sizes of 6 up to 180). Nearly 1000 students In small modules, originally did by paper, where it was fairly easy to manage – but was impractical for large modules

14. Students Prior Experience of group work QuestionPercentage who said yes As part of your pre-University education, have you worked in groups with other students? 92% Did you enjoy working in groups with other students? 71% Have you ever used computers to work online in groups with other students? 'Working online' includes things like communicating and sharing work with other students through discussion boards, email and other computer technology. 58% Did you enjoy working online in groups with other students? 46% Have you ever been assessed by self and/or peer assessment?71% If you have undertaken self and/or peer assessment, do you like it as a form of assessment? 71% of those who said yes, (50% of all respondents)

15. Student Views of group work Some students would not contribute, and those who were active would end up doing all the work; A desire to be able to choose their own groups, so they were confident that people would put in the effort; Concerns about random groups, where members do not contribute fully and the workload is unbalanced; Concerns about having to rely on others; Acknowledgment that group work is essential in the software industry; Concerns that criticising the work of a fellow student may mean that student gives them a low mark in the peer assessment; Perceptions that it is more fun than working alone, but more complex

16. Student views of peer assessment prior to it’s use Asked “do you like the idea of this form of assessment” - This question generated more negative responses, many from those who has not experienced it in practice concerns were about biased marking About other students not taking it seriously. Several students felt that as long as the teacher was involved in the marking process, then it would be acceptable.

17. Solution: WebPA

18. Adding Data

19. Student’s views after module Majority (70%) prefer to work alone (strongly agree or agree # ). BUT 65% would # use peer and self assessment in other modules given the choice and 67% like # a mix of group and individual work 80% comfortable # assessing own performance 69% comfortable # assessing peers

20. Positive views of WebPA

21. More views on WebPA When asked “What changes (if any) to WebPA would you like to see? “: 80% of students said no change 9% asked for “Provide textual feedback to explain the scores you gave yourself“ 1% for a “general feedback to add any other information”

22. Other points Some variety of opinions on whether students would like to submit their own marks Similarly, whether they would want just the tutor, or also their peers to see these comments A small number requested a more complex and in depth mark scheme!

23. My comments on using WebPA Issues where a student had dropped out, but the marks had been applied before I updated that – so having to reverse engineer the mark to stop inappropriate marks of 100% Would be helpful to be able to import groups (I usually manage these in Excel) But generally – excellent

24. Conclusions WebPA can allow the scaling up of group work with peer and self assessment It assists the teacher/lecturer by reducing admin associated with the activity It benefits students as it provides a safe environment in which to carry out the assessment It can encourage self reflection and a focus on assessment criteria It can be integrated fairly well into a VLE