Peer Assessment COMP3120 – Managing Software Development

Contents Value –Students –Academics Purpose How is peer assessment completed? WebPA-x algorithm 2

Value to Student Encourages reflection – learn to make judgements about the quality of one’s own and one’s peers’ work Enables students to recognise individual contributions – quality, quantity, relevance Helps to develop team work Helps improve individual performance 3

Value to Academics Assess how successfully the group worked together and how well they managed their work Assess the group product Mandatory although it carries no marks and there are penalties for late submission 4

Purpose To assess the contribution made by each member of a team Allows the identification of team members who have been active and collaborative and those who have not. Used to calculate individual marks from the team mark awarded for the artefact 5

How is Peer Assessment completed? WebPA-x –Base software developed by Loughborough University in the UK and released as free and open-source –Used by hundreds of universities world-wide –Software was amended by software engineering group project student team –Used by COMP3100/3500/4500 since

How is Peer Assessment completed? Evaluate team members and yourself against a set of categories –which when taken together should give a good overall indication of performance Provide an overall, holistic rating of your and your peers' performance, including a written description which provides an overview of the behaviour, activities, events etc that explain your rating. 7

How is Peer Assessment completed? Does not ask students to make a commentary of other students’ academic abilities Asked to rate performance against a set of professional behaviour / performance categories –Sometime called Key Performance Indicators (KPI) 8

How is Peer Assessment completed? Evaluation categories –Leadership –Professionalism –Respect –Excellence –Collaboration –Accuracy and attention to detail –Timeliness –Enthusiasm 9

How is Peer Assessment completed? Holistic rating of performance, including a written description Rating categories: Excellent Very Good Expected Ordinary Unsatisfactory Margin Deficient Superficial No Show 10

WebPA-x algorithm Each student in a team is given a pie which they divide up among their peers according to their assessment of the value of their peers' contributions. The ratings that a student gives themself is not used when calculating the WebPA-x peer rating. 11

WebPA-x algorithm Normalise ratings to overcome the issues related to one student awarding a maximum rating of 5 and another student awarding a maximum rating of 3, Done prior to calculation of the WebPA-x peer rating. When a team member fails to submit a peer assessment the algorithm acts as if they rated everyone as performing equally. 12

WebPA-x algorithm Worked example –Collate submitted ratings –Normalise ratings awarded by each student –Calculate WebPA-x rating for each student –Rating against professional criteria worth 50% –Holistic evaluation worth 50% –Calculate peer assessed mark Team mark multiplied by WebPA-x rating Individuals rated highly by the team can get more than the team mark 13