Self and Peer Assessment Institute of Teaching and Learning Seminar Series Jan Fermelis Faculty of Business and Law 14 August August 2008

Seminar Outline 1. Team assignment challenges and strategies 2. Online SAPA demonstration 3. Discussion ITL Seminar 14 Aug 2

Team Formation Unit realities Unit realities Size/nature of student cohort Size/nature of student cohort Number of campuses/instructors Number of campuses/instructors Teaching/learning aims Teaching/learning aims Self-selected Self-selected Reduced likelihood of team wars or divorces Reduced likelihood of team wars or divorces Instructor-selected Instructor-selected Improved opportunities for intercultural and teamwork skills Improved opportunities for intercultural and teamwork skills ITL Seminar 14 Aug 3

Teaching/Learning Strategies Integration of teaching/learning/assessment experiences Integration of teaching/learning/assessment experiences Lecture and reading materials Lecture and reading materials Team dynamics Team dynamics Interpersonal communication skills and strategies Interpersonal communication skills and strategies Tutorial activities: Tutorial activities: Discussion of team member attributes and role expectations Discussion of team member attributes and role expectations Speed Meeting exercise Speed Meeting exercise Tutorial time for team meetings Tutorial time for team meetings Student opportunities to reflect and to trial strategies Student opportunities to reflect and to trial strategies Instructor availability to mentor teams Instructor availability to mentor teams ITL Seminar 14 Aug 4

Assessing Team Assignments Shared score for all team members Shared score for all team members Instructor workload Instructor workload Student perceptions of fairness Student perceptions of fairness Individualised scores Individualised scores Paper systems Paper systems Online systems Online systems ITL Seminar 14 Aug 5

Online Self and Peer Assessment Pioneered by Richard Tucker, Architecture & Building Pioneered by Richard Tucker, Architecture & Building Improved quality of work Improved quality of work Student preference for individualised assessment Student preference for individualised assessment STALGS grant for cross-disciplinary pilot within Business Communication and Architecture/Design STALGS grant for cross-disciplinary pilot within Business Communication and Architecture/Design Testing in a more complex educational context Testing in a more complex educational context Ongoing development and refinement Ongoing development and refinement ITL Seminar 14 Aug 6

Research Program Phase 1 Implementation (arch and bus comm) Implementation (arch and bus comm) Compulsory use Compulsory use Penalty for missed entries (arch) Penalty for missed entries (arch) Reward for entries made (bus comm) Reward for entries made (bus comm) Individualisation of assessment Individualisation of assessment Voluntary entry and exit questionnaires Voluntary entry and exit questionnaires Ongoing development and refinement of the software Ongoing development and refinement of the software ITL Seminar 14 Aug 7

Research Program Phase 2 Compulsory implementation (arch) Compulsory implementation (arch) Reward for entries made Reward for entries made Optional implementation (bus comm) Optional implementation (bus comm) Neither rewards nor penalties Neither rewards nor penalties Voluntary entry and exit questionnaires Voluntary entry and exit questionnaires Ongoing development and refinement of the software Ongoing development and refinement of the software ITL Seminar 14 Aug 8

Research Program Phase 3 No implementation of SAPA No implementation of SAPA No individualisation of assessment No individualisation of assessment Voluntary, online questionnaires Voluntary, online questionnaires ITL Seminar 14 Aug 9

Software Enhancements 1 Three types of assessment per SAPA entry Three types of assessment per SAPA entry Relative contribution ( ) Relative contribution ( ) Individual performance (1-5) Individual performance (1-5) Comments Comments Visual attractiveness of SAPA website Visual attractiveness of SAPA website Password protection for students Password protection for students Refinement of Multiplicative Scaling Factor Refinement of Multiplicative Scaling Factor ITL Seminar 14 Aug 10

Software Enhancements 2 Process for keying in team and member details Process for keying in team and member details Username check Username check Add/change team members Add/change team members Password reset Password reset Unit co-ordinator capacity to edit settings Unit co-ordinator capacity to edit settings Easy unit co-ordinator access to results: Easy unit co-ordinator access to results: Students not in groups Students not in groups Number of student entry submissions Number of student entry submissions Student ratings and comments Student ratings and comments ITL Seminar 14 Aug 11

SAPA Procedural Enhancements Motivation and training of staff Motivation and training of staff Motivation and training of students Motivation and training of students User friendliness of entry process User friendliness of entry process Clarity of SAPA instructions Clarity of SAPA instructions Hotlinked reminders Hotlinked reminders ITL Seminar 14 Aug 12

Some Initial Results Positive student attitude to individualisation Positive student attitude to individualisation Almost no complaints about lowered marks Almost no complaints about lowered marks Varying levels of student compliance Varying levels of student compliance Large, complex vs small, single teacher cohort Large, complex vs small, single teacher cohort On-campus vs. off-campus teams On-campus vs. off-campus teams Compulsory vs optional Compulsory vs optional Penalty vs reward Penalty vs reward Staff encouragement vs discouragement Staff encouragement vs discouragement ITL Seminar 14 Aug 13

For Further Research Student self-censorship Student self-censorship Visibility of MSF ratings and comments Visibility of MSF ratings and comments Continuous vs overall SAPA Continuous vs overall SAPA Potential for student collusion Potential for student collusion Inclusion vs. exclusion of SELF assessment Inclusion vs. exclusion of SELF assessment Impact of cross-cultural factors on extreme responses Impact of cross-cultural factors on extreme responses ITL Seminar 14 Aug 14