Peer observation for distance learning e-tutors: University of York Richard Walker & Kevin Hall Evaluation of a cross-programme trial scheme

Description of York trial Shared features with COOLAID model Features specific to York Choice over:  Participation  Focus of observation  Form of feedback  Information flow  Future action Choice over:  role (observer / observee) No choice over: Partner (observer and/or observee) Random selection – based on matching participants from different programmes & subject / disciplinary backgrounds

Breakdown of participants by distance learning programme Distance Learning Programme # of tutors completing observation MA Public Policy & Management 5 MA TEYL2 Certificate in Creative Writing (Lifelong Learning) 3

Participant feedback on the contracting process “I am confident about my teaching style but not clear if this is working online –I wanted to find out.” (Experienced Lifelong Learning tutor) “I felt in control of the process and quite relaxed. I would have reacted differently if told that I had to do it with a random observer.” (Experienced PPM tutor) “I was not concerned about exposing myself – maybe yes if this had been in my first term’s teaching – but I was confident enough to do it”. (Inexperienced PPM tutor) “I was excited by working with an external. We have done our own peer observation internally with all following the same approach – this is an opportunity for another pair of eyes although it was slightly nerve wracking.” (Experienced PPM tutor)

Emergent working models for conduct of peer observation process Model 1: f2f negotiation with oral discussion of feedback Contract by ftf meeting Forms exchanged after meeting Online observation Feedback through f2f meeting Reports exchanged after meeting Model 2: negotiation at a distance with oral & written feedback (interactive) Personal reflection on aims Forms exchanged Clarification by Online observation Delivery of notes to partner Discussion of feedback by Skype Model 3: negotiation at a distance with written feedback (synchronous) Personal reflection on aims Forms exchanged Clarification by Online observation Delivery of written report to partner ModelPreparation Observation & Discussion

Outcomes: Classification of Key Benefits according to Participants  confirmation & reassurance on tutoring practice (what’s working well) The key benefits reported by participants may be categorised as follows: “confirmed that my tutoring style is OK – posts are not too long; student are getting detailed feedback” “the balance of style and substance in the course was confirmed”

Outcomes: Classification of Key Benefits according to Participants  fresh perspectives on current practice (challenging assumptions; surfacing issues & weaknesses in current practice; doing things differently and making improvements to current practice) The key benefits reported by participants may be categorised as follows: “don’t be scared to set boundaries and ground rules for discussion, signalling the beginning and ending of discussions” “introduce video ‘talking heads’ for tutor input to make the course site more engaging - create more of a personal stamp on the course..”

Outcomes: Classification of Key Benefits according to Participants  comparative viewing of platforms / technologies The key benefits reported by participants may be categorised as follows: “I learned about technology such as ‘adaptive release’ and the differences between Moodle and Blackboard”

Outcomes: Classification of Key Benefits according to Participants  ‘side benefits’: unintended benefits – transferable learning approaches (dealing with overseas students) The key benefits reported by participants may be categorised as follows: “helped me to think about how we work with overseas students – useful advice – step-by-step on preparation for writing at Masters level”

Outcomes: Classification of Key Benefits according to Participants  ‘association with a community of practice’ (networking with other tutors) The key benefits reported by participants may be categorised as follows: “it was very nice professionally to feel part of a group of online tutors -I had not felt that before, nor felt before that I was part of the University” Outcomes recap  confirmation & reassurance on tutoring practice  fresh perspectives on current practice  comparative viewing of platforms / technologies  ‘side benefits’  ‘association with a community of practice’

Lessons Learned  Pairings which engaged in face-to-face negotiation and discussion appeared to develop greater levels of trust  Pairings which worked at a distance (online) tended to adhere more closely to the contract and focus on predetermined themes with objective comments, with limited evidence of critical feedback in their exchanges  This suggests a challenge for the further development of this trial and the conduct of peer observation at a distance in supporting observers and observees to develop relationships of trust through online interaction, enabling them to draw the maximum benefits from peer observation to inform their own professional practice

Questions? University of York Richard Walker & Kevin Hall