1. Peer observation, feedback and reflection for online practice Mark H. Jones, Faculty of Science & eSTEeM

2. Overview Motivation for the project Models of peer-review Implementation of a peer-review exercise Evaluation Comments on wider scale adoption

3. Motivation Level 3 physics and astronomy modules have moved completely to online tuition – with online rooms replacing face-to-face contact. Staff Tutor has a responsibility to: – develop staff in their pedagogic practice – monitor the quality of what is provided In an online environment, there are new possibilities to set up peer observation. Can peer-observation be useful in developing practice for synchronous online tuition?

4. Models of peer-review Peer-observation is used throughout HE in several distinct contexts. A useful classification was provided by Gosling (2002): - Evaluation model - Development model - Collaborative model “Observation” is a loaded term.

5. Peer observation models CharacteristicEvaluation modelDevelopment modelCollaborative model Who does it, and with whom? Senior staff, or chosen ‘evaluators’ other staff Educational developers observe practitioners Teachers/peers/ colleagues PurposeQuality assurance (e.g. identify under- performance); appraisal (e.g. promotion cases) Demonstrate or improve competencies in teaching; assessment. Improve teaching and learning through dialogue, reflection; stimulate innovation OutcomeReport / judgementReport / action planAnalysis, discussion, wider experience of teaching methods Status of evidence AuthorityExpert diagnosisPeer shared perceptions Adapted from Gosling (2009)

6. The collaborative ideal? Purpose: improve teaching and student learning through dialogue, self and mutual reflection; stimulate innovation Outcome: analysis, reflection, discussion, wider experience, improvement to teaching and learning. Status of peer review: peer shared understandings and perceptions. Non-judgmental, constructive facilitated dialogue Intended beneficiaries: mutual benefits for both peers, students Conditions for success: a culture in which teaching is valued and discussed. Risks: confirms existing practice, passive compliance, bureaucratic.

7. Implementation Voluntary participation Reviewee is in control of the process: - makes initial contact with reviewer - reports when process is complete Briefing for participants (online meeting – recorded) Aimed for review of live sessions (but some review of recordings did take place) A cohort of tutors on third level physical science modules Tutor group tutorials (rather than module wide presentations) Aimed to observe different modules (wasn’t always possible) SM358, SMT359, S382, S383

8. Practicalities: three stages 1.Pre-observation contact 2.Observation 3.Feedback and reflection meeting 4.(Notification that process is finished)

9. Pre-observation discussion The context of the tutorial (reviewers may not know the module). Note-taking – are you both happy with the note-taking protocols (see below)? Will whiteboards be shared beforehand? Will notes be shared before or after discussion (see below) How will the reviewer be introduced to the students? Agree when feedback meeting will be held. From briefing session

10. The observed session Turn up in good time. Tutor to introduce reviewer as agreed. Reviewer takes notes as agreed. Probably not a good idea to run on from the observed session straight into the feedback/reflection meeting. From briefing session

11. Feedback / reflection meeting Aim: to reflect on a teaching session in order to: - identify good practice - analyse situations which were less successful - feel confident about making changes to practice NOT an aim to either identify everything that wasn’t perfect, or to solve every issue in online teaching. From briefing session

12. Timeline Number of participants: 12 (out of possible 20) Call for volunteers: April 2012 Online briefing: May 2012 Observations took place: June to September 2012 Feedback / reflection sessions after observations

13. Practical problems Timetabling (and use of recordings) Permissions setting on VLE Timescale for the process (too short)

14. Evaluation Main tool for analysis is a structured conversation conducted at the end of the presentation (October 2012). Explores: Practical problems. Practical considerations. Themes around peer observation. The extent to which the process meets the ideal of Gosling’s collaborative model (see also Martin & Double 2006)

15. Evaluation: themes Purpose: learning and reflection? Other peer- observation experience. Apprehension (Bell, 2010) threatening nature (Cosh, 2006) Nature of the feedback (Bell, 2010). Training observers to give constructive criticism (Cosh,2006) Effectiveness (Bell, 2010) and uncritical positive feedback. Benefits to observer, creative reflection (Cosh,2006) and Bell (2010) - 'modelling teaching skills'. Ongoing professional development (Bell, 2010) Identifying good online teaching (Swinglehurst, Russell, Greenhalgh, 2008 )

16. Evaluation: current status Evaluative conversations were conducted in October / November. Currently being transcribed to text. Qualitative data to inform understanding of the themes.

17. Wider adoption Some comments: Findings of small scale qualitative survey should be a useful guide to issues for any larger scale implementation. There is a specific need to develop discussion around practice in teaching using synchronous conferencing. Important to create a model that is perceived to be collaborative. Care needed to avoid the process being appropriated for other purposes (e.g. quality assurance).