DMW 2007 The Open University's Institute for Educational Technology Open Mentor: Supporting tutors with their feedback to students Denise Whitelock Open University, Walton Hall, Milton Keynes Stuart Watt The School of Computing, The Robert Gordon University, Aberdeen

DMW 2007 The Open University's Institute for Educational Technology Formative feedback for tutors: OpenMentor – a training and monitoring e-Tool? What implicit /explicit models do tutors have about marking assignments? What training is given to tutors about marking assignments? Do we hold the company line when marking? Tutor questionnaire

DMW 2007 The Open University's Institute for Educational Technology What is Open Mentor?  “An open source mentoring tool for tutors”  “Open source” = free and easy to use, and to embed in an institutions infrastructure and working practices  “mentoring” = designed to help people learn how to give feedback effectively, through reflection and social networks  “tutors” = primarily intended for teaching staff, but with clear applications for those involved in quality

DMW 2007 The Open University's Institute for Educational Technology Coding into Categories Bales analysis Psychology 1950s Analyses talk Includes socio-emotive categories Flander’s (1970)categories inappropriate as also includes classroom control

DMW 2007 The Open University's Institute for Educational Technology Bales Categories Four main groupings A. Positive reactions; agreeing and boosting the other person B. Directing/teaching C. Questions: requesting information, clarification etc D. Negative reactions: disagreement

DMW 2007 The Open University's Institute for Educational Technology Coding the comments Bales’ Interaction Process

DMW 2007 The Open University's Institute for Educational Technology Identifying trends: H801 Graph to show conflated Bale’s categories against mean number of incidences in H801 scripts

DMW 2007 The Open University's Institute for Educational Technology Identifying trends: H801 Pie Chart to show the mean number of incidences per pass per conflated Bales' Interactional Category for all four levels of pass in H801 scripts Key: A = Positive reactions B = Responses C = Questions D = Negative reactions

DMW 2007 The Open University's Institute for Educational Technology How general is the model? S103 – Foundation Science Course B820 – Masters Business School Course

DMW 2007 The Open University's Institute for Educational Technology Identifying trends Pie Charts to show the mean number of incidences per conflated Bales Interactional Category for ‘Pass 1’ and ‘Pass 4’ in the following courses: Key: A = Positive reactions C = Questions B = Responses D = Negative reactions Pass 4 Pass 1 B820S103H801

DMW 2007 The Open University's Institute for Educational Technology Explaining Open Mentor’s Rules Four categories A – Positive Reactions B – Attempted Answers C – Questions D – Negative Reactions

DMW 2007 The Open University's Institute for Educational Technology ‘A’ - Positive Reactions

DMW 2007 The Open University's Institute for Educational Technology ‘B’ Attempted Answers

DMW 2007 The Open University's Institute for Educational Technology ‘C’ Questions

DMW 2007 The Open University's Institute for Educational Technology ‘D’ Negative reactions

DMW 2007 The Open University's Institute for Educational Technology Example Rules...you could... (B4)...good...but... (B4)...excellent analysis... (A1)...this...does not.... (D10)...what are... (C7)...in my opinion... (B5)

DMW 2007 The Open University's Institute for Educational Technology Building the rule set

DMW 2007 The Open University's Institute for Educational Technology Is the rule set generic?

DMW 2007 The Open University's Institute for Educational Technology Is this a model that works in the way tutors and students expect?  47 students and 44 tutors responded to a questionnaire to test OM’s underlying tutorial model  Findings suggest:  Lower grades should attract more detailed comments and explanation by the tutor (students)  Higher grades should attract more positive comments (students and tutors)  Lower grades attract more questions and suggestions (tutors)  Model supported by pedagogical study

DMW 2007 The Open University's Institute for Educational Technology How Open Mentor handles comments  “Good work”  “Yes, well done”  “Yes, but is this useful?”  “Can you explain what you mean”  “This does not follow”  A = positive reactions  B = attempted answers, and not a positive reaction  C = questions  D = negative reactions

DMW 2007 The Open University's Institute for Educational Technology Open Mentor in action

DMW 2007 The Open University's Institute for Educational Technology

Inside Open Mentor Course info Assess- ments ClassifierExtractor Analyser Web interface Bench- marks

DMW 2007 The Open University's Institute for Educational Technology Current and future developments  Embedding in institutional practice  Enhancing quality of first year provision  Links to VLEs and information systems  Further development  Support for students  Course evaluation  Open Comment  Ensuring emotional support as well as conceptual guidance for learners