1. Evaluating the student learning experience Ivan Moore HE consultant ivan.moore@manchester.ac.uk Centre for Excellence in Enquiry Based Learning Ivan Moore Higher Education Consultant Learning and Teaching

2. Objectives By the end of the session, you should: 1.be able to describe a range of approaches to designing an evaluation form, based on internationally recognised methodologies; 2.have experienced an activity through which you will have begun to develop a customised evaluation methodology; 3.have experienced an approach to evaluating the student learning experience; Centre for Excellence in Enquiry Based Learning Ivan Moore Higher Education Consultant Learning and Teaching

3. What is evaluation? It’s something we talk about in Educational Development. It is important for Learning and Teaching but we often don’t do it very well……or in time to do much about it. Somehow, we often seem to not be able to find the time to evaluate what we are doing or the effects of our interventions. It seems just a little too difficult to do, or we have other priorities.

4. The purposes of evaluation Evaluation for accountability (measuring results or efficiency) Evaluation for development (providing information to help to improve practice) Evaluation for knowledge (to obtain a deeper understanding of some particular area of practice such as student learning or change management) Chelimsky (1997) Summative Formative Research

5. Why should WE evaluate? Because it is part of our professional practice

6. Principles of evaluation An integral part of our teaching practice An ongoing process, so that we learn from systematic reflection Should be participatory Should enable us to make appropriate modifications along the way Should enable us to make judgments on specific sessions, but also to draw out wider implications

7. Purposes of evaluation Mike Prosser Quality Assurance –student satisfaction –the mean score is important as a measure of quality Quality Enhancement –Student conceptions/how they experience the course –The deviation is important: more focused view

8. Purposes of evaluation Mike Prosser Is the learning environment/teaching approach having any influence on student conceptions/approaches? A student experience survey is more important than a student satisfaction survey

9. Approaches Off the peg questionnaires –Learning Styles (Honey and Mumford) –Motivated Strategies for Learning –Learning and Study Strategies Inventory (LASSI) –Approaches to Study Inventory –Course Experience Questionnaire (Ramsden)

10. The Course experience questionnaire (P. Ramsden) Designed as a performance indicator 24 statements relating to 5 aspects 1 overall satisfaction statement Research-based Drawn from statements made by students in interviews Students with positive responses take a deep approach

11. The five sub-scales Good teaching Clear goals and standards Appropriate assessment Appropriate workload Generic skills

12. Learning is driven by a process of enquiry –Students determine the lines of enquiry –They identify what they need to know, how they will learn it, how they will know that they have learned it –They will share this learning with others –They have a range of support resources: Online; paper-based; equipment; room; other students; the tutor Enquiry-Based Learning

13. Students –Academic development:- information skills, analysis, synthesis, evaluation, hypothesis –Professional development:- teamworking, leadership, shared goals, communication –Personal development:- taking responsibility, planning, problem solving, confidence Change of approach –Locus of control, motivation

14. From Ramsden CEQ EBL has helped me develop my ability to work as part of a group I have usually had a clear idea of where I was going and what was expected of me I have found EBL motivating The EBL tutors motivated me to do my best work EBL has helped sharpen my analytical skills There has been more assessment of what I have memorised than of what I have understood As a result of EBL, I believe that it is more important to have found new ways of thinking than to have gained specific knowledge about the subject areas

15. Designing an evaluation questionnaire In groups – 30 minutes Outcomes based design Agree up to 4 goals/objectives for your group For each, design 4 statements that will help to determine if the goal is being achieved Record the goals and statements on a flip chart

16. Approaches Scriven –Goal – free evaluation EBL has helped me develop my ability to work as part of a group –Presupposes and leads the student What parts of the module motivated you most? –(ok so it presupposes motivation, but not EBL) Why? –Illuminative evaluation

17. Goal-free evaluation In pairs Choose another pair on which to focus Devise up to 5 questions that you might ask students to answer that might provide information on what they are experiencing Record your questions Compare them with the other pair Discuss what information you might receive from these questions Draw up your conclusions for feedback

18. Context-free evaluation Since the beginning of the year: What skills, if any, have you developed? What helped you to develop these skills? How have you changed the way you study? What prompted you to make these changes?

19. Other methods Focus groups Structured interviews Continuous feedback –What did you find most difficult/confusing today? –What things did you find helped you learn last week? –What should I: Start/stop/continue?

20. The three minute paper What was the most useful of meaningful thing you learned during this session? What question(s) remain uppermost in your mind as we end this session? What was the ‘muddiest’ point in this session? What would you like me to stop doing? What would you like me to start doing? What would you like me to continue doing?

21. Other instruments Kolb’s Learning Style Inventory (LSI) Honey and Mumford’s LSQ Sternberg’s thinking styles Felder’s Index of Learning Styles Weinstein’s Learning and Study Strategies Inventory (LASSI) Entwistle’s Approaches to Study Inventory

22. Evaluating the student learning experience Ivan Moore HE consultant ivan.moore@manchester.ac.uk Centre for Excellence in Enquiry Based Learning Ivan Moore Higher Education Consultant Learning and Teaching