“Review of Student Evaluation of Teaching at WCPPE” Catherine O’Brien PCUTL Presentation Group Project Jan 2012

The Wales Centre for Pharmacy Professional Education (WCPPE) works: "To motivate and support the pharmacy team, through the provision of learning and development opportunities, to enhance their competence to deliver high quality patient services in Wales." WCPPE

How/What ? 1.Literature Review – COB 2.Current state: process & test v best practice 3.Future State: proposal & test v best practice 4.Costing (!!) & Implementation 5.Review

Student Evaluations of Teaching Harvey (2001) 2 main functions – Internal information to guide improvement – External information for potential students and other stakeholders. levels – Institution – Faculty – Programme – Module – Teacher Appraisal

Student Evaluations of Teaching WCPPE context Learners – Graduate CPD, non graduate – different expectations and needs from UG Range of formats including online and VC Non CU facilities Large number of learning facilitators Repeated courses Learner – development in learning

Satisfaction Cycle, Harvey 2001 Stake- holder determined questions Question distribution Feedback to stake- holders Implement and Monitor Action Plan Consultation Report with action Analysis of results ? X X X X X?X?

Workshop Objectives of the process What is the internal and external purpose of the SET for WCPPE? What are we trying to achieve? What areas do we want to evaluate? What does the current WCPPE process look like and what are the strengths and weaknesses?

Workshop Design of questionnaire ( Rowley, 2003) Will one size fit all for face to face and e-format and for various styles? Should they be tailored to the learning outcomes? Do we take a quantitative or qualitative approach? How do we optimise response rate?

Workshop Design of questionnaire- contd Should they be tick boxes or free text? How do we design in relation to the relative value of the responses? Do we require students to identify themselves or should there be anonymity? What aspects of teacher performance should be evaluated?

Workshop Data Collection and analysis and action When is the right time to get them filled in, immediate or on return to work place? Who should collate and analyse data? Who is responsible for action? What should the new process be and how do we close feedback loop?

Results Its hard- balance size and aspects covered Focus on face to face courses –Venue –Teacher performance –Qualitative approach – free text –Style and content and application to practice 2 forms –Learning facilitator - BOS –Learners – paper

Review Harvey- Satisfaction Cycle Stakeholder determined questions Feedback to stake- holders Implement and Monitor Action Plan Consultation Evaluation to Moodle for each course and more… Standing item on meetings. Developing KPI Add to agenda with Welsh Government and other stakeholders. Consulted Virtual Opinion Panel but could do more.

Harvey- Satisfaction Cycle (2) Stakeholder determined questions Question distribution Feedback to stakeholders Implement and Monitor Action Plan Consultation Report with action Analysis of results

Impact for WCPPE Renewed focus on SET as a tool within WCPPE (not just ticking boxes) Improved quality of courses as feedback is used. Feedback is contextualised by Learning Facilitators. Freed administrative time to market courses. Gained experience to apply to other learning resources. Peer review of learning facilitators.

Current thoughts and conundrums Will we maintain the cycle and review? How do we engage stakeholders in questionnaire design in future? Move to online? Link completion& Certificate What is the picture across CU? How do we consider SETs in the context of CPD across CU?

References Lee Harvey, (2001) “Student Feedback”, “A report to the Higher Education Funding Council for England” Jenifer Rowley, (2003) “Designing student feedback questionnaires”, Quality Assurance in Education, Volume 11 Iss: 3, pp