Handbook for Learning and Teaching Review Team Members 2014/15 1

Contents Purpose of this handbook and LTR training4 The Purpose of LTR6 Key features of the process11 Substantial changes from ISR13 Team Roles and Responsibilities15 Managing the LTR effectively25 How to ask questions38 How to make commendations and recommendations40 2

Contents (cont.) Appendices: 1.List of key documents about the LTR process 2.Suggested LTR Review Visit schedule (1 day and potential variations) 3.LTR Timeline: The Panel Perspective 4.LTR Timeline: The Subject Area Perspective 5.Sample SWOT 6.Recommendations exercise 7.Previous examples of exemplary practice 3

Purpose of the Training and Handbook After reading this handbook and/or attending training, LTR team members should be able to: Articulate the purpose of LTR Appreciate what the key features of the process are Understand how LTRs are planned, and, with support, how to carry out and follow up an LTR Understand the roles of the team members, including the Secretary and Chair Understand how to use the SWOT and other evidence to prepare and ask appropriate questions Recognise the value of recommendations, commendations and exemplary practice, how to identify them, and how to word them 4

Accentuate the positive We can learn more from what works well than from what works badly What works well in one context can often be transferred to another Messages that are constructive and positive in tone are likely to be better received Makes the event a more positive experience for everyone 5

The Purpose of LTR 6

Institutional responsibility The University has primary responsibility for the standards of awards made in its name and for providing the learning opportunities and experiences necessary to enable students to achieve those standards Design of policy guided by the Expectation and indicators of sound practice within Section B8 of the QAA’s UK Quality Code on programme monitoring and review Mechanisms for annual and periodic review: Annual Monitoring and Review (AMR) and Learning and Teaching Review (LTR) 7

QAA requirements – Chapter B8 Effective, regular and systematic processes must exist for the monitoring and review of programmes Deliberate steps must be taken to use the outcomes of programme monitoring and review processes for enhancement purposes Processes, roles and responsibilities for programme monitoring and review must be clearly defined and communicated 8

How LTR can add value: Self-reflection on whether programmes remain current and are effectively taught External and student input Constructive and positive engagement Opportunity to bring matters to the attention of Faculty and University Highlighting areas of exemplary practice 9

Outcomes of the LTR process A positive experience! Confirmation that standards are secure and that there is a high quality learning experience Re-approval of degree programmes for a further six year period Formal report considered by Faculty and University Recommendations to assist the subject area with continuous improvement of provision 10

LTR – Key Features 11

The key features of LTR... Development-led process that encourages reflection, the enhancement of teaching and learning processes, and the dissemination of exemplary practice Focus on quality assurance and quality enhancement – QA summary, QE and Technology-Enhanced Learning summary, and Library report A Review Team visit to the academic unit (approximately one day) The involvement of a student representative and an external subject specialist on the panel Role of LTR Chair as a ‘critical friend’ 12

Significant Changes from Internal Subject Review (ISR) to LTR 13

What’s New with LTR: Focus on centrally-held QA and QE information Academic units asked to produce a SWOT analysis instead of a Commentary Variable lengths of review visits, up or down from one day and depending on a number of parameters 14

Commentary vs. SWOT Commentary – Part of the Internal Subject Review Process – page descriptive assessment of the unit’s history, curriculum, student support, resources, etc. SWOT – Part of the revised Learning and Teaching Review Process – 5 pages maximum, providing overview and then identifying key Strengths, Weaknesses, Opportunities and Threats 15

Reading a SWOT: Overview page: key information, including unit history, student and staff numbers Strengths and Weaknesses are internal to the unit, Faculty and/or University Opportunities and Threats are external (industry-specific, funding shifts, etc.) 16

Reading a SWOT: Look for supporting evidence: can you find evidence in the documentation to back up claims made in the SWOT? Would you interpret that evidence differently? Consider the impact of the claims – what is the potential effect on the subject area? Consider the suggestions made by the unit on what will be done (either to build on a strength/opportunity or react to a weakness/threat) – are these realistic? Will they be effective? 17

Exercise One Read through the sample SWOT (Appendix 5) What information can you gather from a SWOT? How might you develop questions based on the SWOT? 18

Additional Documentation QA summary – prepared by the LTR Secretary based on all available QA documentation (contains overview of Quality Assurance information) QE & TEL summary – prepared by LTDS staff and fed into by the subject area (contains overview of the unit’s use of TEL, including ReCAP and ePortfolio, as well as information on learning & teaching enhancement activities) Library report – prepared by the relevant liaison librarian (contains overview of acquisitions and collections, plus information on engagement activities in the subject area) An Educational Partnerships report (if relevant) Access to selected Blackboard modules QA PDF – contains all subject-level, Faculty-level, and University-level QA documentation in one PDF (e.g. external examiners’ reports, programme regulations, Board of Studies minutes, NSS results). The PDF is searchable and bookmarked so that you can easily find specific documents, and a full list of expected documents can be found in the LTR policy. 19

How to Use Additional Documentation: Not every panel member needs to read every word. Use the summaries and SWOT to target your review. Consider assigning tasks to specific panel members. Use the documentation to find evidence for and more information about items mentioned in the SWOT. Key documents: External Examiners’ reports, student handbooks, Board of Studies minutes, SSC minutes, survey results 20

LTR Team Roles and Responsibilities 21

What does a team consist of? Chair – a UG or PG Dean from another Faculty or a designated nominee An academic from another Faculty (for all but the less-than-one-day reviews) An academic from the same Faculty, but from a different school A student rep nominated by the Students’ Union An external expert nominated by the subject LTR Secretary from the Learning and Teaching Development Service 22

Shared Team Responsibilities: Reading the SWOT, QA summary, QA &TEL summary, and library report before the pre-meeting Reviewing QA evidence as needed before the visit (PDF) Pre-visit meeting (may not include the external) and dinner Agreeing questions and schedule for visit Debriefing after meetings on the visit day Agreeing exemplary practice, commendations and recommendations Commenting on the draft report See Appendix 3 for a ‘swim lane’ of the panel’s responsibilities 23

Role of the External Member: Only subject specialist in the team Plays key role in assessing curriculum, standards, currency, compliance with benchmarks and PSRB requirements, and comparability of provision with that of other institutions Vitally important and required by QAA Often leads on curriculum issues and questions during the visit day 24

Role of the Chair Informal liaison with Secretary and subject area, including providing feedback on a draft SWOT Chairing the pre-visit meeting Ensuring that questions and schedule are agreed Keeping to schedule and keeping focused Managing the debriefing after each meeting, so that the key points are recorded Providing oral feedback at the end of the visit Final say on the draft report 25

Role of the Student Member A full member of the LTR team Selected from another subject area, and not there to represent the students in the subject Participates in pre-visit meeting and dinner May lead on questions about student experience 26

Role of the Secretary: Before the Visit Make arrangements and liaise with the subject area – including over the appointment of external and deadlines for documentation Provides feedback on draft versions of the SWOT analysis (with the Chair) Completes the QA summary Agrees the questions and meeting schedule with the review team Negotiates with the subject area over the schedule and other logistical arrangements 27

Role of the Secretary: During the Visit Takes notes during sessions Helps the team to debrief after meetings and records agreed key points Works with the chair and review team to agree exemplary practice, recommendations and feedback 28

Role of the Secretary: After the Visit Drafts the report and liaises with team members to agree draft Sends draft report to academic unit for factual checking Sends final report to subject, TPSC and relevant FLTSEC secretary Meets with academic unit to discuss potential responses to recommendations (with Chair) 29

Managing the LTR Effectively 30

Using scarce time profitably The visit has to include time for discussion and panel reflection as well as for meetings Questioning has to be very strategic and focused on the most important issues emerging from the team’s review of prior documentation Additional questions may emerge from the meeting with students, and prepared questions may be answered through discussion – be flexible See Appendix 2 for a sample visit day schedule 31

Preparing and asking questions efficiently Meetings and questions should be agreed in advance, including: a chair for the meeting (can rotate around panel), leads for each question, topics to pursue and rough time scales Sometimes you have to leave out a planned question because time runs out, so it is important to know what matters most Session chair needs to keep team and interviewees focused and prevent digression 32

How to Ask Questions Your questions should have a clarity of purpose: What do you want to find out? “Etiquette” of questioning Structuring lines of questioning: – what are you trying to do? – why are you doing it? – how are you doing it? – why is that the best way to do it? – how do you know it works? – how can you improve it? See also appendix 7 33

Exercise 2 In the meeting with students, concerns were raised by some PGT student representatives about their experience of personal tutoring. They indicated that they had raised this at the Student Staff Committee but had not received a response. This does not seem to support the assertion made by the School that in the last three years they have reformed personal tutoring and have fully implemented the new student representation policy. In groups, construct a line of questioning that explores this with members of staff. 34

After each meeting: Follow-Up After each meeting, the team should agree a small number of bullet points relating to the key issues identified in the session Be on the look-out for commendations and recommendations Decide if any questions need to be revised or added before the next meeting 35

Commendations and Recommendations What is exemplary? What is a commendation? Recommendations SMAR(T) goals 36

Exercise 3 Using Appendix 6, draft a recommendation for each of the scenarios listed. Where you feel that you would need more information, note this. Using Appendix 7, review the citations of exemplary practice from previous reviews. Which examples work best as ‘exemplary’ practice and why? 37

And finally... …thank you for attending this workshop Questions Contacts Feedback 38

Key documents The key documents are in the LTR section of the Quality and Standards Handbook: ernal ernal 39