INSTITUTIONAL REVIEWS 2019 PRE-REVIEW WORKSHOP 5 July 2019

Time Activity 8.30 – 9.30 am Registration 9.30  - 10.00 am Introduction: objectives of workshop, programme review process and role of reviewers 10.00 – 10.30 am Documentation: distribution of SERs, Declaration of Interests form, Letter of Appointment and Agreement with UGC 10.30 – 10.45 am Tea 10.45 – 11.15 am Desk evaluation 11.15 am – 12.00 pm Site visit schedule and effective use of meetings 12.00 – 12.30 pm Scoring standards and calculation of final grade 12.30 – 1.30 pm Lunch 1.30 – 2.00 pm Tentative dates for site visits 2.00 – 2.30 pm Preliminary report, draft report and final report formats, deadlines for submission 2.30 – 3.00 pm General discussion 3.00 pm Tea and close

SESSION 1. INTRODUCTION

Objectives of workshop Formal aspects Hand over SERs to assigned review panels + letters of appointment Signatures on Agreement with UGC Agree on dates for site visits Training aspects Ensure that all reviewers know what is required of them in the EQA process conducted by the QA

Background Third year of Institutional Reviews using current manual 2017 – Colombo, Sri Jayewardenepura, Visual & Performing Arts, Uva Wellassa 2018 – Jaffna and Wayamba 2019 – Moratuwa, Sabaragamuwa, OUSL, South Eastern 2020 – Ruhuna, Kelaniya

Role of reviewers in EQA Reviewers are absolutely essential for the effectiveness and success of EQA Need to work as a team, with Review Chair as the team leader Tasks need to be divided up between team members before, during and after site visit Only the initial desk evaluation is to be done individually

Reviewer profile – key characteristics High degree of professional integrity and objectivity An enquiring disposition Ability to readily assimilate a large amounts of disparate information Ability to make appropriate judgments in the context of complex institutions different from their own Personal authority and presence Ability to act as an effective team member Good time management skills Ability to give effective oral feedback

Reviewer profile, ctd. Experience in organization and management, particularly in relation to teaching and learning matters High standard of oral and written communication, preferably with experience in writing formal reports Knowledge and understanding of the review topics, principles, and concepts Knowledge of the special characteristics and conditions of the educational provision to be reviewed Knowledge of quality assurance and quality enhancement procedures

Composition of review panels Tried to include: 6 Members with expertise in broad fields of study relevant to HEI Balance of Review Experience, Universities & Gender Took into account: University affiliation of each reviewer Concerns expressed by HEI under review

Institutional Review Process Desk Evaluation of SER by reviewers SITE VISIT TO VALIDATE CLAIMS IN SER Preliminary report Draft report Comments from HEI on draft report FINAL REPORT FR edited and published by QAC University’s Action Plan for implementation of recommendations

Deliverables required of reviewers Individually: Desk evaluation report As a team: 2. Preliminary report (key findings) 3. Draft report 4. Final report

SESSION 2. DOCUMENTATION

SESSION 3. DESK EVALUATION OF SER

Deliverable 1. Each reviewer is expected to assess the SER using a pre-formatted Excel file provided for this purpose Assign scores for each standard, by comparing what is given in the SER and the best practice listed in the Manual Make notes on any items that you would like clarified during the site visit Not necessary to complete final summary sheet (i.e. award final grade)

Pre-formatted Excel file for IR scores Save file with abbreviated name of university with your initials added at end E.g. OUSL_NRdeS.xls, MRT_NRdeS Email Excel file to QAC (dqac@ugc.ac.lk with copy to qaac@ugc.ac.lk) before pre-site visit meeting on 2 August Bring copy to pre-site visit meeting, for discussion with rest of review panel

Questions?

SITE VISIT SCHEDULE AND EFFECTIVE USE OF MEETINGS

Role of Review Chair Contact VC / Director IQAU ahead of site visit and agree on schedule for site visit Decide on allocation of tasks in consultation with team members Chairing and participating in meetings Going through supporting documents for specific criteria / standards Inspecting infrastructure and facilities Writing up sections of draft report Take the lead in initial and wrap-up meetings with VC Compile and edit draft report Submit final report as soft copies + one hard copy

Site visit schedule Duration – 5.5 days Programme should be broadly agreed upon prior to site visit Generic format of site visit schedule provided by QAC should be modified to suit requirements of University, and any contingencies that emerge during site visit

Effective use of meetings: ‘DO’s Use meetings as an opportunity to triangulate evidence presented in documentary form Go prepared with list of questions that need to be answered by participants in each meeting Make a note of specific questions in relation to the criteria and standards assigned to him /her Use open-ended questions to start with, and specific questions when clarity is needed Be punctual and stay with the agreed program for meetings Keep attendance records and written notes of all discussions

Effective use of meetings: DON’T s Don’t allow one or two persons to dominate a meeting with a group Don’t get distracted into discussions that are irrelevant to the IR Don’t go beyond the time allocated for the meeting, and get late for the next one, and the next…

Questions?

SCORING EACH STANDARD AND CALCULATION OF FINAL GRADE

= = = 145 Standards Programme Review Quality Framework Criterion 1 2 3 6 7 8 9 10 Number of Standards 29 15 10 14 11 25 6 13 7 Programme Review Quality Framework = = = 145 Standards

Assigning Scores for Standards

Guidance for decision-making Question 1. What is the recommended best practice for this standard as stated in the Institutional Review Manual? Question 2. What is the claim made by the University regarding their own practice(s) as stated in SER? Question 3. What evidence does the University provide to support this claim, as stated in the SER? Question 4. Do the Panel’s observations during the site visit support the claim?

1. Degree of internalization of best practices and level of achievement of Standards 2. Degree to which the claims are supported by documented evidence 3. Accuracy of the data and statements made in the SER

Claim of internalization of best practice Meets standard Evidence sufficient to support claim 3 marks Evidence not sufficient to support claim 1 or 2 marks Below standard 1 mark No claim of achievement 0 marks

Calculation of final grade Raw scores will be automatically converted to weighted actual scores on Excel file (Worksheet entitled ‘Summary scores’) For each criterion, check if the weighted actual score is above the weighted minimum score Check on total actual score Use table provided in worksheet entitled ‘Summary scores’ to determine final grade

Award of final grade Criterion-wise actual score Total actual score (%) Grade Equal to or more than the minimum weighted score for all ten criteria 80 - 100 A 70 - 79 B 60 - 69 C <60 D Equal to or more than the minimum weighted score for nine of the ten criteria 70 - 100 Equal to or more than the minimum weighted score for eight of the ten criteria 60 - 100 Irrespective of minimum weighted criterion scores

Final grade Grade descriptors A Very good B Good C Satisfactory D Review panel must decide on final scores and grade together during last session on Day 6 Convey main findings (criterion by criterion) during final wrap-up meeting First talk about observed strengths (commendations) Then talk about observed weaknesses and recommendations for improvement Grade descriptors A Very good B Good C Satisfactory D Unsatisfactory

Questions?

SESSION 4. DATES FOR SITE VISIT

SESSION 5. TEAM REPORTS

DELIVERABLE 2. PRELIMINARY REPORT Completed Excel file + Word document with brief details of programme under review Review Chair to submit to DQAC by email, within 2 weeks of completing the site visit

Format of preliminary report: title page Institutional Reviews 2019 conducted by the Quality Assurance Council University Grants Commission, Sri Lanka    PRELIMINARY REPORT University: Review Panel: Site Visit Dates:

Format of preliminary report: contents Section 1. Background (half to one page)   When the university was started: Number of Students in university at present - breakdown by Faculty and year of study: Maximum number of students enrolled in the last five years: Numbers graduated from the university over the past five years Section 2. Criterion-wise strengths and weaknesses Criterion 1: Governance and Management Strengths: Weaknesses:   Criterion 2: Curriculum design & development Criterion 3 etc

Format of preliminary report: contents, ctd. Section 3. Final evaluation No Criterion Weighted minimum score* Actual criterion-wise score 1 Governance and Management 90   2 Curriculum Design and Development 60 3 Teaching and Learning 50 4 Learning Resources, Student Support and Progression 40 5 Student Assessment and Awards 6 Strength and Quality of Staff 7 Postgraduate studies, Research, Innovation and Commercialization 8 Community Engagement, Consultancy and Outreach 30 9 Distance Education 20 10 Quality Assurance Total score (out of 1000) Total score (out of 100) Final grade:

DELIVERABLE 3. DRAFT REPORT Format set out in Institutional Review Manual (p 103–106) Section 1: Introduction to university Section 2: Observations on SER Section 3: Description of review process Section 4: University’s approach to quality and standards Section 5: Judgment on each of the 10 criteria Section 6: Grading of overall performance Section 7: Commendations and recommendations Section 8: Summary Annexures

Format of Draft Report: cover page University logo on left UGC logo on right Details of the IR on top with the year and dates of review Photograph of University Names of reviewers below photo Name-UGC and QAC bottom of page

Format of draft report: signature page Scanned copy of page with signatures to be inserted after cover page University: Review Panel: Name Signature Date:

Format of draft report List of Contents should include relevant pages of each Section and Criterion (1-10/1-8) Each main Section (1-8) to be started on a new page, with heading aligned centre and subheadings left aligned Font and size Calibri font 14 (bold) for main Section headings Calibri font 13 (bold) for sub-headings Calibri font 12 for text in paragraphs Text colour: black Line Spacing Main Section headings and text – 3.0 Subheadings and content – 2.0 Paragraphs – 1.5 Sentences in paragraphs – 1.15

Annexures Annexures must include: final schedule for site visit all attendance sheets from meetings Optional: additional photographs taken during site visit (reduce file size)

Format of draft report ctd Numbering of tables and figures: Have the number of the section and relevant number of the section: e.g.1.1, 2.3  Capitalization in text: use only for abbreviations and as recommended for names of people, departments, universities, etc. Word limit: 12,000 words

Submission of Draft Report to QAC Deadline for submission to QAC (by email, as Word document): within 6 weeks of completing site visit Draft report will be emailed by DQAC to VC, for corrections / comments (together with Excel file in Preliminary Report), to be sent back within 3 weeks

DELIVERABLE 4. FINAL REPORT Comments from VC will be sent to Review Chair, for consideration in finalizing the report in consultation with the Panel Final Report may be the same as the draft report, if no amendments are requested If the Review Panel considers it appropriate, comments from the VC should be incorporated as changes to the draft Review Chair is required to submit the Final Report in soft copy (Word and pdf formats) and a single hard copy

Publication of Final Report Final Report will be sent back to VC prior to publication Minor factual errors may require correction; but no major changes Final Report is sent to one of a panel of editors prior to publication Edited Final Report will be uploaded to QAC website and sent to VC University is required to prepare an Action Plan for implementation of recommendations in report

Access to document formats etc https://www.eugc.ac.lk/qac/events2015_2017.html

Questions?