Some business of External QA: Transparency (reports), measuring impacts, follow up implementation, expected benefits, strategies for the future Josep Grifoll ENQA & AQU Catalunya

Why do you need external QA for BiH Higher Education? Institutional development Market Purposes Market Purposes National policies for HE Adaptation to EHEA Regulation of QF Autonomy of HEI Adaptation to EHEA IQA Competition (rankings) Public and private HEI Other providers ! !

Why Europe is implementing External QA? Accountability (ESG): HEI should have adequate means (accreditation, recognition, public external evaluation reports) to explain and prove (nationally and internationally) that institutions provide good (and excellent) services. Trend to ensure and to increase the autonomy of HEI (need to enhance internal quality assurance, capacity to improve). Adequate use of public funds (efficiency and efficacy).

Why Europe is implementing External QA? Other elements supporting accountability (ESG): HEI should have a policy for QA (and its own quality concept?) Quality culture development, students role in QA, continuous improving systems… Information systems (students academic performance, satisfaction rates, employability…) Public information: public programme specification and the self- regulation approach.

Why Europe is implementing External QA? Enhancement What is the institutional capacity for designing and implementing improvement plans? Dissemination of good practices: the QA process in one programme can produce changes in other areas. External evaluation reports can lead to consequences of accreditation but is there a complementary added value result. Possible accreditation with conditions or recommendations: What follows the external evaluation process?

Different European Approaches Focus of the evaluation (institutional or programme) Moments of evaluation (initial…) Topics of evaluation (specialised…

Different European Approaches Focus of evaluation: Quality of the programme (NL, E, D…) accreditation process trying to keep the aim of enhancing… Quality of the HE institutions (SF, N, UK, F…) basically is an enhancing oriented process, some cases addressed to institutional recognition. Thematic evaluations (students support, libraries, internationalisation,…)

Different European Approaches DimensionMomentContext Quality of inputsInitial and periodicRules to select new programmes Quality of processessPeriodic after implementationEquity of provision Quality of resultsPeriodic after implementationEquity? Excellence? External QA systems are no statics: standards, objectives and dimensions, moments and contextes change regularly

Transparency (reports) “The external reports” the first immediate product of external QA: information, accountability, enhancement… The content of the external report? Short, long, with recommendations, strong and weak points, indicators. Legitimacy of the external panels…. External report ownership? Agency, panel… who is granting a common template, the similar use of language in the reports… Degree of transparency? Public as it is drafted by the panel, summary, orientation to different stakholders

Some additional ideas about impacts Any formal decisions made as a result of an external quality assurance activity should be based on explicit published criteria that are applied consistently. But, it is important to open the way to measure the future impacts of the external QA: in particular recommendations formulated in the reports. A good idea is to propose critical indicators to be followed (qualitative, quantitative…). The system could measure the satisfaction of the external quality assurance (at short and mid term)

Strategies for the future Having a first programme accreditation round provides a very detailed picture on the quality of the programme. Usually the accreditation standards are a bit higher than the current situation (this is to use the exercice for enhancing purposes). How to prepare the following round?

External panels’ work 1. At first, make an analysis of the self-assessment document (individually). Personal composition about the quality of the programme. Important! in this initial position, consider your perspective and personal background as expert. 2. Composed view with the rest of the experts in the panel. Try to find a common line of argumentations.

External panels’ work 3. Make a list of topics (questions). The core elements that should be treated in an hypothetical improvement plan. 4. Distribution of the topics according different audiences. 5. If you have received and agenda for the site visit, you should confirm the agenda or suggest changes according the information you have after reading the self-assessment document. 6. Suggest complementary evidences, not included in the report. 7. Who is going to formulate the questions during the site-visit? The role of the president, the role of academics and professionals. The role of the student.

External panels’work 8. During the site visit: introduce yourselves and the project (including the aims in which the work of the panel is inscribed). 9. Formulate the questions with a positive tone and avoid to provide conclusions, and ask for evidences, taking into account the self- assessment report and your pre-visit suggestions. 10. Essential to have the time for the panel deliberations during the site visit. Make sure your agenda is including time for deliberations. 12. Direct observation? This is not a common practice but… 13. The use of learning outcomes in your evaluation.

External panels’work 14. Compose a general view for an oral feed-back to be presented at the end of the site visit. 15. Prepare the report* 16. Deal with the university reaction and agree on a final version of your report.