1. Gemma Rauret Director Quality Assurance for Enhancement: a case study of ANECA (Spain) INQAAHE Conference 2009 30st March – 2nd April 2009 Abu Dhabi, United Arab Emirates

2. Tensions and challenges in the evaluation process Simple vs. diversified models for assuring quality The Spanish model for quality assurance The new quality assurance approach Table of contents

3. Tensions and challenges in the evaluation process in QA 1. Tensions and challenges in the evaluation process in QA

4. 1.Is it feasible the evaluation enhancement-oriented model? 2.Is it possible to harmonize an evaluation enhancement-oriented to another accreditation-oriented? 3.How to integrate an evaluation for accreditation with an evaluation for enhancement? Evaluation Process in QA: tensions and challenges

5.  Accountability vs. Quality Enhancement  Governmental Regulation vs. Self-regulation  Performance Indicators vs. Expert Opinion  Accreditation vs. On-going Quality Enhancement Can both approaches be integrated?  Purist perspective vs. Practical position  Stakeholders’ claims  Accreditation legitimacy  Integration in a supra-national framework : the EHEA Tensions in the two approaches in the evaluation Evaluation Process in QA: tensions and challenges

6. Higher Education Institutions:  HEIs are responsible for the quality of their educational offer  HEIs must be able to show quality both at the national and International level  A quality culture within HEIs must be promoted  Quality of academic programmes for students must be developed and enhanced Basic principles that rule the ESG Evaluation Process in QA: tensions and challenges Society:  Society’s interests concerning quality of higher education must be assured and safeguarded

7. Process:  Quality assurance accountability-oriented is compatible with enhancement  Processes should be compatible with diversity and innovation  Transparency and external experts are significant within the quality assurance processes  Processes allowing HEIs can show their responsibility concerning accountability for both public and private funds received should be developed Evaluation Process in QA: tensions and challenges Scope and Goals of the Evaluation:  National Higher Education Academic Rules  Programme and/or institutional accreditation  Users’ protection  Public access to the information about programmes and institutions  Quality improvement and enhancement

8. Simple versus diversified models for assuring quality 2. Simple versus diversified models for assuring quality

9. Simple or different evaluation models?  External and internal determining factors  Reality and evaluation culture of the system  Complexity, volume, tradition and legal frameworks  Problems to be undertaken  Potential of the organization to accomplish the evaluation –Internal quality units in the HEIs and – quality assurance agencies involved in the process  Common aspects of both models  An agency that co-ordinates the process  A self-evaluation report  A peer site-visit  A public final report Evaluation Process in QA: tensions and challenges

10. Stages where different problems will be undertaken/ the role played by the quality assurance/nature of the external review  Doubts regarding educational standards/ identification of programs placed under that threshold/ accreditation  Doubts regarding the efficiency of either the Higher Education system or the HEIs / Public accountability and creating awareness in institutions/ranking or report to state Evaluation Process in QA: tensions and challenges

11.  Doubts regarding Innovation capacity and QA capacity / stimulate self-regulation capacity of institutions. Public accountability/Audit to the institution  Need to stimulate sustainable Quality Culture in Institutions/ Improvement based on self regulation or public accountability /audit report to the institution  Decreasing Comparative Transparency across HE Systems/stakeholder information/ publication of performance indicators Evaluation Process in QA: tensions and challenges

12.  What kind of agency can provide responses to the challenges set up?  The nature of an agency :  learning organization  shared know-how  critical and independent thought  The technology of the agency:  Guides, standards and benchmarks  The process  Interaction with the HEI: moment and type of interaction  Ensuring the presence of different stakeholders Evaluation Process in QA: tensions and challenges

13. What kind of profile, training and role place the experts and agency’s staff in evaluation?  Experts –Highly qualified –Trained on the evaluation model to be applied –International experience  Agency’s staff –Highly qualified staff –On-going training: networking with other agencies’ staff –Network of collaborating experts

14. The Spanish model for quality assurance 3. The Spanish model for quality assurance

15. Spanish Higher Education SystemUniversities 75 universities all over Spain (50 public – 25 private) Public universities – 1.326.734 Private universities – 134.743 Total number of students enrolled 1.461.477

16. The Spanish Setting  Programme evaluation for improvement was introduced but without consequences (1985-2005).  The initial reaction was very stimulated although due to the lack of consequences the impact was rather unequal.  A stage without consequences is useful but cannot be extended for an indefinite period.  This stage led to the creation of QA agencies at the regional and the national level and QA units in HEIs. The Spanish model for QA

17. The Spanish Setting (2)  The EHEA and the quality assurance (2007 onwards). Problem to be undertaken and evaluation model  Ensuring that all official programs fulfilled the standards established in the legal framework.  Adopting the accreditation model with a few elements enhance-oriented  Adopting a national evaluation model for the teaching staff  Mandatory for bachelor and master programs  Implies funding and recognition of degrees for the public service  ANECA and other regional agencies are responsible for the external evaluation The Spanish model for QA

18. Design Programme design UNIVERSITY The programme accreditation process Accreditation Fulfilment evaluation Council Univ. Follow-up Implementation UNIVERSItY ANECA and regional Agencies Ex – ante Evaluation Verification Council Univ. ANECA The Spanish model for QA

19. How ANECA has undertaken the new challenge?  ANECA as a learning organisation  Innovation Area  Involvement at the International level  ENQA, ECA, RIACES and INQAAHE  Taking part in innovative projects  ANECA’s technology  What Tools?  Guide for universities  Guide for evaluation  Methodology, standards and benchmarks  How are they created?  Innovation Unit  Technical Committee for validation  Approval by the Board of Directors The Spanish model for QA

20.  The methodology: An evaluation procedure in three stages  Provisional Report oriented towards the fulfilment of the standards and improvement  Response of universities and enhancement  Final Report on the fulfilment of the standards  ESG Fulfilment  External reviewed in 2007: ENQA full membership  Included in the European Register for Quality Assurance Agencies (EQAR): December 2008 The Spanish model for QA

21. Profile and training of the agents involved in the evaluation Experts profile  Academic peers: expertise in QA, in programme evaluation, in International teams and members of the university staff  Professional Experts in the discipline specific field  Students coming from the discipline specific field at the suggestion of student unions  Experts in quality assurance Training  Specific training sessions in the agency  On the university system of Spain  On the running of the evaluation procedure The Spanish model for QA

22. Profile, training and role of the technical staff  Profile  Expert in quality assurance  Young academic staff with experience in quality assurance of higher education  Training subjects  On the university system of Spain  On the running of the evaluation procedure  On quality assurance Systems and the application of standards  Data processing  Role played by the technical staff of ANECA  Secretary of the committee  Improve the reference book to be used by the Experts in the committees  Co-ordinates the organisation of the meetings The Spanish model for QA

23. The new quality assurance approach 4. The new quality assurance approach

24. The main features: –Strengthening the internal QA system of HEI –A public information system –To give importance to improvement plans –Evaluation agencies should adopt a consultancy role –Funding as an incentive for on-going improvement The new QA approach

25. The internal QA Systems of the institution  Internal systems of the HEI are the only element really sustainable  Based on on-going improvement  Towards international quality levels  Best practices for benchmarking  Reporting on progresses  Improvement of teaching quality  Quality system core  Staff involvement at all levels Public information system  Characteristics of the programs  Planning and implementation of the learning process  Performance indicators The new QA approach

26. Quality improvement plan  Characteristics  Focused on learning quality enhancement  Built up from self assessment  Relevant actions for achieving the improvement goals  Feasible: human and material resources  Adequate time  Institutional commitment  Accountability of results The new QA approach

27. Conclusions remarks  The evaluation approach improvement- and voluntary- oriented creates COMMITMENT  The evaluation approach accountability- and mandatory- oriented only creates FULFILMENT Conclusions Both are necessary components for generating a quality culture

28. www.aneca.es Thanks for your attention