Martí Casadesús Director Effectiveness of Quality Assurance: How can we prove it?

Slides:



Advertisements
Similar presentations
1 Bologna Shaping the Agenda Bologna today and tomorrow Lesley Wilson Secretary-General, European University Association.
Advertisements

Potential impact of PISA
Future Trends on Student Involvement in Quality Assurance Agencies
14 April 2008 Cecilia de la Rosa Using ISO in IQA ENQA Seminar Hosted by AQU Catalunya Internal QA-systems and the ESG Barcelona, April 2208.
Setting internal Quality Assurance systems
Joint Degree Programmes: QA across national borders – ideas and strategies Nick Harris Quality Assurance Agency for HE UK.
European Inventory on Validation of Non-formal and Informal Learning 2010 Jo Hawley, Project Manager Brussels, 12 December 2011.
An Overview of Quality Assurance in the EHEA by Prof. Andreas G. Orphanides President of EURASHE, Rector of European University Cyprus, and Ex-President.
Quality Assurance: Dimension of the Bologna Process Gayane Harutyunyan Bologna Secretariat June 10-11, 2014 Munich.
Review of different stakeholders needs in relation to Joint Assessment of National Strategies and Plans (JANS) Preliminary Findings IHP+ Country Teams.
Project Monitoring Evaluation and Assessment
La qualitat, garantia de millora. La qualitat, garantia de millora Student involvement in QA in Catalonia ALBERT BASART.
1 Experiences of Using Performance Information in the Budget Process OECD 26 th March 2007 Teresa Curristine, Budgeting and Public Expenditures Division,
National Commission for Academic Accreditation & Assessment Preparation for Developmental Reviews.
The quality assurance system in Sweden Håkan Hult Linköping University Gdansk March 13, 2009.
Regulating the engineering profession 1 EC UK Experience in Accreditation of Engineering Programmes Professor Ian Freeston University of Sheffield, UK.
© 2014 Public Health Institute PROPOSAL WRITING.
Standards and Guidelines for Quality Assurance in the European
National Frameworks of Qualifications, and the UK Experience Dr Robin Humphrey Director of Research Postgraduate Training Faculty of Humanities and Social.
1 Module 4: Designing Performance Indicators for Environmental Compliance and Enforcement Programs.
Slide 1 D2.TCS.CL5.04. Subject Elements This unit comprises five Elements: 1.Define the need for tourism product research 2.Develop the research to be.
Quality Assurance in Europe: Challenges and Opportunities Maria Helena Nazaré EUA President Former Rector Universidade de Aveiro, Portugal.
Quality Assurance in a Changing World María José Lemaitre INQAAHE Conference Abu Dhabi, March 2009.
Quality in Education and Training
ERASMUS MUNDUS ACTION 1B – Joint Doctorate Programme Marine Ecosystem Health & Conservation Dr Ian O’Connor GMIT
The Structure and Role of QA Bodies at the University and faculty/department levels UNIVERSITY OF BELGRADE Serbia.
1 External evaluation of Higher Education in the Netherlands and Flanders (case NVAO) Dr ir Guy Aelterman Graz, 11 May 2006.
Quality Assurance in the European HEA Enrique Lopez-Veloso University of Santiago de Compostela, Spain Agustin Merino National Team of Bologna Experts.
Implementing Global Climate Change Research: Assessing the Challenge of Defining and Evaluating Decision Support Rebecca J. Romsdahl, PhD – Earth System.
Swiss Quality Assurance & Accreditation System in Higher Education: Perspectives for Southeast Asia By Dr. Alexandre Dormeier Freire, Graduate Institute.
ENQA a key player in the European Higher Education Area Meeting of the Belarus University System representatives Minsk, March 2013 Josep Grifoll / Жузэп.
MANAGEMENT SYSTEM AND STRUCTURAL ORGANIZATION FOR THE QUALITY ASSURANCE OF SPs ■ Slovak University of Technology in Bratislava ■ Faculty of Civil Engineering.
S WISS Q UALITY A CCREDITATION S YSTEM IN H IGHER E DUCATION : P ERSPECTIVES FOR S OUTHEAST A SIA By Dr. Alexandre Dormeier Freire, Graduate Institute.
Role of University Rankings in Kazakhstan Prof. Sholpan Kalanova BRATISLAVA 2011.
Quality Assurance – European Standards and Guidelines.
National Commission for Academic Accreditation & Assessment Developmental Reviews at King Saud University and King Faisal University.
QUALITY ASSURANCE IN BULGARIAN HIGHER EDUCATION Prof. Anastas Gerdjikov Sofia University March 30, 2012.
FOURTH EUROPEAN QUALITY ASSURANCE FORUM "CREATIVITY AND DIVERSITY: CHALLENGES FOR QUALITY ASSURANCE BEYOND 2010", COPENHAGEN, NOVEMBER IV FORUM-
Professional Development for Citizenship Teachers and Leaders Professor Michael Totterdell Mike Mitchell.
Implementing the WIPO Development Agenda: Comparing National Approaches to Promoting Coherence Between Public Policy Objectives and IP Laws ICTSD Roundtable.
La qualitat, garantia de millora AQU Catalunya. Agency for Quality Assurance in Higher Education of Catalonia Projects Esteve.
ESG Part 2: European standards for the external quality assurance of higher education Conference on self-evaluation July, Belgrade Lewis Purser.
Universitat Autònoma de BarcelonaSplit, October 2002.
CHE Business Plan Mission The mission of the CHE is to contribute to the development of a higher education system that is characterised by.
Strengths and weaknesses of the implementation of the Bologna Process: Spain Round Table Bratislava,10-12 October Guillermo Bernabeu, University of Alicante.
María Amor Barros del Río Gender as content in research in Horizon 2020 GENDER AS CONTENT IN RESEARCH IN HORIZON 2020 CAPACITY BUILDING WORKSHOP FOR RESEARCHERS.
Curriculum reform Tirana 16th-17th March. A bit about ESIB ESIB-the National Unions of students in Europe is an umbrella organization representing over.
ROMANIA MINISTRY OF EDUCATION, RESEARCH AND INNOVATION National Centre for Development of Vocational Education and Training Implementation Unit of Phare.
1 Quality Assurance System UOC Planning and Quality Office
Human Rights management from the Spanish Network perspective 24th January 2011 Valentine Papeians de Morchoven Responsible for Human Rights – GC Network.
Kathy Corbiere Service Delivery and Performance Commission
ESG 2015: Linking external and internal QA Involving stakeholders Tia Loukkola Director for Institutional Development 22 January 2016.
February, MansourahProf. Nadia Badrawi Implementation of National Academic Reference Standards Prof. Nadia Badrawi Senior Member and former chairperson.
30/10/2006 University Leaders Meeting 1 Student Assessment: A Mandatory Requirement For Accreditation Dr. Salwa El-Magoli Chair-Person National Quality.
Some business of External QA: Transparency (reports), measuring impacts, follow up implementation, expected benefits, strategies for the future Josep Grifoll.
PUBLIC ACCREDITATION AGENCY FOR HIGHER EDUCATION “Key aspects of quality assessment for teaching and learning in HE” Niko Hyka Innovation and information.
Quality Assurance in Egypt and the European Standards and Guidelines
Quality Assurance in Higher Education
PRESENTATION OF MONTENEGRO
Accountability: an EU perspective
Project No EPP EL-EPPKA2-CBHE-JP ( ) UES TEAM
NCEQE Conference 6 December 2016
UNECE Work Session on Gender Statistics, Belgrade,
Preparations for post-2020 Impact Assessment European Commission Directorate General for Regional and Urban Policy Unit DGA Policy.
Senior project leader at CIEP Former President of ENQA
quality culture in the EHEA
Recognition Tools and Practices – Criteria and Procedure
Panel “Key performance indicators for Serbian higher education“
Indicators&Criteria in External Quality Assessment
National Correspondents for Qualifications Frameworks (QF-EHEA)
Presentation transcript:

Martí Casadesús Director Effectiveness of Quality Assurance: How can we prove it?

2 Overview 1.Objectives of external QA in HE 2.QAA context 3.Mission of QAA and effectiveness 4.Incorporating objectives to QAA mission 5.Measuring the effectiveness of QAA 6.Questions

1. Objectives of external QA in HE The main objectives for external QA in HE are: -Improving the quality of the HE system -Promoting accountability of the HE system

4 Knowing our own HE system to calibrate the objectives How is our own HE system?  Is it compacted / heterogeneous?  Is it in a process of being reengineered?  Is it in a process of growing?  Is it under a policy of public budget expansion?  Is it experiencing budget cuts?  Is it working under the principle of autonomy for HE? The nature of our HE system has big influence on how QA should be articulated and what can be expected as results of QA and its effectiveness. 2. QAA context

5 Stakeholders with different expectations and different concepts of quality Governments wants to maximise the (social and electoral) benefits of their public investments Universities compete for resources and talent Students wants to maximise their investment considering HE as a private good Etc…. Political pressure on the work of QA agencies

6 Mission A “ambitious”: improving the quality of HE Pressure on the QA agency (QA has a power) Mission B “modest”: Providing information that can be used by stakeholders Pressure on the stakeholders (QA is seen as a service) What is the balance? 3. Mission of QAA and the effectiveness

7 The way in which QA agencies describe their aims and goals is essential to measure the effectiveness of QA. Expectations in agencies missions should have a reasonable system to measure their achievements. Missions require realistic expectations, but ambitious wishes should not be underestimated, even if just realistic goals can be measured.

8 4. Incorporating objectives to QAA mission How objectives are incorporated in the missions of QA agencies? What is your balance? 1. Improving the quality HE 3. Providing relevant information for decision makers 2. Improving the QA management of HEI Some agencies highlight this area Some QA agencies prefer to emphasise this aspect

9 4. Incorporating objectives to QAA mission How objectives are incorporated in the missions of QA agencies? What is your balance? 1. Improving the quality HE 3. Providing relevant information for decision makers 2. Improving the QA management of HEI Some agencies highlight this area Some QA agencies prefer to emphasise this aspect

10 Objective 1: Improving the quality HE Example: Program accreditation 4. Incorporating objectives to QAA mission

ACCESTEACHING DESIGN ACADEMIC RESOURCES GLOBAL PROCESS QAA UNIVERSITY ACADEMICS 11 Qualitative results Accreditation impact: … there is a wide range of opinions… From a Survey acredittacion process from AQU Catalunya (2015)

My assessment is positive. This report leads to the conclusion that the commission analysed the proposal in detail and "detected" both specific technical-type deficiencies and more important defects whose correction substantially improved the final proposal. The suggestions and requests were very well-expressed and therefore have proved to be useful. (Respondent 1) 12 Accreditation impact: … there is a wide range of opinions… It is an opaque, very bureaucratic process. The emphasis was always on formal aspects with no practical importance, instead of aspects that (at least in my opinion) have a real effect on teaching quality. (Respondent 2) From a Survey acredittacion process from AQU Catalunya (2015) Quantitative results

13 4. Incorporating objectives to QAA mission How objectives are incorporated in the missions of QA agencies? What is your balance? 1. Improving the quality HE 3. Providing relevant information for decision makers 2. Improving the QA management of HEI Some agencies highlight this area Some QA agencies prefer to emphasise this aspect

14 Objective 2: Improving the QA management of HEI Example: WINDDAT: Web-site for HEI system indicators 4. Incorporating objectives to QAA mission

15 4. Incorporating objectives to QAA mission How objectives are incorporated in the missions of QA agencies? What is your balance? 1. Improving the quality HE 3. Providing relevant information for decision makers 2. Improving the QA management of HEI Some agencies highlight this area Some QA agencies prefer to emphasise this aspect

16 Objective 3: Providing relevant information for decisions makers Example: Surveys (employability, employers, graduates, …) Study on the labor market outcomes of graduates  Survey carried out once every three years (2001, 2005, 2008, 2011, 2014)  Graduates, Master’s and doctorates  More than surveyed graduates Employers Survey  Employers Survey (2014) Graduates satisfaction Survey  Satisfaction survey degrees and masters Incorporating objectives to QAA mission

17 4. Incorporating objectives to QAA mission How objectives are incorporated in the missions of QA agencies? What is your balance? 1. Improving the quality HE 3. Providing relevant information for decision makers 2. Improving the QA management of HEI Some agencies highlight this area Some QA agencies prefer to emphasise this aspect

18 Objectives 1&2&3: Improving the quality HE & Improving the QA management of HEI & Providing relevant information for decisions makers Example: EUC: Website Quality University Programs in Catalonia 4. Incorporating objectives to QAA mission

19 5. Measuring the effectiveness of QAA How to measure the effectiveness of QAA? For those agencies looking for their impact on how the quality of HE is improved: -How to be sure that this is because of QA? -What is the impact of other factors in that improvement? Measuring the effectiveness of QAA is not easy!!!

20 5. Measuring the effectiveness of external QA is not easy Some direct measures: Providing information for decision-making processes seems less complicated: -Use of the reports for taking decisions on HE -Satisfaction of the users and readers of the reports -Political decisions related to the evaluations reports -QAA evaluations -….

21 5. Measuring the effectiveness of external QA is not easy Some indirect measures: Exemple: The Catalan University System in International Rankings

22 5. Measuring the effectiveness of external QA is not easy Some indirect measures: Exemple: HE evaluation, Employees & Graduates

23 6. Question 1: Why do you need external QA in your HE? Although there are different priorities, according each context, we need to measure its efectiveness on the same way?  QA is required by law.  QA is a must in a global market  QA is a tool in a national system in expansion  QA is used to distribute the use of public funds  QA is to support less developed institutions  QA is to improve our position in world rankings  ….  QA is needed it just to improve HE  QA is needed to improve the QA management of HEI  QA is needed to provide relevant information for decisions makers  ….

24 6. Question 2: How to make external QA more effective in your context? (with more positive impact)  Being specialised in a single procedure for all.  Having a risk-based strategy  Using a combination of procedures  Promoting internal QA in HEI  Using incentives to link quality with greater autonomy or additional funds  Improving the use of quantitative data in QA  Internationalisation of QA  …

25 6. Question 3: Which are the QAA activities that have more impact? Why? Institutional vs Program evaluation Ex-ante vs Post evaluation Improving the QA management of HEI vs Providing relevant information for decisions makers Internal vs External controls Reports: Content vs Publication (transparency) …

26 6. Question 4: How could you measure the effectiveness of the main QAA activity (aligned with your objectives)? Direct or indirect measures? Questionnaires to reviewed units? Questionnaires to the stakeholders? Research on the impact of QA Analysis conducted by independent experts Use of quantitative indicators Use of qualitative indicators / satisfaction Cross-check analysis …

27 6. And don’t forget that many times...

Thanks very much for your attention El copyright d’aquesta presentació (en format paper i electrònic) pertany a l’Agència per a la Qualitat del Sistema Universitari de Catalunya. Qualsevol reproducció, distribució, transformació o presentació, total o parcial, del seu contingut requereix l’autorització expressa i per escrit d’AQU Catalunya, i la referència a AQU Catalunya com a font d’informació.