The Role of Accreditation and Assessment Commissions in HE QA Richard Lewis Former Pro-vice-chancellor of the Open University (UK) Immediate Past President of the International Network of Quality Assurance Agencies in Higher Education (INQAAHE) Role of QA Comm KSU Feb 20111

Summary of Presentation The role of QA agencies To whom they are accountable Their remit The “standard model” and variations thereon Current developments including increased emphasis on quality enhancement and the need to cater for a maturing yet diversifying HE system INQAAHE’s Guidelines of Good Practice for QA Agencies Role of QA Comm KSU Feb 20112

Definitions Quality Assurance Quality assurance is an all-embracing term covering all the policies, processes, and actions through which the quality of higher education is maintained and developed. (Campbell and Rozsnyai, 2002) Quality Assessment Quality Assessment covers both the means by which a judgement is made about the quality and standards of an institution or a programme and the judgement itself. (Vlãscanu, Grünberg and Pârlea, 2004) Accreditation Accreditation is a form of quality assessment where the outcome is a binary (yes/no) decision that usually involves the granting of special status to an institution or programme (CHEA 2001). Role of QA Comm KSU Feb 20113

The prime functions of QA (including accreditation) Agencies Are the institutions that fall within its remit Doing the right thing And are doing it right. Which begs the questions Who decides what is the right thing? How does an institution demonstrate that it is doing it right? Role of QA Comm KSU Feb 20114

To whom are QA agencies accountable? The wider community – not necessarily the same thing as government. QA agencies should work closely with the institutions for whom they have responsibility and should remain aware that it is the institutions that have ultimate responsibility for the quality and standards of the programmes they deliver and the overall academic environment they provide for their students. Role of QA Comm KSU Feb 20115

Remit of QA agencies Programmes only 20% Institutions only 16% Both 64% Notes 2008 figures based on the INQAAHE database. Above exclude programme specific agencies such as professional bodies Role of QA Comm KSU Feb 20116

The “Standard” model In an INQAAHE survey virtually all agencies stated that used the following approach Sets of regulations and guidelines formulated A self evaluation prepared by the institution The appointment of a peer group whose review of the institution or programme would start with a review of the self evaluation Site visits by the peer group. The publication of a report or, in some cases, only the decision Role of QA Comm KSU Feb 20117

Different approaches to applying the basic model Hard or Soft Hard Agency closely specifies content of self study report. Agency issues a grade as a result of the process Agency emphasises checking minimum standards as opposed to encouraging enhancement Assess rather than audit Role of QA Comm KSU Feb 20118

The difference between audit and assessment Audit QAA states that one of three objectives of institutional audit is to ensure that the institution has effective means of ensuring that the awards and qualifications in HE are of an academic standard at least consistent with those referred to in The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ) …. Role of QA Comm KSU Feb 20119

The difference between audit and assessment 2 Assessment “An institutional accrediting body evaluates an entire organization and accredits as a whole. It assesses formal educational activities and also evaluates governance and administration, financial stability, admissions and student personnel services, resources, student academic achievement, organizational effectiveness, and relationships with outside constituencies”. Higher Education Corporation Role of QA Comm KSU Feb

Publication – contrasting approaches Europe “Reports should be published and should be written in a way which is clear and readily accessible to its intended readership” (ENQA 2005) The United States “In most cases, the Commission will not make reports public without the permission of the college or university.” (HEC 2003) Role of QA Comm KSU Feb

Publication of Reports Role of QA Comm KSU Feb

In the early days QA agencies relied heavily on two main elements The intuitive judgement of the academic reviewers based, not on agreed explicit requirements, but on their experience Relatively crude quantitative input measures such as the ratio of academic staff (faculty) to students and the number of books in the library. Role of QA Comm KSU Feb

But things are changing Amongst the more important emerging trends are: A switch in emphasise from quality assurance for accountability to quality assurance for enhancement The change in focus from inputs to outputs The move away from the reliance on the intuitive experience of reviewers to an approach based on explicit statements of the requirements imposed on institutions and programmes Increasing attention to international issues including Cross-border Higher Education (CBHE) Role of QA Comm KSU Feb

Accountability and Enhancement Accountability is concerned with the institution being able to demonstrate that it is operating at or above the basic minimum standard, while quality enhancement is concerned with the continuous process of quality improvement. Role of QA Comm KSU Feb

As QA systems mature QA agencies will need to adapt their approaches after, say, two or three rounds of review or accreditation. But they might then have to deal with a range of institutions from the very well to the very newly established institutions. There are some good examples of dealing with this problem – but not very many! Role of QA Comm KSU Feb

INQAAHE – Guidelines of Good Practice for QA agencies Established in 1991 with about 10 members, now has over 200 members. Has for some years published Guidelines of Good Practice in Quality Assurance; latest edition August Specifically addressed to External Quality Assurance and Accreditation Agencies (EQAAs). Consistent with other guidelines; notably those of the ENQA (European Network of EQA bodies) Been adopted by many agencies as a yardstick against which to judge their own policies and procedures Role of QA Comm KSU Feb

The Guidelines Governance. Ownership and governance structure should be appropriate for the objectives of the agency Practical Implications – relationship with government, independence of EQAA especially in the making of “academic judgement.” 2. Resources - Should have adequate human and financial resources for day to day operations and for development Practical implications – how funded? Role of QA Comm KSU Feb

The Guidelines QA of the EQAA Needs a continuous system of QA for own activities and should be subject to external review at regular intervals Practical implications – how best to do? 4. Reporting Public Information Discloses its criteria etc and its decisions Practical implications – should full reports of evaluations be made publicly available? Role of QA Comm KSU Feb

The Guidelines The relationship between EQAAs and Higher Education Institutions (HEIs) EQAAs (and others) should recognise that quality is the responsibility of the HEI and that EQAA should contribute to quality enhancement as well as accountability. Practical implication – how best deal with system where HEIs exhibit very different degrees of maturity? Role of QA Comm KSU Feb

The Guidelines -4 6 The EQAA’s requirements for institutional/programme performance The EQAA should make clear of its expectations (sometimes described as standards, factors, precepts) which should address all areas that fall within the EQAAs scope including teaching, learning, research, community work etc and necessary resources. May also include specific learning goals Practical implications – how specific should these be, how are they established and how should specific learning goals be specified? Role of QA Comm KSU Feb

The Guidelines The EQAA’s requirements re Institutional Self- Evaluation -EQAA needs to explain to institutions (and others) the self-evaluation process -Should apply standards and criteria that have been subject to reasonable consultation with stakeholders -The process should contribute to quality improvement. Practical implication – how balance the interests of different stakeholders? Role of QA Comm KSU Feb

The Guidelines -6 8 The EQAA’s Evaluation of the institution and/or programme - EQAA should publish standards used, assessment methods, and decision criteria. Should specify approaches to selection and training of reviewers. Should ensure institutions are evaluated in an equivalent way. Where practicable panels should include one reviewer from another jurisdiction. Practical implications – include how best ensure equivalence Role of QA Comm KSU Feb

The Guidelines -7 9 Decisions An EQAA must be independent i.e. has autonomous responsibility for its operations and its judgements should not be influenced by third parties. When the EQAA advises governments or other public bodies the decisions of each agency should be made as independently as practicable Practical implications – how best to ensure independence of decision making (how best balance with needs of government). Is it realistic to expect that the EQAA can have complete autonomy about its style of operation? Role of QA Comm KSU Feb

The Guidelines Appeals The EQAA should have appropriate methods and policies for appeals. Appeals should be conducted by those not responsible for the original decision but appeals should not necessarily be conducted outside the EQAA. Practical implications – should there be an external court of appeal, how best to avoid a situation where there are many appeals? Role of QA Comm KSU Feb

Summary of the GGP Funders -Take their money but not their instructions. But talk to them as well as to other stakeholders. Do unto yourself that you do unto others -Establish effective systems of internal and external quality assurance for the agency The public (broader community) -Speak to it. Role of QA Comm KSU Feb

My thanks for your attention. I very much look forward to the discussion. Richard Lewis Role of QA Comm KSU Feb

References Campbell and Rozsnyai, 2002, Quality Assurance and the Development of Course Programmes. Papers on Higher Education Regional University Network on Governance and Management of Higher Education in South East Europe Bucharest, UNESCO. CHEA, 2001, CHEA web site, section on “International Quality Review”, CHEA glossary of terms ENQA 2005, Standards and Guidelines for Quality Assurance in the European Higher Education Area, European Association for Quality assurance in Higher Education, Helsinki HEC 2003, Handbook of Accreditation, Higher Education Commission, Chicago Vlãsceanu, L., Grünberg, L., and Pârlea, D., 2004, Quality Assurance and Accreditation: A Glossary of Basic Terms and Definitions (Bucharest, UNESCO-CEPES) Role of QA Comm KSU Feb