Institutional quality assurance First Croatian Bologna Seminar International Center for Education of Journalists – ICEJ Opatija, 4 – 5 March 2005 Pero Lučin University of Rijeka Institutional quality assurance Croatia joined the Bologna process 2001. and expressed full commitment to participate in EHEA. Since that, intensive activities and discussions – What is done in two years.

2002. 2004. The Bologna Process Priorities for 2005. Two cycles European countries with two-tier structure (Bachelor-Master) No, planned Yes Yes, process started No, planned Yes Yes, process started 2004. 2002. Priorities for 2005. Two cycles Third cycle (doctoral studies) Quality assurance system Recognition

Poimanje kvalitete visokog obrazovanja u Europi Higher-education institutions Public High degree of institutional autonomy Public responsibility of an institution Internal mechanisms External evaluation Dva ekstrema – većina između Humboltovski pristup – kvaliteta iza autoriteta velikog profesora Problem – interpretacije autonomije institucija Zatvaranje kroz interpretaciju autonomije pojedinca Javni interes sve veći – javno dobro Najčešće javnost ima iskrivljenu sliku – zato što sustav nije transparentan prema javnosti Drugim riječima: Ne znamo jeli to istina – teško je dokazati da smo loši, ali i ne možemo ni dokazati da smo dobri Rezultat: odnos javnosti prema visokom obrazovanju i institucijama – Izoliranost, koja mnogima odgovara Minimum of external regulation or evaluation (and that at the level of the whole institution) Frequent inspection at the level of the programme or qualification.

View of the appropriate relationship that should be established between higher education institutions and their external evaluators Balance Organisations which accredit programmes or institutions Higher education institutions Accountability Improvement improvement External quality assurance is essentially a matter of ‘consumer protection’ Requiring a clear distance to be established between the evaluation agency and the higher education institutions whose work they evaluate Provision of advice and guidance in pursuit of improvements in the standards and quality of programmes of study and associated qualifications. . Close relationship between the evaluators and the evaluated is an unavoidable necessity. Quality in higher education is a description of the effectiveness of everything that is done to ensure that diligent students can derive maximum benefit from the educational opportunities available to them, and also fulfil the requirements for the award for which they are working. Standards are defined as general reference statements describing policies and practices that are intended to ensure that pre-determined and explicit levels of quality are achieved,in the design, delivery and evaluation of higher education courses and awards. A standard does not state ‘how’ an outcome must be achieved – it simply states what the outcome should be. The application and use of standards will be determined within the contexts of individual institutions and national systems; as these change with developments, so the application and use of the standards may need to be revised. Quality assurance is defined as all the actions and activities undertaken by institutions of higher education to ensure that the quality of their programmes and awards meets their own specifications and those of any other bodies legitimately empowered to make specifications. 19 Guidelines are defined as non-prescriptive explanations of the importance of the standards. Their intention is both to indicate why the standards have been included and to promote a wider and fuller awareness of the basic principles of good quality and quality assurance. Peer Review is defined as a process which involves the evaluation of teaching, learning,research and quality assurance, and their effectiveness, by persons external to the body being reviewed who have expertise and experience to command the confidence and respect of those whose work is being reviewed. This criterion might be met, for example, by academic, professional and administrative staff of higher education institutions, students, and appropriate representatives from industry and commerce, professional organisations, or public sector bodies.

Standards and guidelines for quality assurance

Basic principles • providers of higher education have the primary responsibility for the quality of their provision and its assurance; • the interests of society in the quality and standards of higher education need to be safeguarded; • the quality of academic programmes need to be developed and improved for students and other beneficiaries of higher education across the EHEA; • there need to be efficient and effective organisational structures within which those academic programmes can be provided and supported; • transparency and the use of external expertise in quality assurance processes are important; • there should be encouragement of a culture of quality within higher education institutions; • processes should be developed through which higher education institutions can demonstrate their accountability, including accountability for the investment of public and private money; • quality assurance for accountability purposes is fully compatible with quality assurance for enhancement purposes; • institutions should be able to demonstrate their quality at home and internationally; • processes used should not stifle diversity and innovation.

European standards and guidelines for internal quality assurance within higher education institutions Policy and procedures for quality assurance: Institutions should have a policy and associated procedures for the assurance of the quality and standards of their programmes and awards. They should also commit themselves explicitly to the development of a culture which recognises the importance of quality, and quality assurance, in their work. To achieve this, institutions should develop and implement a strategy for the continuous enhancement of quality. The strategy, policy and procedures should have a formal status and be publicly available. They should also include a role for students and other stakeholders. Approval, monitoring and periodic review of programmes and awards: Institutions should have formal mechanisms for the approval, periodic review and monitoring of their programmes and awards. Assessment of students: Students should be assessed using published criteria, regulations and procedures which are applied consistently. Quality assurance of teaching staff: Institutions should have ways of satisfying themselves that staff involved with the teaching of students are qualified and competent to do so. They should be available to those undertaking external reviews, and commented upon in reports. Learning resources and student support: Institutions should ensure that the resources available for the support of student learning are adequate and appropriate for each programme offered. Information systems: Institutions should ensure that they collect, analyse and use relevant information for the effective management of their programmes of study and other activities. Public information: Institutions should regularly publish up to date, impartial and objective information, both quantitative and qualitative, about the programmes and awards they are offering.

Institutional quality assurance What? Study programmes Entire institution What are the standards ? • student progression and success rates; • employability of graduates; • students’ satisfaction with their programmes; • effectiveness of teachers; • profile of the student population; • learning resources available and their costs; • the institution’s own key performance indicators. The quality assurance of programmes and awards are expected to include: • development and publication of learning outcomes; • curriculum and programme design and content; • specific needs of different modes of delivery (e.g. full time, part-time, distance-learning, e-learning) and types of higher education (e.g. academic, vocational, professional); • availability of appropriate learning resources; • formal programme approval procedures by a body other than that teaching the programme; • monitoring of the progress and achievements of students; • regular periodic reviews of programmes (including external panel members); • regular feedback from employers, labour market representatives and other relevant organisations; • participation of students in quality assurance activities. The policy statement is expected to include: • the relationship between teaching and research in the institution; • the institution’s strategy for quality and standards; • the organisation of the quality assurance system; • the responsibilities of departments, schools, faculties and other organisational units and individuals for the assurance of quality; • the involvement of students in quality assurance; • the ways in which the policy is implemented, monitored and revised. Policy and procedures for quality assurance Quality assurance of teaching staff Formal mechanisms for the approval, periodic review and monitoring of their programmes and awards. Learning resources and student support Information systems Public information Assessment of students

European standards for the external quality assurance of higher education Use of internal quality assurance procedures: External quality assurance procedures should take into account the effectiveness of the internal quality assurance processes described in Part 1 of the European Standards and Guidelines. Development of external quality assurance processes: The aims and objectives of quality assurance processes should be determined before the processes themselves are developed, by all those responsible (including higher education institutions) and should be published with a description of the procedures to be used. Criteria for decisions: Any formal decisions made as a result of an external quality assurance activity should be based on explicit published criteria that are applied consistently. Processes fit for purpose: All external quality assurance processes should be designed specifically to ensure their fitness to achieve the aims and objectives set for them. Reporting: Reports should be published and should be written in a style, which is clear and readily accessible to its intended readership. Any decisions, commendations or recommendations contained in reports should be easy for a reader to find. Follow-up procedures: Quality assurance processes which contain recommendations for action or which require a subsequent action plan, should have a predetermined follow-up procedure which is implemented consistently. Periodic reviews: External quality assurance of institutions and/or programmes should be undertaken on a cyclical basis. The length of the cycle and the review procedures to be used should be clearly defined and published in advance. System-wide analyses: Quality assurance agencies should produce from time to time summary reports describing and analysing the general findings of their reviews, evaluations, assessments etc.

European standards for external quality assurance agencies Use of external quality assurance procedures for higher education: The external quality assurance of agencies should take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines. Official status: Agencies should be formally recognised by competent public authorities in the European Higher Education Area as agencies with responsibilities for external quality assurance and should have an established legal basis. They should comply with any requirements of the legislative jurisdictions within which they operate. Activities: Agencies should undertake external quality assurance activities (at institutional or programme level) on a regular basis. Resources: Agencies should have adequate and proportional resources, both human and financial, to enable them to organise and run their external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of their processes and procedures. Mission statement: Agencies should have clear and explicit goals and objectives for their work, contained in a publicly available statement. Independence: Agencies should be independent to the extent both that they have autonomous responsibility for their operations and that the conclusions and recommendations made in their reports cannot be influenced by third parties such as higher education institutions, ministries or other stakeholders. External quality assurance criteria and processes used by the agencies: The processes, criteria and procedures used by agencies should be pre-defined and publicly available. These processes will normally be expected to include: • a self-assessment or equivalent procedure by the subject of the quality assurance process; • an external assessment by a group of experts, including, as appropriate, (a) student member(s), and site visits as decided by the agency; • publication of a report, including any decisions, recommendations or other formal outcomes; • a follow-up procedure to review actions taken by the subject of the quality assurance process in the light of any recommendations contained in the report. Accountability procedures: Agencies should have in place procedures for their own accountability.

Four-year cycle of quality assurance Internal quality assurance University Center + Faculty Units 1 Implementation of new curriculum 2 Institutional self-evaluation Curriculum revision 4 3 The National Council for Higher Education Acreditation External evaluation

Quality assurance of a study unit/module Study unit or module Each department! What is a quality? Is teaching well organized? Are student workloads well balanced and appropriate (balance of ECTS credits)? Is study programme coherent? Did we clearly identify knowledge, skills and competences? Did we clearly define content and learning outcomes? Are students well prepared for teaching? Is participation of students active enough? What is quality of communication between teachers and students? What is an impact of each individual teacher? How do we measure outcomes? Is student marking objective and balanced? What is quality of our examinations and how do we prove it? What is an improvement achieved during last academic year? Are students well informed about study unit/module. Questionnaires for students Questionnaires for teachers Self-evaluations Analysis of exams Web portal (e-learning tools) and share point communication Documentation Measures for improvement Ask students Ask teachers Masure oucomes

Quality assurance of a study programme/intitution Internal quality assurance Examinations Administration E-learning New technologies Departments Unsuccessful study etc. University Center + Faculty Units 1 Implementation of new curriculum 2 Institutional self-evaluation Curriculum revision 4 3 The National Council for Higher Education Acreditation External evaluation

Osiguranje kvalitete studijskog programa Set up quality indicators Student questionnaires Teacher questionnaires Teacher self evaluations Evaluation of exams Evaluation of programme flexibility Evaluation of resources Evaluation of the administrative support Documentation Information system Staff portfolio Enhancement measures Public availability Studijski program 1 2 3 4 Kako znamo da je kvalitetan? Izvodi li se nastava redovito Da li je radno opterećenja primjereno (balans ECTS kredita) Da li je program koherentan Da li su sadržaji učenja primjereni i suvremeni Jesmo li jasno definirali sadržaje Da li se studenti pripremaju za nastavu Da li studenti aktivno sudjeluju u nastavi Koja je razina radne komunikacije nastavnika i studenata Kako bismo ocijenili svakog pojedinog nastavnika Jesmo li definirali opće i specifične kompetencije Kako mjerimo ishod učenja Koliko objektivno provjeravamo napredovanje u učenju i provodimo ispite Pristupamo li organizirano sastavljanju ispitnih pitanja i pratimo li rezultate ispita Unaprjeđujemo li sadržaje i načine izvođenja programa svake godine Imaju li studenti sve potrebne informacije Pitajmo studente Pitajmo nastavnike

Backbone of the quality assurance system National quality assurance agency Faculty units for quality assurance University centre for quality assurance

University center for quality assurance - structure National Agency for Quality Assurance Administrative and organizational support (QA office or center) chair – academic person - part time appointed secretary project manager one project manager on 10.000 students University Quality Assurance Board or Council Teachers + students University center for quality assurance + Institutional units for quality assurance Pero Lucin, May 2004.

Faculty units for quality assurance - organization National Agency for Quality Assurance University center for quality assurance Academic Board (QA Team) teachers + students Administrative support (unit) + Institutional units for quality assurance Pero Lucin, May 2004.

Instituational structure of quality culture development at the University of Rijeka RECTOR/ SENAT BOARD FOR QUALITY PROMOTION / HEAD OF OFFICE TEAM FOR HUMEN RESOURCE DEVELOPMENT EVALUATION CENTER TEAM FOR CURRICULUM DEVELOPMENT (ECTS) TEAMS AT EACH FACULTY UNIT AT EACH FACULTY TEAM AT EACH FACULTY

Zahvaljujem na pažnji! First Croatian Bologna Seminar International Center for Education of Journalists – ICEJ Opatija, 4 – 5 March 2005 Zahvaljujem na pažnji!