Alicante 24-26/4/2013 Organization for QA 1 2 nd DoQuP Training Seminar University of Alicante, April 2013 Session on Organization for the Quality Assurance of Institutions and Study Programmes in Italian Universities Alfredo Squarzoni University of Genoa

Alicante 24-26/4/2013 Organization for QA 2 Items of the presentation a)Internal Quality Assurance of Study Programmes (in Italy) b)Internal QA of Institutions (in Italy) c)Responsibilities for the internal QA of SPs and Institutions (in Italian Universities)

Alicante 24-26/4/2013 Organization for QA 3 Internal Quality Assurance of Study Programmes For “quality assurance of a study programme”it is intended “the whole of the activities (processes) for the management of the educational service aimed at achieving the established educational objectives and then at ‘ensuring trust’ in meeting the quality requirements to all interested parties”.

Alicante 24-26/4/2013 Organization for QA 4 The Guidelines for the definition of QA systems of SPs are suggested in the document “ Autovalutazione, Valutazione e Accreditamento del Sistema Universitario Italiano (AVA)” ( miur/files/ava_documentofinale_0.pdf) (“ Self-Assessment, Assessment and Accreditation of the Italian University System ”), approved by the Italian National Agency of Evaluation of the University System and Research (ANVUR, on 24 July 2012.

Alicante 24-26/4/2013 Organization for QA 5 In that document, ANVUR states that QA of education requires:  the definition of the objectives to be achieved;  the implementation of actions that allow their achievement;  the use of credible ways of assessment of their effective achievement.

Alicante 24-26/4/2013 Organization for QA 6 Furthermore, according to ANVUR, other requirements of a system of QA of education are:  the re-examination of the educational service offered, to be carried out annually and which has to include the drafting of a final report;

Alicante 24-26/4/2013 Organization for QA 7  the compilation of the “ Scheda Unica Annuale (SUA-CdS)” (“ Annual Unique Form ”) of the SPs, where to collect information about the SPs, to be made public and accessible on the net so as to satisfy the conditions considered necessary for proper communication of the characteristics of the SPs offered to students and all stakeholders.

Alicante 24-26/4/2013 Organization for QA 8 The guidelines for the definition of a QA system of SPs are not explicitly defined in the AVA document, but they can be identified analysing the required information and data to be collected in the Annal Unique Form. What suggests the AVA document as for the definition of a QA system of SPs? Basically, five things, consistently with the ESG and the models for the quality assessment and accreditation of SPs of the European agencies.

Alicante 24-26/4/2013 Organization for QA 9 a) First of all, the SP should establish the expected learning or programme outcomes of the educational service offered, in line with the needs and expectations of society and labor market. In particular, the learning outcomes must ensure the personal development of students and be aligned with the skills necessary to perform the roles for graduates in the labor market.

Alicante 24-26/4/2013 Organization for QA 10 b) Then it needs to design and plan an educational path (curriculum and characteristics of course units and other educational activities) able to achieve the expected learning outcomes established and to define methods and criteria of assessment of the student learning able to assess the grade of achievement of the learning outcomes in a credible way.

Alicante 24-26/4/2013 Organization for QA 11 At the same time, it needs to have at disposal resources of teaching staff (lecturers and teaching support staff), facilities (classrooms, laboratories, libraries) and student services (orienteering, tutoring and assistance) adequate, quantitatively and qualitatively, to facilitate the achievement of the expected learning outcomes and to implement the established teaching methods.

Alicante 24-26/4/2013 Organization for QA 12 c) Next it needs to monitor the results of the educational process, in order to check the degree of achievement of the established objectives and, therefore, the quality of the educational service offered.

Alicante 24-26/4/2013 Organization for QA 13 Always with reference to SPs, the results of the educational process that all systems of internal QA take under control include:  the attractiveness of the SP and, therefore, the number of students who enroll in the programme, their provenance, geographical and scholastic, and, not least, their quality, as measured through the assessment of the mastery of the admission requirements;

Alicante 24-26/4/2013 Organization for QA 14  the results relating to the students’ career, in particular: –dropouts, –number of credits earned by students passing from one course year to the next one, –time to graduation and final vote;  the effectiveness of course units and other educational activities and of the learning process as a whole perceived by students and graduates respectively;

Alicante 24-26/4/2013 Organization for QA 15  the placement of graduates in the labor market, in particular: –time to placement, –consistency between granted qualification and working activity, –effectiveness of the degree in the working activity, –opinion of the employed graduates on the adequacy of the education received.

Alicante 24-26/4/2013 Organization for QA 16 Another important monitoring regards the assessment tests of the students’ learning, in order to ascertain their suitability to assess the level of achievement of the established learning, and their results, through, for example, the survey of the average value of the grades attributed to all the students who have passed the examination for all the course units that provide for a final vote.

Alicante 24-26/4/2013 Organization for QA 17 d) Then it needs to promote the improvement if not continuous at least periodic of the educational service, through a re-examination process. The re-examination is the process, periodic and planned, through which the SP evaluates the suitability, adequacy and effectiveness of the educational service offered and takes all the appropriate actions for improvement, which can regard both the management of individual processes for the QA and the revision or redefinition of the management system of the SP.

Alicante 24-26/4/2013 Organization for QA 18 The AVA document requires that the re-examination starts with a self-assessment finalised to the identification of the strong and weak points of the SP, through at least the comparison of the SP’s results with those obtained in the preceding years and the results obtained by other SPs of the same typology, if any, and to the identification of the causes of the weak points (e.g: causes of the dropouts, motivations of delays in graduation, etc.).

Alicante 24-26/4/2013 Organization for QA 19 The self-assessment may bring to the identification of needs of revision or redefinition of the educational objectives and process and of the internal QA system and of opportunities of improvement of the management and/or of the results of single processes. Then, for each identified need of revision and for each opportunity of improvement, the SP should identify and adopt opportune improvement actions. The results of the re-examination should be reported in a Re-examination Report.

Alicante 24-26/4/2013 Organization for QA 20 e) Finally, it needs to document the objectives established, the educational activities carried out, the resources available, the results obtained in order to provide public evidence of the overall quality of the educational service to all stakeholders.

Alicante 24-26/4/2013 Organization for QA 21 Internal Quality Assurance of Institutions In general, in countries where it is practiced, the “ quality assurance of an institution ” is intended as the whole of the activities (processes) aimed at assuring the quality of the SPs offered by the same institution.

Alicante 24-26/4/2013 Organization for QA 22 The Italian Agency for the Evaluation of University System and Research (ANVUR), which practices the institutional accreditation of universities, has defined the following standards for the QA of institutions, borrowed from those developed by the Dutch national Agency NVAO:

Alicante 24-26/4/2013 Organization for QA 23 the institution should establish, state and actually pursue appropriate policies to achieve its vision of the quality of education; the institution knows the extent to which its policies are actually carried out by the SPs; the institution requires that SPs practice quality improvement, aiming at results of increasing value; the institution has an effective organization with powers of decision-making and monitoring of the quality of the SPs and of the education offered to the students.

Alicante 24-26/4/2013 Organization for QA 24 Responsibilities for internal QA The AVA document states that the main responsibility for the QA of a SP or of an institution is with the same SP or institution, but the document identifies three other important bodies for the QA of SPs and institutions. They are:

Alicante 24-26/4/2013 Organization for QA 25  the “ Presidio della Qualità ” ( Commission for the QA, CQA ), at University level;  the “ Commissione Paritetica Docenti – Studenti ”( Commission Lecturers-Students with the same number of lecturers and students, CLS), at Department level (Departments are the structures which study programmes belong to)  the “Nucleo di Valutazione” ( Evaluation Committee, EC), at University level.

Alicante 24-26/4/2013 Organization for QA 26 Commission for the QA (CQA) (Presidio della Qualità) The suggested composition is of 6-10 persons, in part from the academic staff, experts in QA processes, and in part from the administrative staff, involved in the management of information and data on students, experts of informative systems, etc..

Alicante 24-26/4/2013 Organization for QA 27 According to the AVA document, main tasks of the CQA are:  to propose the system of internal QA for both the institution and the study programmes and to organize educational activities in order of their application;  the supervision of the proper and uniform carrying out of the QA procedures throughout the University;  to support study programmes in the common activities for the QA.

Alicante 24-26/4/2013 Organization for QA 28 In particular, according to the indications of the AVA document, the CQA should be in charge of the following functions : Organization and control of the updating of the information contained in the Annual Unique Form of each SP of the University; Organization and control of the carrying out of the QA procedures for the didactic activities; Organization and control of the re-examination activities by the SPs; Organization and check of information transmission to and from the EC and the CLS Evaluation of the effectiveness of the improvements and of their actual consequences.

Alicante 24-26/4/2013 Organization for QA 29 Commission Lecturers-Students (CLS) ( Commissione Paritetica Docenti-Studenti) According to the AVA document, main tasks of the CLS are: a.the design of the SP maintains the due attention to the functions and competences required by both the job perspectives and the perspectives of personal and professional development, taking into account the needs of economic and production system;

Alicante 24-26/4/2013 Organization for QA 30 b.the expected learning outcomes are effective in relation to the tasks and responsibilities of reference; c.the qualification of lecturers, methods of transmission of the knowledge and skills, materials and teaching aids, laboratories, classrooms, equipment are effective in meeting the learning outcomes to the desired level; d.examination methods make possible to determine correctly the results obtained in relation to the expected learning outcomes;

Alicante 24-26/4/2013 Organization for QA 31 e.the annual re-examination is followed by effective corrective measures on the SPs in subsequent years; f.questionnaires for the gathering of the students’ opinion on the educationalprocess are properly managed, analyzed and used; g.academic institution actually makes available to the public, at least through a regular and accessible publication public parts of the Annual Unique Form, information impartial, objective, quantitative and qualitative, on each of SP offered.

Alicante 24-26/4/2013 Organization for QA 32 The CLS expresses its evaluations and make proposals for the improvement in an annual report to be sent to the CQA and to the EC by 31 December of each year.

Alicante 24-26/4/2013 Organization for QA 33 Evaluation Committee (EC) (Nucleo di Valutazione) In general the composition of the EC is of 5-9 persons experts of assessment processes, from both the academic world and the world of production, services and professions, mainly not belonging to the University to be evaluated.

Alicante 24-26/4/2013 Organization for QA 34 According to the AVA document, the EC should carry out an activity of surveillance and leading, with particular reference to the following tasks : a.to evaluate the overall effectiveness of the management in QA of teaching and research; b.to control whether the organization and the documented activities of the CQA are structured to effectively implement the QA in the single internal structures (SPs, Departments, …) and in the university as a whole;

Alicante 24-26/4/2013 Organization for QA 35 c.to control whether the organization of the University and of its internal structures (SPs, Departments, …), through concrete actions appropriately documented, demonstrates that the activities programmed by SPs and Departments are effectively kept under control in a systematic and documented way, i.e. making all the necessary actions to achieve the objectives and to check the level of achievement; d.to verify the persistence of the quantitative and qualitative requirements for the initial and periodic accreditation of SPs and institution;

Alicante 24-26/4/2013 Organization for QA 36 e.to control whether the governing bodies of the SPs and of the University take into account of the activities of the CQA and of the evaluations and proposals made by the CLS in the annual report; f.to verify whether the re-examination reports of the educational activities are drawn up correctly and used to identify and remove all obstacles to the good performance of the educational activities; g.to make recommendations for improving the quality of teaching and research of the University.

Alicante 24-26/4/2013 Organization for QA 37 The EC expresses its evaluations and make proposals for the improvement in an annual report to be sent to the governancr bodies of the University and ANVUR by 30 April of each year.

Alicante 24-26/4/2013 Organization for QA 38 Thanks for your attention