An Evaluation Process of the University Engineering Degrees: the Portuguese Case Carlos Sá Furtado ICEMS Coimbra – Departamento de Engenharia Electrotécnica e de Computadores, LTMEU, Universidade de Coimbra, Pólo 2, Coimbra, Portugal, ICEE 2005 July 25-29, 2005, Gliwice, Poland

Introduction Portuguese university engineering courses called licenciaturas Reasons for the Evaluation/Assessment The Published Legislation Principles and Objectives of the Quality Assessment The Process of Evaluation Criticisms at the Evaluation Process Some General Conclusions and Recommendations

ICEE 2005 July 25-29, 2005, Gliwice, Poland Reasons for the Evaluation/ Assessment 1290 University of Coimbra 1911 Universities of Lisbon and Oporto 1930 Technical University of Lisbon 1970’s eleven public universities, more than twenty polytechnic institutes and dozens of private Higher Education schools The Revolution of the 25th April 1974: Centralised and autocratic regime to a democratic and rooted participated system; Traditional authoritarian rules at the University by a modern representative democracy. Number of students increased exponentially Middle and working classes eroded the exclusive character of university education This explosion not duly prepared, organised, and accompanied

ICEE 2005 July 25-29, 2005, Gliwice, Poland 1980s: adequate legislation has been passed 1st Cycle of Evaluation Public Universitary Licenciaturas Assessment started in Foundation of the Portuguese Universities ( Fundação das Universidades Portuguesas)– FUP 2nd Cycle Evaluation : National Council for the Evaluation of Higher Education ( Conselho Nacional de Avaliação do Ensino Superior- CNAVES).Public and private universities and public and private polytechnics.

ICEE 2005 July 25-29, 2005, Gliwice, Poland The Published Legislation – Law of the Bases of the Educational System, Law nr. 46/86, of 14th October – Law of the Autonomy of the Universities, Law nr. 108/88, of 24th September – Law of Higher Education Evaluation, Law nr. 38/94, of 21st November -- Protocol signed on the 19th June 1995 between the Ministry of Education, CRUP, and FUP – Decree-Law nr. 204/98, of 11th July, regulates Law nr.38/94, and creates CNAVES

ICEE 2005 July 25-29, 2005, Gliwice, Poland Principles and Objectives of the Quality Assessment Main principles of Quality Assessment: The Higher Education Quality Assessment System is unitary Independent and impartial character of the Evaluating Agency Mechanisms and procedures lead to impartial and independent judgements and conclusions Regular periodicity Permanent information through selected and appropriate indicators Encourage the Self- Evaluation

ICEE 2005 July 25-29, 2005, Gliwice, Poland System set on the following principles: The institutions own the evaluation system Based on self-assessment and peer review Cyclical Nationwide Teaching and research are evaluated separately Reports of the expert commissions are made public Faculties/Departments nothing to say about external evaluation committees No link with funding. Objectives of the Quality Assessment System: Promote and improve quality of teaching and research Stimulate self-regulation based on quality Inform Portuguese society on the performance of each Institution

ICEE 2005 July 25-29, 2005, Gliwice, Poland The Process of Evaluation Process established by CNAVES with the agreement of all the Institutions Internal and external stage Self- evaluation of strengths, weaknesses and future prospects External stage peer review includes a site visit and an external report Contradictory hearing of the assessed institution CNAVES announces evaluation in the following year Faculties prepare Self-Evaluation Report ( Relatório de Auto-Avaliação ) Report distributed among the members of the EEC A 2-3 days site visit; Visiting Committee- 3-5 members Preliminary comments at end of visit; afterwards, an interim report EEC approves final report Contradictory arguing

ICEE 2005 July 25-29, 2005, Gliwice, Poland The self-evaluation report contains the following chapters: a) Genesis and evolution of the programme; b) Structure and content of the programme; c) Curriculum organisation; d) Learning and teaching environment; e) Intake; f) Success rates; g) Quality of the graduates; h) Effectiveness of organisation and staff qualities; i) The facilities; j) Internationalisation and external contacts; and k) Internal quality assessment. Guidebook, issued by CNAVES, helps the Visiting Committee: terms of reference and external report Basically, the Committee has: Form an opinion based on supplied information and discussions Make suggestions on quality improvement

ICEE 2005 July 25-29, 2005, Gliwice, Poland Criticisms at the Evaluation Process Evaluation reports made more visible and known Some parameters clarified with assignment of minimum values Lack of indicators and parameters Need for an effective articulation between evaluation and inspection Need to clarify the relations between the evaluation conclusions and the Portuguese Association of Engineers (Ordem dos Engenheiros) Rationalization and increase of financial resources/ foreign experts Consideration of market/ Respect to the independence and creativity of the Institutions

ICEE 2005 July 25-29, 2005, Gliwice, Poland Some General Conclusions and Recommendations All the University Courses of Engineering evaluated General conclusions: Large majority of these courses got a good appreciation Lack of experience on themes related to the enterprises Insufficient attention to ethical and deontological areas and integrated evaluation of students´ competence Great retention of students in two initial years Lecturers find difficulties to deal with great number of badly prepared and little motivated students Professors besides teaching and research must have professional practice Too long time to Final Work of the Course

ICEE 2005 July 25-29, 2005, Gliwice, Poland Some general recommendations have been identified: Government must be aware of the increasing financial constraints Improvement of the preparation obtained at the Secondary School Promotion of a dynamic interaction between University and Enterprises More lecturers with involvement in engineering projects Improvement by the Institutions of a Self-Quality System

ICEE 2005 July 25-29, 2005, Gliwice, Poland Conclusions Main objectives have on the whole been achieved Universities have improved