1. WP4- Piloting and Evaluation
Activity 4.3:
High standards and quality assurance of educational process and teaching
Renate Nantschev, MSc
Meeting – 24. January, 2019: UMIT; Hall in Tirol; Austria
Project web site:

2. Task: 4.3 Objective
To share experience and describing processes in developing and validating quantitative and qualitative evaluation tools.
To develop an approach for new evaluation methods for PH-ELIM teaching and learning activities.

3. Working plan and time table
Three Workshops (as demanded by project description)
Aims
Results
Athens:
22nd-23rd Nov. 2018
To analyse all current evaluating approach of each project partner in their universities.
All project partner presented their specific evaluation concepts used in their universities.
Hall in Tirol:
22nd – 25th Jan. 2019
To find out a practicable solution for an evaluation process that can be useful for Montenegro.
To develop questionnaires for all types of courses and lectures.
Debrecen:
7th – 9th May 2019
To finalize the process for implementation

4. First results We agreed:
first, to develop only an evaluation concept for the new master course at UDG
we would like to prepare questionnaires to evaluate courses, the study environment and the instructors
all documents should be prepared in English – they will be translate later in Montenegrin language

5. First results We have … different quality assurance requirements
examples for evaluation processes
examples for questionnaires
examples how to manage the results of evaluation
examples for evaluation tools
What has the presentation shown.
In the next slides i will give you a summery to this topics of some instituion they presented in Athens

6. Evaluation Concept University of Heidelberg
Project web site:

7. Heidelberg: Quality assurance requirement
Evaluation Statutes of University of Applied Sciences Heilbronn (cooperation partner of medical informatics programs)
§1 Domain
§2 Objective of Evaluations
§3 Responsibilities
§4 Subjects and Deadlines
§5 Standardized Data Collection
§6 Additional Data Collection by the Faculty
§7 Additional Data Collection by Teachers
§8 Implemention of Uniform Data Recording
§9 Use of Evaluation Results within the University
§10 Use of Evaluation Results outside the Univ.
§11 Archiving and Anonymization of Data
§12 Commencement
The quality assurance requirement are regulated in the Statues von Heilbronn

8. Evaluation Medical Informatics Programmes
Heidelberg: Evaluation Process
Evaluation Medical Informatics Programmes
Evaluation statutes
Each course is evaluated every two years
Application system: EvaSys
TANs provided by teachers for students for each course to access questionaire
Evaluation in between lectures to give room for improvement
Individual feedback to teacher
Evaluation Periode
Management of the results

9. Heidelberg: Questionnaires / Evaluation tool
Questionnaires applies to:
the course
for general evaluation (working atmosphere… )
for evaluation of the University
about the workload
EvaSys evaluation software of higher education sector

10. Medical Faculty Heidelberg
Heidelberg: Management of the results
Medical Faculty Heidelberg
1. Quality Management Team
2. Report for each course
3. General report after each term
summary: What was helpful, what to improve
comparison to earlier courses or modules
detailed results: what was helpful
detailed results: what to improve
4. Teachers have to reply to the report

11. Evaluation Concept National and Kapodistrian Unviersity of Athens
Project web site:

12. Project web site: www.ph-elim.net
Athens: Quality assurance requirements
1. Quality Assurance Unit (“MODIP”)
Each institution is responsible for ensuring and continuously improving the quality of its educational and research work as well as for the efficient operation and performance of its services, in line with international practices, in particular those of the European Higher Education Area, and the principles and guidelines of HQA.
For this purpose, the Quality Assurance Unit is responsible in each HEI. In each Higher Education Institution, an internal quality assurance system is developed and implemented, with the responsibility and initiatives of the relevant “MODIP”, which must cover the whole range of functions and activities of the institution.
Each institution is under a modip . Mopid are the requirement by law? John – please can you explane it to you shortly
Thee quality assurance unit is responsible for ensuring and improving the quality in higher education and required higher institutions to develop and implement an internal quality assurance system. Modip balance international practices and guidelines of the quality assurance and accreditationa gency
Project web site:

13. Project web site: www.ph-elim.net
Athens: Evaluation process
“MODIP” - University of Athens
The evaluation criteria and indicators used and monitored are reported on the basis of the above institutional framework in the following four axes:
the quality of the teaching process;
the quality of the research process;
the quality of the curricula;
the quality of other services (administrative services, student care, ICT infrastructure, transparency in the management of financial resources, etc.).
The criteria and indicators used are required to address the following areas 1-4
Project web site:

14. Project web site: www.ph-elim.net
Athens: Questionnaires
e.g. Internal Evaluation of MSc Programmes in the University of Athens, Department of Nursing Evaluation Method and Tools
Evaluation Committee of the Department of nursing consisting of professors and the heads of MSc programmes are responsible for constructing the questionnaires.
Two Questionnaires are distributed to students regarding the:
Evaluation of each course
Evaluation of the MSc programme
Students are requested to answer anonymously the questionnaires.
Project web site:

15. Athens: Management of the results
Actively involving the faculty member in the self-awareness and self-improvement process; using this information as an opportunity to interact with the students and think about alternative teaching methods or material alteration.
Regular meetings among the faculty members are held to discuss about individual results and results about the MSc programme. Allowing sufficient time for dialogue and interaction (between the consultant and faculty members) using feedback information and examining and understanding the faculty member’s approach to teaching (e.g. philosophy and paedagogical strategies); and the setting of improvement goals for the faculty member. Faculty members review the institutional materials taking into consideration the results from the surveys, the institutional expectations and University and Ministry of Education guidelines.
Analysing the results and giving this ffedback to the instructor can involve him in a self-awareness and self improvement process. The instructor take this opportunity to revise his teaching approach and material used. Adiscusiion amog faculty members follows in order to discuss the results

16. Evaluation Concept Universtiy of Mediterranian
Project web site:

17. Law of Higher Education in Montenegro- Self-evaluation and Quality Assessment
An institution shall conduct the procedure of quality self-evaluation concerning study programmes, teaching and working conditions at the end of each academic year, in compliance with statute of the institution.
In the procedure of self-evaluation student surveys should be carried out at all levels, at least once a year with regard to study programmes, teaching, conditions and work of academic staff.
Students are obliged to participate in survey.
Survey is anonymous.
The results of student surveys should be published on the website of institution.
Self-evaluation reports of an institution shall be passed by the governing body of the institution.
An institution shall submit a self-evaluation report to the Ministry and Agency.
Method and criteria of self-evaluation concerning study programmes, teaching equipment, qualifications of academic staff, teaching methods, enrolment of students, exam pass rate, percentage of graduate students and the content of survey are more closely laid down by act of the Agency.

18. Mediterranean: Evaluation Process
University Mediterranean conducts the procedure of quality self-evaluation concerning study programmes, teaching and working conditions twice a year- at the end of winter and summer semester.
Students surveys are carried out at all levels of study- undergraduate, post- graduate and PhD level.
Students are obliged to participate in survey.
Survey is anonymous.
The results of student surveys are published on the website of institution.

19. Mediterranean: Evaluation Process
Method and criteria of self-evaluation concerning study programmes, teaching equipment, qualifications of academic staff, teaching methods, enrolment of students, exam pass rate, percentage of graduate students and the content of survey are more closely laid down by Senate of the University.
Surveys are carried out on-line.
Vice deans and representatives of Students parliament are responsible for the process of carrying out surveys.
The results of the survey are analysed by Faculty’s Council and then by Senate of the University.
In case of unsatisfactory level of students satisfaction regarding some elements of teaching/learning process, measures for their improvement are defined by members of Faculty’s Council and by Senate of the University

20. Evaluation Concept Universtiy of Montenegro
Project web site:

21. University of Montenegro (UCG): Evaluation Process
Quality control at the University of Montenegro, bodies and documents
Centre for study and quality control
Preparation of proposals of strategic plans and programs in the field of quality, as well as proposals for improvement of standards, procedures and methods of quality assurance, helps in preparation of documentation for self-evaluation and accreditation, performs all activities related to quality.
Rulebook on the organization and operation of the system for quality assurance and improvement of UCG
European Standards and Guidelines for Quality Assurance in Higher Education
11th Project Coordination meeting Athens November 2018
Project web site: .

22. Project web site: www.ph-elim.net
University of Montenegro (UCG): Evaluation Process
Students Questionnaires
In April each academic year
All students are required
Runs and processes by the UMN Information System Center
Project web site:

23. Project web site: www.ph-elim.net
University of Montenegro (UCG): Management of results
Analizes of survey and actions
If the survey for a subjects is rated over 3 (very good / excellent), they are considered to be of high quality teaching.
If the survey are rated lower than 3 (bad /very bad), the teacher submits a report to the faculty administration with a proposal that will improve teaching proces.
Project web site:

24. Evaluation Concept Universtiy of Dona Gorica
Project web site:

25. Project web site: www.ph-elim.net
University of Dona Gorica (UDG)
Quality assurance requirements:
Implementation of QA policy and procedures in Montenegro is being promoted by responsible bodies (Ministry of Education, Council for Higher Education, just-established Agency for Quality Assurance in Higher Education) and higher education institutions are encouraged to implement Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)
When it comes to evaluation and assessment of study programs, the process is quite different at different institutions.
At UDC developed a framework for quality assurance system
The quality assurance requirements are the same as in the other institution of higher education in Montenego
Project web site:

26. Quality assurance framework at UDG
I think that is a general framwork for quality assurance: I am not sure if there are the requirement for the evaluation of courses, teacher and so on
November 2018, Athens

27. University of Dona Gorica (UDG)
What questionnaires you have?
How is the evaluation process organized?
How is the management of results organized?
Open questions ?

28. Evaluation Concept UMIT
Project web site:

29. UMIT: Evaluation Process
Contents of evaluation concept:
Process of the course evaluation
Discussion of the evaluation results within study commission
Activities to further develop the quality of teaching
Feedback to the students of the evaluation results
Reporting from the study commission to the senate
Content of the concept are described in five chapters
Stu

30. UMIT: Evaluation Process
The evaluation will be conducted in all courses in each semester
Different questionnaires are used: Lectures; Lectures with exercises; Seminar;
Evaluation will be done before the exam
All Evaluations are electronic based – Moodle or Zensus

31. UMIT: Evaluation Process
Information to the students are provided by the student management
Communication to the instructors about results is provided by the study management
Instructors can see the results after the end of evaluation time
The results are communicated to the students at the end of each semester

32. UMIT: Questionnaires Questionnaires about the course: e.g.
- The overall organization of the course is good (1-5)
- My overall learning gain in this course is high (1-5)
2. Questionnaires about the teacher: e.g.
- Teacher can explain complex contents
- Teacher gives me opportunities to review my learning progress

33. Thank you for your attention 

34. Summary of the first results
Report systems/ Management of the results
general report after each term
Results are available on the website (Debrecen)
Informal meetings: Student representatives may have meetings with subject leaders, students give feedback on quality of education (when they are unhappy)(Debrecen)
The results of student surveys should be published on the website of institution (Mediterranen)
An institution shall submit a self-evaluation report to the Ministry and Agency. (Mediterranen)
If the survey are rated lower than 3 (bad /very bad), the teacher submits a report to the faculty administration with a proposal that will improve teaching process (UCG)

35. Summary of the first results
Questionnaires:
to the course (for all courses the same)
to general aspects of the course (working atmosphere, seminar room, ..
to the personal learning process
questions addressing the teacher performance, educational materials, exams…

36. Summary of the first results
Evaluation Tools:
Online based: EvaSys and Zensus (Heidelberg and Umit)
Paper based
Via educational offices, departments, student organizations Online (evasys) and paper format (Debrecen)
Surveys are carried out on-line (Mediterranen)
Runs and processes by the UMN Information System Canter (UCG)

37. Summary of the first results
Responsibility:
Quality Management Team
Study advisory commission
Evaluation Committee of the Department of nursing consisting of professors and the heads of MSc programmes are responsible for constructing the questionnaires. (Athens)
The results of the survey are analysed by Faculty’s Council and then by Senate of the University. (Mediterranen)