Nottingham – 6 April 2006 PhD thesis title: The Evaluation of Teaching and Learning at Higher Education in Italy, France and England. Antonella Poce PhD Student at the University Roma Tre Innovation and Evaluation of Educational Systems

Nottingham – 6 April 2006 State of the art: assessment methods in use (especially in Italy) are not suitable to improve teaching quality. Hypothesis: if we evaluate innovative teaching methods the results of this procedure can be useful to improve quality. It is worth then looking for and applying new teaching practices.

Examples of good practices Study of cases: the University Roma Tre, the University of Marne la Vallée and the University of Nottingham The evaluation of teaching/learning activity at Higher Education in Italy, France and England The plan of the work Nottingham – 6 April 2006

The Prin Project at Roma Tre. The course Licence Professionnelle- Management des organisations de l’economie sociale at Marne-la-Vallée. The VLL and the TLO application at Nottingham University. Nottingham – 6 April 2006

The IRLTHE: Institute for Research into Teaching and Learning The VLL: Visual Learning Lab The TLO: Teaching and Learning Observatories Nottingham – 6 April 2006

Observation Interviews Documents The effectiveness of TLO: Teaching and Learning Observatories Nottingham – 6 April 2006

Documents The effectiveness of TLO: Teaching and Learning Observatories The use of TLO: focussed observations for trainees, encouragement of dialogue between trainees, participation of all the actors involved to the process of teacher training Nottingham – 6 April 2006

Interviews The effectiveness of TLO: Teaching and Learning Observatories 15 students from the PGCE course 2005/2006 -Everyone shows appreciation with the initiative, -Some highlights that it is a form of support to their education, -Complains are related to technical problems. Nottingham – 6 April 2006

Observation The effectiveness of TLO: Teaching and Learning Observatories Strenghths: possibility of focusing the observation both as regards contents, and teacher and pupils attitude; interaction between trainees and teacher in class. Weaknesses: presence of technicians in the class observed, technical problems with zooms and cameras Nottingham – 6 April 2006

The OFIPE Observatoire des formations, des insertions professionnelles, evaluations La licence professionnelle Management des organisations de l’economie sociale How do students consider and use the web site of the course? An ad hoc model of evaluation for the course: 34 new indicators Nottingham – 6 April 2006

The Prin Project at the University Roma Tre: Experimental Evaluation of e-Learning Effect on the Quality of University Teaching The hypothesis of the experiment was based on the consideration that integrating face-to-face learning with e-learning improves the quality of teaching. Nottingham – 6 April 2006

The students of the module Experimental Pedagogy and Measurement – Course of Primary Education - Faculty of Education could choose among the following possibilities: attending face-to face learning and online learning; attending only online learning; attending only face-to-face learning; none of the above. The Prin Project at the University Roma Tre: Experimental Evaluation of e-Learning Effect on the Quality of University Teaching Nottingham – 6 April 2006

The experimental group was composed of the students who chose the first of the mentioned options. The results of the final exams, taken by these students at the end of the module, in May, June and July 2005, have been compared with the ones of the other groups. Moreover these data have been compared with diachronic series of the results of the final exams, from the academic year 2000/2001. Experimental Evaluation of e-Learning Effect on the Quality of University Teaching Nottingham – 6 April 2006

Experimental Evaluation of e-Learning Effect on the Quality of University Teaching Two different online laboratories (modules) have been created ad hoc: Introduction to Experimental Research – first term; Introduction to Educational Measurement Research – second term Nottingham – 6 April 2006

The project To carry out the project it has been necessary to develop a series of activities such as the: identification of the background characteristics of the students attending the module of experimental pedagogy and measurement; identification of their pre-requisites; creation and development of the online modules; analysis of the exams results for the academic year 2004/2005; analysis of the exam results since the a.y. 2000/2001; report of the results. Nottingham – 6 April 2006 Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings

Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings Percentage values of final exams scores related to the different groups considered MarksFace-to-faceBlendedOnlineNot attending L22802 Failure rates Totals100 Nottingham – 6 April 2006

Diachronic series a.y. 2000/2001 a.y. 2001/2002 a.y. 2002/2003 a.y. 2003/2004 a.y. 2004/2005 Pass Present at the exam Fails Percentage of failure 36,447,531,321,319,1 Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings Nottingham – 6 April 2006

Diachronic series and central tendency values a.y. 2000/2001 a.y. 2001/2002 a.y. 2002/2003 a.y. 2003/2004 a.y. 2004/2005 Mean Median Mode SD3,0972,7883,5933,5763,843 Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings Nottingham – 6 April 2006

Conclusions Lower rate of failures as regards the experimental group; Higher mean rate of scores if compared with previous years Higher frequency of best marks Good participation rate and satisfaction of the students with the initiative support the implementation of similar teaching offers. Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings Nottingham – 6 April 2006

Thank you! Antonella Poce Nottingham – 6 April 2006