1. Nottingham – 6 April 2006 PhD thesis title: The Evaluation of Teaching and Learning at Higher Education in Italy, France and England. Antonella Poce PhD Student at the University Roma Tre Innovation and Evaluation of Educational Systems

2. Nottingham – 6 April 2006 State of the art: assessment methods in use (especially in Italy) are not suitable to improve teaching quality. Hypothesis: if we evaluate innovative teaching methods the results of this procedure can be useful to improve quality. It is worth then looking for and applying new teaching practices.

3. Examples of good practices Study of cases: the University Roma Tre, the University of Marne la Vallée and the University of Nottingham The evaluation of teaching/learning activity at Higher Education in Italy, France and England The plan of the work Nottingham – 6 April 2006

4. The Prin Project at Roma Tre. The course Licence Professionnelle- Management des organisations de l’economie sociale at Marne-la-Vallée. The VLL and the TLO application at Nottingham University. Nottingham – 6 April 2006

5. The IRLTHE: Institute for Research into Teaching and Learning The VLL: Visual Learning Lab The TLO: Teaching and Learning Observatories Nottingham – 6 April 2006

6. Observation Interviews Documents The effectiveness of TLO: Teaching and Learning Observatories Nottingham – 6 April 2006

7. Documents The effectiveness of TLO: Teaching and Learning Observatories The use of TLO: focussed observations for trainees, encouragement of dialogue between trainees, participation of all the actors involved to the process of teacher training Nottingham – 6 April 2006

8. Interviews The effectiveness of TLO: Teaching and Learning Observatories 15 students from the PGCE course 2005/2006 -Everyone shows appreciation with the initiative, -Some highlights that it is a form of support to their education, -Complains are related to technical problems. Nottingham – 6 April 2006

9. Observation The effectiveness of TLO: Teaching and Learning Observatories Strenghths: possibility of focusing the observation both as regards contents, and teacher and pupils attitude; interaction between trainees and teacher in class. Weaknesses: presence of technicians in the class observed, technical problems with zooms and cameras Nottingham – 6 April 2006

10. The OFIPE Observatoire des formations, des insertions professionnelles, evaluations La licence professionnelle Management des organisations de l’economie sociale How do students consider and use the web site of the course? An ad hoc model of evaluation for the course: 34 new indicators Nottingham – 6 April 2006

11. The Prin Project at the University Roma Tre: Experimental Evaluation of e-Learning Effect on the Quality of University Teaching The hypothesis of the experiment was based on the consideration that integrating face-to-face learning with e-learning improves the quality of teaching. Nottingham – 6 April 2006

12. The students of the module Experimental Pedagogy and Measurement – Course of Primary Education - Faculty of Education could choose among the following possibilities: attending face-to face learning and online learning; attending only online learning; attending only face-to-face learning; none of the above. The Prin Project at the University Roma Tre: Experimental Evaluation of e-Learning Effect on the Quality of University Teaching Nottingham – 6 April 2006

13. The experimental group was composed of the students who chose the first of the mentioned options. The results of the final exams, taken by these students at the end of the module, in May, June and July 2005, have been compared with the ones of the other groups. Moreover these data have been compared with diachronic series of the results of the final exams, from the academic year 2000/2001. Experimental Evaluation of e-Learning Effect on the Quality of University Teaching Nottingham – 6 April 2006

14. Experimental Evaluation of e-Learning Effect on the Quality of University Teaching Two different online laboratories (modules) have been created ad hoc: Introduction to Experimental Research – first term; Introduction to Educational Measurement Research – second term Nottingham – 6 April 2006

15. The project To carry out the project it has been necessary to develop a series of activities such as the: identification of the background characteristics of the students attending the module of experimental pedagogy and measurement; identification of their pre-requisites; creation and development of the online modules; analysis of the exams results for the academic year 2004/2005; analysis of the exam results since the a.y. 2000/2001; report of the results. Nottingham – 6 April 2006 Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings

16. Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings Percentage values of final exams scores related to the different groups considered MarksFace-to-faceBlendedOnlineNot attending 180405 197666 200405 210402 220665 23156610 2401308 2501065 2672123 2714663 280208 2914862 3071512 30L22802 Failure rates1464024 Totals100 Nottingham – 6 April 2006

17. Diachronic series a.y. 2000/2001 a.y. 2001/2002 a.y. 2002/2003 a.y. 2003/2004 a.y. 2004/2005 Pass136148189162118 Present at the exam214282275206146 Fails78134864428 Percentage of failure 36,447,531,321,319,1 Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings Nottingham – 6 April 2006

18. Diachronic series and central tendency values a.y. 2000/2001 a.y. 2001/2002 a.y. 2002/2003 a.y. 2003/2004 a.y. 2004/2005 Mean 2423 25 Median2423242325 Mode2523251830 SD3,0972,7883,5933,5763,843 Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings Nottingham – 6 April 2006

19. Conclusions Lower rate of failures as regards the experimental group; Higher mean rate of scores if compared with previous years Higher frequency of best marks Good participation rate and satisfaction of the students with the initiative support the implementation of similar teaching offers. Experimental Evaluation of e-Learning Effect on the Quality of University Teaching: Some Findings Nottingham – 6 April 2006

20. Thank you! Antonella Poce poce@uniroma3.it Nottingham – 6 April 2006