1. Australasian Higher Education Evaluation Forum (AHEEF) 2008 "Evidence based Decision Making: Scholarship and Practice" 2-3 October, 2008 Australasian Higher Education Evaluation Forum (AHEEF) 2008 "Evidence based Decision Making: Scholarship and Practice" 2-3 October, 2008 eVALUAte: development and validation of a teaching survey Beatrice Tucker, Beverley Oliver and Julie-Ann Pegden

2. eVALUate surveys for student feedback eVALUate unit survey (for all current students enrolled in course work units) eVALUate teaching survey (enables students to give feedback to individual teachers on their request)

3. Literature Review Systems for Measuring Teaching Quality – controversial, validity is questionable (Pounder 2007, Kulik 2001, Kember & Leung 2008, Remedios & Lieverman 2008, Burdsal & Harrison 2008) Used for different purposes –teacher development, promotion, awards –for improving quality of courses, quality audits, benchmarking, funding allocation –student choice of study (Tucker, Jones, Straker, & Cole, 2003, Spencer & Schmelkin, 2002, Ballantyne, 2003; Ory, 2000, Department of Education Science and Training, 2004, Carey & Gregory, 2003) Students and teachers have differing opinions about what they perceive to be good teaching (McLean 2001)

4. eVALUate surveys at Curtin eVALUate unit survey – for unit review, annual and comprehensive course review, student choice of units, benchmarking, promotions eVALUate teaching survey - for teacher reflection and development, promotions, awards

5. Development of survey 2003 – committee –students –teaching academics –Deans of teaching and learning –academics with expertise in survey design and evaluation systems (statisticians) Literature was reviewed –excellent teacher characteristics, current instruments used Unit and teaching items included in one survey Oliver, Tucker, Gupta, & Yeo (2008)

6. Method - focus group Students Managers Teaching academics de Bono's Six Thinking Hats

7. Method - item development Characteristics of a good teacher Focus GroupLiteratureOther Surveys Item

8. eVALUate teaching survey 1.Appears knowledgeable in this subject area The teacher seems to have a good understanding of the subject. 2.Is enthusiastic in teaching this unit The teacher makes the subject interesting and conveys his or her enthusiasm for the subject. 3.Is well organised The teacher has material prepared on time, is punctual and structures activities in ways that help learning. 4.Communicates clearly The teacher is easy to understand in face-to-face, online, written and other formats and explains concepts clearly. 5.Is approachable The teacher encourages students to ask questions and seek help. 6.Provides useful feedback The teacher provides timely and helpful feedback so you can learn. 7.Is an effective teacher Overall, this teacher helps you to learn.

9. eVALUate teaching survey 8. Please comment on [the teacher]'s teaching strengths. 9. Please comment on how you think [the teacher] might improve the teaching and learning in this unit.

10. Testing and statistical analysis Piloted (x4) to student groups in 2003-4 tested for face validity Pilot 4 - Feedback from teaching staff and students Surveys separated Trialled to all Faculties in 2006 Factor analysis and Rasch analysis – tested the categorical scale and psychometric properties of the items (instrument measures multiple teacher attributes and not just one dominant trait)

11. How students give feedback

12. Uptake by teachers

13. Teaching surveys drive unit survey response rates *significant difference p<0.001

14. Conclusion All teaching staff are required to conduct a teaching survey annually Teaching surveys help drive unit response rates Teaching eVALUate reports are used to reward staff: for promotion and awards