1. Main components of effective teaching

2. SRI validity A measuring instrument is valid if it measures what it is supposed to measure SRIs measure: Student opinions about/perceptions of effective teaching, their satisfaction with instruction CIOS: Course/Instructor Opinion Survey SRIs are valid because they do measure student satisfaction with instruction

3. However, in my talk I’ve shown that: Student satisfaction with instruction (ratings on the Overall Teaching item) does correlate in a moderate to high extent with: 1.Student learning 2.The conceptual structure/main general behaviors of effective teaching 3.Other measures of effective teaching (i.e. by alumni, peers, experts, trained observers, and by self ratings) And: 4.There are almost no proven factors that bias SRI validity

4. Factors controllable by the teacher, found not to bias validity 1.Expressiveness/enthusiasm 2.Course (perceived) difficulty/overload 3.Course grades

5. Illustration: no relationships between rating of the items: course difficulty and Overall Teaching TAU: All University undergraduate lecture courses (n = 1,770), Fall Semester 2007, r =-.02

6. Risks with factors controllable by the teacher: Faculty may manipulate these factors hoping to get higher ratings—by entertaining students, lowering course demands, grade inflation and altogether, watering down the course… There is evidence that this already happens…

7. Factors uncontrollable by the teacher, found not to bias validity 1.Class size (smaller classes are rated higher) 2.Discipline (“soft” disciplines are rated higher) 3.Student GPA (higher GPA tend to be rated higher) 4.Student motivation (elective courses tend to be rated higher) 5.Student level of studies (graduate courses tend to be rated higher) 6.A variety of extraneous factors

8. Extraneous factors (should not affect teaching or student learning) that were studied (to discredit SRIs) Faculty: academic rank, age, gender, years of teaching experience, personal characteristics (other than enthusiasm or caring), physical attractiveness, with Asian accent, research productivity Students: age, gender, personality Course: the time of day it is offered, length of class meetings, number of rows in the classroom

9. Conclusion (Marsh, 2007): SRIs are valid SRIs correlate only minimally with extraneous variables but correlate reasonably with effective teaching behaviors and with student learning SRIs are relatively valid against a variety of indicators of effective teaching and relatively unaffected by a variety of variables hypothesized as potential biases

10. SRI reliability Reliability is how accurately an instrument measures whatever it measures SRI reliability refers to the stability and consistency of the measurement data

11. Consistency: Repeated measurements should give almost the same result. To illustrate with measurement of same issue in 2 semesters:

12. Stability: Measurement of same issue in different times and by different students provides stable/consistent results: Illustration with teacher ratings profile

13. Conclusion by Marsh & Baily (1993) Instructors appear to have distinct profiles of strengths and weaknesses that are highly consistent across sets of ratings obtained for the same instructor over a 13-year period and across undergraduate- and graduate- level courses.

14. Conclusion: SRIs are valid and reliable to a large extent