1. Measuring Teaching Effectiveness in Higher Education Dr. Jackie Hill

2. Let’s consider a few “age old” questions in academia …. 1) How do we currently measure teaching effectiveness at OU? A) Student evaluations & their uses in faculty evaluation 2) Are student evaluations a good way to measure teaching quality? A) Variables that impact instructor evaluation B) Limitations 3) What are other ways to measure teaching effectiveness?

3. The Student Evaluation At OU Lancaster, student evaluations are used for relied upon heavily for summative decisions and evaluation of teaching quality. At OU Lancaster, the student evaluation form (SEF) contains 13 quantifiable questions, and 3 free response questions. SEFs are completed in person, during class time. The campus average at OUL is typically 4.1 - 4.3 out of 5.

4. The Questions Questions answered on scale of 1-5 (1=Poor; 5=Excellent) 1) Overall rating of the course 2) Clarity of instructor’s oral delivery 3) Instructor’s enthusiasm for course 4) Thoroughness of instructor’s preparation 5) Instructor’s openness to student ideas 6) Instructor’s ability to motivate interest 7) Clarity of instructor’s course objectives 8) Ability to relate content with objectives 9) Clarity of instructor’s grading criteria 10) Correlation of exams and course content 11) Timeliness of comments on student work 12) Intellectual challenge in the course 13) Overall rating of the instructor

5. The Questions Free response questions 1) What do you consider to be the strengths of the instructor and/or the course? 2) What do you consider to be the weaknesses of the instructor and/or the course? 3) What suggestions do you have for the improvement of the instructor (e.g. method, approach, attitude) or the course (texts, content, etc.)?

6. The Issues 1) Student evaluations are used almost exclusively to evaluate teaching effectiveness for Group 1 and 2 merit pay, and P&T decisions, at OU Lancaster –Teaching accounts for 70-80% of job duties 2) Group 3 renewals are dependent almost exclusively upon student evaluations. Should student evaluations be weighted so heavily in summative decisions?

7. Let’s consider a few “age old” questions in academia …. 1) How do we currently measure teaching effectiveness at OU? A) Student evaluations & their uses in faculty evaluation 2) Are student evaluations a good way to measure teaching quality? A) Variables that impact instructor evaluation B) Limitations 3) What are other ways to measure teaching effectiveness?

8. Variables that influence student evaluation of professors 1) Grades. Students that report their expected grade as “A/A-” give the instructor a higher rating. Krautmann and Sander 1999. –The heavy reliance on student evaluations in personnel decisions is likely a contributing factor in grade inflation (Goldman, 1985; Lanning and Perkins, 1995; Weller, 1986; Zirkel, 1999). 2) Professor rank. Full professors tend to receive lower ratings than lower ranked faculty members. Aigner and Thum (1986). 3) Course difficulty, amount of preparation required. The more difficult the course, the lower the ratings. Aigner and Thum (1986) 4) Conflicting reports on variables such as class size and gender of professor.

9. Issues with a heavy reliance on student evaluations 1) At the regional campus, teaching accounts for 70-80% of our job duties. Currently, the most heavily weighted metric to evaluate teaching is student evaluations. Therefore, there is…. 2) Significant pressure on faculty members – especially probationary and adjuncts- to lower grading and/or teaching standards to achieve higher ratings. 3) Lowering grading standards and/or student expectations negatively impacts student learning. Problem that I deal with in the classroom

10. Do I think student evaluations are useful? Yes, student evaluations help give accurate picture of: –Instructor timeliness of grading, punctuality, organization, instructor clarity on expectations, student satisfaction of course But, we should be careful in using student evaluations as the only metric for: –Making merit pay, promotion, and tenure decisions –Evaluation of course content or instructor capabilities. Instructors should not be punished for offering a “difficult course.” Students may offer some good insight into course content, but they should not be the ones exclusively weighing in on course difficulty and rigor

11. Let’s consider a few “age old” questions in academia …. 1) How do we currently measure teaching effectiveness at OU? A) Student evaluations & their uses in faculty evaluation 2) Are student evaluations a good way to measure teaching quality? A) Variables that impact instructor evaluation B) Limitations 3) What are other ways to measure teaching effectiveness?

12. Other ways to measure teaching effectiveness Pros and cons of: Peer evaluation Self-evaluation Videos Student interviews Exit and alumni ratings Employer ratings Administrative ratings Teaching scholarship Teaching awards Learning outcome measures Teaching portfolios

13. How do YOU measure teaching effectiveness and quality in your classroom? Do you rely on student evaluations exclusively, or do you already employ one of the other strategies?