1. + Measuring Teaching Quality in the Online Classroom Ann H. Taylor Director, Dutton e-Education Institute College of Earth and Mineral Sciences

2. + How do we review teaching effectiveness at Penn State? Per Policy HR-23 and the “Statement of Practices for the Evaluation of Teaching Effectiveness for Promotion and Tenure”: Student reviews via the SRTE Peer review Procedures, including frequency, are left to academic units “Your mileage may vary”!

3. + How can we review teaching effectiveness online? Formal reviews Student reviews Informal reviews

4. + Are there differences in what we are looking for? Seven Principles of Good Practice Good practice in undergraduate education: 1. encourages contact between students and faculty, 2. develops reciprocity and cooperation among students, 3. encourages active learning, 4. gives prompt feedback, 5. emphasizes time on task, 6. communicates high expectations, and 7. respects diverse talents and ways of learning.

5. + Good teaching is good teaching. But the evidence may look quite different!

6. +

7. + What are we looking for? Interactions with students (including response rates and availability) Use of technology (including usability of instructor- created content) Pedagogical strategies (including facilitation of discussions) Overall management of the administrative aspects of the course

9. +

10. + What should we do with the results? Celebrate and reward good teaching Provide professional development opportunities Mentoring Learning design support Resources! (weblearning.psu.edu)weblearning.psu.edu

11. + Student Reviews

12. + What are students reviewing? Course content Interactions with instructors Use of technology Appropriateness of the technology Pedagogical strategies

13. + What are some concerns? What can, and cannot, students evaluate? How can you get meaningful feedback? Are students qualified for the task? Will they complete online evaluations? How do I get them to fill them out?

14. + What should faculty do with student feedback? Be prepared to make a few changes Take the small steps approach Read results with a thick skin Look for trends Overlook outliers Seek explanations, but don’t explain away (see http://www.lehigh.edu/~infdli/FD-evaluations.htm)http://www.lehigh.edu/~infdli/FD-evaluations.htm

16. + Tool 1: SEEQ Students’ Evaluation of Educational Quality Mid-semester feedback instrument Developed by educational psychologist Herbert Marsh Normed to determine the kinds of questions students really could answer! 32 standardized questions grouped into nine dimensions of teaching

17. + Tool 2: SALG Student Assessment of their Learning Gains Sample instrument that can be customized Statements about the degree of "gain" (on a five-point scale) which students perceive they've made in specific aspects of the class Includes open-ended questions Delivered to students online Sponsors provide a statistical report of the results

18. + Tool 3: SRTE Student Rating of Teaching Effectiveness 177 Question Choices 4 mandatory items 1. Are you taking this course as an elective? (If uncertain, omit.) 2. What grade do you expect to earn in this course? 3. Rate the overall quality of this course. 4. Rate the overall quality of the instructor. 2 open-ended mandatory items 1. What helped you learn in this course? 2. What changes would improve your learning? Up to 15 department items and up to 5 instructor items

19. + SRTE for online courses in EMS SRTE Question Bank Items: Rate the instructor's availability for individual help and consultation. (#42) Rate the effectiveness of the instructor's responses to student questions. (#48) Rate the instructor's skill in keeping the course on schedule. (#70) Rate the promptness with which graded exams, reports, and other materials were returned. (#101) Additional questions: Rate your agreement with the following statement: “The instructor initiated communication with students on a regular basis.” Rate your agreement with the following statement: “The instructor responded to my inquiries in a timely manner.” Rate your agreement with the following statement: “The instructor provided meaningful feedback on my class assignments.” Rate your agreement with the following statement: “I would recommend this course to others.”

20. + Tool 4: DIY Simple web-based form Ask anything! Lots of examples available Lots of tools available Google Template SurveyMonkey Survey Gold Zoomerang

21. + Another alternative: Gather informal feedback Minute papers, muddiest points Minute papers Self-check using formal instruments Peer Review of Online Teaching Faculty Competencies for Online Teaching Faculty Competencies for Online Teaching Online Mentoring Program (pilot)

22. + How do we get started? Start with faculty How do they define good teaching? What criteria do they think should be measured? How do we involve part-time faculty? Form a steering committee of key stakeholders Take small steps and build on successes Measure impact throughout the process Strive to continuously improve

23. + What can administrators do? Create an institutional environment where good teaching is highly valued & professional growth is expected Provide adequate and accessible faculty development tools & resources Tell faculty, staff, and students how this will benefit them Measure impact! Recognize and reward good teaching at all levels – celebrate! Encourage trial and error Act!

24. + What can faculty do? Utilize a wide array of strategies to collect feedback Document and share evidence of good teaching with superiors Request faculty development programming Work with faculty senate and other faculty bodies to strengthen support Tell your students how this will benefit them

25. + So what are YOU going to do?

26. + Follow-up? Ann H. Taylor atb3@psu.edu 814-863-9070