Faculty Evaluation Systems: Student Evaluations of Faculty What are we measuring? “The evaluation of teachers is a mark of a good college.” Ernest Boyer, 1987, p.155

Introduction:Community College Faculty Evaluation Community College –Teaching Mission –Diverse mission of a comprehensive community college –As a Community College Teaching is at the Center of our Purpose. –RSCC a Teaching Institution – First & Foremost –Can we Measure (Evaluate) Teaching Excellence?

Defining Teaching Excellence: Three Major Emphases 1) Input –Student, teacher and course characteristics 2) Process –Classroom atmosphere, teacher behavior, student learning activities, course organization, & evaluation procedures 3) Product –End-of-course learning, affective/cognitive change, skills acquisition –Long-term learning, affective/cognitive change, skills acquisition

Presentation Outline Part 1: Faculty Evaluation Systems (Purpose) Part 2: Faculty Evaluation System: Student Evaluations of Faculty (Student Ratings - One Piece)

Part 1: Faculty Evaluation- What’s the purpose? Overall institutional purpose is key to proper evaluation. Formative & Summative –Improve Pedagogical Methods & Student Learning –Faculty, Course & Curriculum Development –Excellence in Teaching –Tenure & Promotion

Faculty Evaluation Purpose: Accountability Renewed attention to undergraduate instruction… –State Legislature, Coordinating and Governing Boards –Accrediting Bodies –SACS "The institution regularly evaluates the effectiveness of each faculty member in accord with published criteria, regardless of contractual or tenured status." "Standards for all educational programs include all on-campus, off-campus, and distance learning programs." –Source: The new SACS Comprehensive Standards governing Educational Programs.

Evaluation of College Faculty First student ratings took place in the late 1800s (Sioux City). Last 2 decades, evaluation of college faculty has increasingly become a way of life for those in higher education. Today, virtually all American colleges assess faculty performance using student evaluations along with other instruments.

Faculty Evaluation Systems Comprised of multiple measures –Student Evaluations of Faculty –Peer Evaluations –Self Evaluation –Supervisor Evaluation –Exit Interviews & Alumni Surveys –Teaching Portfolios/Dossier –Classroom observation

Part 2: Student Evaluation of Faculty (Ratings) Most widely used structured method of evaluating faculty in higher education Integral to most systems is the “instrument” by means of which student opinions & observations are collected. Student appraisal of instruction seems appropriate because, students are the only ones who observe teaching for an entire course.

Student Evaluations of Faculty (Ratings) Do these ratings accurately portray teaching effectiveness?

Student Evaluations of Faculty Quality of Ratings? Legitimacy? Higher education tends to question the quality and legitimacy of the info. collected through student ratings of faculty ….

Concerns about Evaluations (student ratings) No surprise a significant # of studies focused on objectivity and validity of such evaluations –2,000 + published studies Concerns include: –Low validity & reliability –Appropriateness of items (instrument) –Correlation between grades & evaluation –Measure “How good an entertainer rather than how good his/her teaching skills are” –Popularity contests that reward classroom entertainers & easy graders –Students don’t take these seriously

Concerns Continued… Student opinion can be affected by variables beyond the faculty member’s control.. –Class size –Course content

Student Evaluations of Faculty – Grades & Ratings Research findings support the notion that instructors CANNOT purchase FAVORABLE student ratings through easy grading. Research also indicates the relationship between grades & ratings is a function of the better achieving students’ greater interest and motivation.

Despite reservations… More than half of the higher education institutions in the nation continue to use student appraisals of faculty within their faculty evaluation systems. Student perceptions of end-of- course evaluations indicate that do take these seriously, but they need to understand how the results are used…

The Quality of Evaluative Information Reliability – Free from error (Computing & Measurement- accurate and consistent results) Validity – Meaning (Measuring what you think you are?) Generalizability – Representativeness (Whose opinions are reflected by data?) (Does sample of info. portray the totality of the person’s teaching?) Utility (What purposes can the data serve?)

The Instrument: Asking the right questions Definition of teaching quality (input, process & product) Purpose of evaluation Usefulness of evaluative info.: Information should be detailed, diagnostic, and focused on specific teaching behaviors and course characteristics to derive maximum benefit of information.

The Instrument: Evaluative Questions Two types of evaluative questions found in most student evaluation instruments –1) Student reaction to instructor traits or behaviors; characteristics of course material; and social and physical environment –2) Student outcomes (Progress toward general or specific educational goals) –3) You might find a third type: Demographics

The Instrument: Format of Questions Whether outcomes or trait questions… –Multiple Choice/Scaled Responses (Best # of points for a particular scale – empirical question) Some better than others –Open-ended Augment multiple choice items Helpful & constructive recommendations for improvement

Areas of Measurement: Grouping the Questions Global: require high inference (making judgments) –Instructor –Course –General (instructional climate) Specific Items: (descriptive & diagnostic) –Course management –Instructor characteristics & style –Reading material/assignments –Exams –Student behavior (effort, involvement, etc.) –Student outcomes of instruction Open-ended:

High Inference Questions vs. Specific Questions Specific multiple choice/open-ended probing questions Diagnostic information Most useful for course/instructor improvement Very general or high inference rating scale items How would you rate this instructor’s overall teaching ability? –Little helpful diagnostic information –Most useful for administrative decisions, advisement of students…

RSCC Faculty Evaluation Instrument - completed by students Examples: Question #9: The Instructor’s examinations and/or assignments reflected the content and emphasis of the course. Question #10: The instructor returned examinations and/or assignments within a reasonable time and with appropriate explanations. Interpretation? –Self or course improvement? –Exam or assignment improvement? Question #9 reworded: The instructor’s examinations reflected the course content.

Flexibility of Instrument Many institutions tried a single institutional instrument –Criticism from faculty and students as irrelevant to the specific needs of particular programs and/or courses –Solutions: Leave room on the questionnaire for the instructor to add his/her own items Module approach – different instruments for different programs/courses (labs, distance learning courses, clinicals, small or large lecture etc.)

Flexibility of Instrument Does one size fit all? Different instruments for different types of courses? –Lab –Clinical –On-line –IDEA Room –Small lecture or large lecture –Seminar –Other

Utility of Assessment Information depends on… Quality of instrument Reporting Relies on faculty perceptions of evaluation process –quality issue Student perceptions of the evaluation process

Summary In sum, then, provided that data are gathered carefully, reported appropriately, and interpreted judiciously, student evaluations appear to make a useful contribution to course improvement, faculty/self development, personnel decisions, and, possibly to student learning.

Remember Research tells us that Students’ early evaluations are quite stable and instructors should be aware that the first few days of class may be very important in determining the eventual image students evaluate at the end of the course.

Questions? Can we measure teaching excellence? Future direction? (Planning/Implementation Time)