Analyzing and Improving College Teaching: Here’s an IDEA Alan C. Lacy, Associate Dean College of Applied Science and Technology Illinois State University

Overview Assessment of Teaching ◦ Components of Teaching ◦ Sources of Evaluation The IDEA Student Rating System ◦ Mission of IDEA ◦ Administration of IDEA ◦ Using the results from IDEA

ASSESSMENT OF TEACHING

How well did I teach it? What are the components of effective teaching? Who should assess these components?

Components of Teaching Content Expertise Course Design Instructional Delivery Course Management

Sources of Assessment Students Peers Administrators Self

1 = not important 2 = of minor importance 3 = somewhat important 4 = very important CJ- Cannot judge The appropriateness of an instructors course objectives Faculty member’s contributions to course/curriculum development Severity, laxness, and consistency of grading standards

1 = not important 2 = of minor importance 3 = somewhat important 4 = very important CJ- Cannot judge The degree to which subject matter presented is balanced and current Relevance and quality of assessment and testing methods Faculty member’s disciplinary knowledge/skills

Students are not qualified to assess: Faculty expertise Appropriateness of goals, content, and organization of course Materials used in delivery How student work is evaluated, including grading practices

Nor are students qualified to assess: Support for departmental efforts Assistance to colleagues Contributing to a positive atmosphere

Student are qualified to assess: Instructional Delivery Course Design Instructor Availability Progress To Course Objectives

SOURCES COMPONENTS StudentsPeersChairSelf Content Expertise NoYes/NoYes Course Design Yes NoYes Instructional Delivery YesNo Yes Course Management No Yes Assessment Matrix

THE IDEA STUDENT RATING SYSTEM

What is IDEA? Individual Development & Educational Assessment (IDEA) Student Rating System The IDEA Center Kansas State University Manhattan, Kansas

Mission of the IDEA Center To serve colleges and universities committed to improving learning, teaching, and leadership performance

IDEA Formal Student Evaluation Developed at Kansas State University Based on research on teaching Has national norms Is tailored to instructor and course Very detailed and helpful reports (Formative/Summative) Currently utilized at 275 institutions ◦ (8 years ago – 110)

IDEA – Improvement of Student Learning Diagnostic tool to guide improvement Evaluative tool to assess teaching effectiveness

Improvement of Student Learning Student ratings have positive impact if…. ◦ The instrument:  Is learning focused  Provides a diagnostic ◦ Emphasis on summative evaluation is appropriate  30-50% of overall evaluation of teaching  6-8 representative classes (more if n<10) ◦ Faculty trust the process

Improvement of Student Learning Campus and college culture has great impact. ◦ Is teaching excellence a top priority? ◦ Are student ratings given an appropriate weight in overall evaluation of teaching? ◦ Does the institution provide resources to aid those seeking to improve instruction? ◦ Is credible support for improvement available?

Other Teaching Activities Teaching Lab Classes Curriculum Development Mentoring Students Supervising Field Experiences

Student Learning Model Assumptions Types of learning must reflect instructor’s purpose Effectiveness determined by student progress on objectives stressed by the instructor Specific teaching behaviors influence certain types of student progress under certain circumstances

IDEA – Assessment of Teaching Pilot tested at ISU in Tenure/Tenure track Faculty 975 students Fully implemented now

Administration of the IDEA System Complete Faculty Information Forms (FIF) Administering Student Course Evaluations Ship Evaluations to IDEA Distribute IDEA Results to Departments

Costs for IDEA Staff time Monetary ◦ $0.13 per course evaluation form ◦ Shipping costs ◦ $6 per class ◦ Small charges for reports

Fall 2008 Administration Assessed 400 classes in 7 departments 11,500 student response forms $3,224 total cost ◦ Purchase of forms ◦ Processing fees

Rationale for Use Shows commitment to teaching Provides validated course evaluation instrument based on research Provides consistency across the College Summative aspects aid personnel decisions Formative aspects aid professional development

Using the IDEA Results  Fall 2008 College Data Compared to the IDEA Database December 2008

Adjusted Scores Accounts for factors beyond instructor’s control Uses regression equation ◦ Student work habits (#43) ◦ Student motivation (#39) ◦ Class size (enrollment, FIF) ◦ Student effort (multiple items) ◦ Course difficulty (multiple items) Nothing like this in homegrown instruments

Using the Results Outcomes ◦ Student Learning Process ◦ Teaching Methods ◦ Instructional Approach ◦ Course and Curriculum Emphasis Program Assessment Inputs ◦ Student Background ◦ Student Characteristics

Guiding Principles to Formal Evaluation Decisions Comprehensive and systematic Valid, summative information Formative information for improvement Consider teaching components Multiple sources of assessment Consider scholarship on best practices Fair and equitable

Thank you for your attention! Any Questions?