1. Analyzing and Improving College Teaching: Here’s an IDEA Alan C. Lacy, Associate Dean College of Applied Science and Technology Illinois State University

2. Overview Assessment of Teaching ◦ Components of Teaching ◦ Sources of Evaluation The IDEA Student Rating System ◦ Mission of IDEA ◦ Administration of IDEA ◦ Using the results from IDEA

3. ASSESSMENT OF TEACHING

5. How well did I teach it? What are the components of effective teaching? Who should assess these components?

6. Components of Teaching Content Expertise Course Design Instructional Delivery Course Management

7. Sources of Assessment Students Peers Administrators Self

8. 1 = not important 2 = of minor importance 3 = somewhat important 4 = very important CJ- Cannot judge The appropriateness of an instructors course objectives Faculty member’s contributions to course/curriculum development Severity, laxness, and consistency of grading standards

9. 1 = not important 2 = of minor importance 3 = somewhat important 4 = very important CJ- Cannot judge The degree to which subject matter presented is balanced and current Relevance and quality of assessment and testing methods Faculty member’s disciplinary knowledge/skills

10. Students are not qualified to assess: Faculty expertise Appropriateness of goals, content, and organization of course Materials used in delivery How student work is evaluated, including grading practices

11. Nor are students qualified to assess: Support for departmental efforts Assistance to colleagues Contributing to a positive atmosphere

12. Student are qualified to assess: Instructional Delivery Course Design Instructor Availability Progress To Course Objectives

13. SOURCES COMPONENTS StudentsPeersChairSelf Content Expertise NoYes/NoYes Course Design Yes NoYes Instructional Delivery YesNo Yes Course Management No Yes Assessment Matrix

14. THE IDEA STUDENT RATING SYSTEM

15. What is IDEA? Individual Development & Educational Assessment (IDEA) Student Rating System The IDEA Center Kansas State University Manhattan, Kansas http://www.idea.ksu.edu/

16. Mission of the IDEA Center To serve colleges and universities committed to improving learning, teaching, and leadership performance

17. IDEA Formal Student Evaluation Developed at Kansas State University Based on research on teaching Has national norms Is tailored to instructor and course Very detailed and helpful reports (Formative/Summative) Currently utilized at 275 institutions ◦ (8 years ago – 110)

18. IDEA – Improvement of Student Learning Diagnostic tool to guide improvement Evaluative tool to assess teaching effectiveness

19. Improvement of Student Learning Student ratings have positive impact if…. ◦ The instrument:  Is learning focused  Provides a diagnostic ◦ Emphasis on summative evaluation is appropriate  30-50% of overall evaluation of teaching  6-8 representative classes (more if n<10) ◦ Faculty trust the process

20. Improvement of Student Learning Campus and college culture has great impact. ◦ Is teaching excellence a top priority? ◦ Are student ratings given an appropriate weight in overall evaluation of teaching? ◦ Does the institution provide resources to aid those seeking to improve instruction? ◦ Is credible support for improvement available?

21. Other Teaching Activities Teaching Lab Classes Curriculum Development Mentoring Students Supervising Field Experiences

22. Student Learning Model Assumptions Types of learning must reflect instructor’s purpose Effectiveness determined by student progress on objectives stressed by the instructor Specific teaching behaviors influence certain types of student progress under certain circumstances

23. IDEA – Assessment of Teaching Pilot tested at ISU in 2001-2002 40 Tenure/Tenure track Faculty 975 students Fully implemented now

24. Administration of the IDEA System Complete Faculty Information Forms (FIF) Administering Student Course Evaluations Ship Evaluations to IDEA Distribute IDEA Results to Departments

25. Costs for IDEA Staff time Monetary ◦ $0.13 per course evaluation form ◦ Shipping costs ◦ $6 per class ◦ Small charges for reports

26. Fall 2008 Administration Assessed 400 classes in 7 departments 11,500 student response forms $3,224 total cost ◦ Purchase of forms ◦ Processing fees

27. Rationale for Use Shows commitment to teaching Provides validated course evaluation instrument based on research Provides consistency across the College Summative aspects aid personnel decisions Formative aspects aid professional development

30. Using the IDEA Results  Fall 2008 College Data Compared to the IDEA Database December 2008

31. Adjusted Scores Accounts for factors beyond instructor’s control Uses regression equation ◦ Student work habits (#43) ◦ Student motivation (#39) ◦ Class size (enrollment, FIF) ◦ Student effort (multiple items) ◦ Course difficulty (multiple items) Nothing like this in homegrown instruments

32. Using the Results Outcomes ◦ Student Learning Process ◦ Teaching Methods ◦ Instructional Approach ◦ Course and Curriculum Emphasis Program Assessment Inputs ◦ Student Background ◦ Student Characteristics

33. Guiding Principles to Formal Evaluation Decisions Comprehensive and systematic Valid, summative information Formative information for improvement Consider teaching components Multiple sources of assessment Consider scholarship on best practices Fair and equitable

34. Thank you for your attention! Any Questions?