© 2008 Brigham Young University–Idaho Course Evaluations at BYU-Idaho 1

© 2008 Brigham Young University–Idaho 1. Assess course quality and teaching performance. 2. Assist in teacher improvement. 3. Give students an opportunity to provide input 4. Satisfy accreditation requirements Primary Objectives of Course Evaluation 2

© 2008 Brigham Young University–Idaho The Numbers 3 F09W10S10 Classes Evaluated 1,5141,4061,264 Instructors Evaluated Evaluations Administered 44,79443,03539,996

© 2008 Brigham Young University–Idaho Completion Rates 4

© 2008 Brigham Young University–Idaho Completion Rates 5 CollegeF09W10S10 Agricultural & Life Sciences65%64%62% Business & Communication69% 67% Education & Human Development72%68%69% Language & Letters64%66% Performing & Visual Arts53% 52% Physical Sci & Engineeering76%78%76% Grand Total66%68%

© 2008 Brigham Young University–Idaho The Overall Instructor Rating 6

© 2008 Brigham Young University–Idaho We aggregate and track... Overall instructor rating Overall course rating Perceived learning Hours of preparation Satisfaction Learning model scales We use the overall instructor rating to... Support CFS decisions Compare classifications (online, adjunct, veteran, 1-year) Watch trend Red flag instructors at or below 10%tile How Does the Administration Use The Data? 7

© 2008 Brigham Young University–Idaho Time of Day Student traits Age Academic aptitude GPA Class level Personality Instructor Traits Age Years of teaching experience What the Research Shows – Non-factors 8

© 2008 Brigham Young University–Idaho Class Size Smaller classes tend to receive higher ratings. Reason for Taking Course Elective courses receive higher ratings than required courses. Expected Grade Positive but low. Discipline The highest ratings go in the following order to: 1. Arts & Humanities 2. Biological & Social Sciences 3. Business & Computer Science 4. Math, Engineering, & Physical Sciences What the Research Shows - Factors 9

© 2008 Brigham Young University–Idaho Difficulty Level Courses that are more difficult or have greater workloads received higher ratings. Course Level Upper division receive higher ratings than lower division. Motivation Prior interest in subject matter or class leads to higher ratings. Student Major Majors are a bit more positive. What the Research Shows - Factors 10

© 2008 Brigham Young University–Idaho Instructor Gender Same-gender instructors receive slightly higher ratings. Instructor Personality Knowledgeable, warm, outgoing, and enthusiastic teachers receive higher ratings. Scholarship Teachers with more publications receive slightly higher ratings. Timing Administration during final exams are lower. PR Framing re promotion & tenure gets slightly higher ratings. What the Research Shows - Factors 11

© 2008 Brigham Young University–Idaho DO THESE FINDINGS HOLD FOR BYU-IDAHO? 12

© 2008 Brigham Young University–Idaho Findings From 2009 – 2010 Data 13

© 2008 Brigham Young University–Idaho Findings From 2009 – 2010 Data 14

© 2008 Brigham Young University–Idaho Findings From 2009 – 2010 Data 15

© 2008 Brigham Young University–Idaho Findings From 2009 – 2010 Data 16

© 2008 Brigham Young University–Idaho Findings From 2009 – 2010 Data 17 PS & Eng Ag & LSBus & Comm Ed & HD

© 2008 Brigham Young University–Idaho Findings From 2009 – 2010 Data 18 Gender of Student Gender of InstructorFemaleMale Female Male Overall

© 2008 Brigham Young University–Idaho Findings From 2009 – 2010 Data 19

© 2008 Brigham Young University–Idaho Findings From 2009 – 2010 Data 20

© 2008 Brigham Young University–Idaho Supplement with direct measures of learning Adjust the composite measures for Discipline Gender Reason for taking class Grade Expected Rework the Learning Model scores Better reports Shorten the instrument Develop some information for students Dreams for the Future 21

© 2008 Brigham Young University–Idaho Are we asking the right questions? Are we asking too many questions? Should we evaluate every course every semester? Is the timing of the evaluation optimal? How do you use course evaluation data (numbers and comments)? Is the information valuable? Are the reports adequate? Should some of the course evaluation data be made available to students to counter biased information on the web? Discussion Items 22

© 2008 Brigham Young University–Idaho

Raw vs Weighted Averages 24 85% see little or no difference (-.1, 0,.1) = 85% 76 get a better rating; 21 get a worse rating 2 get a significantly worse rating