1. Students’ Ratings of University Teaching: A Data Mining Analysis Chuck Dziuban Research Initiative for Teaching Effectiveness Ida Cook Department of Sociology and Anthropology Faculty Senate Morgan Wang Department of Statistics University of Central Florida

2. Rationale for the Study Teaching evaluation data for a 3-year period were available to allow a comparison of two different sets of items (UCF/BOR). Responding to faculty interest, the UCF Faculty Senate requested that an evaluation of the Student Evaluation of Instruction measures be performed. (FS 1995-96-11)

3. The Instrument: UCF Items Feedback on your performance in this course The instructor’s interest in your learning Use of class time The instructor’s overall organization of the course Continuity from one class meeting to the next The pace of the course The instructor’s assessment of your progress The text and supplemental learning materials used

4. The Instrument: Board of Regent Items Description of course objectives and assignments Communication of ideas and information Expression of expectations for performance Availability to assist students In or outside of class Respect and concern for students Stimulation and interest in the course Facilitation of learning Overall assessment of instructor

5. The Study Layout Approximately 450,000 student responses Five Colleges: Arts and Sciences Business Administration Education Engineering Health and Public Affairs Three Levels: Lower Undergraduate Upper Undergraduate Graduate Three Years: 1996-97 1997-98 1998-99

6. Overall Rating of the Instructor Binary Decision Tree SAS Enterprise Miner Remaining 15 Items Level Year College Developmental Sample 1 Developmental Sample 2 Developmental Sample 3 Validation Sample Predictors

7. Decision Tree Example 85.9% n=11,286 85.8% n=6,460 91.5% n=2,079 72.7% n=378 86.7% n=2,369 86.5% n=5,639 74.8% n=821 94.1% n=1,036 89.1% n=1,043 64.7% n=148 79.6% n=230 88.4% n=3,263 84.1% n=2,376 68.9% n=298 78.5% n=526 Arts & Sciences, Business Admin., Hospitality Mgmt. Education Engineering Health & Pub. Affairs F2F, E, MW femalesmalesA&SBA & Hosp. mgmt F2FE, M, WE, MF2F Overall

8. Rules Leading to Overall Rating of Excellent QuestionsExVGGFP Facilitation of Learning● Communication of Ideas and Information ● Probability =.96N=46,805

9. Rules Leading to Overall Rating of Excellent QuestionsExVGGFP Facilitation of learning● Communication of ideas and information ● Organization of the course● Assessment of student progress ●● Probability =.85N=3,462

10. Rules Leading to Overall Rating of Excellent QuestionsExVGGFP Facilitation of learning● Communication of ideas and information ● Organization of the course● Instructor interested in your learning ● Probability =.78N=6,215

11. Overall Ratings Unadjusted and Adjusted for ‘Excellent’ Decision Rules Adjusted for Rule CollegeUnadjusted123 CAS42%95%85%80% CBA36%95%83%77% COE54%97%86%75% CECS32%96%84%78% COHPA48%96%86%74%

12. For more information… Research Initiative for Teaching Effectiveness Dr. Chuck Dziuban 407-823-5478 dziuban@mail.ucf.edu Dr. Patsy Moskal 407-823-0283 pdmoskal@mail.ucf.edu http://rite.ucf.edu