Summary of CPHS Course Evaluations: AY and Chuck Phillips August 2009

IDEA Evaluations: Overview Summary StatisticsAY 07-08AY Number of Classes Evaluated9981 Average Class size7580 Overall Response Rate75%76% On-line Response Rate Mean (s.d.)64% (s.d. 21%)65% (s.d. 19%) Range0-100%33-100% Old 192 and small classes In-class Response Rate Mean (s.d.)91% (s.d. 8%)93% (s.d. 6%) Range69-100%76-100% Average # of Objectives: E or I4.3

Response Rate Issues Low response rates on-line were for either the old therapeutics (192 with several instructors) or classes with very small n (a few electives). Expect average response rate to increase in for on-line evals

Class format Increase in lecture probably due to fewer individual lab sections being evaluated.

AY AY Average% of classes below 3.0 % of classes 4.0 or above Amount of Reading CPHS ✓ 52% 41% 8% 16% IDEA3.233%15% Amount of work in other (non- reading) assignments CPHS % 32% 28% 20% IDEA3.421%18% Difficulty of subject matter CPHS3.4 36% 33% 31% IDEA3.420%18% Values are similar if within 0.3 1=Much less than most courses, 2=less than most, 3=about average, 4=more than most, 5=much more Amount and Difficulty of Course work: Student Ratings

Instructor Related Course Requirements (Some or Much required) Reading and memorization were new categories in

Instructor Related Course Requirements Breakdown

Percent of CPHS classes selecting objective as either Essential or Important (FIF)

Student ratings of progress on objectives chosen as Essential or Important 1=no progress 2=slight progress 3=moderate progress 4=substantial progress 5=exceptional progress

Percent of Reliable CPHS Classes in each Category vs. IDEA (Adjusted scores*) Expected Distribution A. Progress on relevant objectives B. Excellence of teacher C. Excellence of course Summary (Average of A, B, C) Much Higher10%1%0% 4%3%2%1% Higher20%15%13%25%27%19%20% 18% Similar40%53% 53% ✓ 46%39%41%35%46%51% Lower20%13%21%17% 27%15%23%14% Much Lower10%18%13%12% 17% ✓ 9% 27% ✓ 9%16% >30% in top 2 categories: teaching effectiveness appears to be superior to IDEA Database * Results adjusted for 5 factors: Student motivation to take the class regardless of who taught it; Student work habits; Class size; Student effort not attributable to the instructor; Course difficulty not attributable to the instructor

CPHS Adjusted Mean Scores A. Progress on relevant objectives B. Excellence of teacher C. Excellence of course Summary (Average of A, B, C) All CPHS Courses IDEA System Courses

Overview On-line evaluations still have lower response rate  Improving with use of fewer ‘co-teachers’ but can improve more CPHS similar to national database on:  Amount of reading  Amount of work in non-reading assignments  Difficulty of subject matter  Do our students have higher expectations?

Overview Ave. number of objectives still 4.3  3-5 recommended as max.  High on CT, Memorization, reading For computer application, math/quant, & creative/artistic: Almost none require “Much”

Overview Primary class format remains lecture  Should more active learning be stressed? Faculty believe more of our courses address “Learning to apply course material” & “Developing specific skills, competencies, and points of view” compared to IDEA database

Overview Students rate us lower on:  Learning to analyze & critically evaluate  Developing clearer understanding and commitment to personal values  Developing skills in expressing myself orally or in writing  Gaining broader understanding & appreciation of intellectual/cultural activity

Overview We do better in  Excellence of Teacher  Excellence of Course  Vs. Progress on relevant objectives But all three are lower than expected Everyone needs development: almost no course or instructor in 90 th %ile

Questions to Consider Are we balancing teaching methods and course requirements as intended Are we emphasizing correct objectives Are courses rigorous? What has been the affect of the new curriculum? Do we need to require computer application, math/quantitative?

Questions to Consider Too much lecturing? Disconnect on what faculty say vs. students Where should we improve?? Others?

Continued Assessment ‘When you dance with a bear, you can’t quit just because you’re tired” (Russian proverb) Need to continue assessment work Need to improve toward goals Better measure student outcomes