USING STUDENT EVALUATIONS AT STOCKTON Heather McGovern Director of the Institute for Faculty Development Associate Professor of Writing January 2012

What you need to do 1) Select objectives soon. Use the terminology w/students. 2) Allow time on your syllabus for evaluations, last 2 weeks of class in the spring term for paper evals, 5 days near the end of the term for online evals, and shorter times for paper evals in fall and summer terms. 3) Through the Stockton portal, officially indicate which learning objectives are relevant to each class. Your deadline for doing this is the 2 nd precepting day. Directions will follow in from Heather McGovern. 4) When you select learning objectives, double check your disciplinary comparison code; me about changes.

Selecting Objectives I: Your objectives affect your scores In the Progress toward Relevant Objectives scores, items of minor importance do not count at all. Items that are “essential” count double items that are “important.”

Selecting objectives II: Which objectives should I select?  Pedagogy and assignments help students progress on the objective  A significant portion of a student’s grade is comprised of a meaningful measurement of how well students have achieved that objective

Selecting objectives III: Consider program requests Your program may have suggestions. E.g., the writing program and first year seminars suggest objectives for W and first year seminar classes, and science lab courses often have a supervisor who suggests objectives. Programs cannot force selections upon you, but program guidance can help.

Selecting objectives IV: How many objectives should I select? Usually, 1-5. Selecting too many objectives is usually problematic  It is hard for students to make progress on any one objective when the class has many objectives  IDEA research says as larger numbers objectives (over 5) are selected, student ratings tend to decrease

Selecting objectives V: Myths about IDEA objectives  I have to choose 3. No.  I have to have at least one essential or I have to have at least one important. No.

Selecting a disciplinary code I Ideally, your code is as good a match to your class as possible. A match has been selected for you. If it is reasonable, do nothing. If you think it could be better, contact me about a possible new one.

Interpreting Results I: What do results report and 3 major factors Student evaluations gives you students’ perceptions, which is not always the same as student learning or as reality. Three things that can affect scores: 1) Outliers 2) Error of central tendency 3) Halo effect

Interpreting Results II: Use whichever (adjusted or raw) scores are higher Adjusted scores adjust for “student motivation, student work habits, class size, course difficulty, and student effort.

Interpreting Results III: Norming sorts into broad categories (Graph, pg. 1) Scores are normed. Therefore, it is unrealistic to expect most people to score above the similar range. Statistically, 40% of people score in the similar range and 30% above and 30% below that range.

Improving Teaching I Look to the information on page three to see what steps you might take to improve student progress on various objectives. Research strongly indicates that teachers who consult with someone about results are more likely to see improvement in their results in the future.

Improving Teaching II  Consider collecting midterm feedback, especially in your first term at Stockton or when teaching a new or revised course. I’ll a form, but use whatever format you find helpful. And talk to someone about the results!  The IFD (F227) has resources—books, me, video cameras, etc.

Improving Evaluations I Students will more likely perceive progress if you relate to them, early and often, what goals they are working on and use vocabulary similar to that on the IDEA form. This means that your syllabus, assignments, and in-class discussion/lecture should refer to the goals and help students see how activities are intended to help them make progress on the goals.

Improving Evaluations II  Use page four on your IDEA forms to track student feedback over time—the items in bold should match the items on which you’re receiving the highest scores if your students perceive that they have made progress on the learning objectives you selected for the class. High scores in items you didn’t select may point to things you might add later. Low scores in items you did select point to a need to make a change.

References  “Interpreting Adjusted Ratings of Outcomes.” 2002, updated terpretingAdjustedScores.pdf terpretingAdjustedScores.pdf  Pallet, Bill. “IDEA Student Ratings of Instruction.” Stockton College, May  “Using IDEA Results for Administrative Decision- making.” dministrative%20DecisionMaking.pdf dministrative%20DecisionMaking.pdf