Student Ratings of Instruction as Formative Assessment Peg Balachowski Associate Dean of Teaching and Learning Everett Community College

Results of End-of-Course Evaluations Are you discouraged when you see the results of your student surveys? (SRIs)

Results of End-of-Course Evaluations Are you discouraged when you see the results of your student surveys? (SRIs) How do you “right the ship” if results arrive the following term?

Results of End-of-Course Evaluations Are you discouraged when you see the results of your student surveys? (SRIs) How do you “right the ship” if results arrive the following term? Begin early to create a culture of assessment in your classroom!

Results of End-of-Course Evaluations Are you discouraged when you see the results of your student surveys? (SRIs) How do you “right the ship” if results arrive the following term? Begin early to create a culture of assessment in your classroom! How do you accomplish this?

What Can We Learn from End-of-Course Evaluations? By: Maryellen Weimer, PhD, March 8, 2017 No matter how much we debate the issue, end-of- course evaluations count. How much they count is a matter of perspective. They matter if you care about teaching.

“If faculty do not participate in making sense of and interpreting assessment evidence, they are much more likely to focus solely on finding fault with the conclusions than on considering ways that the evidence might be related to their teaching.” Banta and Blaich, Closing the Assessment Loop, Change, Jan/Feb 2011

“A host of studies have shown that when formative assessment is implemented effectively, it can greatly enhance, or even double, the speed of student learning.” What Fitness Bands can Teach Us about Classroom Assessment, Karen S. Buchanan in Faculty Focus, July 22, 2015

NOT RATE MY PROFESSOR

How do we measure student learning? A review of assessment literature in the area of Scholarship of Teaching and Learning (SoTL) will quickly reveal the importance of faculty engagement in the improvement of student learning and engagement in the classroom.

How do we measure student learning? According to the authors of “Rethinking College Student Retention” (Braxton, J. et al, 2014), two of the most significant variables that impact retention are: a genuine interest in students on the part of faculty, and faculty organization and clarity.

What we know Formative assessment provides feedback to students on how they are doing. It is assessment FOR learning rather than OF learning. The feedback helps students understand how to do better next time. - i.e. how to be better students.

What we know Formative assessment provides feedback to students on how they are doing. It is assessment FOR learning rather than OF learning. The feedback helps students understand how to do better next time. - i.e. how to be better students. The same is true for faculty - this will be an opportunity to reflect on their teaching strategies and evaluate how well they are doing.

What we know Formative assessment provides feedback to students on how they are doing. It is assessment FOR learning rather than OF learning. The feedback helps students understand how to do better next time. - i.e. how to be better students. The same is true for faculty - this will be an opportunity to reflect on their teaching strategies and evaluate how well they are doing...BEFORE the student surveys are done.

Feedback as Autopsy Feedback that arrives the next quarter is more like a post-mortem: post-mortem - an examination and dissection of a dead body to determine cause of death or the changes produced by disease http://www.thefreedictionary.com/

Feedback During the Quarter

Two Research Questions: How can faculty effectively close the loop in the formative assessment process? And how can more faculty members be encouraged to engage with this process?

Student Ratings of Instruction (SRI)

Plan of Action During Winter Quarter 2017 a small group of EvCC faculty (17) representing multiple disciplines and class modalities agreed to pilot a specific formative assessment called Instant Feedback (using the online platform called Campus Labs).

Plan of Action During Winter Quarter 2017 a small group of EvCC faculty (17) representing multiple disciplines and class modalities agreed to pilot a specific formative assessment called Instant Feedback (using the online platform called Campus Labs). Faculty were paid a small stipend for their time using an IDEA High Impact Grant.

Who was in the pilot? Tenured/tenure track faculty = 7 Associate (adjunct) faculty = 10 Divisions = 5 of 7 represented (Academic and Prof-Tech) Online classes = 5 On ground = 12 Hybrid = 1 Total # faculty = 424 135 full-time, 289 associates

The Instructor... Displayed a personal interest in you and your learning. Found ways to help you answer your own questions. Demonstrated the importance of the subject matter. Made it clear how each topic fit into the course. Explained course material clearly and concisely. Introduced stimulating ideas about the subject.

Why did these faculty choose to participate? Here’s what they said: First I am a believer in formative assessments but have struggled to find the time to do them during class and this pilot offers students the opportunity to use time outside of class to leave feedback.

Why did these faculty choose to participate? Here’s what they said: First I am a believer in formative assessments but have struggled to find the time to do them during class and this pilot offers students the opportunity to use time outside of class to leave feedback. The Instant Feedback platform will help me to know if the new classroom activities I am planning to use will help my students and thus help me to be an effective teacher and change my teaching techniques if required.

Why did these faculty choose to participate? Here’s what they said: I love the idea of literally having instant feedback. Seeing what my students need/want, immediately, will allow me to make changes quickly rather than waiting for the IDEA surveys to come out once per quarter.

Why did these faculty choose to participate? Here’s what they said: I love the idea of literally having instant feedback. Seeing what my students need/want, immediately, will allow me to make changes quickly rather than waiting for the IDEA surveys to come out once per quarter. The biggest reason I want to use instant feedback is to build a culture of feedback so that students feel comfortable coming to me with their questions and concerns.

Why did these faculty choose to participate? Here’s what they said: I tell my students that learning involves being okay with beginnings and being uncomfortable. If you only stay in the safe, comfortable zone, you're probably not learning much and that's why I volunteered to participate in this pilot project. It does feel uncomfortable, risky and vulnerable because it's an unknown but I'm looking forward to learning lots!

Why did these faculty choose to participate? Here’s what they said: I tell my students that learning involves being okay with beginnings and being uncomfortable. If you only stay in the safe, comfortable zone, you're probably not learning much and that's why I volunteered to participate in this pilot project. It does feel uncomfortable, risky and vulnerable because it's an unknown but I'm looking forward to learning lots! I want to become a better teacher.

Directions for the pilot Each time you complete an Instant Feedback session, you will review the results and make a decision about the area you want to work on. Remember that there are six different areas, all related to your pedagogy, that students will be responding to. (Keep in mind that your pedagogy is something you have control of! You get to make decisions about these things!)

The Instructor... Displayed a personal interest in you and your learning. Found ways to help you answer your own questions. Demonstrated the importance of the subject matter. Made it clear how each topic fit into the course. Explained course material clearly and concisely. Introduced stimulating ideas about the subject.

LOSS GAIN

During the Quarter The feedback session was used 4 times during the quarter, at the end of weeks 2, 4, 6, and 8. This allowed faculty time to review results, read the SoTL literature for each area, let students know what they learned, and implement their plan of action. In all cases, students were kept in the loop.

After each Instant Feedback, respond to these questions: How did you evaluate and utilize the feedback you received from students? How did your teaching strategies shift or change, based on student feedback?

Which areas were most commonly cited? Displayed a personal interest in you and your learning. Found ways to help you answer your own questions. Demonstrated the importance of the subject matter. Made it clear how each topic fit into the course. Explained course material clearly and concisely. Introduced stimulating ideas about the subject.

How do we measure student learning? According to the authors of “Rethinking College Student Retention” (Braxton, J. et al, 2014), two of the most significant variables that impact retention are: a genuine interest in students on the part of faculty, and faculty organization and clarity.

What did faculty do as a result? Displayed a personal interest in you and your learning.

What did faculty do as a result? Displayed a personal interest in you and your learning. Faculty made more of an effort to address students by name, checking in individually when working in groups, doing more outreach, sent individual emails (not just in the class for the pilot) rather than relying on announcements in LMS. Used the discussion board in Canvas as a social part of class. Class related but not so intense. More personal touch. Emphasizing how students can contact the instructor (YouCanBookMe conference tool, one-on-one meetings). Students responded they appreciated the outreach.

What did faculty do as a result? Displayed a personal interest in you and your learning. Goal setting, more class interaction, asking students for feedback on ways the instructor can improve in this area based on recommendations in IDEA paper. Used the discussion board in Canvas as a social part of class. Class related but not so intense. More personal touch. Emphasizing how students can contact the instructor (YouCanBookMe conference tool, one-on-one meetings). Students responded they appreciated the outreach. Using students names, create opportunities to actively engage in classroom activities, etc

What did faculty do as a result? Explained course material clearly and concisely.

What did faculty do as a result? Explained course material clearly and concisely. Faculty found different ways to introduce new topics, assignments, and combined this with how the topics related to each other. Again, not just relying on announcements in Canvas (esp for online classes). This is a change for some faculty who sometimes let the directions speak for assignments, or a post on Canvas

What did faculty do as a result? Explained course material clearly and concisely. Modeling expectations, working through examples with greater emphasis on connecting the dots. This is a change for some faculty who sometimes let the directions speak for assignments, or a post on Canvas

What did faculty do as a result? Introduced stimulating ideas about the subject.

What did faculty do as a result? Introduced stimulating ideas about the subject. Faculty talked more about real-world applications of the subject matter, applications in careers and other courses.

Conclusion What comes next? Will faculty be convinced to continue using Instant Feedback? How will the feedback process change for these and other faculty?

Conclusion What comes next?

Conclusion What comes next? Did using this feedback loop improve retention? Did it improve the grade distribution? How will the course evalus of these faculty compare to other faculty?

Will faculty be convinced to continue using Instant Feedback? Conclusion Will faculty be convinced to continue using Instant Feedback? At this point, I do plan to use Instant Feedback next quarter in both of my classes (Engl 101 and 102), but I will only do it three times, with plus/delta once or twice. In principle, I like the regularity of the every-other-week schedule, and I know that having four data points creates more continuous improvement loops, but I really did sense survey fatigue from my students.

Conclusion How will the feedback process change for these and other faculty? Yet to be determined!

Please feel free to contact me: mbalachowski@everettcc.edu Q&A Please feel free to contact me: mbalachowski@everettcc.edu