Selecting Evidence to Demonstrate Teaching Effectiveness The Institute for Learning and Teaching - TILT [Jen]

Session Outcomes Participants will be able to… Describe the importance of triangulation of evidence sources Review the strengths/limitations of several types of evidence Use a teaching goal to select evidence [Jen] Perhaps have last item be mentioned, not on slide?

Evaluation of Teaching Effectiveness [Jen] Broken stool - where we are Red stool- where we want to end up by defining teaching effectiveness and incorporating multiple sources of data.

[Jen] Where do you start if you want to become a more effective instructor? The TEF is a starting point for looking at the different domains of teaching… TILT is recommending that faculty select evidence based on a domain. ONe way to grow teaching and evaluate it for annual review

Evaluating Teaching Effectiveness Consider the content and quality of your current scale. Which sources will your faculty embrace? Consider strengths/ limitations of each. Which sources should be used for formative and summative assessment? Build sources one at a time, to create an assessment model for each one. Begin with student surveys Review other sources of evidence with faculty Decide which combination of evidence is best for your faculty Map out a plan to build out those sources Recommendations for Developing a Multisource Feedback Model for Evaluating Teaching Effectiveness From Ronald A. Berk, Top five flashpoints in the assessment of teaching effectiveness (2013) According to the research: start with course surveys but include others sources as well. Metric used across campus : Course surveys but as we all know, they have limitations. Based on research and demonstrated by Berk, multiple sources of evidence are needed to get a better picture of teaching effectiveness. This model represents what TILT is recommending to departmental decision makers (chair, committee, dean, etc.) to build a strong 3-legged stool.

Goal-Setting & Self-Reflection Evidence TBD Student Surveys [Jen] Mirrors the research process http:///www.careerpoint-solutions.com/courses/unit/step-2-plan-for-data-collection-and-management/

Matching Sources of Evidence to Departmental Decisions Formative Decisions (to improve teaching) Summative Decisions (annual review) (tenure & promotion) Program Decisions Student Ratings x Peer Observations Peer Review of Course Material External Expert Ratings Self Ratings Videos Student Interviews Mentor’s Review Administrator Ratings Teaching Scholarship Teaching Portfolio Exit and Alumni Ratings Employer Ratings [Jen] Going back to the research. Based on research by Berk, Wieman, Hattie, and others… we consider types of evidence. Strengths and limitations for each type of evidence so we can make informed decisions for our departments and ourselves. [Handout strengths & limitations] [COPUS, TPI, sample observations from TS, portfolio document, Recommendations from Ronald A. Berk, Top five flashpoints in the assessment of teaching effectiveness (2013)

Discussion How does triangulation apply to selecting evidence of teaching effectiveness? “Triangulation means using more than one method to collect data on the same topic. This is a way of assuring the validity of research through the use of a variety of methods to collect data on the same topic, which involves different types of samples as well as methods of data collection. However, the purpose of triangulation is not necessarily to cross-validate data but rather to capture different dimensions of the same phenomenon.” https://www.researchgate.net/post/What_is_triangulation_of_data_in_qualitative_research_Is_it_a_method_of_validating_the_information_collected_through_various_methods [Jen] Table Discussion: Ask one person at each table to report out.

Goal: Improve classroom climate so that students feel safe enough to share ideas even if they make a mistake. SMART Goal: Integrate at least three opportunities for students to interact with each other every class so that they feel more comfortable participating on a regular basis. Realistic, Relevant, or Rigorous [Tonya] Read the goal: What is this goal missing? In pairs, revise the goal to make it SMART.

Domain: Classroom Climate Goal: Worksheet Identify the domain Evidence: Strengths: Limitations: Evidence: Strengths: Limitations: Domain: Classroom Climate Goal: Worksheet Identify the domain Draft a SMART goal Identify 3 potential sources of evidence to support the goal Identify a strength and limitation for each type of evidence. [Tonya] [handout worksheet] Evidence: Strengths: Limitations:

Domain: Classroom Climate Worksheet Identify the domain Evidence: Observation of Classroom Climate - Teaching Square Strengths: Peer feedback, focused on teaching goal Limitations: time-consuming for observer, could be viewed as subjective Evidence: Class climate questions on Course survey Strengths: Focused, direct feedback from students Limitations: Not all students will complete the survey, possible bias based on who completes survey Domain: Classroom Climate Worksheet Identify the domain Draft a SMART goal Identify 3 potential sources of evidence to support the goal Identify a strength and limitation for each type of evidence. Goal: Incorporate practices to to foster an inclusive classroom climate wherein students feel safe to discuss social issues and in an environments that honors differences in experience and perspectives. [Tonya] TONYA - TURN IN TO DOCUMENT Evidence: Sample of student reflections Strengths: Focused, direct feedback from students, a large amount of data Limitations: Could be a lot of data to aggregate, illegible handwriting

Developing and Evaluating Teaching Effectiveness Collaborate with colleagues Join a Community of Practice Attend PD aligned with Teaching Effectiveness Framework/goal Peer Observations Literature research/ Self-Reflection Provost Academy Teaching Squares REFLECT & ASSESS Use evidence to gauge teaching strengths and growth areas ENGAGE Participate in professional learning aligned with the Teaching Effectiveness Framework Teaching Practices Inventory COPUS Peer Feedback Teaching Squares Self-Reflection Teaching Effectiveness Framework Annual Review Course Evaluations REFINE Write/revise a goal IDENTIFY & ALIGN Choose one growth area and align it with a domain on the Teaching Effectiveness Framework Set a SMART goal: Specific, Measurable, Achievable, Relevant, and Time-bound Determine how you will measure achievement of the goal @2019 Colorado State University 2019 CC BY-NC-ND 4.0

Session Outcomes Participants will be able to… Describe the importance of triangulation of evidence sources Review the strengths/limitations of several types of evidence Use a teaching goal to select evidence [Jen]