CIRTL Network Meeting June 19, :00-12:30 ET/ 10:00-11:30 CT/ 9:00-10:30 MT / 8:00-9:30 PT This meeting will be recorded Begin by running the Audio Setup Wizard Select : Tools>Audio> Audio Setup Wizard or select the audio wizard icon Backup Call-in: Call-in Number: 1 (571) Passcode:

Agenda Announcements Integrating Cognitive Science with Innovative Teaching in STEM Disciplines Update Cross-Network Programming Update - possible funding opportunity Video: Joe Hardcastle’s Teaching as Research Project – Boston University (5 min) Topic 1: Local Program Evaluations Cornell University University of Colorado at Boulder

Cross-Network Update

Fall Cross Network Courses Teaching as Research I (first of two semester sequence; students can take one or both) The College Classroom (MOOC) Diversity in the College Classroom (tentative, need co- instructor) Training In Teaching and Learning Modules (or mini course) (titles tentative)

Fall Cross Network Mini Courses and Other Programming Mini Course: The Active Classroom: Supporting Successful Small Group Learning Mini Course: Student Centered Course Design Mini course: Developing a Scholarly Approach to Teaching (tentative) Journal Club, CIRTL Reads Online Learning Community for International STEM students (titles tentative)

CIRTLCast Series Themes for the one-month blocks will include: 1.Diversity Issues in STEM Education 2.Financial Literacy 3.Educational Innovations for the STEM Classrooms (2 months) 4.Job search 5. Surviving the First Year as Faculty Member 6.One other, TBA As soon as we get all the dates, titles and descriptions for the series settled, we will be soliciting speakers and panelists for the sessions. Watch for this solicitation in upcoming weeks. As soon as we get all the dates, titles and descriptions for the series settled, we will be soliciting speakers and panelists for the sessions. Watch for this solicitation in upcoming weeks.

Upcoming Cross Network Dates Mon, July 7, 2014: Fall 2014 Course Offerings Posted Online Mon, July 21, 2014: Fall 2014 Course Registration Opens Mon, August 18, 2014: Fall 2014 Annual Event Calendar Posted Mon, September 22, 2014: Spring 2015 Cat 1 Proposals Due Dates posted on the Cross Network Page at:

Funding Opportunity Update

Joe Hardcastle's Teaching as Research Project - Boston University Vimeo Link:

Cornell University CIRTL Programs on Mentor Training and Their Evaluation 1.Building Mentoring Skills Lunchtime Workshop Series (discussed in May 2013) 2.REU Mentor Trainings (initial training and optional mid- summer check-in lunch) 3.Developing video case studies 4.High-engagement experiential programs (today’s focus)

A Coupled Graduate-Undergraduate Course in Field Research Mentorship “Graduate training programs for future faculty must build a comfort level for engaging with students in the field.” Middendorf & Pohland (2014)

Overall Learning Goals Undergraduate mentees will: Develop an understanding of and appreciation for natural history Build skills and confidence to propose, develop and test hypotheses and to communicate results Practice vital collaboration and cooperation skills needed for research and the job market

Learning Goals Graduate mentors will: Empower mentees to design and confidently execute independent, authentic, inquiry-based research projects Prepare for logistical challenges and to understand and capitalize on student diversity Manage personal dynamics with co-mentors and between mentees

Undergraduate Experience: Bi-weekly course meetings during semester to research and present on aspects of the study system 10-day trip to biological field station immediately after spring semester Lead TA (designing research/evaluation project) Training Structure

Graduate Mentor Experience: Monthly meetings with professors to address learning objectives. Attendance at CU-CIRTL Building Mentoring Skills workshops Field Trip: Facilitated “exploration and inquiry” period Each mentor assigned four students Mentored for design and execution of group research project Ongoing supervision from professors and nightly group discussion (learning community) Training Structure

Questions?

Description of Evaluation Plan Implementation: Pre- and post-course surveys (separate undergraduate and graduate instruments) –Qualitative and quantitative questions –Administered online TA observation and informal interviews in the field Potential for analysis of student artifacts (research posters) Goals: Mentors: Assess mentorship skills development, attitudes toward field mentorship Undergraduate: assess research skills development, provide feedback for mentors Both: Get feedback on pilot course for future development and facilitator guide

Mentor Attitudes: Course “increased the likelihood that they will seek out field-based mentoring opportunities in the future” (for 3/3 mentors) “I feel significantly more confident and excited about field mentoring. This experience has convinced me that this should be something I try to integrate into my future career in some capacity.” Mentor Skills: Skills where self-assessed proficiency went from “low” to “medium/high” (for 3/3 mentors) Building mentee independence Building mentee confidence Developing strategies to deal with mentoring challenges Results

Mentor Confidence: Confidence increased by one or more categories for the following items: Empowering mentees to make their own observations and suggest questions and hypotheses (3/3 mentors) Refocusing mentee attention if necessary in order to complete a specific task (3/3 mentors) Dividing roles and responsibilities between co-mentors (3/3 mentors) Managing mentee interpersonal group dynamics (1/3 mentors) (options: none, low, moderate, high) Results

Open-Ended Questions “I had a conflict about choosing methods with mentees…. After slightly guiding their conversation towards the downsides of the method… I put their decision in the context of peer review. They then understood that if taken seriously and reviewed by scientists, their decisions would reflect bias and a flaw in the experiment.” Did you experience any kind of conflict between yourself and a mentee or have concerns about their actions, attitudes or progress? What steps did you take to address this?

Questions?

Evaluation Challenges and Suggestions Designated time period for surveys to encourage timely participation More targeted questions regarding career perceptions and changes for undergraduates Change trip structure so that mentors arrive early and have field time with professors Advice on further alignment to CIRTL pillars and the new learning outcomes is welcome.

Resources and References Middendorf, G. & Pohlad, B.R. (2014) Ecoliteracy for ecology and ecologists: eroded underpinnings. Frontiers in Ecology and the Environment, 12 : Course manual (in prep): Empowering tomorrow’s field instructors: A coupled graduate-undergraduate course in field research mentorship. Cornell University. Feedback for inclusion in course manual is welcome. Please address to Fiona Soper

Questions?

 TIGER CIRTL Evaluation Report Laura L. B. Border, Director, Graduate Teacher Program C0-PI CU Boulder, CIRTL Project

Program & Purpose  TIGER 2  TIGER 2 is a workshop series designed to highlight effective participation on research teams and professional development for RA’s.

TIGER 2 Workshop Content  5 Workshops on Aspects of Teamwork  3 Workshops on CIRTL Pillars  3 Workshops on Research in General  1 Workshop on Assessing Student Learning  1 Workshop on Preparation for Job Market (Research Statement)

Questions?

Evaluation Questions 1. Who and from which departments participated? 2. How many times did an individual return? 3. Did the participants report that they learned what was presented? 4. Did the participants have an idea about how they might apply the information presented? 5. Were participants satisfied or dissatisfied with the workshop? 6. Which workshops were most effective and why?

Features Measured 1. Participation 2. Dosage 3. Learning 4. Application 5. Satisfaction

Tools Used  Feedback Forms  Sign-in Sheets with demographic information  FileMaker Pro

Data Analysis  Spreadsheets  FileMaker Pro  Tracking and analysis of student responses to questions on feedback forms

Findings: Quantitative  12 workshops over two semesters  61 participants, 35 individuals  12 attended more than one workshop  13 STEM Departments represented  84% of participants returned Feedback Forms  6 individual STEM Faculty presented  Average Satisfaction Score: 4.91 (6=Excellent)

Dosage

Satisfaction  Average Satisfaction Score (6=Excellent) Fall: Workshop overall: 4.54 Fall Presenter Rating: 5.15 Spring: Workshops overall: 5.27 Spring Presenter Rating: 5.33

Effectiveness of Marketing  Where did you hear about the workshop?  GTP Website28  CIRTL Website 0  CU Calendar 4  Lead Graduate Teacher17  Department listserv 5  GTP Poster in Department 3

Feedback: Qualitative  What was most helpful part of this workshop?  What was least helpful part of the workshop?  Describe how you might apply the information in your classroom?  Would you recommend this workshop to a friend?  Suggested topics? Suggested Speakers?  Where did you hear about the workshop? (GTP Website, CIRTL Website, CU Calendar, Lead, Department, GTP Poster in Department)

Most Helpful

Least Helpful

Learning  Measured by statement, “I learned”:  How to integrate research and teaching  How to assess student learning  How to assess teaching  What I need to work on  What I need to articulate  What to keep in mind for proposal defense

Application

Impact & Effectiveness  Most effective workshops were:  Working on a Research Team  Writing Your Research Statement for a Teaching Portfolio  Most impact:  Increased participation in TIGER activities  Content of TIGER 2 was deemed highly relevant  Collaborative workshop environment was highly valued  Self-awareness & professional development was noted by

Potential Change in Evaluation  What changes, if any, might you make to improve or extend your CIRTL-related evaluation activities?  We would like to follow-up with participants via survey or interview.  Build in a focus on inter-disciplinarity.  Add a specific question on “What did you learn today?” to our Feedback Forms

Formative Improvement Possibilities  How might these findings be useful to CIRTL programs?  Emphasize need for shared discussion in workshops.  Emphasize need to highlight professional development aspects of programming.  Emphasize that analyzing your own data according to a rubric pulls out information that you might have not noticed.

Questions?