1. Effective Evaluation Of Teaching And Learning STEPHEN BRADFORTH ADAM LEIBOVICH WILL DICHTEL TOBY SMITH ANDREW FEIG JIM MARTIN FUNDED BY RCSA AS PART OF THE COTRELL SCHOLARS COLLABORATIVE

2. Effective Evaluation of Teaching and Learning (EETL) - Searching for New Approaches to R1 STEM Teaching Evaluation  Premise: student evaluation is not an effective way to promote excellent teaching  Seek: new but easily implementable metrics need to be identified (menu of choices appropriate to different institutions)  Strategy: Align with AAU Undergraduate STEM Education Initiative  Output: workshop, white paper  Project funded: Aug. 2012 2 year scope

3. Project Progress  Conference call to establish detailed work plan and milestones  Involvement of AAU fellow, Kristen Hodge  Each investigator made contact with leaders in area of Teaching Evaluation  Meetings with campus organizations such as Centers for Excellence in Teaching, Education Faculty, Administrators responsible for current evaluation  Creation of a primer on previous projects  1.75 day meeting of investigators in Washington, DC in early November

4. Consultants  Diane O’Dowd (HHMI Professors Group)  James Fairweather  Linda Slakey  Noah Finkelstein  Charles Henderson  Jay Labov  Matthew Hora  Robert Mathieu  Gisele Ragusa

5. Project Progress DC meeting established:  Narrowed down list of effective approaches that project would like to learn more about  Detailed survey prepared addressed at CS Scholars, CCSA awardees and department chairs  Timeline for workshop, grant applications, paper submission: possible locations, dates, grid of invitees and promotion

6. Project Progress December-January: Finalization of Survey Institution Review Board submission for authorization of survey (future use for publication requires human-subject approval) Survey sent out mid-January Collection and analysis of results

7. Survey Asked about:  Current Methods Used to Gauge Teaching Effectiveness, under evaluation of student learning and Faculty teaching effectiveness and Academic Program Effectiveness  Perception of Desirability of Several Best Practices for Assessment of Teaching Effectiveness  Data can be examined by discipline, faculty rank and institution type, and size of academic program.

8. Response rate (1 week out)  Chairs (124 surveyed): 30%  CCSA (776 surveyed): 16%  CS (243 surveyed): 6% response  TOTAL RESPONSES: 215

9. Current Practice: student learning assessments used 4: Used Always 3: Used Often 2: Used Sometimes 1: Used Rarely 0: Never Used

10. Current Practice: student learning assessments used 4: Used Always 3: Used Often 2: Used Sometimes 1: Used Rarely 0: Never Used

11. Current Practice: methodologies to evaluate faculty teaching 4: Used Always 3: Used Often 2: Used Sometimes 1: Used Rarely 0: Never Used

12. Current Practice: methodologies to evaluate faculty teaching 4: Used Always 3: Used Often 2: Used Sometimes 1: Used Rarely 0: Never Used

13. Current Practice: assessing majors program for student learning 4: Used Always 3: Used Often 2: Used Sometimes 1: Used Rarely 0: Never Used

14. Current Practice: assessing majors program for student learning 4: Used Always 3: Used Often 2: Used Sometimes 1: Used Rarely 0: Never Used

15. Desirability of Best Practices for Faculty teaching assessment

16. Tracking Alumni Outcomes is Desirable…

17. Tracking Alumni Job Placements Relative Survey Response … but is performed infrequently! What methodologies are used to determine the effectiveness of the overall curriculum and the student learning outcomes from the curriculum?

18. Desirability of Best Practices for Faculty teaching assessment

19. EETL project Connections with  AAU initiative  CS Collaborative Think and Do Tank  HHMI Professor initiative

20. Next steps  Add investigator from Physics (as defined by survey responses or our wish-list  Follow up survey participants who have particular interesting suggestions  Submit proposal to NSF-DUE for additional workshop funding to expand reach  Finalize workshop, issue invites  Complete white paper after workshop

21. Tentative Structure of Workshop Week 2 or 3 of Nov 2013 In Washington DC at AAAS / AAU or ACS HQ