1. Teaching Methods Related to Student Progress in Lower-level STEM Courses Steve Benton, Ph.D. Senior Research Officer IDEA Emeritus Professor, Kansas State University steve@IDEAedu.org

2.  Role of student ratings  IDEA system  Description of sample  Learning outcomes in STEM classes  Teaching styles/methods in STEM classes  Student motivation in STEM classes Session Overview

3. Measuring Teaching Effectiveness: Include Multiple forms of Assessment Instructional Delivery Instructional Assessment Course Management Content Expertise Instructional Design Learning Outcomes Balanced Plan for Summative Evaluation

4. Measuring Teaching Effectiveness: Include Multiple Feedback Sources Instructional Delivery (Students) Instructional Assessment (Peers) Course Management (Administrator) Content Expertise (Peers) Instructional Design (Peers) Learning Outcomes (Students, Peers) Balanced Plan for Summative Evaluation

5. Purpose of IDEA Individual Development Educational Assessment Teaching Improvement Faculty Evaluation Curriculum Review Program Assessment Accreditation

6. Underlying Philosophy of IDEA Teaching effectiveness is determined primarily by students’ progress on the types of learning the instructor targets.

7. Faculty Information Form

8. Faculty Information Form (FIF)

9. Learning Objective CategoryItem Number Basic Cognitive Background1, 2 Applications of Learning3, 4 Expressiveness6, 8 Intellectual Development7, 10, 11 Lifelong Learning9, 12 Team Skills5

10. Teaching Style CategoryItem Number Stimulating Student Interest4, 8, 13, 15 Fostering Student Collaboration5, 16, 18 Establishing Rapport1, 2, 7, 20 Encouraging Student Involvement 9, 11, 14,19 Structuring Classroom Experiences 3, 6, 10, 12, 17

11.  Classes using IDEA in academic years 2009-2013  STEM Classes - 171,306  Science – 82,200  Computer science – 21,188  Engineering – 12,444  Math – 55,474  Non-STEM Classes – 810,277 Description of Sample

12.  Which learning objectives do faculty select in lower-level (first-year/sophomore) STEM courses? THINK-PAIR-SHARE

13. Learning Objectives Selected in Lower-level STEM versus non-STEM Classes

14. Learning Objectives Selected in Lower-level Science, Math, Engineering, and Computer Science Classes

15.  On which learning objectives do students in lower-level STEM courses report the most progress? THINK-PAIR-SHARE

16. Student Progress in Lower-level STEM versus non-STEM Classes

17. Student Progress in Lower-level Science, Math, Engineering, and Computer Science Classes

18.  Which teaching styles do students observe most frequently in lower-level STEM courses? THINK-PAIR-SHARE

19. Teaching Styles Emphasized in Lower-level STEM and non-STEM Classes

20. Teaching Styles Emphasized in Lower-level Science, Math, Engineering, and Computer Science Classes

21.  Which individual teaching methods are most important in lower-level STEM courses? Teaching Methods Associated with Student Progress on Relevant Objectives

22.  Used BMA to test multiple models  Only included classes where instructor rated objective as relevant  Compared models in first-year/sophomore STEM versus all other classes  Compared models between first-year/sophomore STEM general education classes versus majors Bayesian Model Averaging (BMA)

23. Explained Material Clearly/Concisely Especially important for: Gaining factual knowledge Learning fundamental principles Developing skills (gen. ed.) Problem solving Finding/using resources (gen. ed.) Critical thinking

24. Helped Students Answer Own Questions Especially important for: Developing skills and competencies

25. Inspired Students to Achieve Challenging Goals Especially important for: Critical thinking

26. Asked Students to Share Experiences Especially important for: Finding and using resources

27. Required Original/Creative Thinking Especially important for: Problem solving Finding and using resources

28.  “Explained course material clearly and concisely”  Objectives 1, 2, 3, 11 (knowledge, problem solving, critical thinking)  “Found ways to help students answer their own questions”  Objective 4 (professional skills/competencies)  “Inspired students to set and achieve challenging goals”  Objective 11 (critical thinking)  “Asked students to share ideas and experiences”  Objective 9 (information literacy)  “Gave projects, tests, or assignments that required original or creative thinking”  Objectives 3, 9 (problem solving, information literacy) Key Teaching Methods in Lower-Level STEM Courses

29.  Especially important for gen. ed. students in:  Acquiring an interest in learning more Stimulated Students to Intellectual Effort

30. POD-IDEA Notes IDEA Website

31. POD-IDEA Notes

33.  What percent of lower-level students express a strong desire to take their current STEM course? Students’ Desire to Take the Course

34. Lower-level Students’ Desire to Take STEM versus non-STEM Courses

35. Lower-level Students’ Desire to Take Science, Math, Engineering, and Computer Science Courses

36.  STEM instructors emphasize basic cognitive information and applications of knowledge  Least emphasis on team skills, expressiveness, intellectual development Summary: Learning Outcomes Emphasized in Lower-level STEM Courses

37.  Student progress highest in basic cognitive information/applications  Student progress lowest in expressiveness and intellectual development Summary: Student Progress in Lower- level STEM Courses

38.  Frequent STEM teaching styles: stimulating interest, establishing rapport, and establishing course structure  Least frequent STEM teaching style: fostering collaboration  Key teaching method: Clarity and conciseness Summary: Teaching Methods in Lower- level STEM Courses

39.  STEM students’ desire to take course comparable to non-STEM  Students’ desire to take course lowest in math Summary: Student Motivation in Lower- level STEM Courses

40. Which results confirmed what you might think about STEM courses? Which results were surprising? What additional insights or questions do you have? Discussion

41. Questions? Steve Benton, Ph.D. Senior Research Officer IDEA