1. The Carl Wieman Science Education Initiative at the University of British Columbia Achieving Widespread Change in Teaching Practice Sarah Gilbert, Senior Advisor CWSEI, University of British Columbia, Vancouver, Canada I. A little background – what research tells us about how people learn II. Overview of the Science Education Initiative III. Examples from departments

2. cognitive psychology brain research Science classroom studies Major advances past 1-2 decades Consistent picture of How People Learn Show that there are ways to achieve MUCH greater learning than in the traditional lecture

3. Research on Learning Components of effective teaching/learning 1.Motivation relevant/useful/interesting to learner sense that can master subject 2.Connect with prior thinking 3.Apply what is known about memory short term limitations achieving long term retention 4.Explicit authentic practice of expert thinking interactive engagement focused on developing expert thinking skills (scientific reasoning, sense-making, ….) 5.Timely & specific feedback From: Professor, TAs, fellow students, homework, tests, …

4. Some assumptions of the traditional lecture approach: If you TELL the students something, they will learn it (and they cannot learn it if you do not tell them) Reality: Telling works when person is prepared to learn –AFTER students have grappled with a problem –Experts who already have relevant background knowledge & sophisticated mental models of the subject (e.g. a practicing physicist listening to a talk on new developments in their area of expertise) If the students can do the problems (homework, test), they understand the underlying structure and concepts Reality: Many students memorize facts and problem solving procedures & use them without understanding the underlying concepts or reasoning

5. Science Education Initiatives (SEI) CU-SEI started at the University of Colorado (CU) in 2006 Carl Wieman director, Kathy Perkins associate director CWSEI started at the University of British Columbia (UBC) in 2007 Carl Wieman director, Sarah Gilbert associate director Goals: Widespread improvement in science education at UBC & CU by employing research-based teaching practices Develop successful demonstration model(s) to help inform change efforts elsewhere Lessons from history: A few faculty employing research-based teaching practices in a department does not result in widespread change in dept. (spreading a few seeds approach doesn’t work) TRADITION is strong, especially if little incentive to change

6. SEI Underlying Reasoning & Approach Logical unit of change is the Department – it is the cultural unit and the center of the disciplinary expertise Change must be driven by department – need to –Decide what students should learn (with appropriate consultation) –Adopt or develop good measures of learning –Change instructional approaches Significant 1-time investment of resources – Concentrated (~0.5-1.75 M$/dept. over 6 years) to fund change activities; maintenance of change should not require extra resources Departments compete for funding – Criteria: commitment and readiness to undertake widespread sustained effort to improve undergrad education CWSEI central – hands-on funding agent; continued funding depends on progress. Small central office manages program, provides teaching & learning development, and coordinates cross-dept. communication

7. Implementation Within Departments Department CWSEI Director – Faculty member in the department who manages the department’s program Science Teaching & Learning Fellows (STLF) –Temporary positions within department –Content expertise (typically PhD in discipline) + knowledge of learning research & application (CWSEI central provides STLF development on teaching/learning research & application) –Work with faculty to develop learning goals & activities, measure student thinking & learning, … Transfer teaching expertise This is where most of the $$ are spent

8. SEI “Trinity” for each course What should students be able to do? Measure learning Implement research-based teaching methods Iterative process; learn what is challenging for students and adjust teaching approach accordingly

9. Typical new aspects incorporated in courses: Clearly articulated learning goals (not just a list of topics) What the students should be able to do Pre-reading assignments & quizzes Efforts to increase student interest & motivation to learn subject (context, relevance, why useful/interesting, …) Interactive engagement targeted at learning goals Clicker questions and peer discussion – especially in large classes (challenging questions involving scientific reasoning best) In-class group activities – effective even in large (250 student) classes 2-stage exams (individual + group) Homework problems targeted at learning goals Pre-post testing to measure learning, surveys to gauge perceptions about science...

10. UBC: 171 faculty, 169 courses, 139,000 student-cred hrs/yr (>50%) including small program in Chemisty CU: 112 faculty, 71 courses, 53,000 student-cred hrs/yr. (7-75%/dept.)

11. III. Examples from Departments 1.Earth, Ocean and Atmospheric Science Department 2.Critical thinking in introductory physics lab course 3.4 th year modern optics 4.Group work in large classes

12. Earth, Ocean & Atmospheric Sciences Department CWSEI Department Director: Sara Harris 43 faculty (over 90%) have made significant changes to teaching 48 courses significantly changed 85% of student-credit hours changed teaching How did they do it? Department voted to submit proposal Established a Teaching Initiatives Committee to guide program Consistent Strong Leadership and Support Department Head CWSEI Department Director Detailed planning – who would do what when (though flexible) Science Teaching and Learning Fellows – 4 during peak of program; 2 were with the program the whole time (~ 7 years) Established expectations with faculty

14. By end of planning term By end of 1 st teaching term By end of 2 nd teaching term Project ScopeOutlinedRevised In final documentation Course-level learning goals Draft: Involve stakeholders RevisedBroadly accepted Module- or lecture-level learning goals DraftRevised Mapped to course learning goals AssessmentDraft plan Revised plan & materials Optimized plan & materials Teaching Methods (Pedagogy) Draft plan Revised plan & materials Optimized Short summary of structure & rationale Draft In final documentation Materials archived Completed Plan for sustainability Completed Share progress/problemsAnnual or semiannual mini-retreat Establishing Expectations agreement signed by faculty member and Department Head

15. Upper-year Mineral Deposits class 2 one-hour lectures and 1 three-hour lab per week, ~40-50 students Cross section synthesis activity Number of lecture activities Memorization emphasis Develop expert thinking emphasis

16. Example 2: Critical thinking in intro. physics lab course Noticed lack of critical thinking: students typically think of science being done in a linear “Scientific Method”: Conclusions are often just a statement of results If data doesn’t agree with scientific model, blame “human error” and/or inflate the error bars Never iterate HypothesisConduct experimentAnalyze ResultsConclude

17. Quantitative critical thinking in introductory lab course Natasha Holmes & Doug Bonn, UBC Physics & Astronomy Dept. Make a comparison Act on comparison Reflect on comparison Between datasets or between data & model Agree or not? Can measurements be improved? Should model be improved? Implement plan to improve quality of data

18. Example: Measuring the period of a pendulum Students’ designperiod at 10° (s)period at 20° (s) Trial 1: Measure single period 10 times 1.84 ± 0.081.81 ± 0.08 Instructions: Measure the period of the pendulum at 10 and 20 degrees and compare, then improve the measurements Trial 1: Measure single period 10 times 1.84 ± 0.081.81 ± 0.08 Trial 2: Measure 10 periods 6 times 1.823 ± 0.0081.850 ± 0.008

19. 0.00 0.25 0.50 0.75 1.00 Week 2 Proposed only Proposed & Changed Week 16 Week 17 Fraction of Students Control Exp. First Year lab Students Changing Experimental Methods to Improve Data Quality Control Exp. Second Year lab ControlExp. Prompted to improve data Not prompted

20. 0.00 0.25 0.50 0.75 1.00 Week 17 Fraction of Students Students Evaluating Issues with the Scientific Model Control Exp. Identified Identified & Interpreted Week 17: LR circuit Extra internal resistance affecting time constant – resistance plot Not prompted to evaluate whether the physical model needed improvement, but they did. Holmes, Wieman, & Bonn, Teaching Critical Thinking, Proc. Natl. Acad. Sci., V. 112(36), pp. 11199-11204 (2015)

21. Example 3: Fourth year Optics David Jones & Kirk Madison, UBC Physics & Astronomy dept. Professor David Jones taught course 3 times using traditional lecture format DJ converted course to 100% activities in 2010 Professor Kirk Madison taught in 2013 using Jones’ transformed materials, with slight modifications

22. No Lecture! Complete targeted reading Formulate/review activities Actions Preparation StudentsInstructors Introduction (2-3 min) Listen/ask questions on reading Introduce goals of the day Activity (10-15 min) Group work on activities Circulate in class, answer questions & assess students Feedback (5-10 min) Listen/ask questions, provide solutions & reasoning when called on Facilitate class discussion, provide feedback to class

23. Pre-class Reading Read Chapter 2 Purpose: Prepare students for in-class activities; move learning of less complex material out of classroom Can get >80% of students to do pre-reading if: Online or quick in-class quizzes for marks (tangible carrot) Must be targeted and specific: students have a finite time DO NOT repeat material in class! Spend class time on more challenging material, with Professor giving guidance & feedback Heiner et al, Am. J. Phys. 82, 989 (2014) Read Chapter 2.3 pp 49-50, Chapter 2.4 pp 50-55. Make sure you understand the relationship between Eqn. 2.3 and 2.4. If you are having troubling under- standing Eqn. 2.7, that is okay but make sure you understand everything before it.

24. Lecture Notes Converted to Activities Often added bonus activity to keep advanced students engaged

25. Final Exam Scores nearly identical (isomorphic) problems taught by lecture, 1 st instructor, 3rd time teaching course practice & feedback, 1 st instructor practice & feedback 2 nd instructor 1 standard deviation improvement Yr 1 Yr 2 Yr 3 Jones, Madison, Wieman, Transforming a fourth year modern optics course using a deliberate practice framework, Phys Rev ST – Phys Ed Res, V. 11(2), 020108-1-16 (2015) Course materials: http://cwsei.ubc.ca/departments/physics-astro_course-resources.htm

26. Example 4. Group Work in Large Classes (100-300 students) Successfully done in many large classes at UBC and CU Pre-reading assigned to prepare students for activities Combination of worksheet activities and clicker questions work well Nice to have help (undergraduate or graduate teaching assistants) so that have ~1 per 50 students Instructor circulates during activity to provide guidance and also get better understanding of student thinking Checkpoints every ~10 min for longer activities Some lecturing, but mostly after activities, not before

27. Group Work in Large Classes 2 short videos showing worksheet and clicker activities in real classes highlighting key implementation tips http://blogs.ubc.ca/wpvc/video-4-tutoring-large-classes/ Tutoring practices in large classes: 150 students, Climate Change class http://blogs.ubc.ca/wpvc/intro-physics-active-class/ Active Learning with Worksheets: 250 students, Intro Physics

28. Succeeding in a number of departments ~90% of faculty have changed teaching in 2 UBC departments, impacting over 80% of student-credit hours taught. Several other depts. making very good progress. Factors that help: Actively supportive dept. head/chair Broad faculty support & involvement within dept. Reward structure for SEI-related activities Faculty dissatisfied with student learning Respected faculty who support SEI goals Factors that hinder: Departmental culture that  does not respect education research  expects total individual freedom in teaching  blames lack of learning on students  Last minute teaching assignments Conclusions

29. Reflections on the SEI We thought the most compelling factor would be evidence of improved learning, but … Student engagement was more important for most faculty, and research-based teaching methods greatly increase student engagement Change takes time – there were a few faculty who enthusiastically dove into changes, but most needed time to get used to the idea and get comfortable with new approach Science Teaching & Learning Fellows (STLF) – Very effective! Helped faculty get up to speed on new techniques and rationale; better implementation & fewer problems – everyone happier

30. Lots of resources available: www.cwsei.ubc.ca/resources www.cwsei.ubc.ca/resources Instructor guidance Clicker guidance & resources Videos (also http://blogs.ubc.ca/wpvc/ – Evidence-based science education in action)http://blogs.ubc.ca/wpvc/ Tools (teaching practices inventory, Classroom Observation Protocol for Undergraduate STEM (COPUS), … Course transformation resources, … www.cwsei.ubc.ca/SEI_researchwww.cwsei.ubc.ca/SEI_research - SEI papers & presentations www.cwsei.ubc.ca/departmentswww.cwsei.ubc.ca/departments - details on what happened in UBC departments Questions on this talk Discussion question: how to change teaching practices elsewhere with limited funds Questions on this talk Discussion question: how to change teaching practices elsewhere with limited funds

31. Discussion Question How to achieve widespread change in teaching practices with limited funds? Some ideas: 1.Incentives similar to those driving science research 2.Metrics for effective teaching Use of research-based methods (e.g. Teaching Practices Inventory) Validated pre-post tests of learning 3.Strategic approaches (e.g., paired teaching; instructor experienced with active learning techniques + one not) 4.Use materials produced by others (e.g. SEI materials: sei.ubc.ca) 5.Consultant(s) within dept. (faculty member or STLF with content and pedagogy expertise) 6.Group of faculty give feedback to each other 7.Professional societies conduct workshops on effective teaching

32. Extra slides below

33. “There is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things. For the reformer has enemies in all those who profit by the old order, and only lukewarm defenders in all those who would profit by the new order, this lukewarmness arising partly from fear of their adversaries … and partly from the incredulity of mankind, who do not truly believe in anything new until they have had actual experience of it.” – Machiavelli

34. Example 2: Third year Electricity & Magnetism (CU) Dept. Faculty voted to transform upper division courses under SEI Working group of ~10 faculty formed; developed learning goals 1 Prof, Steve Pollock, & Science Teaching Fellow Stephanie Chasteen developed course materials & taught Also developed the Colorado Upper-Division Electrostatics Assessment (CUE)

35. Learning Goals Examples General: Students should be able to ‒ sketch the physical parameters of a problem ‒ choose and apply the problem- solving technique that is appropriate for that problem ‒ articulate expectations for the solution and justify reasonableness of a solution (by checking the symmetry of the solution, looking at limiting or special cases, …) Specific: Students should be able to ‒ sketch the E field inside and outside a dielectric sphere placed in an electric field ‒ choose when to use Biot-Savart Law vs. Ampere’s Law to calculate B fields, and complete the calculation ‒ predict whether a particular magnetization will result in a bound surface and/or volume current

36. In-class Activities Lecture + ~4 clicker questions with peer discussion Students vote individually and then discuss & vote again Follow-up by instructor

37. Homework Aligned with Learning Goals Consider a field (a)Sketch it. (b)Calculate the divergence and curl of this E-field. Test your answers by using the divergence theorem and Stoke’s theorem. (c)Is there a delta function at the origin like there was for a point charge, or not? (d)What are the units of c? (e)What charge distribution would you need to produce an E field like this? Describe it in words as well as formulas. Is it physically realizable?

38. Measures of Learning Colorado Upper-Division Electrostatics Assessment (CUE) Example problem: DO NOT SOLVE the problem, we just want to know: (1) The general strategy (half credit) (2) Why you chose that method (half credit) Q3. Find E or V at the point P, where P is off-axis, at a distance 50a from the cube.

39. CUE Results Standard Lecture-Based Courses (9 courses) vs Physics Education Research- Based Courses (9 courses) at CU and elsewhere (# students = 515)

40. Measures of Learning Traditional Exam Questions PER-C STND-A PER-D Q1 Q2 Q3 Chasteen, Pollock, Pepper, Perkins, Transforming the junior level: Outcomes from instruction and research in E&M, Phys Rev ST – Phys Ed Res, V. 8(2), 020107-1-18 (2012) Course materials: http://www.colorado.edu/physics/EducationIssues/Electrostatics/

41. Physics & Astronomy UBC Physics & Astronomy Department www.cwsei.ubc.ca/departments/physics-astro.htm 26 courses impacted & counting 34 faculty changed teaching (56% of total) 19,500 student x credit hours (75% of total in dept.) CU Physics Department www.colorado.edu/sei/departments/physics.htm had already transformed first year courses prior to SEI 8 courses impacted (middle & upper division courses) 20 faculty changed teaching (40% of total) 1,800 student x credit hours (7% of total in dept.)

42. Life Sciences STLFs: Jared Taylor & Malin Hansen; Dept. Director: George Spiegelman 1 st year Cell Biology Clicker questions & peer discussion, Invention activities, structured problem solving activities, In-class writing assignments, … Coming up with plausible mechanisms for biological process student never encountered before: control struc. Invention Invention problems (tutorial) (in lecture) Published in CBE-Life Sciences Education Jared Taylor, Karen Smith, Adrian van Stolk, & George Spiegelman Selected for inclusion in the 2010 Highlights issue Published in CBE-Life Sciences Education Jared Taylor, Karen Smith, Adrian van Stolk, & George Spiegelman Selected for inclusion in the 2010 Highlights issue

43. cwsei.ubc.ca/departments/earth-ocean_courses.htm