Using research to improve physics instruction copies of slides to be available Carl Wieman Stanford University Department of Physics and Grad School of Education 1. Told can’t give same talk as 2 years ago, you all remember that perfectly. (Lots of PER results.) 2. Can’t talk about labs, Natasha Holmes doing. Underlying principles of learning from research and how apply to good research-based physics teaching. Cognitive psychology meets physics. Very informal-interrupt. Will stop in 75 minutes.

How learning happens old/current model new research-based view brain changeable ~ same knowledge transformation soaks in, varies with brain Primary educational focus of universities: contents of knowledge “soup” admitting best brains blaming K-12 for deficiencies Changes in response to intense thinking. Improved capabilities. Very dependent on teaching methods, topics not so much.

Strenuous extended mental effort. New connections, new neurons. Develops over time—very much like building muscle. Expertise lies in rewired brain.

Most university instructors and administrators don’t know about advances in sci. ed research but growing recognition: US National Acad. of Sciences (2012) PCAST Report to President (2012) Calling on universities to adopt Amer. Assoc. of Universities (60 top N. Amer. Univ.’s—Stanford, Harvard, Yale, MIT, U. Cal, …) Pre 2011-- “Teaching? We do that?” 2017 Statement by President of AAU-- “We cannot condone poor teaching of introductory STEM courses … simply because a professor, department and/or institution fails to recognize and accept that there are, in fact, more effective ways to teach. Failing to implement evidence-based teaching practices in the classroom must be viewed as irresponsible, an abrogation of fulfilling our collective mission ….”

II. What research says are important factors for learning to think like expert (= physicist) through practice with feedback Prior knowledge & experience Motivation Disciplinary expertise Student variation Brain constraints Social learning Tasks/questions + deliverables Implementation complexities in application—teaching expertise all aspects of course—class, homework, exams group discussion activity—What am I leaving out?

Practicing expert thinking with good feedback Learning through practice with feedback Prior knowledge & experience Motivation Disciplinary expertise Student variation Brain constraints Social learning Tasks/questions + deliverables Implementation Proper brain “exercise” Practicing expert thinking with good feedback (= timely, specific, actionable--use to improve) simple right wrong or delayed have little value

II. What research says are important factors for learning to think like expert (= physicist) through practice with feedback Prior knowledge & experience Motivation Disciplinary expertise Student variation Brain constraints Social learning Tasks/questions + deliverables Implementation complexities in application—teaching expertise

Learning expert thinking*-- Practicing desired thinking skills (expertise) brain “exercise” Decisions when solving science problem with incomplete information Wieman group-ongoing study of expert problem solving across sci., eng, & medicine. Remarkably similar set of decisions.

Expert Problem Solving Process Making particular decisions without complete information (pg 1 of 3) The order is roughly chronological but can often involve reordering and iteration. Models play an essential role in most steps. Evaluation of state of the field • What are important questions or problems? Where is the field heading? Which problems/designs feasible to tackle? Articulation of problem • Form abstract representation/model of core ideas. • Write summary of problem, often in terms of specific questions and hypotheses • Identify or create possible models that are relevant • Identify most important variables • Identify areas of particular difficulty/uncertainty • Delineate goals (in engineering, design criteria). Determine scope of the problem. Refinement of the question/problem into a tractable research/design/construction problem • Identify important underlying structural features of problem, decide what concepts apply and are relevant. Decide which are not. Look at problem from different perspectives • Identify similar problems have seen before, aspects of their solutions that might be useful in present context. (Includes reviewing literature and reflecting on experience). • Decide what approximations or simplifications are appropriate, and test against established criteria. • Evaluate possible models: identify what models are appropriate (based on important features of both the problem and the models) to use in the solution process. This includes mathematical representations. • Identify a small set of potential solutions or solution methods. (Based on experience and fit some criteria for solution to a general problem having key identified features.) • Set priorities: consider optimization and trade-offs, identify most difficult or important/uncertain aspect to focus on, back of envelope estimates for efficient planning • Reflect on solver’s knowledge-- is it sufficient to carry through steps with confidence? • Evaluate resources that are available and requirements • Determine difficulty and risks of problem – decide whether it’s solvable/worth pursuing given the difficulties and risks Reflection on information available and needed • Decide what information is needed to solve the problem & what info is available. Sufficient to test potential models and select between? • Evaluate quality, validity, applicability of available information. Decide what and how to get information required.

Learning expert thinking*-- Practicing desired thinking skills (expertise) brain “exercise” Decisions when solving science problem Decide: how to represent graphically Decide what concepts/models relevant Decide: What information relevant, irrelevant, needed. Decide: what approximations are appropriate. ‘’ : potential solution method(s) to pursue. .... ‘’ : if solution/conclusion make sense- how to test. Learner must practice making decisions. Process & content. Large difference between making decision (good or not) vs. being told outcome to use. (Holmes, Keep, Wieman, TBP) Need scaffolded decision making. Decisions to make is too hard. * “Deliberate Practice”, A. Ericsson research. See “Peak;…” by Ericsson for accurate, readable summary

What research says are important factors for effective learning. through practice with feedback Prior knowledge & experience Motivation Disciplinary expertise Student variation Brain constraints Social learning Tasks/questions + deliverables Implementation complexities in application—teaching expertise

Expert thinking to practice, activity design*- (prior knowledge) Difficult but doable tasks/questions. Need to measure knowledge initially, and ongoing informally. Adjust instruction accordingly. Beliefs about learning physics and topics also important. (Motivation) 1) Interesting & relevant (choice of context, problem driven). 2) Belief that can master the material Messages teachers sends through what they say, and exams and how graded. 3) Some (small) control of learning process.

Learning through practice with feedback Prior knowledge & experience Motivation Disciplinary expertise Student variation Brain constraints Social learning Tasks/questions + deliverables Implementation

Expert thinking to practice, activity design*- Brain constraints 1) short-term (class period) working memory limit 5-7 new items. More it has, more bogs down. Hurts learning. Interesting little digression or joke actually hurts. Jargon, neat looking pictures (with nonessential info) all hurt. Checking facebook, reading email, sending texts—split attention, disaster for learning!

Reducing unnecessary demands on working memory in class Targeted pre-class reading with short online quiz Eliminate non-essential jargon and information Make lecture/class organization explicit. Explicitly connect pieces

Expert thinking to practice, activity design*- Brain constraints 1) short-term (class period) working memory limit 5-7 new items. 2) long term memory– biggest problem is recall after learning additional stuff. “Interference”. Need repeated recall & application of old material intermingled with learning new. Good teaching maximizes efficiency of learning, but will be an ultimate limit on rate/time.

Learning through practice with feedback Prior knowledge & experience Motivation Disciplinary expertise Student variation Brain constraints Social learning Tasks/questions + deliverables Implementation

Implementation— design good tasks as above but with deliverables, individual & group (to define task, and in class you use to guide feedback) Social learning (working in groups, in class 3-4) Feedback, avoids getting stuck so more efficient, do more Teaching/explaining triggers unique cognitive process learning Models discourse & practice of discipline Plus: enormously useful as a teacher to listen in on student conversations! Enables good feedback.

Carefully prepared activity per above Actions Students Instructors Preparation Complete targeted reading Formulate/review activities Introduction (2-3 min) Listen/ask questions on reading Introduce goals of the day Activity (10-15 min) Group work on activities Circulate in class, answer questions & assess students Feedback (5-10 min) Listen/ask questions, provide solutions & reasoning when called on Facilitate class discussion, provide feedback to class

Organization of how a topic is presented Organization of course Examples of common teaching practices, and how research on learning shows you their faults. Organization of how a topic is presented Organization of course Feedback on answers When instructor talking Who making decisions Design of homework and exam problems

Organization of how a topic is presented. Standard teaching practice: Begin with formalism, definitions, principles, equa’s, then use to solve problems. What could be wrong with this? Discuss with neighbors, then I will call on.

Organization of how a topic is presented. Standard teaching practice: Begin with formalism, definitions, principles, equa’s, then use to solve problems. What is wrong with this? Not motivating–no idea as to relevance or value, b) Bigger issue– poor knowledge organization. Random separate pieces of information. Overwhelms working memory, cuts off processing. Expert knowledge organization-- material as tools to solve particular types of problems. Recognizing key features of context and how link to particular concepts, strategies, knowledge to solve.

A better way to present material— “Here is a meaningful problem we want to solve.” “Try to solve” (and in process notice key features of context & concepts—basic organizational structure). Now that they are prepared to learn--“Here are tools (formalism and procedures) to help you solve.” More motivating, better mental organization & links, less cognitive demand = more learning. “A time for telling” Schwartz & Bransford (UW), Cog. and Inst. (1998), Telling after preparation  x10 learning of telling before, and better transfer to new problems.

2. Organization of course and exams. Standard teaching practice--chap. 3 material--Lectures, HW, exam ch. 3, done. chap. 4 ditto, done. What could be wrong with this?

2. Organization of course and exams. Standard teaching practice--chap. 3 material--Lectures, HW, exam ch. 3, done. chap. 4 ditto, done. Material organized in brain chronologically by chap. But real problems not labelled with chap. #! Expertise— deciding when material applies and how to use it. Solution– regularly return to earlier material, how related to current, when does and does not apply.

B. T. 3. Feedback on answers Standard practice– student has something wrong. Feedback—”That is wrong, here is correct solution.” Why bad?

B. T. 3. Feedback on answers Standard practice– student has something wrong. Feedback—”That is wrong, here is correct solution.” Why bad? Research on feedback—simple right-wrong with correct answer ~ zero value. Learning happens when feedback timely and specific. What thinking was incorrect and why, and how to improve.

B. T. 4. Instructor talking. Standard teaching practice— instructor spends 90+% talking while students listen passively, maybe take notes, ask occasional question. Why bad—student brain is not practicing expert thinking– essential for “brain exercise” & rewiring. Learning from expert feedback-telling highly effective, but only if brain prepared first. If students struggle with problem first, then told, x10 learning compared to telling, then practice. (Schwartz & Bransford)

B. T. 5. Making decisions Common teaching practice– “It is much more efficient if I, the expert, just tell them what answer/ decision should be, rather than having them spending time figuring out individually or by talking with peers.” Why bad?

B. T. 5. Making decisions Common teaching practice– “It is much more efficient if I, the expert, just tell them what answer/ decision should be, rather than having them spending time figuring out individually or by talking with peers.” No, not exercising the decision making neurons. Like instructions when driving. Telling you to think about lifting heavy weight over and over to get strong.

Histogram of test scores ave 41 ± 1 % 74 ± 1 % guess Learning from lecture very small. Clear improvement for entire student population.

B. T. 6. Designing homework & exam problems What expertise being practiced and assessed? Provide all information needed, and only that information, to solve the problem Say what to neglect Possible to solve quickly and easily by plugging into equation/procedure from that week Only call for use of one representation Not ask why answer reasonable, or justify decisions Components of expert thinking: recognizing relevant & irrelevant information select and justify simplifying assumptions concepts and models + selection criteria moving between specialized representations (graphs, equations, physical motions, etc.) Testing & justifying if answer/conclusion reasonable How to improve? Don’t do the bad stuff.

“But traditional lectures can’t be as bad as you claim “But traditional lectures can’t be as bad as you claim. Look at us university professors (or Nobel Prize winners) who were taught by traditional lectures.” Bloodletting was the medical treatment of choice for ~ 2000 years, based on exactly the same logic. Need proper comparison group. (science) If better teaching, would have been more successful, along with many other students.

Expertise in teaching. Just gave you lecture on it. Conclusion: Expertise in teaching. Just gave you lecture on it. How much learned?? (some because pretty expert...) To become expert, need to review slides and read references, practice making teaching decisions. Measure learning results to get feedback. Good References: S. Ambrose et. al. “How Learning works” D. Schwartz et. al. “The ABCs of how we learn” Ericsson & Pool, “Peak:...” Wieman, “Improving How Universities Teach Science” cwsei.ubc.ca-- resources (implementing best teaching methods), references, effective clicker use booklet and videos slides will be available

~ 30 extras below

III. How to apply in classroom? practicing thinking with good feedback Example– large intro physics class (similar chem, bio, comp sci, ...) Teaching about electric current & voltage 1. Preclass assignment--Read pages on electric current. Learn basic facts and terminology without wasting class time. Short online quiz to check/reward. 2. Class starts with question:

When switch is closed, bulb 2 will a. stay same brightness, b. get brighter c. get dimmer, d. go out. 2 1 3 answer & reasoning 3. Individual answer with clicker Jane Smith chose a. 4. Discuss with “consensus group”, revote. Different, enhanced cognitive processing = learning Instructor listening in! What aspects of student thinking like physicist, what not?

5. Demonstrate/show result 6. Instructor follow up summary– feedback on which models & which reasoning was correct, & which incorrect and why. Many student questions. Students practicing thinking like physicists-- (applying, testing conceptual models, critiquing reasoning...) Feedback that improves thinking—other students, informed instructor, demo

Learning in the in classroom* Comparing the learning in two ~identical sections UBC 1st year college physics. 270 students each. Control--standard lecture class– highly experienced Prof with good student ratings. Experiment–- new physics Ph. D. trained in principles & methods of research-based teaching. (electricity example) review quickly They agreed on: Same learning objectives Same class time (3 hours, 1 week) Same exam (jointly prepared)- start of next class *Deslauriers, Schelew, Wieman, Sci. Mag. May 13, ‘11

Advanced courses U. Col, UBC, & Stanford 2nd -4th year physics Design and implementation: Jones, Madison, Wieman, Transforming a fourth year modern optics course using a deliberate practice framework, Phys Rev ST – Phys Ed Res, V. 11(2), 020108-1-16 (2015) Worksheets

nearly identical problems Final Exam Scores nearly identical problems practice & feedback 2nd instructor practice & feedback, 1st instructor 1 standard deviation improvement taught by lecture, 1st instructor, 3rd time teaching course Yr 1 Yr 2 Yr 3 Jones, Madison, Wieman, Transforming a fourth year modern optics course using a deliberate practice framework, Phys Rev ST – Phys Ed Res, V. 11(2), 020108-1-16 (2015)

Transforming teaching of Stanford physics majors 8 physics courses 2nd-4th year, seven faculty, ‘15-’17 Attendance up from 50-60% to ~95% for all. Student anonymous comments: 90% positive, only 4% negative (mostly VERY positive, “All physics courses should be taught this way!”) All the faculty greatly preferred to lecturing.

IV. Institutional Change How to make the norm for university teaching?

Large scale change is possible. For administrators: What universities and departments can do. Experiment on large scale change of teaching. Changed teaching of ~250 science instructors & 200,000 credit hrs/yr UBC & U. Colorado Important results: Large scale change is possible. Need to recognize and support teaching expertise. When faculty learn how to teach this way (~50 hrs) they greatly prefer it. Need better way to evaluate teaching- ( Extent using research-based practices?)

Necessary 1st step-- better evaluation of teaching “A better way to evaluate undergraduate science teaching” Change Magazine, Jan-Feb. 2015, Carl Wieman Requirements: measures what leads to most learning equally valid/fair for use in all courses actionable-- how to improve, & measures when do is practical to use routinely student course evaluations do only #4 Better way–characterize the practices used in teaching a course, extent of use of research-based methods. 5-10 min/course “Teaching Practices Inventory” http://www.cwsei.ubc.ca/resources/TeachingPracticesInventory.htm

Activities that develop knowledge organization structure. “ A time for telling” Schwartz and Bransford, Cognition and Instruction (1998) People learn from telling, but only if well-prepared to learn. Activities that develop knowledge organization structure. Students analyzed contrasting cases recognize key features Predicting results of novel experiment

Aren’t you just coddling the students? Emphasis on motivating students Providing engaging activities and talking in class Failing half as many “Student-centered” instruction Aren’t you just coddling the students? Like coddling basketball players by having them run up and down court, instead of sitting listening? Serious learning is inherently hard work Solving hard problems, justifying answers—much harder, much more effort than just listening. But also more rewarding (if understand value & what accomplished)--motivation

A few final thoughts— 1. Lots of data for college level, does it apply to K-12? There is some data and it matches. Harder to get good data, but cognitive psych says principles are the same. 2. Isn’t this just “hands-on”/experiential/inquiry learning? No. Is practicing thinking like scientist with feedback. Hands-on may involve those same cognitive processes, but often does not.

Effective teacher— Designing suitable practice tasks Providing timely guiding feedback Motivating (“cognitive coach”) requires expertise in the content!

2. Limits on short-term working memory--best established, most ignored result from cog. science Working memory capacity VERY LIMITED! (remember & process 5-7 distinct new items) MUCH less than in typical lecture slides to be provided Mr Anderson, May I be excused? My brain is full.

Pre-class Reading Purpose: Prepare students for in-class activities; move learning of less complex material out of classroom Spend class time on more challenging material, with Prof giving guidance & feedback Can get >80% of students to do pre-reading if: Online or quick in-class quizzes for marks (tangible reward) Must be targeted and specific: students have limited time DO NOT repeat material in class! Heiner et al, Am. J. Phys. 82, 989 (2014)

How it is possible to cover as much material? (if worrying about covering material not developing students expert thinking skills, focusing on wrong thing, but…) transfers information gathering outside of class, avoids wasting time covering material that students already know Advanced courses-- often cover more Intro courses, can cover the same amount. But typically cut back by ~20%, as faculty understand better what is reasonable to learn.

clickers*-- Not automatically helpful-- give accountability, anonymity, fast response Used/perceived as expensive attendance and testing device little benefit, student resentment. Used/perceived to enhance engagement, communication, and learning  transformative challenging questions-- concepts student-student discussion (“peer instruction”) & responses (learning and feedback) follow up instructor discussion- timely specific feedback minimal but nonzero grade impact Putting research into practice in classroom (and homework, and exams, etc.) Knowing what students are thinking in classroom and connecting to that. Enhanced communication and feedback. Not about more quizes/test… not just because more alert. Clickers provide powerful psychological combination: personal accountability/commitment peer anonymity 1. Feedback to instructor. 2. Feedback to students. 3. Students intellectually active-- a dialogue. Using clickers for max benefits communication system Class built around series of questions to students: challenging concepts or applications, predictions or explanations of demonstration experiments, ... Small group discussion ("peer instruct."), consensus answers Explicit focus on novice/expert views, reasoning, problem solving. Collaborative problem solving/scientific discourse, self-monitoring. *An instructor's guide to the effective use of personal response systems ("clickers") in teaching-- www.cwsei.ubc.ca

long term retention Retention curves measured in Bus’s Sch’l course. UBC physics data on factual material, also rapid drop but pedagogy dependent. (in prog.) cw

Institutionalizing improved research-based teaching practices. (From bloodletting to antibiotics) Goal of Univ. of Brit. Col. CW Science Education Initiative (CWSEI.ubc.ca) & Univ. of Col. Sci. Ed. Init. Departmental level, widespread sustained change at major research universities scientific approach to teaching, all undergrad courses Departments selected competitively Substantial one-time $$$ and guidance Extensive development of educational materials, assessment tools, data, etc. Available on web. Visitors program