1. An introduction to physics education research, assessments, and modeling in laboratory courses Benjamin Zwickl University of Colorado Boulder & Rochester Institute of Technology (Fall 2013) Lunch Seminar in Fudan Physics Department, May 7, 2013

2. From cavity optomechanics to PER Heather Lewandowski Junior Faculty AMO Physics/JILA HS PhD, Yale Instructor Postdoc BS Jack Harris

3. PART 1: AN INTRODUCTION TO PHYSICS EDUCATION RESEARCH

4. Greetings from the University of Colorado Physics Education Research Group! Right next to the Rocky Mountains

5. A view of campus

6. Physics Education Research @ Colorado Carl Wieman 2001 Nobel Prize in Physics with Eric Cornell (CU) Wolfgang Ketterle (MIT) “Why not try a scientific approach to science education?” The Science Education Initiative at the University of Colorado create and support improvements in science education Research into how students learn

7. A Growing PER Group at University of Colorado Graduate students Ian Her Many Horses Takako Hirokawa George Ortiz Mike Ross Ben Spike Enrique Suarez Ben Van Dusen Bethany Wilcox Rosemary Wulf Staff Shelly Belleau John Blanco Kathy Dessau Jackie Elser Molly Giuliano Kate Kidder Trish Loeblein Chris Malley Susan Nicholson-Dykstra Oliver Nix Jon Olson Emily Quinty Sam Reid Sara Severance Faculty Melissa Dancy Mike Dubson Noah Finkelstein Heather Lewandowski Valerie Otero Robert Parson Kathy Perkins Steven Pollock Carl Wieman (on leave) Post-docs Charles Baily Danny Caballero Stephanie Chasteen Julie Chamberlain Karina Hensberry Katie Hinko Emily Moore Ariel Paul Noah Podolefsky Benjamin Zwickl

8. Jo Handelsman, et al. A scientific approach to teaching A scientific approach includes: 1.Doing research on how students learn 2.Applying those lessons in the classroom. 3.Improving teaching using evidence from your classroom. 4.Using technology effectively. “we [need to] approach the teaching of science like a science.” Wieman, C. Why Not Try a Scientific Approach to Science Education? Change: The Magazine of Higher Learning, (September/October), 9–15. (2007) Carl Wieman

9. What is Physics Education Research (PER)? Theory Application Experiment Docktor, J. L., & Mestre, J. P. (2010). A Synthesis of Discipline-Based Education Research in Physics, A National Research Council Study; McDermott, L. C. (1999). Resource Letter: PER-1: Physics Education Research. American Journal of Physics, 67(9), 755. PER involves: Developing models of learning Creating tools for measurement Gathering evidence of impact Designing new curricula and approaches to teaching

10. Highlights of Physics Education Research Kathy Perkins PhET Director Effective use of technology: PhET simulations > 100 Research-based simulations Free activity guides >100,000,000 downloads phet.colorado.edu

11. Highlights of Physics Education Research Assessment: The Force Concept Inventory (29 multiple-choice questions) Example: Assessing Students’ understanding of Newton’s First Law For this question: Before Instruction: About 45% Correct After Instruction: About 80% Correct For this question: Before Instruction: About 45% Correct After Instruction: About 80% Correct Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force Concept Inventory. The Physics Teacher, 30(3), 141–157.

12. Force Concept Inventory Results Colorado: Tutorials & Learning Assistants Colorado: Traditional recitations R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98). Less Learning More Learning Highlights of Physics Education Research Fraction of courses Red = Traditional lecture & recitation Blue = Research-based curricula & Active learning.

13. Highlights of Physics Education Research Meltzer, D. E., & Thornton, R. K. (2012). Resource Letter ALIP–1: Active-Learning Instruction in Physics. American Journal of Physics, 80(6), 478 Research-based strategies to improve physics learning: 1.Elicit and address students’ difficulties. 2.Engage in a variety of problem solving activities. 3.Work in groups. 4.Explicitly explain reasoning. 5.Use conceptual and quantitative reasoning together. 6.Organize ideas into bigger structures. 7.And more!

14. “Active Learning” in advanced classes Students at University of Colorado Active learning is more effective CU courses transformed using: Explicit Learning Goals Interactive Lectures Transformed Homework problems Common Student Difficulties In-Class Group Activities/Tutorials Concept Tests ("Clicker" questions)

15. “Active Learning” in advanced classes All resources available at: http://www.colorado.edu/sei/departments/physics.htm

16. PART 2: ASSESSMENTS

17. A variety of assessments 1.Introductory conceptual surveys FCI (Force Concept Inventory) FMCE (Force and Motion Conceptual Evaluation) BEMA (Brief Electricity and Magnetism Assessment) 2.Advanced conceptual surveys CUE (Colorado Upper-division Electrostatics Assessment) CCMI (Colorado Classical Mechanics Instrument) 3.Attitudes and beliefs surveys CLASS (Colorado Learning Attitudes about Science Survey) E-CLASS (Experimental Physics CLASS)

18. Attitudes and beliefs: CLASS Students often respond less like experts. Less expert-like students have perform lower. Why do such surveys matter? What is not on these surveys? What are attitudes and beliefs surveys? They ask questions about students views on Understanding physics. Learning physics. Relevance of physics to everyday life. No questions about specific physics concepts. No solving problems.

19. Attitudes and beliefs: CLASS 1. A significant problem in learning physics is being able to memorize all the information I need to know. Strongly Disagree 1 2 3 4 5 Strongly Agree 2. When I am solving a physics problem, I try to decide what would be a reasonable value for the answer. Strongly Disagree 1 2 3 4 5 Strongly Agree 3. I think about the physics I experience in everyday life. Strongly Disagree 1 2 3 4 5 Strongly Agree 41 questions all together, grouped in categories. www.colorado.edu/sei/class Examples of questions from the Colorado Learning Attitudes about Science Survey (CLASS)

20. Example CLASS Results in Chinese High School Zhang, P., & Ding, L. Large-scale survey of Chinese precollege students’ epistemological beliefs about physics: A progression or a regression? Physical Review Special Topics - Physics Education Research, 010110 (2013). 8 CLASS Categories Personal Interest Real world Problem solving general Problem solving confidence Problem solving sophistication Sense-making/effort Conceptual understanding Applied conceptual understanding More expert-like responses Other comparisons include: CLASS scores on traditional and transformed curricula. CLASS scores compared to students’ grades CLASS scores for different majors (physics, engineering, non-science, etc.)

21. 1.A new survey focused on experimental physics 2.Validated for all levels of university students 3.A common evaluation tool than can be applied to a variety of lab experiences across the world. CLASS for Experimental Physics

22. + enjoyment, teamwork, confidence Argumentation DESIGN Statistical analysis for comparison Modeling the measurement system Designing apparatus and experiments Trouble-shooting Test and measurement equipment Computer-aided data analysis Computer-aided measurement Authentic forms in physics Modeling the physical system COMMUNICATION MODELING LEARNING GOALS TECHNICAL LAB SKILLS Use learning goals to develop questions

23. Affect Scientific argumentation Confidence Experimental design Math-Physics-Data connection Modeling the measurement system Physics community Purpose of labs Statistical uncertainty Systematic error Troubleshooting E-CLASS Dimensions

24. Validation 42 interviews with all levels of college students. Students took the survey and then explain how they answered it in an interview format. Students: “What do I think vs. what should I think?” Add: “What would a physicist say?” Students: about lab class or their research lab? Modify: “What would physicists say about their research?” Students: what about theorists? Final: “What would experimental physicists say about their research?” (final)

25. What do you think of when you hear the word scientist? Students: “vague, encompasses geology, biology, chemistry, physics”, “Bill Nye”, “generic person in a lab in a white coat” Can you name any physicists? “Newton, Einstein”, sometimes “Hawking”, “Michio Kaku”, “Archimedes”, “my professor”, a few mentions of “that guy who won the Nobel Prize”/“Wineland” What do you think the difference is between a physicist and an experimental physicist? “A physicist draws pictures on paper”, “A physicist is a paper and pencil physicist”, “I don’t know”, “An experimental physicist is a physicist that does experiments?” Were the questions asking “What would an experimental physicist say about their research?” awkward to you? “No, I think they’re still pretty applicable to what I do.” Scientists vs. Physicist vs. Experimental Physicist

26. Paired Questions Actionable Evidence for Instructor 3. Does this practice help to earn a good grade? 1. Students’ personal attitudes and beliefs 2. Students’ view of experts Core Statement: ( e.g., Whenever I use a new measurement tool, I try to understand its performance limitations.) Pre and Post Post only E-CLASS Design

27. CLASS for Experimental Physics (E-CLASS) Pre: 30 paired questions. Post: Additional 23 tinyurl.com/E-CLASS-Sp13-Post Examples of questions from the E-CLASS 1. When doing an experiment, I try to understand how the experimental setup works. Strongly Disagree 1 2 3 4 5 Strongly Agree What do YOU think? What would experimental physicists say about their research? 2. If I wanted to, I think I could be good at doing research. Strongly Disagree 1 2 3 4 5 Strongly Agree What do YOU think? What would experimental physicists say about their research?

28. Example E-CLASS Results More expert-like Fall 2012 results from an introductory lab course for scientists Circle = Pre, Arrow = Pre/Post Shift Results for all students in similar classes

29. Example E-CLASS Results More expert-like Fall 2012 results from an introductory lab course for scientists

30. Example E-CLASS Results Fall 2012 results from an introductory lab course for scientists This class only had a small correlation between importance for earning a good grade and the shift in attitudes. Positive shifts Negative shifts Number identifies the question

31. E-CLASS Benjamin M. Zwickl, Noah Finkelstein, and H. J. Lewandowski, PERC Proceedings 2012 All 6 faculty chose “agree” or “strongly agree.”

32. MODELING IN THE PHYSICS LABORATORY

33. Modeling emerged as a key lab learning goal B. Zwickl, N. Finkelstein, and H. J. Lewandowski, Am. J. Phys. 81, 1, 63-70 (2013) ModelingDesign Technical lab skills Communication 22 faculty Literature Community Personal experience Many Inputs LEARNING GOALS

34. Defining expertise through learning goals Designing apparatus and experiments Trouble- shooting Test and measurement equipment Computer-aided data analysis Computer-aided measurement Argumentation Authentic forms in physics Statistical analysis for comparison Modeling the measurement system Modeling the physical system DESIGN COMMUNICATION MODELING LEARNING GOALS TECHNICAL LAB SKILLS More info at: http://tinyurl.com/Advanced-Lab-LGshttp://tinyurl.com/Advanced-Lab-LGs B. Zwickl, N. Finkelstein, and H. J. Lewandowski, Am. J. Phys. 81, 1, 63-70 (2013)

35. Model: An abstract version of a real system that is (1)Simplified (2)Predictive (3)has specified limits to its validity Modeling: Developing, testing and refining models Real Abstract 1)Simplified 2)Predictive 3)Limited validity A model of a pendulum

36. 1.Combine quantitative predictive models with quantitative measurement.  The models are as important as the data. 2.Never forget the basic physics ideas and principles that govern the system.  It is never as simple as matching results to an equation. 3.Recognize the idealizations and simplifications that limit the model’s accuracy.  Is it valid to apply the model to the experimental apparatus? Modeling is both a process of experimental physics and a way to “understand” a complex experiment.

37. 4.Measurement devices are not “black boxes."  Understand how the measurement tools work. 5.All sources of error and uncertainty are considered.  Random uncertainty AND systematic error. 6.Experiments are typically iterative and systematic.  Models and apparatus are refined in many stages. Modeling is both a process of experimental physics and a way to “understand” a complex experiment.

38. Specific situation Idealizations? Unknown parameters? Principles Approximations? Physical system model abstraction predictions Principles Approximations? Specific situation Idealizations? Unknown parameters? Data Measurement model Results with uncertainties Real-world physical system Measurement probes How can I get better agreement? Stop Yes No Improve the measurement model Improve the physical model Comparison. Is the agreement good enough? 1. All parts of apparatus included 2. Data and theory compared 3. Iterative cycle A modeling framework for labs

39. ninamccurdy.com Modeling the simple pendulum predictions Are the idealizations valid? What predictions are most useful?

40. Modeling the photogate Simple model for timing gate: Light from IR LED shoots across to photodiode. Gate starts/stops when light blocked Point emitter & point detector

41. Comparison. Is the agreement good enough? Comparison between data and predictions Period (s) Data from photogate Predictions from model of pendulum

42. Model refinement as inquiry RESOLVE THE ERROR: Identify systematic error sources in the: 1.pendulum model & 2.photogate Predict the effect of the systematic error source Experimentally test a systematic error source Refine pendulum model & calibrate photogate Redesign apparatus Systematic Error is just an incomplete model.

43. EXAMPLES OF USING MODELING TO UNDERSTAND COMPLEX EXPERIMENTS Schematics from TeachSpin PS1-A Lab Manual

44. A pulsed NMR experiment is used to measure T2 (spin-spin relaxation time) Good data and a fundamental misunderstanding in the same lab report. Atomic scale dynamics. Invisible fluctuating B-fields Faster than a blink of an eye Sophisticated physics ideas. Sophisticated apparatus. Sophisticated Understanding? Atomic scale dynamics. Invisible fluctuating B-fields Faster than a blink of an eye Sophisticated physics ideas. Sophisticated apparatus. Sophisticated Understanding? (Graph and text from student lab report.) The challenge: Doing without understanding

45. Schematics from TeachSpin PS1-A Lab Manual abstraction Real-world physical system Hydrogen nuclei in sample vial Transmitter coils and pulsed RF drive Constant magnetic field Principles How do proton spins change in time in response to the magnetic fields? Physical system model Mathematica code to numerically solve equations Specific situation Initial state Final state Student’s computer simulation Modeling Nuclear Magnetic Resonance

46. Previous polarization of light lab Qualitative prompts: Examine… Observe… What happens? No predictive model No interpretation of data

47. NEW: Transformed polarization of light lab Compare quantitative data with quantitative models. Refining idealized models. Introduce Jones formalism for polarization Elliptically polarized light: Quarter-wave plate

48. Future directions for modeling 1.Investigate how professional scientists use models To communicate ideas in published journal articles. In the laboratory to understand, design, and troubleshoot experiments. 2.Develop assessments of students’ modeling proficiency 3.Evaluate different styles of laboratory instruction to see how to best develop modeling abilities. The end. Thank you!