Rethinking our goals: What will our students remember when they forget everything? Eugenia Etkina Graduate School of Education Department of Learning and Teaching Rutgers University AAPT-PERC 2010 Portland, Oregon

Former and present members of the Rutgers PER group who contributed to this work A. Van Heuvelen, S. Brahmia, M. Ruibal - Villasenor, M. Gentile, G. Suran D. Brookes, S. Murthy, D. Rosengrant, A. Warren, A. Karelina, V. Shekoyan

What is physics? Body of knowledge A process through which this knowledge is constructed and constantly refined

What do we often focus on when we think of learning or teaching physics? Body of knowledge

If we think of an analogy Athletic body Process is the key

What is this process? What do physicists do when they do physics?

Physicists represent physical processes and ideas in different ways design experimental investigations collect and analyze data devise and test ideas (mathematical models, mechanisms, etc.) modify their ideas in light of new data evaluate communicate Work together and do it again and again

Are those processes that we need to teach?

Observe

Hypothesize

Observe Hypothesize Predict

Observe Hypothesize Predict Test

Do it again and again until I am satisfied

Children are born physicists who do not give up

Can we capitalize on this?

Why should we?

Why should we focus on the process? Our students will encounter different elements of the physics body of knowledge in the course ONCE but if we focus on the process then there are MULTIPLE opportunities to see the same process again and again. + They were good at it at some point. Can we use the physics body of knowledge as a context to help students strengthen the “physics habits of mind” that they already have?

Investigative Science Learning Environment (ISLE) Etkina and Van Heuvelen, Observational experiments Explanation, mechanism, hypothesis, relation Application Revision Assumptions More different assessment Testing experiments Yes

Some of the main goals of ISLE philosophy are to help students understand how the body of knowledge is constructed and develop scientific habits of mind. We call them SCIENTIFIC ABILITIES (in K-12 education they are called “practices”) What are those?

Scientific abilities representing physical processes and ideas designing an experimental investigation (three types) collecting and analyzing data devising and testing a qualitative explanation or a quantitative relation modifying an explanation or a relation in light of new data evaluating communicating

What do our assessments (tests) look like in a course that aims to develop the scientific abilities?

1. You have a loop of wire connected to a galvanometer (shown in the diagram), and a bar magnet. a. Describe an experiment that will make the galvanometer needle deflect to the right. Include a labeled diagram. The needle deflecting to the right indicates that current is flowing into the port on the right side of the galvanometer. b. Explain in detail what causes the current to start flowing in that direction. Paper and Pencil Exam Questions

2. In one of our labs this semester you were asked to determine the elastic potential energy stored in a toy car launcher. You decide to take this one step further and design an experiment to determine the “spring constant” of the launcher. To do this you set the launcher to its second highest setting (which involves pulling the launcher back 12.7cm) and launch the car (mass 60g) three times. Each time you use a stopwatch to record how long it takes the car to travel 50.0 cm down the track. The results are shown in the table. a. Devise a mathematical method that uses these measurements to determine the spring constant of the launcher. Use this method to obtain an answer. Include the uncertainty in your answer. b. Describe one assumption you made in your mathematical method. Does this assumption cause your method to overestimate or underestimate the spring constant? Trial Time s s s

Lab practical exam question You need to develop a simple version of a projector. The image that it projects must be three times as large as the object. As part of your report you must include a precise and detailed drawing of your projector complete with measurements that someone else could use to build it. Available equipment: Assortment of convex and concave lenses, meter stick, viewing screen, light source w. crosshairs pattern (the object).

When and how do students develop those?

One of the strategies: The lab is completely integrated and basically drives the course. In the labs students do initial observations to come up with patterns or models, and then they test and apply them after a discussion in a large room meeting. THEY DO NOT READ THE BOOK BEFORE CLASS!

Students design their own experiments in every lab! Design an experiment to find a relation between a voltage across and current through a commercial resistor. Design an experiment to test whether this relation applies to an incandescent light bulb. Design an experiment to test a hypothesis: interaction of electrically charged objects can be explained by magnetism. Design two independent experiments to determine the specific heat of the given object. The material of the object is not known. Use the list of available equipment (xx, xx) to pose your own scientific question. Investigate the question and write a report.

Students design their own experiments in every lab! Guided by scientific-abilities focused questions Self-assess their work and improve it with the help of rubrics

Rubrics for guidelines, assessment and self-assessmen t The effects of relevant assumptions are determined correctly but assumptions are not validated. Needs improvement (2) The effects of relevant assumptions are determined and assumptions are validated. An attempt is made but effects are described vaguely. No attempt to determine the effects of relevant assumptions. To evaluate specifically the ways in which the assumptions might affect the result Adequate (3) Not adequate (1) Missing (0) LEVEL sub ABILITY

Basic rubric structure A student writes relevant things with some minor omissions (description of what is missing) Needs improvement (2) As perfect as we can expect (a list of all good stuff) A student knows that she/he needs to write something but what is written is vague (description of what is missing) The student does not know that she/he needs to address this issue Small sub ability Drawing a free-body diagram Comparing results of two experiments Adequate (3) Not adequate (1) Missing (0) LEVEL ABILITY

What is the smallest setting of a HotWheels car launcher that will allow the car to do loop the loop without falling? A part of it is to design an experiment to determine the elastic potential energy stored in the launcher

Experiment 1: For each notch, shoot car up into the air, measure the distance that it goes in the air until it starts to fall. Then we can find PE g and use it to find PE spring : PE g = PE spring. U g = mgy Student written lab report - our data source

We measured mass of the car, the distance it travels into the air (shoot against the wall and mark the wall where the car reaches max) U s = mgy We assume that the car is point particle, that it shoots straight up and that we can accurately measure the vertical distance. If we cannot measure accurately our values cannot be accurate. Student written lab report - our data source

We estimate our measurement of the distance is accurate to +/- 2cm. We estimate that uncertainty from assuming car is a point particle is about +/- 6cm (length of the car). This uncertainty is the largest so we will ignore others. This is a relative uncertainty of 11%. ymgy 1st notch0.515m0.141 J 2nd notch0.720m0.198 J 3rd notch0.945m0.259 J 4th notch1.175m0.322 J Student written lab report - our data source

Uncertainties: Most of important uncertainties are identified (Adequate). Evaluation of uncertainty: Random uncertainty is not evaluated. The final result does not incorporate uncertainty. (Inadequate) No attempt to minimize uncertainty (Missing). Assessment of student work Scientific Ability to collect and analyze data; Rubrics: identify, evaluate, and minimize uncertainty

Study 1: How do students develop scientific abilities? Physics for the sciences 193/194 (190 students) ISLE-based with design labs since 2003 Used rubrics to score lab reports (60 x 12 = 720 reports) Reliability > 90% Observed students in the labs and created timelines for their behaviors

Ability to identify uncertainties 0&1 2&3

HW Lab2 Lab4 Lab5 Lab7 Lab8 Lab6 Lab9 Lab10 HW Ability to evaluate uncertainties estimating the largest uncertainty 0&1 2&3

Lab1 Lab2 Lab4 Lab5 Lab7 Lab8 Lab6 Lab9 Lab10 Ability to identify assumptions Lab3 0&1 2&3

Lab1 Lab2 Lab4 Lab5 Lab7 Lab8 Lab6 Lab10 Ability to evaluate assumptions Lab3 0&1 2&3

Summary of Findings Time dependence Content dependence (especially the effects of assumptions!) Significant improvement Saturation Lots of time spent on sense-making and writing!

Study 2: Do students transfer scientific abilities? Experimental and control group Same course Designing a physics experiment Comparison Designing a biology experiment Solving regular exam problems Week 1-10 Week 12 Week 13 Week 5, 11, 15 Design labs+Rubrics PER based labs non-design Experimental groupControl group

Students’ activities during semester labs Design Group Non-Design Group M U U D O U MMDMDMM sense-making 17 min writing 60 min procedure 17 min off-task 5 min TA’s help 31 min MPDD A A AMA R MMMDA U DD M R O A A D M sense-making 44 min writing 64 min procedure 20 min off-task 2 min TA’s help 16 min

Students’ activities during semester labs Design Group Non-Design Group M U U D O U MMDMDMM sense-making 17 min writing 60 min procedure 17 min off-task 5 min TA’s help 31 min MPDD A A AMA R MMMDA U DD M R O A A D M sense-making 44 min writing 64 min procedure 20 min off-task 2 min TA’s help 16 min

SM Distribution

SM vs Time

Time Spent on the lab activities Weeks 1 through 10

Design experiments to determine whether the helium balloon and the air balloon have the same drag coefficients. Physics transfer task: Investigation of the behavior of the balloon

30 60 Sense-making: 43 min Writing: 43 min Procedure: 35 min Off-task: 0 min TA’s help: 8 min Reading: 9 min Min AMU DDDDDMDDMA MUDD MMMUUAA Design Group DDDMDDMMDMMMMMM Sense-making; 20 min TA’s help: 29 min Reading: 11 min Writing: 35 min Procedure: 26 min Off-task: 0 min Non-Design Group

Time spent on lab activities DesignNon-design p - level of significance Total time162±17min120±25min Sense-making52±10min15±5min0.0007

Scientific Abilities Ability to identify the assumptions Ability to evaluate/validate effect of assumptions Difference is statistically significant Chi-square = 67.90, p < Identified relevant and significant assumptions 64% of design students 13% of non-design students “0” – missing “1” – inadequate “2” – needs some improvement “3” – adequate Design Non-design Difference is statistically significant Chi-square = 53.3, p < 0.001

Scientific Abilities Ability to evaluate the uncertainty Ability to evaluate the results by independent method “0” – missing “1” – inadequate “2” – needs some improvement “3” – adequate Design Non-design Difference is statistically significant Chi-square = , p<0.001 Difference is statistically significant Chi-square = 16.36, p < 0.001

Physics understanding Consistency of multiple representations Free Body Diagram Difference is statistically significant Chi-square = 17.73, p<0.001 “0” – missing “1” – inadequate “2” – needs some improvement “3” – adequate Design Non-design Difference is statistically significant Chi-square = 7.838, p< % of design students 22% of non-design students have score “1” - draw wrong FBD

Biology transfer task Conduct two experiments to determine transpiration rate using stem cuttings from a single species of plant.

Summary of findings Increased time on sense making Professionalism in lab reports Coordinated representations Recognized assumptions Evaluated uncertainties Results, evaluated by an independent method

What will our students remember when they forget everything?