Alastair McLean1, Bei Cai1, Lindsay Mainhood2 and Ryan Groome1 Developing Experimental Design Skills: Julia Child or Mission Impossible? Undergraduate laboratories are increasingly the subject of Physics Education Research. Are students developing the experimental design skills that the American Association of Physics Teachers has identified as essential 21st Century competencies? We have made some modest changes to PHYS250, a second year Physics Laboratory. These changes give students the freedom to design their own experiment. We have also attempted to capture the effect that these changes have had on the student experience. Alastair McLean1, Bei Cai1, Lindsay Mainhood2 and Ryan Groome1 1 Department of Physics, Engineering Physics and Astronomy, 2 Faculty of Education.

10 Structured-inquiry experiments PHYS 250/3.0 Foundations of Experimental Physics Laboratory and lecture course that presents techniques and skills that are the foundations of experimental physics. Topics include statistical analysis of data, uncertainties in measurement, propagation of errors, software for data analysis, graphing and reporting…. LEARNING HOURS 132 (24 Lectures; 36 Labs; 72 Practice) When I took over PHYS250, in 2014, I inherited 10 structured-inquiry experiments. Over the last six years we have redesigned two of these as guided-inquiry experiments and introduced a new guided-inquiry experiment, Coupled Oscillators. These changes were assessed using ECLASS surveys, LOPUS and qualitative assessment of in-class recordings. 3 Guided-inquiry and 8 structured-inquiry experiments redesign and assess 10 Structured-inquiry experiments

American Association of Physics Teachers Levels of Inquiry Banchi and Bell, Science and Children, 2008. Open Student formulated question and student designed procedure. Guided Instructor formulated question and student designed procedure. Structured Instructor formulated question and instructor designed procedure. Freedom What is the difference between guided and structured inquiry? In a guided-inquiry experiment a student is given a task or research question and then asked to develop their own procedure. This allows them to develop experimental design skills. In structured-inquiry- the traditional approach in physics - the student is given a detailed procedure to follow. American Association of Physics Teachers

Guided Inquiry Structured Inquiry Degree of Freedom = 2 CD Diffraction vary L finish 1 Structured Inquiry 2 Degree of Freedom = 1 vary theta start In a guided-inquiry lab, the apparatus must have more than one DOF; a DOF being defined as a way of executing the task. This gives the students agency and requires them to make informed choices. In CD Diffraction, for example, there are two DOF (soon to be three). The students can make measurements by systematically varying either L or theta. In structured inquiry there is traditionally only one way to execute the task – and that is to follow the procedure. finish 1 start

Structured-inquiry outline Guided-inquiry outline Here is an example of the structured-inquiry outline versus the guided-inquiry outline. In the structured inquiry lab, the students are told specifically what to do including how to align the laser with the CD and what quantities to measure. In the guided-inquiry lab, the students are asked to first predict what the diffraction pattern would look like. They are then asked to establish procedures to align the CD and the laser. The students had to predict, design and make sense of the lab. Guided-inquiry outline Prescribed procedures are given Students had to predict, design and make sense of the lab

Qualitative Analysis Research Questions Audio recording of students: 8 guided- inquiry labs 5 structured- inquiry labs How does student learning differ between the guided-inquiry labs and structured-inquiry labs? What differences can we observe in student behaviour between the two labs? What elements of lab design best reinforce achievement of desired learning outcomes? In winter 2017, there were 13 pairs of students who did the CD diffraction experiment. The first 8 pairs were provided with the guided-inquiry outline and the last 5 pairs were given the structured-inquiry outline. We audio-recorded the students in the labs and transcribed the audio recordings. We’ve completed a qualitative inductive coding analysis, which basically involves reducing the text data to codes then to categories and finally themes. Here are the 3 research questions for our study. We are interested in finding the differences in student learning and student behaviour between the guided and structured inquiry labs. We also want to find out what elements of the lab design best reinforce achievement of desired learning outcomes. Thematic analysis, Strauss and Corbin, 2015 Inductive approach, Thomas, 2006

Guided-inquiry lab Structured-inquiry lab Theme Category Experimental process and components Process Carrying out scientific method Visual observation Project management Problem solving Measurement and calculation Hypothesizing Constructing apparatus Comprehension experiences Sense making Question asking Learning Sense making/Sense making strategies Of task, peer’s idea or suggestion, observations, instructions, calculations or measurements Learning/Confirmation/Application Question asking/Physics concepts Self and interpersonal experiences Attitude Interpersonal interactions Interpersonal learning Physics language Peer interaction Needing help Student interaction with instructor Instructor interaction with students Laboratory design experiences Design Critical thinking Measurement and analysis The affective experience Reactions Marvelling Emotions This table summarizes our coding results. 4 themes emerged from the guided-inquiry labs. They are ….. These are the categories in each of the themes. There are also 4 themes that emerged from the structured-inquiry labs. These are…… And the categories from each theme are listed here.

Guided-inquiry lab Structured-inquiry lab Themes from qualitative coding of audio transcripts Plotted here are the frequency of each theme. The top is for guided-inquiry labs and bottom is for structured inquiry labs. The y-axis is the themes. The x-axis is the averaged number of references. What this means is that “lab design experiences” is referred to at an average of 110 times in each of the guided-inquiry lab. In comparison, “carrying out scientific method” is referred to 170 times on average in a structured-inquiry lab. Structured-inquiry lab

Codes unique in guided-inquiry lab Here is a graphical representation of the frequency of behaviours and learning experiences in the guided inquiry lab. Green circles represent themes, blue circles represent categories, and red circles represent codes. Only those codes that are unique to the guided-inquiry lab are represented here. Codes that appeared in both data sets are not included. Looking at this image quickly gives you a sense of what the most frequent and not-so-frequent behaviours and learning experiences are. For example, the most frequent behaviours related to lab design experiences, and the least frequent were related to the experimental process and components. Based on the differences in the frequency and nature of the experiences in each type of lab, we will be able to make claims about the behaviour and learning that is happening in each. For example, if we now look at the same representation of the structured-inquiry lab, we see different behaviours and learning experiences and their associated frequencies.

structured-inquiry lab Codes unique in structured-inquiry lab In the structured-inquiry lab, the most frequent experience is interpersonal learning, with similarly lower frequencies in all other experiences: sense making, the affective experience, and carrying out the scientific method. While it may be surprising that interpersonal learning is the most frequent experience in this lab, this is an important finding because it shows the focus of the students’ lab experience is not on measurement or making predictions, among other things, which may be more closely aligned with the course objectives.

Summary We’ve redesigned PHYS 250 to include more guided- inquiry labs. Student behaviors and learning experiences in guided- inquiry labs are more frequently design-like in nature; experimental design is an intended learning outcome. These results and their implications are still under investigation; e.g., the importance of degrees of freedom.

Acknowledgement This work is funded by the TRESTLE network, and the Department of Physics, Engineering Physics and Astronomy, the Center for Teaching and Learning Educational Research Grant, and the Faculty of Engineering and Applied Science at Queen's University.