1. Student Evaluation Differences between Different Physics by Inquiry Courses Gordon J. Aubrecht, II, Ohio State University PERG Abstract: The setting of Physics by Inquiry (PbI) classes is the laboratory. Students do experiments as suggested by the text as well as doing their own experiments to test predictions they have made about nature’s behavior. This is guided inquiry, and students assessments of PbI classes reflect important aspects of inquiry, while not supporting others. We report here on results from different versions of Physics by Inquiry courses (properties of matter, electric circuits, and astronomy by sight and optics) using the Laboratory Program Variables Inventory (LPVI), a Q- sort instrument.

2. The Q-sort mechanism, devised by William Stephenson in 1935, consists in organizing the statements or pictures into a ranking scheme. Q-sort

3. Q-sort instruments can get first-hand data from large numbers of students in a short time. Our Q-sort instrument is called the Laboratory Program Variables Inventory (LPVI). The LPVI contains 25 statements characteristic of most hands-on lab environments; the statements describe common lab activities. It measures what individual students think are the most and least characteristic features of classroom activities, not what they most like or most dislike about the course.

4. Method Students were instructed to sort the 25 statements of the modified LPVI into five groups. Group I is considered most descriptive of the course; Group V least descriptive. Students are forced to rank the statements into groups of size 2, 6, 9, 6, 2, forming a quasi-normal distribution.

5. Most descriptive to the Least descriptive

9. Circuits PbI instructors N = 12 average scorematrix score 171.09170.76 11.0310.76 50.8380.64 8 50.56 230.62160.54 21, 24-0.3221-0.25 14-0.4014-0.29 7-0.477-0.37 4-0.554-0.39 6-1.076-0.65

10. The  2 analysis We perform a  2 analysis on M’ that allows us to see which statements are significantly different from randomly chosen values. These are highlighted below. Items are considered significant if confidence level > 95%, or, in other words, if P(  2 ) < 0.05 that the result could occur randomly.

11. N = 12 average score P(  2 ) matrix score P(  2 ) 171.090.0072170.760.0072 11.030.03910.760.039 50.8380.640.0205 80.640.020550.56 230.62160.540.039 160.5040.039 18-0.180.028318-0.220.0283 21, 24-0.3221-0.25 14-0.4014-0.29 7-0.477-0.37 4-0.554-0.39 6-1.07<0.00016-0.65<0.0001

12. Ranking of Statements The table shows how few of the Physics 107 instructors’ choices are significant. In addition, some non-outlier scores are significant: statements 16 and 18.

13. Distribution of matrix elements for the two greatest outliers and the two non-outlier statements for physics instructors. StatementIIIIIIIVV 6-0.10-0.20-0.360.120.56 170.280.03-0.09-0.240.01 StatementIIIIIIIVV 16-0.080.40-0.18-0.150.01 180.01-0.24-0.180.400.01

14. We have taken LPVI data for all three versions of our PbI class over several years. They are known as 106 (properties of matter), 107 (electric circuits), and 108 (optics and astronomy by sight).

15. Do all three classes’ students see the courses (all taught the same way) as acting identically?

16. Student significant LPVI statements appearing at least in two courses (red denotes among the top/bottom five) Physics 106Physics 107Physics 108 11 44 555 666 777 13 14 16 17 18 21 22 23 24

17. Remember, only significant statements are shown. Statements 1, 4, 5, 6, 7, 13, 14, 16, 17, 18, 21, 22, 23, and 24 are commonly seen as very characteristic or very uncharacteristic.

18. 5. Laboratory activities are used to develop concepts. 17. Students discuss their data and conclusions with each other. 22. During laboratory students check the correctness of their work with the instructor. 23. In discussion with the instructor, assumptions are challenged and conclusions must be justified.

19. 6. The instructor lectures to the whole class. 7. Students are asked to design their own experiments. 18. The instructor or laboratory manual asks students to state alternative explanations of observed phenomenon. 21. Students identify problems to be investigated. 24. Students usually know the general outcome of an experiment before doing the experiment.

20. Statements 1, 4, and 16 are characteristic of just two classes each. 106, 107 1. Students follow the step-by-step instructions in the laboratory manual. 106, 108 4. Students are allowed to go beyond regular laboratory exercises and do experiments on their own. 107, 108 16. Questions in the laboratory manual require that students use evidence to back up their conclusions.

21. The LVPI is a valuable tool for use because it provides instructors information about how students perceive what actually happened in a course without the need for lengthy classroom observations. This work supports the usefulness of Q methodology for assessing laboratory courses and shows that it can be used to provide instructors with formative assessment of their classes.