1. Correlation of Orientation to FCI and CLASS performance
The Effect of Students’ Learning Orientations on Performance in Problem Solving Pedagogical Implementations Andrew J. Mason1 and Charles A. Bertram1,2 1Department of Physics and Astronomy, University of Central Arkansas, Conway, AR Cabot High School and Cabot Freshman Academy, Cabot, AR 72023
Abstract
Students’ learning orientation, as applied towards learning problem solving, may be differentiated into learning a problem solving framework for its own sake, learning for the sake of doing well in the course, and non-goal-related considerations. In previous work, the role of learning orientation in student performance on a metacognitive problem solving exercise appeared to have some correlation with gains in conceptual understanding and attitudes towards problem solving skills. We examine a larger data set, taken from fourteen laboratory sections over six semesters of an introductory algebra-based physics course at the University of Central Arkansas, in which students worked on the problem solving exercise with their laboratory partners prior to a conceptually related laboratory exercise. We discuss the extent to which these preliminary results are robust, and whether they may be interpreted with general IPLS-like populations in mind.
Correlation of Orientation to FCI and CLASS performance
Check orientation results from previous research with larger sample size: framework-oriented (F), performance-oriented (P), vague-oriented (V)
Pretests:
FCI: Vague < Framework (p < 0.03)
No significant differences between any two orientations for any CLASS pretest score
Students tend to express less expert-like opinion on item clusters about application of technique (PS-S, ACU) – makes sense since most students never took physics before
Inflated pretest scores on RWC, SME, PS-G, PS-C? (students asked to consider physical science classes they took if they never took physics)
Gains:
CLASS: Framework-oriented students have positive or zero gains for all item clusters and overall.
F has stronger gains than P, overall & all clusters (p < 0.05) except for PS-C
F has overall stronger gains than V, and stronger on all problem solving and conceptual understanding item clusters (p < 0.05)
FCI: F gains > V gains (p < 0.05); Framework catches up with Vague. No difference between groups on FCI overall posttest.
Orientation (n)
FCI pretest
CLASS pretest, % expert-like answers
All PI
RWC
PS-G
PS-C
PS-S
SME CU
ACU
F (76)
23.6%
57.0
51.3
66.1
62.7
63.5
39.9
72.2
53.1
40.8
P (79)
25.2%
57.1
50.0
65.2
63.1
63.6
41.8
69.8
52.5
40.1
V (63)
29.1%
56.2
47.9
63.9
62.3
65.1
47.4
65.8
56.9
45.1
All (218)
25.8%
56.8
49.8
64.0
42.7
69.5
54.0
Orientation (n)
FCI % gain
CLASS % normalized gain
All PI
RWC
PS-G
PS-C
PS-S
SME CU
ACU
F (76)
+18.2%
+11.1%
+0.59%
+15.6%
+13.7%
+5.44%
+13.5%
+17.7%
+7.96%
P (79)
+14.4%
-2.69%
-21.7%
-9.60%
-2.81%
-2.00%
-14.0%
-0.56%
-5.15%
-7.84%
V (63)
+12.8%
-2.83%
-13.5%
-2.65%
-7.97%
-13.2%
-24.0%
+4.57%
-8.23%
-21.6%
All (218)
+16.6%
+2.06%
-11.6%
+1.18%
+1.45%
-0.76%
-10.1%
+5.81%
+1.92%
-6.31%
Background
IPLS considerations: Life science majors may not be monolithic (e.g. biology majors vs. health science majors)1,2
Initial look at differences found in terms of a) choice of major, b) learning orientation,3 applied towards a group-based physics problem solving exercise in algebra-based physics course section – appeared to relate to attitudes towards physics4,5
However, enrollment fluctuations between semesters, between lab sections introduces variability – relationship to in-class group dynamics & epistemic games remains unclear6
With larger sample size:
Relationship between choice of major and learning orientation
Effect on pre-post gains on FCI measures of conceptual understanding,7 CLASS measures of attitudes towards physics and problem solving8, overall course grade?
Correlation of Choice of Major to FCI and CLASS performance
Pretests:
FCI: Other NS > Biology > Health (p < 0.05, all cases)
CLASS: Health science majors are more novice-like to begin the course than the other majors, both overall and on item clusters in bold (p < 0.05). (Italics = borderline)
Check between pre-professional and non-pre for each major: significant difference between PT/OT majors and other Health science majors on all problem solving clusters (PT/OT > non-PT/OT, 0.02 < p < 0.04, all cases)
Gains:
No significant difference between majors for any comparison on CLASS (overall, item clusters)
Health Science majors actually gain less on FCI than do Biology (p < 0.02) and Other NS (p < 0.02)
Differences between Health Science majors and other majors remain on posttest from beginning of course (p < 0.01 overall, borderline or actual significance on problem solving clusters)
Major (n)
FCI pretest
CLASS pretest, % expert-like answers
All PI
RWC
PS-G
PS-C
PS-S
SME CU
ACU
Biology (91)
25.9%
60.1
58.1
73.1
65.2
64.0
45.2
71.3
57.1
43.6
Health (85)
22.0%
51.3
38.6
56.8
57.5
61.2
34.9
67.1
46.3
36.3
Other NS (35)
33.8%
61.0
54.8
65.7
66.8
69.3
55.7
69.4
51.8
All (218)
25.8%
49.8
65.1
62.7
42.7
69.5
54.0
41.8
Procedure
Six semesters sampled from first-semester introductory algebra-based physics class (Spring 2014 to Spring 2017)
Students given a context-rich problem9 to solve in lab groups during first 50 minutes of lab10
Instructor provides guidance and feedback during session, goes over solution at end of session
Students also perform metacognitive task to “self-diagnose” their strengths and weaknesses in solving problem together11
Data collected: FCI and CLASS pre-post, post-test survey about usefulness of problem solving exercise
Learning orientation determined by 2 independent raters coding students’ post surveys (specifically descriptions regarding what they thought was useful about the exercise) – raters discussed differences and came to agreement after coding
Relevant analysis: FCI rated for correctness on pre-post, CLASS rated for % of opinions that were favorable on pre-post (overall and for each item cluster)
Major (n)
FCI, % gain
CLASS, % normalized gain
All PI
RWC
PS-G
PS-C
PS-S
SME CU
ACU
Biology (91)
+16.9%
+2.77%
-15.2%
+3.75%
+1.11%
+5.49%
-5.39%
+3.61%
+3.08%
+0.73%
Health (85)
+11.3%
-0.35%
-11.5%
-8.63%
-2.47%
-7.06%
-11.2%
+5.82%
+0.65%
-11.4%
Other NS (35)
+20.2%
+4.48%
-8.43%
+9.76%
+12.1%
+9.52%
-17.3%
+12.6%
-1.24%
-12.9%
All (218)
+16.6%
+2.06%
-11.6%
+1.18%
+1.45%
+0.76%
-10.1%
+5.81%
+1.92%
-6.31%
Course Grades & Within-Groups Considerations
By major: Biology majors perform better than Health majors (p < ) & other NS majors (p < 0.05).
By orientation: Vague students are marginally behind Framework majors (p = 0.15) and Performance majors (p = 0.05). Variability is mainly with Health Science majors.
Within groups:
Each orientation group for Biology performs better than Health Science counterpart
(p < 0.05 within each orientation)
No differences between orientations with Biology or Health Science majors
(p > 0.10, within each major)
No differences within majors for each orientation or vice versa for CLASS pretest
Orientation differences in CLASS gains most clear within Health Science majors: F > P (p < 0.05), F > V (p < 0.01)
 GPA
Framework (n)
Performance (n)
Vague (n)
All Orientations (n)
Biology
3.28 (36)
3.20 (30)
3.00 (25)
3.18 (91)
Health
2.56 (32)
2.74 (28)
2.40 (25)
2.58 (85)
Other NS
2.40 (5)
2.94 (18)
2.75 (12)
2.80 (35)
All Majors
2.91 (76)
2.97 (79)
2.68 (63)
2.87 (218)
Acknowledgements
UCA SPS members, for assistance with audiovisual data , useful discussions, work as Learning Assistants,13 & outside-class tutoring hours
Funding: UCA Sponsored Programs Office, Department of Physics and Astronomy
 CLASS gain
Framework (n)
Performance (n)
Vague (n)
All Orientations (n)
Biology
+8% (36)
+1% (30)
-2% (25)
+3% (91)
Health
+10% (32)
-6% (28)
-8% (25)
-0% (85)
Other NS
+19% (5)
-2%( 18)
+9% (12)
+4% (35)
All Majors
+11% (76)
-3% (79)
-3% (63)
+2% (218)
References
S. Mochrie, Am. J. Phys. 84(7), pp , © 2016.
E. Mylott, E. Kutschera, J. Dunlap, W. Christensen, and R. Widenhorn, J. Sci. Ed. Tech. 25(2), pp , © 2016.
Z. Hazari, G. Potvin, R. Tai, & J. Almarode, Phys. Rev. PER 6(1), (2010).
A. Mason & C. Bertram, US-China Ed, Rev. A 4(12), , © 2014.
A. Mason, 2015 PERC Proceedings (Eds: A. Churukian, D. Jones, and L. Ding), pp , © 2015.
A. Mason & C. Bertram, 2016 PERC Proceedings (Eds: D. Jones, L. Ding, and A. Traxler), pp , © 2016.
D. Hestenes, M. Wells, & G. Swackhamer, TPT 30(3), (1992).
W. Adams et al., Phys. Rev. PER 2(1), (2006).
P. Heller & M. Hollabaugh, Am. J. Phys. 60(7), (1992).
P. Heller, R. Keith, & S. Anderson, Am. J. Phys. 60(7), (1992).
E. Yerushalmi, E. Cohen, A. Mason, and C. Singh, Phys. Rev. PER 8(2), and , © “Graduate Programs: 2017
Graduate Programs: “2017 Top Physical Therapy Schools for DPTs.” , retrieved 7/17/2017.
V. Otero, S. Pollock, & N. Finkelstein, Am. J. Phys. 78(11), 1218, © 2010.
Discussion
Health Science majors seem to be disadvantaged in comparison to Biology majors and other Natural Science majors
Begin semester with lower conceptual intuition, more novice-like attitudes
Do not catch up on FCI or CLASS post-tests
Perform worse in the overall course! (non-PT/OT Health Science majors do slightly worse than PT/OT majors)
Seems to particularly affect Health Science majors’ attitudes towards physics if they are not framework-oriented
Learning context? Reside in different college than do Natural Science majors (Biology and otherwise)
Performance orientation or non-goal orientation developed outside Natural Science curriculum context – may be source of disadvantage?
IPLS considerations: Biology majors (natural/biomedical) vs. Health Science majors (PT/OT, exercise science, etc.) may require more scrutiny
Cannot ignore this difference: two sets of majors reside in different colleges, therefore different pedagogical directions
Is not because of negligible students: UCA’s PT school is ranked as high as #3 in the country12 & routinely accepts undergraduates who take this course
Results may differ for intro calculus-based physics courses for these majors?