1. The effect of educational environment on representational competence in introductory physics Patrick Kohl and Noah Finkelstein University of Colorado at Boulder

2. Outline Review of research questions and original study Review of research questions and original study Performance data from additional studies Performance data from additional studies Analysis of representational content of course environments Analysis of representational content of course environments Conclusions Conclusions

3. Introduction Original study investigated: Original study investigated: Effect of problem representation on student performance (student representational competence) Effect of problem representation on student performance (student representational competence) Effect of providing students with a choice of problem representation (student meta- representational competence) Effect of providing students with a choice of problem representation (student meta- representational competence)

4. Introduction Student performance was strongly affected by representation Student performance was strongly affected by representation Effect of representation choice was strong and either positive or negative Effect of representation choice was strong and either positive or negative

5. Research questions Can we explain the choice/control splits? Can we explain the choice/control splits? Will these effects be present in other courses? Will these effects be present in other courses? Why or why not? Why or why not?

6. Study methods Four question pre-recitation homework Four question pre-recitation homework Verbal, mathematical, graphical, pictorial question formats Verbal, mathematical, graphical, pictorial question formats One question recitation quiz in one of four formats One question recitation quiz in one of four formats Half receive format at random, half choose Half receive format at random, half choose Study covered two topic areas in each of Phys 2020 (Trad), 2010 (Reform), 2020 (Reform) Study covered two topic areas in each of Phys 2020 (Trad), 2010 (Reform), 2020 (Reform)

7. Atomic phys. quizzes, pictorial & verbal

8. Choice/control splits Quiz Subject VerbalMathGraphicalPictorial 2020 Diffraction (T) --0.040.03 2020 Spectroscopy 0.0020.00010.00040.001 2010 Springs (R) -0.09-0.07 2010 Pendulums ---- 2020 Diffraction (R) -0.06-- 2020 Spectroscopy ----

9. Choice/Control splits Effect of instruction appears to be strong Effect of instruction appears to be strong Hypothesis: Reform style using broader selection of representation imparts broader representational skill sets, reducing splits Hypothesis: Reform style using broader selection of representation imparts broader representational skill sets, reducing splits Is analysis of lectures and exams consistent? Is analysis of lectures and exams consistent?

10. Study methods Videotaped several lectures from each class, analyzed in terms of representation use Videotaped several lectures from each class, analyzed in terms of representation use Compared representation use on class exams Compared representation use on class exams Conducted problem-solving interviews in 2010 (Reform) and 2020 (Reform) sections Conducted problem-solving interviews in 2010 (Reform) and 2020 (Reform) sections

11. Exams Means by which students are held responsible for representation use Means by which students are held responsible for representation use Each exam sub-problem has its representational content assessed Each exam sub-problem has its representational content assessed We tally fraction of points possible associated with each representation We tally fraction of points possible associated with each representation

13. Averaged exam content

14. Course analysis Both reform-style used more representations in a more integrated way Both reform-style used more representations in a more integrated way Could plausibly account for quiz performance data Could plausibly account for quiz performance data Course also included other components: recitation, lab, homeworks Course also included other components: recitation, lab, homeworks

15. Conclusions Both micro- and macro-level sources of performance variation Both micro- and macro-level sources of performance variation Students have fairly robust opinions of their own representational competence Students have fairly robust opinions of their own representational competence Opinions are cross-context; skills are not necessarily Opinions are cross-context; skills are not necessarily Pervasive use of different/multiple representations in instruction can have a noticeable effect on student skills Pervasive use of different/multiple representations in instruction can have a noticeable effect on student skills

16. Acknowledgements This work was supported in part by an NSF Graduate Fellowship and by Colorado PhysTEC. This work was supported in part by an NSF Graduate Fellowship and by Colorado PhysTEC. Special thanks to the PER group at CU- Boulder and to Drs. Beale, Munsat, and Peterson. Special thanks to the PER group at CU- Boulder and to Drs. Beale, Munsat, and Peterson.

17. Performance data – traditional 2020 VerbalMathGraph Choice35%(N=17) 37% (N=57) 4%(N=26) Control24%(N=17) 56% (N=18) 25% (N=16) p = 0.04 Pictorial 82% (N=72) 58% (N=19) p = 0.03 Wave optics quiz Choice81%(N=21) 90% (N=42) 96% (N=28) Control32%(N=19) p = 0.002 13% (N=15) p < 0.0001 53% (N=17) p = 0.0004 39% (N=58) 83% (N=18) p = 0.0012 Atomic physics quiz