Improving on the Recitation Section: Tutorials in Introductory Physics Wed Brown Bag SJP Fa '05

What are Tutorials? Research-based Student-centered “Elicit-confront-resolve” Process (discussion, consensus) + Sense-making

The Tutorial space (basement, across from Jerry Leigh)

Tutorials at CU Used in 1110 & 1120, twice each (+ 1120, now) Part of successful courses => motivation to continue Today: some data Continue support?

Resources Space, material, training, personnel Ongoing support necessary. “Learning Assistants” (LA’s) - undergrad STEM majors (interested in teaching)

Courses studied so far Recitation Homework Text: Phys 1110 –Fa 03 Tutorials CAPA HRW –Sp 04 Tutorials CAPA HRW –Fa 04 Knight workbookMP Knight –Sp 05 Trad recitations MP Knight Phys 1120 –Fa 04 TutorialsCAPA HRW –Sp 05 TutorialsCAPA Knight

pre/post measures Phys 1110: FMCE Phys 1120: BEMA common exam questions formal + informal survey questions

Phys 1110: Distinguishing features 1: “Tut” (Sp04) Tutorials + CAPA 2: “Workbook” (Fa04) Small groups/Knight (+ Mastering Physics) 3: “(More) Trad” (Sp05) Mostly traditional recit’s (Otherwise much like "2" above) (different instructors, semesters …)

1110 summary - up front! Tutorial courses : strongest results on learning gains and attitude surveys Middle course (“2”) (research-based text and hw, clickers + small-group recitations, not Tutorials): good results. Last course (like “2” except recitations): gains solid, lowest of three.

Phys 1110 Pretest FMCE (Force and Motion concept evaluation) A nationally validated conceptual exam, like the FCI (but harder) Matched, valid data only. (Eliminate pre>93%) Ave Pre ave = 28%(1) / 34%(2) / 28%(3) Spring (2) higher (fall term)

Phys 1110 Posttest Ave Post ave = 74%(1) / 69%(2) / 58%(3)

Phys 1110 normalized gains gain(1)=.66 +/-.02

Phys 1110 normalized gains gain(1)=.66 +/-.02 g(2)=.585 +/ points lower => (more than half a letter grade)

Phys 1110 normalized gains gain(1)=.66 +/-.02 g(2)=.585 +/-.02 g(3)=.45 +/-.02 (trad recit.) => significantly lower gains. (still, double nat’l standards!)

Impact on different pretest populations: "low starters" pretest <=12.5% (% of class in this pool) Course (1) (2) (3)

Impact on different pretest populations: "high starters" 50<pre<93% (% of class in this pool) Course (1) (2) (3)

Beyond the FMCE: Exam comparisons #2 (Knight workbooks/small groups): 34 common exam q’s #3 (Trad recitations): 30 common q’s (17 are “tutor. materials”, 9 are “quant/trad”) (12 are “tutor. materials” 6 are “quant/trad”)

Beyond the FMCE: Exam comparisons ( Tutorial score - other course score) All q’ Tut mater. quant/trad (2) Workbook: (3) Trad: (All ± 2) N.B. 12 points is roughly 1 letter grade. Tutorial courses: significantly better exam scores: both conceptual and quantitative/traditional.

Other data Replication study (compare with UW) CLASS (attitudes/beliefs) Surveys (did you like, did it help?) All favorable (or neutral) for tutorials, ask for details!

1110 Summary Compare Tut-based with “workbook/small group”  measurable diffs (FMCE, exams, CLASS) Compare Tut-based with “more trad rec”  significant diffs. Tutorials only one effect. (Instructors, course structure …!?) But in 1120, changing instructors + text => no impact

Phys 1120: Tut1 (Fa04) and Tut2 (Sp05) Attempt at replication. Main differences: Tut2 has… different instructor different semester different textbook follows up 1110 without Tutorials no long answer on exams

Summary (up front!) Despite changes in course elements, we replicated Tutorials + basic course structure. Result: identical (high) learning gains. (Final score matches our juniors. Hard exam!)

1120 BEMA pre/post BEMA = “Brief E&M Assessment”, F04 (N=319) Pretest ave 26%

1120 BEMA pre/post F04 (N=319) Pretest: 26% S05 (N=232): 27%

1120 BEMA pre/post F04 (N=319) 26% -> 59%, S05 (N=232) 27% -> 59% g(ave, F04) =.44+/-.01 g(ave, S05)=.43+/-.01

1120 E&M pre/post

Other data Common exam questions (no change) Replication study (compare with UW) CLASS (attitudes/beliefs) Surveys (did you like, did it help?) All results similar for both semesters!

Affect: survey results 1120 (Tut2) followed 1110 using Knight “workbook” small-group recitations. Asked at end: Which is better, Tut or trad rec? 39% vs 40% Which would you enjoy more? 39% vs 39%

1120 mini-summary Different instructors, text, exam structures, semesters… same Tutorials and Conceptests: no sig diff on BEMA, CLASS, exams. Validated survey scores high. Slight differences in surveys: ~neutral student satisfaction. No disasters, room to improve

Bottom line Tutorials successful, productive course elements. Cost:$1500/LA/semester * (6-8 LA’s per course) Need 1.5 hr/week training session (TA’s too!) Benefits: Some LA’s => K-12 cert. (+ their learning gains very high) TA’s exposed to research-based learning.

Recommendation We should continue implementing Tutorials (and collecting data) Need support for LA’s, and training infrastructure (=> faculty or experienced grad student assigned to teach the TA’s/LA’s)

Questions, Discussion!

THE END

Replication UW (no Tut) UW (with Tut) CU (with Tut) Atwood: tension Identify Newton III partners 1570 McDermott et al., AJP 62, 1994

Replication UW (no Tut) UW (with Tut) CU (with Tut) Atwood: tension Identify Newton III partners 1570

1-Tut 3-Trad 2-Workbook “Beliefs” survey: CLASS pre/post

CLASS pre/post (full scale) Tutorial-based course: no shift Two others: small but statistically significant declines

CLASS pre/post Tut1 Tut2 pre- to post shifts (attitudes and beliefs survey). Slightly negative (!), though it’s usually worse.

CLASS pre/post (Shown on full scales)

Affect: survey results 1110 ( Likert scale 1= “no” to 5=“yes”) Did the Tutorial help you learn Physics in 1110? /-.05 (20% neg, 60% pos) Did you like the Tutorials? /-.05 (62% neg, 21% pos)

Affect: survey results 1120 Likert scale 1-5: “very bad” to “very good”) How do you feel about the use of Tutorials in Phys 1120? Tut1: /-.07 Tut2: /-.09 (33% neg, 53% pos) (46% neg, 43% pos) How much help was the Tutorial? Tut1: /-.06 Tut2: /-.08

Force Concept Inventory (FCI) R. Hake, ”…A six-thousand-student survey…” AJP 66, (‘98). = post-pre 100-pre Fa03/Sp04 Fa98 red = trad, blue = interactive engagement F C I at C U

Phys 1110 normalized gains gain(1)=.66 +/-.02 g(2)=.585 +/-.02 g(3)=.45 +/-.02 Course (1) with Tutorials has highest gains. (2) is 8 points worse. Course (3) (trad recit.) => significantly lower gains. (but still, double nat’l standards!)

Beyond the BEMA - exam q’s 31 common exam q’s (13 explicitly “tutorial material”, rest => other topics) All Tut material only  (Fa04-Sp05) -1.9% ± 2% -1.8%

~40% ~75% * Mazur, E. Peer Instruction, Prentice Hall