1. Physics Education Research The University of Edinburgh The benefits of giving feedback: investigating patterns of student exchanges over the PeerWise system Alison Kay and Judy Hardy

2. Phys. 1AGGAChem. 1B 2011-122012-132011-122012-132011-122012-13 Students 172 275 215 276 155 141 % of mark 4 4 4 4 3 3 Authored 3 2 4 4 2 2 Answered 15 10 20 10 Rate & comment 9 6 5 5 6 6 Total comments 5058 3923 5910 1216646852798 Implementation of PeerWise

3. 1 Nonsense; totally irrelevant; symbols; undecipherable 2 Clear reply to a previous post – no reference to question or content 3 Basic statement that got question correct or incorrect; hit wrong button 4 Surface comments on question Surface comments on answer 5Deeper comments on question and specific features Deeper comments on own understanding 6 More evaluative - specific about HOW to improve the question or giving more detail WHY the question was good or not Possibly suggesting a different way of working out the answer – showing examples or calculations 7As 6 but much more detail – how to take question or explanation further. More in-depth discussion Comment coding

4. Comments examples 1“Why oh why. Sigh.” 2“Glad I could help ;D” 3“Haha! misread the question :P” 4“Wow! What a question! Superb effort!” 5“I too was caught out with the area of a sphere instead of a circle. Good explanation and good alternatives in multiple choice.” 6 “I'm not sure if your solution is correct. We can use a trajectory equation only if the initial and final heights are the same !!!” 7“Nice question, but bad distractors (imo). I didn't expect an inelastic collision and just worked with energy conservation (m1gh1 = (m1+m2)gh2) and got 1.76 m. Would have totally thrown me off if that was an available answer.”

5. Which activity is most strongly associated with exam performance? DV: Exam score IV: Number of answers or comments made Physics 1A 2012-13 Std. Beta No. answers submitted.248** Total no. comments.239*** No. comments >3.240*** No. comments >4.281***

7. Example: Physics 1A 2012-13 Is writing quality comments associated with higher exam performance when controlling for other factors? Dependent Variable: Exam score Independent Variables: No. comments coded > 4 Pre-test mark (FCI) Scottish Major Gender

8. Building the model R2R2 Adjusted R 2 F ValueP (one tailed).287.27431.4150.000 Independent Variables BetaStandardized Beta P (one tailed) Intercept42.2890.000 Comments > 4.346.1820.000 FCI.371.3750.000 Scottish-6.599-.2030.000 The model predicts that each new high quality comment, is associated with a 0.35% increase in exam score. The minimum 6 comments associated with a 2% increase. The mean of 14 comments associated with a 5% increase.

10. Conclusions Total engagement with PeerWise far exceeded minimum requirements Across all disciplines, writing meaningful comments has a significant, positive association with exam performance when controlling for other known influential factors Especially interesting given that PeerWise requires minimal instructor intervention; that PeerWise is only worth ~4% of the total course mark; and that providing feedback is only one aspect of the PeerWise assessment