1. Physics Education Research The University of Edinburgh Students as co-creators: The development of student learning networks Alison E. Kay and Judy Hardy HEA STEM Annual Conference, Birmingham, 17 April 2013

2. 2 PeerWise: Brief Outline HEA STEM Annual Conference, Birmingham, 17 April 2013 Create multiple choice questions Provide and explain solutions Answer other students questions Rate and comment on other students questions Follow favourite authors Retain complete anonymity

3. HEA STEM Annual Conference, Birmingham, 17 April 2013 The PeerWise Interface

4. PeerWise in Physics 1A 2012/13 HEA STEM Annual Conference, Birmingham, 17 April 2013 4 Write 1 question Answer 5 questions Rate and comment on 3 questions 2 assessments – 3 tasks in each Total Contributions by end of the course from 276 students 630 questions 7156 answers 4025 comments

5. HEA STEM Annual Conference, Birmingham, 17 April 2013 Benefits of PeerWise for students Promotion of deeper learning: question writing and explaining answers imposes a high cognitive load Creation of a bank of questions to test knowledge and understanding Opportunity to engage with a wider network than in traditional offline academic settings.

6. 6 Network Analysis: A Novel Approach HEA STEM Annual Conference, Birmingham, 17 April 2013 Visualise and describe the network Metrics NOT available using traditional analysis New way of thinking about HOW PeerWise is being used and how to exploit relationships

7. 1101 1111 1010 7 HEA STEM Annual Conference, Birmingham, 17 April 2013 Network Analysis: 2 Mode Network Questions Students 1 C A B 23 4

8. 8 2 Mode Questions and Answers Network

9. 9 HEA STEM Annual Conference, Birmingham, 17 April 2013 Question Ego-network 1 EASY QUESTION 17 students (A) DIFFICULT QUESTION 17 students (B)

10. EASY QUESTION 17 students (A) answered 268 other questions Other questions seem less difficult: green squares generally smaller DIFFICULT QUESTION 17 students (B) answered 420 other questions Other questions seem more difficult: green squares generally bigger Question Ego-network 2

11. 11 HEA STEM Annual Conference, Birmingham, 17 April 2013 Student Answering Patterns LOWEST PERFORMING STUDENT 35% for final exam Answered 7 questions MEDIUM PERFORMING STUDENT 68% for final exam Answered 18 questions HIGHEST PERFORMING STUDENT 97% for final exam Answered 152 questions

12. -10 1-1 10- 12 HEA STEM Annual Conference, Birmingham, 17 April 2013 Network Analysis: 1 Mode Network Students B C B A C A

13. 13 HEA STEM Annual Conference, Birmingham, 17 April 2013 1 Mode Student Answers and Comments Density: 57% (100) Diameter: 2 (255)

14. 14 HEA STEM Annual Conference, Birmingham, 17 April 2013 Student ego-networks – answers and comments LOWEST PERFORMING STUDENT 35% for final exam Answered 7 questions 118 students with one question in common (43%) MEDIUM PERFORMING STUDENT 68% for final exam Answered 18 questions 170 students with one question in common (62%) HIGHEST PERFORMING STUDENT 97% for final exam Answered 152 questions 271 students with one question in common (99%) Degree Centrality: 43% Betweenness 0.000 Degree Centrality: 62% Betweenness: 0.001 Degree Centrality: 99% Betweenness: 0.007

15. 15 HEA STEM Annual Conference, Birmingham, 17 April 2013 1 Mode Comments Only Network Density: 28% (100) Diameter: 3 (255)

16. 16 HEA STEM Annual Conference, Birmingham, 17 April 2013 Student ego-networks – comments LOWEST PERFORMING STUDENT 35% for final exam Wrote 3 comments 25 students with at least question in common (10%) MEDIUM PERFORMING STUDENT 68% for final exam Wrote 5 comments 41 students with one question in common (15%) HIGHEST PERFORMING STUDENT 97% for final exam Wrote 24 comments 83 students with one question in common (30%) Degree Centrality: 10% Betweenness0.000 Degree Centrality: 15% Betweenness: 0.000 Degree Centrality: 30% Betweenness: 0.003

17. HEA STEM Annual Conference, Birmingham, 17 April 2013 Conclusions Initial results show both networks to be equitable Clear differences between the comment only network and the comment and answer network Network analysis provides rich, qualitative data Complements traditional statistical analysis of performance and usage data

18. HEA STEM Annual Conference, Birmingham, 17 April 2013 Next Steps? Replicate analyses whilst normalising data to account for number of questions answered or number of times question answered. Incorporate network metrics with more traditional analyses Characterise the nature of the interactions between students – qualitative analysis of comments

19. School of Physics and Astronomy Acknowledgements All participating students Judy Hardy, Ross Galloway HEA STEM Annual Conference, Birmingham, 17 April 2013 Paul Denny