1. Automated feedback in statistics education
Faculty of Science
Freudenthal Institute
Automated feedback in statistics education
Sietske Tacoma
Summer School Freudenthal Institute
August 16, 2017

2. Challenges in university statistics education
Statistics is difficult
Many abstract concepts
Relations between concepts
Statistics anxiety
Groups are large
Statistics relevant for almost all areas of science
Difficult to provide individual guidance and feedback
Castro Sotos, A. E., Vanhoof, S., Van den Noortgate, W., & Onghena, P. (2007). Students’ misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review, 2(2),

3. Solution direction
Solve statistics problems in digital learning environment
Digital Mathematics Environment (DME)
Simulations, interactive graphs, contexts
(Much more on the DME tomorrow morning)
Provide automated feedback
Local: feedback on intermediate steps
Global: feedback on series of tasks

4. Project outline
Educational setting Introductory statistics courses as Utrecht University Biology (230), Economics (450), Social Sciences (1200) Phase 1 – design based research Design of two feedback types (local and global) Pilots: evaluate feedback and student experiences Phase 2 – evaluation study Influence feedback on student knowledge and understanding Interplay between two feedback types

5. This talk: local feedback in hypothesis testing
Introduction/recap hypothesis testing
Stepwise solutions
Designing feedback for intermediate steps
Freudenthal Instituut | Automatische feedback in universitair statistiekonderwijs

6. Introduction/recap hypothesis testing
How would you react if the grade you received for an exam is much lower than you had expected? Research suggests that most students think they can handle such situations better than their peers, but some students think their coping is worse than that of their peers.
In this study, participants were asked to read a scenario of a negative event and indicate how this event would influence their well-being (-5: worsen much, +5: improve much). Next, they were asked to imagine this same event from the perspective of a peer. The difference between both judgements was noted.
Suppose that for the sample of n=25 students the mean difference score was
MD=1.28 points (own judgement minus judgement peer) with standard deviation SD=1.50.
Based on these data, can you conclude that there is a significant difference between the own judgements and judgements of peers? Use a test with α=.05.
Freudenthal Instituut | Automatische feedback in universitair statistiekonderwijs

7. Difficulties in hypothesis testing
Reasoning under assumption that null hypothesis is true
Dealing with variability sample mean
Relation between values (test statistic) and probabilities (in probability distribution)
Freudenthal Instituut | Automatische feedback in universitair statistiekonderwijs

8. Stepwise solutions in the DME
Freudenthal Instituut | Automatische feedback in universitair statistiekonderwijs

9. Stepwise solutions - advantages
Feedback on every step
Possibility to address common errors
Structure for students
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4),

10. Stepwise solutions - disadvantages
Structure for students
Focus on procedural skills

11. Design: freedom to choose steps

12. Designing feedback for intermediate steps
Feedback on separate steps
“incorrect alternative hypothesis”
Address common errors
Feedback on order of steps
“to draw a conclusion, you first need to specify a critical value”
Structure constructed by students themselves
Mechanism: domain reasoner

13. Domain reasoners for stepwise feedback
Buggy rules: error recognition
Feedback
Partial solution
&
previous partial solution no
Constraints: correct solution path?
Rules: solution path recognition
Feedback
yes
Hint
Heeren, B., & Jeuring, J. (2010). Adapting mathematical domain reasoners. In S. Autexier, J. Calmet, D. Delahaye, P. D. F. Ion, L. Rideau, R. Rioboo & A. P. Sexton (Eds.), Intelligent computer mathematics (pp ) Springer Berlin Heidelberg.

14. Demo
DME test environment

15. Designing feedback text
What feedback would you give?

16. Designing feedback text
What feedback would you give?
Try to stimulate thinking and reasoning
Address concepts
Not give away correct answers

17. Further plans September – november 2017: 200 students 6 tasks
Evaluate whether feedback and hints are appropriate
Evaluate whether feedback and hints help
2018:
Use by 1500 students (social sciences and economics)
Inform student model and adaptation