1. Katrina Le, Hamid Tarmazdi, Rebecca Vivian, Katrina Falkner, Claudia Szabo and Nick Falkner
Computer Science Education Research (CSER) Group
Directing Teacher Focus in Computer Science Online Learning Environments

2. Increasing pressures of enrolments
Figure: Commencing Domestic IT enrolments in Australia ( ).

3. Increasing pressures of enrolments
We are seeing a resurgence in Computer Science enrolments.
Increasing pressure on academic staff, and on teaching resources.
Academic staff are also under increasing workload pressures, both in expected research output and the breadth of expected curriculum.
CRA, Computing Degree and Enrollment Trends, Taulbee Survey,

4. Online discussion forums
Online discussion forums provide a valuable method for students to engage with their colleagues and academic staff.
Asynchronous, and flexible.
But can easily get lost amongst the hundreds of messages and requests.

5. Text Classification “Thanks guys, it helped a lot”
“I did an insertion sort but did not use PTList”
“These results are also on the website”
Non-Critical
“Can there be a few more days given for assignment 3?”
“Still not working for me”
“I cannot access the submission system too”
“I am disappointed that no-one from the group has yet responded”.
Critical

6. Research Questions
To what extent can a Naive Bayes classifier effectively classify forum posts in Computer Science courses to determine if a post requires intervention?
Simple technique, cheap in terms of storage, computationally efficient.
But, typically applied for larger discussion items.
Assumes conditional independence.
Can this still be effective in the context of discussion forums?

7. Research Questions
To what extent can a Naive Bayes classifier effectively classify forum posts in Computer Science courses to determine if a post requires intervention?
Can we improve the accuracy by applying negation, parts of speech (POS) tagging, word stemming or removal of stop words ?

8. Text Classifier Applied tag stripping to remove HTML tags
Preprocessing modules able to be applied dependent on experimental conditions, including:
Removal of stop words
Part of Speech tagging
Negation identification
Word stemming

9. Experimental Data
Discussion forum data across multiple undergraduate courses.
Removed user/timeline information.
4,666 sentences.
1,978 manually identified as critical.
2,688 manually identified as non-critical.
Total word count of 83,176.

10. Results

11. Analysis Correctly identified as Critical:
“I’m quite confused.”, p=0.965 “So what exactly are we supposed to do in the Test file?”, p=0.999
Incorrectly identified as Critical:
“Now my code passes example 8.”, p=0.7318

12. Analysis
More importantly, though, is to consider those sentences that are Critical but identified as Non-critical:
“Please help us to solve this problem, thank you very much.” , p=0.885

13. Conclusions
Identified as an efficient technique, NB has also been shown to be effective in the classification of critical discussion forum posts.
Shows promise to be combined with other indicators of student engagement, such as patterns in resource access and overall participation to direct academic attention.

14. Computer Science Education Research Group blogs.adelaide.edu.au/cser
FIN