Core Methods in Educational Data Mining HUDK4050 Fall 2015
Assignment C2
What tools did you use? Packages (i.e. Excel) Features of Packages (i.e. Pivot Tables)
Let’s go back to the list of features from the last class As I read features off If you used this feature (or something very similar), raise your hand
1.last3ednormed-timesdnormed 50%ile in the last three problems, moved up to 75th percentile shift and direction of shift up —> 19down —> 0 2.timeperactions is low, last5wrong is high indicator variable, y/n clicking absent-mindedly or rushing through it up —> 17down —> 5
3.mult of pknow2 and notright percent of knowing is high, but wrong could be careless or off-task up —> 15down —> 5 4.pknow2 * timeofactions normalize — if you don’t know action, you will take more time brings it back down to someone who actually knew the material up —> 18down —> 0
5. ran out of time difference between timelast3ed and timelast5ed, what would you get out of it? up —> 6down —> 3 6.time interact / how many wrongs * pknow2 spend a lot of time but still get answer wrong, he’s off-task up —> 4down —> 6 7.how many wrong up / time per action up —> 7down —> 2
For the features that got used Did it end up in your final model? In what direction? Does this match the class’s overall intuition?
Who created a feature not discussed in Monday’s class? What feature? Did it improve your model?
Let’s… Go through how you created features – Actually do it… Re-create it in real-time, or show us your code… We’ll have multiple volunteers – One feature per customer, please…
Was feature engineering beneficial?
Other questions or comments about assignment?
Textbook
Knowledge Engineering What is knowledge engineering?
Knowledge Engineering What is the difference between knowledge engineering and EDM?
Knowledge Engineering What is the difference between good knowledge engineering and bad knowledge engineering?
Knowledge Engineering What is the difference between (good) knowledge engineering and EDM? What are the advantages and disadvantages of each?
How can they be integrated?
Any questions about cross-validation?
What are the advantages and disadvantages? Flat CV Stratified CV
What are the advantages and disadvantages? Flat CV Student-Level CV
Validity
What is… Generalizability?
What is… Ecological Validity?
What is… Construct Validity?
What is… Predictive Validity?
What is… Substantive Validity?
What is… Conclusion Validity?
What are… Some other validity concerns?
Exercise In groups of 3 Write the abstract of the worst behavior detector paper ever
Any group want to share?
Exercise #2 In different groups of 3 Now write the abstract of the best behavior detector paper ever
Any group want to share?
How many of these were actually feasible?
Real-world EDM involves tradeoffs
Which types of validity do you want to optimize? Which types of validity do you want to satisfice? Can any be safely ignored completely? (at least in some cases)
Other comments or questions About validity issues?
Other questions, comments, concerns about readings?
Assignment B3 Available on TutorShop
Next Class Wednesday, October 15 B3: Bayesian Knowledge Tracing Baker, R.S. (2014) Big Data and Education. Ch. 4, V1, V2. – Most important Corbett, A.T., Anderson, J.R. (1995) Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. User Modeling and User- Adapted Interaction, 4, – Mostly supplemental to video lecture Gweon, G. H., Lee, H. S., Dorsey, C., Tinker, R., Finzer, W., & Damelin, D. (2015). Tracking student progress in a game-like learning environment with a Monte Carlo Bayesian knowledge tracing model. Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, – Only if you have time and interest
The End