1. Modeling Student Knowledge Using Bayesian Networks to Predict Student Performance By Zach Pardos, Neil Heffernan, Brigham Anderson and Cristina Heffernan To evaluate the predictive performance of various fine-grained student skill models in the ASSISTment tutoring system using Bayesian networks. Goal ASSISTment is a web-based assessment system for 8 th -10 th grade math that tutors students on items they get wrong. There are 1,443 items in the system. The system is freely available at www.assistment.org Question responses from 600 students using the system during the 2004-2005 school year were used. Each student completed around 260 items each. The Skill Models The skill models were created for use in the online tutoring system called ASSISTment, founded at WPI. They consist of skill names and associations (or tagging) of those skill names with math questions on the system. Models with 1, 5, 39 and 106 skills were evaluated to represent varying degrees of concept generality. The skill models’ ability to predict performance of students on the system as well as on a standardized state test was evaluated. The five skill models used: WPI-106: 106 skill names were drafted and tagged to items in the tutoring system and to the questions on the state test by our subject matter expert, Cristina. WPI-5 and WPI-39: 5 and 39 skill names drafted by the Massachusetts Department of Education. WPI-1: Represents unidimensional assessment. Background on ASSISTment Predicting student responses within the ASSISTment tutoring system The ASSISTment fine-grained skill models excel at assessment of student skills (see Ming Feng’s poster for a Mixed-Effects approach comparison) Accurate prediction means teachers can know when their students have attained certain competencies. 1. Skill probabilities are inferred from a student’s responses to questions on the system Bayesian Belief Network Student Test Score Prediction Process This work has been accepted for publication at the 2007 User Modeling Conference in Corfu, Greece. SponsorsCollaborators Result: The finer-grained the model, the better prediction accuracy. The finest-grained WPI-106 performed the best with an average of only 5.5% error in prediction of student answers within the system. Result: The finest-grained model, the WPI-106, came in 2 nd to the WPI-39 which may have performed better than the 106 because 50% of its skills are sampled on the MCAS Test vs. only 25% of the WPI-106’s. Predicting student state test scores Conclusions A Bayesian Network is a probabilistic machine learning method. It is well suited for making predictions about unobserved variables by incorporating prior probabilities with new evidence. Bayesian Networks Arrows represent associations of skills with question items. They also represent conditional dependence in the Bayesian Belief Network. Probability of Guess is set to 10% (tutor questions are fill in the blank) Probability of getting the item wrong even if the student knows it is set to 5% 2. Inferred skill probabilities from above are used to predict the probability the student will answer each test question correctly Probabilities are summed to generate total test score. Probability of Guess is set to 25% (MCAS questions are multiple choice) Probability of getting the item wrong even if the student knows it is set to 5%