Addressing the Testing Challenge with a Web-Based E - Assessment System that Tutors as it Assesses Nidhi Goel Course: CS 590 Instructor: Prof. Abbott

Motivation Every hour spent in assessing students is an hour lost from instruction. Student assessment is not always accurate How to instruct a student at the same time of assessment

ASSISTment System E-Assessment + E-Learning ✔ Accurate assessment of a student ✔ Assist a student along with assessment No extra time is spent in assessment Examples of web based systems ✔ ✔ Massachusetts Comprehensive Assessment System (MCAS)

Research questions Does the tutoring provide valuable assessment information Does this continuous assessment system do a better job than more traditional forms of assessment Can we track student learning over the course of the year

Cont'd Can we see what factors affect student learning Can we track the learning of individual skills

Log in Assistment system

Original question Hint 1 st scaffolding question 2 nd scaffolding question Buggy Message

Data Collected Out of 600, 417 students of two middle school who have ✔ Score of MCAS test which is taken in May 2005 ✔ Results of 2 paper practise test which is taken on September 2004 and June 2005 ✔ Results of ASSISTment system which is used every other week from September 2004 to June 2005

Online Measures

Algorithm Step1: Select different variables among the 15 online measures to define the model. Step2: Calculate the coefficients for corresponding variables of a model using linear regression analysis to best match the obtained MCAS scores by the students. Step3: Calculate R2 using the calculated coefficients and sample data of students. R2 is a measure of correlation between curve fit values obtained from regression analysis equation and the sample data. Step4: Using R2 calculate BIC (Bayesian Information Criterion) = n*log(1- R2) + p(log(n)) where n=417 (number of samples), p=number of variables. BIC is a measure of how much information in terms of samples and number of variables is needed to get a reasonably good model. Higher BIC means more information is required, which implies the model is not very good. The authors select the model based on the following criteria ✔ Uses less number of independent variables ✔ Has large value of R2 The value of BIC gives a good assessment for such a model. Authors found that model IV was most significant.

Models

Conclusion Though model V has lowest BIC values, authors think model IV is better because it uses only six variables as compared to ten variables used in model V. Table 3 in this paper contains all included variables' coefficients calculated by regression analysis for model IV. These coefficients are useful in assessment of a student.