Evaluating Learning Using Online Homework Andrew Bennett Center for Quantitative Education Kansas State University

The Big Question Does Online Homework Improve Learning?

The Big Question Does Online Homework Improve Learning? This is the wrong question!

The Big Question Does Online Homework Improve Learning? This is the wrong question! Better questions: – Which students does it help? – Under what circumstances does it help? – Why?

Knowledge Discovery in Databases Classification – Different groups of students react differently. What are the groups and how do they differ? – What problems do students do similarly on? Modeling – Can we track changes through time? – Can we predict success (or failure) for students?

Classification Clustering techniques – Hierarchical clustering (agnes) – K-means (pam) Projections – Singular Value Decomposition – Non-linear methods

Clustering points

Oranges vs. Peaches Suppose an orange is fruit for r 0.9 And a peach is fruit for r>0.5 and pit for r<0.5 Which has more fruit?

The Peach The orange is 72.9% fruit The peach is 87.5% fruit

37 Dimensions Outer 10% of radius encompasses 98% of total 37D volume. Any change in the center of the cluster drastically changes its contents.

Project onto lower dimension using Singular Value Decomposition

Modeling Regression – Linear – Logistic Hidden Markov Models – Lets you model through time

Qualitative Methods Student evaluations Interviews – Asking students to solve problems while you watch is often useful. – $10 or Extra Credit or Help going over an exam can induce students to participate. Key to answering WHY questions. IRB

Use Online Homework to Evaluate Offline Instruction Can handle large numbers easily Maintain standards for scoring “It ’ s not all about technology” – Don’t ignore the context

Useful Measurements Perfect vs. Non-perfect scores – Sometimes 90% works better than perfect Inverse time to perfect (speed) Acceleration (differences of speeds) Don’t neglect non-online data – Problem by problem exam scores – Attendance data

Useful Software R – Free – Can find packages to do almost anything – Very useful in teaching stats Matlab – Need to purchase appropriate toolboxes – I find it easier to use than R – Very useful in teaching numerical methods

Useful Software SQL Database – Allows you to slice and dice the data to form the measures you find most useful. Spreadsheet – Basic statistics – Goal seek for logistic regression – Easy Graphs

Research Nuggets Student ideas about what it means to learn mathematics are very important. Students start interested in learning, but shift to being interested in finishing. After about 4 attempts, outside intervention is appropriate. Timing of deadlines matters. Being mobile-friendly is useful.