Learning Analytics: Process & Theory March 24, 2014

Generalizability Does your model remain predictive when used in a new data set? Underlies the cross-validation paradigm that is common in data mining Knowing the context the model will be used in drives what kinds of generalization you should study

What kind(s) of generalizability Did Baker et al. 2008 look for?

What kind(s) of generalizability Did Baker et al. 2013 look for?

What kinds of generalizability Could one test for?

For each kind of generalizability Has anyone done it? What are the advantages to doing it? What are the practical challenges/barriers/costs?

Given… Given the massive cost of conducting every possible type of generalizability analysis How do we know when to stop?

Let’s go back to the class on validity… Since we didn’t quite finish our discussion…

Types of validity we discussed Ecological Construct Predictive Substantive Content Can anyone define and give a quick example of each?

Exercise In groups of 3 Write the abstract of the worst EDM paper ever

Any group want to share?

Exercise #2 In different groups of 3 Now write the abstract of the best EDM paper ever

Any group want to share?

The End