1. Learning Analytics: Process & Theory
March 24, 2014

2. Generalizability
Does your model remain predictive when used in a new data set?
Underlies the cross-validation paradigm that is common in data mining
Knowing the context the model will be used in drives what kinds of generalization you should study

3. What kind(s) of generalizability
Did Baker et al look for?

4. What kind(s) of generalizability
Did Baker et al look for?

5. What kinds of generalizability
Could one test for?

6. For each kind of generalizability
Has anyone done it?
What are the advantages to doing it?
What are the practical challenges/barriers/costs?

7. Given…
Given the massive cost of conducting every possible type of generalizability analysis
How do we know when to stop?

8. Let’s go back to the class on validity…
Since we didn’t quite finish our discussion…

9. Types of validity we discussed
Ecological
Construct
Predictive
Substantive
Content
Can anyone define and give a quick example of each?

10. Exercise
In groups of 3
Write the abstract of the worst EDM paper ever

11. Any group want to share?

12. Exercise #2 In different groups of 3
Now write the abstract of the best EDM paper ever

13. Any group want to share?

14. The End