1. An Approach to Skill Mapping in Online Courses Presenters: Norman Bier, Carnegie Mellon University, http://oli.cmu.edu Ross Strader, Stanford University, http://oli.stanford.edu Dawn Zimmaro, Stanford University, http://oli.stanford.edu

2. Agenda  Overview of the skills map process  Theoretical learning model used to predict mastery  Current analytics reports  Next steps for future improvements  Implications for improving student learning and courses  Facilitated discussion

3. Overview of the skills map process Activities (resources) linked to skills Skills linked to learning objectives

4. Theoretical learning model  Bayesian hierarchical statistical model  Takes as inputs data about student performance on questions and activities related to a certain skill, and estimates the probability that the student has mastered that skill.

5. Analytics reports  Instructor dashboard gives instructors a live view of student performance as they work

6. Data-driven improvement Learning Curves in DataShop

7. Future directions  Delivering OLI courses on OLI, OpenEdx, and other platforms  Creating ability to author skills maps outside of a platform  Developing outcome analytics service outside of a platform to create and test one or more learning models  Designing customizable data visualizations  Creating data analytics tools for instructors and researchers

8. Implications for improvements in student learning and courses  Design process prompts instructors to think in terms of what they want their students to know or be able to do (learning objectives and skills) and to create activities to address those objectives  Real-time data is shared with instructors and students to guide instructional interventions  Analytics focus on direct measures of learning versus proxies (outcome vs. performance/event analytics)  Data is used to iteratively improve instructional materials  For a copy of the white paper “An Approach to Knowledge Component / Skill Modeling in Online Courses”: http://oli.stanford.edu/news/http://oli.stanford.edu/news/

9. Questions?  Norman Bier: nbier@cmu.edunbier@cmu.edu  Ross Strader: strader@stanford.edustrader@stanford.edu  Dawn Zimmaro: dzimmaro@stanford.edudzimmaro@stanford.edu