1. Making the Learning Analytics Processor (LAP) better, stronger, faster, and more Moodle friendly Lou Harrison Director of Educational Technology Services DELTA North Carolina State Universitylou@ncsu.edu Jeff Webster Senior Associate Director, Applications Development DELTA North Carolina State Universityjsw@ncsu.edu

2. ●Brief History ●Making Moodle data fit into a SAKAI world ●Preview of results ●Institutionalize it ●Future work O VERVIEW

3. ●Open Academic Analytics Initiative (OAAI) ○Educause Next Gen Learning Challenge (NGLC) ○Funded by Bill & Melinda Gates Foundation ●Leverage SIS & LMS data to create an open-source academic early alert system (and interventions) S OME HISTORY

4. Predictive Model worked well and was quite portable to other schools with some tuning. More info, see JAYAPRAKASH, S. M., MOODY, E. W., LAURÍA, E. J., REGAN, J. R., & BARON, J. D. (2014). EARLY ALERT OF ACADEMICALLY AT-RISK STUDENTS: AN OPEN SOURCE ANALYTICS INITIATIVE. EARLY ALERT OF ACADEMICALLY AT-RISK STUDENTS: AN OPEN SOURCE ANALYTICS INITIATIVE. JOURNAL OF LEARNING ANALYTICS, 1(1), 6-47. T HE RESEARCH

5. ●OAAI led to the Learning Analytics Processor (LAP) project part of the Apereo Learning Analytics InitiativeApereo Learning Analytics Initiative ●Exciting results, but all LMS data were based on SAKAI Models LAP IS BORN FROM OAAI

6. ●NC State Partnered with Unicon and Marist College to bring LAP to NC StateUniconMarist College ●We are a Moodle shop! ●Phase 1, Proof of Concept ●Show that LAP could work with Moodle data (and our PeopleSoft SIS data) ○Hope for reasonable predictive accuracy NC S TATE GETS INVOLVED

7. ●From LAP Data Formats Grade and Activity come from MoodleLAP Data Formats ●Grade ○pulling grade items including earned grade - easy ○determining grade weight - messy, abandoned ●Activity ○event logs made this much easier ○combined component and action H OW TO MAKE M OODLE DATA FIT

8. ●Courses use varying components of Moodle ●Combined similar events for modeling H OW TO MAKE M OODLE DATA FIT mod_resource -- viewed content.read mod_url -- viewedcontent.read mod_lesson -- viewedlesson.view mod_ncsubook -- viewedlesson.view

9. ●Our Phase 1 Proof of Concept showed us about 75% accuracy predicting at-risk students* with Recall rates 88-90% but with high false positives (25%) ○Phase 2 is currently wrapping up (with similar numbers) ○Make the LAP, more automated, bigger, badder ○More Enterprise, more nimble ○Similar results with much larger datasets *in a small dataset, of incomplete historical data I T WORKS, IT REALLY, REALLY WORKS

10. ●More model refinement ●Ability to run model on any given day (with live data) ○So when do we run it? ○Tradeoff, earlier gives more time to intervene ○Later gives more predictive power ●Cohorts (different models for different type classes) ○Maybe, if incremental improvement outweighs cost) ○Also look at other ways to make cohorts (like based on LMS penetration) K EEP PLUGGING AT IT

11. ●Review / refine Moodle event to LAP event mapping ●Test model for significance of existing unique actions ○quiz view vs. quiz reviewed ○forum add discussion vs. add post ●Examine Moodle activities for possible additional logging or more unique actions ○Several of our added activity plugins have limited events ●Include activity logs from our other learning tools M ORE FUTURE WORK

12. Lou Harrison lou@ncsu.edu Jeff Webster jsw@ncsu.edu Q UESTIONS ?