1. Making Course Status Visible: The UCF Online Student Performance Dashboard
Thomas Cavanagh, Associate Vice President
Center for Distributed Learning
UCF

2. UCF Orlando, FL Metropolitan, suburban university 64,000+ students
One of the largest universities in U.S.
Carnegie classification: RU/VH Research University: Very High Research Activity
216 degree programs across 11 colleges
11 Campuses throughout Central Florida

3. Academic Year 2016-2017 42.2% total university SCH online/blended
32.5% fully online SCH
81% of all students took at least one online/blended course
83.7% of all undergraduates
62.8% of all graduate students
72% of all students took at least one fully online course
74.9% of all undergraduates
52.6% of all graduate students

4. Course Term Year Degree
STUDENT SUCCESS
Courtesy of Joel Hartman

5. Predictive Analytics Toolkit
UCF-generated analyses
Civitas Learning
EAB SSC-Campus
UCF-developed Student Dashboard
Courtesy of Joel Hartman

6. EAB SSC-Campus

7. Civitas Illume App

8. LMS Data Increases Predictive Power
Comprehensive Data and Modeling
Canvas is a rich data set:
>90% of students in Illume have Canvas data
78 of 427 predictors in Illume come from Canvas data
Strong Model Prediction: 86% Area Under the Curve
LMS Data is Predictive of Persistence at UCF
1 of the top 10 predictors for students overall is their activity on Canvas
2 of top 10 predictors for students taking at least one online course is their Canvas activity
Courtesy of Joel Hartman

9. UCF Student Performance Dashboard

10. Fall Survey Results (n=96)
The majority of respondents were juniors (54%) or seniors (24%), expecting As (41%) or Bs (49%)
41% of students never used the dashboard, but 25% used it daily.
Of the very few students (n=28) who offered open ended responses as to why they did not use the dashboard, the overwhelming majority indicated a lack of awareness regarding the dashboard rather than dislike of the dashboard.
The dashboard was reported as being easy or very easy to use by 79% of respondents (n=52), with the ‘assignments due soon display’ and ‘current score display’ being the most utilized and most helpful features.
The ‘goals display’ was rated as the least helpful feature by 20% of the respondents.

11. Fall Survey Results (n=96)
Most (72%) of the respondents felt the dashboard was somewhat or very accurate, although 15% were unsure.
Students expressed concerns over grades being displayed as artificially low during the beginning of the semester as a lack of grades input made it appear as if they were failing.
Students liked the accessibility and visualization of the dashboard, especially for keeping up to date with assignments and course performance.
57% of students who used the dashboard believed it motivated them to change their behavior either a little or a great deal in regards to the course.

12. Questions
Thomas Cavanagh, Ph.D.
@tbcavanagh