1. Design your Canvas Course to Leverage Current Analytics Tools
PDI 2017
Melody Brake
Stan Kruse
Kevin Nolan
Cale Roger
January 2017

2. Agenda: Highlights of these tools & how to design courses in Canvas to leverage them.
Overview
Canvas Course Analytics
Canvas Student Analytics
Student Access Report
Discussion Participation Count
Canvas Quiz Analytics
Video Analytics
Course Statistics
Student Interactions
Summary
January 2017

3. Overview - Canvas Analytics
What are they and how can instructors use them?
Analytics is the discovery, interpretation, and communication of meaningful patterns in data.
from wikipedia
Canvas records student interactions and provides analytics for instructors.
Interpretation of these analytics is unique to each course.
Potential to provide:
a view of how students are performing in comparison to their peers.
indicators of which students are excelling and which may be falling behind.
benchmarks & measures of achievements when used with Canvas Outcomes.
indicators where improvements can be made in course content for students.
January 2017

4. Overview: Design Considerations for Canvas Courses & Analytics
Add Due Dates to Assignments
Select “Online” Submission Type
Include Participation Activities
Use Canvas Messages Inbox
Use Distinct Titles on All Content
For quizzes, use measured question types
Useful and Universal Video Metrics and Strategies
Use Canvas Assignments, Messages, & Comments.
January 2017

5. COURSE Analytics How do Instructors view them in their Course?
In any course, from Home, click “View Course Analytics”.
January 2017

6. COURSE Analytics Dashboard
Table views
Graph view
OR
January 2017

7. COURSE Analytics Dashboard Details
Relies on DUE DATES
Activity = Page views + Participation
Submissions: Quizzes & Assignments
Grades: a box and whisker plot show the distribution of grades in the course.
January 2017

8. COURSE Analytics Dashboard - A. “Activity by Date”
Shows when students viewed pages or participated in the course.
= Page views + Participation
Count
Includes Page Views
Weeks
Hover over a bar for details.
Time: generally weeks (may be days or months, depending on time span of course)
Course “Participation” includes:
submits a quiz (student)
starts taking a quiz (student)
submits an assignment (student)
update a calendar event's settings or description (instructor)
updates an assignment's settings or description (instructor)
loads a collaboration to view/edit a document
joins a web conference
posts a new comment to a discussion
creates a wiki page
January 2017

9. COURSE Analytics Dashboard – B. “Submissions”
Shows the status of each Graded Assignment in the course.
Relies on due dates
% Students
Each Assignment
Hover for details: title, due date, and %.
Ordered by due date.
Without due date, submissions are “On Time”.
Only “online submission” types display a column.
January 2017

10. COURSE Analytics Dashboard – C. “Grades”
Shows the median, high, and low scores for an assignment.
# Points
Flat line = same score for all
Each Assignment
Hover for title, high/median/low/possible points
Vertical blue line = lowest to highest scores
Blue Box = 25th to 75th percentile
Horizontal black line = median score
January 2017

11. STUDENT Analytics – Overall - Student Analytics
Student Analytics are always shown in a table format and
displays an overview of each student’s participation in the course.
Sum of all Student Analytics = Course Analytics
Sort (ascending/descending) to help evaluate student performance by Student, Page Views, Participation, Current Score.
Click on a student name for their details.
January 2017

12. STUDENT Analytics – for 1 student
Student name & their current total at top.
Can message student by clicking the mail icon (beside total).
Move to next student from dropdown (top right).
Can view by graph or table.
January 2017

13. STUDENT Analytics – for 1 student: “Activity by Date”
= Page Views + Participation
Activity by Date
Includes Page Views
Student “Participation” includes:
loads a collaboration to view/edit a document
joins a web conference
posts a new comment to a discussion
creates a wiki page
submits a quiz
starts taking a quiz
submits an assignment
January 2017

14. STUDENT Analytics – for 1 student: “Communication”
COMMUNICATION between student & instructor include Conversations in the Canvas Inbox.
(Just Messages, not Announcements or Comments.)
Hover for date & # messages.
January 2017

15. STUDENT Analytics – for 1 student: “Submissions”
Each row = 1 assignment & shows date & status of each submission.
Due date = circle, triangle, square
Line = how long before/after due date it was submitted.
Green dot only = submitted on due date.
Yellow triangle = late
Red square = not submitted
White circle = future due date.
Hover for 1) assignment name & due date/time, 2) submission date/time, 3) score.
January 2017

16. STUDENT Analytics – for 1 student: “Grades”
Shows assignment median, high, low, and score class rank indicator compared to peers.
Vertical black line = highest to lowest scores
Grey box extends 25th to 75th percentile
Horizontal black line = median score
Student’s score relative to peers = colored circle, triangle, square
High, Average, Low
Scores
Assignments
January 2017

17. STUDENT Analytics – for 1 student: Table format
January 2017

18. Course and Student Analytics - Design Considerations
Add Due Dates
Select “Online” Submission Type
Include Participation Activities
Use Canvas Messages Inbox
January 2017

19. Course and Student Analytics - Design Considerations
Add Due Dates
Add a due date to each Assignment
Do not create an Assignment for activities that are not graded such as readings; place those as items in Modules instead.
January 2017

20. Course and Student Analytics - Design Considerations
2. Select “Online” Submission Type
For Assignments, select the Submission Type: “Online”, which includes:
Other types can Not display analytics:
January 2017

21. Course and Student Analytics - Design Considerations
3. Include Participation Activities
Activities that qualify as “Participation”:
loads a collaboration to view/edit a document
joins a web conference
posts a new comment to a discussion
creates a wiki page
submits a quiz
starts taking a quiz
submits an assignment
January 2017

22. Course and Student Analytics - Design Considerations
4. Use Canvas Messages Inbox
Use Messages/Conversations via the Canvas Inbox.
Do not use external .
Note: Announcements or Comments
are Not included in Communications Analytics.
January 2017

23. Student ACCESS Report - from People
See a summary of student views & participation in an individual Access Report. How to open…
January 2017

24. Student ACCESS Report - from People
Access Report shows one student’s Views and Participation, summed by content item, and listed in “Last Viewed” order.
This is Not a chronological log.
Assignment icon displays by graded & ungraded assignments.
ICONS:
Assignment
Discussion
Pages
Attachments
January 2017

25. Discussion Participation Counts
Button at top of Discussions page
in Canvas course.
January 2017

26. Student Access & Discussion Count – Design Considerations
5. Use Distinct Titles on All Content
The Student Access Report groups views and participation by Content Item.
Discussion Count groups posts by discussion title.
Distinct titles help define which items the student was accessing in a specific Announcement, Discussion, Page, or Attachment.
January 2017

27. Quick Look at Quiz Analytics
Turn on Access Log.
-Settings, Features Options
-Can turn on after the fact.
-Kept for 6 months
View in SpeedGrader
January 2017

28. Quiz Access Logs
January 2017

29. Quiz Statistics
January 2017

30. Quiz Statistics
Breakdown by question - % who answered correctly, discrimination index
January 2017

31. Quiz Statistics
Download Student Analysis .csv file of students and answers
January 2017

32. Quiz Statistics
Download Item Analysis.csv file which contains statistical information for each quiz question.
Correct and wrong ratios, variance, std. dev., Biserial information, Difficulty Index, etc.
January 2017

33. Quiz Statistics - Design Considerations
6. Understand and use question types that are Measured, when appropriate.
Turn on the Quiz Access Log
Question types measured:
True/False
Multi-Choice
January 2017

34. Video Analytics Overview
Video is supported in Canvas using third party vendors such as YouTube, Vimeo, Kaltura, Echo360, Tech Smith, etc.
Video data strategies are not technology dependent, use any third party vendor that works for your goals
Video vendor data may require LTI if you want video information (such as student user data) passed to Canvas
January 2017

35. Video Analytics: Common Video Data
January 2017

36. Video Analytics: Emerging Video Data
Heat Map – User Engagement
Playlist – User Engagement
January 2017

37. Video Analytics: Video Experiences
Single Video – Multiple Interactions
Canvas and Video Interactions
January 2017

38. 7. Video Analytics: Design Considerations
Design and Data Considerations
Improved user engagement (moving from passive playback to metadata driven video clips)
Specificity of results (qualitative)
Better correlations between videos watched and user performance/value of experience
Fun (for designers, instructors and students)
January 2017

39. Course STATISTICS Course Statistics provide COUNTS of: Discussions
Assignments
Active students
Quizzes
Files
January 2017

40. Course STATISTICS – 4 tabs
January 2017

41. Course STATISTICS – 4 tabs: Totals
In the Totals tab, view running totals of
Discussions
Assignments
Active students
Quizzes
January 2017

42. Course STATISTICS – 4 tabs: Assignments
In the Assignments tab, view counts of assignments including:
submission type,
number of assignments
number of submissions.
These tally separately there
January 2017

43. Course STATISTICS – 4 tabs: Students
In the Students tab, view all students who have recently logged in to Canvas.
Click the name of a student
to view user details.
January 2017

44. Course STATISTICS – 4 tabs: Students
The user details page links to the student’s individual reports.
We are full circle…
We already saw all these reports.
January 2017

45. Course STATISTICS – 4 views: File Storage
In the File Storage tab, view the allotted storage for your course, the number of unique uploaded files, and the number of unique media files in your course.
Please note that any files from a previous course import or course copy is not included in the storage quota or files count.
January 2017

46. Course Statistics - Design Considerations
Nothing to design here - it just counts your stuff.
January 2017

47. “Student Interactions Report” - Open 2 ways
> Dashboard > Grades > Student Interactions Report OR
> In a course > People > Gear > Student Interactions Report
January 2017

48. “Student Interactions Report” – All students listed
Teacher Activity Report for Doug Roberts
History 101
Interactions = conversations or comments on an assignment.
Sortable.
Can message student from envelope icon.
Each Teacher, Asst Teacher, TA can view others’ interaction reports:
from People, select the other person > Student Interactions Report
January 2017

49. “Interactions With You” – Individual student with you
To open: People > Click on Student name > select “Interactions With You”
Can message student from envelope icon.
Can open full Student Interaction Report with link at bottom.
January 2017

50. “Interactions With You Report” - Design Considerations
8. Use Canvas Assignments, Messages, & Comments.
January 2017

51. Summary: Design Considerations for Canvas Courses & Analytics
Add Due Dates to Assignments
Select “Online” Submission Type
Include Participation Activities
Use Canvas Messages Inbox
Use Distinct Titles on All Content
For quizzes, use measured question types
Useful and Universal Video Metrics and Strategies
Use Canvas Assignments, Messages, & Comments
January 2017

52. Resources canvas.colostate.edu CSU Canvas Support:
Canvas guides:
January 2017