Learning Analytics How can I identify and help my struggling students sooner rather than later? How can I see which concepts students struggle with in an online test or quiz? How can I spot potential cheating on a test? How are the students spending time on the course Blackboard site? How much time are my students spending on the course Blackboard site? Is it enough? Are my students participating in discussion boards? Are they making quality posts?

Learning Analytics Presented by the D’Amore-McKim School of Business Instructional Design Group

Learning Analytics “measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Long and Siemens, 2011, p.32)

Questions What are the most helpful analytics in Blackboard? Will the analytics give you the information you need? How often do you need to look at the generated data? Important questions to ask before accessing the many analytics tools available in various locations in Blackboard

Analytics Tools Statistics Tracking Item Analysis View All Attempts Selected Course Reports Course Activity Overview Overall Summary of User Activity User Activity Inside Content Area What we are covering today – A selected list of analytical tools and reports that we think might be useful.

Statistics Tracking Must be turned on for each individual content item to be tracked Content Usage Statistics for selected users are reported: Access by date Access by hour of day Access by Day or Week Can be downloaded and saved in a variety of formats Blackboard Video Tutorial https://www.youtube.com/watch?v=ucSloN NKqng Detailed usage statistics available for selected content items in Blackboard courses. Data for Statistics Tracking is saved for only 180 days after it is generated.

Item Analysis Useful: To see how many students got the correct answer versus wrong in T/F or Multiple Choice To see which wrong answers were selected how many times in a multiple choice question To indicate a question is not understood by students or material needs to be covered more thoroughly Item analysis enables you to look deeply into the multiple choice and true/false test questions in your exams and quizzes.

View All Attempts Summary chart shows: number of attempts date and time test was completed the overall time it took to complete it Review for patterns, possible cheating. View All Attempts gives you a snapshot of an online test and all the students who took it.

Course Reports Questions to ask: Is student activity correlating with performance? Is student activity aligning with expectations/deadlines? Are parts of the course being underutilized? Shows what parts of your course students are accessing, how often and when. Data for Course Reports is saved for only 180 days after it is generated.

Course Activity Overview Check that student activity matches deadlines and times during the week you’d like students to be active. This chart would lead me to believe that there is a deadline structure with assignments due on Mondays and Thursdays. Note that the average student has spend just under ten hours in the course so far. Is this a reasonable amount of time at this point in the semester?

Course Activity Overview Check that individual student activity is up to par. Are students spending enough time in the course? Click a student’s name to obtain more detailed information. Sometimes time spent reflects efficiency more than performance. Other times, it can raise red flags. Should the average student have spent around 10 hours working on the course thus far in the semester? Note that the two students who have spent around four hours are actually doing just fine in the course. They are simply more efficient. The student with no activity actually dropped the course. If he/she hadn’t dropped, this would be concerning. The student who has spent just under two hours may also raise a red flag.

Overall Summary of User Activity Are students visiting the parts of the course you want them to visit? Ensure that students are spending adequate time in the right sections. If there were a discussion board forum in this course, I’d be concerned.

User Activity Inside Content Areas This graph shows the content areas of the course your students are visiting. Check to see that the hits correspond with where you’d like them to spend their time in the course. In this case, most content is organized in the topic folders. So it makes sense that the majority of the hits occur here. The professor in this course expects the PowerPoints section to spike as mid-terms near. Most students use them as a study tool.

User Activity Inside Content Areas Observe patterns in student activity over time. Note that in the first few weeks the course had the most hits. As students become familiar with the navigation, it’s normal for activity to drop off somewhat. They’re not spending as much time clicking around looking for course material. The peaks and troughs likely reflect weekly deadlines for course work. If they don’t, you may want to consider adjusting deadlines to reflect times when students are most active.

Taking Action Offer credit or create assignments to get important materials in front of students Reevaluate quiz questions based on performance Notify student that below average activity is affecting grade Post announcement reminding students to visit certain materials Reorganize material to increase likelihood of site visits, streamline navigation Remember: it’s OK to link to materials from multiple locations; there’s no one best way to arrive at what needs to be seen or done What data requires action? Some suggested interventions.