Presented by Marie Lippens Engaging faculty in learning analytics. There’s a qualitative side too! Presented by Marie Lippens

Outline What is Learning Analytics Anyway? The Role of the Teacher Data Results Let’s Do This!

Learning Analytics…? Learning Analytics (LA) has been defined as the “measurement, collection, analysis, and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs” (Siemens 2013). What is still unclear: how are we to collect, manage, interpret and use this data (this depends on who you ask…) In Practice, LA allows for data-driven continual improvement measures Image: http://www.edtech.ku.edu/new/courses/760/sessions/session_12/instruction.shtml

Research is Outpacing Practice The MAJORITY of research on the topic of Learning Analytics does not involve teachers. Rather, there is an abundance of high level and computer science-based work. We’re missing out on data gathering on the qualitative side of the learning experience. And we’re missing out on what teachers can do with LA right now.

Data-Informed Decision Support Teacher Inquiry: Teachers engage in reflective activities to strengthen their practice, and share their experiences with peers. Learning Design (LD): Sharing tried- and-true methods of design principles in teaching and learning with technology.

Methodology Study Design Mixed methods exploratory case study Sample Multiple iterations of two established online graduate courses taught by long-standing, highly experienced faculty.

Methodology Step 1: Quantitative Analysis of LMS data → Extract LMS data in the form available to faculty → Analyze according to: Learning Design Visualizations As described in Lockyer et al, 2013 and supported in Bakharia, Corrin, de Barba, Kennedy, Gašević, Mulder, Williams, Dawson and Lockyer (2016) Total log counts (as proxy for total presence) See Whitmer’s study (as cited in Jisc, 2016) Discourse Analysis using the Community of Inquiry (CoI) framework See analytics models: Rockinson-Szapkiw, Wendt, Whighting & Nisbet (2016), and original framework by Garrison, Anderson & Archer (1999)

Methodology, continued Step 2: Qualitative Analysis – insight from faculty LMS data representations were provided to faculty with no explanations, then an open-ended interview was conducted Step 3: Compare/contrast findings from the data and insights from faculty. Goal: Look for practical implications emerging from this process

Results Important highlights: Faculty believe that their regular use of the crude LMS data is beneficial to their practice for both monitoring and reflection Alignment between learning design and user activity was clearly evident Variations in user activity (presence) was explained by faculty in surprising ways. Most notable was flexibility provided to students which was vital to the success of some, and various technology and communication preferences that caused uneven data capture. The LMS captured only a fraction of learning activities

Results There’s more! A rich CoI was evident with high levels of the three presences (cognitive, social and teaching). The dynamics (establishment and timing) of the three presences were consistently unique in each course. Faculty insight made it clear that CoI differences were influenced by the teacher’s leadership approach. The visualizations provided to faculty were not well understood. Each faculty had their own ideas about what they wanted to see, supporting a call for on-demand analytics.

Let’s make a Change! Next moves… Gašević, Dawson, Rogers and Gasevic (2016) study the extent to which instructional conditions influence predictions of academic success and caution that generalized models miss this important parameter. Next moves… Give teachers the resources to: inform their practice share their experience Remove barriers by providing: Technology support Academic freedom Time and incentive to foster a community of practice Image: http://change-challenge.blogspot.ca/

A Few References.. Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., Williams, D., Dawson, S. & Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. LAK ’16 Conference Proceedings: The Sixth International Learning Analytics & Knowledge Conference, pp 329-338. Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in text-based environments: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87–105. Gašević, D., Dawson, S., Rogers, T. & Gasevic, D. (2016). Learning analytics should not promote one size fits all: the effects of instructional conditions in predicting academic success. The Internet and Higher Education 28, 68– 84. Kovanović, V., Gašević, D., Joksimović, S., Hatala, M. & Adesope, O. (2015). Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussions. Internet & Higher Education, (27), 74-89. doi:10.1016/j.iheduc.2015.06.002 Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230-248. doi:10.1111/bjet.12207 Rockinson-Szapkiw, A., Wendt, J., Whighting, M. & Nisbet, D. (2016). The Predictive Relationship Among the Community of Inquiry Framework, Perceived Learning and Online, and Graduate Students’ Course Grades in Online Synchronous and Asynchronous Courses. The International Review of Research in Open and Distributed Learning, 17(3).

Why the CoI? In the context of graduate level online learning, a collaborative-constructivist model applies, which is the basis of the Community of Inquiry (CoI) Model (Garrison, Anderson and Archer, 1999). Supplementary slide in case of audience query RE: CoI Validation support of the CoI model notes the dual nature of the element of teaching presence, with course design and instructor behaviour emerging as separate and related factors Image: https://coi.athabascau.ca/coi-model/