Staff and student perceptions of the Canvas Learning Management System Preliminary findings Presented by James Kite (@jkite13) Faculty of Medicine and Health, School of Public Health, Prevention Research Collaboration

Acknowledgement of Country

Context Exponential increase in online teaching & learning Every unit in the School of Public Health now has an online presence University is transitioning from Blackboard to Canvas “To support our educational goals around blended learning with engaging, interactive and collaborative learning designs” The rise of the internet has opened new opportunities for teaching & learning in online environments As access has increased and technology has improved, more and more T&L is done online Every unit at SSPH must now have an online presence through the University’s preferred learning management system Many units allow students to study fully online, where no face-to-face contact is required at all. Online only units are also increasingly common. USyd has switched its LMS to Canvas, after using BB for over a decade The switch was made “to support our educational goals around blended learning with engaging, interactive and collaborative learning designs.” An emphasis was also placed on the user experience being supportive of these goals. During the transition, staff were encouraged to ‘unlearn Blackboard’ and ‘think in Canvas’, with Canvas being billed as ‘fundamentally different’ to BB.

The evidence so far - Students Strong associations between satisfaction and learning outcomes Learning content and course design among the most influential determinants of satisfaction Issues: Communication between staff and students Self-efficacy Higher engagement possible with more sophisticated use of LMS When it comes to learning in any environment, we know that there are strong associations between student satisfaction and learning outcomes Among the most influential factors influencing student satisfaction are learning content (what students actually learn) and, importantly in this context, course design (or how the course is taught). The learning management systems (LMS) is part of this. While students report being happy with the convenience of online learning, among the most common LMS influences on student satisfaction is communication, with the amount and quality of interaction highlighted as a disappointing feature of online learning. We also know that the student’s self-efficacy when it comes to using the LMS is important, meaning that technical support and training opportunities are important. Finally, we know that more sophisticated uses of LMS (e.g. use of communication tools to facilitate student and staff interactions, collaboration tools, and the ability to tailor learning to the student’s preferred speed and interests) leads to an increase in student engagement online.

The evidence so far - Staff Limited evidence! Perceived usefulness of a LMS is associated with the extent of staff engagement Barriers to engagement: Technical issues Lack of access Lack of appropriate support Student perception is that negative attributes of LMS reflect implementation, not inherent problems with the platform This means it is important to understand what influences staff engagement with LMS but much of the research to date has been done on students, with little attention paid to staff. It does appear that the perceived usefulness of a LMS is associated with staff engagement. There is some evidence that technical issues, lack of access, and lack of appropriate support are the top barriers to staff engagement with a LMS. Importantly, students perceive the negative attributes of LMS to reflect the implementation of that LMS, and not as inherent problems within the platform itself. In other words, students are likely to believe that their satisfaction with an online learning environment is driven by how well teaching staff use the platform, making it vital that we understand what motivates and influences teaching staff’s engagement with a LMS. Students tend to view LMS more positively than academics, suggesting that they may be more willing to give online learning a go than staff are to give online teaching a go.

Aims and research questions Aim: To explore the staff and student user experience of Canvas Features that enhance or detract from learning experience Staff perceptions of design and management Staff and student experience of technical support From the available evidence, we can see that understanding the elements that influence student satisfaction with LMS and how staff engage with the LMS is vital information for improving online teaching and learning. This study will help the School improve its learning and teaching practices in online environments. In addition, this research has broader relevance to the higher education sector given the large and increasing role that e-learning has. There is also relatively little research in this space, particularly in relation to the barriers and facilitators that influence teaching staff. This research will help fill that gap.

Method Interviews with unit coordinators and students from School of Public Health Audio and screen recorded Thematic analysis Progress: 8 staff interviews completed 4 student interviews completed We are conducting interviews with staff and students from the Sydney School of Public Health. The interviews are open to any staff who have coordinated units of study in Semester 1, 2018 or are coordinating units in Semester 2. Students are eligible to participate if they completed any units of study offered by SSPH in Semester 1, 2018, regardless of what degree program they are doing, how they study (predominately online, mixed, or predominately face-to-face), whether they are enrolled full- or part-time, or their academic performance in the units in question. Staff interviews explore their experience of designing and managing their units, including their perceived competence in using Canvas, features of Canvas that they like and dislike, how Canvas compares to Blackboard, and their experience with available support. Student interviews examine their experience of using Canvas, including how they use Canvas in their study, features of courses that they like or dislike, and, where relevant, their previous experience with Blackboard. Has been a mix of staff interviewed so far, with their unit size and structures varying considerably. However, all of the student interviews were with students who study predominately face-to-face.

Staff results Improved aesthetics and navigation Limited interaction through Canvas Managing larger classes is harder than BB Minimal use of design and editing features Agreement that there is improved aesthetics and more intuitive navigation compared to BB but overall, not much difference in perceived functionality and use of Canvas. There is generally limited interaction through Canvas, with most units designed as a repository of information i.e. students take what they need out of Canvas and return to complete or submit assessments. Where there is interaction, asynchronous discussion boards seem to be the most common tool. However, these are perceived as not particularly conducive to actual ‘discussion’. They seem to function more as a Q&A session, with limited interaction between students. For those units that have used web conferencing (e.g. Zoom, Adobe Connect), there is a perception that they create a much better environment for generating genuine discussion but there are underlying concerns about the reliability of connection. Managing larger classes is perceived as harder than BB, principally because of the lack of analytics and the complexity of establishing and managing sub-groups of students. The lack of analytics is particularly problematic when it comes to identifying and managing struggling students, which is something staff want to be able to do. Most of the staff interviewed made minimal use of design and editing features. This seems to be partly because it is perceived as more complex than BB, which has led to an over-reliance on ELSU to do even relatively basic editing. It also seems to reflect a lack of time available to learn the skills required.

Student results Improved aesthetics and navigation Limited interaction through Canvas Communication is better through other means Bear in mind that so far, I have only spoken to F2F students Again, there is limited interaction through Canvas. These students are predominately using Canvas as a repository of information i.e. a place they go to find and download information and only return when they have to (e.g. to submit assignments). There doesn’t appear to be any desire for increased interaction through Canvas, either, as the perception is that they learn better in F2F environments. Communication with staff and other students is done in other environments, particularly F2F (in class) and via email. Discussion boards are not a popular means of communication because the stilted nature of the discussions are not conducive to genuine discussion. As a result, when forced to use discussion boards (e.g. for assessment purposes), students tend towards only doing the minimum required.

Preliminary thematic map Harder class management Limited interest in design and editing Staff Staff and students Interaction is easier off Canvas Canvas as information repository Aesthetics are better

Summary and implications Limited use of online environment Fine for predominately face-to-face students Online students? Need e-learning pedagogical specialists – not just tech support Options for managing large classes needs further exploration Both staff and students were generally happy with the functionality of Canvas. There was agreement that the differences between Canvas and BB were minimal, although Canvas is perceived as better looking and easier to navigate. However, it appears that the sites are being designed and used as repositories. This seems to be fine for F2F students, as it meets their expectations but it will be interesting to see what online students have to say. Given the limited interest in design and editing among staff, it may be necessary to employ specialists in e-learning who can help with pedagogical issues and not just function as tech support.

Email: james.kite@sydney.edu.au Twitter: @jkite13 Contact me Email: james.kite@sydney.edu.au Twitter: @jkite13 Also: Careers Night in Sydney, 1 November