1. Staff and student perceptions of the Canvas Learning Management System Preliminary findings
Presented by
James Kite
Faculty of Medicine and Health, School of Public Health, Prevention Research Collaboration

2. Acknowledgement of Country

3. Context Exponential increase in online teaching & learning
Every unit in the School of Public Health now has an online presence
University is transitioning from Blackboard to Canvas
“To support our educational goals around blended learning with engaging, interactive and collaborative learning designs”
The rise of the internet has opened new opportunities for teaching & learning in online environments
As access has increased and technology has improved, more and more T&L is done online
Every unit at SSPH must now have an online presence through the University’s preferred learning management system
Many units allow students to study fully online, where no face-to-face contact is required at all. Online only units are also increasingly common.
USyd has switched its LMS to Canvas, after using BB for over a decade
The switch was made “to support our educational goals around blended learning with engaging, interactive and collaborative learning designs.” An emphasis was also placed on the user experience being supportive of these goals.
During the transition, staff were encouraged to ‘unlearn Blackboard’ and ‘think in Canvas’, with Canvas being billed as ‘fundamentally different’ to BB.

4. The evidence so far - Students
Strong associations between satisfaction and learning outcomes
Learning content and course design among the most influential determinants of satisfaction
Issues:
Communication between staff and students
Self-efficacy
Higher engagement possible with more sophisticated use of LMS
When it comes to learning in any environment, we know that there are strong associations between student satisfaction and learning outcomes
Among the most influential factors influencing student satisfaction are learning content (what students actually learn) and, importantly in this context, course design (or how the course is taught). The learning management systems (LMS) is part of this.
While students report being happy with the convenience of online learning, among the most common LMS influences on student satisfaction is communication, with the amount and quality of interaction highlighted as a disappointing feature of online learning.
We also know that the student’s self-efficacy when it comes to using the LMS is important, meaning that technical support and training opportunities are important.
Finally, we know that more sophisticated uses of LMS (e.g. use of communication tools to facilitate student and staff interactions, collaboration tools, and the ability to tailor learning to the student’s preferred speed and interests) leads to an increase in student engagement online.

5. The evidence so far - Staff
Limited evidence!
Perceived usefulness of a LMS is associated with the extent of staff engagement
Barriers to engagement:
Technical issues
Lack of access
Lack of appropriate support
Student perception is that negative attributes of LMS reflect implementation, not inherent problems with the platform
This means it is important to understand what influences staff engagement with LMS but much of the research to date has been done on students, with little attention paid to staff.
It does appear that the perceived usefulness of a LMS is associated with staff engagement.
There is some evidence that technical issues, lack of access, and lack of appropriate support are the top barriers to staff engagement with a LMS.
Importantly, students perceive the negative attributes of LMS to reflect the implementation of that LMS, and not as inherent problems within the platform itself. In other words, students are likely to believe that their satisfaction with an online learning environment is driven by how well teaching staff use the platform, making it vital that we understand what motivates and influences teaching staff’s engagement with a LMS.
Students tend to view LMS more positively than academics, suggesting that they may be more willing to give online learning a go than staff are to give online teaching a go.

6. Aims and research questions
Aim: To explore the staff and student user experience of Canvas
Features that enhance or detract from learning experience
Staff perceptions of design and management
Staff and student experience of technical support
From the available evidence, we can see that understanding the elements that influence student satisfaction with LMS and how staff engage with the LMS is vital information for improving online teaching and learning.
This study will help the School improve its learning and teaching practices in online environments. In addition, this research has broader relevance to the higher education sector given the large and increasing role that e-learning has. There is also relatively little research in this space, particularly in relation to the barriers and facilitators that influence teaching staff. This research will help fill that gap.

7. Method
Interviews with unit coordinators and students from School of Public Health
Audio and screen recorded
Thematic analysis
Progress:
8 staff interviews completed
4 student interviews completed
We are conducting interviews with staff and students from the Sydney School of Public Health. The interviews are open to any staff who have coordinated units of study in Semester 1, 2018 or are coordinating units in Semester 2. Students are eligible to participate if they completed any units of study offered by SSPH in Semester 1, 2018, regardless of what degree program they are doing, how they study (predominately online, mixed, or predominately face-to-face), whether they are enrolled full- or part-time, or their academic performance in the units in question.
Staff interviews explore their experience of designing and managing their units, including their perceived competence in using Canvas, features of Canvas that they like and dislike, how Canvas compares to Blackboard, and their experience with available support.
Student interviews examine their experience of using Canvas, including how they use Canvas in their study, features of courses that they like or dislike, and, where relevant, their previous experience with Blackboard.
Has been a mix of staff interviewed so far, with their unit size and structures varying considerably.
However, all of the student interviews were with students who study predominately face-to-face.

8. Staff results Improved aesthetics and navigation
Limited interaction through Canvas
Managing larger classes is harder than BB
Minimal use of design and editing features
Agreement that there is improved aesthetics and more intuitive navigation compared to BB but overall, not much difference in perceived functionality and use of Canvas.
There is generally limited interaction through Canvas, with most units designed as a repository of information i.e. students take what they need out of Canvas and return to complete or submit assessments.
Where there is interaction, asynchronous discussion boards seem to be the most common tool. However, these are perceived as not particularly conducive to actual ‘discussion’. They seem to function more as a Q&A session, with limited interaction between students.
For those units that have used web conferencing (e.g. Zoom, Adobe Connect), there is a perception that they create a much better environment for generating genuine discussion but there are underlying concerns about the reliability of connection.
Managing larger classes is perceived as harder than BB, principally because of the lack of analytics and the complexity of establishing and managing sub-groups of students. The lack of analytics is particularly problematic when it comes to identifying and managing struggling students, which is something staff want to be able to do.
Most of the staff interviewed made minimal use of design and editing features. This seems to be partly because it is perceived as more complex than BB, which has led to an over-reliance on ELSU to do even relatively basic editing. It also seems to reflect a lack of time available to learn the skills required.

9. Student results Improved aesthetics and navigation
Limited interaction through Canvas
Communication is better through other means
Bear in mind that so far, I have only spoken to F2F students
Again, there is limited interaction through Canvas. These students are predominately using Canvas as a repository of information i.e. a place they go to find and download information and only return when they have to (e.g. to submit assignments). There doesn’t appear to be any desire for increased interaction through Canvas, either, as the perception is that they learn better in F2F environments.
Communication with staff and other students is done in other environments, particularly F2F (in class) and via .
Discussion boards are not a popular means of communication because the stilted nature of the discussions are not conducive to genuine discussion. As a result, when forced to use discussion boards (e.g. for assessment purposes), students tend towards only doing the minimum required.

10. Preliminary thematic map
Harder class management
Limited interest in design and editing
Staff
Staff and students
Interaction is easier off Canvas
Canvas as information repository
Aesthetics are better

11. Summary and implications
Limited use of online environment
Fine for predominately face-to-face students
Online students?
Need e-learning pedagogical specialists – not just tech support
Options for managing large classes needs further exploration
Both staff and students were generally happy with the functionality of Canvas. There was agreement that the differences between Canvas and BB were minimal, although Canvas is perceived as better looking and easier to navigate.
However, it appears that the sites are being designed and used as repositories. This seems to be fine for F2F students, as it meets their expectations but it will be interesting to see what online students have to say.
Given the limited interest in design and editing among staff, it may be necessary to employ specialists in e-learning who can help with pedagogical issues and not just function as tech support.

12. Email: james.kite@sydney.edu.au Twitter: @jkite13
Contact me
Also: Careers Night in Sydney, 1 November