7th Annual QM National Conference The Noel Levitz Priority Survey for Online Learners at NMSU: A Baseline for Future Research 7th Annual QM National Conference November 1-4, 2015 Sharon Lalla, Ed.D. & Susie Bussmann, Ph.D. Instructional Innovation & Quality (IIQ), New Mexico State University

Start a research project to get a baseline about online course quality and services

Research Questions What are student perceptions about quality in the online courses that have been influenced by QM versus NonQM? How satisfied are our online learners with issues that are really important to them at NMSU in the areas of institutional perception, academic services, student services, instructional services, enrollment services, and course quality?

Our Alternative Hypothesis… Courses influenced by Quality Matters will have a positive effect on quality at NMSU

Nov. 2014 Nov. 2014 How did we start? Get Buy-in from Colleges (Pilot) Add 4 questions Nov. 2014

How did we start? Jan. 2015 Feb.- May Submit IRB Application Collaborate with Noel Levitz; Distribute Survey Feb.- May Instutional Analysis IA department, integrity and anonymity

The Instrument: Noel Levitz PSOL 26 NL Questions 6 QM and 4 NMSU Questions Importance to me 1 - not important at all 2 - not very important 3 - somewhat unimportant 4 - neutral 5 - somewhat important 6 - important 7 - very important N/A - does not apply My level of Satisfaction 1 - not satisfied at all 2 - not very satisfied 3 - somewhat dissatisfied 4 - neutral 5 -somewhat satisfied 6 - satisfied 7 - very satisfied N/A - not available/not used The Noel Levitz Priority Survey for Online Learners at NMSU

Questions about Quality QM Questions This institution’s online courses are of high quality. Courses have clear instruction about getting started and where to begin. Technologies required in my courses are readily available, provided or easily downloaded. Navigation in the online courses is logical, consistent, and efficient. Course learning objectives, and instructions on how to meet them, are made clear. Courses are well-designed, readable and not distracting.

Questions about Quality Our Questions: The purchase of course materials is a worthwhile investment. The learning experience is engaging and supportive. Courses use a variety of instructional materials such as audio, video, transcripts, etc. Course assignments are meaningful and relevant.

Target Population 3,270 online learners in Business, Health & Social Services, and Education Colleges, Spring 2015 Susie 1-10: Last Slide for Susie Two surveys – can we see noticable Impact on qm and non qm Survey distributed to three colleges in Spring 2015 533 out of 3,270 students responded to both surveys NonQM: 424 out of 2750 (15%) response QM: 109 out of 520 (21%) responses

Data Sample N=533 (16%) out of 3,270 Responses 109/520 (21%) Two surveys – can we see noticable Impact on qm and non qm Survey distributed to three colleges in Spring 2015 533 out of 3,270 students responded to both surveys NonQM: 424 out of 2750 (15%) response QM: 109 out of 520 (21%) responses

Demographics Female 73% 19-34 Age 63% Masters 34% Rent or Own 79% Dona Ana County 63% Online 54% Single 62% 4=Year 43% Full-time 48% 19-34 Age 63%

Satisfaction with Online Services The smaller the gap, the greater the satisfaction

Online Services Strengths Registration for online courses is convenient. Billing and payment procedures are convenient for me. The technologies required in my courses are readily available, provided or easily downloaded. Instructional materials are appropriate for program content. Challenges The institution's online courses are of high quality. Faculty provide timely feedback about student progress. There are sufficient offerings within my program of study. The quality of online instruction is excellent.

Qualitative Responses Please enter any comments you would like to share with this institution. 154 (29%) responses

Findings: About Quality History of Online Courses at NMSU Conflicted Responses Excellent, Fair, Poor Across colleges “I do not think that this survey is set up very well to obtain accurate results. Half my classes have great instructors, who respond timely and have very nicely set up online classes. Other instructors are terrible and do not respond and often times the online course conflicts with the syllabus. As a result, it is difficult to answer about all classes with just one question. I either give a higher rating than some online courses deserve or a lower rating than others deserve. “

Learner Voice on “Quality” Course Organization Instructor Feedback Instructor Interaction Instructor Response Instructor (Social) Presence Instructor Engagement Work Overload Canvas Training Online Teacher Training (Design/Delivery) Consistency Time put in to class Validates importance of QM and some not qm

Learner Voice on “Quality” (cont’d.) Course Organization Some of the courses have been well organized and others have not been and have been difficult to navigate. Instructor Feedback On a positive note though, my instructor has been very timely with the grading of assignments and always provides practical and meaningful feedback. Instructor Interaction I have VERY good and VERY bad experiences in online classes. Some classes are perfectly fine, some are exceptionally great, and a couple have been wastes of time; incorporating out of date textbooks, lack of professor-student interaction/feedback. Time put in to class

Learner Voice on “Quality” (cont’d.) Instructor Response For example, the instructors do not reply to emails in a timely manner or at all (I have yet to receive replies from my current professors on emails I sent several weeks ago… Instructor (Social) Presence I have been very pleased with the program thus far. The teachers and staff are excellent and I loved coming down to see the University even though I am taking my online classes in Canada. I feel connected to the University and am so pleased I choice NMSU. Instructor Engagement Many faculty are not very engaged in student learning. Time put in to class

Learner Voice on “Quality” (cont’d.) Work Overload The on-line classes seem to require so very much work that it is really hard to take a full-time schedule compared to taking in-class classes on campus since I have done both. Canvas Training My only other pet peeve, which should not be misconstrued to be less irritating than the previous, is PROFESSORS WHO DON'T KNOW HOW TO USE CANVAS OR EVEN BE AN ONLINE INSTRUCTOR!!! Online Teacher Training (Design/Delivery) A few instructors might benefit from training on how to facilitate online courses and the importance of timely feedback. Time put in to class Gap Analysis Strengths in online services Challenges in online services Mixed qualitative responses regarding quality of courses

Results of Goal/Hypothesis… Goal: Start a research project to get a baseline about online course quality and services ✓ About online course services ✓ Student perceptions of quality Alternative Hypothesis: Courses influenced by Quality Matters will have a positive effect on quality Unproven

Closing the Loop Disseminate the results to administration, faculty, students. Disseminate the results to academic support, student support, instructional, and enrollments services for service improvement. Add online teaching to our professional dev. Revisit lens for further research.

What’s Next? Redesign: Pilot study in Business college Research trends in retention, student satisfaction, and grade improvement in QM recognized courses Work with NMSU System-Wide QM Coordinators to evaluate COP Possibly conduct Noel Levitz PSOL for entire university

Thank you for attending. Questions?

New Mexico State University Contact Information Sharon Lalla, Co-Principal Investigator Email: slalla@nmsu.edu Susie Bussmann, Co-Principal Investigator Email: suceppib@nmsu.edu New Mexico State University