1. Assessing the Impact of Implementing Quality Matters at NMSU Sharon Lalla –Las Cruces Sherrell Wheeler –Alamogordo Nicole Olsson-Dail –Carlsbad Krista MacDonald – Doña Ana Community College NMSU Alamogordo RoundUp Friday September 12th, 2014

2. Welcome! Photo by satosphere @ Flickrsatosphere

3. It’s All About Discovering… Quality Online

4. New Mexico State University – Las Cruces Sharon Lalla, New Mexico State University - Las Cruces

5. OCIP Program Goals 1. Assist faculty in enhanced web course design and delivery. 2. Provide faculty instructional design services with a framework that supports peer interaction and best practices for online learning. 3. Facilitate participation in national digital content consortiums. 4. Create and support a culture of quality for online courses.

6. Quality Matters (QM) is the theoretical and organizational framework for the OCIP program Employs continuous improvement approach Activities are thematically organized around QM Rubric One-Year-Plus Fellows (Informal QM Reviews) Subscriber-Managed Formal Peer Reviews National Course Recognition

7. QM Activities Since 2013 It’s a start…….. 8 Courses QM Nationally Recognized 59 Informal Course Reviews 33 Have Taken the Applying the QM Rubric Training 14 Are Certified Peer Reviewers 3-Year Implementation Submitted to Quality Matters NMSU-Wide Subscription Sought

8. Online Course Improvement Program (OCIP) 2013 Faculty Focus Group 2013 Student Evaluations

9. “My online courses will never be the same!” “Rubrics really work to help the alignment process.” “A start here is so simple and so helpful for students.” “I had heard of quality matters, but I really didn’t know what it was regarding so this definitely has forced me to improve my course.”

10. “Objectives started me getting to think about my past evaluations. I used to think the student did not try hard enough.” “I really took a look at alignment and made my course better. In addition, my face-to-face classes are now benefiting from my new understanding of course organization and alignment.” “Things that were obvious to me were not obvious to my students. I have to pay attention to what students say.”

11. How easy was it for you to find things in the course when needed? Very easy 30%, Easy 52% How clear were the course requirements in terms of what you were expected to learn? Clear 70%, Somewhat Clear 29% How helpful were the learning activities in promoting your learning in this course? Very Helpful 63% Do you feel that you were assessed on what you were supposed to learn in the course? Yes 89.9%

12. What’s Next?

13. New Mexico State University – Alamogordo Sherrell Wheeler, Director of Online Quality Assurance

14. Accurate Prospective “The QM peer review process is pretty intense. I might compare it to being dipped into an icy mountain lake--a bit of a shock, but stimulating and awakening at the same time.” Kathy Roark-Diehl, Professor, English and Women’s Studies

15. Grade Comparison

16. Student Survey – Spring 2014 Sent to students who were in a course that had been QM Approved Sent to 95 students 26 students responded 27% response rate on an online survey during finals week

17. Was this course easier to navigate than a typical online course you have taken in the past? “Everything was very organized and there was no question about what I had to do for the week.” “This course was clearly laid out and organized.” “Everything was laid out that made accessing assignments and documents very easy.”

18. How clear were the expectations that were set for you? “The instructor was very clear as to what he wanted and made all information about the class and assignments easily accessible.” “My instructor made everything very clear on what to expect out of this course and what he expected from us.”

19. How well did this course engage you? “It required more online communication between students.” “It was the most engaging class to date.” “I felt more obligated to follow along in the lectures and be an active part of it, rather than flying by the seat of my pants and words in a textbook.”

20. The overall design of the course was… “The organization was astounding. I had taken three other online courses this semester and this one was by far the easiest to follow.” “The design was simple and clear, couldn't be better.” “The design was great. The modules all varied but at the same time had a similar thread.”

21. General Comments “This course and the layout of it were so easy to navigate that I couldn't even use the excuse that I couldn't turn a paper in because of not knowing where a link was; everything is at your finger tips it's a solid design!” “The course really impacted me and helped me to look forward to other classes using the QMS.” “Amazing job of presenting the material in a way that I was engaged and retained the information.” “I was not that familiar with the subject and was very impressed with what I learned. I was more engaged in this course than any other course by far.”

22. General Comments “Students taking an online course often have to deal with misunderstandings, confusing due dates, and unorganized information. Using a course designed by Quality Matters™ Standards, those problems are no longer an issue. Information is exactly where it is supposed to be and in exactly the right order. Due dates are no longer confusing and there are so many less misunderstandings. Material is even so much more clear than a professor who does nothing more than create a long and boring PowerPoint presentation on exactly what is in the textbook, without taking that extra step to make the course interesting.”

23. Instructor Comments “I knew that QM would help us improve our online courses by setting a standard that we all worked to achieve. What I didn't realize is that it would open up so many conversations about online teaching. We, as faculty, are sharing ideas constantly and showing one another tips and tricks that we have learned. The best thing though is the sheer number of NMSU-A faculty and staff I have gotten to know through this process! It has opened up a dialogue with people on campus that I have never known and would never have interacted with to this extend if it hadn't been for QM! ​ ” Tanya Allred, Associate Professor of English

24. Instructor Comments “After revising my course to meet QM standards, I know I have a much better product for students. The added course overview information allows students a better understanding of what to expect from me and what I expect of them. I feel students in my course this semester were able to find information within the course, and I saw a decrease in questions about course content and navigation issues, which I took to be a good sign. The QM Rubric is a wonderful tool for improving course design.” Haley Lehoski, Part-time Instructor

25. Instructor Comments “As a faculty member, I have always been proud of my assignment design, but working through QM showed me that my instructions were not always as clear as I believed them to be. Gaining a better understanding of course navigation online has added new depth and clarity to the way I design all elements of a course, be it online or face-to- face. This new clarity is already paying off, making my role that of a facilitator of learning, and not facilitator of instructions.” Kathy Roark-Diehl, Professor, English and Women’s Studies

26. New Mexico State University – Carlsbad Nicole Olsson-Dail, Manager Instructional Technology

27. NMSU Carlsbad Implementation Faculty Professional Development –87% of Faculty have successfully completed Internal Peer Reviews –18 peer reviews conducted through September 2014 QM Subscriber-Managed or QM-Managed Peer Reviews –1 course received QM recognized

28. How has Quality Matters positively impacted your online teaching? “I now look at my classes in a very different light. I much more critical of them from the build stand point and will be looking how to increase their productivity from the student’s view” “QM has already impacted my current face-to-face course design in very positive ways. I will not only give video tutorials, but I also plan to create step by step PDFs and provide written explanations as well.” “The next online course I teach will definitely want to use the rubric during course development and then submit my course to a Quality Matters review”

29. “I think that having ALL instructors take this course would be a good idea. I plan to be more obvious in incorporating the Learning Objectives into my course materials. Many of the standards could be adapted to face-to-face classes.” “QM has offered a more focused approach for creating, designing, and delivering my courses with an appreciation for my students’ perspectives. I find that I consider my students’ positions and interpretations more often as I work through my courses each semester.” How has Quality Matters positively impacted your online teaching?

30. Student Quality Matters Standards Questionnaire “I wanted to tell you how much I appreciated the structure and organization of your class. I can tell you put a lot of work into this class and I thank you for it!” “Easy to navigate, material was nice to learn for newer PC users, and the instructor put a lot of time, not online into the course itself, but she is always willing to help students.” “The online course was very easy to navigate through and if I had other classes with this amount of energy put in, I would have no complaints in online learning!”

31. How would you rate the CS 110 course overall?

32. New Mexico State University – Doña Ana Community College Krista MacDonald, Online Education Coordinator and Assistant Professor

33. IEPO Data Student Survey Faculty Survey Faculty Focus Group

34. DACC Started QM

35. Difference Between Online and Face-to-Face Pass Rates Pre-implementation -7.17% Post implementation -4.03% DACC has cut the difference between online and face-to-face pass rates by half (46%).

36. Spring 2014 VLIT Online Education Student Survey (N=237) The course objectives and procedures were clearly communicated. 51% (109) Strongly Agree, 29%(62) Agree The overall usability of the course web site was good. 47% (102) Strongly Agree, 35% (76) Agree The course material was organized into logical and understandable components. 48% (104) Strongly Agree, 31% (68) Agree The instructor was actively involved in facilitating the course. 51% (112) Strongly Agree, 25% (55) Agree

37. VLIT Online Education Faculty Focus Group Spring 2014 Please describe how using the Quality Matters Rubric to assess your own course was helpful or not helpful. “There are many things. Most of all I’ve reorganized the layout of my course.” “The concept of alignment has had the biggest impact on my course.” “Helped with discussions…it made all the difference in the world.” “The biggest thing for me is organization and student expectation.”

38. VLIT Online Education Faculty Focus Group Spring 2014 Do you believe that your Quality Matters training affected how your students performed in your online courses? “Definitely improved retention right off the bat.” “Immediately I had fewer students asking me ‘What does that mean?’” “Retention improved and grades improved” “Pass rates increased”

39. VLIT Online Education Faculty Focus Group Spring 2014 Are you planning to assess whether the changes you made to your online courses affected student learning? “Already done it…I’m looking at the retention rates of students before and after my QM training.” “I hadn’t thought about it, but now that I have, I’ll go back and compare.” “Already done…the semester after the changes, I got straight 5’s on the online class evaluations-that was the first time that had ever happened.” “Too many factors, too hard to compare…”

40. Spring 2014 VLIT Online Education Faculty Survey (N=141) 65 DACC Faculty have designed an online course. 38 DACC Faculty have attended the DACC Quality Matters Workshops. 46 DACC Faculty have completed The Applying the Quality Matters Rubric Course. 21 DACC Faculty plan to submit an online or blended course for an official Quality Matters Review this year.

41. QUESTIONS? Thank you for attending.

42. Contact Information Sharon Lalla, Quality Matters Coordinator/Liaison New Mexico State University – Las Cruces Email: slalla@nmsu.eduslalla@nmsu.edu Nicole Olsson-Dail, Manager, Instructional Technology New Mexico State University – Carlsbad Email: nolsson@nmsu.edunolsson@nmsu.edu Sherrell Wheeler, Director of Online Quality Assurance New Mexico State University – Alamogordo Email: swheeler@nmsu.eduswheeler@nmsu.edu Krista MacDonald, Online Education Coordinator New Mexico State University – Doña Ana Community College Email: kkozel@nmsu.edukkozel@nmsu.edu