1. Joyce Bahhouth Bladen Community College
Assessing OL Courses
Joyce Bahhouth
Bladen Community College

2. What is the common ground?
Rich, rewarding, meaningful OL courses?
Last minute course preparation
Updating due dates
Revamping course

3. Have we had students who …
… skipped through the modules and went straight to the assignments?
… never followed instructions as they prepared their assignments?

4. Complementary Tasks Interaction Format Template Dep. of Distance Ed:
Learning
Pedagogy
Content
Department Chairs:

5. Is it a matter of who does more?
Instructor?
Student?

6. Challenges to assess the complexities of OL courses
Minimum level of standards
Comprehensive/ evaluative tool
Institution-wide

7. What do we want to see in a course?
Ideal Course
Instructor’s presence
Ease of navigation
Supplemental material
Activities requiring critical thinking

8. What makes a course unique?
Students’ input
Students’ perceptions of their learning and course experience
Developments in the field and current events
Student/ instructor interactions
Student/ student interaction

9. Quality Matters Online Instructor Skill Set
Institutional Context
Technologies
Instructional Design
Pedagogy
Assessment
Social Presence

11. Cognitive Presence Definition
The extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of learning.
It has the potential to assess the quality of critical inquiry in terms of providing a means to assess the systematic progression of thinking over time.
The process more than the product is important.

12. Social Presence
Definition
The ability of learners to project their personal characteristics
socially and emotionally into the community of inquiry
thereby, presenting themselves as ‘real people.’

13. Teaching Presence Definition design, facilitation, and direction
of cognitive and social processes
for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes.

14. A community of inquiry
community involves (re)constructing experience and knowledge through the critical analysis of subject matter, questioning, and the challenging of assumptions

15. Practical Inquiry Model

17. How Learning Occurs

18. Five Steps to Assess an Online Course (Morrison, D. 2015, May 26
Five Steps to Assess an Online Course (Morrison, D. 2015, May 26. How good is your online course? Five steps to assess course quality. Online learning insights.)
Assess using a rubric
e.g. QM, CSU Chico, OCAT, QOLT
Analyze course from a student perspective
Assess course artifacts, materials, and feedback
Consider level and type of student-to-student and student-to-instructor interactions
Examine results: Are students learning?

19. Road map to design a course
Assess OL Courses Using a Rubric
Road map to design a course
Self-evaluation tool
Recognition

20. Quality Matters Rubric
Getting started, purpose and structure of the course, self introductions, netiquette
Prerequisite skills, minimum technical skills, policies
Course Overview Introduction
Course objectives, module objectives, clear link between objectives and activities, measurable, learners’ perspective
Clearly stated objectives suitable for the course level
Learning Objectives
Competencies assessed, grading policy clearly stated
Specific and descriptive grading criteria linked to grading policy, multiple opportunities, sequenced and varied
Assessment

21. Quality Matters Rubric
Properly cited, the purpose and how the material is to be used are explained
Current, varied, clear distinction between required and optional
Instructional Materials
Promote achievement of learning outcomes, provide opportunities for interaction that leads to learning
Clear requirements for interaction, planned classroom response and feedback
Course Activities and Learner Interactions
Provide opportunities for learning, current, readily obtainable
Links to resources, policies, explanations about how students can succeed
Course navigation facilitating ease of use, access to course materials in different formats to meet diverse learners’ needs
Couse Technology
Learner Support
Accessibility and Usability

22. California State University (CSU) Chico Rubric
Learner support and resources
Online organization and design
Instructional design and delivery
Assessment and evaluation of student learning
Innovative teaching with technology
Faculty use of student feedback

23. Coulter Faculty Center eLearning Faculty Fellows: Online Course Assessment Tool (OCAT)
Course Overview and Organization
Elements of Course Design
Elements of Teaching
Learner Objectives and Competencies
Resources and Materials
Learner Interaction
Learner Assessment

24. Learner Interaction
Course Design
Elements of Teaching
Assessments are frequent enough to provide formative feedback
Rubrics are provided to define assessment criteria, if applicable
Methods of submitting assignments are appropriate to the online learning environment
The instructor:
clearly describes assignments (student discussion, participation, and projects)
provides formative assessment/progress feedback (e.g. feedback that guides the student about academic performance before the end of the course)
provides summative assessment feedback (e.g. substantive instructor feedback on academic performance)
provides timely assessment feedback

25. Quality Online Learning and Teaching (QOLT) Instrument (qolt
Quality Online Learning and Teaching (QOLT) Instrument (qolt.csuprojects.org)
1. Course Overview and Introduction 2. Assessment and Evaluation of Student Learning 3. Instructional Materials and Resources Utilized 4. Students Interaction and Community 5. Facilitation and Instruction 6. Technology for Teaching and Learning 7. Learner Support and Resources 8. Accessibility and Universal Design 9. Course Summary and Wrap-up 10. Mobile Design Readiness (optional)

26. How a course can be assessed from a learner’s perspective
Student Feedback
forum posts of student questions, 
feedback from group interaction,
end-of-course student surveys
Your feedback
As a student in a course/field you are not familiar with
MOOC
Colleagues’ Feedback
Same department
Other department

27. How course artifacts, materials, and feedback can be assessed
Course materials
Student generated content
Student feedback
Comparative data

28. How social interaction can be assessed
Student-student and student-instructor interaction:
If students are connected,
More motivated,
More satisfied with the learning experience
If students are disconnected, …

29. Demonstrate process and product:
Assess Learning
Assignments
Demonstrate process and product:
what students know, how they think, how they can apply course concepts, and how they can articulate what they have learned
Learning occurs
Instructor can evaluate learning effectively and can determine level of critical thinking

30. How learning can be assessed
Goals
Clearly defined
Aligned with objectives
Learning Activities
Appeal to all learning styles
Develop critical thinking and problem solving skills
Interaction
Student/content
Student/student; student/instructor

31. Suggested Rubric (Work in Progress) Instruction
The Instructor
Provides clear objectives to modules or chapters
Provides personalized teaching
Addresses all learning styles
Provides organized and coherent lessons
Integrates supplemental materials successfully

32. Assignments and Activities
Instructor
Provides clear instructions
Varies assignments and activities
Provides a clear rubric or criteria for evaluation
Addresses higher order critical thinking skills in assignments

33. Feedback Instructor Provides prompt, detailed, and thorough feedback
Has effective communication skills
Provides progress feedback

34. Conclusion
Start a
course
with
time! at
one

36. References
Anderson, T., Rourke, L., Garrison, D. R., Archer, W. (2001). Assessing Teaching presence in a Computer Conference Environment. Journal of asynchronous learning networks, 5(2), 1-17.
California State University Chico Rubric
Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical Thinking, Cognitive Presence, and Computer Conferencing in Distance Education. American journal of distance education.
Morrison, D. 2015, May 26. How good is your online course? Five steps to assess course quality. Online learning insights.
Online course assessment tool (OCAT) and peer assessment process. Cleveland State University.
Quality Matters ubric.pdf Qualitymatters.org
Quality Online Learning and Teaching (QOLT) Instrument
Rourke, L., Anderson, T. Garrison, D. R., & Archer, W. (2001). Assessing social presence in asynchronous, text-based computer conferencing. Journal of Distance Education, 14(3),