(c) 2011 MarylandOnline, Inc. QM Continuing and Professional Education (CPE) Rubric Developed in Collaboration with UPCEA Ron Legon Executive Director The Quality Matters Program (c) 2011 MarylandOnline, Inc.

Development of the CPE Rubric

Foundation of the CPE Project Approach by representative of the University Professional and Continuing Education Association (UPCEA), Chris Sax Steering Committee and Working Group of CPE specialists formed Definition of Scope: “Online and blended courses, facilitated, mentored, or self-managed, that may have pass/fail, skills-based, or other completion criteria, but do not provide academic credit hours.”

Stages of Development Fit/Gap Analysis of QM Higher Education Rubric applied to a range of CPE courses Adaptation of the Higher Ed Rubric Removal of standards that were not applicable Modification of standards that required additional annotation Creation of standards unique to non-credit CPE courses Testing of new rubric on a range of CPE courses Recognition near end of process that the rubric could apply to MOOCs

Distinctive Characteristics of CPE Courses And their Impact on the CPE Rubric

Open Enrollment Course takers may not have gone through a formal admissions process and may not be considered matriculated students at host institutions Learners are less likely to meet knowledge and technical skills The importance of Standards 1.5 (Prerequisite Knowledge) and 1.6 (Technical Skills) is increased 1.6 is increased to a 2-point standard in the CPE Rubric Learners are less likely to have access to institutional resources and support General Standard 7 is greatly impacted

Non-Credit and Varying Length Non-credit status requires alternative ways of documenting course completion 3.2 shifts from grading policy to criteria for successful course completion and recognition CPE courses vary tremendously in length and scope - unlike semester or quarter length Higher Education courses 1.2 (Introduction to Purpose & Structure) modified to expect annotation specifying that the length of the course be clearly stated 3.4 and 3.5 adjusted for the range and variety of opportunities for course takers to measure their learning progress

Non-Facilitated and Minimally Facilitated Courses Many CPE courses have limited or no access to an instructor or facilitator Courses may not enable learner-to-instructor or learner-to-learner communication Courses may be self-paced, student managed, and/or machine graded

Impact of Non-Facilitation on Rubric Affected standards include 1.3 (Etiquette), which specifies in the annotations that the standard be waived and the points awarded 1.7 and 1.8 (Facilitator and Learner Self-Introductions) 5.3 (Facilitator Response Time) 5.2 (Learner Activities), with a number of examples given as ways to ensure that the course is engaging 5.4 (Learner Interaction), which specifies that the standard be waived and the points awarded 6.2 (Tools for Learner Engagement), with a number of examples to ensure active learning.

Increased Emphasis on Content Quality The CPE Rubric places more emphasis on content mastery The more rigorous content standards of the QM Publisher Rubric have been incorporated into the CPE Rubric 4.3 The instructional materials are current and authoritative 4.4 The instructional materials have sufficient breadth and depth for the course taker to learn the subject 2.5 (Learning objectives appropriate to content & course level ) Increased emphasis on content in the annotation Greater reliance on the SMEs on the CPE Review Teams.

Results of the Early CPE Reviews (11 of 13 foundation and developmental MOOCs supported with Gates Foundation grants)

Results of Early MOOC Reviews Only 3 of 11 MOOCs met standards without amendment and one more since amendment Most frequently missed standards, with the essential ones asterisked: 1.6 Minimum Technical Skills Required (more than half) 7.3 Support Services and Resources for Students (more than half) ***2.2, 2.3 and 5.1 Measurable Module/Unit Learning Objectives, Instructions and Activities (nearly half) *2.1 Measurable Course Objectives (a third)*7.2 Link to Accessibility Services (nearly half) 8.1 Guidance on Accessibility Support (nearly half) 1.8 Learner Self-Introductions (nearly a third) *5.3 Facilitator feedback (nearly a third)

Reviewers’ Comments (1) MOOC content was widely praised by QM Peer Reviewers: “I think the course did a great job of providing the design necessary to meet that and other related objectives.” “I think the MOOC I reviewed was EXCELLENT!”

Reviewers’ Comments (2) The majority of QM Peer Reviewers doubt these courses could substitute for traditional online or classroom courses for college credit for the “typical” student: “Considering the requirement for personal maturity and discipline one has to have to take an online course, and the fact that the MOOC arrangement cannot provide extensive support to the student, a MOOC of this quality can serve the described student quite well.” “A typical developmental student needs lots of support, is easily frustrated, and needs constant encouragement to trust in their abilities. I don't see MOOCs being able to offer enough resources to a developmental student who already is in a higher risk category to stop out.”

Reviewers’ Comments (3) QM peer reviewers see these courses as remediation and preparation for college-level course work: “A fully online course to help students prepare for general education courses will be very useful.” “I think this course would be great for learners in need of a self-paced review work before entering the next level of course.”

Your Questions and Comments

Thank You!