1. The IR Role in Subscriber Managed Course Reviews QM Institution Representative Training © MarylandOnline, Inc., 2010. All rights reserved.

2. Agenda for Today’s Workshop Types of course reviews and their requirements Readiness for a review – the course review and the faculty developer IR responsibilities and time frames in a course review How to use the Peer Reviewer Search Tool How to apply for a course review How to set up a subscriber managed course review Reporting to QM Scoring and certification of courses Re-certification of courses

3. 8 Topics for QM IR Webinars 1. The Role of the QM Institution Representative 2. Using the QM Website and MyQM 3. What IRs should know about the QM Rubric 4. The IR role in Internal (Unofficial) and QM-Managed Course Reviews 5. The IR Role in Subscriber-Managed Course reviews for QM Recognition 6. Selecting and Registering for QM Training 7. Communication by, with, and about QM 8. The QM Implementation Plan

4. Course Review Process

5. Types of Course Reviews and Requirements  Official Course Reviews – Any course reviews, QM-managed or subscriber- managed, that are conducted in compliance with QM Course Review Standards with the intention of receiving QM recognition. QM-Managed Course Reviews – Course reviews that are managed by Quality Matters (QM) staff and/or designees in compliance with QM Course Review Standards. Subscriber-Managed Course Reviews – Official course reviews, managed by a trained Subscriber representative, that adhere to the QM Course Review Standards and use the QM online tools.  Internal Course Reviews - Unofficial course reviews managed by the subscribing institution. Courses reviewed in this way are not eligible for QM recognition.  QM Course Recertification – Courses older than 5 years that have previously received QM recognition are eligible for an expedited course recertification. See policy on QM Course Recertification (in QM IR Manual)

6. Steps in a Subscriber Managed Course Review  IR notifies faculty developer of course review  IR submits course review application for all types of course reviews  IR contacts peer reviewers: confirms availability and discusses stipends  IR approves course review application  IR initiates completion of faculty developer worksheet  IR assigns peer review team and opens up the rubric for peer review team  Chair sets up pre-review conference call with faculty developer, IR and peer review team  Review team reviews course (3-4 weeks)  Review team conducts post review call if needed  Chair submits final report and sends review outcome  Faculty developer submits online faculty response form  Faculty developer makes changes during amendment period if needed  Quality Matters awards QM recognition * (duration may vary, see next slide)

7. How Long does Recognition Last?  Up to 5 years, as long as  Any improvements to the course is made by original instructor or design team  Material added by other instructors do not alter the basic structure and IR verifies compliance with standards (Master Course model)  Versions of a QM certified course offered by other instructors must undergo their own review  See Multi-Section (Master Course ) policy

8. Preparing for a Course Review  QM course reviews are designed for mature courses that have been taught previously and for courses that are fully developed online such that reviewers can see all evaluated components.  Note: Courses that have not yet been taught MAY be reviewed only if all components have been fully developed  What kind of course access does the review team need?  Student-level access.  Quality Matters recommends using a full-developed course shell rather than a live course or the most recent archive, with steps taken to ensure student privacy rights.

9. Checklist for Review Readiness The following checklist can help you determine if a course is ready to be submitted for review: _____ Has the faculty developer/ instructor had time to review the Quality Matters rubric and make modifications before the review? Faculty that are prepared for a Quality Matters review have better outcomes and get more out of the review process. _____ Are the course-level outcomes specified and are there module/unit objectives for each of the course module/units? The lack of course-level outcomes and module/unit-level objectives is among the most frequently missed standards. _____ Are all discussion board questions or topics posted for review? This might include samples faculty responses designed to facilitate the desired interaction. (If student responses are also provided, they should be stripped of identifying information.) _____ If the course uses email in any part of the instruction, is this information made available to the review team? Examples of such email exchanges should be provided to the review team during its review of the course. _____ Are all course activities, including all audio-visual components, available to the review team? Sometimes instructors make assignments “not available” after a specified “due date.” All such assignments will need to be available to the review team. _____ Are all assessment tools available for review by the team? The review team will need to be able to access quizzes, exams, and tests, and it would be beneficial to the review team to also have access to the (blank) grade book as it is set- up for the “student view.” _____ Can the review team see and experience the course as a student would?

10. IR Responsibilities  Institution Representatives play an important role in the success of a review team. Keep in contact with the Course Developer to assist him/her in completing the faculty responsibilities. Contact the Team Chair if you have questions about the review in progress. 1. Keep in contact 2. Complete & Approve application 3. Set up Peer Review Team 4. Send & Approve Instructor Worksheet 5. Participate in a Pre-Review Team Discussion 6. Enroll Peer Course Review team members in your course 7. Send login directions 8. Return Faculty Response Form 9. Pay Reviewers 10. Post Review - Revise course (if needed)

11. Review Requirements Use of current and unmodified QM rubric standards Review of online or hybrid course Three person peer review team meeting the following criteria: All reviewers are certified eligible peer reviewers At least one reviewer is external to the institution submitting the review At least one reviewer is designated as a subject matter expert Team chair is certified QM Master Reviewer Process Requirements – The following are the QM process requirements that must be met on the reviews of all courses seeking QM recognition: Instructor Worksheet, Final Report, and Course Amendment Form (if applicable) have been submitted online. Team chair evaluates amendments to courses. Pre-review communication with Course Developer and review team. Post-review survey completed by the institution representative, Course Developer, and all reviewers. No more than 20 weeks between the start of the review and the date documentation is submitted to Quality Matters for course recognition.

12. Scoring and Recognition To meet standards and gain QM recognition a course must: Have followed all QM review requirements Met all 17 essentials standards (3 point standards) Have a score of at least 72 points out of 85 Complete all necessary paperwork such as faculty developer worksheet and survey If amendment process is needed: fill out course amendment form, chair approval, completed within 20 week time frame

13. How to Apply for a Course Review http://qmprogram.org/review/ The new Course Review Management System CRMS (online rubric tool) will officially launch November 15, 2010 All types of reviews will start with an application being submitted. Info needed: who will serve as IR faculty developer email address course number course name discipline area

14. Peer Reviewer Search 1

15. Peer Reviewer Search 2

16. Peer Reviewer Search 3

17. Peer Reviewer Search 4

18. CRMS Homepage

19. Application

20. Institution selection

21. Select IR for review

22. Choose Review Type

23. Application information

25. Display Applications

26. Process Application

27. Send Faculty Developer Worksheet

28. Faculty Developer Worksheet

29. Status FD Worksheet

30. Approve Faculty Developer Wksht

31. FD Worksheet Upload

32. Upload Course Objectives

33. File Uploaded

34. Status FD Worksheet

35. Faculty Developer Worksheet

36. Add Reviewers

39. Review Team Verification

40. Mark Reviewer Selection Complete

41. Review Started

42. Quality Matters Questions?

43. Thanks to YOU… Quality Matters! Fall 2010 © MarylandOnline, Inc., 2010. All rights reserved.