What is Peer Online Course Review? Your guide to improving course quality! December 7, 2016 Ales will introduce us and read our bios Autumn Bell Chief Professional Development Officer, OEI Lené Whitley-Putz Training and Development Coordinator, @ONE

Overview History of Peer Online Course Review Future of Peer Online Course Review Training Peer Reviewers Building Local Peer Reviews Insights from 2 years of Peer Review Lené: I’ll just do a very brief overview of the main topics

2014-2015 Rubric #1 was approved and adopted 30 Reviewers were trained 60 courses were reviewed in their “native” CMS Lené: Rubric was developed by the PD work group, with attention paid to the iNacol standards, Quality Matters course review rubric, and the CSU rubrics, ROI and QOLT. @ONE and OEI began an official collaboration to train the first peer reviewers in San Diego, where we convened for two days for in-depth training on the rubric. We followed this training with a two week online course in Moodle. That Fall, 60 courses were reviewed, in 5 different learning mgmt systems. Each course was reviewed by 3 peers, so that is a total of 180 reviews! During this review cycle, we discovered we were woefully underprepared for gauging accessibility, and we discovered different course management systems priviliged different course design elements

2016 Canvas became the official Common Course Management System The rubric was revised 30 more reviewers were trained 60 more courses were reviewed—in Canvas! Lené: Changes had a dramatic effect on the process and on the results, pointing out places where the professional development could be modified for more gain, including revisions in the rubric to eliminate “double-dinging” Needed more reviewers Canvas reviews led to insights about the effect of the CMS on the course quality. The first courses had similar design issues, but each subsequent round is getting better and better. OEI and @ONE are developing workshops and webinars to raise awareness about designing in Canvas. We’ll show you a few design “tricks” we’ve learned at the end of this presentation

2017 Newly revised rubric New review cycle New resources to help instructors Autumn: This slide is just introducing this next section. More slides follow to allow you to provide details of each part

The revised rubric Eliminates vague and unproductive scoring categories Re-organized criteria to clarify what is actually being assessed Clearly identifies areas that are exchange ready The next slide is a screenshot of the A section of the revised rubric

Section A The new rubric includes only three rating categories—incomplete, exchange ready, exemplary Criteria are clear and succinct Redundancy was eliminated

New Review Cycle Begins with an introductory webinar Instructors use the rubric to self-assess their course The instructor submits their self-assessment and the course is reviewed Instructors meet with the review team via a virtual conference

New Resources for Instructors Technical support is offered in a variety of ways, including webinars, help from instructional designers, and hands-on workshops Courses “caught” in the review cycle due to accessibility receive help from trained accessibility specialists

Training Peer Online Course Reviewers The review process benefits both the instructors being reviewed, and the instructors reviewing In 2017, we’ll offer quarterly POCR courses Introductory webinar, followed by 3 week online course Lené: Explain the training we offer—this is an introductory slide

Introduction to Peer Review Webinar The Rubric Self-Assessment Discussion of the criteria The categories covered in the Pre-POCR workshop

An Example: B1 through B4 Why discussing the rubric is so important. What is each criterion measuring? Where do we look for evidence of this? What can we learn from others? How do we make our design choices explicit for students?

Online POCR course Week 1: Discussion of the Rubric Week 2-3: Course Review Successful candidates are eligible to become peer reviewers once they are approved by Faculty Senate Spring POCR begins January 30, with the online portion beginning February 6th The sections of the Online Course

Local peer review Foster an environment that supports course development Identify campus resources and campus needs Create a community of practice Autumn: What can campuses gain by engaging in and supporting local peer review?

Insight: Home Canvas offers you choices, but each has their strengths and weaknesses Lené:

Activity Stream

Content Page

Page Limitation

Modules

Assignments

Syllabus

Assignment Summary

Take-Aways Reviewers learn as much as the instructors being reviewed Sharing raises the quality of all the courses It takes a village Autumn:

Support from OEI for all campuses Rubric is available on the OEI website:

Evaluation Survey https://www.surveymonkey.com/r/POCRWebinar16FA Help us improve our webinars by filling out a short online evaluation survey at: https://www.surveymonkey.com/r/POCRWebinar16FA www.onefortraining.org