1. What is Peer Online Course Review?
Your guide to improving course quality!
December 7, 2016
Ales will introduce us and read our bios
Autumn Bell Chief Professional Development Officer, OEI
Lené Whitley-Putz Training and Development

2. Overview History of Peer Online Course Review
Future of Peer Online Course Review
Training Peer Reviewers
Building Local Peer Reviews
Insights from 2 years of Peer Review
Lené:
I’ll just do a very brief overview of the main topics

3. 2014-2015 Rubric #1 was approved and adopted 30 Reviewers were trained
60 courses were reviewed in their “native” CMS
Lené:
Rubric was developed by the PD work group, with attention paid to the iNacol standards, Quality Matters course review rubric, and the CSU rubrics, ROI and QOLT.
@ONE and OEI began an official collaboration to train the first peer reviewers in San Diego, where we convened for two days for in-depth training on the rubric. We followed this training with a two week online course in Moodle.
That Fall, 60 courses were reviewed, in 5 different learning mgmt systems. Each course was reviewed by 3 peers, so that is a total of 180 reviews!
During this review cycle, we discovered we were woefully underprepared for gauging accessibility, and we discovered different course management systems priviliged different course design elements

4. 2016 Canvas became the official Common Course Management System
The rubric was revised
30 more reviewers were trained
60 more courses were reviewed—in Canvas!
Lené:
Changes had a dramatic effect on the process and on the results, pointing out places where the professional development could be modified for more gain, including revisions in the rubric to eliminate “double-dinging”
Needed more reviewers
Canvas reviews led to insights about the effect of the CMS on the course quality. The first courses had similar design issues, but each subsequent round is getting better and better.
OEI are developing workshops and webinars to raise awareness about designing in Canvas.
We’ll show you a few design “tricks” we’ve learned at the end of this presentation

5. 2017 Newly revised rubric New review cycle
New resources to help instructors
Autumn:
This slide is just introducing this next section. More slides follow to allow you to provide details of each part

6. The revised rubric
Eliminates vague and unproductive scoring categories
Re-organized criteria to clarify what is actually being assessed
Clearly identifies areas that are exchange ready
The next slide is a screenshot of the A section of the revised rubric

7. Section A
The new rubric includes only three rating categories—incomplete, exchange ready, exemplary
Criteria are clear and succinct
Redundancy was eliminated

8. New Review Cycle Begins with an introductory webinar
Instructors use the rubric to self-assess their course
The instructor submits their self-assessment and the course is reviewed
Instructors meet with the review team via a virtual conference

9. New Resources for Instructors
Technical support is offered in a variety of ways, including webinars, help from instructional designers, and hands-on workshops
Courses “caught” in the review cycle due to accessibility receive help from trained accessibility specialists

10. Training Peer Online Course Reviewers
The review process benefits both the instructors being reviewed, and the instructors reviewing
In 2017, we’ll offer quarterly POCR courses
Introductory webinar, followed by 3 week online course
Lené:
Explain the training we offer—this is an introductory slide

11. Introduction to Peer Review Webinar
The Rubric
Self-Assessment
Discussion of the criteria
The categories covered in the Pre-POCR workshop

12. An Example: B1 through B4
Why discussing the rubric is so important. What is each criterion measuring? Where do we look for evidence of this? What can we learn from others? How do we make our design choices explicit for students?

13. Online POCR course Week 1: Discussion of the Rubric
Week 2-3: Course Review
Successful candidates are eligible to become peer reviewers once they are approved by Faculty Senate
Spring POCR begins January 30, with the online portion beginning February 6th
The sections of the Online Course

14. Local peer review
Foster an environment that supports course development
Identify campus resources and campus needs
Create a community of practice
Autumn:
What can campuses gain by engaging in and supporting local peer review?

15. Insight: Home
Canvas offers you choices, but each has their strengths and weaknesses
Lené:

16. Activity Stream

17. Content Page

18. Page Limitation

19. Modules

20. Assignments

21. Syllabus

22. Assignment Summary

23. Take-Aways Reviewers learn as much as the instructors being reviewed
Sharing raises the quality of all the courses
It takes a village
Autumn:

24. Support from OEI for all campuses
Rubric is available on the OEI website:

26. Evaluation Survey https://www.surveymonkey.com/r/POCRWebinar16FA
Help us improve our webinars by filling out a short online evaluation survey at: