1. “Pathways to Excellence”
September , 2017 | Fort Worth, Texas

2. QM Peer Review Past, Present & Future
Ron Legon, Executive Director Emeritus & Senior Advisor for Knowledge Initiatives
Quality Matters
Melissa Poole, Director of Quality Assurance

3. The INQAAHE Project
INQAAHE = International Network of Quality Assurance Agencies in Higher Education
Project: The Impact of Faculty Peer Review on External Quality Assurance:The Quality Matters Case
Methodology: Interviews and surveys of QM founders; analysis of post-review surveys; review of literature on peer review in higher ed.
Published Study: forthcoming in Quality Assurance in Higher Education

4. What We’ll Cover Brief description of QM peer review
Origins and intended purpose of QM peer review
The historic role of peer review in higher education
Acceptance of QM peer review in online learning
Challenges of sustaining peer review
The future of QM peer review - Discussion

5. What is QM Peer Review?
The Peer Review Team for Higher Ed Online Courses
3 members
Led by Master Reviewer
All must have recent online teaching experience
All must be QM trained, certified, and periodically updated
Team must include at least one external member
Team must include one Subject Matter Expert
Faculty Course Developer treated as member of the team

6. What is QM Peer Review?
Reviews must be collegial – aiming to make courses better
Focusing on design, not on reviewers’ content preferences
Constructive – suggesting steps to eliminate any deficiencies
Encouraging – identifying what is done well

7. When Did QM Adopt Peer Review?
Faculty to faculty peer review - was assumed from the beginning of the QM development process in 2003
Carried over from precursor process at Frederick Community College
And never seriously questioned thereafter

8. Why Did QM Adopt Peer Review?
Broad agreement among founders on desired effects of requiring faculty peer review
Assert Faculty Ownership of a faculty only review process
Demonstrate Independence from administrative control and P&T considerations to encourage voluntary participation
Build Trust among faculty course developers by relying on peers
Establish Rigor and Credibility by requiring online teaching experience & subject matter expertise
Insure Objectivity by including one or more external reviewers
Spread Standards through peer reviewers and Create Champions

9. Summative vs. Formative Peer Review
QM founders were heirs to a long-standing tradition
Centuries’ old practice of senior faculty reviewing juniors
More recent tradition of faculty to faculty mentoring
Mid-20th Century distinction between
Formative and Summative Peer Review
QM Founders adopted the lessons of Formative Peer Review
Principles developed for the classroom setting
Relevant elements were easily adapted to the online environment

10. Principles of Formative Peer Review*
“Formative evaluation including rigorous descriptive strategies, along with ample feedback and opportunities for practice and coaching, is necessary for improved teaching.”
“Care must be taken to minimize potential problems that can arise from having the same faculty involved in formative and summative evaluation of a colleague.”
See Keig, L., & Waggoner, M. D., 1994. ‘Collaborative Peer Review: The Role of Faculty in Improving College Teaching’, ASHE-ERIC Higher Education Report No. 2. ASHE-ERIC Higher Education Reports, George Washington University, pp Available at
Gosling D, ‘Models of peer observation of teaching’, Generic Centre: Learning and Teaching Support Network; Retrieved, 8(10), 08. Google Scholar;
Teoh, S. L., Ming, L. C., & Khan, T. M, ‘Faculty Perceived Barriers and Attitudes Toward Peer Review of Classroom Teaching in Higher Education Settings: A Meta-Synthesis’, SAGE Open, 6(3),

11. Principles of Formative Peer Review*
“Formative evaluation should include nonjudgmental descriptions of faculty members’ teaching by colleagues, administrators, and, where available, teaching consultants as well as students; each constituency should be asked for data only in areas where it is qualified to provide it.”
“Faculty should be provided opportunities for training in the skills needed to conduct formative peer evaluation.”
See Keig, L., & Waggoner, M. D., 1994. ‘Collaborative Peer Review: The Role of Faculty in Improving College Teaching’, ASHE-ERIC Higher Education Report No. 2. ASHE-ERIC Higher Education Reports, George Washington University, pp Available at
Gosling D, ‘Models of peer observation of teaching’, Generic Centre: Learning and Teaching Support Network; Retrieved, 8(10), 08. Google Scholar;
Teoh, S. L., Ming, L. C., & Khan, T. M, ‘Faculty Perceived Barriers and Attitudes Toward Peer Review of Classroom Teaching in Higher Education Settings: A Meta-Synthesis’, SAGE Open, 6(3),

12. Evidence that QM Peer Review Has Achieved Its Aims
Wide adoption of QM Standards and review practices
From the original 20 MarylandOnline institutions in 2006
To 80 institutions that joined QM in
To 1,000+ institutions by 2017
Grassroots support of faculty champions
Steady conversion of skeptics to trust in QM’s review process

13. Surveys of Reviewers and Course Designers from 2008 – 2014
On a 5-point Likert Scale:
1,578 Course Developers gave the peer review process a 4.65 approval score, and
5,601 Peer Reviewers gave their reviewer colleagues a 4.78 approval score

14. Challenges to Sustaining QM Peer Review at Scale
Constant efforts to replenish the reviewer pool
Expanding training to certify QM reviewers and keep them current
Ongoing communication and monitoring to assure inter-rater reliability and resolve disputes
Time demands – As institutions increase their use of course reviews there are calls for expedited reviews
Cost - As institutions increase their use of course reviews there are calls for less costly reviews
Relevance for team-designed courses – client may have less concern that review is based on peer faculty principles

15. The Future of QM Peer Review
Preparatory Reviews
Expedited reviews from one Rubric to another
New review types for those creating/revising all courses to meet QM Standards
Coaching
Resource Sites – MRRS/PRRS

16. Audience Discussion Thank you! Discussion Questions:
Evidence of positive (or negative) impact of QM peer review
Value of preserving QM peer review process
Acceptable modifications and alternatives to meet challenges
Your questions
Thank you!