1. Building Trust with Quality Assurance Strategies in TAACCCT
Gerry Hanley
SkillsCommons.org (TAACCCT Repository Services)
Executive Director, MERLOT
Assistant Vice Chancellor, Academic Technology Services
California State University, Office of the Chancellor

2. What Do You Want to Achieve?
Your students recognize their potential and become motivated to learn from you.
Your faculty colleagues who share your passion and vision recognize your insights in teaching others and want to follow the journey of learning you have laid out for your students
Employers who are searching for people to make their businesses better hire your students and students who learned from your courses

3. How Do You Get Strangers…
Students
Faculty
Employers
… To trust what you have created?

4. Quality and Symbols of Trust
The content of your curriculum captures the skills, knowledge, and attitudes needed for success on the job and in life
The design of the instructional process enables rather than interferes with the desires to learn
The delivery of the curriculum enables all learners, including those with disabilities to succeed
The demonstration of the learning represents the enrolled students ‘performances.

5. And When You Put Your Programs Online – Fully Online or Partially Online How do you assure quality?

6. Quality Assurance Strategies
An Overview For TAACCCT Grantees and Planning Our Future

7. Quality of Content
TAACCCT projects working with industry partners and subject matter experts
How do strangers know that your curriculum has been created by experts?
Describe the expert authors in SkillsCommons.org
3rd Party experts review the content – Professional Organizations, Editorial Boards, Employers
Other institutions adopt your curriculum
MERLOT’s peer review processes by editorial boards – 14 years building tools and processes
Many Universities and Colleges using rubrics to evaluate the quality of instructional design of courses
Quality Matters
Institutional developed guidelines (CSU QOLT)

8. Learn More at
Wednesday’s
10 am session
commons.org

11. SkillsCommons.org Can Leverage MERLOT’s Peer Review Process
MERLOT’s peer review processes by editorial boards – 14 years building tools and processes
Managing 23 Editorial boards
Opportunities for TAACCCT grantees to build communities and develop your quality assurance strategies for your disciplines

12. Quality of Instructional Design
Many universities and colleges use rubrics to evaluate the quality of the instructional design of their courses
Quality Matters TM
Online Learning Consortium (AKA Sloan-C) Quality Scorecard (Institutional Capabilities)
Institutional developed guidelines (CSU QOLT- Quality Online Learning and Teaching)

13. “Certified” training for faculty learning to apply the QM Rubric
“Certified” Peer Reviewer
“Certified” Master Reviewer/Train the Trainer
“Certified” evaluation of course against the QM Rubric
Great value – Consortium pricing
Rubric for Course Design and Delivery
Aligns with QM
Recognition Program within the CSU
Free

14. QM General Standards
#1: The overall design of the course is made clear to the student at the beginning of the course (8 criteria) #2: Learning objectives are measurable and are clearly stated (5 criteria) #3: Assessment strategies are designed to evaluate student progress by reference to stated learning objectives; to measure the effectiveness of student learning; and to be integral to the learning process (5 criteria)

15. QM General Standards
#4: Instructional materials are sufficiently comprehensive to achieve stated course objectives and learning outcomes (6 criteria) #5: Forms of interaction incorporated in the course motivate students and promote learning (4 criteria) #6: Course navigation and technology support student engagement and ensure access to course components (5 criteria)

16. QM General Standards www.qualitymatters.org
#7: The course facilitates student access to institutional support services essential to student success (4 criteria)
#8: The course demonstrates a commitment to accessibility for all students (4 criteria)

17. Total of 41 Checkpoints for Quality Instructional Design for Online and Hybrid Courses
Over 300 CSU faculty and staff have completed QM workshops.
Over 50 CSU certified peer-reviewers

18. CHOICE of Quality Assurance Rubrics

20. Recognition & Dissemination of Quality Instruction
Since 2010, 500+ CSU faculty have participated in the CSU QOLT Awards Program.
Three levels of evaluation
Self-evaluation: Copy goes to instructor, copy goes to Campus Coordinator
Student ratings: Anonymous. Viewed by peer- reviewer. Instructor views after grades submitted.
Peer-review: Analysis by faculty/staff colleague
Participation letter to all; Recognition letter to campus finalists; Letter, certificate, postings to CSU awardees
Campus and CSU presentations by awardees
Online repository of exemplars by objective

22. QM and MERLOT Partnership
MERLOT’s free and open library of instructional materials includes over 5,000 Open CourseWare.
Online curriculum for an entire course that is free of cost and free to use/reuse/revise.
MERLOT’s peer reviewer apply rubrics to Learning Objects/Modules not entire courses.
QM reviewers apply rubric to entire courses
MERLOT’s open library services enables easy access to QM’s reviews

23. Free and Open Access to OCW

24. Calculus I – Add QM review

25. Contributing QM Review in MERLOT

26. MERLOT Integration - Comment

27. QM Review in MERLOT Content Builder

28. QM Member Profile in MERLOT

29. Quality and Accessibility
All learners, including those with disabilities, have equally effective access to learning and succeeding
Make it a forethought not an afterthought
Very important and lots of work to do
We will be working with and you to make it work

30. CAST: Center for Applied Special Technology

31. Making Open for All
DIGITAL +
OPEN LICENSE
ACCESSIBLE ≠

33. MERLOT Accessibility Checkpoints
Aligned with Section 508 and WCAG guidelines
Balanced between brevity and breadth of coverage
Tailored to common eLearning formats
32 total checkpoints organized in 15 functional areas
Collectively represent baseline accessibility support by addressing the most common, high-impact barriers
Easily validated using free or low-cost tools and methods:
Firefox WAVE toolbar extension
Firefox Web Developer extension
Manual evaluation

36. Structural Markup Criterion Rationale Evaluation Steps
The text includes markup that allows for navigation by key structural elements (6A).
Rationale
Headings allow those who are blind to understand the structure of the page and easily navigate by sections.
Headings allow those with low-vision to apply their own styles to more easily locate specific sections.
Evaluation Steps
In the WAVE toolbar, click the Outline button

37. Structural Markup - Demonstration 1
DNA From the Beginning (Concept 1 links page)
Standard View
Evaluation View
Presenter Note:
The Concept 1 links page on the DNA From the Beginning ( site contains sections that are marked up as headings as revealed by the WAVE toolbar.

38. Structural Markup - Demonstration 2
Biology Tutorials for Cell, Metabolism and Genetics
Standard View
Evaluation View
Presenter Note:
The Table of Contents page on the Biology Tutorials for Cell, Metabolism and Genetics site ( contains visual sections that aren’t marked up as headings.

42. Images & Accessibility
Criteria
Non-decorative images have descriptive alternative text (11A).
Decorative images have null alternate text (11B).
Rationale
Alternate text should describe non-decorative images so they’re accessible to those who are blind or have visual impairments.
Decorative images should have null (alt=“”) alternate text so screen readers can skip over unnecessary content.
Evaluation Steps
In the Web Developer toolbar, select the Images submenu and then select the Display Alt Attributes command

43. Images - Demonstration 1
DNA From the Beginning (Concept 1 landing page)
Presenter Note:
The Concept 1 page at DNA From the Beginning ( provides alternate text for non-decorative images in the body and search field; however it also provides non-meaningful alternate text for thumbnails (i.e. the alt text only indicates the concept number). Ideally, the thumbnails would have null alternate text and the hyperlink text would provide the concept number and name for all users.

44. Images - Demonstration 2
Biology Tutorials for Cell, Metabolism, and Genetics (Figure 30)
Presenter Note:
On the Biology Tutorials for Cell, Metabolism and Genetics site, the page containing Figure 30 ( conveys a significant amount of visual information to sighted users but does not provide alternate text that describes the image for those who cannot visually perceive or interpret it (that’s why there’s no red annotation containing alternate text).

47. CSU+MERLOT+Universal Design = Open For U

48. Quality by Academic Integrity
How can you assure that the student performance in your class was by same person who enrolled in your class?
How can you assure that the student performance represents the students’ knowledge, skills, and attitudes that they will bring to the job?

49. Aspects of Assessment Integrity
Before the assessment
Authentication of student (No confederate)
Controlled access to assessments (No learning the test)
Unique design your assessments (No plagiarism)
During the assessment
Proctoring of student performance (No “study aids”)
After the assessment
Controlled access to assessments (No sharing of test)
Authentication of the performance (No Plagiarism)

50. OER and Assessment
How can we have academic integrity in student performance if the assessments are OER?
OPTIONS:
CC BY on assessments but only authenticated faculty/staff have access to these materials AND commit to protect the access
CC BY on assessments that are released as “study guides”. Have 3rd party organization create derivative assessments that are securely managed
Create assessments with other funds not requiring CC BY license

51. How will we build trust in transforming the education of our workforce?
Mass = 1

52. Building A Community To Trust
Mass = Putting Quality Assurance into Educational Practices
Mass = TAACCCT Community

53. Thank You!
Questions?

54. Gerry Hanley ghanley@calstate.edu
For more information:
Gerry Hanley