Measuring Course Quality: Development of a Micro-Analysis Tool Laurie Beatt Sonja Chamberlin

SAIT has just celebrated it’s 100 year! When SAIT opened in 1916 there were 11 students in motor mechanics and metal working and 7 staff. Within 2 weeks there were 370 students and almost all of those were enrolled in the evening. SAIT currently offers 83 Diploma, Certificates, Applied degrees and Baccalaureate Degrees serving over 13,000 students. We offer 31 apprenticeship programs serving over 9,000 apprentices. SAIT has over 62,000 registrations in 1,200 courses in continuing education. We have students at five campuses.

Strategic Plan Strategic Priorities Education Plan Sustainable Growth Student Success Employee Success Applied Education Innovation Partnerships Education Plan Focus Areas Student-first Culture Engaging Students Effective Teaching Methods Focused Applied Education Programs Relationships Before we take a closer look at the quality framework, we thought it important to first overview SAIT’s Strategic and Education Plans, the institutional education system and our student success framework which all informed the development of our program and course quality frameworks.

Strategic Plan Strategic Priorities Education Plan Sustainable Growth Student Success Employee Success Applied Education Innovation Partnerships Education Plan Focus Areas Student-first Culture Engaging Students Effective Teaching Methods Focused Applied Education Programs Relationships First, we will focus on the Strategic Plan which has five strategic priorities: - sustainable growth - student success - employee success - applied education innovation - partnerships

The SAIT Education System MARKETS Mature Changing Emerging SAIT SUSTAINABLE GROWTH PARTNERSHIPS GRADUATE CAPABILITY Technical skills 21st century skills Digital skills SAIT Graduate Outcomes Market Demand Input GRADUATE REQUIREMENTS Which programs Skills Number of graduates STUDENT SUCCESS EMPLOYEE SUCCESS APPLIED EDUCATION INNOVATION Now we will consider SAIT’s Education System and how the strategic priorities fit within it. We believe one of our strengths as an institution is our close relationships with industry. All of our programming is based on market needs. Through market analysis gleaned from Business Intelligence, advisory councils, accreditation bodies, and quality reviews, we determine which programs to offer, the types of skills needed, and the number of graduates needed to meet market demands. We also strive to develop graduates with the appropriate balance of technical skills and 21st century skills including digital literacy. Then as we overlay the five strategic priorities onto this framework, you can see how these support and drive the system. In order to achieve our vision of becoming a global leader in applied education, we are putting our efforts into developing a learning organization culture in which systems are aligned, there is a common language and understanding, and collaboration between departments as well as time and space for reflection, both individually and institutionally. Of course, this requires ongoing focused employee development and in the end, we are striving for an environment where there is consistency in routine processes but room for creativity and innovation to thrive within the current complexities of the higher education landscape. Now we’ll take a closer look at what’s going in the big red circle shown here in terms of a student success framework. Enablers: Systems Thinking/Alignment Focused Employee Development Common Understanding Collaboration Reflection Learning Organization Consistency Creativity

Strategic Plan Strategic Priorities Education Plan Sustainable Growth Student Success Employee Success Applied Education Innovation Partnerships Education Plan Focus Areas Student-first Culture Engaging Students Effective Teaching Methods Focused Applied Education Programs Relationships The student success framework is guided by the Education Plan which has the themes of: - student-first culture - student engagement (through) - effective teaching methods (and) - focused applied education programs (which are supported by effective) - relationships

Student Success Framework SAIT Accreditation SAIT Graduate Outcomes --- Market Graduate Requirements Institutional Inputs Pedagogical Subject Matter Educational Technologies Facilities Professional Development Learner Support Technical skills 21st century skills Digital skills Content Delivery Assessment There are several external forces which influence programming such as market needs and accreditation or other regulatory requirements. Then there are institutional inputs which require consideration such as: - pedagogy - educational technology - facilities - professional development - learner supports like counseling, accommodations for accessibility and so on - professional development of faculty and staff These factors are all critical as part of the content, delivery of curriculum and assessment of competencies in developing the desired technical, digital, and 21st century skills of our graduates to meet the expected outcomes. We are also focusing on the student’s complete learning experience and considering not just curricular activities, but co-curricular and extra-curricular as well. So we are hoping that this provides you with some context and a high-level framework to better understand our program and course quality framework and how it aligns. Now, I’ll hand it back to Laurie to describe our program quality assurance framework. Student Experience Curricular Co-Curricular Extra-curricular

Program Quality Assurance Framework

Conduct Program Self- Study Conduct External Review Create Program Response and Action Plan Review by Academic Quality Council & Approved by VP Academic Program Action Plan Implemented

Program Quality Assurance Inquiry Framework Program Quality Assurance Standards: The framework for the program quality review ensures that each program is assessed against a consistent standard. The standards inform the review in a comprehensive way which creates consistency within the review process. Indicators are seen as a total picture and no particular area dominates. Quality Management: The program engages in regular quality review that involves faculty, students, industry representatives and other potential stakeholders to ensure continual improvement. Program Goals: The program results in employment in a specific field or discipline and provides the graduate with the opportunity for career advancement and further learning. Educational Design: The program’s educational design, including critical factors such as curriculum, instructional methods and faculty, ensures that students are able to meet expected goals and outcomes. These elements impact what happens in the learning environment among learners, faculty and staff. Educational Experience: The student’s experience is successful and leads to the possibility of lifelong learning and career progression. This includes the program learning experience, SAIT experience, academic relationships and industry application of learning. Program Resources: The existing physical, technological, financial resources and time are used effectively and appropriately given the program’s anticipated goals and outcomes. This category includes some learning materials, library materials, equipment, facilities, computer hardware, and other tools, specifically as they are used by and affect the program.

Learner and Academic Services: CORE Education

CORE Education Provide training in essential employability skills for programs across campus Offer foundational & elective courses in communications, liberal arts, math, engineering sciences, and technology literacy Deliver 80+ courses into more than 50 apprenticeship, certificate, diploma, and degree programs Serve nearly 5,000 students annually I lead an academic department called CORE Education which offers what we call service instruction to provide training in…..

Course Quality Framework Because my area does not offer programs, the institutional program quality framework that Laurie just outlined did not really work for us. So we collaborated with folks from Institutional Planning and Analysis to develop a course quality framework.

Two Pronged-Approach Course Review Program Review This framework is a two-pronged approach that looks at course quality at the course level and then within the context of the program in which it is offered.

Course Review Framework Student Success Strategies, techniques, and practices to engage students within the classroom context enable student success Curriculum Design Effective curriculum design enables students to achieve learning outcomes Student Progression Progression data informs curriculum design and teaching practices Faculty Faculty expertise, development and support enable effective teaching practices Feedback Feedback processes enable continuous quality improvement to support student success We’ll start by looking first at the course review component. It has five main standards. If you look on your tables, you will see a document called “Course Review”. On here, you can see the five standards as well as a list of enabling questions for each standard. <go through each standard>

Course Review Program Review Course review findings inform the program self-study. CORE courses are evaluated in the context of the program – how do course outcomes facilitate achievement of program outcomes? Findings from the program self-study will then feed back to inform required course improvements

World Café

Instructions There will be 4 rounds of conversations; participants switch tables after each round Each table will have a facilitator and a question to discuss; record thoughts and insights on sticky notes 7 minutes to discuss per round After 7 minutes move to a new table

Discussion Thank you