National Learning Infrastructure Initiative Transformative Assessment Focus Session March 15, 2002

Mount Royal College Context Four year undergraduate college Comprehensive credit program base: 11,000 credit students Large credit-free operation: 51,000 registrations in continuing education, languages institute, international education and Conservatory of Music operations 80% of credit students from Calgary area 240 FT faculty; ~ 350 PT faculty

“Sensibilities” of Mount Royal College’s Assessment Program Informed by contemporary views and activity that complement institutional cultures Supported and resourced Transformative/Inclusive (Mertens, 1998) Utilization Focused: R2 (Patton, 1997) Change “aware” (Rogers, 1962)

Fundamental Principles for Developing an Assessment Framework A function of our Vision: Canada’s leading undergraduate college educating and training individuals for success in the new economy A function of our Stakeholders and what they Value A function of our Mission (Core Purpose): Student Success & Satisfaction Assessment is:

Principles for Development: Assessment is a Function of Institutional Mission MRC’s Core Purpose: Student Success and Satisfaction

Principles for Development: Assessment is a Function of Institutional Vision Vision StatementSpeaks to the need for:  Canada’s leadinginnovation/progressiveness undergraduate collegerelevance of what we offer educating & training individualsquality of how its delivered for success in the new economyvalue-added benefits

Principles for Development: Assessment is a Function of Institutional Values MRC’s Values: Student-centered Progressive, innovative, leading Personalized teaching & services of highest quality Education as an instrument for individual, community & economic development Open communication, honesty, integrity, respect for individuals Value-added

What Does This Mean? MRC’s assessment framework answers the following fundamental questions: 1. What do key stakeholders value from MRC? 2. Are MRC’s educational offerings & services relevant to the needs & values of our stakeholders? 3. Does our student-client focus foster student academic success and satisfaction?

What Does This Mean? 4. Do our educational & business practices meet or exceed nationally/internationally recognized standards for quality? 5. Do our students/clients & other stakeholders realize value-added benefits that enhance their success in the new economy? 6. Is MRC positioned as being among Canada’s most innovative and progressive undergraduate colleges?

Values of Primary Stakeholders JUDGED ON OUTCOMES Students, Alumni, Community, Faculty/Staff, Governments, Investors, other comparable providers. Who are they? What do they value? What criteria do they use to measure value?

Public Accountability PERFORMANCE PROGRESS ON VISION What must we do to achieve our vision? What are our implementation strategies? How will success be defined?

Creating Shared Goals for Institutional Transformation Identifying/Creating the Context for Change Major goals in strategic documents Priorities Issues of the day Expectations of key stakeholders Perceptions of the institution Demonstrate and communicate quality and effectiveness of what is done

How To... Institutional Research Assessment Task Group Key Questions: Institution-wide dialogues Project-based work Program-based initiatives (Academic/Service) Collaborative Environmental Sensitivity & Scanning

CASE STUDY Course Adaptation Research Project Mount Royal College

Classroom Plus  …use of learning technologies to augment, enhance and extend face- to-face instruction.

Course Adaptation for Web- Supported Delivery Pilot Project Over-riding goal: use of learning technologies to foster more flexible teaching-learning environments for faculty and students End-result: “hybrid” courses, blending face-to- face and online in various proportions 20 projects over a two-year period Significant research/evaluation component Funding: Provincial grant (Learning Enhancement Envelope)

Course Adaptation for Web Supported Activity Project GOAL #4 To study the impact of such course adaptation on teaching-learning environments from a variety of perspectives including Impact on the student learning experience Impact on student learning Impact on faculty teaching practice Impact on faculty role, responsibilities and workload

Course Adaptation Research Project: CARP Targeted Outcomes Reality Based & Results Oriented Instructional Technology and Pedagogical Quality Models Cost: time is the most important currency Professional “Location” Institutional Diffusion Inventive yet rigorous methods Authentic and visible evidence of student learning

CARP Project Overview: Research Components Activity-based costing Flashlight, NCHEMS, Custom Assessment of teaching,learning and technology Flashlight, FAST Evidence for student learning Visible Knowledge Maps Support Faculty Research Interests

Map Coding & Analysis (QSR Nud.ist & SPSS) Visible Knowledge Class Benchmarked Learning Map Individual Learning Maps Consistency and frequency of key conceptual ideas Accuracy and application of key conceptual ideas Correlations between time concepts are introduced and their appearance on maps Significance of change over time Relationship between measures (e.g. extent to mastery) Relationship between Flashlight results and student grades Totality of knowledge, sources of learning