Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D. Russell T. Osguthorpe, Ph.D. Danny R. Olsen, Ph.D. Tom Mallory Jeff Fox, Ph.D. Brigham Young University, Provo UT

2

 11 Academic Colleges  56 Academic Departments  400 Degree Programs  1,600+ Faculty  8,000+ Courses  33,000+ Students 3  Carnegie Classification: Research University (High Research Activity)  Northwest Commission on Colleges and Universities  30 Specialized Accrediting Organizations BRIGHAM YOUNG UNIVERSITY

U.S. Regional Accreditors 4

5 Assessment Charge  Publish expected learning outcomes for all of their courses and programs  Provide evidence that the expected learning outcomes are realized by students  Demonstrate how such data collection and analyses leads to continuous improvement of student learning, the curriculum and the university Colleges and universities across the country are being asked to:

6 Why “Student Learning Outcomes”? 1.Ethics: Assessment should ultimately (first and always) be about student learning  Student major selection  Aligning institutional, department goals  Student “take-aways” 2.Pressure:  Stakeholders  Accrediting entities  Government agencies  Employers

7 What are Student Learning Outcomes? 1.Definition: Statements defining what students should know, understand and/or be able to do as a result of their learning experiences in a degree program  Cognitive (knowledge)  Behavioral (skills)  Affective (attitudes) 2.Assessment: What counts as evidence that a given level of competence has been achieved. 3.Clear and assessable statements of the “essential and enduring knowledge, abilities, and attitudes or dispositions” that enable a learner to apply what he or she has learned to situations encountered in the real world.

Direct & Indirect Evidence DIRECT MEASURES  Imbedded performance assessment mapped to learning objectives  Course and homework assignments  Exams, quizzes  Standardized tests  Term papers, reports  Research projects  Capstone projects, senior theses, exhibits or performances  Pass rates, scores on licensure, certification, area tests  Rubric scores INDIRECT MEASURES  Course evaluations  Alumni questionnaire data  Senior survey data  National Survey of Student Engagement data  Employer surveys  Graduate school placement rates  Department exit interviews  Focus group interviews with students, faculty, etc.  Job placement  Registration or course enrollment information 8

Evidences of Learning on Pathways to Graduation 9

University Goal Awareness To Publish Course Level Learning Outcomes by August 31,

Published, Mapped Course Outcomes

12 Obstacles & Challenges  Culture of assessment disparity across campus.  Quality of stated “Expected Learning Outcomes”.  Obtaining faculty consensus within departments.  Faculty engagement in the process as part of the status quo without any extra compensation.  Moving from a faculty-centric to a student centric language of stated outcomes.  Appropriate channels of evidence—direct, indirect.  Demonstrating closing the assessment loop  Applied Technology—cost, time, scalability, usability

Design Goals Ease of use  Editing, viewing, and linking of program and course outcomes must be easy Leverage existing IT web services  Security, identity, course information Data accessibility  Existing university applications  Course catalog, registration, syllabus builder

Achieving Design Goals 14 Ease of use  Usability Leverage existing OIT web services  University IT department adopted SOA  SOA registry Data accessibility  Adopted a service model  Common interfaces for interoperability  Service registered in SOA registry

Learning Outcomes Service Learning Outcomes Web site Exams Syllabus Builder Discussions Course Content Program Assessment Tool Course Registration Course Catalog Course Assessment Tool Course Evaluations

Challenges Course Learning Outcome Outliers  Certificates  Emphases  GE  Service Course Building a Learning Outcomes Assessment tool that meets disparate needs.

Faculty Involvement oy6ztc1kq0

Course Catalog

Course Evaluations

Program Assessment Tool Just Beginning Design…

Course Content

Discussions

Exams

Syllabus Builder

26

LO Website 1

29