1. OCTOBER 2011 Considerations for Assessing Program Objectives Malcolm LeMay Director of Operations College of Business

2. Considerations and approaches  Focus on accreditation requirements and expectations  Minimize burden on faculty while ensuring participation  Close the loop from assessment to improvement  Ensure value-added activities and focus on tracking/reporting relevant info  Use technology to reduce confusion, ensure consistency and improve process Assessment of Learning – Lessons Learned

3. In other words… 1.What will our students learn in our program? What are our expectations? 2.How and when will they learn it? 3.How will we know they have learned it or not? 4.What will we do if they have not learned it? Process for Assessment of Student Learning 1.Define goals and objectives 2.Align curricula with goals 3.Identify instruments and measures to assess learning 4.Collect, analyze and disseminate assessment information 5.Use assessment information for continuous improvement

4. Concept of Accreditation Maturity Model Capability Maturity Model: 1.Initial - ad hoc 2.Repeatable practices 3.Practices defined 4.Managed and measured 5.Optimizing

5. Developed interpretation of AMM levels for all AACSB standards Training during faculty retreat 18 months prior to Accreditation visit Conducted self-assessment on AMM levels Results  Increased faculty awareness and buy-in  Process gaps identified  Improved understanding of expectations Accreditation Maturity Model

6. Applying AMM for Assessment Standard 16 – Learning Goals Adapting expectations to the school’s mission and cultural circumstances, the school specifies learning goals and demonstrates achievement of learning goals. Learning goals address: General knowledge and skills (i.e., communications, problem-solving, ethical reasoning) Management or discipline specific knowledge and skills (entrepreneurial perspective, professional behavior) Basis for judgment: Demonstrate that students meet the learning goals If learning goals are not being met, demonstrate efforts to eliminate the discrepancy

7. Applying AMM for Assessment Level 1 – General objectives exist and are measured through traditional course evaluation (initial) Level 2 – Program objectives exist, assessment of learning relies on indirect measures (repeatable) Level 3 – Program objectives integrated with curriculum development; relies on direct assessment (defined) Level 4 – Assessment process generates quantitative and qualitative data, improvements are planned and implemented (managed) Level 5 – Assessment is an ongoing process aimed at continuous improvement of programs, curricula and course delivery. Processes are periodically reviewed and improved (optimizing)

8. OSU Vision/Mission COB Vision/Mission Assessment Approaches: 1.Direct assessment of course outcomes mapped to program objectives 2.Direct assessment of program objectives through course embedded measures (e.g., exams, projects, papers) 3.Direct assessment through stand-alone performance (MBA oral exams) 4.Indirect assessment of program objectives (student surveys) Program Goal #1 Program Objective 1.1 Program Objective 1.2 Program Goal #2 PO 2.1PO 2.2 Results: Exceeded – xx% Met – xx% Below – xx% Responses/Actions: Changes to curriculum, pedagogy, coverage, prerequisites Impact on program offerings: New courses, changes to sequencing, adjustments to program objectives and course outcomes Faculty Involvement in Assurance of Learning Process Courses

9. Mapping Core Courses to Program Objectives

10. Mapping Course Outcomes to Objectives

11. Example (Business Administration) Program Goal #1: Students will identify, assess and shape entrepreneurial opportunities in a variety of contexts.  Program Objective 1.1: Recognize entrepreneurial opportunities for new business ventures and evaluate their potential for business success.  Course Outcome (BA-260.6): Apply venture opportunity screening techniques to an actual start-up idea, and subsequently develop and prepare a feasibility plan. Course Outcome Assessment: Measured by evaluating team projects requiring a presentation analyzing the feasibility of a new business concept.

12. Measurement methods Direct and Course Embedded:  Individual/team presentations  Oral exams  Mock interview reviews  Group collaboration assignments  Simulation results  Essay exam questions  Course entrance exams  MBA business plan competition Indirect  Exit surveys  Recruiter feedback

13. Use of Technology Simplify collection, analysis and reporting Ensure consistency Improve overall awareness of learning and student success Ease burden on faculty Reduce reliance on Excel spreadsheets

19. To the Cloud  COB Force.com course informationForce.com course information

20. Goal 2 - Professional communication and behavioral skills 2.1 Communicate effectively and professionally in business situations through physical or virtual presence, writing, speaking, listening, and electronic media. 2.2 Demonstrate the ability to lead by using team building skills and facilitating collaborative behaviors in the accomplishment of group goals and objectives. 2.3 Demonstrate good work habits, time management and self discipline. Assessment Findings  Written communication skills assessed below desired level  Oral communication skills for professional situations assessed below desired level  Limited opportunities to expose students to virtual collaboration.  Students lack professional behavioral skills  Students’ leadership skills assessed as less than strong  Desire greater participation in experiential learning Actions Taken  Added BA 353 Professional Development course to the business core as a “Writing Intensive Course” to provide students with the opportunity to learn business communications earlier in the curriculum.  BA 353 added assessment of oral and presentation skills through mock interview sessions.  BA 370 added course projects using SharePoint to emphasize virtual collaboration skills.  BA 352 revised to emphasize practical skills in leadership and team building. The course is now taught exclusively to business students.  Implemented a professional development program which has improved the awareness levels of internship opportunities through greater publicity efforts, created a system for monitoring the number of internships undertaken by students during the year, and provided a year round program of professional development.

21. Target level 4 for all programs Ongoing  Involve faculty in program mapping and analysis  Use technology to reduce support requirements, encourage consistency and ease demands on faculty  Improve reporting and follow-up to “close the loop”  Keep in mind “big picture” - continuous improvement  Aspire for “accreditation-ready” Moving Forward with Program Assessment

22. Back up slides

23. 1.Knowledge: Recall data or information 2.Comprehension: Understand the meaning, translation, interpolation, and interpretation of instructions and problems. 3.Application: Use a concept in a new situation or unprompted use of an abstraction. 4.Analysis: Separates material or concepts into component parts so that its organizational structure may be understood. 5.Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure. 6.Evaluation: Make judgments about the value of ideas or materials Bloom’s Taxonomy of Learning Domains