Assessed: 5 Cycles 2006, 2009, 2010, 2011, 2013

 General Education Requirements  Required course in Intermediate Composition and Critical Thinking  Required course in Quantitative Reasoning  Additional lower and upper division General Education requirements  Preparation for the Major Requirements  Core Courses in the BSBA Major  Advanced Courses in the BSBA Major

The refinement of the process over five cycles of assessment.

 Case analyses completed for the college-wide CBA capstone course (MGT 405) were examined.  CBA’s former Director of Assessment developed a rubric after extensive review of existing critical thinking instruments  2 CBA faculty worked with the former Director of Assessment to rate the case analyses using the rubric following a training and norming session

Exceeds Expectations (4) Meets Expectations (3) Approaches Expectations (2) Fails (1) Issue IdentificationExplicitly identifies the key issue(s) Implicitly identifies (discusses) the key issue(s) Identifies subsidiary issues as key Fails to identify issue(s) or question(s) Use of evidence and data Interprets/analyzes data in a way that improves understanding of case Cites data and uses it to analyze case Mentions/cites data but fails to apply it to case issues Fails to use data provided; provides little to no support for analysis Models and Frameworks Explicitly applies models or frameworks to case analysis Analyzes case using concepts from models or frameworks Uses models/frameworks inappropriately or incorrectly Uses no models or frameworks to analyze case Conclusions & Recommendations Recommends and defends a conclusion based on the analysis Recommends a solution congruent with the analysis Recommends a solution not congruent with the analysis Does not offer a specific recommendation or conclusion

 Final exams completed for the college-wide CBA capstone course (MGT 405) were examined.  Course Instructor and Director of Assessment developed a rubric after reviewing the 2006 rubric and others from the critical thinking literature.  Rubric was widely distributed to students during the semester.  Instructor & Director of Assessment rated the final exams using the revised rubric following training and norming sessions.

 7 Point scale from Weak to Excellent  Expectations determined post-hoc based on examination of means for dimensions.  Dimensions: 1. Clear understanding of the question and issues to be addressed. 2. Understanding of the relevant concepts and frameworks. 3. Effective application of relevant concepts to address question/issues. 4. Ability to effectively use case data (depth & breadth) to illustrate a position. 5. Ability to think strategically (i.e. integrate across internal & external environment; across functional areas; take a general management view). 6. Ability to arrive at logical and well reasoned conclusions based on the discussion.

DimensionFail (1) Below (2 to Expectations 3) Approaches Expectations (4) Meets Expectations (5) Exceeds (6 to Expectations 7)

 Final exams completed for the college-wide CBA capstone course (MGT 405) were examined.  Course Instructor and Director of Assessment refined 2009 rubric based on mapping & experience.  Rubric was widely distributed to students during the semester.  Two independent CBA faculty members rated the final exams using the revised rubric following training and norming sessions.

 7 Point scale from Weak to Excellent retained.  Expectations determination post-hoc based on examination of means for dimensions retained.  Dimensions from Cycle #2 Rubric were reduced by one and wording modified in some cases: 1. Clear understanding of the question and issues to be addressed. 2. Knowledge of case facts & ability to effectively use case data to illustrate position. 3. Effective understanding and application of the relevant concepts and frameworks. 4. Ability to think strategically (i.e. integrate across internal & external environment; across functional areas; holistically from firm perspective). 5. Ability to arrive at logical and well reasoned conclusions/recommendations.

DimensionFail (1) Below (2 to Expectations 3) Approaches Expectations (4) Meets Expectations (5) Exceeds (6 to Expectations 7)

 Process adopted for Cycle #3 retained.  MGT 405 final exams used  Two independent CBA faculty members rated.  Additional rubric revision/refinement in Cycle #4.  Descriptions of levels added to each dimension.  “Approaches Expectations” dropped in favor of 3 point scale to provide consistency across assessment measures: ▪ Exceeds Expectations ▪ Meets Expectations ▪ Below Expectations

 Process adopted for Cycle #3, used in Cycle #4 retained.  MGT 405 final exams used  Two independent CBA faculty members rated.  Rubric revision in Cycle #5:  Descriptions of levels as added in Cycle #4 retained.  3 point scale added in Cycle #4 retained.  Point ranges within three point scale dropped in favor of: ▪ 1 – Below Expectations; 2 – Meets Expectations; 3 – Above Expectations ▪ However, ½ and ¼ point judgments acceptable when appropriate.  Dimensions reduced to two to better align with Critical Thinking SLOs. ▪ Third dimension rating writing used, results not included in this report.

DimensionBelow Expectations 1 Meets Expectations 2 Above Expectations 3 Collect and organize critical data and information to solve a problem. Response does not use relevant concepts to address question; response does not demonstrate a clear understanding of the relevant concepts; response discusses irrelevant concepts. Limited use of quantitative and qualitative case data to support position. Response uses relevant concepts to address question; response demonstrates an understanding of the relevant concepts; however, the response is not succinct and directly informed by the concept. Some extraneous concepts summarized and discussed. In-depth use of selective quantitative and qualitative case data to support position. Response is informed directly by relevant concepts; response demonstrates a clear and in-depth understanding of the concepts; no extraneous concepts invoked. Comprehensive and in-depth use of appropriate quantitative and qualitative case data to support position. Analyze information and follow logical steps to reach an effective decision. Response does not demonstrate an ability to think holistically; does not integrate internal and external factors and strategy; does not take an organizational perspective; considers functional areas in isolation and takes an operational perspective. Response does not arrive at logical conclusions and inferences. Statements are made and not supporting logic is offered. Response demonstrates some ability to think holistically; integrates a few internal and external factors and some connection to strategy and integrate across a couple of functions. Some evidence of taking an organizational perspective. Response arrives at logical conclusions and inferences. Statements are made and some supporting logic is offered. Response demonstrates a clear ability to think holistically; integrate several internal and external factors to strategy and integrate across multiple functions; clear evidence of an organizational perspective with an understanding of operational issues. Response arrives at very logical conclusions and inferences. Position is well argued and tightly presented with supporting logic. Ability to communicate effectively in writing. Series of individual paragraphs that are not well connected nor well presented; paragraphs do not have strong lead sentences; sloppy paper with numerous spelling and grammatical errors. Individual paragraphs that are reasonably well connected and cogent; reasonably strong lead sentences; generally competent paper marred with a few spelling, punctuation and grammatical errors. Very well organized response with inter-connected paragraphs. Strong lead sentences; craftsmanship and style exhibited throughout the entire report.

 Cycle #1: Fall Semester 2006 & Spring Semester 2007 in MGT 405 (International Business Strategy & Integration): ▪Sample Size: 175  Cycle #2: Fall Semester 2009 in MGT 405 ▪Sample Size: 124  Cycle #3: Fall Semester 2010 in MGT 405 ▪Sample Size: 119  Cycle #4: Fall Semester 2011 in MGT 405 ▪Sample Size: 120  Cycle #5: Data collected In Fall Semester 2012 in MGT 405, analyzed Spring 2013 ▪Sample Size: 122

Updated through here...

 SLO #1: Collect and organize critical data and information to solve a problem.  Cycle #1: Issue Identification  Cycle #2 - #4: Clear understanding of the question and issues to be addressed.  Cycle #1: Use of Evidence & Data  Cycle #2: Ability to effectively use case data (depth & breadth) to illustrate a position.  Cycles #3 & 4: Knowledge of case facts & ability to effectively use case data to illustrate position.  Cycle #5: Single dimension to assess SLO #1 as indicated on rubric.

 SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision.  Cycle #1: Models & Frameworks  Cycle #2: 1) Understanding of relevant concepts & frameworks; 2) Effective application of relevant concepts to address question/issues  Cycle #3 & #4: Effective understanding and application of the relevant concepts & frameworks.  Cycle #1: Conclusions & Recommendations  Cycle #2: Ability to arrive at logical and well reasoned conclusions based on discussion.  Cycle #3 & #4: Ability to arrive at logical and well-reasoned conclusions/recommendations.  Cycle #2: Ability to think strategically (i.e. integrate across internal & external environment; across functional areas; take a general management view.  Cycle #3 & #4: Ability to think strategically (integrate across internal & external environments, functional areas, holistically from firm perspective).  Cycle #5: Single dimension to assess SLO #2 as indicated on rubric.

 BENCHMARKS:  85% of our students should meet or exceed expectations for critical thinking skills  50% of our students should exceed expectations for critical thinking skills

Cycle #1

53% 23% 13% 11% “85% should meet or exceed expectations”; 83% actually did “50% should exceed expectations”; 53% actually did

22% 43% 27% 8% “85% should meet or exceed expectations”; 65% actually did “50% should exceed expectations”; 22% actually did

14% 17% 3% 65% “85% should meet or exceed expectations”; 31% actually did “50% should exceed expectations”; 14% actually did

26% 45% 23% 7% “85% should meet or exceed expectations”; 71% actually did “50% should exceed expectations”; 26% actually did

“85% OF OUR STUDENTS SHOULD MEET OR EXCEED EXPECTATIONS”  NO  Issue Identification  Use of Evidence & Data  Models & Frameworks  Conclusions & Recommendations “50% OF OUR STUDENTS SHOULD EXCEED EXPECTATIONS”  YES  Issue Identification  NO  Use of Evidence & Data  Models & Frameworks  Conclusions & Recommendations

 4 recommendations were made by the assessment team regarding ways to improve the analytic and critical thinking skills of our students.  Faculty should encourage students to make clear problem statements without hedging.  Students should be asked to show their use of models and tools and to demonstrate a clear connection between those models and their analysis.  Faculty should model the use of data in developing solutions to cases and problems.  Students should be given opportunities to learn how to support recommendations with evidence by writing and revising these sections of their papers.  An additional recommendation was to urge faculty to grade assignments with a rubric and to distribute the rubric to students prior to assignment submission

 A Memo was sent to all CBA faculty detailing the recommendations and strongly urging them to adopt them  A number of faculty members across the college expressed interest in learning more about the use of rubrics and rubric development.  The CBA Assessment Committee developed a short “rubric primer” which was distributed to the CBA faculty along with examples of rubrics available online and rubrics currently being used in the college.

Cycle #2

39% 53% 5% 3% “85% should meet or exceed expectations”; 92% actually did “50% should exceed expectations”; 39% actually did

19% 52% 11% 18% “85% should meet or exceed expectations”; 71% actually did “50% should exceed expectations”; 19% actually did

27% 50% 10%13% “85% should meet or exceed expectations”; 77% actually did “50% should exceed expectations”; 27% actually did

24% 47% 17%12% “85% should meet or exceed expectations”; 71% actually did “50% should exceed expectations”; 24% actually did

21% 63% 7% 9% “85% should meet or exceed expectations”; 84% actually did “50% should exceed expectations”; 21% actually did

21% 53% 15% 11% “85% should meet or exceed expectations”; 74% actually did “50% should exceed expectations”; 21% actually did

“85% OF OUR STUDENTS SHOULD MEET OR EXCEED EXPECTATIONS”  YES  Understanding question & issues  NO  Understanding relevant concepts & frameworks  Application of relevant concepts to issues  Effective use of case data  Ability to think strategically  Ability to arrive at conclusions “50% OF OUR STUDENTS SHOULD EXCEED EXPECTATIONS”  NO  Understanding question & issues  Understanding relevant concepts & frameworks  Application of relevant concepts to issues  Effective use of case data  Ability to think strategically  Ability to arrive at conclusions

“85% OF OUR STUDENTS SHOULD MEET OR EXCEED EXPECTATIONS”  NO  SLO #1: Collect and organize critical data and information to solve a problem.  SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision. “50% OF OUR STUDENTS SHOULD EXCEED EXPECTATIONS”  NO  SLO #1: Collect and organize critical data and information to solve a problem.  SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision.

SLO #1SLO #2 * Dimensions combined to form single SLO measure

 Our students appear to have improved slightly in meeting expectations between Cycles #1 & #2.  The benchmark was not achieved for any dimension in Cycle #1 but was achieved on one and very nearly on a second dimension in Cycle #2.  Overall, however, benchmarks were not met for either SLO.  Our students continue to struggle with critical thinking.

 The Undergraduate Committee was heartened by the improvement between Cycles #1 & #2 albeit small.  A decision was made to reassess Critical Thinking in one year to determine if the improvement was the beginning of a trend prior to making additional “Loop Closing” decisions.

Cycle #3

17.6% 67.2% 10.1% 5% “85% should meet or exceed expectations”; 85% actually did “50% should exceed expectations”; 18% actually did

16.8% 57.1% 16.8% 9.2% “85% should meet or exceed expectations”; 74% actually did “50% should exceed expectations”; 17% actually did

10.1% 52.1% 21% 16.8% “85% should meet or exceed expectations”; 62% actually did “50% should exceed expectations”; 10% actually did

11.8% 57.1% 18.5% 12.6% “85% should meet or exceed expectations”; 69% actually did “50% should exceed expectations”; 12% actually did

6.7% 56.3% 18.5% “85% should meet or exceed expectations”; 63% actually did “50% should exceed expectations”; 7% actually did

“85% OF OUR STUDENTS SHOULD MEET OR EXCEED EXPECTATIONS”  YES  Understanding question & issues  NO  Understanding & application of relevant concepts & frameworks  Effective use of case data  Ability to think strategically  Ability to arrive at conclusions “50% OF OUR STUDENTS SHOULD EXCEED EXPECTATIONS”  NO  Understanding question & issues  Understanding & application of relevant concepts & frameworks  Effective use of case data  Ability to think strategically  Ability to arrive at conclusions

“85% OF OUR STUDENTS SHOULD MEET OR EXCEED EXPECTATIONS”  NO  SLO #1: Collect and organize critical data and information to solve a problem.  SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision. “50% OF OUR STUDENTS SHOULD EXCEED EXPECTATIONS”  NO  SLO #1: Collect and organize critical data and information to solve a problem.  SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision.

 Although the number of students meeting expectations continues to increase, there was a significant decline in those exceeding expectations between Cycles #2 & #3.  The percent of students failing to meet expectations for SLO #2 was particularly discouraging.

 The UG curriculum map indicated that virtually all faculty members believed they were delivered critical thinking skills in their courses.  The Undergraduate Committee and the Assessment Committee discussed and concluded that faculty needed specific models of exercises and assignments that address critical thinking.  A “Best Practices in Critical Thinking” workshop for faculty was planned and implemented.

Cycle #4

41.7% 54.2% 4.2% “85% should meet or exceed expectations”; 95.8% actually did “50% should exceed expectations”; 41.7% actually did

15.8% 74.2% 10.0% “85% should meet or exceed expectations”; 90.0% actually did “50% should exceed expectations”; 15.8% actually did

10.8% 76.7% 12.5% “85% should meet or exceed expectations”; 87.5% actually did “50% should exceed expectations”; 10.8% actually did

13.3% 78.3% 8.3% “85% should meet or exceed expectations”; 91.6% actually did “50% should exceed expectations”; 13.3% actually did

23.3% 70.0% 6.7% “85% should meet or exceed expectations”; 93.3% actually did “50% should exceed expectations”; 23.3% actually did

“85% OF OUR STUDENTS SHOULD MEET OR EXCEED EXPECTATIONS”  YES  Understanding question & issues  Understanding & application of relevant concepts & frameworks  Effective use of case data  Ability to think strategically  Ability to arrive at conclusions “50% OF OUR STUDENTS SHOULD EXCEED EXPECTATIONS”  NO  Understanding question & issues  Understanding & application of relevant concepts & frameworks  Effective use of case data  Ability to think strategically  Ability to arrive at conclusions

“85% OF OUR STUDENTS SHOULD MEET OR EXCEED EXPECTATIONS”  YES  SLO #1: Collect and organize critical data and information to solve a problem.  SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision. “50% OF OUR STUDENTS SHOULD EXCEED EXPECTATIONS”  NO  SLO #1: Collect and organize critical data and information to solve a problem.  SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision.

SLO #1SLO #2 * Dimensions combined to form single SLO measure

 The improvement in the % of students meeting and/or exceeding expectations was heartening.  Improvements in the % of students exceeding expectations were noted but it is acknowledged that the % remains well below the established benchmark.  The UG Committee believes that with 4 cycles of data complete it may be time to reassess the “Exceeding” benchmark.

Cycle #5

“85% should meet or exceed expectations”; 94.3% actually did “50% should exceed expectations”; 19.7% actually did

“85% should meet or exceed expectations”; 93.5% actually did “50% should exceed expectations”; 15.6% actually did

“85% OF OUR STUDENTS SHOULD MEET OR EXCEED EXPECTATIONS”  YES  SLO #1: Collect and organize critical data and information to solve a problem.  SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision. “50% OF OUR STUDENTS SHOULD EXCEED EXPECTATIONS”  NO  SLO #1: Collect and organize critical data and information to solve a problem.  SLO #2: Find appropriate models and frameworks to analyze information and follow logical steps to reach an effective decision.

SLO #1SLO #2 * Dimensions combined to form single SLO measure ** “Approaches” dropped after Cycle #3

 Slight improvement in the % of students meeting and/or exceeding expectations for both SLOs. Not significant.  Percent of students exceeding expectations for SLO #1 remained constant from Cycle #4 however 9 % decline in % of students exceeding expectations for SLO #2.  Acknowledged that percent exceeding expectations continues to remain well below the established benchmark.  The UG Committee believes that with 5 cycles of data complete it may be time to reassess the “Exceeding” benchmark.