6. McCoy College of Business Administration (BBA Assessment)
Why Assessment?
We give grades don’t we?
What is an Assessment Program?
What can we do?
How do we do it?
Special thanks to Dr. Doug Eder and Dr. Kathryn Martell for much of the content in this presentation.

7. The First and Only Goal:
To teach for long-term retention of information and application to new situations
--after Halpern & Hakel
0-5 min. Cathy Relax and warm up audience, introduce team and theme. Identify the thesis statement on our two-week NFO.

8. Universities are systems
Deep (expert) learning is a curriculum function.
It is not a course function.
Universities are systems

9. Direct vs. Indirect Assessment
Direct assessment acquires evidence about student learning and the learning environment: Exams, projects, logs, portfolios, observations....
Indirect assessment acquires evidence about how students feel about learning and their learning environment: Surveys, questionnaires, interviews, focus groups, reflective essays....

10. Whatever Assessment Is...
...It’s Multiple Measures Over Time.

11. Why Assessment Won’t Go Away
Limits on money
Shifts in public view of Higher Education
State laws and education standards
Accountability to accrediting agencies to maintain operating credentials
* * * * *
Result: We must demonstrate quality of outcome and return on investment.
Through assessment educators meet responsibilities to themselves, to students, and to the public.

12. What is “Program-Level” assessment as specified by AACSB?
The college or school must specify program-level learning goals for each separate degree program. Generally, such goals are anticipated for each degree, not for separate majors or concentrations within a degree. For example, the McCoy College offers a Bachelor of Business Administration (BBA) degree with defined majors in accounting, CIS, economics, finance, management, and marketing. A set of program-level learning goals for the BBA degree must be provided and assessed; goals for each major will not be required for AACSB program review accreditation. The MBA program has program-level learning goals reflecting graduate-level learning outcomes.

13. Strongest Links: The Scholarship of Teaching
Clear Goals
Adequate Preparation
Appropriate Methods
Significant Results
Effective Presentation
Reflective Critique

14. Assessment vs. Evaluation
Assessment focuses on the student and the learning environment.
Evaluation focuses on the professor and the teaching performance.

15. We Give Grades, Don’t We?
Aren’t grades
(by themselves)
enough?
NO!

16. When used alone, grades are no longer regarded as adequate indicators of student learning.
Professor autonomy: Grades in one course or section may be recorded using a different standard than grades in another.
General accountability: Much of the current public annoyance with higher education comes from a lack of skill and content mastery by students who have received good grades.

17. Criterion Performance vs. Value Added
Grades may reflect many things besides student mastery of course objectives:
Verbal ability
Participation
Cooperation
Extra credit
Attendance
Effort
Criterion Performance vs. Value Added

18. Good Assessment Asks Good Questions
Do we have a curriculum or a set of common courses? Which of these do our students experience?
What do we want our students “to be” or “to have” when they have completed the program? What are our curricular goals?
What do our students “do” to demonstrate accomplishment of curricular goals (i.e. course objectives)? How much and what kinds of writing? What kinds of math? Oral Presentations? Analysis and Reports? Are these relevant to our curricular goals?
How do we demonstrate to ourselves that students are achieving or have achieved curricular goals?

19. BBA Common Core Courses
1. CIS Introduction to Microcomputer Applications. (3)
2. QMST Business Statistics (3)
3. ACC Introduction to Financial Accounting (3)
4. ACC Introduction to Managerial Accounting (3)
5. ECO Principles of Microeconomics (3)
6. ECO Principles of Macroeconomics (3)
7. BLAW Legal Environment of Business (3)
8. MGT Business Communication (3)
9. MGT Management of Organizations (3)
10. MKT Principles of Marketing (3)
11. CIS Personal Productivity and Information Technology (3)
12. FIN Business Finance (3)
13. MGT Strategic Management and Business Policy (3)

20. Bachelor of Business Administration LEARNING GOALS (BBA) Upon completion of the BBA, a student will be able to:
1. Conceptualize a complex issue into a coherent, persuasive written or oral statement.
2. Use critical thinking skills to evaluate information, solve problems, and make sound decisions.
3. Use information technology skills in decision-making.
4. Apply general concepts of ethical behavior in dealing with stakeholders.
5. Understand the importance of group dynamics in achieving organizational goals and use the skills needed for effective teamwork.
6. Understand the importance of culture and diversity.

21. An Assessment Question: How Do You Know...
...that students walk out your door looking like you want them to? What behaviors have they exhibited or products have they produced? What are the indicators for your goals?

22. Goals and Objectives for Students
Goals express what we want our students to be.
Objectives describe what we want our students to do.
.....
Objectives are indicators of goals.

23. Faculty Teaching Common Core Courses Should:
Talk to each other (if multiple faculty teach the course) . . .
Agree on specific objectives for the course . . .
Make sure course objectives are relevant to Program-Level goals . .

24. To be assessment friendly, objectives should:
Focus on students
Make the learning goals visible (serve as indicators)
Describe behaviors or products (doing, making) that can be captured by assignments.

25. Bloom’s Taxonomy LEVEL SOME COGNITIVE BEHAVIORS
Evaluation Appraisal of an Analysis
or Synthesis
Synthesis Assembly of Application
Analysis Disassembly of Application
Application Use of Understanding
Understanding Management of Knowledge
Knowledge Memorization of facts, language, concepts, principles, theories

26. The Next Step . . .
Faculty Should Agree on How Best to Measure Student Performance on each Objective Contributing to Program-Level Goals.

27. Some Assessment Ways and Means
Assessment days or centers
Case studies
Classroom assessments
Completion and retention studies
Content analyses
Debates
Direct observations
Focus groups
Graduate success
Internships and service learning
Interviews (including videotapes)
Exams for certification and licensure
Matrices
Performances
Portfolios of several kinds
Projects (Primary Trait Analysis)
Questionnaires and surveys (Direct and telephone; employer, alumni, and student attitude and satisfaction)
Reflective essays
Study and activity logs
Tests
(Locally-developed and standardized)
Transcript analyses

28. Another Step . . .
Depending on the method (ways and means) used to assess a course objective, develop a rubric or a method of measurement.
Evaluate student performance on the specific objective on three levels: Exceeds Expectation; Meets Expectation; Does Not Meet Expectation.
Report your findings.
Discuss the result(s), determine any actions to take (if necessary).

29. Example Rubric for MGT. 4335

30. Responsibilities of Core Course Coordinators
1. completing the Course Alignment Matrix which specifies which program-level learning goals are covered in a specific core course,
2. developing course learning objectives that lead to fulfillment of program-level learning goals,
3. collecting data from faculty teaching course sections to support or measure student learning of program-level learning goals (including a representative sample of measurement devices/techniques/assignments to be maintained in the office of the core course coordinator),
4. reporting collective results on program-level learning goals to the McCoy College Assessment Committee, and
5. discussing and acting on specific recommendations made by the college assessment and curriculum committees.

33. What documentation must be retained by the Core Course Coordinators?
Core course coordinators should retain copies of instruments used for direct assessment of student learning, such as assignments, written products, test questions, rubrics, scoring grids, etc. The course coordinators also must retain sample student products relating to the instruments used. Aggregate results of direct assessments for various program-level learning goals gathered and/or submitted to the assessment committee should be retained by the course coordinator by semester or year for review. Minutes of core course faculty meetings should be retained with special emphasis given to items concerning improvements in instruments used to assess student learning and acknowledgement and review of data.

34. Must all students be assessed?
For the purpose of meeting AACSB-International standards relating to Assurance of Learning (16, 18 and 20), sampling may be utilized as long as it is representative. Faculty teaching college core courses for various programs, as scholars in their respective fields and under the leadership of the Core Course Coordinator, have the responsibility for determining appropriate representative sample sizes. Graduate core course coordinators may use census data from graduate core courses for assessment purposes.

35. Should college core course faculty and coordinators gather data and submit results every year?
Program-level learning goals for undergraduate (BBA) and graduate (MBA) programs are reviewed on a scheduled rotational basis. At least two program goals are reviewed each year, and college core courses covering that particular program goal are required to submit the results of course-embedded assessment to the assessment committee during the scheduled review year. Since the McCoy College views assessment as a continuous process, core course coordinators and core course faculty are encouraged, but not required, to gather data each semester for the reporting of assessment results.

36. Undergraduate Goal Rotation
Program-Level Goal Rotation Sequence:
Academic Year :
Goal 1 – Conceptualize a complex issue into a coherent, persuasive written or oral statement.
Goal 5 – Understand the importance of group dynamics in achieving organizational goals and use the skills needed for effective teamwork.
Academic Year :
Goal 2 – Use critical thinking skills to evaluate information, solve problems, and make sound decisions.
Goal 3 – Use information technology skills in decision-making.
Academic Year :
Goal 4 – Apply general concepts of ethical behavior in dealing with stakeholders.
Goal 6 – Understand the importance of culture and diversity.

37. We have never been asked...
...to assess everything all at once.

38. Who sets performance standards
Who sets performance standards? How should Core Course Coordinators report student performance on various college-level learning goals?
For each program-level learning goal assessed in a college core course, the faculty will determine their minimum expectation or standard. For reporting data on any specific program-level learning goal, core course coordinators should aggregate data and report the results by three levels: Exceeds Expectations; Meets Expectations; or Does Not Meet Expectations. In order to preserve the integrity of the assessment process, core course coordinators should release aggregate data by course (all sections) and not individual results by the professor teaching the core course. Program Assessment in the McCoy College is not the evaluation of faculty performance in the classroom; it concerns the aggregate level of student learning of various program-level goals.

39. Example Report on Program Goal

40. An Important Lesson from the Farm
A pig doesn’t get any fatter merely by weighing it.

41. How many students must meet or exceed the performance standard on a college-level program goal?
There is no prescribed percentage of students that must meet the standard articulated by faculty. AACSB and other constituent groups will look at whether and how these data are used for continuous improvement purposes. Thus, a poor showing of student mastery of a given program-level learning goal would only be a concern if the curriculum was not subsequently modified to improve student skills/performance of that learning goal. A second emphasis in the Assurance of Learning Standards is that program-level learning goals are used to communicate competencies of graduates to students, employers and other constituents. Thus, faculty have the responsibility to develop and modify curricula to insure that program-level goals are achieved by most graduates and not just a portion.

42. What Assessment Asks of You!
One day of initial development time to determine course objectives and agreeing on a grading rubric for specific course objectives.
Two to three hours per semester (depending on the goal rotation) to extract student performance on course objectives leading to the fulfillment of program-level goals.
Writing a short report (1 page) concerning student performance.
Maintaining examples of student work, grading rubrics or other examples demonstrating student achievement of course objectives.
Talking to other faculty about continuously improving classroom processes and products to improve student learning of program level goals over time.

43. The enemy of the good is the perfect.
Closing Thought
The enemy of the good is the perfect.