1. From Inputs to Process to Outcomes: the Quality Management Program at the American University in Bulgaria Steven F. Sullivan Dean of Faculty

2. How to assess quality?

3. Assess Inputs Does the meat have the correct weight, fat content? Is there the correct amount of cheese? Are the other ingredients according to the recipe?

4. Input-based assessment Advantage: If the inputs necessary for a quality product are not there, the quality cannot be there. Disadvantage: Having the inputs doesn’t guarantee that the output is high- quality. Standards may lag behind changes in technology

5. Process Evaluation Was the meat cooked for the right amount of time at the right temperature? Does the restaurant observe the limits on holding time? Was the sandwich packaged correctly?

6. Does process evaluation guarantee high quality? Following the correct processes ensures that the actions taken are those that were intended, but doesn’t guarantee that the desired outcome was achieved.

7. Outcome assessment Were the organization’s goals achieved? Are society’s goals achieved?

8. AUBG’s assessments of quality External Constituencies – New England Association of Schools and Colleges (US accreditation) – National Evaluation and Assessment Agency (Bulgarian accreditation) – Employers – Donors

9. Internal Constituencies Board of Trustees Students Parents

10. NEASC (US Accreditation) The institution’s academic programs are consistent with and serve to fulfill its mission and purposes.

11. NEASC (continued) The institution works systematically and effectively to plan, provide, oversee, evaluate, improve, and assure the academic quality and integrity of its academic programs and the credits and degrees awarded. The institution develops the systematic means to understand how and what students are learning and to use the evidence obtained to improve the academic program.

12. Outcome assessment Advantage: you set out to measure the very objective your organization is striving to achieve. Disadvantage: It’s hard!

13. Do we engage in the processes that reflect our goal to accomplish the institutional mission? Do we monitor our success in graduating students? Do we evaluate our students’ career success after graduation? Do we take steps to ensure that the quality of our academic programs are upheld? Do we monitor student, faculty, and institutional performance and take appropriate steps to improve performance?

14. NEAA Criteria (Bulgarian accreditation) 1.2. HEI performs procedures for developing, approving, observing and renewing the educational documentation (qualification descriptions, curricula, syllabi) 3.2. HEI ensures professional level, qualification and development of academic staff 3.2.2. Ensures the normative minimum of habilitated teachers appointed

15. Examples of outcomes to track…

16. Example: allocation of resources to fulfill academic mission

17. Academic Planning Faculty member’s observation: “I feel like my obligation to try to reach out to the bottom 15% of the class drags down my whole class. Why are these students even taking my course?”

18. What does the data say? Student1st 4 GPACGPABUS GPA 111.352.652.43 41.672.32.20 181.682.642.59 21.832.592.17 561.852.973.05 91.902.992.40 51.932.52.22 441.932.852.97 282.003.042.72 402.083.112.92 72.082.872.31 122.082.462.44 222.083.052.66 472.083.012.99 512.153.023.01 172.182.372.59

19. Overall conclusion Summary Data -- 2009 Business Majors 1st 4 GPA# of studentsoverall BUS GPA (average) below 2.082.5 from 2.0 to 2.49232.68 from 2.5 to 2.99433.03 from 3.0 to 3.49273.29 from 3.5 to 4.0503.69

20. Response 2010-11 Academic Catalog: Upper-level required courses in Business Administration (those numbered 3XX and 4XX) are available only to declared Business Administration majors. To be qualified for declaring the Business Administration major, a student must earn a 2.5 GPA in the first four required Business Administration courses – BUS 203, BUS 208, BUS 209, and BUS 210.

21. Example: fulfillment of mission 3. To what degree has the overall AUBG experience refined your personal values and integrity? Marking of multiple answers allowed A great deal, expectations were clearly modeled by faculty and administration5328.0% Moderately, there has been only moderate discussion of matters of integrity9047.6% Very little, integrity is not consistently valued at AUBG2613.8% Not at all, AUBG has less impact than the overall environment in this regard105.3% Not applicable, integrity should not be a University issue but only a personal issue105.3%

22. Example: survey of graduating students: 3. How important was AUBG for developing your leadership skills? extremely important5932.4% relatively important7842.9% somewhat important3820.9% not important at all42.2% does not apply31.6%

23. Evaluation of instruction

25. Distribution of coursework throughout the semester

26. Consistency of academic standards

27. Conclusions Assessment occurs throughout the institution Types of assessment include inputs, processes, and outcomes Facts and data can (and should) be collected to support the decision-making process A commitment to quality requires that decision-making be driven by relevant data.