1. Quality and Effectiveness Assessment in Higher Education Milton Krivokuca DBA Quality Program Coordinator

2. Objectives for this presentation: Develop and deliver an academic program using Quality tools and techniques.

3. Academic Planning ä Academic planning must support university goals ä Faculty have primary responsibility for curriculum development ä Faculty Senate typically reviews/approves/ recommends course and program development ä Course/program learning outcomes must be clearly stated ä Learning outcomes assessment must be defined ä Resource implications must be stated ä Faculty, staff, facilities, funding requirements, implications for accreditation

4. Academic Planning ä Periodic review of programs ä Provost/Academic Vice President makes recommendation to the President ä In university systems special approval is required for some programs.

5. College of Extended & International Education California State University, Dominguez Hills College of Extended & International Education Master of Science In Quality Assurance

6. Master of Science - Quality Assurance The master of Science in Quality Assurance (MSQA) is designed to prepare professionals in quality, engineering, science, management, health care, government and service industries for career advancement.

7. FACULTY ä ONE FULL TIME COORDINATOR ä SIXTEEN PART TIME ä FIVE DOCTORATES ä ELEVEN MASTERS - INCLUDING FIVE PROFESSIONAL ENGINEERS ä MAJORITY OF FACULTY ARE PRACTICING PROFESSIONAL QUALITY ENGINEERS AND MANAGERS/DIRECTORS/VP’S ä SEVERAL INDUSTRY RECOGNIZED SUBJECT MATTER EXPERTS

8. MSQA Curriculum Design and Assessment Overview of MSQA Curriculum and Assessment Processes 8 College of Business Administration & Public Policy

9. Curriculum Design & Assessment: Alignment of Learning Objectives MSQA Mission Learning Standards for the Program Program Goals for Each Course Learning Objectives Assessment at the Course Level 9

10. MSQA Assessment Process ä Verify MSQA SLO’s align with university mission ä Map the curriculum to SLO’s per course ä Assess effectiveness of meeting each SLO ä Analyze results ä Advisory board meeting

11. University Mission – Program Mission

12. Basis to Map Curriculum

13. MSQA Program Course Lists Manufacturing Concentration: ä QAS 510 Advanced Probability and Statistics3 Units QAS 510 QAS 510 ä QAS 511Quality Function Management and TQM3 Units QAS 511 QAS 511 ä QAS 512 Reliability3 Units QAS 512 QAS 512 ä QAS 513 Statistical Quality Control and Sampling3 Units QAS 513 QAS 513 ä QAS 514 Advanced Experimental Design3 Units QAS 514 QAS 514 ä QAS 515 Human Factors in Quality Assurance3 Units QAS 515 QAS 515 ä QAS 516 Measurement and Testing Techniques3 Units QAS 516 QAS 516 ä QAS 518 Quality Project Management & Productivity3 Units QAS 518 QAS 518 ä QAS 598S Directed Research3 Units QAS 598S QAS 598S ä QAS 599 Project3 Units QAS 599 QAS 599 ä Elective (with approval of Academic Coordinator)3 Units Elective Total:33 Units

14. MSQA Program Course Lists Service and Health Care Concentration: ä QAS 511Quality Function Management and TQM3 Units QAS 511 QAS 511 ä QAS 515 Human Factors in Quality Assurance3 Units QAS 515 QAS 515 ä QAS 516 Measurement and Testing Techniques3 Units QAS 516 QAS 516 ä QAS 518 Quality Project Management & Productivity3 Units QAS 518 QAS 518 ä QAS 530 Statistical Quality Control for Service Professionals 3 Units QAS 530 QAS 530 ä QAS 531 Customer Satisfaction & Quality Assurance3 Units QAS 531 QAS 531 ä QAS 532 Quality Assurance for the Service Delivery Process3 Units QAS 532 QAS 532 ä QAS 598S Directed Research3 Units QAS 598S QAS 598S ä QAS 599 Project3 Units QAS 599 QAS 599 ä Elective 3 Courses (with approval of Academic Coordinator)9 Units Elective Total: 33 Units

15. MSQA Program – New Courses for 2010! QAS 5QAS 534 Change Management QAS 5 QAS 535Lean Manufacturing QAS 536 Six Sigma Principles and Application AS 536 AS 536 QAS 5QAS 537 Qualify Function Deployment: Understanding Customer Requirements QAS 5 QAS 538 QAS 538 Evaluation and Outcome for Healthcare Delivery QAS 538 QAS 539 Good Manufacturing Practices QAS 540Food & Drug Law QAS 541 Biomedical Quality Control methods QAS 542 Risk Management in FDA Regulated Industries

16. Curriculum Map

17. Program SLO Mapping

18. Assessment Planning

19. Assessment Letter to Faculty Faculty assessor: ___William Trappen______________ MSQA Student Learning Outcomes #1, #3, #5, #7, and #8 MSQA Course Level Objective #1 Apply advanced principles and tools from quality and measurement science to problem solving and measuring reliability and performance in production and service industries MSQA Course Level Objective #3 Evaluate complex, integrated organizational systems and processes in order to recognize and measure system failures scientifically, synthesize data and form solutions MSQA Course Level Objective #5 Demonstrate the ability to conduct and present independent research using primary and secondary sources, analyze information, interpret data, draw conclusions MSQA Course Level Objective #7 Design a quality system or process to meet industry standards or quality system compliance requirements. MSQA Course Level Objective #8 Identify, formulate and solve quality problems involving physical, human, and economic parameters inherent to core processes for both manufacturing and service focused industries.

20. Spring 2013 Assessment

21. MSQA SLO Map Spring 2013

22. Unbalanced SLOs

23. Survey Questions – Lickert Scale 1 -5 ä The MSQA program provided me with skills that I can immediately apply 4.5 ä The MSQA staff at the university were supportive of my needs and were a dependable resource. 4.25 ä The faculty were knowledgeable and approachable to assist with issues or questions. 4.5

24. Survey Questions – Lickert Scale 1 -5 ä I feel that the personal value I received from form time and effort will be rewarding in my career 4.25 ä I would recommend the MSQA program to a friend or professional associate interested in advancing his or her knowledge of Quality. 4.6

25. Graduate Survey Results 2011 Graduate Survey Results Q1Q2Q3Q4Q5 11344315 25544523 35125417 45555525 55554524 65555525 75554423 84454421 94555524 105545524 115555525 125355523 5451545155269 4.54.254.54.254.58333322.41667 Milt Krivokuca

26. Survey – Two years after graduation MSQA Graduate Survey Please circle the term that is most applicable to your current job assignment. Since graduating from the MSQA program: 1.I utilize tools and methodologies learned in the program: Not ApplicableNeverOccasionallyFrequently 2.Since completing the MSQA, I have (circle all that apply) Received a raise Received a promotion Made a positive career change Made a negative career change Received a performance award Became unemployed Gave back to the community through volunteer efforts Became active in ASQ or another professional society 3. 3. I would recommend the MSQA to someone interested in attaining an advanced education in quality YesNo 4.To help my career, the MSQA could have included (circle all that apply): More practical SPC exercises Additional case studies More industry specific elective course opportunities ________________________________________ 5.Any additional comments that will help us to improve the MSQA curriculum? ______________________________________________________________

27. Survey Results

28. Spring 2013 Advisory Board Meeting March 15, 2013

29. Agenda for Advisory Board Meeting ä General Welcome - 15 Minutes ä Introductions – 15 minutes ä Program Status – 15 minutes ä Assessment -20 minutes ä Breakout Sessions – 95 minutes ä Closing reports – 20 minutes ä 3 Hours total!

30. Advisory Board Report Spring 2013 CSUDH Advisory Board Meeting Following are the action plans developed as a result of the March 15, 2013 Advisory Board meeting. The information gained from this meeting was evaluated and developed into a realistic action plan. The planned activities will assure the CSUDH Quality programs remain current in the content and support the mission of the University, College of Extended and International and Extended Education, as well as the program itself. Item I:Develop CEU certificate program to support the Measurement Science Conference. Item II:Request additional information as to the goal for developing CEU credit for CCE CAPE training. Item III:Formalize BSQA Metrology Certificate of Completion Item IV:Contact instructors for QAS 512, 513, and 514 to include class assignment or activity supporting MSQA SLO #4. Item V:Revise QAS 538 to address biostatistics. This revision will further address SLOC #7 and review if revised QAS 538 can replace QAS 530 for a Healthcare specific MSQA track. Item VI: Revise SLOCs #4 and #7 to be more applicable to the current needs of Quality professionals. Item VII:Develop new version of QAS523 Software Quality. Place QAS 523 in refrigerator. New course to include elements of software quality, but move focus to IT and suspect/counterfeit issues. Item VIII: Develop three (3) new MSQA elective courses: Hoshin Planning, Sustaining Quality Improvement, Laboratory Accreditation and Proficiency Testing IX:Revise MSQA program to 3 tracks, Manufacturing (minor program changes), Service (minor program changes), and Healthcare (new track, but existing classes available for this track). Milt Krivokuca April 1, 2013

31. Next Steps: Prioritize the Action Plan per the University Program and Course revisions policy.

32. The Final Results Educational Excellence!

33. Thank You! Milton Krivokuca DBA Quality Program Coordinator mkrivokuca@csudh.edu