Accreditation at the Masters Level Lessons Learned and Continuous Improvement

Presentation Outline Institution and Department Profile Program Accreditation History Masters Level Accreditation – EAC Continuous Improvement

College of Petroleum and Minerals, 1963 KFUPM … Then and Now College of Petroleum and Minerals, 1963 KFUPM now… A college 2-year technical college 76 students 14 faculty members 7 Colleges BS, MS, and PhD 9000 students 950 faculty

Admission of Students Entrance tests ( Aptitude & Achievement) Among the best 2% in the Kingdom 45% of them comes to KFUPM and the rest to other places. Nationwide Recruitment program for high achievers.

Student Enrollment* * At Dhahran (Main Campus)

Academic Programs In addition to two Community Colleges Engineering Sciences Industrial Management Environmental Design Computer Sciences & Engineering Applied & Supporting Studies Aerospace Civil Chemical Electrical Mechanical Petroleum Chemistry Physics Mathematics & Statistics Earth Sciences Accounting MIS Finance & Economics Management Marketing Architecture Architectural Engineering Construction Eng. & Mang. City & Regional Planning Computer Sciences Computer Eng. Systems Eng. Software Eng. Prep-year Skills Development Islamic & Arabic Studies Physical Edu. English Language General Studies In addition to two Community Colleges

Accreditation/Assessment Accrediting Body Programs Accredited Year (ABET)* All Engineering Programs 1993 (Berkeley Team) College of Science Programs 1996 (UNESCO) Research Institute 1997 Engineering Programs 2001 Association to Advance Collegiate Schools of Business (AACSB) College of Industrial Management 2002 Applied Engineering Programs 2003 Self Assessment 7 Programs 11 Programs 2005 (ABET)* - Extension 2007 2009 (ABET) All Engineering Programs (18) 2010

The relative distribution of U. S The relative distribution of U.S. patents assigned to higher educational institutions in the Arab world

Construction Engineering and Management Department No undergraduate degree 7 Full Time Faculty Members 200+ Graduate Students (mostly part-time)

Construction Engineering and Management Department Offer 3 graduate degrees MS in Construction Engineering and Management – Since 1984 ME in Construction Engineering and Management – Since 1999 ME in Engineering Management – Since 2009

Program Accreditation History MS and ME in CEM are ABET accredited Substantial Equivalence in 1993, 2001, and 2007 (Extension) Full Accreditation starting 2010 (Visit in 2009) Next comprehensive review:2015-2016

Accreditation – Last ABET visit in 2009 Self Study Report PEV Questions Prior to Site Visit Site Visit (2 Evaluators) Construction Engineering Engineering Management ABET Draft Statement Due Process Response Report ABET Final Statement

Self Study - ABET EAC Criteria General Criteria for Masters Level Programs (2009-10 Cycle) Develop, Publish, and Periodically Review Educational Objectives and Student Outcomes Fulfillment of BS Level General Criteria One Year Study beyond BS Level Fulfillment of Program Criteria Appropriate to Masters Level Specialization Area Students have ability to apply Masters Level Knowledge

Educational Objectives CEM Educational Objectives (Revised) Graduates will advance in their professional careers and attain leadership roles in their respective organizations. Graduates will pursue advanced degrees or seek professional certification in Construction Engineering and Management or related fields Graduates will actively participate in professional and scientific activities relevant to Construction Engineering and Management. Assessment of all PEO* *(changes to criteria 2) The criteria 2 allows more latitude in the way PEOs can be phrased. “The program must have published program educational objectives that are consistent with the mission of the institution, the needs of the program’s various constituencies, and these criteria. There must be a documented, systematically utilized, and effective process, involving program constituencies, for the periodic review of these program educational objectives that ensures they remain consistent with the institutional mission, the program's constituents' needs, and these criteria.” vs. from 2009-10 (c) an assessment and evaluation process that periodically documents and demonstrates the degree to which these objectives are attained.

Student Outcomes Program overwhelmingly admits students with ABET accredited undergraduate degrees Assess Masters Level Outcomes only Outcomes “a-k”: met by virtue of the accredited undergraduate degree Outcome “l”: Ability to conduct sound research in Construction Engineering & Management

One Year Study Beyond BS Level ME: 42 Credit Hours Program Core (24) , Elective (15), Report (3)   Course Description Core Courses CEM 510 Project Planning & Scheduling 511 Construction Estimating 520 Construction Contracting & Adm. 530 Construction Engineering 540 Construction Project Management ACCT 501 Financial Accounting MGT Principles of Management OM 502(1) Statistical Analysis for Business 599 Research Seminar 600 Master of Engineering Report 3 CEM Electives (9 Credits) and 2 Free Electives (6 Credits)

Program Criteria – Construction Engineering Criteria Item Courses that demonstrate graduates’ proficiency / understanding Proficiency in mathematics through differential and integral calculus, probability and statistics, general chemistry, and calculus-based physics By virtue of successful completion of an Engineering BS Degree from an ABET accredited / ABET equivalent institution. Proficiency in engineering design in a construction engineering specialty field CEM 540 (Construction Project Management) involves application of knowledge gained through the various CEM courses. In addition, ME Reports focus on real life issues faced by the industry. An understanding of legal and professional practice issues related to the construction industry Core: CEM 520, 540 Electives such as: CEM 515, 513, 525 for further proficiency

Program Criteria – Construction Engineering (contd.) Criteria Item Courses that demonstrate graduates’ proficiency / understanding An understanding of construction processes, communications, methods, materials, systems, equipment, planning, scheduling, safety, cost analysis, and cost control Successful completion of CEM core courses CEM 510, 511, 520, 530 and 540 demonstrates that a student understands these areas. Specialized electives are offered as well to develop a deeper understanding. An understanding of management topics such as economics, business, accounting, law, statistics, ethics, leadership, decision and optimization methods, process analysis and design, engineering economics, engineering management, safety, and cost engineering ACT 501, MGT 501, OM 502 specifically target these areas along with elective course such as CEM 514, 516, 517,518

Program Criteria – Engineering Management Criteria Item Courses that demonstrate graduates’ proficiency / understanding An understanding of the engineering relationships between the management tasks of planning, organization, leadership, control, and the human element in production, research, and service organizations   CEM 510, 520, 540 MGT 501 Electives such as CEM 513 An understanding of and dealing with the stochastic nature of management systems CEM 514, CEM 516, OM 502 Demonstrating the integration of management systems into a series of different technological environments CEM 530, 540, 600

Accreditation Oversight – Administrative Perspective Assessment Challenges Collecting Evidence Student Outcomes Program Educational Objectives Faculty Involvement Critical to the success Ownership and Engagement Sustainable Assessment Program

How to assess Ability to Apply Masters Level Knowledge? Indirect Assessment Peer Reviewed Publications resulting from MS Thesis and ME Reports Direct Assessment CEM 540 – Capstone Course – Team work Masters Report / Thesis – Rubric Based Assessment – Individual Work

Rubric Development – KPIs for Masters Level Performance Abstract Problem Formulation Relevance Definition Research Objectives Report Contents Literature Review Research Methodology Data Collection and Analysis Results and Conclusions Communication Skills Report Organization Presentation Writing Style

Sample Rubric – 2 Pages

Sample Rubric – KPI Scoring Performance Indicator Exceeds Criteria 4 Meets Criteria 3 Progressing to Criteria 2 Below Expectations 1 Score Data Collection and Analysis Sufficient data has been collected. Acceptable analysis techniques have been employed. Analysis is complete. Sufficient data has been collected. Acceptable analysis techniques have been employed. Analysis is somewhat incomplete. Insufficient data has been collected. Acceptable analysis techniques have been employed. Analysis is incomplete. Very little / No data has been collected. Analysis is incomplete.   Results and Conclusions Clear conclusions have been reported. The findings are supported by the analysis. Clear conclusions have been reported. The findings are partially supported by the analysis. Somewhat clear conclusions have been reported. The findings are partially supported by the analysis. The conclusions are unclear.

Mapping from KPIs to Student Outcomes

Closing the Loop Collect Data-> Assess/Analyze -> Implement Rubrics to be filled out by Report Advisor Report Reader Improvement Actions

Continuous Improvement Changes since the Accreditation

Changes since the accreditation ABET-FAQ for Masters Level Engineering Programs (2010-11) No exceptions for programs that admit students with ABET accredited undergraduate degrees In case of exceptions, programs must assess all Outcomes for all students or Exception Items for specific Students Additional Student Outcomes One Year Study beyond undergraduate level Ability to apply Masters Level knowledge Changes to Criterion 2 and Criterion 4 Changes to Criteria 2 allow for more flexibility in wording the PEOs. The emphasis is also on making sure that the objectives are in line with stakeholder expectations. There has to be a process in place to review that. Criterion 4 says: The program must regularly use appropriate, documented processes for assessing and evaluating the extent to which the student outcomes are being attained. The results of these evaluations must be systematically utilized as input for the continuous improvement of the program. Other available information may also be used to assist in the continuous improvement of the program.   Past Language Read: Each program must show evidence of actions to improve the program. These actions should be based on available information, such as results from Criteria 2 and 3 processes.

Curriculum to Student Outcomes Program Courses Student Outcomes a b c d e f g h i j k l CEM 510  CEM 511 CEM 512 CEM 513 CEM 514 CEM 515 CEM 516 CEM 517 CEM 518 CEM 520 CEM 525 CEM 530 CEM 531 CEM 540 CEM 549 CEM 599 CEM 600

Assessment Tools => Improvement Actions Internal Assessment Graduating Student Survey Rubrics for Master Report / Thesis Instructor Report CEM 600 Coordinator’s Report External Input Alumni Survey Employer Survey External Advisory Board

Example of process improvement: CEM 600 Report Coordinator Masters Report– One semester course Most students were not finishing in time Problems with finding a topic and advisor in one semester Modified Process Topic and Advisor – Research Seminar Execution of Report – Masters Report

Example for continuous improvement Actions in Progress Program Benchmarking and Review ASCE – Civil Engineering Body of Knowledge / 2E – 24 ASCE Outcomes Other leading universities Impact of the new Engineering Management Program Impact on Enrollment Student/Faculty Ratio

Actions for the next cycle Curriculum Revision Revision of Student Outcomes in Response to Curriculum Revision Results New Student Outcomes? Consistency with PEOs? Consistency with 1 Year Study? Accreditation of the EngMgt Program

MS Level Accreditation Challenges and Opportunities

Stakeholders - Faculty Accreditation cannot succeed without active faculty involvement Process Ownership Value of Accreditation Faculty Champions Paradigm Shift is needed Counting Beans vs. Process Quality Embed accreditation into the way business is done

Role of Program Leadership Most programs seek accreditation at the BS Level MS Level focuses in a narrow specialization area Value of accreditation at the MS Level? Justification for the stakeholders? Top management commitment Accreditation – The Deming way? Stay up-to-date with Changes to the criteria and revisions

External Factors ABET Guidelines Different external requirements Unclear requirements for MS Programs FAQ for master level programs Different external requirements Government – MoHE (42 Credit Hours) ABET (1 Year of Study) Alignment may be possible for some requirements Accepting students with non ABET Undergraduate Degrees ABET on the International Stage Symposiums at International Locations

Thank you