Accreditation Cycle EAC Accreditation Site Visit XYZ University October 24-26, 2004 Introductions Expectations for the team –documents –role of observers –support program assignments Overview of Schedule EC2000 Visit details Issues Program overviews

Accreditation Cycle Important Team Expectations Evaluators represent the EAC of ABET We are accrediting programs to state to the public that they satisfy the criteria Team effort—team decisions Confidentiality Conflict of interest – every visitor should have signed a conflict-of-interest statement Observers –no evaluative statements –exit interview—thank-you only

Accreditation Cycle Confidentiality Do not discuss conclusions with faculty, students, and others Keep all materials until the July 2005 EAC meeting. At conclusion of accreditation process (July 2005) materials are to be destroyed Information specific to the institution is to remain confidential without time limit Institutional data is confidential except with written authorization of institution ABET materials only released by ABET staff

Accreditation Cycle Communication Maintain open line of communication with the department head Identify deficiencies as soon as possible Discuss all issues with the department head at the debriefing Do not discuss the recommended accreditation action with anyone except team members

Accreditation Cycle Today’s Schedule Sunday meetings –Sunday a.m. Brief: initial thoughts and recommendations –Sunday p.m. More detailed discussion “Pre-visit” recommended action Questions Lunch

Accreditation Cycle Visit Schedule (Example) Sunday, October 24 10:30 AM – 11:30ABET team meeting in team room Brief: initial thoughts and recommendations 11:30/noonTeam lunch or evaluator/chair meetings 1:30 - 4:30 PMMeetings/Examine course materials and documentation of outcomes. 4:30-6:00 PMTeam meeting More detailed discussion “Pre-visit” recommended action 6:00 PMABET team dinner

Accreditation Cycle Visit Schedule (Example) Monday, October 25 [7:30 AMTransit time—meet in hotel lobby] 8:00 AM - 9:00 AM Team Meeting with engineering administration. 9:00 AM - 10:00 AM Evaluators meet with chairs or begin meetings with faculty 10:00 AM - 12:00 NMeet with faculty members to discuss program processes, outcomes, improvement 12:00 N - 1:30 PMJoint ABET team and XYZ administration lunch 1:30 PM - 3:00 PM Visit supporting departments 3:00 PM - 4:00 PM Meet with students or faculty 4:00 PM - 4:45 PM Meet with faculty or students 5:00 PM - 11:00 PM ABET team meeting and dinner

Accreditation Cycle Visit Schedule (Example) Tuesday, October 26 [7:30 AM Meet in hotel lobby—evaluators may choose to schedule meetings earlier—on their own] 8:00 AM - 11:30 AM Inspect classrooms, laboratories, offices, equipment >> 11:45 AM - 12:30 AM ABET team meeting to prepare debriefing 12:30 PM - 1:00 PMDebrief chairs/dean 1:15 PM - 2:45 PM ABET team meeting and working lunch 3:00 PM - 4:00 PM Summary meeting with administration 4:00 PMDepart for airport/hotel

Accreditation Cycle Support Program Assignments MathPEV name * Time Location PhysicsPEV nameTime Location ChemistryPEV nameTime Location Hum/S.S.PEV nameTime Location LibrariesPEV nameTime Location ComputingPEV nameTime Location * TC may help in support program reviews

Accreditation Cycle Visit details Important forms –Transcript & curriculum analyses –Level of Implementation –Program Audit Form + Explanation of Shortcoming (left on campus) –Draft Statement—NEW FORMAT—on disc to me –Program Evaluator Worksheet—you keep –Short form—recommended actions –Program Evaluator Report

Accreditation Cycle PROGRAM AUDIT FORM (PROVIDE A COPY TO INSTITUTION AT EXIT MEETING) If it doesn’t have this column, you are using an old form

Accreditation Cycle Exit Statement Format INTRODUCTION—USEFUL PROGRAM STATISTICS PROGRAM ISSUES –Strengths (special, unique or particularly conspicuous strengths) –Deficiencies (In order, only for those criteria where deficiencies exist) 1.XXX 2.etc. –Weaknesses (In order, only for those criteria where weaknesses exist) 1. YYY 2.etc. –Concerns (In order, where concerns exist) 1.ZZZ 2.etc. –Observations (do not have to relate to criteria) 1.etc.

Accreditation Cycle EC2000 Visit details Consistency—things to look for –Evaluation and measurement of objectives –Assessment and demonstration of outcomes –Program improvement (closing the loop) –Curricular and program issues –Faculty and students As they relate to undergraduate education

Accreditation Cycle Terminology Deficiency -- criterion is NOT satisfied. Weakness -- criterion is satisfied, but lacks strength of compliance to assure the quality of the program will not be compromised prior to next general review. Concern -- criterion is satisfied, but potential exists for non-satisfaction in the near future.

Accreditation Cycle Working Definition of Key Terms Deficiency: assigned to any criterion that is totally or largely unmet Weakness: criterion is met to some meaningful extent, but compliance is insufficient to fully satisfy requirements Concern: criterion is fully met, but there is potential for non-compliance in the near future Observation: general commentary possibly, but not necessarily, related to criteria

Accreditation Cycle Limit Use of Key Terms Use Key Term only in reference to overall evaluation of each criterion

Accreditation Cycle Accreditation Actions NGRNext General Review IRInterim Report IVInterim Visit SCShow Cause REReport Extended VEVisit Extended SEShow Cause Extended NANot to Accredit Interim visit only

Accreditation Cycle Linking Actions to Terms TermResults of Evaluation Weaknesses ?noyesyes-- Deficiencies ?nononoyes Type of ReviewPossible Actions General (comprehensive)NGRIRIV SC or NA (following a SC)NGRIRIV NA or SC* New ProgramNGRIRIVNA * Show Cause can follow a Show Cause only if progress is demonstrated. General Review Visits

Accreditation Cycle Duration of Accreditation Actions Duration Weak?Def?Action (years) NoNoNGRNext General Review6 YesNo IRInterim Report2 YesNoIVInterim Visit2 --Yes SCShow Cause1 General Review Visits

Accreditation Cycle Interim Actions Interim evaluations focus on identified deficiencies (if Show Cause ) or weaknesses. Interim Visit: recommended only when degree of resolution requires the evaluator to be on campus and cannot be determined by review of a report or when previous written information has not been effective in providing the necessary evidence. (e.g., student work). A new team is sent. Interim Report: recommended when resolution of shortcomings can be described by a report (e.g., faculty hiring); the current program evaluator and team chair may review the interim report, assess progress, prepare a statement (Minor U) and recommend accreditation action.

Accreditation Cycle Consistency Issues for the Team The depth and completeness of the evaluation from program to program Consistency across all programs in an institution The assignment of appropriate key terms (deficiency, weakness, concern) to describe shortcomings For weaknesses, consistency on interim recommendations--IR vs IV

Accreditation Cycle Definitions Program Educational Objectives Program educational objectives are broad statements that describe the career and professional accomplishments that the program is preparing graduates to achieve.

Accreditation Cycle Program Educational Objectives (continued) These are also often referred to by institutions as goals, career outcomes, or standards. There are two types of objectives those that all graduates are expected to accomplish and those that some subgroups, but not all graduates, are expected to accomplish. The audiences for objective statements are normally external constituents, such as prospective students, employers, and transfer institutions.

Accreditation Cycle Program Outcomes Program outcomes are narrower statements that describe what students are expected to know and able to do by the time of graduation. These relate to the skills, knowledge, and behaviors that students acquire in their matriculation through the program.

Accreditation Cycle Assessment Assessment is one or more processes that identify, collect, and prepare data to evaluate the achievement of program outcomes and program educational objectives. This reference to “or program improvement” seems out of place, since evaluating achievement of outcomes and objectives is a prerequisite to improvement, not an alternative. Besides, in our set of definitions, improvement comes after ‘evaluation’ rather than after ‘assessment.”

Accreditation Cycle Assessment (continued) Often, the entire process is referred to as assessment, and the program or institution does not subdivide the overall process into component parts. While assessment data are useful for display during accreditation reviews, data alone do not provide the documented evidence for continuous improvement.

Accreditation Cycle Evaluation Evaluation is one or more processes for interpreting the data and evidence accumulated through assessment practices. Evaluation determines the extent to which program outcomes or program educational objectives are being achieved, and results in decisions and actions to improve the program..

Accreditation Cycle Evaluation (continued) Evaluation is often referred to generically by the term “assessment.” Also, “closing the loop” is a term used to describe the process of evaluation of assessment data and making improvements as a result of the analysis. It is expected that the programs would perform the evaluation of assessment data, and the documented results of the evaluation would serve as evidence of achievement of objectives and outcomes and/or evidence of an effective continuous improvement system.

Accreditation Cycle Consistency Issues for Criteria 2 & 3--where issues arise Most consistency issues are likely to center on Criteria 2 & 3 TCs and evaluators should understand the institution’s use of its own terminology relevant to these criteria

Accreditation Cycle Level of Expectation How should you assess the claim “We’re working on it”? –Educational objectives must be defined and based on needs of constituencies –All major outcomes must be defined and processes in place to assure continuous improvement –Assessment is an on-going activity and takes some time to reach full maturity, but some data should be available and results demonstrated

Accreditation Cycle  Exactly what attributes must each graduate have? –A system must be in place to ensure that all graduates have, to some minimum extent, achieved the prescribed outcomes (Criterion 3) and all elements of Professional Component (Criterion 4) –The level of achievement may vary, consistent with program objectives Level of Expectation

Accreditation Cycle Criterion 2 Program Educational Objectives Program Educational Objectives-- statements that describe the expected accomplishments of graduates during the first few years after graduation 1 Unique to the program and institution Consistent in all publications 1 Working definition from ABET Faculty Workshops

Accreditation Cycle Criterion 2 Program Educational Objectives Consider a deficiency if the general intent of Criterion 2 is not met. Contributing factors may include: –no involvement of constituencies –no process-oriented approach to achieving objectives (links to curriculum) –no process-oriented approach to evaluate achievement of objectives –no data that demonstrate the extent to which objectives are met –no evidence of program improvement based on evaluative processes

Accreditation Cycle Criterion 2 Program Educational Objectives Consider a weakness if the general intent of Criterion 2 is met to some extent, but not fully met. Contributing factor may include: –Objectives are published but are not accessible to constituencies and potential students –limited or ad hoc involvement of constituencies –incomplete process to achieving objectives (links to curriculum are not clear) –incomplete process-oriented approach to evaluating achievement of objectives –evidence of program improvement based on ad hoc processes

Accreditation Cycle Criterion 2 Program Educational Objectives Consider a concern if the general intent of the criterion is fully met, but minor issues may lead to lack of compliance in the future. Contributing factors may include: –Objectives are published, but are changed frequently –Objectives are evaluated, but there is limited involvement of constituencies in this process or it varies from year to year (2b) –Program improvement processes may rely too heavily on one person

Accreditation Cycle Criterion 3 Program Outcomes & Assessment Program outcomes: statements that describe what students are expected to know and are able to do by the time of graduation, the achievement of which indicates that the student is equipped to achieve the Program Educational Objectives 1 ABET designated (a-k) included in some way Program may add others 1 Working definition from ABET Regional Faculty Workshops

Accreditation Cycle Criterion 3 Program Outcomes and Assessment Consider a deficiency if the general intent of Criterion 3 is not met. Contributing factors may include: –No documented working process(es) to produce outcomes –Loop not closed on any outcomes –Absence of defined goals and documented assessment results –No evidence that demonstrates achievement of outcomes –No evidence of efforts at program improvement based on assessment

Accreditation Cycle Criterion 3 Program Outcomes and Assessment Consider a weakness if the general intent of Criterion 3 is met to some extent, but not fully met. Contributing factor may include : –Absence of a working process(es) to produce some outcomes –Loop closed on some outcomes –Defined goals and documented assessment results for some outcomes –Absence of demonstration of a small number of outcomes –Incomplete or ad hoc evidence of efforts at program improvement based on assessment

Accreditation Cycle Criterion 3 Program Outcomes & Assessment Consider a concern if the general intent of the criterion is fully met, but minor issues may lead to lack of compliance in the future. Contributing factors may include: –Process to produce some outcomes is possibly inconsistent and may lead to circumstances in which their quality is insufficient to meet program metrics –Loop closed on most outcomes, but some important assessment results have not been acted upon –Inconsistent coverage or demonstration of a small number of outcomes e.g., with variation from year to year in the same course taught by different faculty members—may be overly dependent on one person

Accreditation Cycle 30-Day Due Process Response from institution TC edits “Final” Statement to add 30-day response (Major U or Minor U format); checks with evaluators if needed TC updates PAF and Short Form EAC takes final accreditation action ABET sends Final Statement and accreditation letter to institution Team Chair and Program Evaluators fill out on- line evaluation forms Post-Visit Process

Accreditation Cycle Team Chair Editor EAC Chair Team Institution: Due Process 14-day response n Institution n EAC n Team ABET Headquarters Draft statement  Due Process: 14-Day Response

Due Process Team Chair Editor EAC Chair ABET HQ Team Due Process: Institutional Response n Institution n EAC n Team Due Process: After 30-Day Response

Accreditation Cycle Team Chair Editor EAC Chair Team ABET Headquarters Institution EAC Meeting Final statement   ABET President Professional Societies Final Statement