The On-Site Evaluation Visit Module 6 The On-Site Evaluation Visit

Learning Objectives Following training, the participant will: Know the role and responsibilities of the evaluation team. Know the role and responsibilities of the program director. Have an example of a typical schedule for an on-site visit. Module 6 FEPAC Feb 2010

Evaluation Team Philosophy The Evaluation Team has an obligation to both FEPAC and the Institution being evaluated to be fair and objective. On-site evaluators represent FEPAC to the Institution. Compliance/non-compliance must be to FEPAC accreditation standards. Findings should NOT be Based on personal opinion or Compared to the way the program is conducted at the evaluators’ institutions. Module 6 FEPAC Feb 2010

Evaluator Roles and Conduct Evaluators should Remember that they are guests of the Institution, Demonstrate a professional attitude, behavior, speech, and dress, Be aware of the possibility that the institution faculty and staff may be stressed and/or sensitive to perceived criticism, Remain objective, Only provide FEPAC materials and guidance, Remember that the primary purpose of the on-site evaluation is fact finding, not consultation. Module 6 FEPAC Feb 2010

Evaluator Roles and Conduct Evaluators should NOT Interrogate or lecture to institution personnel, Disagree, argue, or negotiate - just collect facts, Opine as to what the Commission will do, Advise: You are not a consultant, Show Applicant the report and/or comments. Module 6 FEPAC Feb 2010

Evaluator Responsibilities 1. Understand the accreditation standards and the requirements for writing the self-study report. 2. Thoroughly read the self-study before the visit. 3. Compare the reported activities with those required by standards. 4. Plan to arrive so that a team meeting can be held before the initial visit to the Institution. 5. Conduct interviews. Purpose is fact-finding and clarification of submitted documents. 6. Listen/assess situation. Module 6 FEPAC Feb 2010

Evaluator Responsibilities 7. Provide the Team leader with report/comments/ deficiencies in assigned areas. 8. Adhere to the schedule. 9. Have a draft of the evaluation report BEFORE leaving the institution. 10. Send a final report to the Accreditation Director within 10 days of the site visit. Module 6 FEPAC Feb 2010

Additional Responsibilities of the Lead Evaluator (Academic Member) Communicate with the Institution contact. Eliminate any final summation surprises. Act as facilitator for all Evaluation Team members. Confidentially mediate any issues between Team members and Institution staff. Coordinate all Evaluation activities to avoid time-wasting activities and duplicated efforts. Assimilate and finalize all reports for the FEPAC Review Committee and Commission review. Do not leave a copy of the draft Evaluation Report with the Institution. This report is pre-decisional until all reviews and Commission actions are complete. Module 6 FEPAC Feb 2010

Evaluator Checklist Checklists are provided as a resource for Evaluation Team use during evaluations. Can be copied for each member of the Evaluation Team to facilitate the collection of all relevant information. Address each point of a specific standard. Helps evaluators remember what to look for. Module 6 FEPAC Feb 2010

Responsibilities of the Program Director Submit the application for accreditation. Prepare the self-study report, with the assistance and input of other interest parties. Work with the accreditation director and team leader to schedule the on-site visit. Provide the FEPAC office with suggestions for lodging and travel. All is coordinated through the FEPAC office. Module 6 FEPAC Feb 2010

Before the On-Site Visit Pre-Visit Checklist: A checklist of activities that should be conducted by the Lead Evaluator before the On-site visit takes place. Each evaluator should read the institution’s self-study report, prepare questions, and know which documents will provide evidence of compliance or non-compliance with each standard. Module 6 FEPAC Feb 2010

MEETINGS: Pre-Visit Team Meeting This meeting must take place before the evaluation. Typically occurs the evening before the evaluation, either at the hotel or a site selected near the institution being evaluated. Arranged by team leader. Purpose is to plan the visit. Module 6 FEPAC Feb 2010

Meeting Room at the Institution Items that should be in the team meeting room: 1. A lockable meeting room for the on-site team to meet privately. 2. Telephone (speaker phone preferable) for both inside and outside calls. 3. A PC with internet access and a printer. 4. A copy of the application and all supporting documentation. Module 6 FEPAC Feb 2010

MEETINGS: Program Director The first meeting with the applicant Institution generally involves the Program Director. Usually longer than most others. Serves as an introductory meeting. Serves as the first step in the evaluation. Team should get information on: program structure mission workload budget institution management. Module 6 FEPAC Feb 2010

MEETINGS: Faculty & Staff The Team should offer to meet the Faculty and Staff to explain the evaluation process. The Team will defer to the Program Director regarding staff participation in this type of a meeting. There may be some operational, logistic, or space issues that may dictate the format and staff participation. Suggested topics to cover during this meeting are included in the Evaluator’s Manual. Module 6 FEPAC Feb 2010

MEETINGS: Meeting with Administrators/CEO The Team must meet with at least one representative of the Institution’s upper management (Dean, Provost, President). This meeting should: introduce all those involved, briefly outline what the institution can expect during the process, determine the degree of the administration’s support for the program, should be fairly brief, should cover “top level” topics and issues, concerns raised by administrative assessors in self-study. Module 6 FEPAC Feb 2010

MEETINGS: Students Meet with students separate from faculty. Use to assess student perceptions and thoughts about the program. Meet with each faculty member individually if possible. Module 6 FEPAC Feb 2010

Meetings Program Graduates Likely to be done by phone. Can be used to ask about post-graduate assessment efforts, whether graduate feels adequately prepared for workforce, quality of laboratory training. Module 6 FEPAC Feb 2010

MEETINGS: Conduct of Institutional Meetings Meet with each group separately, outside the presence of the Program Director or others. Ask each their views of the program. Compare each group’s answers with self-study documentation and provided materials for Evaluators’ review, to determine the reliability of this information. It is also recommended that the Evaluation Team hold a pre-summation meeting with the Program Director alone. This affords the opportunity to correct misperceptions, if any, before the Summation Conference is held. Module 6 FEPAC Feb 2010

MEETINGS: Summation Conference Held at the end of the evaluation. Includes the Evaluation Team, program director and any others the director chooses. Defer to the Institution’s administration and/or program director’s choice whether this is open to the staff or to a smaller group consisting of department chairs, etc. Suggested topics and format provided in the Evaluators Manual. Module 6 FEPAC Feb 2010

Sample Agenda Visits typically last 2-3 days. This sample agenda is for a two day on-site visit, but could be expanded to three days. Allow at least 2 hours each in the first two days at the end of each day for “team private time” to review and assess where you are in the visit and for 3 hrs on the third day prior to the exit briefing to prepare the report. Module 6 FEPAC Feb 2010

Sample Agenda Day 1 8:30 Opening meeting of Team and Program Director 9:00 Tour of Forensic Science facilities 10:00 Meeting with program faculty (a separate meeting with a Department Chair may be warranted) 10:45 Meeting with College Dean 11:30 Lunch 1:00 Meeting with Provost/President 1:45 Tour of additional on-campus facilities available to forensic science students (library) 3:30 Review of documentation – team only 5:00 Team returns to hotel; dinner and evening meetings for the team; begin preparation of Draft Report Module 6 FEPAC Feb 2010

Sample Agenda Day 2 8:30 Observe class 9:30 Meetings with students 11:00 Phone call to graduates 12:00 Lunch 1:00 Finish draft report 4:00 Exit briefing – team and program representatives Module 6 FEPAC Feb 2010

The On-Site Evaluation Report Module 6 FEPAC Feb 2010

On-Site Evaluation Report: Content Each standard is described in the online system. Record specific observations and evidence. Address all requirements of the standard and initial commission review. Do not indicate compliance or non-compliance, rather tell why or why not a standard is met. Evidence and results, not conclusions. “Minutes from the curriculum review committee meeting were provided”. Be specific in your observations: lack of detail may cause administrative delays. Module 6 FEPAC Feb 2010

On-Site Evaluation Report: Review by Commissioners Team leader sends report to Accreditation Director within 10 days of site visit. Accreditation Director assigns 2-person committee to review report. This may be done in advance of the On-site visit. Committee made up of FEPAC Commissioners. Module 6 FEPAC Feb 2010

On-Site Evaluation Report: Review by Commissioners The Review Committee reviews the report Committee looks for: Evidence of compliance, inconsistent application of standards, ambiguities in discussion of supporting criteria, non-compliance issues, correct assignment of identified problems to the appropriate standard. Lead Evaluator and Review Committee have the authority to change preliminary findings and/or commentary. Module 6 FEPAC Feb 2010

On-Site Evaluation Report: Institution Response The Director of Accreditation sends reviewed report to Applicant. Applicant may dispute findings. Evaluation Reports are “Pre-decisional.” Module 6 FEPAC Feb 2010

Final Commission Review & Decision Making Process Commission makes accreditation decision (January meeting or February AAFS meeting) Each team member receives a copy of the determination letter sent by FEPAC to the Institution. Letter provides: a complete summary of FEPAC deliberations, final decision on the accreditation application for each forensic program reviewed. Module 6 FEPAC Feb 2010

CHEA Disclosure According to FEPAC Policy 3.13 and CHEA requirements, all accreditation decisions and the basis for those decisions must be publicly disclosed. These decisions are posted to the FEPAC website. Programs are given the opportunity to provide a public response related to the decision, which is also posted to the FEPAC website. Module 6 FEPAC Feb 2010

Decision Appeal When a program is denied accreditation, the program may appeal this decision. The program may not appeal conditional or probationary status. During an appeal the Self-Study and team report are provided to the Appeal Board. Therefore, it is imperative that the report and self-study represent a clear and detailed view of the on-site evaluation. Module 6 FEPAC Feb 2010