1. The On-Site Evaluation Visit
Module 6
The On-Site Evaluation Visit

2. Learning Objectives Following training, the participant will:
Know the role and responsibilities of the evaluation team.
Know the role and responsibilities of the program director.
Have an example of a typical schedule for an on-site visit.
Module 6 FEPAC Feb 2010

3. Evaluation Team Philosophy
The Evaluation Team has an obligation to both FEPAC and the Institution being evaluated to be fair and objective.
On-site evaluators represent FEPAC to the Institution.
Compliance/non-compliance must be to FEPAC accreditation standards.
Findings should NOT be
Based on personal opinion or
Compared to the way the program is conducted at the evaluators’ institutions.
Module 6 FEPAC Feb 2010

4. Evaluator Roles and Conduct
Evaluators should
Remember that they are guests of the Institution,
Demonstrate a professional attitude, behavior, speech, and dress,
Be aware of the possibility that the institution faculty and staff may be stressed and/or sensitive to perceived criticism,
Remain objective,
Only provide FEPAC materials and guidance,
Remember that the primary purpose of the on-site evaluation is fact finding, not consultation.
Module 6 FEPAC Feb 2010

5. Evaluator Roles and Conduct
Evaluators should NOT
Interrogate or lecture to institution personnel,
Disagree, argue, or negotiate - just collect facts,
Opine as to what the Commission will do,
Advise: You are not a consultant,
Show Applicant the report and/or comments.
Module 6 FEPAC Feb 2010

6. Evaluator Responsibilities
1. Understand the accreditation standards and the requirements for writing the self-study report.
2. Thoroughly read the self-study before the visit.
3. Compare the reported activities with those required by standards.
4. Plan to arrive so that a team meeting can be held before the initial visit to the Institution.
5. Conduct interviews.
Purpose is fact-finding and clarification of submitted documents.
6. Listen/assess situation.
Module 6 FEPAC Feb 2010

7. Evaluator Responsibilities
7. Provide the Team leader with report/comments/ deficiencies in assigned areas. 8. Adhere to the schedule. 9. Have a draft of the evaluation report BEFORE leaving the institution. 10. Send a final report to the Accreditation Director within 10 days of the site visit.
Module 6 FEPAC Feb 2010

8. Additional Responsibilities of the Lead Evaluator (Academic Member)
Communicate with the Institution contact.
Eliminate any final summation surprises.
Act as facilitator for all Evaluation Team members.
Confidentially mediate any issues between Team members and Institution staff.
Coordinate all Evaluation activities to avoid time-wasting activities and duplicated efforts.
Assimilate and finalize all reports for the FEPAC Review Committee and Commission review.
Do not leave a copy of the draft Evaluation Report with the Institution. This report is pre-decisional until all reviews and Commission actions are complete.
Module 6 FEPAC Feb 2010

9. Evaluator Checklist
Checklists are provided as a resource for Evaluation Team use during evaluations.
Can be copied for each member of the Evaluation Team to facilitate the collection of all relevant information.
Address each point of a specific standard.
Helps evaluators remember what to look for.
Module 6 FEPAC Feb 2010

10. Responsibilities of the Program Director
Submit the application for accreditation.
Prepare the self-study report, with the assistance and input of other interest parties.
Work with the accreditation director and team leader to schedule the on-site visit.
Provide the FEPAC office with suggestions for lodging and travel.
All is coordinated through the FEPAC office.
Module 6 FEPAC Feb 2010

11. Before the On-Site Visit
Pre-Visit Checklist:
A checklist of activities that should be conducted by the Lead Evaluator before the On-site visit takes place.
Each evaluator should read the institution’s self-study report, prepare questions, and know which documents will provide evidence of compliance or non-compliance with each standard.
Module 6 FEPAC Feb 2010

12. MEETINGS: Pre-Visit Team Meeting
This meeting must take place before the evaluation.
Typically occurs the evening before the evaluation, either at the hotel or a site selected near the institution being evaluated.
Arranged by team leader.
Purpose is to plan the visit.
Module 6 FEPAC Feb 2010

13. Meeting Room at the Institution
Items that should be in the team meeting room:
1. A lockable meeting room for the on-site team to meet privately.
2. Telephone (speaker phone preferable) for both inside and outside calls.
3. A PC with internet access and a printer.
4. A copy of the application and all supporting documentation.
Module 6 FEPAC Feb 2010

14. MEETINGS: Program Director
The first meeting with the applicant Institution generally involves the Program Director.
Usually longer than most others.
Serves as an introductory meeting.
Serves as the first step in the evaluation.
Team should get information on:
program structure
mission
workload
budget
institution management.
Module 6 FEPAC Feb 2010

15. MEETINGS: Faculty & Staff
The Team should offer to meet the Faculty and Staff to explain the evaluation process.
The Team will defer to the Program Director regarding staff participation in this type of a meeting.
There may be some operational, logistic, or space issues that may dictate the format and staff participation.
Suggested topics to cover during this meeting are included in the Evaluator’s Manual.
Module 6 FEPAC Feb 2010

16. MEETINGS: Meeting with Administrators/CEO
The Team must meet with at least one representative of the Institution’s upper management (Dean, Provost, President).
This meeting should:
introduce all those involved,
briefly outline what the institution can expect during the process,
determine the degree of the administration’s support for the program,
should be fairly brief,
should cover “top level” topics and issues,
concerns raised by administrative assessors in self-study.
Module 6 FEPAC Feb 2010

17. MEETINGS: Students Meet with students separate from faculty.
Use to assess student perceptions and thoughts about the program.
Meet with each faculty member individually if possible.
Module 6 FEPAC Feb 2010

18. Meetings Program Graduates
Likely to be done by phone.
Can be used to ask about post-graduate assessment efforts, whether graduate feels adequately prepared for workforce, quality of laboratory training.
Module 6 FEPAC Feb 2010

19. MEETINGS: Conduct of Institutional Meetings
Meet with each group separately, outside the presence of the Program Director or others.
Ask each their views of the program.
Compare each group’s answers with self-study documentation and provided materials for Evaluators’ review, to determine the reliability of this information.
It is also recommended that the Evaluation Team hold a pre-summation meeting with the Program Director alone. This affords the opportunity to correct misperceptions, if any, before the Summation Conference is held.
Module 6 FEPAC Feb 2010

20. MEETINGS: Summation Conference
Held at the end of the evaluation.
Includes the Evaluation Team, program director and any others the director chooses.
Defer to the Institution’s administration and/or program director’s choice whether this is open to the staff or to a smaller group consisting of department chairs, etc.
Suggested topics and format provided in the Evaluators Manual.
Module 6 FEPAC Feb 2010

21. Sample Agenda Visits typically last 2-3 days.
This sample agenda is for a two day on-site visit, but could be expanded to three days.
Allow at least 2 hours each in the first two days at the end of each day for “team private time” to review and assess where you are in the visit and for 3 hrs on the third day prior to the exit briefing to prepare the report.
Module 6 FEPAC Feb 2010

22. Sample Agenda Day 1 8:30 Opening meeting of Team and Program Director
9:00 Tour of Forensic Science facilities
10:00 Meeting with program faculty (a separate meeting with a Department Chair may be warranted)
10:45 Meeting with College Dean
11:30 Lunch
1:00 Meeting with Provost/President
1:45 Tour of additional on-campus facilities available to forensic science students (library)
3:30 Review of documentation – team only
5:00 Team returns to hotel; dinner and evening meetings for the team;
begin preparation of Draft Report
Module 6 FEPAC Feb 2010

23. Sample Agenda Day 2 8:30 Observe class 9:30 Meetings with students
11:00 Phone call to graduates
12:00 Lunch
1:00 Finish draft report
4:00 Exit briefing – team and program representatives
Module 6 FEPAC Feb 2010

24. The On-Site Evaluation Report
Module 6 FEPAC Feb 2010

25. On-Site Evaluation Report: Content
Each standard is described in the online system.
Record specific observations and evidence.
Address all requirements of the standard and initial commission review.
Do not indicate compliance or non-compliance, rather tell why or why not a standard is met. Evidence and results, not conclusions. “Minutes from the curriculum review committee meeting were provided”.
Be specific in your observations: lack of detail may cause administrative delays.
Module 6 FEPAC Feb 2010

26. On-Site Evaluation Report: Review by Commissioners
Team leader sends report to Accreditation Director within 10 days of site visit.
Accreditation Director assigns 2-person committee to review report. This may be done in advance of the On-site visit.
Committee made up of FEPAC Commissioners.
Module 6 FEPAC Feb 2010

27. On-Site Evaluation Report: Review by Commissioners
The Review Committee reviews the report
Committee looks for:
Evidence of compliance,
inconsistent application of standards,
ambiguities in discussion of supporting criteria,
non-compliance issues,
correct assignment of identified problems to the appropriate standard.
Lead Evaluator and Review Committee have the authority to change preliminary findings and/or commentary.
Module 6 FEPAC Feb 2010

28. On-Site Evaluation Report: Institution Response
The Director of Accreditation sends reviewed report to Applicant.
Applicant may dispute findings.
Evaluation Reports are “Pre-decisional.”
Module 6 FEPAC Feb 2010

29. Final Commission Review & Decision Making Process
Commission makes accreditation decision
(January meeting or February AAFS meeting)
Each team member receives a copy of the determination letter sent by FEPAC to the Institution.
Letter provides:
a complete summary of FEPAC deliberations,
final decision on the accreditation application for each forensic program reviewed.
Module 6 FEPAC Feb 2010

30. CHEA Disclosure
According to FEPAC Policy 3.13 and CHEA requirements, all accreditation decisions and the basis for those decisions must be publicly disclosed.
These decisions are posted to the FEPAC website.
Programs are given the opportunity to provide a public response related to the decision, which is also posted to the FEPAC website.
Module 6 FEPAC Feb 2010

31. Decision Appeal
When a program is denied accreditation, the program may appeal this decision. The program may not appeal conditional or probationary status.
During an appeal the Self-Study and team report are provided to the Appeal Board. Therefore, it is imperative that the report and self-study represent a clear and detailed view of the on-site evaluation.
Module 6 FEPAC Feb 2010