1. Fire & Safety Division Evaluator Certification
Charlene Jordan-Jones
January 20 & 21, 2017

2. https://evaluator.jibc.ca/

3. Course Overview
The intent of this course is train Fire & Safety Division (FSD) evaluators.
Upon successful completion of this course evaluators should be able to:
Explain the relationship between accreditation, certification and evaluation of learning;
Adhere to JIBC policy, procedures and FSD Accreditation Operational Guidelines;
Evaluate students’ practical skills;
Administer and proctor written exams; and
Grade and provide feedback on students’ participation, projects and assignments.
This open course is designed for current and future JIBC evaluators.
Current evaluators: must complete this course and the follow-up written exam
New evaluators: must apply as an evaluator and be accepted prior to completing this course plus the written exam
There is no time limit for completion of this course. The course is self-paced and any activities completed are not graded but are designed to reinforce learning

4. What is Evaluation of Learning?
According to JIBC’s Evaluation of Learning Policy (policy #3302), “Evaluation of learning is the measurement of student learning and performance in relation to stated Justice Institute of British Columbia (JIBC) learning outcomes” (JIBC, 2015). This involves ensuring that students are informed about the learning objectives of a course (typically via a course outline or syllabus) and the methods that will be used to evaluate their learning.
In this course, for instance, it is expected that students will learn how to do the following:
Recognize the relationship between accreditation, certification and evaluation of learning;
Adhere to JIBC policy, procedures and Fire & Safety Division Accreditation Operational Guidelines;
Evaluate students’ practical skills;
Administer and proctor written exams; and
Grade and provide feedback on students’ participation, projects and assignments.

5. New Evaluators vs Current Evaluators
Complete the course by studying the content and complete a 40-question multiple-choice written exam and score a grade of 75% or higher. This exam will be both proctored and timed.
Submit your application with the following documentation:
CV or Resume showing proof of Fire Service experience (min. of 5 years) and Subject Matter Expertise (to at least a level higher than you are applying to evaluate)
Proof of NFPA 1041 Fire Service Instructor I
Proof of a minimum of 3 years teaching or instructional experience
Letter of support and reference from your fire department
Current Evaluators
Complete the course by studying the content and complete the same 40-question multiple-choice written exam and score a minimum of 75%. This exam will be timed and proctored.
All Evaluators
Must conduct a minimum of four (4) documented FSD student evaluations per year. Qualifying evaluations include the conducting of skills-based assessments and/or the marking of assignments for face-to-face, blended or online courses.
In addition, all JIBC evaluators must re-certify every three (3) years following their initial certification. Note that re-certification only includes cognitive assessment (i.e., there is no practical skills evaluation requirement). However, the FSD intends to periodically audit, train and provide professional development opportunities for all evaluators to continue to hone their skills.

6. Information in Evaluator Course
Overview of accreditation
JIBC policies and FSD accreditation operational guidelines
Understanding NFPA Standards
Practical Skill Evaluations
Written Exams
Assignments
Getting certified

7. Practical Skill Evaluations
JIBC evaluators conduct criterion-referenced evaluations, which involves measuring student performance against set NFPA® standards
An evaluator is assessing student performance as objectively as possible
To support test validity and reliability, FSD practical evaluations are conducted with the assistance of practical skills sheets
Each skill sheet (i.e., skills scenario) is developed by a validation team of Subject Matter Experts (SMEs) and a trial is conducted prior to skill sheet launch in order to uncover any challenges and increase test validity

8. Practical Skill Evaluations
Preparation
Ways to effectively prepare to conduct a practical evaluation:
Ensure there is no perceived or actual conflict of interest
Familiarize yourself with the skill sheets/performance checklists being used
Inspect the station, facilities and/or equipment to ensure availability, functionality and safety
Ensure candidates understand exactly what they will be tested on (e.g., ensure candidates have been provided with skill sheets to review in advance)
Be sure to adhere to evaluator scripts where these have been provided
Answer any immediate questions candidates may have
Outline expectations around testing confidentiality and process (e.g., reinforce the need for other candidates being evaluated to absent themselves from the test site until it is their turn to be evaluated)

9. Practical Skills Evaluations
Delivery
Some pointers on how to effectively conduct a practical evaluation:
Actively use the provided skill sheet(s)—they are not for reference only!
Instruct candidates to verbalize their thought process where visual observation alone would not suffice
Observe candidates closely, paying explicit attention to the skills being demonstrated but without getting in the way
If necessary, reposition yourself to see fine detail (e.g., move in closer to inspect a halyard knot or an SCBA cylinder connection)
Stop the activity if any clear danger presents itself
Ensure appropriate safety procedures are followed
Don’t interrupt unless there is justifiable reason

10. Practical Skill Evaluation
Feedback & Administration
Ensure that skill sheets are properly completed and submitted
Make sure that a student’s name, student ID number, home department or class number, and the date on which the evaluation occurred are clearly indicated
Indicate whether that student’s performance was successful or unsuccessful, and if applicable, their score
Inform the candidate of his/her test results with respecting student confidentiality
Obtain the student’s signature, which is not meant to prove necessarily that the student agrees with the results communicated, but acknowledges that the evaluation took place
Take the time to recognize and reinforce what the candidate did well
Point out how their performance could be improved
Ask candidate(s) if they have any questions

11. Written Exams
Basic principles that always apply when invigilating a written exam include:
Conflict of interest must be avoided
Acting as an authorized agent of the FSD, proctors must agree not to disclose any information regarding exam content to any person other than authorized FSD personnel
Students must present photo identification in order to write an exam in order to confirm their identity
Restroom breaks are allowed for exams that exceed three hours in length
Proctors are to explain emergency evacuation procedures - safety first! In the unlikely event of a fire or power failure, etc., the exam will need to be rescheduled.
Students should be seated with sufficient space between them in order to help prevent the risk of cheating
Course materials (e.g., textbooks, notes, etc.) are not permitted in the examination room unless officially authorized by the JIBC Program Manager or Coordinator, as part of the examination process
The exam room must be free of posters, bulletin boards, equipment or models that could inadvertently be of assistance to students

12. Written Exams No food or drinks are permitted in the examination room
Mobile devices are not permitted in the testing area
The exam start and end times must be communicated to students, and notice should be provided when they have reached the half-way mark for exam completion
There is to be no talking during an exam
Proctors are not allowed to discuss the exam content or assist students in answering questions in any way
FSD students are allowed to challenge test items following exam completion
If a student would like to challenge a particular question they should be instructed to their challenge or concern to an assigned, program-specific address (e.g., OR the responsible Manager or Coordinator/Fire Officer if the program has no address of this nature.
In the event a question is successfully challenged, exam results will be adjusted for all students in that particular test group
Pursuant to the JIBC Student Code of Conduct, any infractions, such as cheating, must be referred as quickly as possible to the appropriate JIBC Manager or Coordinator

13. Assignments
Some FSD courses are either exclusively or primarily assignment/project-based
A student’s performance must be measured against the applicable NFPA® standard and not relative to the performance of other learners
The exact nature of assignments varies considerably, as each is purposefully designed to allow students the opportunity to demonstrate specific knowledge and skills
Assignments may include tasks such as writing a report, drafting a policy, or completing a research project, etc
In order to support evaluators in the assessment of project or assignment-based work, assignments are typically accompanied by a marking rubric
A rubric is a tool that clearly indicates marking criteria

14. How to Use the Marking Rubric
A rubric is a tool that clearly indicates marking criteria
It can be used for marking assignments, class participation, discussion board posts or overall grades
Please feel free to include any additional comments to a learner that do not fit within the rubric’s criteria
Once you have reviewed the component using the rubric, you will decide upon a final grade for the component based on the rubric.

15. Getting Certified Step 1 Complete the Evaluator Application Form
Attach supporting documents to the application
New Evaluators
CV or Resume showing proof of Fire Service experience (min. of 5 years) and Subject Matter Expertise (to at least a level higher than you are applying to evaluate)
Proof of NFPA 1041 Fire Service Instructor I
Proof of a minimum of 3 years teaching or instructional experience
Letter of support and reference from your fire department
Re-certification for Current Evaluators (every 3 years)
Evaluators are required to perform a minimum of four (4) documented evaluations/year
Evaluators should keep a log or documentation of evaluations performed, as proof may be required for re-certification
Step 3 Submit your application with supporting evidence

16. Getting Certified Step 4
Application will be reviewed by the JIBC FSD staff
Once accepted, JIBC FSD staff will notify you about the registration and exam process
Your exam will be scheduled and the assigned proctor will be sent the exam access information and proctor agreement for online exam
Step 5 Complete online exam with a minimum passing grade of 75%
Step 6
JIBC FSD staff will add you to the JIBC Certified Evaluator Registry
JIBC FSD Program Manager/Coordinator will contact you with information about evaluations for the area you are certified
Step 7
Congratulations! You are now a JIBC Certified Evaluator
Remember to re-certify in three (3) years!