Fire & Safety Division Evaluator Certification Charlene Jordan-Jones January 20 & 21, 2017

https://evaluator.jibc.ca/

Course Overview The intent of this course is train Fire & Safety Division (FSD) evaluators. Upon successful completion of this course evaluators should be able to: Explain the relationship between accreditation, certification and evaluation of learning; Adhere to JIBC policy, procedures and FSD Accreditation Operational Guidelines; Evaluate students’ practical skills; Administer and proctor written exams; and Grade and provide feedback on students’ participation, projects and assignments. This open course is designed for current and future JIBC evaluators. Current evaluators: must complete this course and the follow-up written exam New evaluators: must apply as an evaluator and be accepted prior to completing this course plus the written exam There is no time limit for completion of this course. The course is self-paced and any activities completed are not graded but are designed to reinforce learning

What is Evaluation of Learning? According to JIBC’s Evaluation of Learning Policy (policy #3302), “Evaluation of learning is the measurement of student learning and performance in relation to stated Justice Institute of British Columbia (JIBC) learning outcomes” (JIBC, 2015). This involves ensuring that students are informed about the learning objectives of a course (typically via a course outline or syllabus) and the methods that will be used to evaluate their learning. In this course, for instance, it is expected that students will learn how to do the following: Recognize the relationship between accreditation, certification and evaluation of learning; Adhere to JIBC policy, procedures and Fire & Safety Division Accreditation Operational Guidelines; Evaluate students’ practical skills; Administer and proctor written exams; and Grade and provide feedback on students’ participation, projects and assignments.

New Evaluators vs Current Evaluators Complete the course by studying the content and complete a 40-question multiple-choice written exam and score a grade of 75% or higher. This exam will be both proctored and timed. Submit your application with the following documentation: CV or Resume showing proof of Fire Service experience (min. of 5 years) and Subject Matter Expertise (to at least a level higher than you are applying to evaluate) Proof of NFPA 1041 Fire Service Instructor I Proof of a minimum of 3 years teaching or instructional experience Letter of support and reference from your fire department Current Evaluators Complete the course by studying the content and complete the same 40-question multiple-choice written exam and score a minimum of 75%. This exam will be timed and proctored. All Evaluators Must conduct a minimum of four (4) documented FSD student evaluations per year. Qualifying evaluations include the conducting of skills-based assessments and/or the marking of assignments for face-to-face, blended or online courses. In addition, all JIBC evaluators must re-certify every three (3) years following their initial certification. Note that re-certification only includes cognitive assessment (i.e., there is no practical skills evaluation requirement). However, the FSD intends to periodically audit, train and provide professional development opportunities for all evaluators to continue to hone their skills.

Information in Evaluator Course Overview of accreditation JIBC policies and FSD accreditation operational guidelines Understanding NFPA Standards Practical Skill Evaluations Written Exams Assignments Getting certified

Practical Skill Evaluations JIBC evaluators conduct criterion-referenced evaluations, which involves measuring student performance against set NFPA® standards An evaluator is assessing student performance as objectively as possible To support test validity and reliability, FSD practical evaluations are conducted with the assistance of practical skills sheets Each skill sheet (i.e., skills scenario) is developed by a validation team of Subject Matter Experts (SMEs) and a trial is conducted prior to skill sheet launch in order to uncover any challenges and increase test validity

Practical Skill Evaluations Preparation Ways to effectively prepare to conduct a practical evaluation: Ensure there is no perceived or actual conflict of interest Familiarize yourself with the skill sheets/performance checklists being used Inspect the station, facilities and/or equipment to ensure availability, functionality and safety Ensure candidates understand exactly what they will be tested on (e.g., ensure candidates have been provided with skill sheets to review in advance) Be sure to adhere to evaluator scripts where these have been provided Answer any immediate questions candidates may have Outline expectations around testing confidentiality and process (e.g., reinforce the need for other candidates being evaluated to absent themselves from the test site until it is their turn to be evaluated)

Practical Skills Evaluations Delivery Some pointers on how to effectively conduct a practical evaluation: Actively use the provided skill sheet(s)—they are not for reference only! Instruct candidates to verbalize their thought process where visual observation alone would not suffice Observe candidates closely, paying explicit attention to the skills being demonstrated but without getting in the way If necessary, reposition yourself to see fine detail (e.g., move in closer to inspect a halyard knot or an SCBA cylinder connection) Stop the activity if any clear danger presents itself Ensure appropriate safety procedures are followed Don’t interrupt unless there is justifiable reason

Practical Skill Evaluation Feedback & Administration Ensure that skill sheets are properly completed and submitted Make sure that a student’s name, student ID number, home department or class number, and the date on which the evaluation occurred are clearly indicated Indicate whether that student’s performance was successful or unsuccessful, and if applicable, their score Inform the candidate of his/her test results with respecting student confidentiality Obtain the student’s signature, which is not meant to prove necessarily that the student agrees with the results communicated, but acknowledges that the evaluation took place Take the time to recognize and reinforce what the candidate did well Point out how their performance could be improved Ask candidate(s) if they have any questions

Written Exams Basic principles that always apply when invigilating a written exam include: Conflict of interest must be avoided Acting as an authorized agent of the FSD, proctors must agree not to disclose any information regarding exam content to any person other than authorized FSD personnel Students must present photo identification in order to write an exam in order to confirm their identity Restroom breaks are allowed for exams that exceed three hours in length Proctors are to explain emergency evacuation procedures - safety first! In the unlikely event of a fire or power failure, etc., the exam will need to be rescheduled. Students should be seated with sufficient space between them in order to help prevent the risk of cheating Course materials (e.g., textbooks, notes, etc.) are not permitted in the examination room unless officially authorized by the JIBC Program Manager or Coordinator, as part of the examination process The exam room must be free of posters, bulletin boards, equipment or models that could inadvertently be of assistance to students

Written Exams No food or drinks are permitted in the examination room Mobile devices are not permitted in the testing area The exam start and end times must be communicated to students, and notice should be provided when they have reached the half-way mark for exam completion There is to be no talking during an exam Proctors are not allowed to discuss the exam content or assist students in answering questions in any way FSD students are allowed to challenge test items following exam completion If a student would like to challenge a particular question they should be instructed to email their challenge or concern to an assigned, program-specific email address (e.g., basicffexams@jibc.ca) OR email the responsible Manager or Coordinator/Fire Officer if the program has no email address of this nature. In the event a question is successfully challenged, exam results will be adjusted for all students in that particular test group Pursuant to the JIBC Student Code of Conduct, any infractions, such as cheating, must be referred as quickly as possible to the appropriate JIBC Manager or Coordinator

Assignments Some FSD courses are either exclusively or primarily assignment/project-based A student’s performance must be measured against the applicable NFPA® standard and not relative to the performance of other learners The exact nature of assignments varies considerably, as each is purposefully designed to allow students the opportunity to demonstrate specific knowledge and skills Assignments may include tasks such as writing a report, drafting a policy, or completing a research project, etc In order to support evaluators in the assessment of project or assignment-based work, assignments are typically accompanied by a marking rubric A rubric is a tool that clearly indicates marking criteria

How to Use the Marking Rubric A rubric is a tool that clearly indicates marking criteria It can be used for marking assignments, class participation, discussion board posts or overall grades Please feel free to include any additional comments to a learner that do not fit within the rubric’s criteria Once you have reviewed the component using the rubric, you will decide upon a final grade for the component based on the rubric.

Getting Certified Step 1 Complete the Evaluator Application Form Attach supporting documents to the application New Evaluators CV or Resume showing proof of Fire Service experience (min. of 5 years) and Subject Matter Expertise (to at least a level higher than you are applying to evaluate) Proof of NFPA 1041 Fire Service Instructor I Proof of a minimum of 3 years teaching or instructional experience Letter of support and reference from your fire department Re-certification for Current Evaluators (every 3 years) Evaluators are required to perform a minimum of four (4) documented evaluations/year Evaluators should keep a log or documentation of evaluations performed, as proof may be required for re-certification Step 3 Submit your application with supporting evidence

Getting Certified Step 4 Application will be reviewed by the JIBC FSD staff Once accepted, JIBC FSD staff will notify you about the registration and exam process Your exam will be scheduled and the assigned proctor will be sent the exam access information and proctor agreement for online exam Step 5 Complete online exam with a minimum passing grade of 75% Step 6 JIBC FSD staff will add you to the JIBC Certified Evaluator Registry JIBC FSD Program Manager/Coordinator will contact you with information about evaluations for the area you are certified Step 7 Congratulations! You are now a JIBC Certified Evaluator Remember to re-certify in three (3) years!