First level incident commanders’ decision-making assessment for learning Stella Polikarpus

What am I talking about? Teaching means designing the learning environment in a way that it supports students learning. Learning means to seek new knowledge and understanding, exploring alternative solutions and welcome different perspectives, think evaluatively and check impact of ones transformative actions.

Work with me! Perhaps you learn something? Open your PC or phone and type in internet browser Socrative.

Work with me! EASS EASS EAS EA E Open your PC or phone and type in internet browser Socrative.

Learning outcome Student knows the main components of the process of new assessment methodology implementation in Estonia rescue service.

45 227 km² 1 323 820 residents 1st January 2019 Statistikaameti andmetel elas 1. jaanuaril 2019 Eestis 1 323 820 inimest. 1. Estonia has one of the lowest population densities in the world (28 per sq km; 72.5 per sq mi) – placing it 188th in the world. Estonian highest peak is 318 meters (1,043 feet) above sea level 

72 national rescue brigades in Estonian Rescue Board 24/7 In Estonia there are : 72 national rescue brigades 350 shift workers in Estonian Rescue Board 24/7

Level I brigades (72 stations) Level II Stand-by groups (total 12) operational/Bronze level Level II Stand-by groups (total 12) operational or tactical/Silver level Level III Regional duty officer in charge tactical/Silver Level IV Operational duty officer of the Estonian Rescue Board strategic/Gold

The ESTONIAN ACADEMY OF SECURITY SCIENCES (EASS) The ESTONIAN RESCUE BOARD (ERB)

Work with me! Room nr: EASS Open your PC or phone and type in internet browser Socrative.

AIM to give an overview of the process of new assessment methodology implementation in Estonia rescue service. In presentation the e-learning course, Effective Command platform and XVR On-Scene software is showed. The summary of assessment results is presented together with suggestions of how to improve the decision-making training.

The presentation structure Before During After 3 years´ project of training and assessment of first level incident commanders 2016-2018 Assessment project of first level incident commanders was carried out in cooperation with the Estonian Academy of Security Sciences (EASS) and Estonian Rescue Board (ERB) in years 2016-2018. The assessment method is called Effective Command [1]. Assessors where trained and certified by Dr Katherine Lamb from United Kingdom. Assessment criteria where compared with Estonian Rescue unit leader occupational qualification standard [2] to build the development and assessment system connected to occupational qualification framework. Additional training curriculum was developed [3] together with scenarios in software XVR On-Scene [4]. Incident commanders in Estonia where divided in three groups of 120 commanders, as that was maximum EASS could assess in a year.

Occupational qualifications Rescue unit leader, level 5 Rescuer, level 4 Rescue commander, level 6 IC is incident commander

Preparations in ERB Changing the rescue unit leaders´ attestation regulations Incident command competences being assessed in EASS every 3 years If failed, then re-attestation in a year Meanings of the assessment colours Finances training (3 years) Selecting staff to be attestated Assessment project of first level incident commanders was carried out in cooperation with the Estonian Academy of Security Sciences (EASS) and Estonian Rescue Board (ERB) in years 2016-2018.

Preparations in EASS Mapping competences with standard Training the assessors (12 persons) Incident command XVR user competences Effective Command and SPAR decision making module The additional training curriculum Developing the assessment scenarios Assessors where trained and certified by Dr Katherine Lamb from United Kingdom. Assessment criteria where compared with Estonian Rescue unit leader occupational qualification standard [2] to build the development and assessment system connected to occupational qualification framework. Additional training curriculum was developed [3] together with scenarios in software XVR On-Scene [4]. Incident commanders in Estonia where divided in three groups of 120 commanders, as that was maximum EASS could assess in a year.

Assessment method The assessment method is called Effective Command [1]. Assessors where trained and certified by Dr Katherine Lamb from United Kingdom. Assessment criteria where compared with Estonian Rescue unit leader occupational qualification standard [2] to build the development and assessment system connected to occupational qualification framework. Additional training curriculum was developed [3] together with scenarios in software XVR On-Scene [4]. Incident commanders in Estonia where divided in three groups of 120 commanders, as that was maximum EASS could assess in a year. Adopted from dr Katherine Lamb presentation in ForeFront meeting April 2019

SPAR decision module Adapted from Launder & Perry 2014 Assessment project of first level incident commanders was carried out in cooperation with the Estonian Academy of Security Sciences (EASS) and Estonian Rescue Board (ERB) in years 2016-2018. The assessment method is called Effective Command [1]. Assessors where trained and certified by Dr Katherine Lamb from United Kingdom. Assessment criteria where compared with Estonian Rescue unit leader occupational qualification standard [2] to build the development and assessment system connected to occupational qualification framework. Additional training curriculum was developed [3] together with scenarios in software XVR On-Scene [4]. Incident commanders in Estonia where divided in three groups of 120 commanders, as that was maximum EASS could assess in a year.

Development and assessment day 6 training classes for IC Lecture (2 academic hours 1:4) Exercise (2 academic hours 1:1) Assessment (2 academic hours 2:1) The development and assessment day consisted 6 training classes [3]. In 2016 we started with lecture about decision-making module SPAR (Situational Awareness, Plan, Action, Review) [5], followed by one to one training session. In training session, the joystick exercise was carried out first to explain the trainee, how is it possible to collect information in 3D environment. Secondly, the exercise was carried out where traffic accident had occurred in two-level intersection [6]. After that conversation between trainer and trainee was carried out based on SPAR module and Effective Command assessment criteria.

Development and assessment day After lunch the trainers did the assessments in pairs. In dynamic phase of the assessment one assessor had the roll of technical instructor manipulating with XVR. Other made voices to avatars and made sure that important changes in situations are played out to test that the commander reviews the situation. The dynamic phase always ended with second level commander arriving on scene played by one of the assessors. Trainee had to give overview of the situation to the second level commander. Third, the conversation between trainee and assessors was carried out and the certificated filling started. The aim of the conversation is to find out what incident commander was thinking when he commanded and communicated with avatars. Certificate was filled after two assessors had joint understanding of each assessment criteria. Trainee was asked back to room and certificate was introduced and signed digitally by all three.

During the project 2016 30 days 0 day 2017 2 days 2018 35 days 14 days YEARS 1st time assessment 2nd time assessment 2016 30 days 0 day 2017 2 days 2018 35 days 14 days

HITSA Moodle e-learning platform In 2017 we had e-learning course ready [7] to introduce the SPAR decision-making module and commanders who had got red assessment results where obligated to go through the course. Others were asked to do the same, but if they had not done that it did not change their participation in the development and assessment day. In year 2018 everyone where asked to join the e-learning course however not before March 2019 all had done that.

Participant feedback Web-based anonymous survey at the end of year 13 questions in the survey 137 reponses (351 where asked)

Participant feedback to the purposefulness

Feedback to scenario difficulty level

Master´s theses about the project Author: Andre Tammik Title: “Mapping of the learning motivation and the factors related to the student engagement during the training and assessment of incident commanders”

Main findings in Tammik´s thesis The survey (224 responses): positive engagement learning motivation of incident commanders helped to develop leadership competencies their actions where perceived as autonomous

Changes to be made Meaning of assessment results colours: Red next year Yellow in two years Green in three years Development of assessors, assessment scenarios, e-learning platform and training tools Assessment results where expressed in colours: red: failed; yellow: threshold; green: excellent performance. Commanders with red had to be assessed again next year, with yellow in second year and green on third year.

How did you do working with me? Lets see in Socrative…. Assessment results where expressed in colours: red: failed; yellow: threshold; green: excellent performance. Commanders with red had to be assessed again next year, with yellow in second year and green on third year. Any questions?

Päästekolledži kriisireguleerimise õppetool 27.07.2019 TÄNAN! Stella Polikarpus lektor Päästekolledži kriisireguleerimise õppetool Polikarpus, S., Bøhm, M., and Ley, T., “Training Incident Commanders ’ Situational Awareness – A Discussion of How Simulation Software Facilitates Learning” in Digital Turn in Schools—Research, Policy, Practice: Proceedings of ICEM 2018 Conference, 2019, pp. 1–15. https://www.springer.com/gp/book/9789811373602