1. First level incident commanders’ decision-making assessment for learning
Stella Polikarpus

2. What am I talking about?
Teaching means designing the learning environment in a way that it supports students learning.
Learning means to seek new knowledge and understanding, exploring alternative solutions and welcome different perspectives, think evaluatively and check impact of ones transformative actions.

3. Work with me! Perhaps you learn something?
Open your PC or phone and type in internet browser Socrative.

4. Work with me! EASS EASS EAS EA E
Open your PC or phone and type in internet browser Socrative.

5. Learning outcome
Student knows the main components of the process of new assessment methodology implementation in Estonia rescue service.

6. 45 227 km² 1 323 820 residents 1st January 2019
Statistikaameti andmetel elas 1. jaanuaril 2019 Eestis inimest. 1. Estonia has one of the lowest population densities in the world (28 per sq km; 72.5 per sq mi) – placing it 188th in the world.
Estonian highest peak is 318 meters (1,043 feet) above sea level 

7. 72 national rescue brigades in Estonian Rescue Board 24/7
In Estonia there are :
72 national rescue brigades
350 shift workers
in Estonian Rescue Board 24/7

8. Level I brigades (72 stations) Level II Stand-by groups (total 12)
operational/Bronze level
Level II Stand-by groups (total 12)
operational or tactical/Silver level
Level III Regional duty officer in charge
tactical/Silver
Level IV Operational duty officer
of the Estonian Rescue Board
strategic/Gold

9. The ESTONIAN ACADEMY OF SECURITY SCIENCES (EASS)
The ESTONIAN RESCUE BOARD
(ERB)

10. Work with me! Room nr: EASS
Open your PC or phone and type in internet browser Socrative.

11. AIM
to give an overview of the process of new assessment methodology implementation in Estonia rescue service.
In presentation the e-learning course, Effective Command platform and XVR On-Scene software is showed. The summary of assessment results is presented together with suggestions of how to improve the decision-making training.

12. The presentation structure
Before
During
After
3 years´ project of training and assessment of first level incident commanders
Assessment project of first level incident commanders was carried out in cooperation with the Estonian Academy of Security Sciences (EASS) and Estonian Rescue Board (ERB) in years The assessment method is called Effective Command [1]. Assessors where trained and certified by Dr Katherine Lamb from United Kingdom. Assessment criteria where compared with Estonian Rescue unit leader occupational qualification standard [2] to build the development and assessment system connected to occupational qualification framework. Additional training curriculum was developed [3] together with scenarios in software XVR On-Scene [4]. Incident commanders in Estonia where divided in three groups of 120 commanders, as that was maximum EASS could assess in a year.

13. Occupational qualifications
Rescue unit leader, level 5
Rescuer, level 4
Rescue commander, level 6
IC is incident commander

14. Preparations in ERB
Changing the rescue unit leaders´ attestation regulations
Incident command competences being assessed in EASS every 3 years
If failed, then re-attestation in a year
Meanings of the assessment colours
Finances training (3 years)
Selecting staff to be attestated
Assessment project of first level incident commanders was carried out in cooperation with the Estonian Academy of Security Sciences (EASS) and Estonian Rescue Board (ERB) in years

15. Preparations in EASS Mapping competences with standard
Training the assessors (12 persons)
Incident command XVR user competences
Effective Command and SPAR decision making module
The additional training curriculum
Developing the assessment scenarios
Assessors where trained and certified by Dr Katherine Lamb from United Kingdom. Assessment criteria where compared with Estonian Rescue unit leader occupational qualification standard [2] to build the development and assessment system connected to occupational qualification framework. Additional training curriculum was developed [3] together with scenarios in software XVR On-Scene [4]. Incident commanders in Estonia where divided in three groups of 120 commanders, as that was maximum EASS could assess in a year.

16. Assessment method
The assessment method is called Effective Command [1]. Assessors where trained and certified by Dr Katherine Lamb from United Kingdom. Assessment criteria where compared with Estonian Rescue unit leader occupational qualification standard [2] to build the development and assessment system connected to occupational qualification framework. Additional training curriculum was developed [3] together with scenarios in software XVR On-Scene [4]. Incident commanders in Estonia where divided in three groups of 120 commanders, as that was maximum EASS could assess in a year.
Adopted from dr Katherine Lamb presentation in ForeFront meeting April 2019

17. SPAR decision module Adapted from Launder & Perry 2014
Assessment project of first level incident commanders was carried out in cooperation with the Estonian Academy of Security Sciences (EASS) and Estonian Rescue Board (ERB) in years The assessment method is called Effective Command [1]. Assessors where trained and certified by Dr Katherine Lamb from United Kingdom. Assessment criteria where compared with Estonian Rescue unit leader occupational qualification standard [2] to build the development and assessment system connected to occupational qualification framework. Additional training curriculum was developed [3] together with scenarios in software XVR On-Scene [4]. Incident commanders in Estonia where divided in three groups of 120 commanders, as that was maximum EASS could assess in a year.

18. Development and assessment day 6 training classes for IC
Lecture (2 academic hours 1:4)
Exercise (2 academic hours 1:1)
Assessment (2 academic hours 2:1)
The development and assessment day consisted 6 training classes [3]. In 2016 we started with lecture about decision-making module SPAR (Situational Awareness, Plan, Action, Review) [5], followed by one to one training session. In training session, the joystick exercise was carried out first to explain the trainee, how is it possible to collect information in 3D environment. Secondly, the exercise was carried out where traffic accident had occurred in two-level intersection [6]. After that conversation between trainer and trainee was carried out based on SPAR module and Effective Command assessment criteria.

19. Development and assessment day
After lunch the trainers did the assessments in pairs. In dynamic phase of the assessment one assessor had the roll of technical instructor manipulating with XVR. Other made voices to avatars and made sure that important changes in situations are played out to test that the commander reviews the situation. The dynamic phase always ended with second level commander arriving on scene played by one of the assessors. Trainee had to give overview of the situation to the second level commander. Third, the conversation between trainee and assessors was carried out and the certificated filling started. The aim of the conversation is to find out what incident commander was thinking when he commanded and communicated with avatars. Certificate was filled after two assessors had joint understanding of each assessment criteria. Trainee was asked back to room and certificate was introduced and signed digitally by all three.

20. During the project 2016 30 days 0 day 2017 2 days 2018 35 days 14 days
YEARS
1st time assessment
2nd time assessment
2016
30 days
0 day
2017
2 days
2018
35 days
14 days

21. HITSA Moodle e-learning platform
In 2017 we had e-learning course ready [7] to introduce the SPAR decision-making module and commanders who had got red assessment results where obligated to go through the course. Others were asked to do the same, but if they had not done that it did not change their participation in the development and assessment day. In year 2018 everyone where asked to join the e-learning course however not before March 2019 all had done that.

23. Participant feedback Web-based anonymous survey at the end of year
13 questions in the survey
137 reponses (351 where asked)

24. Participant feedback to the purposefulness

25. Feedback to scenario difficulty level

26. Master´s theses about the project
Author: Andre Tammik
Title: “Mapping of the learning motivation and the factors related to the student engagement during the training and assessment of incident commanders”

27. Main findings in Tammik´s thesis
The survey (224 responses):
positive engagement
learning motivation of incident commanders
helped to develop leadership competencies
their actions where perceived as autonomous

28. Changes to be made Meaning of assessment results colours:
Red next year
Yellow in two years
Green in three years
Development of assessors, assessment scenarios, e-learning platform and training tools
Assessment results where expressed in colours: red: failed; yellow: threshold; green: excellent performance. Commanders with red had to be assessed again next year, with yellow in second year and green on third year.

29. How did you do working with me?
Lets see in Socrative….
Assessment results where expressed in colours: red: failed; yellow: threshold; green: excellent performance. Commanders with red had to be assessed again next year, with yellow in second year and green on third year.
Any questions?

30. Päästekolledži kriisireguleerimise õppetool
TÄNAN!
Stella Polikarpus
lektor
Päästekolledži kriisireguleerimise õppetool
Polikarpus, S., Bøhm, M., and Ley, T., “Training Incident Commanders ’
Situational Awareness – A Discussion of How Simulation Software Facilitates Learning”
in Digital Turn in Schools—Research, Policy, Practice: Proceedings of ICEM 2018
Conference, 2019, pp. 1–15.