Application of NIST Technical Note 1822 to CA crowd dynamics models verification and validation Jakub Porzycki, Robert Lubaś, Marcin Mycek, Jarosław Wąs The Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering Department of Applied Computer Science ACRI 2014 Conference, CCA Workshop, Kraków 22.09.2014 www.agh.edu.pl 1

The need of standardization Lack of well defined, comprehensive and commonly accepted V&V procedures. Models are tested by authors using arbitrary chosen criteria. The need of standard, impartial tool to check model usefulness for a given task. www.agh.edu.pl 2

Existing guidance for models V&V MSC/Circ.1033 and MSC/Circ.1238 – ships' evacuation, by International Maritime Organization. SAFEGUARD – suggestions of improvements to MSC/Circ.1238 and validation data. RiMEA – adjustment of MSC/Circ.1033 to building evacuation (mostly in German). Work of C. Rogsch et al. - number of comparison test between models and proposition of validation test cases. NIST Technical Note 1822: The Process of Verification and Validation of Building Fire Evacuation Models. E. Ronchi, E. D Kuligowski, P. A Reneke, R. D Peacock, D. Nilsson www.agh.edu.pl 3

NIST Technical Note 1822 Summary of current knowledge. Proposes a set of 17 verification test. Recommends validation data set. Suggests procedure of model uncertainty analysis. The note is focused on fire evacuation. „ ...this document is aimed at opening discussion on the topic rather than being a definitive guidance on the topic.” www.agh.edu.pl 4

2.1 Speed in corridor and 2.2 Speed on stairs Expected result: Pedestrian should maintain designed speed 1m/s in 40m corridor and 100m stairs. Thoughts: Detailed tests for CA models – corridor/grid orientation. Exact values not available in CA models due to space and time discretization. www.agh.edu.pl 5

2.3 Movement around corner Expected result: Pedestrian navigate around corner without penetrating the boundaries. Thoughts: Unnecessary test for CA models - boundary penetration impossible due to CA allowed/forbidden states. This test could also verify space usage. www.agh.edu.pl 6

2.3 Movement around corner www.agh.edu.pl 7

2.8 Horizontal counter-flows Expected result: Passage time should decrease when number of people in counter flow increase Thoughts: This test can also verify lane formation, and expect more detailed passage time changes (based e.g. On Kretz T. et al. “Experimental study of pedestrian counter flow in a corridor”) www.agh.edu.pl 8

2.10 People with movement disabilities Expected result: Group with disabled people will evacuate slower than another. Thoughts: Representation of disabled people is problematic in standard CA models – challenge for future ! Should this test be obligatory for all kind of simulations. www.agh.edu.pl 9

3.1 Exit choice/usage Expected result: Occupants leave the simulation with the allocated exit. Thoughts: Many simulation models calculate exit assignment – why they should fail this test ? Test 4.1 When occupants should change its exit assignment in case when it becomes unavailable during simulation ? www.agh.edu.pl 10

5.1 Congestion, 5.2 Maximum flow rates Expected results: 5.1 Congestion at the room exit and at the base of the stairs. 5.2 Outflow trough 1m wide exit should not exceed 1.3 person/s Thoughts: Test 5.1 is suggested to be executed using visual tool – there is still no objective measure. Why only maximum flow is tested in 5.2? Why not compare to experimental fundamental diagram ? www.agh.edu.pl 11

Summary NIST Technical Note 1822 – very valuable initiative, summary of current knowledge, step to V&V standardization, uncertainty measurement We propose to take into account: More detailed and extended specification of expected test results. Tests for emerging phenomena and fundamental diagram. Specification of data collection methods. Division types of tests into general and specialised for different simulation types. Execution of V&V process by third party – detailed list of functionalities is required. www.agh.edu.pl 12