The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Visual Surveillance Technologies for Enhancing ABC Secure Zones Project: FastPass Csaba Beleznai, Stephan Veigl, Michael Rauter David Schreiber, Andreas Kriechbaum Video- and Security Technology Safety & Security Department AIT Austrian Institute of Technology GmbH

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Motivation  Significantly increasing passenger flows from 2012 to million  Border guards face big challenges in-depth document checks reliable identity checks check of entry conditions discover possible threats 2

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Motivation Issues to be addressed  Exactly one person per passport  Single person detection  Clean secure zone  Left object detection  Situation overview  Queue length estimation Video Surveillance in Gate  Reliable detections with 3D stereo video camera 3 Demo eGate: Vienna Airport (Terminal 2, Non-Schengen-Arrivals)

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. ABC Gates 4

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. The Sensor 3D stereo-camera system developed by AIT  Area-based, local-optimizing, correlation- based stereo matching algorithm  Specialized variant of the Census Transform  Video image Spatial Resolution: 752x480  rectified to 608x328 RGB  Depth information 16 bit depth image  USB 2 interface 5

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Camera Set-Up (inside the gate)  Top-view of eGate interior  ~15 frames per second  Mounting height: 279 cm  depth resolution: ~2 cm  Ground floor 180x60 cm  spatial resolution: ~1 cm 6

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Single Person Detection Motivation / Challenges  Ensure only one person is inside the eGate reliable detection and counting of persons multiple persons must not pass!  Real-time processing low latency  Better than existing system error-prone to tailgating / piggybacking number of false alarms  Developed for secure zones 7

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Single Person Detection Method 1.Head candidates hypothesis generation  test positions 2.Classification with head-shoulder model support-vector-machine filter false hypothesis 3.Tracking of detections 8  use door signals to increase robustness

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Single Person Detection Examples - single person 9 video 1 - single person.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Single Person Detection Examples - multiple persons 10 video 2 - multiple persons.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Single Person Detection Examples - closed doors 11 video 3 - closed doors.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Left Object Detection Motivation / Challenges  Detect left objects in eGate eGate has to be empty after person left visualize left object  Real-time processing low latency  Difficult object size and / or apperance small size (e.g. passport) low contrast (e.g. empty bottle) 12

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Left Object Detection Method  3D Images color image depth information data fusion  Motion suppression 13

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Left Object Detection Examples - Trolley 14 video 4 - trolley.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Left Object Detection Examples - Mobile Phone 15 video 5 – mobile phone.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Left Object Detection Examples - Trolley 16 video 6 – bottle.avi

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Queue Length Estimation Motivation / Challenges  Enhanced queue management number of persons per queue  Visual tracking of queue dynamics estimated waiting time  Overcome occlusion problems eliminate top-view requirement (for low ceiling height environments) use 3D information 17  4 Min.

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Queue Length Estimation Outlook (Method)  Stereo camera (e.g. mounted on eGate)  Detection distance up to 12 meters 18

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Conclusion Stereo Video Surviallance  New dimension of information  Robust and reliable detection under variable situations  Decrease false alarms for single person detection left object detection  New options for queue length estimation 19

The work has been supported by the FastPass project. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/ ) under grant agreement n° This publication only reflects the author’s view and the European Union is not liable for any use that may be made of the information contained therein. All document contained therein cannot be copied, reproduced or modified in the whole or in the part for any purpose without written permission from the FastPass Coordinator with acceptance of the Project Consortium. Stephan Veigl Video- and Security Technology Safety & Security Department AIT Austrian Institute of Technology GmbH 20 AIT Austrian Institute of Technology your ingenious partner