WP3 INERTIA Local Control and Automation Hub Task 3.4 Multi-Sensorial Activity Flow Detection & Modelling in controlled environments CERTH-ITI

T3.4 Overview Task 3.4 “Multi-Sensorial Activity Flow Detection and Modeling in controlled environments We are here Deliverable contributes to: D3.3 Ambient User Interfaces, User Behavioural Profiling and Activity Flow Framework (Prototype, Public, M24 September 2014) Milestone MS6 Final Version of the Occupancy Flow Modeling and Prediction available (M24 September 2014) Involved Partners: CERTH/ITI 11 ALMENDE 2

T3.4 Main achievements so far (1/2) Thorough analysis of available occupancy detection systems identifying the requirements, advantages and disadvantages of each one. Definition of various business case scenarios for multi-sensorial installations in a wide range of different types of tertiary buildings, commercial building rooms and zones. Detailed specification of the proposed combination of sensors needed for various types of tertiary buildings along with their optimal placement based on use-cases and system requirements.

T3.4 Main achievements so far (2/2) Planning of sensor installations at pilot premises of CERTH. Investigation of algorithmic methods for extracting human motion and occupancy flows in buildings combining multi-sensor network data. Prototype development extending functionalities and exploring more algorithmic approaches (e.g. HSMM: Hidden semi-Markov Model), in order to cover all different aspects identified in Task T2.3.

Analysis of available occupancy detection systems (1/7) Environmental Sensors CO2 Sensor Pros Cons Estimates occ. density High cost Easy to install Slow response to changes (~10-15min delay) Easily influenced by environmental factors (e.g. open windows) Acoustic Sensor Pros Cons Easy to install Affected by environmental noise Provides indication for occ. density Used only in areas where occupants produce sound (e.g. meeting room, kitchen)

Analysis of available occupancy detection systems (2/7) Motion/Presence Sensors: Used to detect if there is presence/motion within an area Beam Sensor Pros Cons Used as counter in entrances/exits More than one needed to extract direction Easy installation Possibility of false alarms and of loosing occupants entering or exiting together Low cost PIR Motion Sensor Pros Cons Simple installation (possible use of the existing alarm system) Detects only presence/absence and not exact number of occupants Low cost Fails to detect occupants who remain relatively still Possible false positives (e.g. wafts of warm or cold air)

Analysis of available occupancy detection systems (3/7) Pressure Sensors: Based on the pressure that is forced by a person’s weight Pressure Mat Pros Cons Used as counter in entrances/exits Expensive for a wide installation Easy installation Satisfactory accuracy Chair Pressure Sensor Pros Cons Low cost Needs suitable placement (e.g. into pillow) so that it is not visible or obtrusive Easy installation (Mounted on chair)

Analysis of available occupancy detection systems (4/7) Human Capacitance Sensors Pros Cons They use electric fields to detect human movement Difficult installation Unobtrusive Possible false signals due to electromagnetic noise Context Sources (e.g. keyboard/mouse software detector, instant messaging clients, user calendar etc) Pros Cons Low cost Produce errors when the source is not used but the occupant is present or the opposite No real sensors, context information is used instead

Analysis of available occupancy detection systems (5/7) Vision-based Systems Depth-Image Camera Pros Cons High accuracy Privacy and ethical issues Low cost Requires time consuming setup and calibration Sensitive to changes in image background Thermal Sensor Pros Cons Used for counting near entrances/exits (can count multiple people entering or leaving at the same time) High cost Quick setup and configuration Sensible to small and high environment temperature No privacy issues

Analysis of available occupancy detection systems (6/7) Wi-Fi System Pros Cons Occupancy tracking and identification capability with Wi-Fi tags High cost for market solutions Tracking capability for Wi-Fi enabled devices Possible privacy and ethical concerns High accuracy

Analysis of available occupancy detection systems (7/7) RFID System Readers/antennas can be placed near entrances/exits (extract the number and identity of the occupants inside a space but not necessarily their specific coordinates)  Passive technology Readers/antennas can be located strategically in various places to cover the whole area for indoor localization purposes  Active technology Pros Cons Individual tracking and identification Privacy and ethical issues Widely used by the construction industry Occupants may not carry their tags always with them Cost efficient Affected by environmental factors

Analysis of available occupancy detection systems - Overview Environmental Sensors Motion/ Presence Sensors Pressure Sensors Human Capacitance Sensors Vision-based Systems Wi-Fi Systems RFID Systems Context Sources Spatio-temporal properties Presence v Count (density) x Count (number) Location Track Identity Scenario Individual Group-based Need to carry tag Privacy/Ethical Issues

Business case scenarios for multi-sensorial installations Various aspects to be taken into account per each occupancy extraction sensor cloud: Installation cost (procurement and setup) Privacy issues Required accuracy Use of existing equipment/infrastructure (alarm system, BMS, wirings, walls etc) Obtrusiveness of sensor installations Other special requirements

Multi-sensorial installations: Technical criteria Technical criteria to be taken into account for the selection of sensor cloud: Quick response in order to capture real-time occupancy state (e.g. CO2 needs 10-15mins to respond) Desired spatio-temporal properties to be measured: presence/absence, occupancy density/exact number of occupants, location, track, identity Need for carry-on device (e.g. RFID tag) Need for self-validation of occ. Extraction, by using more than one inputs (e.g. door counter + motion detector) Other special requirements

Multi-sensorial scenarios - Proposed solutions Performance-driven scenario high accuracy, possible zoning, no privacy concerns PIR Motion detectors maybe already available through the alarm system Depth-Image Cameras + PIR Motion detectors

Multi-sensorial scenarios - Proposed solutions Resource-constrained scenario low cost, simple least obtrusiveness, minimal intervention PIR motion detectors maybe already available through the alarm system PIR Motion detectors

Multi-sensorial scenarios - Proposed solutions Combine satisfactory accuracy with least possible obtrusiveness Count the number of people that enter or exit a space PIR Motion detectors maybe already available through the alarm system Beam Sensors OR Pressure Mats + PIR Motion detectors

Multi-sensorial scenarios - Proposed solutions Individual Extraction Readers/antennas installed only near entrances/exits OR Readers/antennas and Reference tags located strategically to cover the full area for indoor localization RFID System

Planning of sensor installations at CERTH Detection System Applicable (Implemented in pilots) Applicable (Not implemented in pilots) Not applicable for INERTIA CO2 Sensor v Acoustic Sensor Beam Sensor PIR Motion Sensor Ultrasonic/Microwave Sensor Pressure Sensors Human Capacitance Sensor Keyboard/Mouse detector Depth-Image Camera Thermal Sensor Wi-Fi System RFID System (Active) RFID System (Passive)

Planning of sensor installations at CERTH Will use different sensor combinations in parallel to evaluate extraction accuracy of each combination against the others under the same time period and conditions Sensors will be used in turns in different sites for different periods of time Ground truth data will be collected to explore the efficiency of the various combinations by using Kinect Cameras or observation

Occupancy Extraction: Current Status Extraction using Kinect Cameras: Already implemented and verified – will only be used for ground-truth occupancy extraction. Started development of new prototype combining the use of multiple sensors. Exploring and developing algorithms for occupancy extraction combining data flows from various sensors. Will define most suitable combinations of sensors to produce results. collect occupancy data from different sensors (by mid March)

Occupancy Prediction: Current Status Continued prototype development Extending functionalities and exploring more algorithmic approaches to be implemented HSMM (Hidden semi-Markov Model): seems not applicable as there are no hidden states SMM (Semi-Markov Model): seems more relevant to our case Association Rule Mining: seems promising – needs more investigation

T3.4 Next efforts Develop occupancy extraction algorithms for the various combinations of sensors. Continue prototype development, extending functionalities and implementing more algorithmic approaches for occupancy prediction. Acquire more occupancy data from pilot installations at CERTH or other projects (e.g. Adapt4EE), towards better assessment of the different approaches. Take into account scheduling information.