1. WP3 INERTIA Local Control and Automation Hub
Task 3.4 Multi-Sensorial Activity Flow Detection & Modelling in controlled environments
CERTH-ITI

2. T3.4 Overview
Task 3.4 “Multi-Sensorial Activity Flow Detection and Modeling in controlled environments
We are here
Deliverable
contributes to:
D3.3 Ambient User Interfaces, User Behavioural Profiling and Activity Flow Framework
(Prototype, Public, M24 September 2014)
Milestone
MS6 Final Version of the Occupancy Flow
Modeling and Prediction available
(M24 September 2014)
Involved Partners:
CERTH/ITI 11
ALMENDE 2

3. T3.4 Main achievements so far (1/2)
Thorough analysis of available occupancy detection systems identifying the requirements, advantages and disadvantages of each one.
Definition of various business case scenarios for multi-sensorial installations in a wide range of different types of tertiary buildings, commercial building rooms and zones.
Detailed specification of the proposed combination of sensors needed for various types of tertiary buildings along with their optimal placement based on use-cases and system requirements.

4. T3.4 Main achievements so far (2/2)
Planning of sensor installations at pilot premises of CERTH.
Investigation of algorithmic methods for extracting human motion and occupancy flows in buildings combining multi-sensor network data.
Prototype development extending functionalities and exploring more algorithmic approaches (e.g. HSMM: Hidden semi-Markov Model), in order to cover all different aspects identified in Task T2.3.

5. Analysis of available occupancy detection systems (1/7)
Environmental Sensors
CO2 Sensor
Pros
Cons
Estimates occ. density
High cost
Easy to install
Slow response to changes (~10-15min delay)
Easily influenced by environmental factors (e.g. open windows)
Acoustic Sensor
Pros
Cons
Easy to install
Affected by environmental noise
Provides indication for occ. density
Used only in areas where occupants produce sound (e.g. meeting room, kitchen)

6. Analysis of available occupancy detection systems (2/7)
Motion/Presence Sensors: Used to detect if there is presence/motion within an area
Beam Sensor
Pros
Cons
Used as counter in entrances/exits
More than one needed to extract direction
Easy installation
Possibility of false alarms and of loosing occupants entering or exiting together
Low cost
PIR Motion Sensor
Pros
Cons
Simple installation (possible use of the existing alarm system)
Detects only presence/absence and not exact number of occupants
Low cost
Fails to detect occupants who remain relatively still
Possible false positives (e.g. wafts of warm or cold air)

7. Analysis of available occupancy detection systems (3/7)
Pressure Sensors: Based on the pressure that is forced by a person’s weight
Pressure Mat
Pros
Cons
Used as counter in entrances/exits
Expensive for a wide installation
Easy installation
Satisfactory accuracy
Chair Pressure Sensor
Pros
Cons
Low cost
Needs suitable placement (e.g. into pillow) so that it is not visible or obtrusive
Easy installation (Mounted on chair)

8. Analysis of available occupancy detection systems (4/7)
Human Capacitance Sensors
Pros
Cons
They use electric fields to detect human movement
Difficult installation
Unobtrusive
Possible false signals due to electromagnetic noise
Context Sources (e.g. keyboard/mouse software detector, instant messaging clients, user calendar etc)
Pros
Cons
Low cost
Produce errors when the source is not used but the occupant is present or the opposite
No real sensors, context information is used instead

9. Analysis of available occupancy detection systems (5/7)
Vision-based Systems
Depth-Image Camera
Pros
Cons
High accuracy
Privacy and ethical issues
Low cost
Requires time consuming setup and calibration
Sensitive to changes in image background
Thermal Sensor
Pros
Cons
Used for counting near entrances/exits (can count multiple people entering or leaving at the same time)
High cost
Quick setup and configuration
Sensible to small and high environment temperature
No privacy issues

10. Analysis of available occupancy detection systems (6/7)
Wi-Fi System
Pros
Cons
Occupancy tracking and identification capability with Wi-Fi tags
High cost for market solutions
Tracking capability for Wi-Fi enabled devices
Possible privacy and ethical concerns
High accuracy

11. Analysis of available occupancy detection systems (7/7)
RFID System
Readers/antennas can be placed near entrances/exits (extract the number and identity of the occupants inside a space but not necessarily their specific coordinates)  Passive technology
Readers/antennas can be located strategically in various places to cover the whole area for indoor localization purposes  Active technology
Pros
Cons
Individual tracking and identification
Privacy and ethical issues
Widely used by the construction industry
Occupants may not carry their tags always with them
Cost efficient
Affected by environmental factors

12. Analysis of available occupancy detection systems - Overview
Environmental Sensors
Motion/ Presence Sensors
Pressure Sensors
Human Capacitance Sensors
Vision-based Systems
Wi-Fi Systems
RFID Systems
Context Sources
Spatio-temporal properties
Presence v
Count (density) x
Count (number)
Location
Track
Identity
Scenario
Individual
Group-based
Need to carry tag
Privacy/Ethical Issues

13. Business case scenarios for multi-sensorial installations
Various aspects to be taken into account per each occupancy extraction sensor cloud:
Installation cost (procurement and setup)
Privacy issues
Required accuracy
Use of existing equipment/infrastructure (alarm system, BMS, wirings, walls etc)
Obtrusiveness of sensor installations
Other special requirements

14. Multi-sensorial installations: Technical criteria
Technical criteria to be taken into account for the selection of sensor cloud:
Quick response in order to capture real-time occupancy state (e.g. CO2 needs 10-15mins to respond)
Desired spatio-temporal properties to be measured: presence/absence, occupancy density/exact number of occupants, location, track, identity
Need for carry-on device (e.g. RFID tag)
Need for self-validation of occ. Extraction, by using more than one inputs (e.g. door counter + motion detector)
Other special requirements

15. Multi-sensorial scenarios - Proposed solutions
Performance-driven scenario
high accuracy, possible zoning, no privacy concerns
PIR Motion detectors maybe already available through the alarm system
Depth-Image Cameras +
PIR Motion detectors

16. Multi-sensorial scenarios - Proposed solutions
Resource-constrained scenario
low cost, simple
least obtrusiveness, minimal intervention
PIR motion detectors maybe already available through the alarm system
PIR Motion detectors

17. Multi-sensorial scenarios - Proposed solutions
Combine satisfactory accuracy with least possible obtrusiveness
Count the number of people that enter or exit a space
PIR Motion detectors maybe already available through the alarm system
Beam Sensors OR
Pressure Mats +
PIR Motion detectors

18. Multi-sensorial scenarios - Proposed solutions
Individual Extraction
Readers/antennas installed only near entrances/exits OR
Readers/antennas and Reference tags located strategically to cover the full area for indoor localization
RFID System

19. Planning of sensor installations at CERTH
Detection System
Applicable (Implemented in pilots)
Applicable (Not implemented in pilots)
Not applicable for INERTIA
CO2 Sensor v
Acoustic Sensor
Beam Sensor
PIR Motion Sensor
Ultrasonic/Microwave Sensor
Pressure Sensors
Human Capacitance Sensor
Keyboard/Mouse detector
Depth-Image Camera
Thermal Sensor
Wi-Fi System
RFID System (Active)
RFID System (Passive)

20. Planning of sensor installations at CERTH
Will use different sensor combinations in parallel
to evaluate extraction accuracy of each combination against the others under the same time period and conditions
Sensors will be used in turns in different sites for different periods of time
Ground truth data will be collected to explore the efficiency of the various combinations
by using Kinect Cameras or observation

21. Occupancy Extraction: Current Status
Extraction using Kinect Cameras: Already implemented and verified – will only be used for ground-truth occupancy extraction.
Started development of new prototype combining the use of multiple sensors.
Exploring and developing algorithms for occupancy extraction combining data flows from various sensors.
Will define most suitable combinations of sensors to produce results.
collect occupancy data from different sensors (by mid March)

22. Occupancy Prediction: Current Status
Continued prototype development
Extending functionalities and exploring more algorithmic approaches to be implemented
HSMM (Hidden semi-Markov Model): seems not applicable as there are no hidden states
SMM (Semi-Markov Model): seems more relevant to our case
Association Rule Mining: seems promising – needs more investigation

23. T3.4 Next efforts
Develop occupancy extraction algorithms for the various combinations of sensors.
Continue prototype development, extending functionalities and implementing more algorithmic approaches for occupancy prediction.
Acquire more occupancy data from pilot installations at CERTH or other projects (e.g. Adapt4EE), towards better assessment of the different approaches.
Take into account scheduling information.