1. WP3 - INERTIA Local Control and Automation Hub
T3.5- Ambient and Personalized UI & User Profiling Mechanism
Hypertech S.A.

2. Task 3.5 Overview
T3.5- Ambient and Personalized UI & User Profiling Mechanism
Duration: 19 Months (M6-M24)
D3.3 Ambient User Interfaces, User Behavioural Profiling and Activity Flow Framework
Report + Prototype, Public.
Lead Beneficiary: HYPERTECH (11), Involved Beneficiaries: CERTH-ITI (2), ENG (4), CNET (2), INDESIT (6), ALMENDE (4), ETC EL(3)
Highlight (during the presentation) the role and contribution of each partner
HYPERTECH: User profiling models
CERTH-ITI: Occupancy flow and prediction module (linking with the work of task T3.4 and Task 2.3…Specs).
CNET: Interfacing with the Semantic Middleware mostly focusing on state and event data extraction
ALMENDE: Incorporation of User profiling engine in the MAS system (this will be the ideal approach). In addition to cooperate for the optimal algorithmic approach on Local Hub … set the boundaries and interfaces between user profiling and the DER flexibility models
ENG to ensure the compliance and optimal adoption of U&B requirements from End Users (what else ??)
INDESIT : Ambient UI development
ENELSI: Evaluation of the approaches

3. T3.5 General Purpose Main Expected outcomes:
To design and develop User Profiling and a Personalized User Interface
(Facility Manager + Occupants) that support:
The review & development of techniques for the extraction of robust, reliable prosumer-based profiles, which fully exploit the “context” that affects Users behaviour
The development of an Ambient and Personalized Environment will offer the opportunity to users to monitor and control (facility manager) their workplace through interfaces applying to user monitors or other smart devices
Main Expected outcomes:
Ambient & Personalized User Interface Module
User profiling Models
Incorporation of User Profiling Models& Ambient UI into the Holistic Framework
The UI comes first
The second bullet aims at capturing all contextual parameters affecting User Control behavior and preferences
The third bullet aims at establishing a continuously adaptive framework that never ceases to analyze the user behavior (we consider the problem highly dynamic … or non-stationary in terms of learning techniques)
(Main partner involved: HYPERTECH)
(Main partner involved: INDESIT)

4. T3.5 - User Profiling Scope
High- level overview of User profiling Approach (Review)
- Introduce also the notion of user agents (as well as agents managing different contextual parameters).
- Provide the first clue that the user profiling should allow for the quantification of the user preferences (and the way it affects DER elasticity) in order to allow later on the negotiation and optimization process to take place.
Comments: Point out the role of the User profiling in this schema. Define the barrier for external signals and pose the need to examine User profiling as affected only by the occupants and their operations.
Overview
Non intrusive mechanism for the extraction of User Profiling mechanism
User profiling estimation in Environmental Conditions Related Devices: Thermal / Visual User Preferences for HVAC / Lighting Devices
User profiling for Operational Devices : Operational profiles for specific devices optimal forecasting operations

5. Environmental Conditions Related User Profiling
User Profiling Analysis Objectives - Steps
to continuously monitor occupancy related data in order to align the states of the environmental conditions with the additional occupancy patterns examined in the pilot premises
to continuously monitor environmental conditions that reveal the comfort and discomfort preferences of the occupant’s. The environmental conditions will be available through the respective sensors (Temperature, Humidity, Luminance), to be installed within the INERTIA pilot area.
to continuously monitor user control actions - reactions in specific environmental conditions, in order to seamlessly provide an effective and efficient learning scheme for the extraction of User Preferences
INERTIA user profiling mechanism will minimize or eventually eliminate required occupant
interaction and thus overall occupant disturbance
-----
Highlight main challenges
- elasticity is more a function of occupancy and user preferences than price
- the complexity problem due to dimensionality as well as dynamic nature of the contextual parameters (occupants some times change their behavior, also in certain spaces (and respective DER) new occupants/owners are introduced each day that must be captured by the system, new DER are introduced, new environmental conditions could be encountered that have not been part of the training data/period).
Therefore we need :
- a multidimensional contextual approach
- a continuously adaptive approach
- an approach that should quantify the relation between DER real time elasticity and user behavior
------

6. T3.5 User Profiling Main Aspects (I)
User Profiling Dependencies
Learning Approach
Ambient User Interface. External settings of User Preferences (Temperature &Luminance Conditions as initial settings)
INERTIA Middleware. Delivering the real time contextual information (e.g. luminance, temperature, humidity, control actions on switches, dimmers and HVAC control panels)
Real -time Occupancy flow. Delivering real - time occupancy details (Occupants with ID, number of occupants).
Event Reader: to handle information related to real time events
INERTIA Information Access Module. Management of the historical related information (environmental events and control actions). Information to be provided in a personalized way.
Markov Model Learning Mechanism (Bayesian Approach) for the extraction of User Preferences under specific environmental conditions
User profiling dependencies. The relationship with external components of INERTIA Architecture.
Occupancy events are needed in order to align the control actions to the specific Users in the area examined
For groups of occupants as this information will be further examined in order to extract patterns that may lead on the operational dependence to number of users.

7. T3.5 - User Profiling Approach
A high level overview of User Profiling algorithmic approach – Learning approach
Normalized data from raw data events are delivered in the State Estimator Normalization of data in 60 lux level for Luminance events & 0.5 oC for Temperature events.
Occupancy Detector that feeds the State Estimator with the status of occupancy related to the specific area examined.
State Estimator is a knowledge based components that defines the rules for the learning mechanism (Aggregation of events, decision about the learning period of the model, weighted values on control action events)
Learning Model Defines the Utility Function Parameters and User related statistics that further be stored in a database (in INERTIA: Local Hub IAM) for further evaluation
Business Services Component : Provides an API with information related to the learning model about User Preferences. The context information is useful for the control and optimization mechanism of the INERTIA Local Hub .
----
The Evaluation – learning Cycle should be highlighted and provide hints for the re-inforcement learning technique.

8. T3.5 User Profiling Main Aspects (II)
User Profiling Dependencies
Utility Estimation Approach
INERTIA Holistic Flexibility Component.
Input : Occupancy Status, Environmental Conditions (Luminance, Humidity, Temperature),[time period in the current condition]
Output: Thermal or Visual User Comfort/ Discomfort level in a scale of [0…1]
INERTIA Information Access Module.
Input: Historical events (environmental conditions, occupancy details, Control actions delivered by the users) in order to provide a continuous learning operation
Output: Learning Model Parameters as defined by the learning mechanism
Providing the necessary input for the extraction of the Holistic Flexibility Models and therefore the optimal controlling of the DER units
Comments:
To examine different approaches
User profiling implemented by ALMENDE (HYPERTECH to provide the algorithmic approach)
User profiling as an engine – out of the MAS
User profiling implemented by HYPERTECH (Not possible)
User profiling dependencies. The relationship with external components of INERTIA Architecture.

9. User Profiling Architecture
A High Level view of the User Profiling Architecture:
State Estimator: Evaluates the current state (learning or estimation mechanism - Parameters to be examined)
Learning Controller: For the Utility Function learning process
Utility Estimator: For the estimation of the Utility function when asked

10. T3.5 - User Profiling Cost function
Main issues on the proposed framework
The utility function is a continuous function for a defined set of multi-criteria analysis (occupancy, context, context environment, user control actions)
Common format value [ 0…1] to be provided for different types of DERs, occupancy, environmental conditions)
Utility Function as a generalization function approximating the training set
A diagrammatic representation of an indicative utility function is provided (discomfort function-luminance)
We have to quantify the utility function for each occupant on the respective Environmental Conditions
It more or less reflect the underlying model of our universe (building space and occupants behavior).
Να τονισουμε την διασυνδεση των preferences με τις κλιματολογικες συνθηκες.

11. T3.5 - User Profiling & DER Modeling
Different types of DERs in INERTIA Project:
Environmental Conditions related Devices (HVAC, Lighting). The estimation of the Utility function (User Preferences) is based on the environmental conditions and not the operational status of the devices. Thus, an (Environmental conditions  Comfort Performance) model is defined
Operational Devices: Shiftable devices. The role of the User Profiling mechanism is to provide the operational patterns for the specific devices (based on time & occupancy based characteristics). This information could be handled by the Local Hub Control mechanism for the optimal management of the operational devices.
Operational Devices (Monitor PC, plugged office equipment). The estimation of the Utility function is based on the absence - presence of the occupants. Thus the INERTIA User profiling mechanism will not provide a utility estimation for these DERs {Occupancy On Status On, Occupancy Off Status off}.
Special interest for the Context related Devices, as they are going to provide a significant
amount of flexibility needed during a DR event.
INERTIA DER Flexibility Model P(occupancy)*P(DER Control Action| Context, Occupancy)*DER_Model(Context, Control Action)
Also connect the slide with the reference in the DOW about specific control strategies i.e.
- Limiting (mild but predictive)
- shedding (more intrusive and sudden and a bit predictive)
- Shifting (highly intrusive, highly predictive)
Comments: We have to define the role of each device as examined in pilot area.
INERTIA User profiling = Occupancy * Context * DER Operation (Who * What* How)
Eg. Non Controllable devices ( Cafeteria machine, PV, PC untits)
Curtailable devices (HVAC. Lighting with ballast etc)
Shiftable devices (Washing Machine, PHEV)

12. T3.5 - User Profiling Approach Overview
What we need - Summary
Utility function estimation for each occupant taking into account the real time context conditions  Occupancy data
Utility function totally dependent on the context (environmental impact, operational situations etc.) Context data (environmental conditions, control actions on DERs)
Different approaches to be examined in INERTIA Project:
Single occupant with ID. The estimation of the preferences is delivered in a personalized way. Initial settings of User preferences are provide through the Ambient UI.
Multiple Occupants in a specific zone  Group Profiling Main Aspects. To further examine the impact of different occupancy patterns and define additional group profiles if needed.
Stereotypes. For specific areas (Cafeteria-kitchenette). Concept of Virtual Occupant. A typical user’s patterns are examined. The ID of the occupant is not known but the patterns of usage are common.
Flexibility estimation for each type of occupancy provided to Holistic Flexibility Model

13. T3.5 – Ambient UI - Overview
Ambient UI Objectives:
Thus, the development of the Ambient UI concept includes the implementation of different
types of interfaces:
for the occupants: different ways to visualize high-level information about the energy consumption of area, in order to increase occupants' awareness. Only monitoring functionalities are provided as the direct enrollment of the User in Demand Response Actions are out of the scope of the INERTIA project
for the manager of the building: visualization of real-time (aggregated and correlated) information on the energy consumption of the building, in order to have real-time information and apply suitable energy strategies. (Monitoring and control functionalities)
This slide defines the main objectives and the role of Ambient Interface.
You are pleased to refine or add more detail on the existing information.
You should further check the DoW (p.25 and p ) mentioning the role of INERTIA Ambient UI.

14. T3.5 – Ambient UI - Specifications
Based on the Demo Presentation- Review
Individual Occupants: 
Login: This page allows the user to authenticate him in the system inserting its own credential (login and password).
Homepage: A box containing a summary of the most important data (consumption in real time-environmental conditions). A box including all the main aspects examined by a User (Menu list)
User Preferences: settings of environmental parameters (luminance, temperature). Additional settings on specific DERs ( Electric Vehicles, Water Heater?)
User Preferences: Statistics as delivered from the User Profiling Component: Preferred environmental conditions, Number of Control Actions on specific states etc…
Consumption Monitoring: Personalized information about energy consumption (per Device & time period examined)
Historical Consumption Monitoring: This sub-section includes a time-consumption graph, divided in time slot. (To be defined: Either predefined time slots or selection menu for the occupant)
New Schedule: The user will be able to select the presence in a specific area /absence as examined by CERTH. In addition the User will be able to set an activity (activity, time start, time end)
Schedule Manager: Calendar form with all the events marked by the user
Notifications: Notifications section includes the list of all notifications generated during the testing period. Each notification include the time at which it has been generated and a description of the event.

15. T3.5 – Ambient UI – Specifications (II)
Groups of Occupants: 
Main functionalities as Individual Occupants
Differentiations:
Tablet view
Not personalized information but a spatial –oriented analysis
Schedule Manager: Settings only about the specific area presented
Not external preferences defined by the users Implicit profiling only
The same view also available in a web based environment (remote view or view from PCs)

16. T3.5 – Ambient UI – Specifications (III)
Facility Manager: 
Homepage 
A box containing a summary of the most important data (Total consumption, Total consumption of the day, Environmental conditions, System Information operation)
A graphical overview of the area with the spaces and the zones as defined for the INERTIA pilot area (CERTH Premises)
A box including all the main parameters to be examined.
Consumption Monitoring: Real time information for the pilot premises
Extrapolation of this information in DER types & Spaces
Historical Consumption Monitoring: Historical Energy consumption representation
Business Domain Monitoring : Real time and Historical Cost of Energy (Based on retailer prices)
Occupancy data: Real time and historical information about the occupancy levels. Aggregate data of occupancy per space
User Preferences: Based on the occupancy levels, extraction of the User Preferences Spatial analysis for groups of occupants

17. T3.5 – Ambient UI – Specifications (IV)
Facility Manager (II): 
System Statistics : A list of KPIs related to the facility manager operations has to be defined.
Schedule Manager : Facility Manager to handle the scheduling procedure within each zone/sub-area
Notifications : Different types of Notifications – Logging Mechanism (Abnormal situations /Events)
Operational mode Settings : The FM to handle the operational mode of the entire infrastructure. (Green operation, Cost minimization , Sustainable operation, Customized, On vacation).
Management rule setting : FM settings about the operational modes of the INERTIA
DR activation mode will be also handled: 
ON: The DR event bypasses the operational mode of the building
OFF: The DR event is not taken into account.
Device Control Operation : Apart from the above mentioned monitoring functionalities on different types of DERs, the facility manager will be able to remotely control the devices.

18. T3.5 – Ambient UI - Architecture
Architectural Approach View
ENG Suggestion: MVP Modeling approach
Model: Representation of the additional models ( Occupants with ID, Groups of Occupants, Facility Manager) through the respective agents
Presenter: Web Presenter + Mobile Presenter Modules
View: Additional Views available for the End Occupants & Facility Manager
Overall approach provided on the document of Architecture D1.4
Development – Deployment View
Occupants with ID: Smartphone applications (Android + iOS Applications)
Groups of occupants: Tablets in 2-3 zones. (to define the installation zones)
Groups of occupants: Information also available through a web interface
Facility Manager: A single Web Interface for remote access control and management.

19. T3.5 Dependencies – Required Contributions (M17-M24)
Contribution from other WPs (WP1, WP2) and involved partners
WP1: Requirements for the Ambient User Interfaces& User Profiling Mechanism
WP1: Pilot survey (Installation of the tablets User Interfaces)
WP1: Overall architecture approach for the placement of the components within the Architecture
WP2 – Task 2.1 – Task 2.4 Fully Completed KPI values for the Ambient UIs
WP3 – Task 3.1 – Task 3.2 Fully Completed || Task 3.3 & 3.4 in Parallel Progress
Prerequisites for providing feedback to T3.5 work
All involved partners will contribute in relation to the following :
HYPERTECH : Task Leader, User Profiling Algorithms and Semantic Models (Task 2.3)- Ambient User Interfaces specs
ALMENDE : MAS Components incorporating User Profiling Models & Ambient User Interface Modules
ALMENDE & CNET : Web based User Interface (Facility Manager)
ENG& CERTH : Mobile Devices User Interfaces (???)
INDESIT: Interfacing with Smart Appliances - Other??

20. T3.5 Work Plan for the next period
Work in Parallel
User Profiling mechanism:
First prototype version covering some aspects of the User Profiling mechanism (M16). (Based on the HYPERTECH Pilot area)
Continuous fine-tuning of the data model-algorithmic approach in order to extract a fully fledged approach (M20) Incorporation of time parameter in the model (TTCA parameter)
Integration of the User Profiling Mechanism to the Holistic Flexibility Framework (M24)
We need data in order to examine our approach
Ambient User Interface mechanism (Based on Initial Plan) :
1st Version of Smartphone & Tablet Apps. (Android: M16- iOS: M20) (ENG - CERTH)
Final Version of Smartphone & Tablet Apps. (Android: M24) (ENG - CERTH)
Web Interface for the Facility Manager – Monitoring Functionality. (M18) (CNET, ALMENDE)
Web Interface for the Groups of Occupants. (M20) (CNET-ALMENDE)
Web Interface for the Facility Manager – Control Functionality. (M20)- Final Version (CNET, ALMENDE)
Demo presentation of partially results on Android Apps & Web UI for FM (Monitoring). (Next Plenary meeting) (ENG - CNET)
Integration of the Ambient UI Mechanism to the Holistic Flexibility Framework. (M24) (ALMENDE, ALL)
Preliminary results (next meeting)

21. Discussion
Questions??
Thank You !!!