1. SENSOR-INDEPENDENT PLATFORM FOR CIRCADIAN RHYTHM ANALYSIS Andrea Caroppo Institute for Microelectronics and Microsystems (IMM) National Research Council (CNR) c/o Campus Ecotekne, Via Monteroni, Lecce, Italy Co-authors: Giovanni Diraco, Gabriele Rescio, Alessandro Leone, Pietro Siciliano Institute for Microelectronics and Microsystems, (IMM) National Research Council (CNR) c/o Campus Ecotekne, Via Monteroni, Lecce, Italy

2. INTRODUCTION (1/3) The need for Personal assistance with everyday activities increases with age. Use of smart sensor technologies can help through the creation of Intelligent Environments Keeping the privacyLiving in their own homesReducing the need for assistance Platforms of heterogeneous sensors as key technology player in AAL scenarios

3. INTRODUCTION (2/3) Topic of interest: - Human Behaviour Analysis - Human Behaviour Understanding Information about user’s health and lifestyle patterns Methodologies for the detection of abnormal behavior pattern Detection of anomalies in circadian rhythm (CR) AIM: Design and implementation of a tool for CR analysis  Automatic detection of anomalies with unsupervised methodology  Platform with sensing technologies invariant interface

4. INTRODUCTION (3/3) OVERVIEW OF THE PLATFORM DETECTOR LAYER Time-Of-Flight 3D Vision (TOF) Ultra-wideband Radar (UWV) MEMS-based Accelerometer (ACC) Human posture sequences SIMULATION LAYER Ground-Truth Posture Simulator Calibrated Posture Simulator Long-Term Posture Simulator REASONER LAYER Feature Extraction Reinforcement Learning (Unsupervised Clustering) CR anomalies detection

5. MATERIALS AND METHODS (1/5) DETECTOR LAYER Human postures are detected using sensing approaches implemented with both ambient and wearable solutions POSTURE DETECTORS MESA SR-4000 TOF sensorSmartex WWS PulsON 410 UWB radar Embedded-PC running detection algorithms

6. MATERIALS AND METHODS (2/5) DETECTOR LAYER COMPARISON OF THREE POSTURE DETECTORS CharacteristicTOFACCUWB Invasiveness LOWMEDIUMVERY LOW Accuracy VERY HIGHMEDIUM Robustness to object occlusion and cluttering LOWVERY HIGHHIGH Data richness VERY HIGHMEDIUM POSTURE CLASSIFICATION PERFORMANCE (Confusion Matrix)TOF ACCUWB COMMON EXPERIMENTAL SETUP FOR POSTURE RECOGNITION AND CLASSIFICATION: -18 subjects (9 males and 9 females) -Age 38±6 years, height 175±20 cm, weight 75±22 kg -Execution of typical ADLs (household tasks, meal preparation, sitting and watching TV, relaxing and sleeping, …) -Data collected simultaneously by one TOF sensor, one UWB radar, and one MEMS accelerometer worn by each participant

7. MATERIALS AND METHODS (3/5) SIMULATION LAYER Behaviour analysis  long-term observations Lack of dataset containing long-term posture sequences Conceptual representation of the posture simulator

8. MATERIALS AND METHODS (4/5) SIMULATION LAYER CALIBRATED SIMULATION OF POSTURES Simulation of posture sequences by using a calibrated approach based on real observation given by each detector node Short term observations (actions as posture sequences) Expectation-Maximization (EM) Method Model Error Modelling (MEM) Method Estimated parameters Calibrated Simulation for each node detector (Prediction Error Model) Long-term data

9. MATERIALS AND METHODS (5/5) REASONER LAYER Feature Extraction Identification of sleep periods (start time and relative duration)  ADLs recognition from posture sequences using supervised methodology (HMM)  For CR, start time of actions: “going to bed” – “sleep in bed” – “wake up” are extracted for the following module: Reinforcement Learning (Unsupervised Clustering) Mapping into two-dimensional space the features: 1) sleep start time (x-axis) 2) sleep duration (y-axis) … for each simulated day Iterative K-means Clustering (K=2) for online detection of change in CR patterns CR anomalies detection A new cluster is detected if distance between centroids > TH and features extracted for N consecutive days belong to the same new cluster. N and TH  settable according to physician’s indications

10. RESULTS (1/2) ROBUSTNESS OF CALIBRATED SIMULATION STEP For each sensor evaluation of posture classification performance MRE = Mean Relative Error by computing confusion matrix < 2% ADLs RECOGNITION STEP (7 kind of activities: sleeping, waking up, eating, cooking, housekeeping, watch TV and physical training.)

11. RESULTS (2/2) Detector N (days) 714212835 M (days) 60 TOF 76,4 69,7 73,1 90,2 73,5 84,6 90,7 77,2 86,1 94,6 81,4 89,5 95,0 84,5 92,3 UWB ACC 120 TOF 72,1 65,2 69,8 74,4 67,6 70,1 80,3 70,2 73,9 88,7 73,5 76,8 88,9 78,5 80,2 UWB ACC 180 TOF 60,4 56,4 58,7 73,7 60,1 62,6 79,7 63,0 66,9 80,2 67,9 70,4 87,4 72,7 74,2 UWB ACC DETECTION RATE (%) OF DEVIATIONS FROM THE REFERENCE CR AT VARYING OF M AND N M  Time interval in day (Reference Period) N  Periods within which a change in the CR pattern was detected VALIDATION OF CLUSTERING STEP Start Time σ = 0.6 Duration σ = 0.5 Ref Days = 60 Start Time σ = 0.9 Duration σ = 0.5 Ref Days = 60 Start Time σ = 1.2 Duration σ = 0.5 Ref Days = 60 Start Time σ = 0.6 Duration σ = 0.5 Ref Days =120 Start Time σ = 0.9 Duration σ = 0.5 Ref Days = 120 Start Time σ = 1.2 Duration σ = 0.5 Ref Days = 120

12. CONCLUSIONS Design and evaluation of sensor-independent platform for detection of CR anomalies Use of abstract features (human postures) produced by a generic detector Framework optimized for embedded processing (home application requirements: low-power consumption, noiseless and compactness) ONGOING ACTIVITIES Classification (not only detection) of sleep disorders Integration of other sensing technologies able to detect human postures (e.g. Kinect, … ) Ability to manage other behavior abnormalities (Sedentary Behavior, Hyperkinetic Behavior, …), adding other features (e.g. motion level or spatial position)

13. THANK YOU FOR THE ATTENTION