1. Vikramaditya R. Jakkula, Diane J. Cook & Aaron S. Crandall Washington State University Presented by Aaron S. Crandall

2. Index Introduction MavHome Project Experimentation Environment Temporal Relations Experimentation Process Results Conclusion & Future Work VJ © AI LAB EECS@WSU 2007

3. Smart Homes: Goals Adapt to Needs Cost Effective and Reliable Maximum Comfort and Security VJ © AI LAB EECS@WSU 2007

4. Smart home Challenges Health care Tele-Health and Health Monitoring Smart Devices Device Automation such as smart watch to read blood pressure and other vital health parameters. Assisted Living Reminder Assistant Systems Learning and Adaptation Prediction of activity and detecting anomaly in activity. Lifestyle Patterns such as preferences of daily activities, such as exercise preferences. Robotics Virtual Pets Robot enabled wheelchair and more. Human Computer Interface Audio Video based applications for event detection and analysis VJ © AI LAB EECS@WSU 2007

5. MavHome: Smart Home Project Project Unique – Focus on entire home House perceives and acts – Sensors – Controllers for devices – Connections to the mobile user and Internet Unified project incorporating varied AI techniques, cross disciplinary with mobile computing, databases, multimedia, and others VJ © AI LAB EECS@WSU 2007

6. Experimentation Environment 1 VJ © AI LAB EECS@WSU 2007

7. Experimentation Environment 2 MavHome Environment  MavLab  MavKitchen  MavPad VJ © AI LAB EECS@WSU 2007

8. Experimentation Environment 3 MavLAB Argus Sensor Network  around 100 Sensors.  include Motion, Devices, Light, Pressure, Humidity and more. VJ © AI LAB EECS@WSU 2007

9. What is a temporal relation? Food “Contains” Water or Water “Before” Pills or Food “Meets” Pills or Food “Contains” Water “before” Pills Food Water Pills Time Interval  “It is common to describe scenarios using time intervals rather than time points” - James F. Allen VJ © AI LAB EECS@WSU 2007

10. Why Temporal Relations? Reminder system based on temporal relations. Reminder Assistance If Pills are to be taken “After” Food, we can notice violation of this activity! Anomaly Detection If Cooker is Spoiled should we call emergency or a normal repair? Maintenance If Oven used for Turkey, Is turkey at Home? Temporary Need Analysis Increase prediction accuracy with association rules! Improve Prediction VJ © AI LAB EECS@WSU 2007

11. Temporal Relations Before After X During X Contains Overlaps Overlapped-By X Meets Met-by X Starts Started-By Finishes Finished-By Equals VJ © AI LAB EECS@WSU 2007 T EMPORAL R ELATION V ISUAL D IAGRAM C ONSTRAINT U SABLE X Before Y Y Contains X X Overlaps Y Start(X)<Start(Y); End(X)<Start(Y) Start(X)>Start(Y); End(X)<End(Y) Start(X)<Start(Y); Start(Y)<End(X); End(X)<End(Y) X Meets Y XFinishesY YFinishedbyX XStarts Y YStartedby X X Equals Y Start(Y) = End(X) Start(X)≠Start(Y); End(X) = End (Y) Start(X)≠Start(Y); End(X) = End (Y) Start(X)=Start(Y); End(X)≠End(Y) Start(X)=Start(Y); End(X)=End(Y) X X X X Y Y X X Y Y X X Y Y X X Y Y X X Y Y X X Y Y X X X X Y Y Y Y Y Y

12. Datasets: Real vs. Synthetic Real Dataset and synthetic datasets consist timestamp of the activity with the activity name and the state it is in. VJ © AI LAB EECS@WSU 2007 Real Dataset (Sample): 3/2/2003 12:40:0 AM, (Studio E) E9 OFF 3/2/2003 2:40:0 AM, (Living Room) H9 ON 3/2/2003 2:40:0 AM, (Living Room) H9 OFF 3/2/2003 6:4:0 AM, (Living Room) H9 OFF 3/3/2003 3:43:0 AM, (Studio C) C14 ON Synthetic Dataset (Sample): 2/1/2006 10:02:00 AM, off, oven 2/1/2006 11:00:00 AM, on, lamp 2/1/2006 11:11:00 AM, off, thermostat 2/1/2006 12:02:00 PM, off, lamp 2/1/2006 12:35:00 PM, off, cooker Datasets Parameter Setting No of DaysNo of Events No of Intervals Identified Size of Data Synthetic6081729106KB Real60171623104KB Train: 59 Days Test: 1 Day

13. Experimentation Step 1: Find frequent activities in the sensor data. Step 2:. Parse raw sensor data and formulate time intervals for event occurrences and Associate temporal relations. Step 3: Run for Anomaly detection. VJ © AI LAB EECS@WSU 2007

14. The Apriori Algorithm Example for detecting frequent events Database D Scan D C1C1 L1L1 L2L2 C2C2 C2C2 C3C3 L3L3 VJ © AI LAB EECS@WSU 2007

15. Temporal Intervals Process raw data to form temporal intervals. Raw Sensor Data Timestamp Sensor State Sensor ID 3/3/2003 11:18:00 AM OFF E16 3/3/2003 11:23:00 AM ON G12 3/3/2003 11:23:00 AM ON G11 3/3/2003 11:24:00 AM OFF G12 Identify Time Intervals Date Sensor ID Start Time End time. 03/02/2003 G11 01:44:00 01:48:00 03/02/2003 G19 02:57:00 01:48:00 03/02/2003 G13 04:06:00 01:48:00 03/02/2003 G19 04:43:00 01:48:00 Associated Temporal Relations Date time Sensor ID Temporal Relation Sensor ID 3/3/2003 12:00:00 AM G12 DURING E16 3/3/2003 12:00:00 AM E16 BEFORE I14 3/2/2003 12:00:00 AM G11 FINISHESBY G11 4/2/2003 12:00:00 AM J10 STARTSBY J12 Raw Sensor DataInterval Data Temporal Relations Data VJ © AI LAB EECS@WSU 2007

16. Temporal Interval Analyzer Algorithm : Temporal Interval Analyzer Input: data timestamp, event name and state Repeat While [Event && Event + 1 found] Find paired “ON” or “OFF” events in data to determine temporal range. Read next event and find temporal range. Identify relation type between event pair from possible relation types (see Table 1). Record relation type and related data. Increment Event Pointer Loop until End of Input. VJ © AI LAB EECS@WSU 2007

17. Anomaly Detection Process 1] Calculate the evidence of occurrence with other frequent events as shown in probability model in next slide. 2] Calculate the anomaly. 3] Check if the calculated anomaly is equal to or greater than (mean + 2 * St. Dev.), If yes, declare it as an Anomaly. VJ © AI LAB EECS@WSU 2007

18. Probability Model P(Z|Y) = |Before(Y,Z)| + |Contains(Y,Z)| + |Overlaps(Y,Z)| + |Meets(Y,Z)| + |Starts(Y,Z)| + |StartedBy(Y,Z)| + |Finishes(Y,Z)| + |FinishedBy(Y,Z)| + |Equals(Y,Z)| / |Y| Anomaly z = 1 – P(Z|Y) VJ © AI LAB EECS@WSU 2007

19. Experimentation Results Frequent EventEvidenceAnomalyDetected J100.450.55No J110.320.68No A110.330.67No A150.240.76No A110.230.77No A150.220.78No I110.270.73No I140.340.66No Anomaly Mean0.7 Anomaly St. Dev.0.071764 Anomaly Cut-off Threshold0.8435 Anomaly Detection on Real Dataset Frequent EventEvidenceAnomalyDetected Lamp0.30.7NO Lamp0.230.77NO Lamp0.010.99YES Fan0.320.68NO Cooker0.290.71NO Lamp0.450.55NO Lamp0.230.77NO Lamp0.010.99YES Lamp0.230.77NO Fan0.30.7NO Cooker0.340.66NO Lamp0.330.67NO Lamp0.20.8NO Lamp0.020.98NO Lamp0.0020.998YES Fan0.340.66NO Cooker0.420.58NO Anomaly Mean0.763412 Anomaly St. Dev.0.135626 Anomaly Cut-off Threshold1 Anomaly Detection on Synthetic Dataset VJ © AI LAB EECS@WSU 2007

20. Experimentation results VJ © AI LAB EECS@WSU 2007

21. Conclusion Unique and new Approach. Larger datasets would be incorporated and experimented soon. Promising performance on the synthetic datasets. VJ © AI LAB EECS@WSU 2007

22. Future Direction Expansion of the temporal relations by including more temporal relations, such as until, since, next, and so forth, to create a richer collections. Adapting to multiple inhabitants using Entity discovery approach. VJ © AI LAB EECS@WSU 2007

23. Questions VJ © AI LAB EECS@WSU 2007 Thank You