1. January 20081 Smart Environments: Artificial Intelligence in the Home and Beyond Diane J. Cook cook@eecs.wsu.edu http://eecs.wsu.edu/~cook

2. BodyNets Topics 2 Smart Environments Both generate large amounts of data Both need: data mining prediction classification / identification

3. What Is a Smart Environment? An environment able to acquire and apply knowledge about the resident and the physical surroundings to improve the resident’s experience.

4. Imagine… Face recognition, automated door entrySmart sprinklers Lighting control Door/lock controllers, Surveillance system Robot vacuum cleaner Robot lawnmower Intelligent appliances Climate control Automated blinds Remote site monitoring and control

5. What Good Is a Smart Environment? Comfort and Productivity Health and Safety Energy and Sustainability

6. Our Initial Hypothesis Resident interactions with smart environments can be automated by analyzing and predicting resident behavior and using the information to learn an automation model.

7. Smart Apartment

8. Smart Workplace

9. Technology Perception Sensor network Off-the-shelf sensors Action PLC Sensor network Reason Data mining Prediction Decision maker Rule engine

10. Data Mining Raw data 2005-01-03 09:47:30 i 5 1 100 mavpad 2005-01-03 09:56:17 i 5 0 0 mavpad 2005-01-03 13:04:45 a 1 1 100 mavpad 2005-01-03 13:05:37 i 3 1 mavpad 2005-01-03 13:05:37 i 3 1 100 mavpad 2005-01-03 13:06:11 c 4 1 100 mavpad 2005-01-03 13:06:22 c 4 0 0 mavpad 2005-01-03 13:16:33 S 19 1 100 mavpad 2005-01-03 13:16:33 S 70 1 100 mavpad 2005-01-03 13:16:34 i 5 1 100 mavpad 2005-01-03 13:16:44 i 3 0 0 mavpad … Patterns % Turn on kitchen light when in room [PRODUCTION_PATTERN] ID = 56008634 4 V_19_1 V_19_0 i_1_1 x_0_0 NumberOccurrences = 5 V_19_1 V_19_0 V_19_1 V_19_0…x_0_0 1 % Kitchen Activities [ABSTRACT_PATTERN] ID = 54964 4 1487735365 1487712271 56008634 100100 NumberOccurrences = 1 1487735365 1487712271 56008634 100100 1101000101101000…

11. Prediction Parse sensor events using text compression Given recent history, predict next symbol (sensor event) with highest probability  a (5)b(4)d(2)c(1) b(2)a (2) c(1)a (1) c(1) b(2)a (1) “aaababbbbbaabccddcbaaaa”

12. Decision Model Learn hierarchical model from data Learn automation rules with user and rule engine feedback

13. Results - Workplace

14. Results - Apartment

15. Prototype Smart Apartment

16. What Good Is a Smart Environment? Comfort and Productivity Health and Safety Energy and Sustainability

17. The Crisis of an Aging Population By 2040, 23% of U.S. population will be 65+ Facilities cannot handle coming “age wave”

18. The Crisis for Individuals Virtually ALL want to stay at home

19. Financial Impact Americans spend $44 billion/year for home care Family members provide $197 million/year U.S. healthcare spending will increase by 25% SenseIdentifyRemindAssistAssess

20. 10 / 2 / 15 If we can keep 10% of population home 2 years longer, we will save Washington State $341 million/year $15 billion/year nationally

21. Mission Our vision is to keep older adults functioning independently in their own homes as long as possible. Our method integrates artificial intelligence, machine learning, and pervasive computing technologies into their everyday environments, turning them into smart environments.

22. What Assistance Is Needed? Activity identification Functional assessment Tracking Eating Toileting Medical monitoring Medication management NowNowPartialPartialFutureFuture

23. Activities of Daily Living (ADL) Technology 23 Sense Identify RemindAssist Assess

24. Sense IdentifyRemindAssistAssess

25. SenseIdentifyRemindAssistAssess Identify

26. Motion1Motion1 ENDEND Motion2Motion2 Motion3Motion3WaterOnWaterOn Motion4Motion4 WaterOffWaterOff 0.4 0.6 1.0 0.1 1.0 0.9 Model SenseIdentifyRemindAssistAssess P(“wash hands” | [Motion 1, Motion 3, Water On, Water Off]) = 0.54 P(“cooking” | [Motion 1, Motion 3, Water On, Water Off]) = ?

27. Remind SenseIdentifyRemindAssistAssess

28. Remind Medicine compliance SenseIdentifyRemindAssistAssess

29. Assist 29 SenseIdentifyRemindAssistAssess Voice / Hand Disability

30. Assess Replace lab with everyday environments Detect dementia as it occurs Monitor as it progresses SenseIdentifyRemindAssistAssess

31. What Good Is a Smart Environment? Comfort and Productivity Health and Safety Energy and Sustainability 31

32. Load Profiling for Smart Home Visualization of energy usage Play in real time or rewind Annotation with activity IDs

33. Energy-Efficient Automation Alternative Automation of Daily Activities Open south windows to warm room Automatically learn strategy based on Positive feedback (reduced energy usage) and Negative feedback (resident reverses automation)

34. Smart Communities Learn usage profiles for entire community Use information to balance load Avoid brownouts and blackouts Reward customers who sign up for conservation plan

35. What’s Next? 35 Multiple residents Multiple environments Multiple perspectives

36. To Learn More 36 http://eecs.wsu.edu/~cook cook@eecs.wsu.edu