January Smart Environments: Artificial Intelligence in the Home and Beyond Diane J. Cook

BodyNets Topics 2 Smart Environments Both generate large amounts of data Both need: data mining prediction classification / identification

What Is a Smart Environment? An environment able to acquire and apply knowledge about the resident and the physical surroundings to improve the resident’s experience.

Imagine… Face recognition, automated door entrySmart sprinklers Lighting control Door/lock controllers, Surveillance system Robot vacuum cleaner Robot lawnmower Intelligent appliances Climate control Automated blinds Remote site monitoring and control

What Good Is a Smart Environment? Comfort and Productivity Health and Safety Energy and Sustainability

Our Initial Hypothesis Resident interactions with smart environments can be automated by analyzing and predicting resident behavior and using the information to learn an automation model.

Smart Apartment

Smart Workplace

Technology Perception Sensor network Off-the-shelf sensors Action PLC Sensor network Reason Data mining Prediction Decision maker Rule engine

Data Mining Raw data :47:30 i mavpad :56:17 i mavpad :04:45 a mavpad :05:37 i 3 1 mavpad :05:37 i mavpad :06:11 c mavpad :06:22 c mavpad :16:33 S mavpad :16:33 S mavpad :16:34 i mavpad :16:44 i mavpad … Patterns % Turn on kitchen light when in room [PRODUCTION_PATTERN] ID = V_19_1 V_19_0 i_1_1 x_0_0 NumberOccurrences = 5 V_19_1 V_19_0 V_19_1 V_19_0…x_0_0 1 % Kitchen Activities [ABSTRACT_PATTERN] ID = NumberOccurrences = …

Prediction Parse sensor events using text compression Given recent history, predict next symbol (sensor event) with highest probability  a (5)b(4)d(2)c(1) b(2)a (2) c(1)a (1) c(1) b(2)a (1) “aaababbbbbaabccddcbaaaa”

Decision Model Learn hierarchical model from data Learn automation rules with user and rule engine feedback

Results - Workplace

Results - Apartment

Prototype Smart Apartment

What Good Is a Smart Environment? Comfort and Productivity Health and Safety Energy and Sustainability

The Crisis of an Aging Population By 2040, 23% of U.S. population will be 65+ Facilities cannot handle coming “age wave”

The Crisis for Individuals Virtually ALL want to stay at home

Financial Impact Americans spend $44 billion/year for home care Family members provide $197 million/year U.S. healthcare spending will increase by 25% SenseIdentifyRemindAssistAssess

10 / 2 / 15 If we can keep 10% of population home 2 years longer, we will save Washington State $341 million/year $15 billion/year nationally

Mission Our vision is to keep older adults functioning independently in their own homes as long as possible. Our method integrates artificial intelligence, machine learning, and pervasive computing technologies into their everyday environments, turning them into smart environments.

What Assistance Is Needed? Activity identification Functional assessment Tracking Eating Toileting Medical monitoring Medication management NowNowPartialPartialFutureFuture

Activities of Daily Living (ADL) Technology 23 Sense Identify RemindAssist Assess

Sense IdentifyRemindAssistAssess

SenseIdentifyRemindAssistAssess Identify

Motion1Motion1 ENDEND Motion2Motion2 Motion3Motion3WaterOnWaterOn Motion4Motion4 WaterOffWaterOff Model SenseIdentifyRemindAssistAssess P(“wash hands” | [Motion 1, Motion 3, Water On, Water Off]) = 0.54 P(“cooking” | [Motion 1, Motion 3, Water On, Water Off]) = ?

Remind SenseIdentifyRemindAssistAssess

Remind Medicine compliance SenseIdentifyRemindAssistAssess

Assist 29 SenseIdentifyRemindAssistAssess Voice / Hand Disability

Assess Replace lab with everyday environments Detect dementia as it occurs Monitor as it progresses SenseIdentifyRemindAssistAssess

What Good Is a Smart Environment? Comfort and Productivity Health and Safety Energy and Sustainability 31

Load Profiling for Smart Home Visualization of energy usage Play in real time or rewind Annotation with activity IDs

Energy-Efficient Automation Alternative Automation of Daily Activities Open south windows to warm room Automatically learn strategy based on Positive feedback (reduced energy usage) and Negative feedback (resident reverses automation)

Smart Communities Learn usage profiles for entire community Use information to balance load Avoid brownouts and blackouts Reward customers who sign up for conservation plan

What’s Next? 35 Multiple residents Multiple environments Multiple perspectives

To Learn More 36