1. Assisted Cognition Henry Kautz kautz@cs.washington.edu 590 AI – Autumn 2001

2. An Epidemic of Alzheimer’s Disease

5. Early Diagnosis Early-stage Alzheimer’s disease can be diagnosed with a high degree of confidence with non-invasive tests First time in human history we have a population of people who know they have Alzheimer’s

6. Pathology of Alzheimer’s Brain tissue destroyed starting from hippocampus – plaques and tangles Precise location of neuronal loss determines what abilities lost in what order Reverse child development – retrogenesis (Barry Reisberg, NYU neurologist)

7. Lost Competencies Short-term memory Memory of events Memory of concepts Ability to plan for future Desire to plan for future Ability to orient self in time and space Dressing, bathing, cooking, eating Recognize friends, relatives Introspection / awareness

8. Cognition in Context Can often compensate for physical disabilities by change in environment Wheelchairs Redesigned appliances Cognitive competence also depends on environment Can you cook dinner, given a dead animal, a stone knife, and set of flints?

9. Social Context The context for cognition involves both the physical and social environments Stability & organization of physical environment may reduce cognitive load Other people (e.g. a spouse) can actively assist in problem solving What do I do after taking a shower? Where is my razor?

10. The Million-Dollar Question Can we build computer systems that can (like a caregiver) actively assist a person with Alzheimer’s perform the tasks of day-to-day living? Enhance quality of life Prolong independence Lessen burden on other caretakers

11. Assisted Cognition Systems Learn patterns of behavior, and actively offer help / prompting as needed Remember where objects are and how they are used Recognize signs of mental distress and alert other caregivers

12. Computational Infrastructure Ubiquitous computing environment Vision/sound/motion sensors Speech input/output Portable wireless PDA’s Home robots Software layer on top of “smart house” technology

13. Scenarios

14. Monitoring Repeated Events Learn when prototypical activities (eating, sleeping, grooming) are performed Monitor behavior – notice when an activity is missed “Would you like to have lunch now?” “Did I eat lunch yet?” “You’re still awake – it’s awfully late.” Alert family member – person not moving/eating.

15. Task Sequencing Alzheimer’s suffers often have difficulty in completing a task, or moving on to the next step of a task “You’ve been standing in the hall for a while now – can I show you where the bathroom is?” “Now open the drawer and take out your socks. Okay. Get out a shirt. The closet is over here.” “Here is how you always have made a cake. First, get out the flour...”

16. Guiding Restore orientation in space In home, and/or involving a PDA-based GPS system, and/or a robot “Where’s the kitchen?” “Are you lost? This isn’t the way back home.” “Please follow me to the exercise room.”

17. Identification Compensate for of ability to recognize objects or people AC system might “whisper” name to user “The thing in your hand is a saltshaker.” (By recognizing voice) “That is your Uncle Bob who just spoke.”

18. Object Location Person with Alzheimer may have difficulty in noticing objects, especially if they are out of place “Where are my eyeglasses?” “I do not think you want to put your eyeglass in the refrigerator.”

19. Research Goals

20. Research Topics Plan and behavior recognition in the real world uncertainty plans with errors Cooperative planning and execution with an cognitively-impaired partner Representing and refining commonsense knowledge about everyday tasks Object tracking – using vision, sound, direct object sensing

21. Topics, continued User interfaces for people with Alzheimer’s (Then might make a computer usable by ordinary people too) Robust person/face/voice recognition Robotic caregivers A special kind of user interface to a AC system Evaluation

22. Related Efforts

23. Area Heating Up Georgia Tech “Aware Home” MIT Media Lab / U. Rochester Center for Future Health CMU/Pitt Nursebot Project U. Colorado Adaptive House

24. Resources at UW

25. Computer Science Artificial intelligence Ubiquitous computing Other CS areas? Intel Lab, Intel People & Practices MSR Information Science

26. Medical Community UW Alzheimer’s Disease Research Center (ADRC) UW Medical Center UW Center on Aging Schools of Medicine, Nursing, Health & Public Policy

27. Next Steps

28. The Three Ingredients of Success Ideas More brainstorming needed! Want to help paddle, or just look out for rocks? Email kautz@cs.washington.edu. Money NSF ITR coming up Nov 13 th (soon!) Students Help us define and create this vision!

29. End of Alzheimer’s in our Lifetime? Some scientists believe a vaccine effective in preventing Alzheimer’s can be developed Dale Schenk – knock-out mice Assisted Cognition Systems could also help people suffering from traumatic brain injuries, mental retardation, stroke,...