1. Semi-automated Coaching for Elderly Collaborative effort between UCBerkeley, OHSU and NorthEastern University

2. Motivation We are interested in developing predictive models of physical activities of human using robotic methodology and technologies. This approach is facilitated by the advances in sensing (vision, force, touch, accelerometers, etc.), computational power and new algorithms. This domain is very reach for investigation since every person is different and even the same person over time changes its physical capabilities.

3. Semi-automated Coaching for Elderly Improve health-status of elderly by semi-automated health-coaching. Goal of this project is to model behavior change of elderly during health-coaching intervention. Development of automated coaching platform based on Microsoft Kinect camera. o Š. Obdržálek, G. Kurillo, F. Ofli, R. Bajcsy, E. Seto, H. Jimison, M. Pavel, "Accuracy and Robustness of Kinect Pose Estimation in the Context of Coaching of Elderly Population, EMBC, 34th International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, California, August 2012Accuracy and Robustness of Kinect Pose Estimation in the Context of Coaching of Elderly Population o Š. Obdržálek, G. Kurillo, E. Seto, R. Bajcsy, "Architecture of an Automated Coaching System for Elderly Population", Stud Health Technol Inform. 2013;184:309- 11 (Proceedings of MMVR 2013).Architecture of an Automated Coaching System for Elderly Population o F. Ofli, G. Kurillo, S. Obdrzalek, R. Bajcsy, H. Jimison, M.Pavel, "Design and Evaluation of an Interactive Exercise Coaching System for Older Adults: Lessons Learned", Journal of Biomedical and Health Informatics. 2015. Accepted.

4. Prototype System Kinect Exercise system: Unobtrusive, low-maintenance, and low-cost. Guides user through a series of exercises Observes proper execution of the exercises Provides feedback about performance Records the performance achieved Tracks exercise system usage

5. Real-time Data Processing

6. User Interface Design Elderly have various sensory and cognitive impairments. User interface was adapted based on feedback from coaches and users. We considered: mouse & keyboard, voice control, gesture control, PowerPoint remote for UI.

7. In-Exercise Feedback Visual Video of the coach Data form Kinect Repetition counting Messages Auditory Exercise Instructions Spoken messages Exercise Instructions Exercise Feedback

8. Experimental Results Experimental Study 1: 6 subjects at home 10 sessions over period of 2-4 weeks Only 12 exercises Experimental Study 2: 7 subjects at home 16 weeks, exercise every other day About 40 exercises derived from 12 basic exercises

9. Example of Buddha’s Prayer Exercise: Skeletal data recorded during exercise with motion feature-based segmentation. Wrist lift measurement in Buddha’s Prayer during first 2 weeks of exercise.

10. Results for Subject #2 After 10 weeks Daily chart of number of exercises completed in each exercise session. Weekly chart of differences in survey responses before and after the exercise

11. Conclusions Lessons Learned: Every household is different and one needs to accommodate the system to these differences. The system must be simple to the user! The user interfaces must be simple yet rich enough not support boredom. There are great differences of expectations from the healthcare provider and the behavioral scientist. The first is interested in coarse classifications of the state of the user, such as endurance, strength, stability, while the behavioral/medical scientist wants to know detailed range of movements of joints in order togive a proper rehab. Advice.