T-Party Joint Steering Committee September 20, 2005Slide 1 Personalized Virtual Caregivers Randall Davis (for Grimson, Guttag, Darrell, Freeman)

T-Party Joint Steering Committee September 20, 2005Slide 2 Personalized virtual caregivers Personalized caregivers – –monitoring cognitive and physical activity to allow subjects to age in place; –monitoring and assisting with the needs of subjects with chronic conditions; –supporting wellness monitoring. Enormous potential benefit to individuals, if such systems could be deployed. Huge potential for mass market products for home or institutional markets, associated with these needs: –Individual monitoring components, communication systems for monitoring

T-Party Joint Steering Committee September 20, 2005Slide 3 Other Applications Surveillance, home and business security. Monitoring of automobile driver’s awareness and mental state.

T-Party Joint Steering Committee September 20, 2005Slide 4 Outcomes Allow aging or infirm patients to lead normal lives at home Monitor everyday aspects of wellness in the full population.

T-Party Joint Steering Committee September 20, 2005Slide 5 Key technologies Enabling computer vision technologies: –Activity monitoring and recognition, tracking activity trends, detecting unusual events. –Tracking changes in physical movement. –Visual recognition of physical features. Sensor network technology: –Data acquisition and management in distributed networks of sensors. Monitored skin mole

T-Party Joint Steering Committee September 20, 2005Slide 6 Key technologies Vision algorithms for monitoring, recognition, tracking Consumer level modules for use in home, and other settings

T-Party Joint Steering Committee September 20, 2005Slide 7 Two Examples Fall prevention –Track episodes of instability –Correlate with place, time, activity –Suggestion environmental/behavioral changes Medication compliance –Use vision+other sensors to estimate Taking of medication Factors governing recommended dosage Physiological state

T-Party Joint Steering Committee September 20, 2005Slide 8 Deliverables and schedule Wellness monitor –Year 1: acquire data in real settings; create algorithms to extract relevant features for analysis. –Year 2: full prototype: handheld unit with wireless communication to analysis system. –Year 3: extension to other wellness parameters, second generation prototype. Compliance monitor –Year 1: recognition of specific arm and hand movements, and objects. Recognize medication ingestion. –Year 2: integrate system into full prototype; deploy in home settings. Test system in conjunction with medical experts. –Year 3: include exercise and nutrition wellness compliance parameters, second generation prototypes. Activity level monitor –Year 1: develop distributed camera system, wireless communications system and algorithms to acquire visual data and analyze coarse level activities. –Year 2: full prototype; integrate into deployable package. Test in range of settings. –Year 3: second generation prototype, refined to analyze more intricate or subtle behavior.