Feature Extractors for Integration of Cameras and Sensors during End-User Programming of Assistive Monitoring Systems Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science 1 of 16 ? Motion sensor

Sensors and actuators in MNFL [1] for end-user programming Alex Edgcomb, UC Riverside2 of 16 “Person at door” LED lights in house “Person at door” Outdoor motion sensor Doorbell Assistive monitoring User customizability essential [2][3] [1] Edgcomb, A. and F. Vahid. MNFL: The Monitoring and Notification Flow Language for Assistive Monitoring. Proceedings 2nd ACM International Health Informatics Symposium, Miami, Florida. [2] Philips, B. and H. Zhao. Predictors of Assistive Technology Abandonment. Assistive Technology, Vol. 5.1, 1993, pp [3] Riemer-Reiss, M. Assistive Technology Discontinuance. Technology and Persons with Disabilities Conference, 2000.

Porch light LED lights in house Expanding the previous example Alex Edgcomb, UC Riverside3 of 16 “Person at door” Outdoor motion sensor Doorbell Light sensor

Webcams are cheap 4 of 16Alex Edgcomb, UC Riverside

Webcams can do more than sensors Fall down at home In room for extended time Can do same as some sensors Motion sensor Light sensor 5 of 16Alex Edgcomb, UC Riverside Identify person at front door

Problem: Integration of webcams and sensors 6 of 16 Homesite Commercial approach: Alex Edgcomb, UC Riverside ? Outdoor motion sensor

Solution: Feature extractor 7 of Integer stream output Alex Edgcomb, UC Riverside Extract some feature Video stream input

Identify person at door in MNFL Alex Edgcomb, UC Riverside8 of 16 Outdoor motion sensor

Person in room for extended period of time in MNFL 9 of 16 Video’s YouTube link Alex Edgcomb, UC Riverside

Many feature extractors are possible 10 of 16Alex Edgcomb, UC Riverside

Are feature extractors usable by lay people? Two usability trials. 51 participants Trials required as 1 st lab assignment Non-engineering/non-science students at UCR 11 of 16Alex Edgcomb, UC Riverside

Participant reference materials One-minute video showing how to spawn and connect blocks. Overview picture 12 of 16Alex Edgcomb, UC Riverside

Example challenge problem 13 of 16Alex Edgcomb, UC Riverside actual participant solution

Trial 1: Increasingly challenging feature extractor problems 25 participants 14 of 16Alex Edgcomb, UC Riverside

Trial 2: Feature extractor vs logic block 26 participants 15 of 16Alex Edgcomb, UC Riverside

Conclusions Feature extractors – Elegant integration of cameras and sensors – Quickly learnable by lay people Future work – Develop additional feature extractor blocks – Trade-off analysis between privacy, communication, and computation 16 o f 16Alex Edgcomb, UC Riverside