1. Feature Extractors for Integration of Cameras and Sensors during End-User Programming of Assistive Monitoring Systems Alex Edgcomb Frank Vahid University of California, Riverside Department of Computer Science 1 of 16 ? Motion sensor

2. Sensors and actuators in MNFL [1] for end-user programming Alex Edgcomb, UC Riverside2 of 16 “Person at door” LED lights in house “Person at door” Outdoor motion sensor Doorbell Assistive monitoring User customizability essential [2][3] [1] Edgcomb, A. and F. Vahid. MNFL: The Monitoring and Notification Flow Language for Assistive Monitoring. Proceedings 2nd ACM International Health Informatics Symposium, 2012. Miami, Florida. [2] Philips, B. and H. Zhao. Predictors of Assistive Technology Abandonment. Assistive Technology, Vol. 5.1, 1993, pp. 36-45. [3] Riemer-Reiss, M. Assistive Technology Discontinuance. Technology and Persons with Disabilities Conference, 2000.

3. Porch light LED lights in house Expanding the previous example Alex Edgcomb, UC Riverside3 of 16 “Person at door” Outdoor motion sensor Doorbell Light sensor

4. Webcams are cheap 4 of 16Alex Edgcomb, UC Riverside

5. Webcams can do more than sensors Fall down at home In room for extended time Can do same as some sensors Motion sensor Light sensor 5 of 16Alex Edgcomb, UC Riverside Identify person at front door

6. Problem: Integration of webcams and sensors 6 of 16 Homesite Commercial approach: Alex Edgcomb, UC Riverside ? Outdoor motion sensor

7. Solution: Feature extractor 7 of 16 92 Integer stream output 0 100 Alex Edgcomb, UC Riverside Extract some feature Video stream input

8. Identify person at door in MNFL Alex Edgcomb, UC Riverside8 of 16 Outdoor motion sensor

9. Person in room for extended period of time in MNFL 9 of 16 Video’s YouTube link Alex Edgcomb, UC Riverside

10. Many feature extractors are possible 10 of 16Alex Edgcomb, UC Riverside

11. Are feature extractors usable by lay people? Two usability trials. 51 participants Trials required as 1 st lab assignment Non-engineering/non-science students at UCR 11 of 16Alex Edgcomb, UC Riverside

12. Participant reference materials One-minute video showing how to spawn and connect blocks. Overview picture 12 of 16Alex Edgcomb, UC Riverside

13. Example challenge problem 13 of 16Alex Edgcomb, UC Riverside actual participant solution

14. Trial 1: Increasingly challenging feature extractor problems 25 participants 14 of 16Alex Edgcomb, UC Riverside

15. Trial 2: Feature extractor vs logic block 26 participants 15 of 16Alex Edgcomb, UC Riverside

16. Conclusions Feature extractors – Elegant integration of cameras and sensors – Quickly learnable by lay people Future work – Develop additional feature extractor blocks – Trade-off analysis between privacy, communication, and computation 16 o f 16Alex Edgcomb, UC Riverside