BeNeFri University Vision assistance for people with serious sight problems Moreno Colombo Marin Tomić Future User Interfaces, BeNeFri University

BeNeFri University 1.Introduction a) Background and Problem Statement b) Research Questions c) Project Description 2.Architecture and Sfotware a) Epson BT-200 b) Implementation c) Special case 3.Multimodal Interaction a) CASE b) CARE c) Output 4.Results and Conclusion 5.Future Work Summary

BeNeFri University Disabilities Various techonology Visually impaired Can wearable computing be used to help people with sight problems? How can such technology be implemented? Is it possible to augment reality through smart glasses in a way that helps specific users and provide multimodal intuitive interface? Background & Research Questions

BeNeFri University Smart glasses which can, with input of a user, mark specific items in a room to help user recognize a specific item Epson BT-200 & Unity3D Help visual impaired Visual markers Head gestures Sound output Project Description

BeNeFri University Transparent smart glasses VGA Camera Wireless Android Connected to „Epson smartphone” Epson BT-200

BeNeFri University 55yrs old Felix Wife and two sons Accident Cannot differentiate objects Red color Felix looking for restroom „Asks” glasses Follow directions Persona & Scenario

BeNeFri University „Explorer” marker + audio/blinking Head gesture + audio/blinking Special case No inputs, no choice Output combined Entire screen red Implementation

BeNeFri University Privacy concerns Prove the concept Can see, cannot perceive Memory problems Special case

BeNeFri University Demonstration

BeNeFri University Machine-side Exclusive One at a time Speech recognition Not completely implemented CASE classification

BeNeFri University Usability and fusion in MI Equivalence Necessary to use one of input CARE classification

BeNeFri University Audio for choice Always Audio for directions Meters and beeping Blinking for directions Combination for „Special case” Output

BeNeFri University Orientation towards „Special case” Qualitative evaluation Positive Detect bright red Can orient Challenges Optimization Frequency of beeping/blinking Hardware related Results

BeNeFri University Concept proved Many ideas poped out Speech recognition Change size, shape, color Smart bracelet/watch (vibration) SLAM for room mapping Depth perception Future work

BeNeFri University Thank you! Questions? Comments? Moreno Colombo Marin Tomić