1. A Lightweight Computer- Vision-based Electronic Travel Aid Andrew B. Raij (raij@cs.unc.edu)raij@cs.unc.edu Enabling Tech Project Final Report 4/17/2003 Department of Computer Science, UNC-Chapel Hill

2. Long-term Goals Design and build a wearable, inconspicuous device that assists people with visual disabilities with orientation and mobility (O&M) Gathers info from the environment with one or more cameras and presents the data to the user in a “useful form” Focus on gathering depth info this semester Explore useful ways to present depth to user

3. Previous ETAs (1) Cane A depth and acoustics probe for nearby things No long distance probing No probing up high Requires scanning back and forth across world

4. Previous ETAs (2) NavBelt Spatial audio – possibly interferes with user’s auditory feedback from the world Uses sonars to build a local, 2D, birds-eye map Vertical resolution of depth is poor Modes Guidance Mode – guides users around obstacles, but requires dead-reckoning Image Mode – scans across map, increases pitch and volume as depth decreases. Too much info, requires lots of training

5. Previous ETAs (3) GuideCane Cane attached to robot that turns at obstacles but maintains user’s desired course Guidance tech based on NavBelt Guidance Mode Very intuitive, natural force feedback Restricted to relatively flat surfaces How does the user feel about being led by a robot? Control issues? Concerns about appearing different?

6. Previous ETAs (4) vOICe (http://www.seeingwithsound.com)http://www.seeingwithsound.com Scans across images (like the NavBelt image mode) and translates images to different sound cues. Uses images - Inherently good vertical and horizontal resolution, scans world for user Auditory feedback could be lots of information to deal with at once Can require lots of time practicing to get used to it Some users claim it almost makes them feel like they are seeing Can also work with depth images

7. Depth Extraction We want to determine X given the two views First find the fundamental matrix F  x’Fx = 0 Use the fundamental matrix to find the projection matrices P and P’ for C and C’ Use least-squares linear triangulation method to calculate X fast This method is more prone to errors from noise

8. Estimating F Estimating the fundamental matrix F 7 or more corresponding points Insert known points in scene Structured light Checkerboard robust searching for corresponding points in 2 views possible, but not at every frame Need a scene with good features if geometric relationship between views is constant, so is F, P, P’ Only need to estimate F, P, P’ once, or maybe just every once in a while

9. Depth Sampling At every frame we need to choose points to sample for depth All can be fast if epipolar lines are parallel to scanlines Edges and surrounding areas? Object extraction? Correspond them in 2 views Points that do not correspond well in both images are not processed further

10. Spatial Perception (Ungar) Near-Space (Haptic) vs. Far-Space (Locomotor) Both spaces directly connected through vision Early and congenitally blind generally code the world in terms of their own bodies, i.e. near-space Spatial relationships are understood but spatial inference, rotation can be difficult Recent research suggests people who are blind tend to not learn/use strategies for understanding locomotor space and its connection to haptic space Hypothesis: tactile maps can bridge the gap between locomotor and haptic space

11. Tactile Display Convey information through touch Two kinds of tactile displays I read about Vibration Thermal Others Pin Arrays Electric

12. Vibration Vibration for phones, PDAs, other small form factor devices Phone Ambient Touch Tactile Virtual Buttons Gloves, vests with piezo-electric buzzers Lots of others

13. Thermal Not enough work here Mostly for VE research blowing hot air on skin Thermal Phantom, Gloves Peltier cell, water-cooled heat sink Temperature Perception Spatial resolution is poor Interesting spatial summation properties Slow temp changes not very noticeable but fast changes are Body adapts to temperature change over time Unlikely to perceive clearly several areas on body changing temp Safety issues!