ARGUS “Assisting peRsonal GUidance System for people with visual impairment” Oihana Otaegui Vicomtech-IK4

ARGUS – European Space Solutions 2012 Page 2 European Space Solutions 2012, List of content  Argus Consortium  Motivation  Objectives  Argus System  Headphonests  Argus Preliminary Tests  Conclusions

ARGUS – European Space Solutions 2012 Page 3 European Space Solutions 2012, ARGUS Project: Assisting personal guidance system for people with visual impairment  FP7-ICT  Duration: 30 Months  Start date:  End date: 

ARGUS – European Space Solutions 2012 Page 4 European Space Solutions 2012, Motivation  Visually disabled people have striking needs for trustful navigation systems providing them efficient mobility services, mainly considering safety and autonomy.  Limitations of existing products and solutions: Lack accuracy and integrity: they do not provide a suitable and efficient man- machine interface adjusted to this user segment, or rely on costly infrastructures. Are based on GNSS providing information through graphical displays or verbal output. Are not convenient for people who cannot see or have to operate in environments where verbal instructions could interfere with other external sounds.

ARGUS – European Space Solutions 2012 Page 5 European Space Solutions 2012, Motivation People with visual impairment require guidance support in outdoor activities.

ARGUS – European Space Solutions 2012 Page 6 European Space Solutions 2012, Motivation GNSS technologies are being implemented in innovative personal navigation devices Track perception will allow blind people to imagine the geometry of the path and walk along it.Track perception will allow blind people to imagine the geometry of the path and walk along it.

ARGUS – European Space Solutions 2012 Page 7 European Space Solutions 2012, Overall Goal  To develop a GNSS-based mobility service for people with impaired visually capabilities, to guide them along a pre-defined track, using acoustic and audio-haptic signals, which meets the level of accuracy and reliability they need in urban environment for improving their day-to-day life autonomy  Other outcomes: Develop innovative tools which could help blind and visually impaired people to move around autonomously and confidently. They could be used also by people working in low-visibility and hazardous conditions. Implemenation of a user-friendly portable satellite-based navigation device with acoustic and haptic user interfaces enabling users to obtain a 3D spatial insight of their surrounding environment, and providing continuous assistance to follow a predefined path in urban, rural or natural areas. Retrieve benefits from navigation services to increase the level of positioning accuracy and reliability as well as the level of service availability.

ARGUS – European Space Solutions 2012 Page 8 European Space Solutions 2012, ARGUS Guidance Concept The sequence of the bell rings allows any person to build a mental map of the path. Although the sound is the same it is perceived in a different way depending on the distance and orientation ARGUS PROPOSES 3D SOUND FOR GUIDANCE USING STEREO OUTPUT

ARGUS – European Space Solutions 2012 Page 9 European Space Solutions 2012, Binaural audio What are binaural sounds? Binaural technologies reproduce the interaural differences (arrival time and amplitude between the ears). Binaural sounds create the illusion that sounds produced by a stereo headphone emanate from specific directions and distances in the surrounding space.

ARGUS – European Space Solutions 2012 Page 10 European Space Solutions 2012, System Operation GPX file is loaded in the user terminal and activated for track guidance to reproduce the path and reach the destination. A.Track Recording B. Track on Map Processing C. Track Server Obtaining Track Data Track Recreation GPX data (Tracks and Waypoints) can be obtained through different options Absolute System Reference User System Reference GPX data recorded and activated Guidance instructions are provided based on user position and orientation obtained through GNSS and electronic compass Optional Head Mounted

ARGUS – European Space Solutions 2012 Page 11 European Space Solutions 2012, System architecture

ARGUS – European Space Solutions 2012 Page 12 European Space Solutions 2012, Headphone set Main Objective: The selection and validation of headphones that meet the safety needs of the blind and partially sighted in outdoor navigation tasks. And specifically, headphones non-disturbing the general hearing of ambient sounds. Headphones non-disturbing the general hearing 1. Audiobone 2. Vibe BS NVE Normal headphones: 3. Elecom 4. Sennheiser

ARGUS – European Space Solutions 2012 Page 13 European Space Solutions 2012, Conclusions Headphone set 95% of respondents identified one or more headphone devices that they thought could be used to safely guide people outdoors. 40% of users selected the Vibe, 30% Elecon, 19% Sennheiser and 13% Audiobone. Argus should be designed to accommodate a wide range of headphone types, which can then be chosen by each individual user to meet personal needs and preferences.

ARGUS – European Space Solutions 2012 Page 14 European Space Solutions 2012, Wizard of Oz Develop a tool in order to carry out early tests on binaural sounds perception and guiding Very valuable for user training and to help to define the values of the most relevant parameters of the Binaural Module of the PPoC Binaural Test and Validation Platform -WYSIWYG style android development tool from Google Functionality -Set of applications for testing and training users. -Not automated, two person are needed  Conductor: who triggers and simulated ARGUS operation  User: who is being guided through a route -Oral STOP message to avoid risky situations Objective -Get the personalization options for the future ARGUS system  Sound latency  Orientation perception  Distance between consecutive binaural points.  Velocity between points  Height of the perceived sound -Trainning

ARGUS – European Space Solutions 2012 Page 15 European Space Solutions 2012, User Tests: Technical Setup Positioning Unit -GPS L1: u-blox 6T evaluation kit with an USB interface for polling raw GPS L1 measurements and ephemeris -Power Supply: USB interface -Antenna: u-blox ANN-MS antenna (active)  located at the head of the applicant -notebook used for the position calculation was carried in a backpack -PVT  Snap-shot solution for position computation, based on least-square adjustment, was not adequate  Enhanced Kalman Filter (EKF) for pedestrian navigation was chosen  EDAS correction information was not used while operating with the EKF. The ionospheric influence was reduced using the Klobuchar model -PPoC Setup  Code-pseudo-ranges and Doppler information was used with an update rate of 1 Hz  Position output rate was chosen with 1 Hz  Following NMEA sentences were forwarded to the Smartphone via Bluetooth  All output information forwarded to the Smartphone was logged into a file in the notebook

ARGUS – European Space Solutions 2012 Page 16 European Space Solutions 2012, User Tests: Technical Setup Mobile device -ASES Eee Pad Transformer Tablet  Processor: NVidia Tegra 1GHz  RAM: 1GB  OS: Android 3.2  Sensors: G-Sensor, 3D Gyroscope, E-Compass, GPS -> IMU used for the PPoC Mobile Device Application -Android operating system using the Android SDK(for GUI) & Android NDK (for core) -integrates all the other components used at the PPoC Navigation Module -Radius of 2 m was set to define the reaching area for each waypoint Binaural Module -One binaural ring (2m) with sounds each 15 degrees -Finger Snap sound selected -Sound duration 1s Headset -Audiobone, Vibe and Senheiser

ARGUS – European Space Solutions 2012 Page 17 European Space Solutions 2012, User Tests: Test sites Open field -Public meadow near the Airfield Paderborn-Haxterberg -PPoC user tests

ARGUS – European Space Solutions 2012 Page 18 European Space Solutions 2012, Test Results: User’ Routes (TRK_U2_01) GNSS Reception Environment -Lower number of used satellites -No jumping  Good GNSS position performance -User walked at higher speed 3-4 km/h  COG and SOG smoother User Graphical Route -maximum deviation distance by the external GPS was 3.23 meters -maximum distance from the reference track during the first section was less than one meter

ARGUS – European Space Solutions 2012 Page 19 European Space Solutions 2012, Test Results: User’ Routes (TRK_U3_02) GNSS Reception Environment -10 satellites on average: min7 max 11 -a static period of the user of about 30 seconds -At the last third of the track, the user reduced his walking speed, when he approached the end point User Graphical Route -Except the third section, the other are inside the -/+ 3 meters corridor during most of the time -Section very sharp angle. Not very common in real life. Good overview of the behaviour of the system and the user. User must make an orientation change of more than 120º degrees to the left. The first instruction given by the audio at the reference point is a 90º change to the left (it is maximum value). Audio keeps asking for more orientation change, which may be perceived by the user as the first instruction being repeated

ARGUS – European Space Solutions 2012 Page 20 European Space Solutions 2012, Graphical comparison of heading reaction vs. heading instructions From test TRK_U2_01 and TRK_U3_02, a KML file has been generated from the internal logs of the audio played during the tests and the individual compass heading measurements -Impossible to plot all the information, only relevant point selected -Powerful overview of the performance of the system and how the user reacts to each order Test TRK_U2_01 -a good “order following” -user reacts correctly to the orders -The user process correctly the direction of the sound and therefore the user has no problem following the track

ARGUS – European Space Solutions 2012 Page 21 European Space Solutions 2012, Graphical comparison of heading reaction vs. heading instructions Zoom view in TRK_U3_02 user test where a late reaction of the user can be appreciated. -The arrows above the white line show the direction of the played sound to guide the user -the user takes a long time to change his walking direction. -Although the system is trying to get the user back to the ideal track the user keeps deviating due to his late reaction -To avoid this cases ARGUS system should increase the sound rate to warn the user and to encourage him to change his walking direction

ARGUS – European Space Solutions 2012 Page 22 European Space Solutions 2012, Numerical track Analysis Maximum distance (m) to the reference track in each section Average distance (m) to the reference track in each section Waypoints pass distance evaluation from the different tests

ARGUS – European Space Solutions 2012 Page 23 European Space Solutions 2012, Overall Conclusions The results of the Preliminary Proof of Concept tests in Paderborn have been compiled in order to show the first tangible development of the ARGUS concept. Preliminary Proof of Concept of the guidance support system, using map/tracks, binaural sounds and satellite signals has been tested successfully with four end-users performing nine navigation tasks Although the users were aware that this was a preliminary prototype they were satisfied with ARGUS system guiding capability, simplicity and non disturbance of the general hearing -the binaural guiding principle should be explained in detail before using the system in real navigation tasks in order to achieve optimum results The examined test parameters led to satisfying results with regard to the early stage of the current development phase The interviews highlighted the need of providing personalization options for the binaural guiding at the final prototype to cope with the high variability of technical skills and visual capabilities of the end users 7 out of 9 tests have an average distance error of below 5 meters to the ideal track in the four waypoint sections of the proposed route

ARGUS – European Space Solutions 2012 Page 24 European Space Solutions 2012, Overall Conclusions

ARGUS – European Space Solutions 2012 Page 25 European Space Solutions 2012, Thanks for your attention Questions?