Blind-enT: Making objects visible for blind people* Waldemar Bartyna (speaker) S. Ambroszkiewicz; M. Faderewski; S. Jakubowski; D. Kocieliński; D. Mikułowski; G. Terlikowski Institute of Computer Science, University of Podlasie, POLAND *)The work was partially supported by a grant from the British Council

The goal of our project a blind person can perceive visual attributes of objects, e.g., door, pillar, ditch, elevator, passage, room, hall, etc., having only a mobile cellular phone in his/her hand, e.g., Nokia can familiarize environment (e.g., building, street, town) by getting to know a cognitive map can be given a short and safe path on the map Guidance along this path

What is needed? mobile phone alone (even as smart as Nokia 6600) is not enough special infrastructure is needed to provide information about objects and environment to the mobile the information is processed and delivered to the blind user as voice mobile is a computer connected (via GPRS, Bluetooth, and infrared) to other computers and devices

System architecture Internet R-server elementary object complex object Blue- server mobile IrDA GPRS Bluetooth

The basic idea unlike the famous „Talking Signs”, the object description as well as a map of complex object is not in a voice format it is a special data structure expressed in terms of generic attributes and types it can be processed automatically only the result of such processing is delivered to a blind user as voice THE IDEA: generic object attributes and types

Attributes and types object is of some type, and has some attributes type and attributes are specified in uniform and digital form once the object is identified, they can be delivered to the mobile how to get object’s identifier? via infrared or/and Bluetooth transceivers embedded into the objects

Object description Object’s identifier and short textual info Type name List of attribute names and their values. Object position: X, Y, and Z coordinates List of identifiers of superior objects List of identifiers of sub objects List of neighbours. A neighbour consists of an identifier, the azimuth along with the distance from the object to this neighbour, as well as a short textual info.

Elementarny Objects wall Neighbours: straight safe and direct path between objects azimuth and distance to neighbor exit2 exit1

Object oriented maps: complex objects … … platform1 Railway station hall ticket office exit2exit1 track lamp post exit3 hall and platform1 are neighbors

Object hierarchy Object hierarchy: super and sub objects Object oriented map: hierarchical collection of object descriptions –the root object (e.g., a complex object: town or large building), –its sub-objects, sub-sub-objects, and so on up to the elementary objects. Map can be represented as a tree Restriction of the tree to a branch of length n can be viewed as the scale n of this map

Virtual tours having object oriented map, blind user can familiarize the surrounding area the mobile can display descriptions of neighbour objects, superior object, and sub objects this is delivered to blind user as voice the blind user can navigate between the objects on the map acquiring knowledge about them

XML data structures data structure for representing object attributes, types, identifiers, and descriptions are specified in XML Schema attribute, type, object identifier and description are instances of the corresponding schema these schemas are available from the project web site

The algorithm of hierarchical routing … building2 street1 Railway station building1 … street2 Graph of the neighbourhood of the immediate sub-objects of town

The algorithm of hierarchical routing 1-st floor … stairs 1 ground floor 2-nd floor stairs 2 Graph of the neighborhood of the immediate sub-objects of a building

Prototype implementation object oriented maps constructed the maps served for testing the routing algorithm and virtual tours by blind and sighted users elementary objects were simulated by the second mobile phone (Nokia 6600). Blue-server was implemented on this very notebook

Prototype implementation R-server implemented on a PC- workstation: ii4.ap.siedlce.pl Bluetooth, infrared and GPRS connectivity has been verified the application (for blind user) on the mobile phone (Nokia 6600) implemented in C++ directly on the Symbian OS

Prototype implementation experiments proved our claim: mobile phone is enough for making objects visible for blind people

Conclusions THE PROBLEM: short range of infrared connectivity (up to 6 meters) and Bluetooth connectivity (up to 10 meters). Once this problem is solved (by a manufacturer) the system can be applied in the real environment. Prototype implementation is ready for demonstration after the session. Visit our web site,