1. Screen Readers Cannot See (Ontology Based Semantic Annotation for Visually impaired Web users) Yeliz Yesilada, Simon Harper, Carole Goble and Robert Stevens Information Management Group (IMG) Department of Computer Science University of Manchester

2. ICWE2004 2/24 Introduction Problem: For visually impaired, mobility is reduced; the objects that support travel are inaccessible or missing altogether. Solution: Analysis of travel support offered by Web pages and provide tool support for enhancing travel (mobility). Message: Travel support can be improved if the objects that support travel are presented in a way that they can fulfill their intended roles and ease the travel.

3. ICWE2004 3/24 Travel - Navigation and orientation with purpose, within an environment. Mobility - Confident navigation and orientation with purpose, ease and accuracy within an environment. Travel and Mobility Orientation - ‘Where am I?’ Navigation - ‘Where can I go?’

4. ICWE2004 4/24 Problem Statement with an Example Page has 185 links image map ref equals top_nav_sb_gateway/ 104-1393710-1148754 image map link ref equals cm_wl_topnav_gateway/104-1393710-1148754?type=wishlist

5. ICWE2004 5/24 Travel objects: Way points, orientation points and travel assistants Mobility techniques and actions: Consulting and reviewing, landmark recognition and avoidance Mobility instruments: In-journey guidance, previewing, probing and feedback A Model of Travel [Goble et al., 2000]

6. ICWE2004 6/24 Travel Objects Travel memory Way Points Orientation Points Decision point Way edge Navigation point Identification point Alert Attention Reference point Location & Position Direction Distance Travel Assistants Information point Travel aid Travel support

7. ICWE2004 7/24 Travel Objects- Inventory Examples (1) Decision Point Reference Point Attention Distance Way Edge

8. ICWE2004 8/24 Travel Objects- Inventory Examples (2) Location and Position Identification Point Way Edge Reference Point

9. ICWE2004 9/24 What do we need to do? Analyse Web pages; Identify objects that support mobility; Discover their roles; Make these objects and knowledge about them explicit; Associate the knowledge to these objects; Transform pages by using the explicit knowledge in order to fulfil the intended roles of these objects.

10. ICWE2004 10/24 A Mobility Support Tool-- DANTE 1.Extracting travel objects; 2.Discovering their roles; Travel Ontology Analysts  3.Annotating extracted objects; 4.Transforming the page. Web page Provide Suggestions  DOM Enhanced DOM Travel Analysis Heuristics (1)  Annotation 1 Heuristics (2) Transformation   User Agent 2 3 4 Manual Auto. COHSE Annotation Service Ontology Service Annotation DB     

11. ICWE2004 11/24 Introducing the Travel Ontology In the context of DANTE, the ontology is used as A representation of a shared conceptualisation of knowledge about the mobility of visually impaired people and structures widely supported by Web pages; The controlled vocabulary to guide page transformations. It consists of three parts: The mobility concepts The authoring concepts The context of a journey

12. ICWE2004 12/24 The Travel Ontology– Mobility Semantics WayPoint WayEdge DecisionPoint IdentificationPoint ReferencePoint NavigationPoint Encapsulates the knowledge about the travel objects from the real world mobility studies. Aims to capture knowledge about how the travel objects are used (their role in supporting movement) in a typical journey. E.g., EnvironmentalRole Travel Assistant TravelSupport TravelMemory TravelAid InformationPoint MobilityConcept JourneyRole ObstacleCueOutOfView

13. ICWE2004 13/24 The Travel Ontology– Authoring Semantics Holds information about including hypermedia concepts and vocabularies used in previous work on transcoding. Aims to capture knowledge about how the travel objects are presented (their structural properties). E.g., Chunk Footer Header List SearchEngine BreadcrumbTrail NavigationalList AuthoringConcept Collection Node Atom Advertisement Label Caption Heading Separator

14. ICWE2004 14/24 The Travel Ontology– Context Semantics A Web journey can take place in different contexts and concepts in this group provide contextual knowledge about a journey such as the purpose of the journey being undertaken. E.g., Purpose TravellerPurposeObjectPurpose Searching Scanning Querying AidsNavigation AidsOrientation

15. ICWE2004 15/24 The Travel Ontology– Example Header WayEdge Logo ReferenceP oint LinkMenu Decision Point Navigation Point Chunk WayEdge Heading IdentificationPoint

16. ICWE2004 16/24 Annotation Example– COHSE Annotator 2 3 1 2 3 1

17. ICWE2004 17/24 Annotation Example– DANTE RDF Editor

18. Fragmentation into several pages Transformation Examples (1)

19. ICWE2004 19/24 Transformation Examples (2) Providing the overview of the page

20. ICWE2004 20/24 Eliminating repetitions Transformation Examples (3)

21. ICWE2004 21/24 Enabling movement to the focus of the page Transformation Examples (4)

22. ICWE2004 22/24 Open Issues and Future Work Dynamic pages need dynamic annotation; Automate the process of annotation; Deep annotation (annotate pages at early stages of design); Over-annotation vs. under-annotation; Appropriateness of transformation alternatives (prioritise); Annotations might result in conflicting transformation heuristics; The combination effect of annotation and transformation; Using reasoning mechanism of ontologies; Evaluation of annotations and transformations.

23. ICWE2004 23/24 Summary Travel and mobility on the Web is likened to travel and mobility in the physical world; A mobility support tool, DANTE, is presented for extracting travel objects, discovering their roles, annotating and transforming pages by using the annotations; Travel ontology is introduced– mobility and authoring concepts and information about the context of a journey; Web pages are manually annotated by using COHSE annotator; Web pages can be automatically transformed based on the manual annotations; Visually impaired Web users could also benefit from Semantic Web technologies and we demonstrate a possible approach.

24. ICWE2004 24/24 References Carole Goble, Simon Harper, and Robert Stevens. The travails of visually impaired web travellers. In Proceedings of the Eleventh ACM on Hypertext and Hypermedia, pages 1-10, 2000. Yeliz Yesilada, Robert Stevens, and Carole Goble. A foundation for tool based mobility support for visually impaired web users. In Proceedings of the Twelfth International Conference on World Wide Web, pages 422–430, 2003. Http://Augmented.man.ac.uk/ontologies/TravelOntology.owl Http://Augmented.man.ac.uk Http://Towel.man.ac.uk URLs