1. Slides by Yevgen Borodin (slides adapted for Psych 384, 3/3/09) Department of Computer Science, Stony Brook University A Vision for a Universally Accessible Web

2. The web is designed for those who can filter out irrelevant information

3. Non-Visual Web Browsing Jaws, Windows Eyes, Hal Serial audio interface Shortcut-driven navigation in HTML DOM-tree Navigation between links, headers, lines, etc. Inaccessible images, links, multimedia, etc.

4. How blind people browse the web Landmarks Searches List of links Headings Structure Speed of speech Static sites [Bigham, et. al., ASSETS’07]

5. What is HearSay? Started: Motivated by work in AI Information extraction from web pages Process modeling Added an audio interface Now : Working with HKSB and HKNC Collaborate with IBM, UW Several faculty members, Ph.D., MS., undergrad

6. HearSay 3 Free!!! Multi-platform Focused on Web browsing Flexible multimodal interface Supports text-to-speech engines Supports voice recognition engines

7. Improving navigation would make web browsing more efficient. Segment pages Identify patterns Add 2D navigation Summarize content All this helps, but...

8. Project Goals Filter out irrelevant information Discover relevant information Provide quick access to relevant content Evaluate the usability of HearSay Compare HearSay to other screen readers Distribute a stable version of the program for free

9. Scenarios Relevancy in Ad-hoc Web Browsing Relevancy when Web content changes Relevancy in Online Transactions (e.g. shopping, paying bills) But what is relevant?

10. Manual Annotations of Content (the user could tell us what is relevant) Beginning of Main Content Search Button

11. Needed for Manual Labeling: Provide an interface for creating annotations Store annotations in a database Query the database when the page loads Apply the metadata to the page Provide an interface for reviewing the annotations

12. Collaborative authoring of accessibility metadata [Takagi et al, 2008] Social network connecting end-users and volunteers  [http://socialaccessibility.alphaworks.ibm.com/] Accessibility Commons (AC) DB to store metadata [Kawanaka, Borodin et al, 2008] Web-based infrastructure for sharing metadata

13. Benefits of Social Accessibility Shortens the time for accessibility renovations Supported: headings, ALT tags, and titles  Workshop at UW – formed a consortium Defined the Accessibility Commons DB schema Identified the object addressing methods: XPath, MD5, URI

14. Automatic labeling of content to support web transactions

15. Labeling content Non-visual web transactions are difficult Consider all problems with non-visual browsing Need to locate relevant concepts (buttons and links) Relevant concepts are similar across websites Some variations, e.g. “add to cart”, “add to bag” Different labels, e.g. “Search”, “Go”, “Find” Evolution of relevance and form over time

16. Or instead of annotating elements, the whole process could be automated. Ex: AT&T Log-in Page

17. AT&T: Account Overview Page

18. AT&T: Make a Payment Page

19. AT&T: Confirm Payment Details

20. Macro Recording Interface Create a recording (non-)visually Save recording with a description Voice interface to replay the macro-recording Page (in)dependence Customizing what is read Specifying variables

21. Context-Directed Browsing

24. Something has Just Changed…?

25. Dynamic Web Content Dynamic content: Our actions often cause change We pay attention to changes of content New information is often in the changes Affects relevancy of information Types of updates: Page refresh, redirect, JavaScript and AJAX updates Source of updates: User-invoked and Timer-based

26. Another example:

27. Dynamic Content Paradigm Treat any content changes as “updates”: AJAX, JavaScript, refresh, redirect Navigation by following links Using back and forward buttons Analyze and diff the updated Web content Provide interface for reviewing the changes

28. Page Refresh

29. Filtering Repeated Content

30. User-Centric Goals Discover and present relevant information first Minimize access-time to relevant information Keep users focused on tasks and information Facilitate multi-tasking and refocusing Enable automation of repetitive tasks Keep the context of user actions Minimize system distractions

31. HearSay for the Sighted Browsing on handhelds Browsing over the phone Browsing on-the-go Other services

32. Web Accessibility in Handhelds

33. Mobile Browsing Problems Data Transfer Cost is High Connection is Slow Small Screens Lots of Scrolling

34. Context-driven Browsing

35. External Collaborators Accessibility Group at IBM Japan Accessibility Group at Google HKSB and HKNC Arizona State University Conferences: ASSETS, W4A, CSUN

36. Conclusion Web Accessibility is an important problem Glimpse of Interesting Approaches Much remains to be done: E.g. Integration, Robustness (Specification and Verification) Sonification Other modalities – touch, pen,.. Extensive end user studies to probe mental models to drive technology development (feedback) Other Disabilities – cognitive, motor impairment, etc. 36

37. Questions? Comments? Concerns? Suggestions? www.cs.sunysb.edu/~hearsay mailto:borodin@cs.sunysb.edu www.cs.sunysb.edu/~borodin