Interaction Media & Communication, Department of Computer Science, Queen Mary University of London THE INFLUENCE OF SCREEN READERS ON WEB COGNITION Tony Stockman & Oussama Metatla Queen Mary, University of London

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London OVERVIEW Examine implications of SR technology for web cognition Report findings from survey and study of collaborative web use Propose draft taxonomy of errors in collaborative web interaction Examine potential role of non-speech sound in addressing identified problems

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London FEATURES OF SR WEB INTERACTION Default linear model of page presentation No ambient representation of location on page No representation of spatial layout No immediate indication of information density Little to assist formulation of mental model of page

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London CURRENT FUNCTIONALITY Focus: Jaws, WE and VO. Cursor key navigation – strength and weakness of analogy with other apps Other mechanisms: By listing of navigating forward/back between links, headers, frames, tables, forms, text elements, markers etc. Forms mode

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London OVERVIEWS Important but often under supported/neglected Typically lists no. of links, frames, headers, and forms with reminder of related hot keys

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London CURRENT NON-SPEECH SOUND Jaws schemes Window-eyes events VoiceOver defaults

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London VoiceOver Embedded in OS Factors in switching Group v. DOM mode navigation Overview followed by interaction model Non-speech sound more “visible”

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London IMPLICATIONS FOR COGNITION AND INTERACTION Hindered by linear navigation EG pageing through search results Hot keys and markers help, but ignore density of info and spatial layout - consequences for collaboration Overviews neglect spatial layout, ordering and esthetics, Tables are navigable but lack overviews

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London DESCRIBING WEB PAGES Employed widely known pages Characteristics of sighted descriptions: Relatively short but covering main features Column layouts, colours, mood, style, pictures, emotional response to message Characteristics of VI descriptions: Longer, more factual and granular, more focus on function and usability Conclusion: the two groups approach web tasks from widely differing contexts

Interaction Media & Communication, Department of Computer Science, Queen Mary University of London COLLABORATIVE TASKS Involved pairs of sighted and visually impaired users Both could read the web pages used One gave instructions while the other performed the task The tasks involved: Simple information searches Comparisons of data values Navigating pages and filling forms

VISUALLY IMPAIRED INSTRUCTOR These tasks were performed generally quite straightforwardly VI user was generally familiar with sites Sighted users perspective generally compensated for difference in views of each user Sources of problems: Screen-reader’s focus unavailable to sighted user Sighted user referring to spatial layout unavailable to screen- reader Interaction Media & Communication, Department of Computer Science, Queen Mary University of London

SIGHTED INSTRUCTOR Substantially more problems: Point in task unclear because screen-reader focus unavailable to sighted user Sighted references to spatial layout CAPTCHEs Non-standard form controls Column headers not spoken on forms Dynamic updating of form fields Interaction Media & Communication, Department of Computer Science, Queen Mary University of London

TOWARDS A TAXONOMY OF COLLABORATIVE ERROR 1 Location disconnects Layout disconnects Missing objects Navigation disconnects Contextual disconnects Interaction Media & Communication, Department of Computer Science, Queen Mary University of London

TOWARDS A TAXONOMY 2 Affordance disconnects Modal disconnects Hollistic disconnects Multi-focus disconnects Esthetics disconnects Interaction Media & Communication, Department of Computer Science, Queen Mary University of London

NON-SPEECH AUDIO Audio is inexpensive and widely used Screen-readers only gradually adopting limited non-speech sound, eg forms and progress bars Growing body of knowledge on how to design and use ( Range of techniques that could be examined as part or whole solutions to problems described Interaction Media & Communication, Department of Computer Science, Queen Mary University of London

Monitoring for dynamic changes The structure of Earcons could for EG reflect object type and nature of update Ambient sound might convey esthetics and/or interaction mode Auditory icons might signal affordance open/closed Spatial sound might convey overall layout, density, locations of users NON-SPEECH AUDIO 1 Interaction Media & Communication, Department of Computer Science, Queen Mary University of London

NON-SPEECH AUDIO 2 Example of non-speech auditory overviews compared with speech, more like a glance Spearcons for typical radio button options Crucial to avoid auditory overload, masking etc. Interaction Media & Communication, Department of Computer Science, Queen Mary University of London

CONCLUSIONS Speech-only model struggles to convey rich web content and enable increasingly complex interactions This is highlighted by gap in first impressions of common web pages Cross modal web collaboration os subject to a range of disconnects due to differences in presentation and interaction Non-speech audio is an under-used mechanism that, with careful design, could help to address some of the issues sited Interaction Media & Communication, Department of Computer Science, Queen Mary University of London