Diagrammar Use Cases: A Framework for Making Images and Graphics Accessible 2016 CSUN Conference March 25, 2016

Page 2 Our Panelists Sue-Ann Ma (Moderator) Product Manager, DIAGRAM Center, Benetech Markku Hakkinen Managing Sr. Research Developer, Educational Testing Service Becky Gibson Sr. Technical Staff Member, IBM Emerging Technologies Group Charles La Pierre Technical Lead, DIAGRAM and Born Accessible, Benetech

Page 3 Options for making images & graphics accessible [Tactile Graphic][Sonification] [MusicXML] A diagram shows properties for the element Chlorine from the periodic table of elements. Atomic number, 17. Element symbol, Cl. Element name, Chlorine. Atomic mass, [Text Description] [3D] [Haptic] [MathML]

Page 4 Diagrammar: a framework for including options ● One sample framework developed through the DIAGRAM Center ● Makes it possible to associate several alternatives to a single image or diagram: –Summary description –Simplified language description –Annotations –Target age/grade range –Tactile source –Tactile tour

Page 5 Exploring applications for Diagrammar ● Questions we’ve started addressing: –What types of alternatives should we provide? –Do people really want options? –How do we expose these alternatives to users? –Will people be able to find the options on their own? –How do we get reading systems to support these options? –Are there applications beyond education? –How do preferences and needs differ by user (e.g. end-user vs. educator or facilitator)?

Page 6 ETS Research ● Explore innovative applications of technical standards for improving access to alternatives for the simulations (long descriptions, short descriptions, tactile graphics). ● Key standards: –Diagrammar (DIAGRAM Center) – W3C Web Components

Page 7 Requirements ● Need for a more seamless experience when using alternatives for simulations –Alert students when alternatives for simulations are available –Help students orient themselves to the alternatives –Include all needed resources, to the extent feasible ● Move beyond one-off solutions –Standards-based solutions to allow other designers and developers to be able to replicate the solution

Page 8 Initial Proof of Concept ● Diagrammar focus on authoring, but what about delivery? ● No clear standard for presenting multiple alternatives on the Web, and what exists has limitations –Alt –Long description –Aria-describedBy ● Could we bring Diagrammar over to the Web? ● We created a prototype Web Component called which replicated the Diagrammar structure in HTML, providing both default and scriptable interfaces accessing alternatives

Page 9 Prototype of the Initial Proof of Concept

Page 10 Applying Diagrammer to a ScienceTask ● Water Condensation Task ● Includes Simulation and Static Images ● Updated Web Component Model ● Provided: –Short Descriptions –Long Descriptions –Embossed Tactiles (with Tour) ● Conducted small scale usability study (reported by Hansen, et al here at CSUN 2016)

Page 11 The ScienceTask Simulation

Page 12 Simulation showing the long description

Page 13 Short description

Page 14 Tour of tactile graphics

Page 15 Why Did IBM Chose EPUB? Focused on the design and delivery of human-centric solutions that personalize the user experience on any device so everyone has equal access to digital knowledge.

Page 16 Customers want mobile friendly documentation served up in the Cloud

Page 17 IBM Needs Standards-based, Accessible, Adaptable, & Globally Enabled Docs

Page 18 Leverage IBM’s Contributions to Accessibility & Open Web Platform WAI-ARIA

Page 19 IBM Needs an Accessible, Open, Multi- platform Reading System ● Deliver documentation on multiple platforms ● Unified reader so not dependent upon varying feature sets ● Deliver documentation via the cloud ● Need support for MathML ● Must be accessible ● Support multiple renditions and user preferences

Page 20 Readium.org ● Open Source Technology for EPUB 3 and the Open Web Platform ● Develops technology to accelerate the adoption of EPUB 3 by the global digital publishing industry ● Non-profit membership organization ● Formed in February, 2013 ● Hosted at github.com/readiumgithub.com/readium

Page 21 Readium Features ● Support for EPUB 2.0 and 3.0 Rendering ● Media Overlays ● MathML (via MathJax) ● EPUB Reading System Object ● Obfuscated font support ● Accessibility ● Plug-in architecture ● Digital Rights Management (DRM) in progress

Page 22 Try it Yourself! ● Live Readium Demo Live Readium Demo ● Chrome Extension at Google Web Store Chrome Extension at Google Web Store ● Becky’s Captioned Demo Video Becky’s Captioned Demo Video

Page 23 Diagrammar ● Prototypes with Different Interfaces: –HTML Prototype with content inlineHTML Prototype –Readium Prototype with content on auxiliary pagesReadium Prototype ● DIAGRAM Center – Focus Group Test Results –Testing: UI/UX preferences for surfacing alternatives –Feedback

Page 24 Implementation: How it Works ● Web Components vs. aria-details or something else? ● Aria-details: ARIA 1.1 new extension –A common use for aria-details is in digital publishing where an extended description needs to be conveyed in a book that requires structural markup or the embedding of other technology to provide illustrative content. ● Caution: customization vs. standards (Personalization Profile)

Page 25 Future of Diagrammar and Augmented Description ● Building more multi-modal frameworks: –Math equation: text description, MathML, LaTex, braille, discipline –Math diagrams: text description, haptics, 2D tactile, 3D model –Math graphs: text description, haptics, sonification, 2D tactile ● Personalization ● Mainstream adoption – assessments, corporate and more…

Page 26 Questions?

Page 27 For More Information Resources: ● DIAGRAM Center Research: ● Readium: Us:

Page 28 Thank you for attending!