1. Design and Implementation of Online Braille Math Support in the Accessible Equation Editor Sam Dooley Pearson Assessment Dan Brown Pearson Assessment Susan Osterhaus Texas School for the Blind and Visually Impaired

2. 1Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math is hard!  Blind students need high-quality braille math  Advanced preparation is costly and takes time  Math teachers are often unfamiliar with braille  TVIs are often unfamiliar with math notation

3. 2Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math software is harder!  Software translation can be problematic  Forward translation takes many steps  Real-time back translation is non-existent  No support for online or classroom use

4. 3Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math needs to be online! Blind students need:  A level playing field for STEM instruction  To read and write online braille math  To interact with sighted instructors and peers  To participate in online activities

5. 4Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math should be math!  Math concepts are independent of notation  Braille math codes capture all math notation  Math software can be independent of notation Blind students only have full access to math if their math is treated the same as printed math.

6. 5Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Classroom challenges for blind math students  They cannot read math the teacher can write  They cannot write math the teacher can read  Speech solutions incur high cognitive load  They need to read and write for themselves

7. 6Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math translation  Visual math software is not truly accessible MS Word, MathType, Scientific Notebook  Braille math translation software  is not instantaneous  is not totally accurate LaTeX, Duxbury, HIMS, HumanWare

8. 7Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Online Braille Math with the Accessible Equation Editor  Sighted user can create math for a blind user  Blind user can create math for a sighted user  Real-time, two-way braille math translation  Instantaneous interactions with math content  Discoverable braille math encoding rules

9. 8Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Nemeth Braille (1952)  Abraham Nemeth (AFB, APH, BANA)  Tactile encoding for print math  Math for technical publications  Presentational math structures  Consistent with content markup

10. 9Design and Implementation of the Accessible Equation Editor 10 Apr 2016 TeX/LaTeX (1978/1985)  Donald Knuth, Leslie Lamport  Typesetting language for print math  Math for technical publications  Presentational math structures  Fully programmable macro language

11. 10Design and Implementation of the Accessible Equation Editor 10 Apr 2016 W3C MathML (1998)  World Wide Web Consortium (W3C)  XML element/attribute vocabulary  Typical K-12 and higher ed. math  Presentation and content markup

12. 11Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Presentation v. Content  Presentation encodes signs/symbols  Content encodes functional structure  "x^2+1" v. "1346, 45, 23, 5, 346, 2"  "x^2+1" v. (plus (power x 2) 1)

13. 12Design and Implementation of the Accessible Equation Editor 10 Apr 2016 QWERTY keyboard input  Math symbols on the keyboard  Other symbols on the buttons  Keyboard cursor navigation  Implicit and explicit selection  Backspace, delete, clear

14. 13Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille terminal output  Math output on the screen  Braille output on the screen  Braille output on the terminal  Screen reader support

15. 14Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille terminal input  Each braille key is treated like a keyboard key  Sequences of braille keys create the math  Button tooltips with text and braille  Input cursor position and routing

16. 15Design and Implementation of the Accessible Equation Editor 10 Apr 2016 QWERTY terminal output  Braille input creates math content  Math content becomes math output  Math output becomes braille output  The math markup is exactly the same

17. 16Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math is math!  Blind students can read the same math  Blind students can create the same math  The math can be shared the same way  The math can be scored the same way Blind students can now have full access to math since their math is the same as printed math.

18. 17Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Equation Editor Architecture  Syntax expression trees  Expression tree transformations  Configuration and customization  Universal design decision points

19. 18Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Expression trees  An operator with its child arguments  The operator as the initial child argument  Parse trees, syntax expressions, XML elements  Often annotated with semantic attributes  Functional programming languages (Lisp)

20. 19Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Content expressions  The functional structure of a math expression  Does not itself define any semantic behavior  The operators carry the semantic behaviors  Operator, child elements, child attributes  Direct translations to and from XML markup  Boxed reference to presentation expression

21. 20Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Presentation expressions  The visual notation for a math expression  Does not itself define functional structure  The operators carry the notational schema  Operator, child elements, child attributes  Direct translations to and from XML markup  Boxed reference to content expression

22. 21Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Expression references  Math document user interface elements  Content node matched with a presentation  Pointers to child expression references  References to the browser display nodes  Resolves from DOM to presentation to content

23. 22Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Browser display nodes  Browser document user interface elements  Direct references into the browser DOM tree  Attached pointers into content, presentation  Listeners for math content change events  Wrap nodes not owned by the equation editor

24. 23Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Expression trees are universal!  Generic encodings for math formulas  Separating notation from function  Separating interface from function  Enabling automated regression testing  Universal design for math behaviors

25. 24Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Transformation architecture  Input transformations create expressions  Output transformations create XML markup  Layouts transform content to presentation  Templates create new content from old  Generic rewrite rules on expression trees  Enabling data-driven math behaviors

26. 25Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Configuration architecture  External data defines conventional behaviors  Allows highly customizable math interactions  Fine control for solving math usability issues  Allows reconfiguration for new populations  Layouts, templates, buttons, palettes, keys

27. 26Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Layout transformations  Transform structure into notation  Structure encoded as Content MathML  Notation encoded as... pick your target!  Layout rules encoded as custom XML  Source pattern as Content MathML  Output pattern as layout schemata

28. 27Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Content to Presentation  Math operators determine layout rules  Minimal contextual dependencies  Presentation links back to content  (plus x 1) v. "x + 1"

29. 28Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Content to Braille  Math operators determine braille symbols  Additional contextual dependencies  Flattening to linear output format  (plus x 1) v. "1346, 346, 2" ⠬

30. 29Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Template transformations  Create new structure from old structure  Template rules encoded as custom XML  Template names and key event names  Source pattern as Content MathML  Output pattern as Content MathML  Template variables control rewriting

31. 30Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Keyboard to Content  Keys and button names select template rules  Minimal sequential dependencies  Presentation updated on each key  "x + 1" v. (plus x 1)

32. 31Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Braille to Content  Braille cells indirectly select template rules  Non-trivial sequential dependencies  Presentation updated on each key  "1346, 346, 2" v. (plus x 1)

33. 32Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille key transformations  Transform braille cells to template names  Braille cells are received as ASCII braille  Each braille cell assigned a symbolic name  Symbolic names for braille cell sequences  Braille input mode transforms to key names  Previous template rules operate unchanged

34. 33Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille cells to keys  Single cells that appear as keys  Single cells that cause actions  Sequences that cause actions  Sequences that extend others

35. 34Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille finite state machine  Symbolic names for braille prefixes  Symbolic names identify input states  Braille cells identify state transitions  Accepting input states invoke templates  Subsequent states may replace inputs

36. 35Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Universal design for online math  Expression trees provide universal data  Data-driven universal tree transformations  Flexible configuration customization rules  Universal presentation output notation  Universal input event transformations

37. 36Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Braille Input Examples  Numeric indicator  Baseline indicator  Type form indicators  Shape indicators  Negated operators  Composed relations

38. 37Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Braille Input Testing  Web browser input test page  JavaScript equation editor  Screen reader device drivers  Web accessibility APIs  Braille terminal input

39. 38Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Research Studies  Two research studies in Fall 2015 (KY/AZ - Sep, TX - Oct)  Goal - to collect feedback from multiple populations on EE functionality  Populations - blind, low vision, regular print readers, learning disabled  Criteria - high school students who had completed Algebra I

40. 39Content MathML Markup from Nemeth Braille Input 23 Mar 2016 Research Outcomes  Students had limited knowledge of Nemeth  Erasing math content was unpredictable  Working with grouping symbols was difficult  Entering and closing fractions was unexpected  Ending trigonometric expressions was unusual

41. 40Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Advanced preparation  Create math content in advance  Share with students by email  Include math content with slides  Students use the equation editor

42. 41Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Extemporaneous content  Teacher uses the equation editor  Projects the screen to the class  Blind student uses braille terminal  Everyone can read the same content

43. 42Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Shared presentations  WebEx, Google Hangouts, etc.  Shared screens for visual users  Individual work shown to the class  Blind students can show their work  Blind students viewing others work

44. 43Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Online assessments  Common Core, PARCC, TestNav  Spring 2015 admin (15m responses)  Machine scoring for constructed math  Blind students can use online forms  Blind students create online responses

45. 44Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Nemeth Braille curriculum  Interactive Nemeth braille exercises  Immediate practice for lessons learned  Experimentation with braille concepts  Discoverability for unfamiliar encodings

46. 45Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Homework assignments  Blind student uses the equation editor  Email homework files to sighted teacher  Teacher reads using the equation editor  Teacher marks up the math content  Returns marks to the blind student

47. 46Design and Implementation of the Accessible Equation Editor 10 Apr 2016 TVI support  The role of the TVI remains essential  The technical work load can be reduced  The software can help assist the process  Discoverability for print math notations

48. 47Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Math teachers  The role of the math teacher is vital  The technical work load can be reduced  Engaging directly with blind students  Discoverability for braille math encodings

49. 48Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Group interactions  Effective group work with blind students  Each student has an opportunity to contribute  Each can learn from the work of others  The software can help assist the process

50. 49Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Online Accessible Math! Blind students can have:  A level playing field for STEM instruction  Tools to read and write online braille math  Interaction with sighted instructors and peers  Active participation in online activities

51. 50Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Expression trees are universal!  Generic encodings for math formulas  Separating notation from function  Separating interface from function  Enabling automated regression testing  Universal design for math behaviors

52. 51Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Braille math is math!  Blind students can read the same math  Blind students can create the same math  The math can be shared the same way  The math can be scored the same way Blind students can now have full access to math since their math is the same as printed math.

53. 52Design and Implementation of the Accessible Equation Editor 10 Apr 2016 Accessible Equation Editor  Try it out, give us feedback http://accessibility.pearson.com/mathex-app/  Please let us know how you would use it sam.dooley@pearson.com dan.brown@pearson.com osterhauss@tsbvi.edu