Making a Computer Speak Algebra However You Want CEC 2013 Convention & Expo SAN ANTONIO, TEXAS April 5,

2 Presented by Susan A. Osterhaus Texas School for the Blind and Visually Impaired Outreach Programs 1100 West 45th Street Austin, TX

Presentation Overview Project Motivation, Goals, and Background Development to date: discussion and demonstration Research Studies Next Steps 3

Project Motivation Text-to-speech (TTS) vs. Math-to-speech Limited choice of syntax/semantics Limited screen-reader support Limited or no within-expression navigation Difficult to author Works in limited environments ETS Experience 4

Project Goals High school algebra Classroom-like synthesized speech “style”: ClearSpeak Speech is flexible so computer will speak the way the teacher wants Can be used in any math document – including both instruction and assessment Interactive navigation Author in Word with easy playback 5

Not Part of Project Braille Support for other formats Authoring by people with visual impairment Solving math problems Tutoring/Curriculum Integration with standardized testing 6

What is MathML? Mathematical Markup Language (MathML) is an application of XML for describing mathematical notations and capturing both its structure and content. It aims at integrating mathematical formulae into World Wide Web pages and other documents. It is a recommendation of the W3C math working group. From Wikipedia, the free encyclopediaXML mathematicalWorld Wide WebW3Cworking group MathML = Math Accessibility 7

Behind the Scene: MathML (What is MathML?) 8 x 3

Project Software for Math MathML Tools: Authoring: MathType+Word Audio rendering: MathPlayer Screen reader Window-Eyes (Word & IE) 9

Many ways to Speak Math How do you say… 10

Many ways to Speak Math How do you say… 11

Many ways to Speak Math How do you say… 12

Many ways to Speak Math How do you say… 13

Many ways to Speak Math How do you say… 14

Many ways to Speak Math How do you say… 15

How Math Speech Works Rules Preferences Exact Speech 16

ClearSpeak Many ways to speak math ClearSpeak design philosophy Other design philosophies MathSpeak SimpleSpeech 17

Demonstration Authoring Using MathType in Word to enter math and use preferences Entering exact speech in Word Playback Word with Window-Eyes 18

Project Goals Recap ClearSpeak Author in Word Allow setting preference and exact speech Playback in Word and IE Interactive navigation 19

Creating Expressions and Questions for Studies Level Degree of complexity Relevance of speech variations Provide useful data Test comprehension, not computation Guide decisions about speech rules, preferences 18

Feedback Study #1 Compared ClearSpeak, MathSpeak, SimpleSpeech Fractions, exponents, parentheses Focus: was expression understood? Research/Statistical consultation 16 HS students: blind or low vision 21

Results Summary ClearSpeak more understandable than Simple Speech or MathSpeak Students overall preferred ClearSpeak Students got more answers correct with ClearSpeak than with Simple Speech or MathSpeak 22

One Expression’s Results 23 P+ Very Familiar Somewhat Familiar Somewhat Unfamiliar Very Unfamiliar Simple Speech30817 MathSpeak31546 ClearSpeak*66810

24 ClearSpeakSimpleSpeakMathSpeak Very familiar48%24% Somewhat familiar50% 31% Not very familiar2%9%23% Very unfamiliar0%16%22% Very easy to understand52%17%16% Somewhat easy to understand33%37%38% Somewhat hard to understand13%24%23% Very hard to understand3%22% Very sure understood71%42%39% Somewhat sure understood24%37%33% Not sure understood5%7%11% Definitely did not understand0%14%17% Familiarity and Understanding

Feedback Study #2 Can Prosody help understanding? Pauses Pitch Rate Changes Volume Changes Rejected pitch and volume changes non-speech sounds not an option Focus on pauses, rate-change vs. start/end 25

Feedback #2 Example vs. With Pauses / rate change Expression 1 Expression 2 With “end root” Expression 1 Expression 2 26

Feedback #2 Another Example Nested Parentheses: Uniform Pauses Non-uniform pauses Non-uniform pauses and “First Paren” 27

Navigation Simple char by char mode for beginners Powerful navigation features for experts Read Describe Multiple ways of moving Tree Character, Placemarker, Semantic After tree move: read or describe 10 Placemarkers, 2 Cursors, Where am I? 28

Participation Opportunities Blind or Visually Impaired Students Teachers 29

Contact Information Lois Frankel: Neil Soiffer: Beth Brownstein: Susan Osterhaus: MathPlayer: The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324A to the Educational Testing Service. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. 30

Thank you for attending this session. 31