How can we speak Math? Richard Fateman Computer Science Univ. Calif, Berkeley The Evolution of Mathematical Communication in the Age of Digital Libraries December, 2006

Humans speaking math into computers A primary input method Correction of errors (secondary) Simultaneous with handwriting, supportive of ambiguity resolution, error correction

The reverse: computers speaking math aloud AsTeR (T.V. Raman) – for blind readers Display-free communication (e.g. telephone) Simultaneous with handwriting, supportive of ambiguity resolution, error correction

Ambiguity and Expectation minus b plus or minus square root of b squared minus 4 a c divided by 2 a . Without prior knowledge of this formula how could you know if the 4ac or even the 2a belongs within the square-root? Could it be -b± b ^2 -4ac/2a ? In Lisp, ambiguity is removed with one mechanism, parentheses/prefix (/ (+ (- b) (sqrt (- (^ b 2) (* 4 a c))) (* 2 a)) (/ (+ (- b) (-(sqrt (- (^ b 2) (* 4 a c)))) (* 2 a))

Digression on math input/output Written Math Output TeX, other typesetting systems Interactive systems with selection MathML, other “notations” Not entirely solved but good enough for most work Written Math Input Traditional keyboard Menu selection Handwriting Mixture of above

Speaking math into computers Why bother? Speed. Compare “bold italic gamma” γ Convenience: Write with hand or mouse, modify with voice. Correction. Markup. Keyboard-impaired users Note: there is a trivial non-solution. Spell out G-A-M-M-A

Pick a subset of the problem. Numbers are easy, right? 1/10, 9/10/ 10/11 14/100, 14/10000 3/100, 300 34/100, 30/400 1-3, 1 2 3. 7-4-6, 7-4-1776 Pick a number from 1, 2, 10 .

OK, we can delimit the speaker’s flexibility on numbers. Then what? Still problems Homophones (sine, sin, sign) Near Homophones (p,b,t) Learning discrimination of ambiguous math a(b+c) f(b+c) Bracketing []{}(), “The quantity … end quantity”

Looks tough We believe it is not as tough as handwriting. Grammar-based tools are part of advanced speech recognition. No special hardware (maybe cheap microphone, already used for Skype etc) Combined with selection, menu, keyboarding, looks promising.

What next Continued development of tools and interfaces (Lisp, .net) Relying on technology from other domains (fun, frustration) Integration with Computer Algebra Systems, Math Browsers, Education projects, Disabled-access