1. Math-Puzzle: Equation Tutor for Sighted and Visually-Impaired Children Jarno Jokinen Department of Computer Sciences University of Tampere Finland jarno.jokinen@uta.fi April, 2005 AAFG 2005

2. Math-Puzzle J. A. Jokinen2 / 1518. 04. 2005 communication of mathematics is usually visual formulas, diagrams, graphs etc. it is very difficult for blind and partially sighted people / students to do mathematics and is one of the biggest obstacles for them in school and at the university introduction

3. Math-Puzzle J. A. Jokinen3 / 1518. 04. 2005 most of the work reported in developing techniques that deal with mathematics can be presented through next categories: tactile as in Braille and other raised representations audio aids that read equations with tools to help in reading process tonal representing equations and graphs (sonification / audification) haptic or forced feedback devices represent shapes and curves integrated /multimodal approaches

4. Introduction into the games Math-Puzzle J. A. Jokinen4 / 1518. 04. 2005 http://www.boowakwala.com/kids/math-game-kids.html http://www.learn4good.com/games/kids/double_digits.htm

5. Math-Puzzle J. A. Jokinen5 / 1518. 04. 2005 http://www.gamealbum.com/keyword/math/

6. Math-Puzzle J. A. Jokinen6 / 1518. 04. 2005 project impetus created a game instead of making tests with an existing game mathematical game for blind children after different ideas, Math-Puzzle was chosen training in logics, memory and equations’ manipulation strategy

7. the problems in question is it possible to solve equations using only short speech cues? what are the limitations? what is the easy way for blind interaction to edit the equations in static or dynamical puzzle? (memory capacity or external memory aid see next slide) how long does it take to solve a puzzle? what are the parameters of the gameplay progress and player performance? how much does it make difference when player has visual feedback or all the tasks (including navigation within the game-field) are presented through sounds? how and in which order the equations could be solved? (strategies) Math-Puzzle J. A. Jokinen7 / 1518. 04. 2005

8. Math-Puzzle J. A. Jokinen8 / 1518. 04. 2005 different matrix approaches static puzzle on the left dynamic puzzle on the right

9. game concept 5x5 matrix with one equation in each row equations are predefined and randomly picked for the matrix the goal of the game is to change the places of the equation members so that all of the equations are true the figures (only in the same column) can be swapped by clicking one number and then the other one operators cannot be swapped as well other controls are: alt+R -> new game (reset) alt+M -> minimize / maximize browser window alt+S -> show / hide figures and operators the puzzle completion time and the number of moves are calculated the game is currently implemented only in www with limitations Math-Puzzle J. A. Jokinen9 / 1518. 04. 2005

10. testing procedure 5 technically aware adults (age ranged from 26 to 34) 10 games per player with all three playing modes: 1 st visual, 2 nd blind (hidden labels with sound cues), 3 rd blindfolded laptop pc with external mouse and headphones Math-Puzzle J. A. Jokinen10 / 1518. 04. 2005

11. problems sound feedback is not supported with mozilla browsers completing the fourth equation usually completes also the fifth if not, the answers are crossing each other in a way that makes it very difficult to solve the puzzle the program sometimes gives 1 or more correct equations at the start which may also result in an error on the browser Math-Puzzle J. A. Jokinen11 / 1518. 04. 2005

12. results and discussions the game (blind mode) is hard for adults that can see what about kids that can’t? They have better spatial understanding some matrixes can be a lot faster solved than others – repetitions needed for a good estimate of skills if you wan’t to hear the sound cues you need to be patient sound is heard when the mouse is moved over the square, so if you want to hear it again you must move the mouse away and over again completing the fourth equation is crucial as explained on next slide Math-Puzzle J. A. Jokinen12 / 1518. 04. 2005

13. about the fourth equation Math-Puzzle J. A. Jokinen13 / 1518. 04. 2005 OK

14. conclusions game requires a lot of memorizing, but there are strategies that help completing equations in some order using mathematical rules (division and multiplication) when playing the game blindfolded, the increase in speed was bigger than when playing the visual version Math-Puzzle J. A. Jokinen14 / 1518. 04. 2005

15. Math-Puzzle J. A. Jokinen15 / 1518. 04. 2005 References http://www.boowakwala.com/kids/math-game-kids.html http://www.learn4good.com/games/kids/double_digits.htm http://www.gamealbum.com/keyword/math/ Children’s math project http://www.udel.edu/educ/cmp2/http://www.udel.edu/educ/cmp2/ http://www.educational-software-directory.net/math/ Lambda-project: Linear Access to Mathematic for Braille Device and Audio-synthesis http://www.lambdaproject.org/ http://www.lambdaproject.org/ Karshmer, A.I., Gupta, G., Gillan, D. Architecting an Auditory Browser for Navigating Mathematical Expressions, ICCHP 2002, LNCS 2398, p. 477 http://link.springer.de/link/service/series/0558/papers/2398/23980477.pdf Gaura, P. REMathEx - Reader and Editor of the Mathematical Expressions for Blind Students, 2002, LNCS 2398, p. 486, http://link.springer-ny.com/link/service/series/0558/papers/2398/23980486.pdfhttp://link.springer-ny.com/link/service/series/0558/papers/2398/23980486.pdf Fitzpatrick D. Speaking Technical Documents: Using Prosody to Convey Textual and Mathematical Material, ICCHP 2002, LNCS 2398, p. 494, http://link.springer.de/link/service/series/0558/papers/2398/23980494.pdf http://link.springer.de/link/service/series/0558/papers/2398/23980494.pdf http://www.computing.dcu.ie/~dfitzpat/publications.html Math project, http://www.cs.york.ac.uk/maths/index.htmlhttp://www.cs.york.ac.uk/maths/index.html Prosody in Mathtalk http://www.cs.york.ac.uk/maths/robert/prosody.htmlhttp://www.cs.york.ac.uk/maths/robert/prosody.html Mathematical Access for Technology and Science, http://www.papenmeier.de/reha/research/mathe.htm http://www.papenmeier.de/reha/research/mathe.htm Edwards, A. D. N., Stevens, R. D. and Pitt, I. J. Représentation non visuelle des mathématiques, (translated by A. Assimacopoulos) in A. B. Safran and A. Assimacopoulos (editors) Le Déficit Visuel, Éditions Masson, pp. 169–178 (1995), http://www.cs.york.ac.uk/ftpdir/pub/alistair/publications/ps/geneva.ps http://www.cs.york.ac.uk/ftpdir/pub/alistair/publications/ps/geneva.ps Karshmer, A.I., Gupta, G., Geiger, S., and Weaver, C.: Reading and Writing Mathematics: The MAVIS Project, BIT (Behaviour & Information Technology), January 1999