Designing a Handwriting Recognition Based Writing Environment J C Read, S J MacFarlane, C Casey Department of Computing, University of Central Lancashire, Preston, UK

Introduction Background Information Description of the Observational Study Findings from the study General Findings Findings relating to the Handwriting Recognition Satisfaction measuring Informing the design Description of the Prototype Conclusions

Who? Why? What? Where? How? Janet C Read, lecturer and mother! Elodie, PhD study Speech, handwriting: - Human Language Technology, Free text not command. Lancs..UK; white rural primary – age 7 – 9 Research, Observations, Usability studies

Previous Work QWERTY keyboard difficult (WES2000) HLT attractive to children, HR feasible (HCI2000, HCI2001) Measuring Fun (CandF2000, CandF2001) Participatory design (IDC2002) Errors in HR interfaces (NordiCHI2002)

The Observational Study Children aged 7 and 8 Normal classroom activities In twos Laptop (HR), Desktop (QWERTY), Desk (Pencil) Different writing tasks Difficulties, Errors, Corrections

Pen and Paper Errors Errors made – missing words, spelling, letters written backwards Error prevention – asked, avoided, looked Error discovery – reading back, self, teacher or another child Error repair – rub out, scribble out, cross out, overwrite, re-write, squeeze in, change

QWERTY - Errors Errors made – missing words, spelling, hit wrong key Error prevention – asked, avoided, looked Error discovery – reading back, self, teacher or another child, wiggly lines! Error repair – position and rub out, rub back to, rub all, retype, change

Handwriting Recognition - Overview Hardware – Graphics tablet and pen Software – Recognition software Fuzzy computing Disobedient – ambiguous Character or word based On line – ‘t’ stroke problems Demonstration

Demonstration of handwriting

Handwriting Recognition – Errors (1) Child Errors made – miss words, spellings, letters backwards, pen up Computer Errors made – Bad recognition, hardware Child Error prevention – ask, avoid, look Computer Error prevention – spell checker (not used)

Handwriting Recognition – Errors (2) Child Error discovery Before Recognition – reading back, teacher, other child After Recognition – as above + wiggly lines! Error repair Before recognition – scribble out, overwrite, insert letter After Recognition – rub back to, rub all, rewrite all or some, use QWERTY

Satisfaction Measuring Errors do not imply dissatisfaction WHY? Sticky – addictive vs. nothing better Funny – humour with recognition – easy to use

Designing a Prototype - method Users Children, environment, characteristics, mental models Tasks Goal oriented – hierarchy System States – dangerous states Interface UI design guidelines

Child Classroom based – standard equipment, needs to be easy to use, robust, minimal help needed Children – varied pen control, different levels of expertise with technology, different reading skills, poor or very good letter formation Mental model – see tablet as paper – want to scribble out and insert missed words

Child writing

User Goal To produce good written work Planning Translation Reviewing and Editing (Hayes and Flower)

Supporting the writer (1) Ideas – pop up in clouds, can have many, child can re-order them and can put them away, use handwriting that is not recognised Translation – training supported, lines can be drawn on screen or on the tablet (or both!); recognition can be immediate or delayed;

Supporting the writer (2) Reviewing – computer can read back recognised text, child can read recognised or script text; spellings may be highlighted in recognised text – teacher controls Editing – child can edit with rubber and pen on script, or with keyboard on recognised text

System States Entry state Recognition state Edit state DANGEROUS STATES Pens that point Cursors that confound Spaces that stop

Interface Design (1) Full writing screen Ability to place new pages Menus at the bottom Haptic boundary preferred Tablet matched to screen Pen can be turned on and off

Interface design (2) Video clip facility Teachers screen Assistant Customisable Training activities

And so…………… The designs for a product for a small group of users, for a narrow application Keyboard interface Error repair Speech recognition

Thank you Janet C Read University of Central Lancashire Preston Up North! England