Imager Laboratory for Graphics, Visualization, and HCI Designing ESI Planner: The Participatory Design of a Sound and Image Enhanced Planner for People with Aphasia Karyn Moffatt, Joanna McGrenere, Barbara Purves, Maria Klawe Department of Computer Science University of British Columbia Canada 1

Imager Laboratory for Graphics, Visualization, and HCI Aphasia is a loss of words – not intelligence Acquired language disorder Acquired language disorder Caused by brain damage (e.g. stroke, trauma, etc.) Caused by brain damage (e.g. stroke, trauma, etc.) Impairment of communication abilities Impairment of communication abilities Relative sparing of other cognitive abilities Relative sparing of other cognitive abilities What is Aphasia? A hp a a i s i Y a l Q n ssxo f o w r d T o y i t l g iir e nc e - 2

Imager Laboratory for Graphics, Visualization, and HCI Skip has Aphasia Video and name used with permission. 3

Imager Laboratory for Graphics, Visualization, and HCI Key Design Problems mismatch of individual abilities and skills required to use the technology mismatch of individual abilities and skills required to use the technology failure to address the spectrum of communication contexts failure to address the spectrum of communication contexts Sports Work Doctor SpotFamilyHi/Bye Hi, I have an appointment with Dr Smith New Back Home More 4

Imager Laboratory for Graphics, Visualization, and HCI Research Goals Understand the strategies used by individuals with aphasia to communicate and to carry out language-based tasks. Understand the strategies used by individuals with aphasia to communicate and to carry out language-based tasks. Seek opportunities where technology can support individuals in those activities. Seek opportunities where technology can support individuals in those activities. 5

Imager Laboratory for Graphics, Visualization, and HCI ESI Planner The Enhanced with Sound and Images Planner PDA application (iPaq from HP) PDA application (iPaq from HP) A computerized daily planner designed for people with aphasia A computerized daily planner designed for people with aphasia Uses images, sound, and text to represent people and places in appointments Uses images, sound, and text to represent people and places in appointments 6

Imager Laboratory for Graphics, Visualization, and HCI Design Approach Two phase design: Two phase design: Phase One: Participatory Design Phase One: Participatory Design Phase Two: Experimental Evaluation Phase Two: Experimental Evaluation 7

Imager Laboratory for Graphics, Visualization, and HCI Phase One: Participatory Design Four sub-phases: 8

Imager Laboratory for Graphics, Visualization, and HCI Phase One: Participatory Design Four sub-phases: 1. Idea brainstorming Interviews Interviews Identified needs: Identified needs: Daily planner Daily planner Recipe book Recipe book Word dictionary Word dictionary Personal history recorder Personal history recorder Conversation primer – to help preplan conversations Conversation primer – to help preplan conversations Photo used with permission 9

Imager Laboratory for Graphics, Visualization, and HCI Phase One: Participatory Design 1 - Drawn by hand2 - Using computer tools June: 9/6/03 7:00 am - 9:30 am Marilyn MonroeEiffel Tower 11 : : Starts Ends Four sub-phases: 1. Idea brainstorming 2. Low-fidelity paper prototyping 10

Imager Laboratory for Graphics, Visualization, and HCI Phase One: Participatory Design Four sub-phases: 1. Idea brainstorming 2. Low-fidelity paper prototyping 3. Medium-fidelity software prototyping 11

Imager Laboratory for Graphics, Visualization, and HCI Phase One: Participatory Design Four sub-phases: 1. Idea brainstorming 2. Low-fidelity paper prototyping 3. Medium-fidelity software prototyping 4. High-fidelity software prototyping and formal evaluation > Demo! 12

Imager Laboratory for Graphics, Visualization, and HCI ESI Planner Demo 13

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Evaluation Hypothesis: A planner using triplets of images text and sound would be more usable than a text-only planner. Hypothesis: A planner using triplets of images text and sound would be more usable than a text-only planner. vs. Text-Only: NESI Planner (The Not Enhanced with Sound and Images Planner) Images, Sound, and Text: ESI Planner 14

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Methodology 8 Participants: 8 Participants: 1 female, 7 male; ages: 47 to 86 1 female, 7 male; ages: 47 to 86 All at least one year post onset All at least one year post onset Participants mild moderate severe SpeechAuditionReadingWriting 15

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Methodology 8 Participants: 8 Participants: 1 female, 7 male; ages: 47 to 86 1 female, 7 male; ages: 47 to 86 All at least one year post onset All at least one year post onset Participants mild moderate severe SpeechAuditionReadingWriting 15

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Methodology 8 Participants: 8 Participants: 1 female, 7 male; ages: 47 to 86 1 female, 7 male; ages: 47 to 86 All at least one year post onset All at least one year post onset Participants mild moderate severe SpeechAuditionReadingWriting 15

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Methodology 8 Participants: 8 Participants: 1 female, 7 male; ages: 47 to 86 1 female, 7 male; ages: 47 to 86 All at least one year post onset All at least one year post onset Participants mild moderate severe SpeechAuditionReadingWriting 15

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Methodology Two Sessions: Two Sessions: 16

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Methodology Two Sessions: Two Sessions: Session 1: Planner Evaluation Session 1: Planner Evaluation 30 min with each planner 30 min with each planner 10 tasks per planner 10 tasks per planner retrieval, creation, modification retrieval, creation, modification presentation format: text and oral presentation format: text and oral Quantitative performance measures: Quantitative performance measures: Number of tasks completed correctly Number of tasks completed correctly Number of tasks completed Number of tasks completed Qualitative self-reported preferences Qualitative self-reported preferences 5:15pm End Time: 4:00pm Start Time: September 14, 2003 Date: When? Eiffel Tower Place:At Marilyn Monroe Person:With4A Create an appointment Example Task 16

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Methodology Two Sessions: Two Sessions: Session 1: Planner Evaluation Session 1: Planner Evaluation Session 2: Western Aphasia Battery Session 2: Western Aphasia Battery Standardized battery used to assess: Standardized battery used to assess: speech, auditory comprehension, reading, and writing speech, auditory comprehension, reading, and writing Used to describe abilities in terms of severity: Used to describe abilities in terms of severity: mild, moderate, or severe mild, moderate, or severe 16

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Results Qualitative Self Reported Measures Qualitative Self Reported Measures Preference varied with severity of impairment Preference varied with severity of impairment Participants mild moderate severe preference ESINESINESINESIESIESIESIESI SpeechAuditionReadingWriting More severely impaired More mildly impaired 17

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Results Qualitative Self Reported Measures Qualitative Self Reported Measures Preference varied with severity of impairment Preference varied with severity of impairment Participants mild moderate severe preference ESINESINESINESIESIESIESIESI SpeechAuditionReadingWriting More severely impaired More mildly impaired 17

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Results Qualitative Self Reported Measures Qualitative Self Reported Measures Preference varied with severity of impairment Preference varied with severity of impairment Participants mild moderate severe preference ESINESINESINESIESIESIESIESI SpeechAuditionReadingWriting More severely impaired More mildly impaired 17

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Results Quantitative Performance Measures: Tasks Correct Quantitative Performance Measures: Tasks Correct Significantly more tasks correct with ESI Planner Significantly more tasks correct with ESI Planner (F(1,7) = 27.00, p<.01) Averages: ESI Planner: 7.9 NESI Planner: Participants Total Tasks Correct ESINESI 18

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Results Quantitative Performance Measures: Tasks Correct Quantitative Performance Measures: Tasks Correct Significant interaction between tasks correct and interface order (F(1,7) = 8.33, p<.05) Significant interaction between tasks correct and interface order (F(1,7) = 8.33, p<.05) ESI NESI NESI ESI Trial 1 Trial 2 ESI-NESI NESI-ESI Average Tasks Correct 19

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Results Quantitative Performance Measures: Tasks Correct Quantitative Performance Measures: Tasks Correct Significant interaction between tasks correct and interface order (F(1,7) = 8.33, p<.05) Significant interaction between tasks correct and interface order (F(1,7) = 8.33, p<.05) ESI NESI NESI ESI Trial 1 Trial 2 ESI-NESI NESI-ESI Average Tasks Correct 19

Imager Laboratory for Graphics, Visualization, and HCI Phase Two: Results Quantitative Performance Measures: Tasks Correct Quantitative Performance Measures: Tasks Correct Significant interaction between tasks correct and interface order (F(1,7) = 8.33, p<.05) Significant interaction between tasks correct and interface order (F(1,7) = 8.33, p<.05) ESI NESI NESI ESI Trial 1 Trial 2 ESI-NESI NESI-ESI Average Tasks Correct 19

Imager Laboratory for Graphics, Visualization, and HCI Ongoing Research Additional Analysis Additional Analysis Patterns of Behavior Patterns of Behavior Longitudinal Field Study Longitudinal Field Study Adoptability vs. Usability Adoptability vs. Usability Explore Customization Explore Customization 20

Imager Laboratory for Graphics, Visualization, and HCI Research Outcomes ESI Planner: An application to support a specific daily living activity ESI Planner: An application to support a specific daily living activity Guidelines Guidelines For working with special populations For working with special populations For improving the accessibility of handheld technology For improving the accessibility of handheld technology 21

Imager Laboratory for Graphics, Visualization, and HCI Questions? Contact: Karyn Moffatt 22