Usability Evaluation Evaluation should occur continually through the design and implementation process. Evaluation methods are applied as the interface is being developed, rather than at the end.

Usability Evaluation Evaluation (Preece et al), is concerned with gathering data about the usability of a design or product by a specific group of users for a particular activity within a specified environment or work context. Evaluate the design in terms of its usability (the efficiency, effectiveness and satisfaction)

Usability Evaluation "When the cook tastes the soup, that's formative; when the guests taste the soup, that's summative." – Robert Stakes Formative Evaluation Designs evaluated as you go along Evaluation of the interaction design to find usability problems to fix to improve usability (part of formation of design) Early and continual Should have something to evaluate no later than 10% into a project schedule Carefully selected participants

Usability Evaluation Summative Evaluation Conducting full evaluation at end of design Evaluation of interaction design to assess level of usability (as a summation of achievement) Often used for comparison with previous version, or with another system Randomly selected participants

Usability Evaluation Conducting the Formative Evaluation Before evaluation session Identify the usability properties to be measured Make ready the prototype to be used for the evaluation Design the experiment During evaluation session Run the tests and collect data After evaluation session Analyse the data to establish how the conditions of testing have affected the performance of activities Draw conclusions about any improvements needed in the design

Usability Evaluation Conducting the Formative Evaluation Example from Prof. H. Rex Hartson, Computer Science, Virginia Technology.

Usability Evaluation Qualitative Formative Evaluation: Subjective Qualitative Techniques: Introspection, Direct Observation, Direct Observation via ‘Think Aloud’ Protocols, Query Techniques, Interviews, Questionnaires/Surveys See Saul Greenberg presentation slides

Usability Evaluation Conducting the Formative Evaluation Participants (the subjects/users) must be treated with respect ethical rules of behaviour by Saul Greenberg

Usability Evaluation Quantitative Formative Evaluation: Objective Techniques: Controlled Experiments Determine a causal relationship between interface characteristic and user performance/behaviour Method is complex Statistical analysis of results can be complex See Alan Dix: example of method and statistical analysis

Usability Evaluation Results from Tests: What we hope for From Stephen Brewster

Usability Evaluation Results from Tests: What we hope to avoid From Stephen Brewster