CSM18 Usability Engineering Evaluation: test the usability and functionality of an interactive system Goals of an evaluation • assess the extent of the system’s functionality • assess its usability - see 10 heuristics assess the effect & affect of the interface on the user • identify any specific problems with the system or with its use UniS Department of Computing Dr Terry Hinton 17 April, 2019

Evaluation Methods for Interactive Systems Analytical Methods Experimental Methods Observational Methods Query Methods UniS Department of Computing Dr Terry Hinton 17 April, 2019

Evaluation Methods for Interactive Systems Analytical Methods Predict performance based on a model e.g. analysis of cash dispenser based on number of key strokes required, time needed to press a key, time needed to think, time needed to react. Experimental Methods design experiments in the laboratory e.g. speed of recognition of key words dependent on font & colour UniS Department of Computing Dr Terry Hinton 17 April, 2019

Evaluation Methods for Interactive Systems Observational Methods - in the field Users: expert users typical users novice users Query Methods - survey opinions, attitudes, easy, enjoyable also consider contextual issues Interviews Questionnaires UniS Department of Computing Dr Terry Hinton 17 April, 2019

UniS Department of Computing Usability Usability defined: usability = efficiency + effectiveness + enjoyment Can’t compute a usability parameter - J Neilson proposed 10 Usability heuristics see later heuristics - set of rules for solving problems other than by an algorithm (Collins English Dictionary 2nd Ed.) UniS Department of Computing Dr Terry Hinton 17 April, 2019

Experimental methods - in the laboratory design an experiment for laboratory conditions make an hypothesis - testable select your subjects - sample size select the variables - change one at a time statistical measures - time, speed, number of events, number of errors (recoverable, fatal) UniS Department of Computing Dr Terry Hinton 17 April, 2019

Observational techniques - in the field Observe behaviour - arbitrary activity - set the tasks Task analysis Specify a set of tasks gives insight into usability Specify a goal gives insight into cognitive strategy used Record - actions, time, errors etc. UniS Department of Computing Dr Terry Hinton 17 April, 2019

Observational techniques - in the field Verbal Protocol - Think aloud Protocol analysis paper and pencil audio recording video recording computer logging user notebooks Automatic protocol analysis tools Post-event protocol - teach-back or Post-task “walkthroughs” UniS Department of Computing Dr Terry Hinton 17 April, 2019

Query techniques - Attitudinal Data Interviews (see page 35 et seq.) design an interview schedule Questionnaires general open-ended scalar multi-choice UniS Department of Computing Dr Terry Hinton 17 April, 2019

Planning an evaluation Factors to be considered in planning an evaluation purpose - who are the stake holders laboratory vs field studies qualitative vs quantitative measures information provided immediacy of response intrusiveness resources UniS Department of Computing Dr Terry Hinton 17 April, 2019

Ten usability heuristics by J Neilson Visibility of system status system should keep users informed about what is going on Match between system and the real world system should speak users’ language - words, phrases and concepts familiar to the user (rather than system-oriented terms) User control and freedom users often choose system functions by mistake - support undo/redo Consistency and standards follow platform conventions (users shouldn’t have to wonder whether diferent words, situations, or actions mean the same thing. UniS Department of Computing Dr Terry Hinton 17 April, 2019

Ten usability heuristics Error prevention better than error messages Recognition rather than recall make objects, actions, and options visible (users shouldn’t have to remember information) Flexibility and efficiency of use accelerators (unseen by novice users) my speed up interaction for expert users - system allows users to tailor frequent actions Aesthetic and minimalist design simplicity is beauty UniS Department of Computing Dr Terry Hinton 17 April, 2019

Ten usability heuristics Help users recognise, diagnose, and recover from errors Express error messages in plain language (no codes), indicate the problem, and suggest solution Help and documentation Ideally, its better if system can be used without documentation, Most often it is necessary to provide help and documentation. Such information should be easy to search, focused on the user’s task, list concrete steps to be carried out and not be too long. Examples are always helpful. UniS Department of Computing Dr Terry Hinton 17 April, 2019

Questionnaire Design (see page 35 et seq.) A simple checklist yes no Don’t’ know copy paste Example of 6-point rating scale (avoid a middle value) Very useful Of no use UniS Department of Computing Dr Terry Hinton 17 April, 2019

UniS Department of Computing Questionnaire Design An example of a Likert Scale strongly agree agree slightly agree neutral slightly disagree disagree strongly disagree An example of a semantic differential scale extremely slightly neutral slightly extremely easy difficult UniS Department of Computing Dr Terry Hinton 17 April, 2019

UniS Department of Computing Questionnaire Design An example of a ranked order question: Place the following commands in order of usefulness using the numbers 1 to 4, 1 being the most useful. copy paste group clear UniS Department of Computing Dr Terry Hinton 17 April, 2019

Evaluation in the Design Phase Participatory Design - user is involved in the whole design life cycle Number of methods to help convey information between user and designer brainstorming storyboarding workshops pencil & paper exercise role playing UniS Department of Computing Dr Terry Hinton 17 April, 2019

UniS Department of Computing Evaluating the design Cognitive walk through Heuristic evaluation Review based evaluation Model based evaluation UniS Department of Computing Dr Terry Hinton 17 April, 2019

UniS Department of Computing Choosing an evaluation method Ref: Dix, A., Finlay, J.,Abowd, G., Beale, R. (1994) UniS Department of Computing Dr Terry Hinton 17 April, 2019

Choosing an evaluation method UniS Department of Computing Dr Terry Hinton 17 April, 2019

Choosing an evaluation method UniS Department of Computing Dr Terry Hinton 17 April, 2019