1. Case study - usability evaluation Howell Istance

2. SOFT3057 - Interactive Systems ELISE II - Web- based Image Retrieval System n This allows users to enter search terms which relate to descriptions of images stored in image collections e.g. ‘sunflower’

3. SOFT3057 - Interactive Systems

8. Methods proposed for usability evaluation of ELISE II n Structured expert reviewing u Use of cut-down version of cognitive walkthrough technique n Observation of users completing standard tasks u Studied via a number of user trials n Extended in-situ testing u Use of ELISE II in normal work during 1 month u Participant keeps diary of system usage

9. SOFT3057 - Interactive Systems User trials – pilot testing n Used 2 graphics designers n Insufficient scope in the tasks used to test the advanced search and the browse facilities u Requires additional tasks which require these to be used

10. SOFT3057 - Interactive Systems Participants in evaluation trials n Limerick 5 librarians, 1 student n ULB 10 librarians n DMU 2 graphics designers (pilot), 2 lecturers, 2 research students n V&A 1 researcher, 2 cataloguers, 2 curators n Tilburg 1 librarian, 2 cataloguers, 1 manager

11. SOFT3057 - Interactive Systems Procedures for user trials n introduction to the trial and its purpose n initial interview to establish background and nature of image use n initial exploration phase n formal explanation of the functions n search for last image actually used n Searches for 3 sample images n post-trial interview

12. SOFT3057 - Interactive Systems Macro Usability Issues n How well does the system support the user in formulating a query? n Are the retrieval results presented in such a way that enables the user to comprehend the relevance of the results to his search needs? n Is the user supported in making advanced queries? n Can the image retrieval system be integrated into the working environment of the user?

13. SOFT3057 - Interactive Systems Search strategies n Most – began with restrictive search (several terms and boolean operators), removed terms successively of increase number of hits, only if no hits found then different generic term used n Few – began with generic term and added terms to reduce hits n Very few – used browse facility to check search term n Many tried to minimise numbers of record retrieval in view of long wait

14. SOFT3057 - Interactive Systems Support in formulating query n Level of information about individual collections, type of images and search terms n Truncation not supported n Language of collections not clear n Order of precedence in Boolean expression evaluation n collections visible to user or one large virtual collection?

15. SOFT3057 - Interactive Systems Assessing relevance of search results n Order of image retrieval unclear n Detail of retrieved images n Local copies of image records n Decision to buy image records

16. SOFT3057 - Interactive Systems

17. Support for Advanced search n Understanding meaning of index fields n Information about individual collections n Complexity of search expressions

18. SOFT3057 - Interactive Systems Integration with user’s normal working environment n Speed of image retrieval and system reliability n Saving retrieved images n General usefulness

19. SOFT3057 - Interactive Systems Micro usability issues n Consistency n Appropriate user support n Minimal user input n Reduce user memory load n Feedback n Flexibility n Navigational support

20. SOFT3057 - Interactive Systems Navigation n Mechanisms inconsistent n For example.. u No obvious means of returning to the simple search page from the search results page u Clicking on buttons containing the numbers of hits to view the record list is not obvious n Requires consistent mechanism to move query – search – view sequence, n Consistency in object used, in labelling, in location

21. SOFT3057 - Interactive Systems System feedback n System messages are u too small in size u appear in a non-prominent part of the page u language and content is often very obscure n e.g ERROR: HTTP return error – Broker not found=?HTTP/1.1 500 Internal Server error.. n empty return – contact the administrator - meaningless to most users

22. SOFT3057 - Interactive Systems Login Facility n Location of command not prominent n Feedback about outcome (success or failure) is poor n Dialogue box persists on-screen with no visible change during login attempt n User encouraged to quit rather than logging off n Have separate login page

23. SOFT3057 - Interactive Systems