2. Understanding methods in a ‘system of usability practice’ Dominic Furniss PhD Supervisors: Ann Blandford & Paul Curzon

3. 2 Contents The problem Overview of the research process Introduction to accident models and RE Details of a ‘system of usability practice’ Validation Conclusion and the bigger picture

4. 3 Why I chose the problem and why it’s interesting We don’t know enough about usability methods in practice… 1. Many methods do not transfer from research to practice …we need to know more about our users (practitioners), tasks and their contexts to do better applied research… 2. Literature on methods is criticised as being irrelevant to practitioners …we need to know more grounded in practitioner perspectives… 3. Research is moving out from the technical capability of methods, to their use, to their use in practice, to wider factors that impact on the performance of practice. …we need to know how methods integrate with the wider system.

5. 4 Overview of the Research Process 1.Literature review: composition of usability practice knowledge. PITC. 2.Grounded theory: exploratory, engages with practitioner perspectives. Bottom-up mode 3.Website domain analysis (9 interviews) 4.Safety domain analysis (13 interviews) Top-down mode 5.Distributed Cognition perspective 6.Resilience Engineering perspective

6. 5 Accident models and RE Sequential Epidemiological The alarm for reactor signals Operator turns off alarm The reactor overheats ‘Human error’ Design – too many alarms Training – poor Procedures – not followed Management – weak Safety culture – weak Latent conditions

7. 6 Accident models and RE Systemic Non-sequential Events Come together by Bad luck -Murphy’s Law “This cannot be happening to me”

8. 7 Accident models and RE Playing the devil’s advocate: 3 reasons why practitioners don’t use methods (Senior HCI lecturer, 2005, personal communication)

9. 8 Accident models and RE Transferring HCI Modelling Techniques and Design Techniques to Practitioners (Buckingham Shum & Hammond, 1994) Prerequisite gulf Cost gulf Payback gulf Consultancy gulf Organisational factors

10. 9 Resilience Engineering account. Highlights: 1.Methods are coupled with project structures and processes 2.Methods should provide leverage for understanding the project/domain issues 3.Methods should support rapport, be persuasive, and lead to the development of the practitioner’s reputation 4.Methods are coupled with staff development and management 5.The development of tools, methods, and communication practices affect practice 6.Methods should support documentation where appropriate

11. 10 FRAM: Overall system

12. 11 FRAM 1: Project process

13. 12 FRAM 2: HF/usability practitioner understanding

14. 13 FRAM 3: Persuasion, rapport and reputation

15. 14 FRAM 4: Staff development and management

16. 15 FRAM 5: Tools, methods and reporting practices

17. 16 FRAM 6: Auditing and documentation

18. 17 FRAM: Overall system

19. 18 Validation Constant comparative method as part of GT Internal member checking of quotations Internal (10) and external (8) member checking of the model Model vs instantiations Scoping model in terms of detail and complexity Some parts are not unique to HF/usability Differences in tone: facilitating and helping vs persuading and selling Triangulation People, projects and domains Literature; e.g.: DC, RE, and Fitzgerald (1996) in IS development Corroborating and shaping usability practice research

20. 19 Key Contribution to Knowledge A systems-thinking perspective Methods should be understood within a theory of system performance i.e. how they affect, and are affected by, wider aspects of HF/usability practice. In practice methods are simultaneously coupled to the… … technical substance of project … communication of results, issues and reporting … client resources, budgets, and structure … capabilities of the practitioners involved … tools – to enhance and extend capability … rapport, relationships and reputations … creation of documentation where appropriate In practice methods … should not be thought of as tools merely to find problems … should not be thought of in terms of solely how they are prescribed A’ A

21. 20 The Bigger Picture CommunityMessage HCI researchers developing methodsMethods should take advantage of couplings Methods should be reported ‘adaptably’ HF/usability practitionersFRAM network can be used as a reflective tool for fine tuning and diagnosing issues in practice. Gives the ‘bigger picture’. HCI educators and MSc studentsMethods should not be limited to their common prescriptions but understood within a system of practice, e.g. via the functional network. Students should more easily see themselves as a resource for effective usability practice.

22. 21 FRAM: Overall system

23. 22 Thank-you