4th VRE Programme meeting July 2006, Manchester User engagement: five key challenges and strategies for addressing them Chris Ramsden and Kerstin Junge The Tavistock Institute VRE Formative Evaluation Team

4th VRE Programme meeting July 2006, Manchester We are going to talk about… Some key challenges for involving users in technology development activities Some strategies for addressing them that have worked in practice … and then: an exercise

4th VRE Programme meeting July 2006, Manchester Five key challenges of user engagement Identifying the end-users Recruiting end-users Securing commitment Getting the most out of interactions with users Capturing information and using results

4th VRE Programme meeting July 2006, Manchester Five key challenges of user engagement Identifying the end-user Moving from the envisaged user to the real end-user Developing an ongoing understanding of Who the technology is for How it fits in with peoples working practices and therefore What the purpose of the technology is Acknowledging that people shape technology The real user emerges from the project work

4th VRE Programme meeting July 2006, Manchester Five key challenges of user engagement Recruitment Methodological challenges: How many people do we need to give us meaningful data? Practical challenges: Locating the end-users to involve in the study Convincing users of the value of participating People are too busy, dont see the benefit of getting involved, are sceptical or dont care And: recruiting end-users is time-consuming It takes an average of 1.15 hours of staff time to recruit one participant (NNG, 2003)

4th VRE Programme meeting July 2006, Manchester Five key challenges of user engagement Securing commitment Translating agreement in principle into arrival on the day The drop-out rate for usability testing events is one in ten (NNG, 2003) Keeping people motivated throughout an iterative development cycle Continued involvement and longer term commitment can be problematic

4th VRE Programme meeting July 2006, Manchester Five key challenges of user engagement Getting the most out of interactions with users Knowing what to get out of user engagement How and when to extract appropriate data from users relevant to the project Deciding on the boundaries of user involvement Using interactions effectively Designing tasks and processes so that end-users provide relevant and meaningful information Effective facilitation of workshops

4th VRE Programme meeting July 2006, Manchester Five key challenges of user engagement Capturing information, using results Recording what people do, there and then Assessing the results… … and acting on them.

4th VRE Programme meeting July 2006, Manchester Strategies to address the key challenges Identifying and engaging the real end-user Developing an ongoing understanding of the user Acknowledging that people shape technology Involve diverse users recognising and testing the possibility of diversity, e.g.: Different levels of ICT skills Different roles Different working practices Different needs Engaging non-users and sceptics: their scepticism might be important for developing the technology If possible: understand how users operate in their real working environment to accommodate different working practices

4th VRE Programme meeting July 2006, Manchester Strategies to address the key challenges Recruitment How many users? Finding them Getting buy-in Between 5 and 20 Some end-users are better than none Tap into professional networks to recruit your users Use key contacts within organisations Using product champions to help with social marketing Unusual methods: Notes on University blackboards and discussion lists Specialist and non-specialist conferences? Classifieds Social marketing: Sell the value of the technology and the benefit to the individual

4th VRE Programme meeting July 2006, Manchester Strategies to address the key challenges Securing commitment No shows Long-term commitment Incentives Financial incentives Book tokens Gold pass for users to use the technology outside the session Creating a culture of involvement: Making users feel valued Showing them that their input makes a difference Giving them structured personal feedback ( , letter)

4th VRE Programme meeting July 2006, Manchester Strategies to address the key challenges Getting the most out of interactions with users Knowing what to get out of user engagement Effective interactions with users Straightforward structure and questions for the event Continuous data extraction throughout the session Having a clear framework for setting boundaries and understanding constraints Designing the right task Thinking about the context of use: real-life scenarios where people would use the technology Allowing the end-user to explore the technology in an unstructured way Avoid implicitly proposing a certain path Do a trial-run of usability event with at least two users Facilitation Structured process design of the event But: keep it interesting, use non-technical language where possible and let your end-users speak

4th VRE Programme meeting July 2006, Manchester Strategies to address the key challenges Capturing information, using results Recording what users do Analysing results Acting on them Take as many notes as possible during the event watching video footing / listening to tapes is time- consuming Have 2 or 3 members of your team present, and divide roles: e.g. facilitator, observer, note-taker Analyse and report on results as soon as possible, and share these results within the team Reflect on the event within the team: What went well, and why? What did not go so well, and why? What have we learnt for next time? Having a clear mechanism for making changes (timetable, deadlines, quality control)

4th VRE Programme meeting July 2006, Manchester Strategies of user engagement Summing up Viewing the user as a collaborator and source of expertise A technology of user involvement Taking risks

4th VRE Programme meeting July 2006, Manchester The exercise Discuss with the person sitting next to you: What was your most positive user engagement experience in the last months?

4th VRE Programme meeting July 2006, Manchester The exercise Discuss in groups of four: How could you use the learning from this experience to improve what your projects are currently doing to engage users?

4th VRE Programme meeting July 2006, Manchester The exercise For the whole group: On reflection of this exercise, as a community, what new things can we do with our users now and in future?