IS4445 Principles of Interaction Design Lecture 10: Testing cards Rob Gleasure R.Gleasure@ucc.ie www.robgleasure.com

Course structure Or more specifically Week 1: Introduction Week 2: Empathise 1 (personas) Week 3: Empathise 2 (empathy maps) Week 4: Define 1 (journey maps) Week 5: Define 2 (value curves) Week 6: Ideate 1 (mind maps) Week 7: Ideate 2 (6 hats) Week 8: Prototyping 1 (storyboards) Week 9: Prototyping 2 (wireframes) Week 10: Test 1 (Testing cards) Week 11: Test 2 (UX audits) Week 12: Revision

IS4445 Survey http://bit.ly/robgleasure102

The next next impossible problem We now have a system that not only explored user needs, considered new values, and operationalised a new interaction, it actually showed the system to users and integrated further suggestions. However, these people weren’t really typical users by the end of the process. Many of them helped us create the new system, meaning they’re hardly objective “I told you that was what I wanted so… yeah, I guess I want it” We need to get hard, objective evidence to make sure we’re not building on bad assumptions We need to give our system a chance to fail.

Why testing cards? It’s important we use logic and understand the causes why users say, do, think, and feel the things they do. However, we can easily get trapped on a bad path if we build on a flawed or incomplete assumption. This gets worse the more we feel we understand our users. We need to get past the logic and judge our new interaction based on results.

Why testing cards(continued)? More than 75% of new products and services fail. Why? Ideas tend to be neat and logical, the world doesn’t Forecasting unknowns has limited value New ventures are different from existing ventures Butterfly wings

Pivoting Image from https://hbr.org/2013/05/why-the-lean-start-up-changes-everything

Minimum viable product Don’t try and build something complete Release it once it’s usable and spend your time observing use and feedback Drop anything that isn’t adding significant demonstrable value Integrate changes that you have seen actually work Image from https://hbr.org/2013/05/why-the-lean-start-up-changes-everything

Examples of lean testing Buffer was designed to be an application that would allow people to queue social media posts in advance, along with the times they should be shared Rather than build it and hope it sold, the founders decided to launch the homepage and see how many people clicked on the pricing link (at which point a message popped up explaining the site was work in progress but users could send an email if they were interested) Once they had proof people wanted it, they then repeated the process a level deeper with pricing plans

Examples of lean testing (continued) Bounce is a calendar that uses GPS and web data to tell people when they need to leave for specific meetings/appointments/events/etc. They set up a pre-purchase option to generate early revenue but they decided to A/B test* different price points 1.4% of visitors pre-purchased at $5 1.7% of visitors pre-purchased at $10 Why?

Examples of lean testing (continued) An online cosmetics vendor in Norway called BliVakker (about 20K visits/day) decided to add Facebook login to the checkout process Almost 50% of their visitors are probably logged in anyway Image and story from https://vwo.com/blog/facebook-login-reduces-sales-ecommerce-store/ They ran comparisons with and without by splitting half their visitors for each Better conversion rates without (3%, $10,000 per week) Why?

What are testing cards? Everything that has ever happened to every single user impacts on a piece of software. Not everything makes the cut in terms of consideration in our design Think lean! Testing cards and lean design let us be brutal in our evaluation Is the problem big enough to matter? Does the system make a big enough difference to matter? Does each feature make a big enough difference to matter? If not, we drop them and look elsewhere Often testing cards are about stripping the design right back to its most valuable core

What are testing cards? Proposed by Alex Osterwalder c/o Strategyzer.com Lays out simple structure to design tests for each assumption Encourages designer to ask themselves simple questions, e.g. what result is enough? How much does this test cost? How critical is this assumption? Image from https://strategyzr.s3.amazonaws.com/assets/vpd/resources/the-test-card.pdf

What are testing cards? Image from https://issuu.com/business.model.innovation/docs/vpd_sneakpeek/92

How to use testing cards for lean design? Basic principles Experimentation over elaborate planning Prioritise assumptions according to how critical they are Sketch out alternative tests for each until you have one or more with which you are satisfied Prioritise tests that identify flawed assumptions early to minimise the cost Discovering a failed assumption should be considered a huge victory Fail quickly, fail often, fail better

How to use testing cards for lean design (continued)? Basic principles (continued) Feedback over intuition Mantra of ‘get out of the building’ Test assumptions with people that actually represent intended users, partners, or other actors involved This isn’t just about the product or service; all aspects of the design should be tested, e.g. user groups, engagement channels, costs, resources, etc.

How to use testing cards for lean design (continued)? Basic principles (continued) Iterative design over ‘big design up front’ Old model was to keep idea hidden to maintain first-mover advantage, i.e. ‘stealth mode’ New model is to develop your customer base along the way Get something up and running, based on the aspects of the design with which you are most comfortable Get people using it Test and adapt to their needs as quickly as possible

Some disclaimers on this form of testing and lean design in general Not really any established toolkit – more of a series of a revelations shared by enthusiasts Seriously susceptible to ‘local maxima’ Often uses the idea of ‘scientific’ to gain a sort of false legitimacy

When to use testing cards? Testing cards can be used whenever we recognise we are making important assumptions, e.g. the representativeness of our personas, the qualities on our value curve, specific interface items and configurations, etc. Testing cards are often used to ease into a beta-test This is the antithesis of prototyping, where we wanted to knock together features we would re-build later Here we want to build only the most essential code but we want to build it as solidly as possible

Where to use testing cards? The most important thing is to ensure the new design is subject to the most naturalistic conditions possible – it needs to prove it works in the real world The ultimate goal is about simplifying the design so it’s easier to use and more free to grow and expand into emergent future uses

Building your report Identify critical assumptions and test cards for your new system What are the distinct interaction/interface components that make up your new system and what are those features supposed to be accomplishing? How could you test whether those features are actually having the desired impact? Are there any interdependencies between distinct interaction/interface components in your new system? What would happen if you removed one or more of those components and how could you test it?

Posting online Post one or two of your favourite test cards You should continuously ask yourself two questions Is there any part of the new system that hasn’t been tested, i.e. given a chance to fail? If we went and did this next week, would we be able to actually run the identified tests?

Reading Ries, E. (2011). The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses, Crown Publishing, USA. Wong, K. (2015). Making It Through The Startup 'Trough Of Sorrow, Forbes online, retrieved from http://www.forbes.com/sites/kylewong/2015/08/09/making-it-through-the-startup-trough-of-sorrow/