1. IS4445 Principles of Interaction Design Lecture 10: Testing cards
Rob Gleasure

2. Course structure Or more specifically Week 1: Introduction
Week 2: Empathise 1 (personas)
Week 3: Empathise 2 (empathy maps)
Week 4: Define 1 (journey maps)
Week 5: Define 2 (value curves)
Week 6: Ideate 1 (mind maps)
Week 7: Ideate 2 (6 hats)
Week 8: Prototyping 1 (storyboards)
Week 9: Prototyping 2 (wireframes)
Week 10: Test 1 (Testing cards)
Week 11: Test 2 (UX audits)
Week 12: Revision

3. IS4445
Survey

4. The next next impossible problem
We now have a system that not only explored user needs, considered new values, and operationalised a new interaction, it actually showed the system to users and integrated further suggestions.
However, these people weren’t really typical users by the end of the process. Many of them helped us create the new system, meaning they’re hardly objective
“I told you that was what I wanted so… yeah, I guess I want it”
We need to get hard, objective evidence to make sure we’re not building on bad assumptions
We need to give our system a chance to fail.

5. Why testing cards?
It’s important we use logic and understand the causes why users say, do, think, and feel the things they do.
However, we can easily get trapped on a bad path if we build on a flawed or incomplete assumption.
This gets worse the more we feel we understand our users.
We need to get past the logic and judge our new interaction based on results.

6. Why testing cards(continued)?
More than 75% of new products and services fail.
Why?
Ideas tend to be neat and logical, the world doesn’t
Forecasting unknowns has limited value
New ventures are different from existing ventures
Butterfly wings

7. Pivoting
Image from

8. Minimum viable product
Don’t try and build something complete
Release it once it’s usable and spend your time observing use and feedback
Drop anything that isn’t adding significant demonstrable value
Integrate changes that you have seen actually work
Image from

9. Examples of lean testing
Buffer was designed to be an application that would allow people to queue social media posts in advance, along with the times they should be shared
Rather than build it and hope it sold, the founders decided to launch the homepage and see how many people clicked on the pricing link (at which point a message popped up explaining the site was work in progress but users could send an if they were interested)
Once they had proof people wanted it, they then repeated the process a level deeper with pricing plans

10. Examples of lean testing (continued)
Bounce is a calendar that uses GPS and web data to tell people when they need to leave for specific meetings/appointments/events/etc.
They set up a pre-purchase option to generate early revenue but they decided to A/B test* different price points
1.4% of visitors pre-purchased at $5
1.7% of visitors pre-purchased at $10
Why?

11. Examples of lean testing (continued)
An online cosmetics vendor in Norway called BliVakker (about 20K visits/day) decided to add Facebook login to the checkout process
Almost 50% of their visitors are probably logged in anyway
Image and story from
They ran comparisons with and without by splitting half their visitors for each
Better conversion rates without (3%, $10,000 per week)
Why?

12. What are testing cards?
Everything that has ever happened to every single user impacts on a piece of software. Not everything makes the cut in terms of consideration in our design
Think lean!
Testing cards and lean design let us be brutal in our evaluation
Is the problem big enough to matter?
Does the system make a big enough difference to matter?
Does each feature make a big enough difference to matter?
If not, we drop them and look elsewhere
Often testing cards are about stripping the design right back to its most valuable core

13. What are testing cards?
Proposed by Alex Osterwalder c/o Strategyzer.com
Lays out simple structure to design tests for each assumption
Encourages designer to ask themselves simple questions, e.g. what result is enough? How much does this test cost? How critical is this assumption?
Image from

14. What are testing cards?
Image from

15. How to use testing cards for lean design?
Basic principles
Experimentation over elaborate planning
Prioritise assumptions according to how critical they are
Sketch out alternative tests for each until you have one or more with which you are satisfied
Prioritise tests that identify flawed assumptions early to minimise the cost
Discovering a failed assumption should be considered a huge victory
Fail quickly, fail often, fail better

16. How to use testing cards for lean design (continued)?
Basic principles (continued)
Feedback over intuition
Mantra of ‘get out of the building’
Test assumptions with people that actually represent intended users, partners, or other actors involved
This isn’t just about the product or service; all aspects of the design should be tested, e.g. user groups, engagement channels, costs, resources, etc.

17. How to use testing cards for lean design (continued)?
Basic principles (continued)
Iterative design over ‘big design up front’
Old model was to keep idea hidden to maintain first-mover advantage, i.e. ‘stealth mode’
New model is to develop your customer base along the way
Get something up and running, based on the aspects of the design with which you are most comfortable
Get people using it
Test and adapt to their needs as quickly as possible

18. Some disclaimers on this form of testing and lean design in general
Not really any established toolkit – more of a series of a revelations shared by enthusiasts
Seriously susceptible to ‘local maxima’
Often uses the idea of ‘scientific’ to gain a sort of false legitimacy

19. When to use testing cards?
Testing cards can be used whenever we recognise we are making important assumptions, e.g. the representativeness of our personas, the qualities on our value curve, specific interface items and configurations, etc.
Testing cards are often used to ease into a beta-test
This is the antithesis of prototyping, where we wanted to knock together features we would re-build later
Here we want to build only the most essential code but we want to build it as solidly as possible

20. Where to use testing cards?
The most important thing is to ensure the new design is subject to the most naturalistic conditions possible – it needs to prove it works in the real world
The ultimate goal is about simplifying the design so it’s easier to use and more free to grow and expand into emergent future uses

21. Building your report
Identify critical assumptions and test cards for your new system
What are the distinct interaction/interface components that make up your new system and what are those features supposed to be accomplishing?
How could you test whether those features are actually having the desired impact?
Are there any interdependencies between distinct interaction/interface components in your new system?
What would happen if you removed one or more of those components and how could you test it?

22. Posting online Post one or two of your favourite test cards
You should continuously ask yourself two questions
Is there any part of the new system that hasn’t been tested, i.e. given a chance to fail?
If we went and did this next week, would we be able to actually run the identified tests?

23. Reading
Ries, E. (2011). The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses, Crown Publishing, USA.
Wong, K. (2015). Making It Through The Startup 'Trough Of Sorrow, Forbes online, retrieved from