Gojko Adzic gojko@gojko.net http://gojko.net http://www.neuri.co.uk Agile Acceptance Testing Closing the communication gap in software projects Gojko Adzic gojko@gojko.net http://gojko.net http://www.neuri.co.uk

Why should you care (PM/BA)? Developers will actually read the specifications They will understood the stuff correctly They will not skip parts of the specs You can track the development progress Save time on acceptance/smoke testing

Why should you care (dev)? Requirements will be unambiguous and without functional gaps Business analysts will really understand those special cases you mentioned You will have automated tests to guide development It will be easier to take-over and hand-over code

Why should you care (QA)‏ Finally stop those guys from making the same mistakes over and over Avoid doing the same stuff all the time Build quality in from the start Vefify business rules by a click on a button

An experiment with four active battalions in US Army Commander expectations matched actions in only 34% of the cases L.G.Shattuck, 2000 http://www.au.af.mil/au/awc/awcgate/milreview/shattuck.pdf

Traditional process is like the Telephone game! http://www.sxc.hu/profile/scol22

How many points are there?

How many points are there?

A “small” misunderstanding can cost lots of money!

Pay ₤2 for a 1000 clicks what about 2500 clicks? what about 1000 clicks over 2 days?

One of the most effective ways of testing requirements is with test cases very much like those for testing the completed system Donald Gause and Gerald Weinberg Exploring Requirements - 1989!

As formality increases, tests and requirements become indistinguishable. Robert C. Martin and Grigori Melnik Tests and Requirements, Requirements and Tests: a Mobius Strip IEEE Software January/February Issue 2008

Key practices Discuss real-world examples to build a shared understanding of the domain Use those examples as an acceptance criteria Automate acceptance tests Focus the development on those tests Use the tests as a live specification to facilitate change

It is not UAT! It is not even “testing” in the QA sense. Customers, developers, business analysts involved as well as QA.

Better names “Acceptance testing” is misleading! Behaviours Executable specifications Examples

Although it is called “testing”, it is not really about QA It is about improving the communication and providing guidance for development

Jim Shore: “Describe-Demonstrate-Develop” A very useful way to think about acceptance tests in practice http://www.jamesshore.com/Blog/How-I-Use-Fit.html

Step 1: Building a shared understanding of the domain

The Example-Writing workshop Business people, developers and QA in the same room Transfer the knowledge Ensure that we all understand the same thing

Can you give us a few examples? How do we verify that this thing we are going to write is implemented completely and correctly? Can you give us a few examples?

Pretend it is magic… …and we deliver this software. How would you test it?

Inconsistencies and gaps are easy to spot when you write the rules down!

Real-world examples help flush out incorrect assumed rules find real business rules!

People have think at a more detailed level and can't brush questions off…

People approach the same problem from different perspectives, so this avoids groupthink!

Step 2: Select a formal set of acceptance tests and automate them

The Toyota Way Check at the source Inexpensive Verifications Test to prevent defects, not to discover them

Save time on manually (acceptance/smoke) testing Verify business rules with the click of a button

Automate tests, but still keep them human-readable And you can add pictures as well….

This is a specification:

It is also a test

This as well

And this

And this Given a stock of prices 0.5,1.0 When the stock is traded at 2.0 Then the alert status should be OFF When the stock is traded at 5.0 When the stock is traded at 11.0

The only technical slide...

Step 3: Providing focus for development No just-in-case code

Developers will have to code exactly what was specified … not just the rules they see

Automated test reports show where we are… When all the tests are green, the job is done

Tests promote a consistent use of the DDD ubiquitous language!

Iteration flow (just a suggestion)‏

Step 4: Keeping in touch with changes

Previous examples help you ensure to discuss all important edge cases.

Automated tests show straight away when something is obsolete or broken

“significant and valuable business resource” [tests became a] “significant and valuable business resource” Rick Mugridge, Doubling the Value of Automated Tests, Google Tech Talks 09/2006

Common Excuses for not using Agile Acceptance Testing

“It’s extra work and I don’t have time” Save time by not writing those big requirements docs that nobody reads…

“But they will only look at the tests and not read the requirements…” Tests ARE Specifications

“What if I leave something out “What if I leave something out? If they are using tests as the scope, and we do not specify some tests, what happens then?“

“That’s cheating! We give the developers exactly what we are going to test, and they develop the software only to pass those tests…”

“I won’t understand user stories and tests, I’m used to use cases” Story tests still contain triggers, steps, exceptions…

“It’s impossible to describe UI layouts as FIT tests” So use something else – share the knowledge when you discuss tests

“Rules are too scattered, there is no big picture” Communicate intent when writing stories, clean up tests to isolate rules, add some general docs…

“We don't want to give way control over testing” If anything, you are getting more people to participate in testing...

Recap (PM/BA)‏ Developers will have to read the specifications to implement tests Discussion makes sure that everyone understood the stuff correctly To make all the tests green, they cannot skip parts of the specs You can track the development progress Save time on acceptance testing with automated verifications

Recap (dev)‏ Discuss and suggest examples until the gaps and inconsistencies are flushed out Make sure that business analysts understood special cases by suggesting them as examples and discussing them Acceptance tests are a live specification/documentation for the code

Recap (QA)‏ Suggest examples and discuss them to cover mistakes that people make over and over Automated tests help you avoid doing the same stuff all the time Build quality in from the start by suggesting tests that prevent problems Verify business rules by a click on a button

Questions? http://gojko.net http://www.neuri.co.uk gojko@gojko.com