1. Acceptance Test Driven Development
Michael Eselgroth
Daryn Holmes

2. ATDD helps to improve on many of the short comings in software development
Communication, Design, Defects

3. No guarantees, Manual testing is a good thing
It’s not free, it’s not easy

4. Not optional

5. Story Mapping
Story Mapping Demo

6. Who, what, why but not how…
User Stories
As a …
I want …
So that …
Who?
What?
Why?
They do not contain enough implementation details for developers – intentionally vague
They are a place holder for a future discussion about the detailed requirements
They put the requirement in context within the business
They should explain the requirement not the solution
The smaller the better
How – to be decided later
Who, what, why but not how…

7. Requirements Stories are reminders for a future discussion
Drive out the requirements, in the form of examples
Specification by example
Preferably in a format that can be automatically executed and verified
Tools often use the following advertising:
“Business users can now write their own requirements in their own language”
Specific examples to clarify the requirement
Naturally use examples to explain things
Although this would be nice, it seldom happens
It can take a long time to get business analysts to actually write the tests
It is more common for the team to work together to get the requirements
good for communication
As long as the business users can read the tests, then they help to focus discussions and flesh out requirements.
The goal is not really to get business analysts to write the tests on their own, the goal is to get clear, unambiguous specifications.

8. Specification by Example
Story
High level requirement
May contain high level acceptance criteria
Specification by Example
Low level requirements
Specific examples\scenarios of acceptance criteria
Example 1
Example 2
Example n
Code
Includes unit testing and fleshing out the scenarios\examples into complete automated acceptance tests

9. ATDD (BDD)
TDD = Unit Testing + Acceptance Testing + …

10. Anatomy of ATDD System Under Test (SUT) BDD Framework SUT Driver
Runs examples\scenarios
Cucumber, Fitnsesse
SUT Driver
Execute and drive the SUT
White, Selenium WebDriver
Verification Framework
Built into Fitnesse
Unit Test verification
JUnit, Nunit, RSpec

11. FIT, Cucumber etc
Demo

12. Intelligent Testing No strict rules for all situations Unit testing
Feature testing
Sensible slices
Somewhere in between
Unit testing
Preferably objects isolation
Feature testing
Automated acceptance testing
Sensible slices
Somewhere in between
Integration of a subset of the system
Engineering Trade-off
Run slower than unit tests

13. Mainframe
Oracle
Java Stored Proc
PL\SQL
Reporting Framework
Browser (End user)

14. Mainframe
Oracle
Java Stored Proc
PL\SQL
Reporting Framework
Test Harness

15. Mainframe
Oracle
Java Stored Proc
PL\SQL
Reporting Framework
Test Harness

16. Mainframe
Oracle
Java Stored Proc
PL\SQL
Reporting Framework
Test Harness
Java Stored Proc

17. Smart Home SUT Smart Home Controller We can’t have fake (mock) devices
Too many
To hard to get the results back
Smarthome RWE
Smart Home Controller

18. Business logic, database etc
Smart Home
Message Adapter
Business logic, database etc
Transmitter

19. Smart Home Message Adaptor Business logic, database etc
‘Test’ Message Transmitter

20. Smart Home Message Adapter Business logic, database etc
‘Test’ Message Transmitter

21. Smart Home Message Adapter Business logic, database etc
‘Test’ Message Transmitter

22. Smart Home Message Adapter Message Adapter Test Harness
Business logic, database etc
‘Test’ Message Transmitter

23. Example: Automated Data Input
Database
Background Application (scheduled task)
Feed Provider
Content Team
Calculated Data
Raw Data
Not Requested

24. End To End Integration Test
DatabaseRepository
WebHandler
Notifier
Domain
Mocked
Mocked
Mocked

26. What part do expert testers play
They should do what they always done, just sooner and more often
They should be embedded in the team
More opportunities for better tests if they are on-board from the start
They should do what they always done, just sooner and more often
Developers get more time to recover
Continuous integration testing must happen
The sooner the better, the more often the better
Usability Testing can start from the beginning
So can performance testing
Don’t prematurely optimise, but test performance early
Performance can be difficult and costly to fix at the end of a project
Having separate testing teams is not as good as having expert testers within the team
Their work can be made easier if they are onboard from the start
Developers can help support the tester, and testing strategy

27. Why bother… http://en.wikipedia.org/wiki/British_Airways_Flight_5390
British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life.
At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls. 
Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air.
Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window.
Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster.
Accident investigators found that a replacement windscreen had been installed 27 hours before the flight, and that the procedure had been approved by the Shift Maintenance Manager. However, 84 of the 90 windscreen retention bolts were 0.026 inches (0.66 mm) too small in diameter, while the remaining six were 0.1 inches (2.5 mm) too short. The investigation revealed that the previous windscreen had been fitted with incorrect bolts, which had been replaced on a "like for like" basis by the Shift Maintenance Manager without reference to the maintenance documentation.[2] The air pressure difference between the cabin and the outside during the flight proved to be too much, leading to the failure of the windscreen. The incident also brought to attention a design flaw in the aircraft of the windscreen being secured from the outside of the aircraft, putting a greater pressure on the bolts than if they were secured from the inside.
Investigators blamed the British Airways Birmingham Airport Shift Maintenance Manager for installing the incorrect bolts during the windscreen replacement and for failing to follow official British Airways policies. They also found fault with British Airways' policies, which should have required testing or verification by another individual for this critical task. Finally, investigators blamed the local Birmingham Airport management for not directly monitoring the Shift Maintenance Manager's working practices. The AAIB chief investigator was Stuart Culling.
Safety recommendations
Review their quality assurance system and encourage engineers to provide feedback.
25 February 2011
Daryn Holmes

28. Why bother…
On 10 June 1990, an improperly installed pain of the windshield failed
Plane was at feet
No loss of life
2 people were seriously injured
British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life.
At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls. 
Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air.
Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window.
Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster.
25 February 2011
Daryn Holmes

29. Why bother…
A replacement windscreen had been installed 27 hours before the flight
90 bolts
84 were 0.66mm too small in diameter
6 were 2.5mm too short
Previous windscreen has been fitted with the wrong bolts
British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life.
At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls. 
Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air.
Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window.
Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster.
25 February 2011
Daryn Holmes

30. Why bother…
Investigators found found fault with British Airways' policies, which should have required testing or verification by another individual for this critical task
They found engineers were working by themselves, under extreme pressure
Safety recommendations
British Airways should review their quality assurance system and encourage engineers to provide feedback
British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life.
At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls. 
Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air.
Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window.
Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster.

31. Why bother… http://en.wikipedia.org/wiki/British_Airways_Flight_5390
British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life.
At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls. 
Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air.
Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window.
Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster.
Accident investigators found that a replacement windscreen had been installed 27 hours before the flight, and that the procedure had been approved by the Shift Maintenance Manager. However, 84 of the 90 windscreen retention bolts were 0.026 inches (0.66 mm) too small in diameter, while the remaining six were 0.1 inches (2.5 mm) too short. The investigation revealed that the previous windscreen had been fitted with incorrect bolts, which had been replaced on a "like for like" basis by the Shift Maintenance Manager without reference to the maintenance documentation.[2] The air pressure difference between the cabin and the outside during the flight proved to be too much, leading to the failure of the windscreen. The incident also brought to attention a design flaw in the aircraft of the windscreen being secured from the outside of the aircraft, putting a greater pressure on the bolts than if they were secured from the inside.
Investigators blamed the British Airways Birmingham Airport Shift Maintenance Manager for installing the incorrect bolts during the windscreen replacement and for failing to follow official British Airways policies. They also found fault with British Airways' policies, which should have required testing or verification by another individual for this critical task. Finally, investigators blamed the local Birmingham Airport management for not directly monitoring the Shift Maintenance Manager's working practices. The AAIB chief investigator was Stuart Culling.
Safety recommendations
Review their quality assurance system and encourage engineers to provide feedback.

32. Really? ToEven AwayFromZero
When a number is halfway between two others, it is rounded toward the nearest even number
AwayFromZero
When a number is halfway between two others, it is rounded toward the nearest number that is away from zero
At the start of a project
start with a walking skeleton
and start this with acceptance tests
Introducing it on an existing project
Introduce it incrementally
Do as much TDD as possible on the next feature
Fix all defects using TDD
First expose the defect in a test
Make the test pass

33. What part do developers play
Nr. Of Stories
Develop
Test
Requirements
(Spec)

34. What part do developers play
Nr. Of Stories
Develop
Test
Requirements
(Spec)

35. What part do developers play
Nr. Of Stories
Develop
Test
Requirements
(Spec)

36. What part do developers play
Nr. Of Stories
Develop
Test
Requirements
(Spec)

37. What part do developers play
Nr. Of Stories
Develop
Test
Requirements
(Spec)

38. Crystal: Efficiency is expendable in non-bottle-neck activities

40. Barry Boehm, “Equity Keynote Address” March 19, 2007.

41. As a rule of thumb, every hour you spend on defect prevention will reduce your repair time from three to ten hours
Steve McConnell, Software Quality At Top Speed,

42. Should Start-ups Use TDD?

43. Should Start-ups Use TDD?

45. Introducing ATDD At the start of a project
Introducing it on an existing project
Outside in
At the start of a project
start with a walking skeleton
and start this with acceptance tests
Introducing it on an existing project
Introduce it incrementally
Do as much TDD as possible on the next feature
Fix all defects using TDD
First expose the defect in a test
Make the test pass

46. TDD Culture It’s not about the technology, it’s about the culture
The right tools are important but it’s not about the code, it’s about the culture
Green
Team is actively pursing TDD
all they need is support from management
Amber
Interested (chicken or the egg problem)
Need help in getting started and for getting over set backs
Red
Against TDD
No direct influence will have any effect on the team
It is difficult to introduce it to a team that is already considered successful
These teams are usually doing something else instead
Such as picking one developer to do all the tests, usually ends up with one developer retrofitting unit tests and doing loads of manual testing before resigning

47. TDD Culture “The development team is responsible
for first line support, 24/7”
- Steve Freeman

48. TDD Culture “The development team is responsible
for first line support, 24/7”
“Team members ended up volunteering
to be on support over Christmas”
- Steve Freeman

49. Thank you…