Acceptance Test Driven Development Michael Eselgroth Daryn Holmes

ATDD helps to improve on many of the short comings in software development Communication, Design, Defects

No guarantees, Manual testing is a good thing It’s not free, it’s not easy

Not optional

Story Mapping Story Mapping Demo

Who, what, why but not how… User Stories As a … I want … So that … Who? What? Why? They do not contain enough implementation details for developers – intentionally vague They are a place holder for a future discussion about the detailed requirements They put the requirement in context within the business They should explain the requirement not the solution The smaller the better How – to be decided later Who, what, why but not how…

Requirements Stories are reminders for a future discussion Drive out the requirements, in the form of examples Specification by example Preferably in a format that can be automatically executed and verified Tools often use the following advertising: “Business users can now write their own requirements in their own language” Specific examples to clarify the requirement Naturally use examples to explain things Although this would be nice, it seldom happens It can take a long time to get business analysts to actually write the tests It is more common for the team to work together to get the requirements good for communication As long as the business users can read the tests, then they help to focus discussions and flesh out requirements. The goal is not really to get business analysts to write the tests on their own, the goal is to get clear, unambiguous specifications.

Specification by Example Story High level requirement May contain high level acceptance criteria Specification by Example Low level requirements Specific examples\scenarios of acceptance criteria Example 1 Example 2 Example n Code Includes unit testing and fleshing out the scenarios\examples into complete automated acceptance tests

ATDD (BDD) TDD = Unit Testing + Acceptance Testing + …

Anatomy of ATDD System Under Test (SUT) BDD Framework SUT Driver Runs examples\scenarios Cucumber, Fitnsesse SUT Driver Execute and drive the SUT White, Selenium WebDriver Verification Framework Built into Fitnesse Unit Test verification JUnit, Nunit, RSpec

FIT, Cucumber etc Demo

Intelligent Testing No strict rules for all situations Unit testing Feature testing Sensible slices Somewhere in between Unit testing Preferably objects isolation Feature testing Automated acceptance testing Sensible slices Somewhere in between Integration of a subset of the system Engineering Trade-off Run slower than unit tests

Mainframe Oracle Java Stored Proc PL\SQL Reporting Framework Browser (End user)

Mainframe Oracle Java Stored Proc PL\SQL Reporting Framework Test Harness

Mainframe Oracle Java Stored Proc PL\SQL Reporting Framework Test Harness

Mainframe Oracle Java Stored Proc PL\SQL Reporting Framework Test Harness Java Stored Proc

Smart Home SUT Smart Home Controller We can’t have fake (mock) devices Too many To hard to get the results back Smarthome RWE Smart Home Controller

Business logic, database etc Smart Home Message Adapter Business logic, database etc Transmitter

Smart Home Message Adaptor Business logic, database etc ‘Test’ Message Transmitter

Smart Home Message Adapter Business logic, database etc ‘Test’ Message Transmitter

Smart Home Message Adapter Business logic, database etc ‘Test’ Message Transmitter

Smart Home Message Adapter Message Adapter Test Harness Business logic, database etc ‘Test’ Message Transmitter

Example: Automated Data Input Database Background Application (scheduled task) Feed Provider Content Team Calculated Data Raw Data Email Not Requested

End To End Integration Test DatabaseRepository WebHandler EmailNotifier Domain Mocked Mocked Mocked

What part do expert testers play They should do what they always done, just sooner and more often They should be embedded in the team More opportunities for better tests if they are on-board from the start They should do what they always done, just sooner and more often Developers get more time to recover Continuous integration testing must happen The sooner the better, the more often the better Usability Testing can start from the beginning So can performance testing Don’t prematurely optimise, but test performance early Performance can be difficult and costly to fix at the end of a project Having separate testing teams is not as good as having expert testers within the team Their work can be made easier if they are onboard from the start Developers can help support the tester, and testing strategy

Why bother… http://en.wikipedia.org/wiki/British_Airways_Flight_5390 British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life. At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to 17300 feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls.  Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air. Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window. Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster. Accident investigators found that a replacement windscreen had been installed 27 hours before the flight, and that the procedure had been approved by the Shift Maintenance Manager. However, 84 of the 90 windscreen retention bolts were 0.026 inches (0.66 mm) too small in diameter, while the remaining six were 0.1 inches (2.5 mm) too short. The investigation revealed that the previous windscreen had been fitted with incorrect bolts, which had been replaced on a "like for like" basis by the Shift Maintenance Manager without reference to the maintenance documentation.[2] The air pressure difference between the cabin and the outside during the flight proved to be too much, leading to the failure of the windscreen. The incident also brought to attention a design flaw in the aircraft of the windscreen being secured from the outside of the aircraft, putting a greater pressure on the bolts than if they were secured from the inside. Investigators blamed the British Airways Birmingham Airport Shift Maintenance Manager for installing the incorrect bolts during the windscreen replacement and for failing to follow official British Airways policies. They also found fault with British Airways' policies, which should have required testing or verification by another individual for this critical task. Finally, investigators blamed the local Birmingham Airport management for not directly monitoring the Shift Maintenance Manager's working practices. The AAIB chief investigator was Stuart Culling. Safety recommendations Review their quality assurance system and encourage engineers to provide feedback. 25 February 2011 Daryn Holmes

Why bother… On 10 June 1990, an improperly installed pain of the windshield failed Plane was at 17300 feet No loss of life 2 people were seriously injured http://en.wikipedia.org/wiki/British_Airways_Flight_5390 British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life. At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to 17300 feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls.  Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air. Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window. Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster. 25 February 2011 Daryn Holmes

Why bother… A replacement windscreen had been installed 27 hours before the flight 90 bolts 84 were 0.66mm too small in diameter 6 were 2.5mm too short Previous windscreen has been fitted with the wrong bolts http://en.wikipedia.org/wiki/British_Airways_Flight_5390 British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life. At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to 17300 feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls.  Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air. Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window. Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster. 25 February 2011 Daryn Holmes

Why bother… Investigators found found fault with British Airways' policies, which should have required testing or verification by another individual for this critical task They found engineers were working by themselves, under extreme pressure Safety recommendations British Airways should review their quality assurance system and encourage engineers to provide feedback http://en.wikipedia.org/wiki/British_Airways_Flight_5390 British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life. At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to 17300 feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls.  Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air. Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window. Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster.

Why bother… http://en.wikipedia.org/wiki/British_Airways_Flight_5390 British Airways Flight 5390 was a British Airways flight between Birmingham Airport in England and Málaga, Spain. On 10 June 1990 an improperly installed pane of the windscreen failed, blowing the plane's captain halfway out of the aircraft, with his body firmly pressed against the window frame. The first officer managed to perform an emergency landing in Southampton with no loss of life. At 07:33, the cabin crew had begun to prepare for meal service. The plane had climbed to 17300 feet over Didcot, Oxfordshire. Suddenly, there was a loud bang, and the fuselage quickly filled with condensation. The left windscreen, on the captain's side of the cockpit, had suffered a catastrophic failure. Lancaster was jerked out of his seat by the rushing air and forced head first out of the cockpit, his knees snagging onto the flight controls.  Nigel Ogden quickly latched his hands onto the captain's belt. Susan Price and another flight attendant began to reassure passengers, secure loose objects, and take up emergency positions. Meanwhile, Lancaster was being battered and frozen in the 500 mph slipstream, and was losing consciousness due to the thin air. Ogden, still latched onto Lancaster, had begun to suffer from frostbite, bruising and exhaustion. He was relieved by the remaining two flight attendants. By this time Lancaster had already shifted an additional six to eight inches out the window. From the flight deck, the flight and cabin crew were able to view his head and torso through the left direct vision window. Atchison eventually received clearance from air traffic control to land at Southampton, while the flight attendants managed to free and hold on to Lancaster's ankles for the remainder of the flight. By 07:55 the aircraft had landed safely on Runway 02 at Southampton. Passengers immediately disembarked from the front and rear stairs, and emergency crews retrieved Lancaster. Accident investigators found that a replacement windscreen had been installed 27 hours before the flight, and that the procedure had been approved by the Shift Maintenance Manager. However, 84 of the 90 windscreen retention bolts were 0.026 inches (0.66 mm) too small in diameter, while the remaining six were 0.1 inches (2.5 mm) too short. The investigation revealed that the previous windscreen had been fitted with incorrect bolts, which had been replaced on a "like for like" basis by the Shift Maintenance Manager without reference to the maintenance documentation.[2] The air pressure difference between the cabin and the outside during the flight proved to be too much, leading to the failure of the windscreen. The incident also brought to attention a design flaw in the aircraft of the windscreen being secured from the outside of the aircraft, putting a greater pressure on the bolts than if they were secured from the inside. Investigators blamed the British Airways Birmingham Airport Shift Maintenance Manager for installing the incorrect bolts during the windscreen replacement and for failing to follow official British Airways policies. They also found fault with British Airways' policies, which should have required testing or verification by another individual for this critical task. Finally, investigators blamed the local Birmingham Airport management for not directly monitoring the Shift Maintenance Manager's working practices. The AAIB chief investigator was Stuart Culling. Safety recommendations Review their quality assurance system and encourage engineers to provide feedback.

Really? ToEven AwayFromZero When a number is halfway between two others, it is rounded toward the nearest even number AwayFromZero When a number is halfway between two others, it is rounded toward the nearest number that is away from zero At the start of a project start with a walking skeleton and start this with acceptance tests Introducing it on an existing project Introduce it incrementally Do as much TDD as possible on the next feature Fix all defects using TDD First expose the defect in a test Make the test pass

What part do developers play Nr. Of Stories Develop Test Requirements (Spec)

What part do developers play Nr. Of Stories Develop Test Requirements (Spec)

What part do developers play Nr. Of Stories Develop Test Requirements (Spec)

What part do developers play Nr. Of Stories Develop Test Requirements (Spec)

What part do developers play Nr. Of Stories Develop Test Requirements (Spec)

Crystal: Efficiency is expendable in non-bottle-neck activities

Barry Boehm, “Equity Keynote Address” March 19, 2007.

As a rule of thumb, every hour you spend on defect prevention will reduce your repair time from three to ten hours Steve McConnell, Software Quality At Top Speed, 1996 http://www.stevemcconnell.com/articles/art04.htm

Should Start-ups Use TDD? http://www.redhills.ie/2012/08/08/should-a-startup-use-tdd/

Should Start-ups Use TDD? http://www.redhills.ie/2012/08/08/should-a-startup-use-tdd/

Introducing ATDD At the start of a project Introducing it on an existing project Outside in At the start of a project start with a walking skeleton and start this with acceptance tests Introducing it on an existing project Introduce it incrementally Do as much TDD as possible on the next feature Fix all defects using TDD First expose the defect in a test Make the test pass

TDD Culture It’s not about the technology, it’s about the culture The right tools are important but it’s not about the code, it’s about the culture Green Team is actively pursing TDD all they need is support from management Amber Interested (chicken or the egg problem) Need help in getting started and for getting over set backs Red Against TDD No direct influence will have any effect on the team It is difficult to introduce it to a team that is already considered successful These teams are usually doing something else instead Such as picking one developer to do all the tests, usually ends up with one developer retrofitting unit tests and doing loads of manual testing before resigning

TDD Culture “The development team is responsible for first line support, 24/7” - Steve Freeman

TDD Culture “The development team is responsible for first line support, 24/7” “Team members ended up volunteering to be on support over Christmas” - Steve Freeman

Thank you…