1. Google Confidential and Proprietary Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group Oct 18, 2012 Alan Myrvold Google

2. Google Confidential and Proprietary About me Feb 2011 - now 2005 - 2011 1998 - 2005 1988 - 1998 Google Test Engineer, Ads Microsoft SDET in Office Security + Outlook Entrust Test manager, development manager, security assurance manager Cognos Tester, developer, test manager, development manager

3. Google Confidential and Proprietary More about me

4. Google Confidential and Proprietary Buzzword bingo ATDD BDD Cucumber Cucumber - JVM Gherkin Jasmine SpecFlow

5. Google Confidential and Proprietary Buzzword bingo ATDD- acceptance test driven development BDD- behavior driven development Cucumber- a Ruby tool that supports BDD Cucumber - JVM - a Java tool that supports BDD Gherkin - the language used by Cucumber Jasmine- a javascript tool for BDD SpecFlow- a.NET tool for BDD

6. Google Confidential and Proprietary Cucumber Example Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two numbers Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen

7. Google Confidential and Proprietary Given / When / Then Given - precondition When - action Then - assertion on expected result And - same action as before

8. Google Confidential and Proprietary Cucumber step implementation @When("I have entered (.*) into the calculator") public void enterNumber(int number) {... }

9. Google Confidential and Proprietary Scenario Outline Example Scenario Outline: Add two numbers Given I have entered into the calculator And I have entered into the calculator When I press add Then the result should be on the screen Examples: | x | y | z | | 2 | 3 | 5 | | 0 | -1 | -1 |

10. Google Confidential and Proprietary Why BDD? Clarifying requirements by example Demystifying automated tests by using English Demystifying repeated manual tests by emphasizing why and what to verify.

11. Google Confidential and Proprietary Some test types

12. Google Confidential and Proprietary How I used BDD at Microsoft Clarifying requirements in my test plan.

13. Google Confidential and Proprietary How we are using BDD at Google My group, DoubleClick Bid Manager, uses BDD for Java API-level system tests and repeated manual tests. Other groups use BDD tests for Java WebDriver tests. We share the same framework, developed internally.

14. Google Confidential and Proprietary Tools support + Books Tools: o Cucumber - Ruby http://cukes.infohttp://cukes.info o Cucumber JVM https://github.com/cucumber/cucumber-jvmhttps://github.com/cucumber/cucumber-jvm o SpecFlow - Binding business requirements to.NET code http://specflow.org http://specflow.org Books o The Cucumber Book o The RSpec Book o Cucumber Recipies (beta, scheduled 3/7/2013) All books from Pragmatic Programmers, http://pragprog.comhttp://pragprog.com

15. Google Confidential and Proprietary When / Then elsewhere Mockito … a Java unit testing framework @Mock CalculationEngine engine; Calculator calculator = new Calculator(engine); when(engine.add(2, 2)).thenReturn(4); calculator.parse("2 + 2 ="); assertEquals("4", caclulator.getResult()); Compare the syntax to EasyMock: expect(engine.add(2, 2)).andReturn(4);

16. Google Confidential and Proprietary describe / it Rspec … a Ruby unit testing framework describe Calculator, "#basics" do it "return 4 for 2+2" do calc = Caculator.new calc.add(2, 2) calc.result.should eq(4) end Jasmine … a Javascript unit testing framework describe("calc", function() { it("2+2 is 4", function() { expect(calc(2, 2).toEqual(4); });

17. Google Confidential and Proprietary Failure mode #1 - Too implementation dependent BAD Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen BETTER When I add 50 and 70 Then the result is 120

18. Google Confidential and Proprietary Failure mode #2 - Programmatic Scripts BAD When I set x to 1 And while x < 50 And set row x to "empty" BETTER Then set the first 50 rows to "empty"

19. Google Confidential and Proprietary Failure mode #3 - Too low level BAD When I go to the login page And enter "bob" into the username field And enter "pass123" into the password field And click login Then I am logged in as "bob" BETTER When I log in as "bob"

20. Google Confidential and Proprietary Failure mode #4 - Not exploring interesting cases BAD When I add 50 and 70 Then the result is 120 BETTER When I add 50 and 70 Then the result is 120 When I add 1e90, 0.1, and -1e90 Then the result is 0.1

21. Google Confidential and Proprietary Failure mode #5 - Not using regex BAD @When("Set the budget to 100") public void setBudget() {... } BETTER @When("Set the budget to (.*)") public void setBudget(int amount) {... }

22. Google Confidential and Proprietary Failure mode #6 - Complex parsing logic BAD @When("(.*) the (.*) to (.*)") public void doAction(String action, String name, String value) { if (action.equals("set") && name.equals("budget") } BETTER @When("Set the budget to (.*)") public void setBudget(int amount) {... }

23. Google Confidential and Proprietary Failure mode #7 - Leaking code details BAD When I click the BTN- REFRESH-ALL button BETTER When I refresh all

24. Google Confidential and Proprietary Failure mode #8 - Bad test architecture BAD Calling entry points in code that are fragile, or disappear BETTER Using supported public or test API's

25. Google Confidential and Proprietary Success tips from me Use BDD for a small set of tests Focus on human readability, as a domain expert using the feature would describe a test

26. Google Confidential and Proprietary Success tips from The Cucumber Book, by Matt Wayne and Aslak Hellesoy DAMP beats DRY DAMP: Descriptive and meaningful phrases DRY: Don’t repeat yourself Declarative better than imperative Declarative: Given I am logged in Imperative: Log in as user “Bob”

27. Google Confidential and Proprietary Questions? http://testapprentice.com amyrvold@google.com