1. Zhuhai Test Automation Overview Ryan Li Team lead of test automation Foglight QA

2. Overview

3. Requirement Collection First Release of Framework Integrate with Webdriver and Cucumber Concurrent Execution Web IDE for Running Test 2011.6 2012 2012.6 2013 2011 The team was created 2 years ago. Zhuhai test automation mainly focus on building a unify testing environment for tester. We keep providing new features to help tester to reduce effort on manual/performance/load testing. What is Zhuhai test automation?

4. Team structure Ryan Li (Team lead) xxx (Scripter for PulseNET, FglAM) xxx (Scripter for DME and vFoglight) xxx (Testing tool developer) xxx (Scripter for Foglight, APM)

5. Project Status 5 Confidential Zhuhai automation currently covering on following projects: Foglight FglAM PulseNET vStorage DME F4VD vFoglight 1EU …

6. What is Workbot

7. Introduction to Workbot 7 Confidential Workbot is a testing automation system which provides many pragmatic functions for managing test cases, and setting up test environment for web based application. Here are the main features of Workbot: Platform independent Browser independent Distributed testing and Multitasking

8. The Features of Test Framework - Workbot What Benefits test automation bring to us? The Zhuhai test automation framework, we call it “Workbot”, the latest release is v3.3.2, and it’s good at the factors below: One-click test environment setup Browser test, it’s available for IE6-10, Firefox 3-latest, Chrome and Safari. Interface test (gray-box test), including web services, http... Distributed log analyst. Command line test. Load test. Performance test.

9. What Benefits Will Automation Bring to Us? Test automation that doesn’t hurt 9 Confidential EFFICIENCY AGILITY RESILIENCY

10. What Benefits Will Automation Bring to Us? 10 Confidential Provide a Continues Deployment (CD) environment Improve software quality by reducing time to market Improve reliability and consistency of testing processes Allow manual testers do more complex & more suited for human kinds of tests (exploratory, usability etc.) Enforce SLAs (service level agreements) Ultimate goal is to save your business $$$...

11. Our Goal Make test more easier 11 Confidential To build a easy-to-use, efficient, effective, distributed testing and multitasking automation environment for QA tester, to removes the tedium of creating the automation infrastructure. On-going automation activities. Automation technical sharing Create documentation (scripting API, user guide) Easy environment setup Easy scheduling Easy reporting – receive report from email Encapsulate business logic and create reusable test script libraries …

12. OS Platform (Windows / Redhat / Mac etc) Components are developed in difference Language Web Tools Lang OS Integrate with cutting edge Testing Tools Server is able to setup on difference Platform Web UI and Services support STAF RSpec Cucumber Selenium JavaRubyJRubyGroovy Gherkin Web ServicesRuby on Rails Chrome Firefox IE Multiple Browser support Workbot System Stack Workbot is a platform and browser independent test framework

13. Modules of Workbot Test management, report management … 13 Confidential Workbot Test Reports Management Test Cases Management Test Schedule Module User Module VM Management Test Suites Management Report Sender

14. Testing Automation Working Principle 14 Confidential

15. Technical

16. Keyword - Language and Tools What Workbot is composed of 16 Confidential Zhuhai Automation System (Workbot) Ruby Java STAF Cucumber Selenium-Webdriver Ruby on Rails

17. Three-tier Architecture How does it work? 17 Confidential Feature, written in Gherkin language (tier 3): Scenario: Clicking the add button to show the dialog Given opened 'Dashboards -> foo -> bar' screen using default credentials When I click 'add' button Then the 'Hello' dialog should be displayed Step Definition (tier 2): When /^I click '(.+)' button$/ do |name| Button.new(name, '').click end Core infrastructure (tier 1): class Button < DwrElement # the implementation end

18. Three-tier Architecture 18 Confidential The core infrastructure (or components) and step definitions are the part of the project, so they should have the same version, we don't treat the components as a library, it's the source code of the test scripts, project team is response for the scenario and the test automation team is for the step definition and component, take the following codes as example, for clicking the 'add' button, script writer says "When I click 'add' button", and when he run the scenario, the framework will pick up the step definition for this step, and execute the code. Because the step definitions and components are done by the test automation team, so script writer no need to care about how it implement (the code in red).

19. Cucumber A example of scenario (in RubyMine IDE) 19 Confidential

20. Example of Cucumber Scenario 20 Confidential Feature: support bundle generation In order to see what is going wrong and where is the problem As a administrator I want to generate a support bundle @automation @ryan @smoke @regression Scenario: generate support bundle Given opened 'Homes -> Administration' screen using default credentials Then 'Administration' screen should be displayed When I click 'Support' link Then 'Support' screen should be displayed When I click 'Generate Support Bundle' link Then 'Create Support Bundle Confirmation' dialog should be displayed When I click 'Yes' button in 'Create Support Bundle Confirmation' dialog Then 'Generate a support bundle' dialog should be displayed And 'Your support bundle was created successfully.' message should be displayed Type of tests: smoke, regression etc. Author: the name of scenario writer The tag “@automation” mean this scenario have been done and ready for execution Type of tests: smoke, regression etc. Author: the name of scenario writer The tag “@automation” mean this scenario have been done and ready for execution

21. Cucumber The component that makes automation more easier 21 Confidential I think you may ask why we need Gherkin? That's because Gherkin is a business-readable domain-specific language (DSL) for behavior-driven development (BDD), and it's business-readable for not only project owner, but also customer, developer and QA etc. Moreover, it's good for test automation, we can use a tool called Cucumber to execute plain-text functional descriptions as automated tests. And Gherkin is the language that Cucumber understands. It is a Business Readable, DSL that lets you describe software’s behavior without detailing how that behavior is implemented. Here is an example, a feature file may contains one or more scenarios.business-readable domain-specific language

22. Reporting

23. Workbot Test Result 23 Confidential Features will be used for acceptance / smoke / regression test (for more detailed information of test configuration, it will be introduced in another document), the test automation will be trigged by Jenkins and the test report will be send via email when the new build is ready.

24. Receiving Test Result 24 Confidential We receive test result everyday from mailbox

25. Receiving Test Result 25 Confidential From detailed message page, we can very easily to locate the problem by screenshot

26. Automation Work Process

27. QA Work Process Review QA use Agile process to manage the daily test activity 27 Confidential Example project for reference: https://commons.quest.com/community/monitoring/pulsenet/projects/scrum

28. Scrum Process with Test Automation QA use Agile process to manage the daily test activity 28 Confidential

29. The Over View of Current Scrum Process 29 Confidential The diagram above shows us the Scrum process that we are using currently, we discuss product backlog at the beginning of every spring to let everybody get familiar with requirement, for supporting the QA and automation join our process, the REQ (requirement in JIRA) writer need to use Gherkin language to write down requirement, fortunately, the format of Gherkin language and our REQ description is very similar.

30. Scenario Process 30 Confidential Focus on the scenario (Although it’s a failing scenario, but no worry ) Review the scenario, and see if it can be automated Writing step definition When step definition completed, it means the scenario should be easy to read and PASS in the test, then add @automation to the scenario

31. QA – Start Point 31 Confidential QA uses JIRA issue for tracking requirement as well as defect, to create relation between issue and scenario, we usually add the link to the scenario in JIRA issue: Add link to the scenario

32. Managing Test Scenario 32 Confidential https://auto-server/svn/PulseNET/trunk/features/.../Support/Ability_to_Alert_Labels_for_Roles.feature Click the link in PR and browse the scenario

33. Composing and Executing Scenarios 33 Confidential Online Scenario Editing

34. Planning

35. What we need to do next? 35 Confidential Workbot has provided us a very easy-to-use test automation environment, but it’s still not enough. For composing scenario, case writer need to spend time to setup their development environment, and moreover, it’s not easy to getting start, there are so many step they need to remember, to solve this problem, there is a application called “Recubot”, it provide a online IDE that tester no necessary to setup there develop environment, and this server will response for executing the test.

36. Dashboard of Recubot 36 Confidential This is the screenshot of Recubot, scenario writing is able to manage their test scenarios, and execute test case just by clicking the “Run” button in browser

37. Thank you! 37 Confidential