1. Designing Reusable Frameworks for Test Automation
Presenter: Byron Goodman
National Director of Quality Assurance
Neudesic

2. Overview Some thoughts on automation Successful test automation
Components of a framework
Some coding standards
Standard routines

3. Automated Testing: Why We Love It
Let the machines do the tests
Easy to create alternate scenarios by leveraging existing tests
Using tools can create more interest in testing activities
Potential to reduce testing cycle time
Effective at catching regression bugs

4. Automated Testing: Why We Hate It
Tests break too often!
Pass or fail results may not be reliable
Tools are expensive, although there are some good open source solutions
Tools may oversimplify or overcomplicate the degree of effort required to test applications

5. Have you Experienced Any of These Symptoms?
More time spent working on automated tests than time needed to do the manual testing
Sporadic interest in implementing automated testing
Statements by others that automated testing is: a sham, ineffective, waste of time
Shelving of test automation in the interest of project success

6. What is Successful Test Automation?
As with all project related work, the time spent creating automated testing must offset this effort by the return of value. This is demonstrated in various ways:
Less time expended during the testing cycle than if manual testing is the only solution
More thorough and consistent regression coverage
Reduce human error and boredom
Frees up people to test new functionality while computers test existing functionality

7. Basics of Beginning
Strategy – start simple and work towards a more complete and complex solution
Timing – start using tests as soon as they are written, Do not wait for a completed suite
Buy-in – Demonstrate to others that each test does exactly what it is intended to so that there is confidence in results
Traceability – Each automated test must trace back to a specific automated test

8. Start Somewhere
Generally speaking, some automated testing is better than no automated testing
Select the most simple of tests to automate
Begin by using record and playback to minimize time investment but to show that the tool does prove to be effective

9. Record and Playback
For years, I have insisted that no tests will be created using record and playback
I will admit that there is a place for R&P
Very light use to prove out the tool and to minimize the reluctance people may have to use complicated technology
However, R&P is does not have much gain in efficiencies over manual testing

10. Testing Framework Many of us have experience with testing frameworks
Other names– DLLs, namespace, test library
Its just a fancy name for code that is used to automate tests
A testing framework is a set of routines that are reused by tests that will speed up test development while increasing test reliability

11. Testing Framework Goals
Needs to be recognized to have the same rigor as any development effort
Adds value to the overall test effort
Implements complex and redundant routines for tests to leverage
Minimizes the impact of a changing application to the tests, which reduces test maintenance

12. Layers & Design
Each layer is developed to promote portability and reusability
The more generic a layer, the more reusable it needs to be
i.e. the Generic Routines should be able to work for everyone all the time regardless of the environments and AUTs

13. General Data Flow

14. Major Components of a Framework
Common routines that do not interact with any application
Generic UI layer, non-technology-specific manipulation routines
Technology-specific layer routines
Application-specific layer routines
Application-specific Scenario/transactional layer routines
Tests layer

15. Common routines that do not interact with any application
Not related to the application or an interface
Build more capability and utility that the test tool may not have
Functionality that a tester may need for any reason
If written properly, these routines will work with test tools that use the same programming language

16. Generic UI layer routines
Some control types, like grids, that are generic and rather independent from the technology that they are written in
This is a good place to create additional or alternate logging

17. Technology-specific layer routines
For handling specific technologies – i.e. Browsers, WinForms and WPF.
Custom controls will be wrapped with calls that are identical to similar standard controls
Create interfaces for new versions of technology if the tool vendor does not yet have the new technology versions integrated

18. Application-specific layer routines
Routines that are specific to the AUT
Repository for UI mappings
Defines a handler for every individual form/page
AUT service interfaces defined and wrapped
Serves the calls needed for tests to interact with the AUT

19. Application-specific Scenario/transactional layer
Use this as a distinguishable layer if the routines for scenarios are complex and extensive
May include multiple forms/pages
In cases where there will not be a lot of routines, this can be combined into the Application-specific manipulation routines

20. Tests layer This layer will contain only tests
No test calls any other test
Tests call into the other layers
Tests should be very simple as all the complexity is implemented in the other layers

21. General Data Flow – Strictly Enforce

22. Standards
Using consistent coding standards promoted reusability and maintainability
Standards need to be documented and followed similar to what project development would do
Use code reviews to enforce standards

23. Typical Coding Standards
Naming conventions for variables and methods/functions
Use human readable names!
Limit code complexity
Separation of layers
Comments

24. Automation Coding Standards
Do not use the control type in the name, Address, not AddressEdit
No duplication of control names for any specific window if the tools would normally allow this
Consistent naming of similar controls form to form, even if the forms label the controls differently

25. Automation Coding Standards
Execute anywhere – tests need to written so that they do not rely upon a very specific environment configuration, unless that is the purpose of the test
Unattended testing – tests that need human interaction to execute should be segregated so that the other tests can all be run unattended

26. Automation Coding Standards
No hard coding – use configuration files or similar mechanisms to store information or environment values
In tests, avoid using the automation tool syntax
Tests should call into the other layers
This will make tests less likely to fail if the tool vendor modifies their syntax
Can make it easier to port tests to new automation tools

27. Automation Coding Standards
Overuse logging
Design for distributed testing and for concurrent execution of tests
An automated test will test just a single test
No test ever depends upon the successful actions or completion of a previous test
Computers don’t care if they do repetitive actions

28. Standard Routines Baseline (Initialization) Navigation Editing
Verification
Tools already have their own routines defined for some of this – however, you will get more flexibility and maintainability using these patterns

29. Baseline Will set the environment to the proper state to start a test
This state is defined for each AUT and test team
Can be simple to very complex
Determined by defining a known state that all tests can reliably execute and get consistent results

30. Navigation
Used to determine if a page/form is active and ready for interaction
Will navigate to the desired page/form if the steps required are not complex, or the steps will not alter the outcome of any test
Every page/form will have an unique navigation routines created

31. Editing
Enters data into fields, pushes buttons, sets checkboxes, selects values in lists, etc.
The control types should not matter when calling this routine – it will figure out what the control type is during test execution and interact with the controls appropriately
Defined as an unique routine for every individual page/form

32. Verification Reads data from all controls types
Determines if the values read result in a pass or fail
Responsible for logging results
Defined as an unique routine for every individual page/form

33. Areas we did not cover Solution layout Actual code examples
Dealing with particularly difficult controls
Configuration management
Skill sets
Specific tools’ capabilities

34. What we did cover Beginning automated tests Test framework goals
Layers of a framework
Coding standards
Initial routines that every page/form requires

35. Questions?
Thanks for your participation