Exploring an Open Source Automation Framework Implementation

Journey of Test Automation Record and Playback Test Script Modularity creating small, independent scripts that represent modules, sections, and functions of the application-under-test combine them in a hierarchical fashion to construct larger tests Test Library Architecture divides the application-under-test into procedures and functions (or objects and methods depending on your implementation language) instead of scripts requires the creation of library files that represent modules, sections, and functions of the application-under-test library files are then called directly from your test case scripts Data-Driven Testing test input and output values are read from data files and are loaded into variables in captured or manually coded scripts navigation through the program, reading of the data files, and logging of test status and information are all coded in the test script Keyword-Driven Testing the entire process is data-driven, including functionality requires the development of data tables and keywords

Data Driven Testing Driver Scripts Perform initialization (if required), then call the Test Case scripts in the desired order Test Case Scripts Perform application test case logic using Business Function scripts Business Function Scripts Perform specific Business Functions within the application Subroutine Scripts Perform application specific tasks required by two or more Business scripts User-defined Functions General, Application-specific, and Screen-access functions

Keyword-Driven Testing Driver Script Performs initialization, if required Calls the Application-Specific “Controller” Script, passing to it the file-names of the Test Cases (which have been saved from the spreadsheets as a “tab-delimited” files) The “Controller” Script Reads and processes the file-name received from Driver Matches on “Key Words” contained in the input-file Builds a parameter-list from the records that follow Calls “Utility” scripts associated with the “Key Words”, passing the created parameter- list Utility Scripts Process input parameter-list received from the “Controller” script Perform specific tasks (e.g. press a key or button, enter data, verify data, etc.), calling “User Defined Functions” if required Report any errors to a Test Report for the test case Return to “Controller” script User Defined Functions General and Application-Specific functions may be called by any of the above script- types in order to perform specific tasks

WRAFS Architecture Application Under Test Application Map – WRAFS GUI MAP CycleDriver (TSL Script) WRAFS DRIVER Suite Driver (TSL Script) Step Driver (TSL Script) Suite Tables (.STD) WRAFS ENGINE Component Functions (TSL SCRIPT) Support Libraries (TSL SCRIPT) Suite Tables (.STD) Cycle Tables (.CDD) Step Tables (.SDD) START CYCLE TEST Step Tables (.SDD)

High Level Overview The SAFS framework consists of 5 layers. In order from the bottom up: Component Function Modules – Generic independent modules of code controlling functions for different objects Maps – Files that link given names to automation tool names for all components on each screen Step Tables – Low level tables linking data variables and actions to objects Suite Tables – Higher level tables assigning values to variables used in Step Tables Cycle Tables – The Highest level of abstraction, used to link Suite Tables (test cases) to form a complete test cycle to be executed

Cycle Table Cycle Table: VerifyAuthenticationFunction KEYWORDS (Suite Tables)TABLE PURPOSE VerifyInvalidLogin Tests with Invalid UserID and/or Password VerifyBlankLogin Tests with Missing UserID and/or Password VerifyValidLogin Tests with Valid UserID and Password

Cycle Driver Cycle Tables combine intermediate-level Suites into Cycles. The Suites can be combined in different ways depending upon the testing Cycle we wish to execute (Regression, Acceptance, Performance…). Each Cycle will likely specify a different type or number of tests. Different cycles are defined to exercise different modules or components of an application A regression test cycle might select an appropriate subset of suite tables from each of the other test cycles CycleDriver reads each record from the Cycle Table, passing SuiteDriver each Suite Table it finds during this process.

Cycle Driver Control Flow Execution starts from StartCycleTest Script All Libraries are loaded in DDEngine script CycleDriver.CDD file is opened Read the next record if it is present else execution is completed If 1 st field ; or # If 1 st field “T” If 1 st field “C” Goto DriverCommands Goto SuiteDriver Skip the Line Complete the Suite Process Execute the Command, set return codes, print Log messages

Suite Table Suite Table: VerifyInvalidLogin KEYWORDS (Step Tables)USERIDPASSWORD LaunchSite LoginBadUserGoodPassword VerifyLoginError LoginGoodUserBadPassword VerifyLoginError LoginBadUserBadPassword VerifyLoginError ExitLogin

Suite Table Suite Table: VerifyBlankLogin KEYWORDS (Step Tables)USERIDPASSWORD LaunchSite Login"" VerifyLoginError ExitLogin Suite Table: VerifyValidLogin KEYWORDS (Step Tables)USERIDPASSWORD LaunchSite LoginGoodUserGoodPassword VerifyLoginSuccess ShutdownSite

Suite Driver Suite Tables tables typically combine Step Tables into Suites in order to perform more useful tasks. The same Step Tables may be used in many Suites. We then mix-and-match them in Suites according to the purpose and design of our tests, for maximum reusability. SuiteDriver reads each record from the Suite Table, passing StepDriver each Step Table it finds during this process.

Suite Driver Control Flow FileName.STD file is opened Read the next record if it is present else give control back to the CycleDriver If 1 st field ; or # If 1 st field “T” If 1 st field “C” Goto DriverCommands Goto StepDriver Skip the Line Execute the Command, set return codes, print Log messages Complete the Step Process SuiteDriver

Step Table Step Table: LaunchSite COMMAND/DOCUMENTARG/COMPONENTACTION EXPECTED RESULTS LaunchBrowserDefault.htm Browser VerifyCaption"Login" Step Table: Login &user &password DOCUMENTCOMPONENTACTIONINPUT DATA LoginPageUserIDInputText&user LoginPagePasswordInputText&password LoginPageSubmitButtonClick Step Table: VerifyLoginError DOCUMENTCOMPONENTACTION EXPECTED RESULT ErrorWin VerifyCaption"Error" ErrorWinErrorMessageVerifyText"The User ID or Password was invalid." ErrorWinOKButtonClick

Step Table Step Table: ExitLogin DOCUMENTCOMPONENTACTION LoginPageCancelButtonClick Step Table: VerifyLoginSuccess DOCUMENTCOMPONENTACTION EXPECTED RESULT HomePage VerifyTitle"Home" Browser VerifyCaption"Welcome" Step Table: ShutdownSite DOCUMENTCOMPONENTACTIONINPUT DATA Browser SelectMenuItemFile>Exit

Step Driver Step Tables contain the detailed step-by-step instructions of our tests. Using the object names found in the Application Map, and the vocabulary defined by the Component Functions; these tables specify what document, what component, and what action to take on the component. StepDriver parses records in the Step Table and performs some initial error detection, correction, and synchronization making certain that the document and\or the component we plan to manipulate are available and active. StepDriver then routes the complete instruction record to the appropriate Component Function for final execution.

Step Driver Control Flow FileName.SDD file is opened Read the next record if it is present else give control back to the calling SuiteDriver If 1 st field ; or # If 1 st field “T” If 1 st field “C” Goto DriverCommands Check the ObjectType of the component Skip the Line Execute the Command, set return codes, print Log messages Goto the respective class of the component StepDriver Execute the called wrapper function Set proper return status and print corresponding Log Messages

Component Functions Component Functions are those functions that actively manipulate or interrogate component objects. Different Component Function module for each type of component we encounter Component Function modules are the application-independent extensions applied to the functions already provided by the automation tool with added extra code to help with error detection, error correction, and synchronization. The component Functions takes the windows name in which the component resides, the actual component name on which the action is to be performed, the values needed for performing the action and the type of action to be performed as its arguments. Each Component Function modules will define the keywords or "action words" that are valid for the particular component type it handles. The Component Function keywords and their arguments define the low-level vocabulary and individual record formats will be used to develop the test tables.