1. IBM Software Group Practical Approaches to End-to-End Automation with STAF and STAX

2. IBM Software Group Agenda  Introduction  STAF Services  STAX Service  Automation Tasks  Extending the Infrastructure

3. IBM Software Group Introduction  STAF (Software Testing Automation Framework) is a multi-platform, multi- language framework designed around the idea of reusable components, called services (which provide functionality such as process invocation, resource management, logging, and monitoring)  STAF helps you increase the efficiency, productivity, and quality of your testing by improving your level of automation and reuse in your individual testcases as well as your overall test environment  STAF removes the tedium of building an automation infrastructure, thus enabling you to focus on building your automation solution

4. IBM Software Group STAF Services  A STAF service is a reusable component which provides a focused set of functionality (such as logging)  STAF's services can be leveraged to provide end-to-end automation of your test environment. Here are some examples. EventProvides a publish/subscribe notification system EventManagerAllows you to call STAF services when a specified Event occurs CronCalls into STAF services at a specified time interval ProcessAllows you to start, stop,and query processes QueueProvides a network-enabled IPC mechanism for STAF programs MonitorAllows a testcase to publish its current running execution status for others to read LogProvides a full-featured logging facility SemaphoreProvides network-enabled named event and mutex semaphores FileSystemAllows you to get and copy files across the network, list directories, and delete files HTTPAllows you to quickly and easily make HTTP requests ResourcePoolAllows you to manage exclusive access to pools of elements, e.g. UserIDs or Software Licenses EmailAllows you to send email messages

5. IBM Software Group Introduction  STAX is an execution engine which helps you thoroughly automate your entire test environment ƒSystem Setup ƒProduct Installation ƒTestcase Distribution ƒTestcase Execution/Monitoring ƒResults Analysis ƒNotification  STAX builds on top of three existing technologies, STAF, XML, and Python, to place great automation power in the hands of testers  STAX provides a powerful GUI monitoring application which allows you to interact with and monitor the progress of your jobs

6. IBM Software Group STAX Service  The STAX Service is what drives all of your testcase automation, from beginning to end. Your end-to-end testcase automation may be comprised of a single STAX job, or multiple STAX jobs.  Features provided by the STAX Service ƒSupport for sequential and parallel execution ƒUser-defined granularity of execution control ƒSupport for nested testcases ƒAbility to control amount of execution time ƒAbility to import modules at run time ƒSupport for existing Python and Java modules/packages ƒXML-based ƒIncludes common utility modules ƒAbility to extend the STAX XML language  Features provided by the STAX Monitor GUI ƒAbility to view and interact with your entire test environment from any number of remote machines ƒDynamic updating of the GUI to reflect the current state of the job ƒAbility to extend the STAX Monitor GUI

7. IBM Software Group Automation Tasks Execution Synchronization Monitoring Resource Management Automation Completion Testcase Output Analysis Results Notification Testcase Cleanup Testcase Execution System Setup Automation Startup

8. IBM Software Group Automation Tasks - Automation Startup  Automation startup refers to the stage where your end-to-end automation begins and is usually the result of a prior piece of work, such as a product build successfully completing or by the passage of a designated time interval ƒIn the first case, the Event and EventManager services can be used to initiate STAF/STAX automation  STAF-enabled testcases can register for a certain Event, and take appropriate action when the Event (such as a product build completion) occurs  Instead of using STAF-enabled testcases to listen for these events, you can use the EventManager service to execute an arbitrary STAF command when the Event occurs  In most scenarios, the STAF command is the submission of a STAX Job which will kick off the target automation ƒIn the second case, the Cron service can be used to execute an arbitrary STAF command at a designated time interval (for example, every hour, every day at 12 midnight, every 15th day of month, etc.)  In most cases, the STAF command is the submission of a STAX Job which will kick off the target automation

9. IBM Software Group Automation Tasks - System Setup  During the setup phase, a baseline OS can be established by using Ghost/Drive images and/or VMWare images. STAF's Process Service allows you to boot these images.  Prior to testcase execution, STAF's FileSystem and Process services can be used to install the product under test and any pre-requisite software  In most cases, testcase files will need to be extracted from a server or a code repository. STAF's Process service allows testcases to be extracted from code repositories, and STAF's FileSystem service allows testcase files to be transferred from a server machine to test machines.

10. IBM Software Group Automation Tasks - Testcase Execution  Testcases are typically files (such as binary executables, Java class files, shell scripts, Perl/Python/Tcl/Rexx scripts etc.) or system commands that will be executed on a test machine  STAF and STAX allow you to start any executable file/script/command, just as if you were executing that file or command while logged into a terminal on the machine  Testcases are executed in the STAF environment by the Process service ƒThe Process service can execute testcases that are completely unaware that they are being executed by STAF/STAX, or they can leverage STAF services in order to take advantage of additional automation capabilities ƒIn your STAX job, the element is used to submit a START command to the Process service  To automate GUI testing, you would need to use tools such as SilkTest, Rational Robot, WinRunner, etc. ƒOnce you have the GUI testing automated, you would then use STAF/STAX to kick off the GUI automation tools to run the tests ƒAfter the tests complete, STAF/STAX can analyze the output to determine if the tests were successful, and take appropriate action

11. IBM Software Group Automation Tasks - Testcase Execution - Monitoring  It is desirable to have the means to easily determine the status of testcases as they are executing  Testcases can leverage the Monitor service in order to publish their current execution status ƒAs an example, a testcase that loops through a set of tests 1000 times could publish the current loop number as its latest status ƒBy using the Monitor service, you could then, from a central machine in the STAF environment, query the current status of all of your testcases ƒFurthermore, when using the STAX Monitor, all of your testcases' current status will be dynamically updated in the STAX Monitor ƒThe STAX Monitor allows you to remotely view all of your executing testcases in a single view  Throughout your STAX job, you can define blocks, which represent a portion of your job over which you would like to have runtime control ƒThe STAX Monitor allows you to hold, release, and terminate any blocks that are defined in your job

12. IBM Software Group Automation Tasks - Testcase Execution - Synchronization  Testcases may need to synchronize at certain points before proceeding with additional execution. STAF and STAX allow testcases, even when on remote machines, to synchronize via the Semaphore service.  The STAX service allows you to synchronize the completion of multiple activities via the and elements  The STAX service also allows you to place time constraints on portions of your automation jobs via the element ƒAllows cleanup of "infinite" tasks after the time limit is reached ƒAllows detection and cleanup of "hung" tasks ƒThe STAX engine handles all cleanup activities and gives you direct control over the outcome of an expired timer

13. IBM Software Group Automation Tasks - Testcase Execution - Resources  Testcases can have pools of resources that can be shared among a number of other testcases. Resources are typically a set of machines, printers, or other hardware devices to which you want to manage access. Resources can also be "logical" resources, such as software licenses or userids. ƒOne example would be if part of your test effort includes a certain software package for which you only have 5 software licenses, but you have 20 testcases that may, at some point during their execution, need to obtain one of these software licenses. The ResourcePool service could be used to set up a resource pool with 5 software licenses, from which the 20 testcases would request acquisition of one of the resources before continuing with their execution. ƒAnother example would be if you had 3 userids that your testcases could use to log into a system. You could again use the ResourcePool service to control access to the 3 userids. ƒA third example would be if you had a machine pool of 10 machines on which your testcases can execute. You could leverage the ResourcePool service to control access to the machines.

14. IBM Software Group Automation Tasks - Testcase Output Analysis  Testcases often write output to log files or to the standard output stream  When using STAF and STAX, you can have any number of files and/or the entire standard error/output stream returned when the testcase completes  Every STAF command and process returns a return code which can be used to determine the automation flow  The files and/or standard error/output stream can then be parsed through, or otherwise analyzed in STAX, and appropriate action can then be taken depending on the content of the files and/or standard error/output  Throughout your STAX job, you can define elements and corresponding elements to record testcase passes and failures  The FileSystem service can be used to check for the presence/absence of files as part of the output analysis  The FSExt service can be used to search files for particular strings as part of the output analysis  Within your STAX job, you can use Python’s Regular Expression search capabilities to analyze testcase output

15. IBM Software Group Automation Tasks - Testcase Cleanup, Results Notification  Testcase Cleanup ƒAfter your testcases have completed execution, STAF's FileSystem and Process services allow you to cleanup any temporary output and/or the testcases themselves ƒThe FileSystem service can also transfer any output or log files that have been created by the testcases to a central server  Results Notification ƒAfter your automation has completed, you can use STAF's Email service to distribute the automation results ƒSTAF's HTTP Service allows you to post forms with the automation results on the web

16. IBM Software Group End-to-End Automation with STAF and STAX Event, EventManager, Cron FileSystem, Process Process, Monitor, Log, Variable, Queue, ResourcePool, Semaphore Log, Process Process, FileSystem Email, HTTP STAXJobSTAXJob Automation Completion Results Notification Testcase Cleanup Testcase Output Analysis Testcase Execution System Setup Automation Startup

17. IBM Software Group End-to-End Automation with STAF and STAX Test systems in your lab STAX Service Machine STAX Monitor Machine - this could be your office or home system Automation Startup  Event  EventManager  Cron Log Process Monitor FileSystem ResPool Http Email Variable Queue Semaphore

18. IBM Software Group End-to-End Automation with STAF and STAX Extract source code STAX Service Machine Cron service starts build job at 1am every day WIN32 AIX Linux OS/400 z/OS Transfer code to build machines Build product on each platform Transfer product installation packages to common network location Transfer build results to project web site Transfer product binary files to ISMP machine Build ISMP packages Generate "Build Available" event

19. IBM Software Group End-to-End Automation with STAF and STAX Download product installation packages from common network location to SVT machines STAX Service Machine EventManager service receives a notification for the "Build Available" event, and kicks off the SVT job WIN32 Solaris HP-UX OS/400 z/OS Install product on SVT machines Post SVT results to project website Run SVT testcases If SVT fails, send a page to project manager Acquire SVT test machines from the machine pool

20. IBM Software Group Extending the Infrastructure  STAF ƒYou can extend the STAF infrastructure by writing you own custom STAF Services in Java, C++, or REXX ƒThese services can then be dynamically plugged into the STAF infrastructure  STAX ƒExtensions to the STAX service can be written which define new elements that can be used in a STAX xml file ƒExtensions to the STAX Monitor can be written which define plug-in views which can be displayed via the STAX Monitor

21. IBM Software Group End of Presentation