1. Software Development Conference 2007 IBM RATIONAL TOOLS FOR TESTING Presentation: Vítek Linduška Demo: Václav Novotný

2. Software Development Conference 2007 2 Agenda  Manual Tester  Functional Tester  Performance Tester  ClearQuest

3. Software Development Conference 2007 3 Manual Tester

4. Software Development Conference 2007 4 Why Teams use Manual Testing? Recruitment of subject-matter experts and end-users to test Poor or non-existent test automation-ready interfaces Strict regulatory or process requirements aren’t met with automation tools

5. Software Development Conference 2007 5 Challenges of Manual Testing  Labor Intensive  Too much time is spent creating and maintaining manual test documentation  Labor Intensive  Too much time is spent creating and maintaining manual test documentation “More time is spent maintaining test documentation than on executing tests.” “We struggle to document even a minimum set of tests.”

6. Software Development Conference 2007 6 Challenges of Manual Testing  Labor Intensive  Too much time is spent creating and maintaining manual test documentation  Unreliable  Inadvertent human-error often skews test results  Labor Intensive  Too much time is spent creating and maintaining manual test documentation  Unreliable  Inadvertent human-error often skews test results “Our results are unreliable due to human error.” “Our tests aren’t detailed enough for the testers to consistently execute them.”

7. Software Development Conference 2007 7 Challenge 1: Labor Intensive  Challenge  Creating and maintaining manual tests is labor intensive and difficult with tools such as Microsoft Word or Excel.  Challenge  Creating and maintaining manual tests is labor intensive and difficult with tools such as Microsoft Word or Excel.  Resolution  Implement a solution that promotes the best practice of modular tests which can share common content  Leverage existing Word or Excel test documents to quick- start test development  Resolution  Implement a solution that promotes the best practice of modular tests which can share common content  Leverage existing Word or Excel test documents to quick- start test development Share test content

8. Software Development Conference 2007 8  Challenge  Human error often skews tests results  Challenge  Human error often skews tests results  Resolution  Assist testers with the tasks most prone to error – data entry and data verification  Build manual tests that are clear, concise and easy for the tester to follow  Resolution  Assist testers with the tasks most prone to error – data entry and data verification  Build manual tests that are clear, concise and easy for the tester to follow Challenge 2: Unreliable

9. Software Development Conference 2007 9 Leverage Existing Test Documents Import from Word or Excel Microsoft Word documents Microsoft Excel spreadsheets  Jump-start adoption of Rational Manual Tester by leveraging existing assets  Bulk import manual tests documented in Word or Excel  Jump-start adoption of Rational Manual Tester by leveraging existing assets  Bulk import manual tests documented in Word or Excel Rational Manual Tester test

10. Software Development Conference 2007 10 Assisted Data Entry and Data Verification Reduces human error  Store critical data used during the test as part of the test instructions Test execution window

11. Software Development Conference 2007 11 Assisted Data Entry and Data Verification Reduces human error  Store critical data used during the test as part of the test instructions  Automate data entry during test execution to speed entry and reduce errors  Store critical data used during the test as part of the test instructions  Automate data entry during test execution to speed entry and reduce errors Test execution window Application under test

12. Software Development Conference 2007 12 Assisted Data Entry and Data Verification Reduces human error  Store critical data used during the test as part of the test instructions  Automate data entry during test execution to speed entry and reduce errors  Automate data validation during test execution to ensure accurate results  Store critical data used during the test as part of the test instructions  Automate data entry during test execution to speed entry and reduce errors  Automate data validation during test execution to ensure accurate results

13. Software Development Conference 2007 13 Enables Shared Content Across Multiple Tests Reduces test maintenance and ensure consistency  Create commonly used test statements once  Reuse statements across multiple tests through simple drag and drop  Maintenance of shared content is efficient with single-point updating of shared content  Ensures consistency and reduces test maintenance  Create commonly used test statements once  Reuse statements across multiple tests through simple drag and drop  Maintenance of shared content is efficient with single-point updating of shared content  Ensures consistency and reduces test maintenance

14. Software Development Conference 2007 14 On-Screen Prompted Test Execution Immediate and accurate results collection  On-screen prompting of test steps  Immediately record test results and attach images or files for clarity  Tool will assist with data entry and data verification  Results recorded in permanent log  On-screen prompting of test steps  Immediately record test results and attach images or files for clarity  Tool will assist with data entry and data verification  Results recorded in permanent log

15. Software Development Conference 2007 15 Spread-Sheet Ready Test Results Use with any 3 rd party reporting tool  Results easily exported to common file format  Import into any 3 rd party tool for evaluation and consolidation with other test data  Results easily exported to common file format  Import into any 3 rd party tool for evaluation and consolidation with other test data

16. Software Development Conference 2007 16 Functional Tester

17. Software Development Conference 2007 Manual testing can’t keep pace! Rerun the first tests and... Time Tests...take more time...the new tests... Build 1Build 2Build 3, 4, 5 … Challenge of Manual Testing with Short Test Cycles  Manual testing alone can’t keep pace with the rapid development of applications  Quality will be sacrificed to keep on schedule

18. Software Development Conference 2007 18 Challenges of Functional Testing  Controlling cost of test automation  Inexperienced testers face steep learning curve for tools  Test maintenance outpaces testing  Controlling cost of test automation  Inexperienced testers face steep learning curve for tools  Test maintenance outpaces testing “Our testers are not experienced in building automated tests.” “Application UI changes are always breaking our test scripts.”

19. Software Development Conference 2007 19 Challenges of Functional Testing  Control cost of test automation  Inexperienced testers face steep learning curve for tools  Test maintenance grows with project  Testing Tools Lack Flexibility  Power users hands are tied by tool limitations  Control cost of test automation  Inexperienced testers face steep learning curve for tools  Test maintenance grows with project  Testing Tools Lack Flexibility  Power users hands are tied by tool limitations “I need one tool for both my expert and novice testers.” “Our script language limits our ability to overcome some automation obstacles.”

20. Software Development Conference 2007 20 Challenges of Functional Testing  Control cost of test automation  Inexperienced testers face steep learning curve for tools  Test maintenance grows with project  Testing Tools Lack Flexibility  Power users hands are tied by tool limitations  Test Team Integration  Managing script changes with parallel development  Sharing test assets with the larger software team  Control cost of test automation  Inexperienced testers face steep learning curve for tools  Test maintenance grows with project  Testing Tools Lack Flexibility  Power users hands are tied by tool limitations  Test Team Integration  Managing script changes with parallel development  Sharing test assets with the larger software team “How do I keep my team in connected with the project team?” “We’re geographically distributed and we’re worried about parallel script development.”

21. Software Development Conference 2007 21  Challenge  Overcome the complexity of test automation to ensure teams spend more time on high value quality assessment rather than on low return test maintenance.  Challenge  Overcome the complexity of test automation to ensure teams spend more time on high value quality assessment rather than on low return test maintenance.  Resolution  Provide technologies to enable novice testers to craft complex tests with no programming  Data driven wizard  Dynamic data verification  Reduce script maintenance  Wizard driven global updates to centralized object map  Resolution  Provide technologies to enable novice testers to craft complex tests with no programming  Data driven wizard  Dynamic data verification  Reduce script maintenance  Wizard driven global updates to centralized object map Challenge 1: Controlling the cost of Automation

22. Software Development Conference 2007 22  Challenge  Fulfill both standard test needs as well as expert requirements to ensure usage limited only by the imagination of the project team.  Challenge  Fulfill both standard test needs as well as expert requirements to ensure usage limited only by the imagination of the project team.  Resolution  Provide two test script development languages  Java  Visual Basic.NET  Give professional testers the flexibility to overcome testing challenges in the language of their choice  Resolution  Provide two test script development languages  Java  Visual Basic.NET  Give professional testers the flexibility to overcome testing challenges in the language of their choice Challenge 2: Tools Lack Flexibility

23. Software Development Conference 2007 23 Challenge 3: Test Team Integration  Challenge  Accommodate the requirements of both dedicated, independent QA teams as well as integrated project teams to ensure high operational integrity.  Challenge  Accommodate the requirements of both dedicated, independent QA teams as well as integrated project teams to ensure high operational integrity.  Resolution  Provide version control support in the box with ClearCase LT  Provide integrations for standard version control features  Check In/Out  Compare with Previous Versions  Show change history  Resolution  Provide version control support in the box with ClearCase LT  Provide integrations for standard version control features  Check In/Out  Compare with Previous Versions  Show change history

24. Software Development Conference 2007 24 Reduce Test Script Maintenance Reliable Playback with Script Assure Determines Match Version 1.0Version 2.0 Tester Sees Tool Sees No User Intervention Required With ScriptAssure TM

25. Software Development Conference 2007 25 Performance Tester

26. Software Development Conference 2007 26 Challenges of Performance Testing  Complex Tools & Complex Task  Tool complexity and lack of experience intimidates many first time users  Complex Tools & Complex Task  Tool complexity and lack of experience intimidates many first time users “We don’t have a lot of experience building good load tests.” “Most load test tools are much more complicated than we require.”

27. Software Development Conference 2007 27 Challenges of Performance Testing  Complex Tools & Complex Task  Tool complexity and lack of experience intimidates many first time users  Testing Tools Lack Flexibility  Power users hands are tied by tool limitations  Complex Tools & Complex Task  Tool complexity and lack of experience intimidates many first time users  Testing Tools Lack Flexibility  Power users hands are tied by tool limitations “We don’t have a good understanding of what exactly is occurring during a load test.” “We have highly technical performance testers that need to customize their tests.”

28. Software Development Conference 2007 28 Challenge 1: Complex Tools & Complex Task  Challenge  Tool complexity and lack of experience intimidates many first time users  Challenge  Tool complexity and lack of experience intimidates many first time users  Resolution  Remove the need for programming  Tests are represented in a tree view as a sequential flow web pages  User interface enables  Data pooling  Data correlation  Looping  Conditional events  Resolution  Remove the need for programming  Tests are represented in a tree view as a sequential flow web pages  User interface enables  Data pooling  Data correlation  Looping  Conditional events

29. Software Development Conference 2007 29  Challenge Power users hands are tied by tool limitations  Challenge Power users hands are tied by tool limitations  Resolution  Tests may call custom Java code provided by the user  Offers flexibility for power users to perform complex tasks  Expose behind the scenes details which are hidden from novice users by default  Provide the flexibility to overcome any testing obstacles  Resolution  Tests may call custom Java code provided by the user  Offers flexibility for power users to perform complex tasks  Expose behind the scenes details which are hidden from novice users by default  Provide the flexibility to overcome any testing obstacles Challenge 2: Tools Lack Flexibility

30. Software Development Conference 2007 30 Performance Problem Identification During Test  Performance Testing finds bottlenecks  Next logical question is Why?  Root Cause Analysis provides to tools to answer this question Page Performance Report Shows average response time per page. Highest bar = Performance Problem

31. Software Development Conference 2007 31 Performance Problem Identification During Test  Performance Testing finds bottlenecks  Next logical question is Why?  Root Cause Analysis provides to tools to answer this question

32. Software Development Conference 2007 32 Response Time Breakdown  Feature:  Breakdown page response times into composite element response time  Benefit:  Breakdown data drills down to highlight slowest components of a given page. Breakdown into tiers and components is a unique competitive advantage for RPT. Response data broken down by tier and by transaction component (JDBC, JSP, Servlet, etc…)

33. Software Development Conference 2007 33 Deep Diagnostic Data Providing the details you need to solve the performance problem mystery UML Sequence diagram showing method calls between classes with timing information. User can drill down to source code. Execution Statistics Shows response time for every method

34. Software Development Conference 2007 34 ClearQuest

35. Software Development Conference 2007 35  Eclipse Based Test Management  Joins Functional and Performance Tester to complete transition of testing tools to eclipse platform  Native and web clients also available  Key Functional Areas  Test Planning  Create test plans, test cases and configured test cases  Test Execution  Create suites or run individual test cases  Test Reporting  Extensive query and reporting facilities to measure test progress Test Management with ClearQuest Eclipse, Windows and Web Clients for test project tracking

36. Software Development Conference 2007 36 Lifecycle Traceability Query one source for all aspects of project progress  Query to find  Test results associated to requirements  Defects associated to test results  Defects associated to tests associated to requirements  Traceability from requirements to tests and defects closes the loop on the software development cycle Requirements Tests Defects

37. Software Development Conference 2007 37 Test Case Trending Test Verdicts Lifecycle Traceability Query one source for all aspects of project progress Leverage existing reports or Create new reports with Crystal Reports To provide the information you need for decision making

38. Software Development Conference 2007 38 ClearQuest Web Interface Distributed access for test planning and reporting  Instant access for remote users  Access for alternative platforms  No additional licenses required  Perfect for test planning and reporting

39. Software Development Conference 2007 39 ` ClearQuest MultiSite Replication and synchronization for geographically distributed teams  Automatic replication and synchronization of ClearQuest repositories  Access current information, regardless of location  Automatic recovery of repositories in the event of system failure maintains data integrity  Transmission of only incremental changes for improved network performance

40. Software Development Conference 2007 40 Centralized User Management LDAP authentication for common user id support Reduce administration with centralized user management – maintain only one list of user ids LDAP authentication supports common user ids for ClearQuest and RequisitePro Rational RequisitePro Authorization Rational RequisitePro Rational ClearQuest LDAP 3.0 Compliant Server Rational ClearQuest Authorization Rational ClearQuest User DB Logon Dialog LDAP

41. Software Development Conference 2007 41 Compliance and Control Using Audit Trails and eSignatures to track change  Audit Trails can track changes to individual records  Provides immutable change log of all record change activity  e-Signature requirement can be added to any event on any artifact  Will record who changed a record, when and how the record was changed  e-Signature provides immutable evidence of process completion Validate Results Test Execution Test CaseRequirementApprove e-Signature Approval

42. Software Development Conference 2007 42 Configure and Enforce Your Process Customize appearance and behavior of all forms and processes  Customize everything  Modify forms  Configure State Transition  Add fields and control behavior  Extend your system with custom coded hooks  Extend functionality using pre-defined Packages  eSignatures  Audit Trail  Many, many more…

43. Software Development Conference 2007 43 Extensible Test Ecosystem Integrated with Rational, open source and 3rd party test automation tools  Collect and report on results from Rational test execution engines  Rational Manual Tester  Rational Functional Tester  Rational Performance Tester  Collect and report on results from Eclipse Test and Performance Tools Project  jUnit tests  Manual tests  URL tests  Open API Extensibility  SPI Dynamics QAInspect

44. Software Development Conference 2007 44 Rational Software Quality Solutions Developer Test Functional Test AutomatedManual Rational RequisitePro Rational ClearQuest Defects Project Dashboards Detailed Test Results Quality Reports Performance Test SOFTWARE QUALITY SOLUTIONS Test and Change Management Test Automation Quality Metrics DEVELOPMENT OPERATOINS BUSINESS Rational ClearQuest Requirements Test Change Rational PurifyPlus Rational Test RealTime Rational Functional Tester Plus Rational Functional Tester Rational Robot Rational Manual Tester Rational Performance Tester

45. Software Development Conference 2007 45 WAKE UP, IT’S DEMO TIME!