1. Testing Planning with Test Plan Templates HCA / Nashville, TN March 9, 2010 Sponsored by: NASQP Board

2. Nashville Association Software Quality Professionals Where IT-QA professionals come to network, to learn, and to exchange ideas. Web address http://nasqpros.ning.com

3. Agenda 5:30 PM Registration and Networking 6:00 PM Presentation Testing Planning with Test Plan Templates Test Planning within SDLC - Katherine McFarland (Vanderbilt) Traditional and COTS Test Plan Templates – John Bell ( Nissan) Test Planning for Agile – Charlie Williams (Ingenix Technology Engr.) Questions / Open discussion 7:00 PM Program Closure: NASQP Group Sign-up and Surveys

4. Overview: Test Planning within SDLC

5. Test Planning Project Rarities…NOT!

6. Test Phases: Review Unit (Component, Module, or Class) Testing This type of testing is defined as the testing of individual software components after changes have been made “White-box” testing Integration Testing At its lowest level, it is testing done by developers to check the interactions between components or modules. On the other extreme, system integration testing tests interactions between different systems and can be done after system testing.

7. Testing Definitions (cont) System (Functional) Testing Verifies that the integrated system satisfies the specified business requirements. Investigates both functional and non-functional (commonly known as the “-ilities” – reliability, portability, maintainability, usability, and efficiency) “Black-box” testing Acceptance Testing (User Acceptance Testing) Ensures that the system satisfies the pre-defined acceptance criteria and to permit the customers to determine whether or not to accept the system. Determines whether or not the system is fit for purpose and ready for deployment.

8. V-Model Diagram User Requirements System Requirements Global Design Detailed Design Implementation Component Test Execution Integration Test Execution System Test Execution Acceptance Test Execution Operational System Need, wish, policy, law Preparation Acceptance test Preparation System test Preparation Integration test

9. QA and Test Planning Early involvement If you relegate QA responsibilities to just testing the end product, you overlook an opportunity to integrate quality into the entire software development life cycle. By getting involved upfront, the QA Specialist also has time to prepare for the actual testing. Testing and requirements are synergistic The “yin and yang” of the system

10. Traditional Test Planning

11. Traditional test plans are those which are following the waterfall, waterscrum, or the “pitch it over the wall” methodologies.

12. Traditional Test Plan A test plan/strategy is a document describes the testing portion of the software development cycle. It is created to inform project managers, testers, and developers about some key issues of the testing process. In the test strategy is described how the product risks are mitigated via testing, which test types are performed, and which entry and exit criteria apply. The QA Master Test Plan defines what testing is to be conducted, who will do it and when.

13. Traditional Test Plan Templates Major elements of the traditional test plan are: Introduction Project Administration Resource Requirements In Scope and Out of Scope Requirements Test Approach Test Deliverables Dependencies/Risks Approval Log

14. Traditional Test Plan Templates Reason for the test plan: Provide clear and concise information to all stakeholders on the testing approach Defines the deliverables of the project Differentiates between “Optional” and “Required” testing Defines what “done” looks like

15. COTS Test Planning

16. Organizations, large and small, use Commercial- Off-The-Shelf software Testing COTS provides it’s own challenges especially due to unknowns and constraints Testing templates for COTS testing are the same as used for traditional testing Testing requires a slightly different mind-set since all application functionality may not be used

17. Does COTS software have defects? Is there a gap between marketing promises, management expectations and software functionality? How should we plan for installing and testing COTS software releases? What about software security, back-doors, hacks and Easter eggs? COTS Test Planning

18. Template for Testing COTS software The QA Master Test Plan - Identify testing to be conducted Identify testers Document test steps Document expected / acceptable results COTS Testing

19. Steps to conduct COTS Testing 1. Plan for install / upgrade 2. Build QA environment 3. Test the environment (smoke testing) 4. Automated testing for upgrades 5. Stress testing upgrades 6. UAT testing 7. Reporting results COTS Testing

20. Agile Test Planning

21. Agile Scrum Tester Charlie Williams February18, 2010

22. ©2006 Ingenix, Inc

24. Planning Stages 20 Points 10 Points 3 Points

25. Scrum — Project Vision Drives the Features Fix These Estimate These Features ScheduleCost ScheduleCost Features Plan Driven Value / Vision Driven The Plan creates cost/schedule estimates The Vision creates feature estimates WaterfallScrum

26. ©2006 Ingenix, Inc Centralized or Distributed Scrum Teams

27. ©2006 Ingenix, Inc The Advantages of Scrum

28. The Scrum Framework 3 Roles: Scrum Master Product Owner Delivery Team 3 Artifacts: Product Backlog Sprint Backlog Burndown Charts 5 Meetings: Release Planning Sprint Planning Daily Scrum Review / Demo Retrospective

29. ©2006 Ingenix, Inc

31. Story Points Story points are often used on agile teams to determine the complexity of the work being done. The number of points completed each sprint determines their velocity ( points per sprint ) and gives management an approximation on how much work can be completed in a given sprint. ©2006 Ingenix, Inc

32. Estimation In an agile software team you don't estimate your work till right before you begin. And you only estimate, in the case of Scrum, the next sprints work. So how do you know how long it will take? You don't. Furthermore, you really don't care. You'll continue to deliver functionality every sprint. As soon as product management and QA say you have a good enough product; it'll be released as a production version. You might have a guess, but until the team estimates it....you really don't know long it will take. ©2006 Ingenix, Inc

34. Tester Activities

35. ©2006 Ingenix, Inc

39. Exploratory Testing Learning Areas for Story Testing : 1.Usage Scenarios 2.Capabilities / Confirming 3.Across Stories Relationships 4.Creative Ideas 5.Failure Modes 6.Quality Factors “As a Tester I want to confirm that …, So that …” Type of Test Case Stories

40. ©2006 Ingenix, Inc Test Automation Pyramid

41. ©2006 Ingenix, Inc Automate Non-Functional Testing Performance Testing Load / Stress Testing Security Testing Scalability Testing Data Migration Infrastructure Testing

42. ©2006 Ingenix, Inc

43. Testing/Coding: Don’t sit and wait! Is any testable part of a story ready? - Test with behind-the-GUI tool such as FIT? - Or other harness to bypass GUI Pair with programmers - Test together before check-in - Show them issues - Bugs found here are cheap & easy to fix Copyright 2007: Lisa Crispin and Janet Gregory

44. Automation Tools Automation can be accomplished by a broad range of technologies. Effective automation starts by focusing on what is to be automated/tested and then applies the appropriate tools and approaches – Common automation options include: Automated acceptance test frameworks (e.g. FIT, StoryTeller, or Cucumber, Business Process Test) Technically oriented automation tools (e.g. SoapUI, SOATest, STaF) Functional Automation tools (QuickTest Pro) Scripting Languages (Ruby, Python/IronPython, Perl, VBScript) Performance Testing Tools (e.g. JMeter, The Grinder, LoadRunner) © Ingenix, Inc. 44

45. ©2006 Ingenix, Inc Sprint Documentation The Stories, Code, and Test Cases that are written in the Sprint are the source of documentation. It is NOT true that Scrum has no documentation.

47. All stories within a sprint must be defect free in order to be considered "done" Any defects that are found during a sprint for a previously completed story are logged and put into the backlog. They're estimated just like anything else and prioritized by the product owner. If a product owner prioritizes new features over defects, they're choosing functionality over quality and vice-versa.

48. Sample Definition of Done Working Software Unit tested Code Review Tests (Test Strategy/Tollgate) Functional Testing Product Owner Acceptance Performance/Load Tests Regression tests (Automated preferably) Documentation User documentation/Help Design Release Notes API documentation Statement of work

49. Sprint Demo and Review Team reviews what it accomplished during the Sprint Typically takes the form of a demo of delivered product backlog items and/or underlying architecture and a review of metrics and data Informal – 2 hour prep time rule Attendees – Team (Customer Representative, Developers, Testers, Architect, etc.) – ScrumMaster – Product Owner, Customers – Stakeholders – All other interested parties A PowerPoint is not a demo!

50. Sprint Retrospective Inspection of process and team practices at the end of every Sprint Attended only by the delivery team Facilitated by ScrumMaster or third party What went well, what could be improved ScrumMaster prioritizes improvements based on team direction Team devises solution to most vexing problems If there are many ideas, create an “Implementation Backlog” with ranked ordering of recommendations

51. ©2006 Ingenix, Inc Ready ? Jump In Thank You

52. ©2006 Ingenix, Inc

53. Why Agile Adoption Fails in Some Organizations Posted by Christopher Goldsbury on Nov 04, 2009Christopher Goldsbury Agile Testing A Practical Guide For Testers and Agile Teams By Lisa Crispin and Janet Gregory

54. Open forum for additional questions, comments, etc. Next Event : Test Planning Workshop April 2010

55. Thank you for coming. In an effort to make these meeting useful and beneficial to all, please complete the survey on our home page. http://nasqpros.ning.com