1. October 15, 2004 – 1 Welcome IPMA and SolutionsIQ Professional Event Testing, Testing, 1…2…3… Improving software quality -- one bug at a time

2. October 15, 2004 – 2 Agenda Building the Test Framework –Jan McCollum, SolutionsIQ Break Practical Panel Discussion –Cheryl Hainje – AFRS Product Manager, OFM –Dotti Lane – QA Project Manager, OFM –Tim Vessey – POS Project Manager, LCB –Stein Wang – Quality Assurance Lead, SolutionsIQ Break Testing Templates & Checklists

3. October 15, 2004 – 3 SolutionsIQ Overview SolutionsIQ is a full-spectrum IT services company 25 years of technology services and solutions 400+ consultants Corporate headquarters (Bellevue, WA) Professional Services (Bellevue, WA) Oregon Branch Office (Lake Oswego, OR) 8+ years of serving the State of WA –DOC, AOC, LCB, DNR, DOL, LNI, & DSHS

4. October 15, 2004 – 4 SolutionsIQ Expertise Professional Services Division –Consulting and Analytical Solutions Project management Assessments and feasibility studies Design and architecture roadmaps –Development and Test Solutions Full life cycle development projects Custom application development EAI, portals, and business intelligence Quality assurance and testing solutions

5. October 15, 2004 – 5 Building the Testing Framework Jan McCollum Manager, Quality Assurance and Testing Solutions

6. October 15, 2004 – 6 Setting Goals Knowing WHAT you want is as important as knowing how to get it –Defining the vision –Defining the timeline –Gaining acceptance and buy in

7. October 15, 2004 – 7 Defining the Vision To define the vision look at what came before –What went well –What went badly –What now –Where do you want to go

8. October 15, 2004 – 8 Testing vs. Quality Assurance Testing is about finding bugs Quality Assurance is about preventing them!

9. October 15, 2004 – 9 Quality Assurance Takes time Is about the overall effort – including development Methodologies can be very formal

10. October 15, 2004 – 10 QA Applied to Testing Quality assurance principals applied to the testing effort will produce higher quality work

11. October 15, 2004 – 11 Establishing a Timeline The 6 month / 1 year / 3 year plan –Implement processes and strategies that give the best return on investment

12. October 15, 2004 – 12 Quality Testing Roadmap After the goals and objectives are complete, make them real by publishing the quality testing roadmap

13. October 15, 2004 – 13 Quality Testing Roadmap Roadmap should include… –Test team structure –Communications plans –Test processes –Test procedures

14. October 15, 2004 – 14 Quality Testing Roadmap Test scope Test dependencies and impacts Automation transition plan Test deliverables

15. October 15, 2004 – 15 Gaining Acceptance and Buy In Development Business management Project management IT management Customer/product support

16. October 15, 2004 – 16 Making it Happen! Organizational structure Qualified candidates Roles and responsibilities

17. October 15, 2004 – 17 Test Planning The master test plan: a one-stop shopping guide for your project –Contents –Contributing documents –Sign-off procedures

18. October 15, 2004 – 18 Test Planning Test matrix and test suites –Detailed test steps –Pass/Fail results –Tester who performed tests

19. October 15, 2004 – 19 Test Planning Test case design – what is a good test case? –Accurate – tests what it’s designed to test –Repeatable, reusable – has a life after this release –Economical – no unnecessary steps

20. October 15, 2004 – 20 Test Planning Test case design – Traceable to a requirement – Appropriate for test environment, testers – Self-standing has enough information for anyone to run

21. October 15, 2004 – 21 Test Planning Test case design: How to make good test cases better –Setup, environment, data –Steps, actions and expected results –Use active voice in expected results –System displays this, does that –Simple, conversational language

22. October 15, 2004 – 22 Test Planning Test case design: Why work to improve test cases? –Productivity – less time to write and maintain cases –Testability – less time to execute them –Scheduling – better reliability in estimates

23. October 15, 2004 – 23 Defect (Bug) Management Deciding upon a tool –Easy of configuration –Ability to add/change fields –Reporting capabilities Integrated solution

24. October 15, 2004 – 24 Defect (Bug) Management The bug lifecycle –Who can create bugs –Who can assign bugs –Who can close bugs

25. October 15, 2004 – 25 Defect (Bug) Management The bug triage meeting –Purpose and who should go Reporting –Determining a trend Bug metrics –Number of bugs found –Bugs found in production vs. test cycle

26. October 15, 2004 – 26 Moving On Improving the process: Requirements traceability –Test cases for each requirement –Requirements matrix –Tracing requirements to defects

27. October 15, 2004 – 27 Moving On Improving the process: Risk-based testing –You can’t test everything so test what is important –The risk list and how to use it to drive test strategy

28. October 15, 2004 – 28 Broadening Your Scope Build verification testing –Also called smoke or acceptance tests –Is a subset of the major functional areas Integration testing –Testing the entire system

29. October 15, 2004 – 29 Broadening Your Scope Compatibility testing –How application works with other apps Configuration testing –Testing on different configurations Setup testing –Testing the installation Regression testing –Verify if bug fixes are successful

30. October 15, 2004 – 30 Broadening Your Scope Black box testing White box testing Grey box testing

31. October 15, 2004 – 31 Improving Quality Testing metrics – measure your success Bug tracking metrics –Number found –Number found per component –Daily bug find rate

32. October 15, 2004 – 32 Improving Quality Test case effectiveness –Metric: Test case effectiveness; test case effectiveness = bugs found in test/total found * 100 Test coverage –Metric: Test coverage (absolute) = tests conducted/total tests * 100

33. October 15, 2004 – 33 Improving Quality Test team performance –Metric: Test process effectiveness: test process effectiveness = bugs fixed/bugs found * 100 –Metric: Planned days vs. actual days in test

34. October 15, 2004 – 34 Improving Quality QA and test involvement early! Design reviews –Why testers should attend Develop and use checklists Project closeout meetings –You should have them

35. October 15, 2004 – 35 Questions? For additonal information, email –JMcCollum@SolutionsIQ.comJMcCollum@SolutionsIQ.com