1. Q Ver. 1.21 TPI at Capital One CVSQAA Meeting 11/18/03 Tom Anderson Thomas.Anderson@capitalone.com 804-967-1688 Lamar Barnett Lamar.Barnett@capitalone.com 804-934-8860

2. Q Ver. 1.22 What is TPI? TPI is the Test Process Improvement model for assessing the maturity of test processes One of several models generally accepted by testing profession Prescriptive model – Tells you how to improve, as well as describing current state 4 Levels, A, B, C & D, where D is better than A 20 different Key Process areas, with levels in each Assessment is a point-in-time, a basis for improvement For more info, read: Test Process Improvement: A Practical Step-by-Step Guide to Structured Testing by Tim Koomen and Martin Pol, ISBN: 0201596245

3. Q Ver. 1.23 TPI Model Attributes Pinpoints strong and weak points of an organization Helps formulate improvement actions Factors in life cycle, organization, infrastructure and tools, and techniques Independent of development methodology

4. Q Ver. 1.24 TPI Key Areas Key Areas Levels Improvement SuggestionsCheckpoints Test Maturity Matrix

5. Q Ver. 1.25 Types of Assessments Self-assessment Some or all of the people doing the assessment are performers of the work being assessed Internal assessment Assessors are from the same organization as the performers, but are not normally involved in the same projects Formal Assessment Assessment is performed by an independent firm Assessors usually certified in assessment model and techniques In 4Q 2003, QS Enterprise Systems Test Teams will perform Internal Assessments on each other

6. Q Ver. 1.26 TPI at Capital One Quality Services adopted TPI as the model for assessing and improving test process maturity in 2002 Many test teams were assessed by Sogeti in late 2002 Used TPI Assessment on OneView program in mid-2003 as a basis to recommend improvements Test managers are now being trained to conduct TPI assessments internally Goal is to do Internal Assessments on all teams by 1/31/04, use as basis for improvement in 2004 Model Selected Sogeti Assessments 20032004 Improvement Plans OneView Assessment Internal Assessor Training Internal Assessments Complete Improvements Based on 1Q04 Assessments Re-assess 2005

7. Q Ver. 1.27 My Experience in Process Audit ISO 9000 training TickIT Lead Auditor Duval County School Board HR Dept CIS, HTE, London Bridge process audits and improvements parts.com

8. Q Ver. 1.28 Assessing & Auditing How can I guarantee a successful assessment? Commando audit techniques Intelligence work Reporting Findings Follow up OneView example

9. Q Ver. 1.29 TPI as an Audit Model Audit for Compliance Findings – benchmarking, not strict adherence to regulations & policies Make practical recommendations to improve work life

10. Q Ver. 1.210 Audit Interviews? Why should internal audits involve interviews? –One viewpoint is not as robust. –“Reading between the lines”. Adherence to achieve compliance is through consistent, uniform practice!

11. Q Ver. 1.211 What does a successful audit interview look like?

12. Q Ver. 1.212 The Steve Irwin Model In search of the phenomenon.

13. Q Ver. 1.213 The Stephen Covey Model Begin with the End in Mind. First Things First. Think Win-Win.

14. Q Ver. 1.214 The Tom Brokaw Model Separate Fact From Perception & Feelings.

15. Q Ver. 1.215 Interview Techniques

16. Q Ver. 1.216 Interviews – Etiquette & Personalities Pre-interview – Provide an agenda & background material. Dress for success. Be punctual. Address with respect. Frame open-ended questions with charity. Avoid confrontational questioning.

17. Q Ver. 1.217 Interviews – The Offensive Game The Red Herring Obfuscation Stonewalling Disorganized / Can’t Find It Interruptions Schedule conflicts Vacillation, uncertainty, indecisiveness

18. Q Ver. 1.218 Interviews – The Defensive Game Prevent defensive game by preparing. Remain calm. Remember you can leave.

19. Q Ver. 1.219 Interview Difficulties Misunderstandings sometimes occur. –Listen carefully & actively. –Address concerns fully & repeat for feedback. –Stay grounded in facts.

20. Q Ver. 1.220 Validation of Responses Review and complete your notes soon after the interview. Refer to documentation. Chat informally with others in the group. Set up additional interviews. Phone or stop by for follow-up questions.

21. Q Ver. 1.221 Pacing the Assessment Two weeks includes interviews, research, and reporting. Total effort should be less than 40 hours. Don’t forget scheduling delay! Schedule all interviews and the end report up front!

22. Q Ver. 1.222 Tracking Results Develop working hypotheses. Prove right or wrong. Follow-up with questions & calls.

23. Q Ver. 1.223 Scoring the Assessment

24. Q Ver. 1.224 Develop the Score

25. Q Ver. 1.225 Develop the Score KA 1.A.1 “A motivated consideration of the product risks takes place…” Question: “What is your test strategy, and how did you formulate it?”

26. Q Ver. 1.226 Develop the Score

27. Q Ver. 1.227 Develop the Score

28. Q Ver. 1.228 Develop the Score

29. Q Ver. 1.229 Finalizing the Presentation Remember your audience. Should be a mere formality. Tell them more than they know. Provide actionable plan

30. Q Ver. 1.230 The Report Presentation

31. Q Ver. 1.231 Reporting Findings Management team receives report Written as well as verbal presentation No surprises

32. Q Ver. 1.232 Leveraging TPI for OneView The 20 TPI Key Areas provided concrete suggestions for improvement –Treat testing as a project, dedicate a manager –Consolidate defect reporting with a DB tool –Dedicate professional testing resources –Involve test design resources early

33. Q Ver. 1.233 Postscriptum

34. Q Ver. 1.234 The Postscriptum After the presentation, follow up will be required. Share the formal document with participants. Schedule any f/u meetings.

35. Q Ver. 1.235 The Take-Home What’s the take-home?