Q Ver TPI at Capital One CVSQAA Meeting 11/18/03 Tom Anderson Lamar Barnett
Q Ver What is TPI? TPI is the Test Process Improvement model for assessing the maturity of test processes One of several models generally accepted by testing profession Prescriptive model – Tells you how to improve, as well as describing current state 4 Levels, A, B, C & D, where D is better than A 20 different Key Process areas, with levels in each Assessment is a point-in-time, a basis for improvement For more info, read: Test Process Improvement: A Practical Step-by-Step Guide to Structured Testing by Tim Koomen and Martin Pol, ISBN:
Q Ver TPI Model Attributes Pinpoints strong and weak points of an organization Helps formulate improvement actions Factors in life cycle, organization, infrastructure and tools, and techniques Independent of development methodology
Q Ver TPI Key Areas Key Areas Levels Improvement SuggestionsCheckpoints Test Maturity Matrix
Q Ver Types of Assessments Self-assessment Some or all of the people doing the assessment are performers of the work being assessed Internal assessment Assessors are from the same organization as the performers, but are not normally involved in the same projects Formal Assessment Assessment is performed by an independent firm Assessors usually certified in assessment model and techniques In 4Q 2003, QS Enterprise Systems Test Teams will perform Internal Assessments on each other
Q Ver TPI at Capital One Quality Services adopted TPI as the model for assessing and improving test process maturity in 2002 Many test teams were assessed by Sogeti in late 2002 Used TPI Assessment on OneView program in mid-2003 as a basis to recommend improvements Test managers are now being trained to conduct TPI assessments internally Goal is to do Internal Assessments on all teams by 1/31/04, use as basis for improvement in 2004 Model Selected Sogeti Assessments Improvement Plans OneView Assessment Internal Assessor Training Internal Assessments Complete Improvements Based on 1Q04 Assessments Re-assess 2005
Q Ver My Experience in Process Audit ISO 9000 training TickIT Lead Auditor Duval County School Board HR Dept CIS, HTE, London Bridge process audits and improvements parts.com
Q Ver Assessing & Auditing How can I guarantee a successful assessment? Commando audit techniques Intelligence work Reporting Findings Follow up OneView example
Q Ver TPI as an Audit Model Audit for Compliance Findings – benchmarking, not strict adherence to regulations & policies Make practical recommendations to improve work life
Q Ver Audit Interviews? Why should internal audits involve interviews? –One viewpoint is not as robust. –“Reading between the lines”. Adherence to achieve compliance is through consistent, uniform practice!
Q Ver What does a successful audit interview look like?
Q Ver The Steve Irwin Model In search of the phenomenon.
Q Ver The Stephen Covey Model Begin with the End in Mind. First Things First. Think Win-Win.
Q Ver The Tom Brokaw Model Separate Fact From Perception & Feelings.
Q Ver Interview Techniques
Q Ver Interviews – Etiquette & Personalities Pre-interview – Provide an agenda & background material. Dress for success. Be punctual. Address with respect. Frame open-ended questions with charity. Avoid confrontational questioning.
Q Ver Interviews – The Offensive Game The Red Herring Obfuscation Stonewalling Disorganized / Can’t Find It Interruptions Schedule conflicts Vacillation, uncertainty, indecisiveness
Q Ver Interviews – The Defensive Game Prevent defensive game by preparing. Remain calm. Remember you can leave.
Q Ver Interview Difficulties Misunderstandings sometimes occur. –Listen carefully & actively. –Address concerns fully & repeat for feedback. –Stay grounded in facts.
Q Ver Validation of Responses Review and complete your notes soon after the interview. Refer to documentation. Chat informally with others in the group. Set up additional interviews. Phone or stop by for follow-up questions.
Q Ver Pacing the Assessment Two weeks includes interviews, research, and reporting. Total effort should be less than 40 hours. Don’t forget scheduling delay! Schedule all interviews and the end report up front!
Q Ver Tracking Results Develop working hypotheses. Prove right or wrong. Follow-up with questions & calls.
Q Ver Scoring the Assessment
Q Ver Develop the Score
Q Ver Develop the Score KA 1.A.1 “A motivated consideration of the product risks takes place…” Question: “What is your test strategy, and how did you formulate it?”
Q Ver Develop the Score
Q Ver Develop the Score
Q Ver Develop the Score
Q Ver Finalizing the Presentation Remember your audience. Should be a mere formality. Tell them more than they know. Provide actionable plan
Q Ver The Report Presentation
Q Ver Reporting Findings Management team receives report Written as well as verbal presentation No surprises
Q Ver Leveraging TPI for OneView The 20 TPI Key Areas provided concrete suggestions for improvement –Treat testing as a project, dedicate a manager –Consolidate defect reporting with a DB tool –Dedicate professional testing resources –Involve test design resources early
Q Ver Postscriptum
Q Ver The Postscriptum After the presentation, follow up will be required. Share the formal document with participants. Schedule any f/u meetings.
Q Ver The Take-Home What’s the take-home?