QA Transformation “Get on board or risk NOW!” Ray Scott QA Transformation Consultant

Who am I? 25 years I.T (15 QA, 10 Agile) Sr. QA Manager & Agile – Walt Disney Theme Parks & Resorts Director Agile and professional services – Grid Tools Bank of Boston, Fidelity Investments QA Transition Manager/Interim Head of QA Ladbrokes/Aditi UK Leading airline Head of Performance SOA, Consultant Speaker at: Agile and QA conferences throughout Europe and USA 2

AGILE – True of False?  CULTURE  METHODOLOGY  SET OF PRACTICES  Practicing SCRUM is AGILE  Performing TDD, BDD and Pair programming means you are AGILE  Is hard to do!

What is Software Testing? 4

Testing practices Test plans Test cases/scenarios Aligning to requirements Wait for code to be delivered Automation Expect defects and report KPI 5 Defect management Triage manager Test manager Test lead Tester QA Manager

What is the cost of exposing defects? 6

Costs of Issues 7 Req. Arch. Dev. Prod Total 400 hrs ~ £40,000 - Clients £ £ £ £ £ £300 Detail Design. £800£1800£3800 £6000 SIT UAT £10,000 PreProd £20,000

Off-shoring has become BIG- BUSINESS

What are we trying to save? 9

Pushing “LEFT” 10 Req. Arch.Dev. Prod Detail Design. SIT UAT PreProd 1X6X 13X 20X35X 65X 140+X 3X Testing finds defects, QA prevents defects

How does it all change for a tester in an Agile environment? 11

Some core concepts 12 Collaboration Communication Transparency Innovation How we work together How we move information Know what is going on Trying something different Does this sound like Testing?

Traditional Tester 13

Transforming 14

WoW - Ways of Working 15 Testing finds defects, QA prevents defects

Quality Assurance agenda 16 Testing is the responsibility of the TEAM Testing occurs ALL THE TIME “Push-Left” Reduce risk and increase confidence with predictability Alignment to business goals in the SPRINT Validates a potential release candidate DONE is DONE

How can testing be the responsibility of the TEAM? Development testing practices, TDD and BDD Non-Functional requirements Product Owner feedback Test Scenarios over Test cases Manage technical debt (Sev 3 and 4s) Identifying Sev 1 and 2s early leaves time for 3s and 4s 17

Quality Assurance agenda 18 Testing is the responsibility of the TEAM Testing occurs ALL THE TIME “Push-Left” Reduce risk and increase confidence with predictability Alignment to business goals in the SPRINT Validates a potential release candidate DONE is DONE

How can testing occur ALL THE TIME? Continuous Integration (Build & Test) - TDD, BDD QA reviews Product Owner reviews Plan for the Unplanned Avoid more than needs to be tested 19

Quality Assurance agenda 20 Testing is the responsibility of the TEAM Testing occurs ALL THE TIME “Push-Left” Reduce risk and increase confidence with predictability Alignment to business goals in the SPRINT Validates a potential release candidate DONE is DONE

Push Left = How do we detect defects earlier? The best protection is EARLY PROTECTION Understand the Sprint delivery Create relationships with team Communication plans Participate in user story maintenance (Func. & NFR) Perfection is only a test case away 21

Quality Assurance agenda 22 Testing is the responsibility of the TEAM Testing occurs ALL THE TIME “Push-Left” Reduce risk and increase confidence with predictability Alignment to business goals in the SPRINT Validates a potential release candidate DONE is DONE

How can RISK be reduced? “The probability of LOSS” AND “ Exposing to harm or danger” Understand the limitations of what is under test Testing is NOT trying to break it Shorter, incremental & cumulative Understand the data 23

Quality Assurance agenda 24 Testing is the responsibility of the TEAM Testing occurs ALL THE TIME “Push-Left” Reduce risk and increase confidence with predictability Alignment to business goals in the SPRINT DONE is DONE

How to align with the business Know your Product owners (Business Representative) Prove the value you bring to the product development Intelligently challenge Walk through demo’s Introduce Quality Controls (KPI’s) 25

Quality Assurance agenda 26 Testing is the responsibility of the TEAM Testing occurs ALL THE TIME “Push-Left” Reduce risk and increase confidence with predictability Alignment to business goals in the SPRINT DONE is DONE

When are we DONE? Define DONE Work towards DONE Communicate DONE to all Stop when DONE enough DONE-DONE-DONE 27

Joining an Agile team Testers are pack animals Joining an Agile can be a lonely existence Reporting into a Dev. Manager Reduced structure, stage gates and documentation Loss of control 28

The new world of Agile Testers no longer need to be siloed or feel alone in Agile Work with support groups Influence team members Focus on Quality not quantity of test cases Testing finds defects, QA prevents defects Requires courage to move out of comfort zone Learn new skills (Customer focus) Team up with those that build the quality into product delivery 29

WoW - Ways of Working 30 A real world worked example

Environment Agile-like 3 rd party development team, distributed 3 rd party testing team, distributed (Not same as above) Product Owner engaged and committed Some SCRUM practices encouraged Too much technical debt Sev 3 and 4 pushed aside UAT duplicated SIT testing 31

WoW – original lifecycle 32 Development SIT UAT

Goals Reduce cycle delivery time for Sprints Reduce number of defects found late in the cycle Increase confidence with delivery of quality product 33

KPIs –Cumulated defects

WoW Early engagement of QA with Dev and Business team Sharing product and testing objectives Added value to Dev testing effort Last few days of Sprint became a Pre-SIT period Execution of “Happy Path” Early detection of Sev 1s & 2 Entry stage gate criteria met Fast turn around of defects 35

WoW Entry to formal SIT with “Known Knowns” More confidence of product quality Less “Unknown Unknowns” Next Sprint Technical debt (Sev 3/4) Product owner can “Go-Live” earlier UAT cycle shorter Less technical debt More communication….more reliability….earlier value 36

WoW – Modified lifecycle 37 Development SIT UAT Development SIT Pre- SIT UAT QA involvement

KPIs –Cumulative KNOWN defects

WoW – comments from team “Much more engagement created a collaborative environment” “Gut feel entering SIT stage was superior” “Quality product delivered – but more effort needed” 39

In closing AGILE is HARD to do! Agile is NOT for everyone Leaders over Mangers A Pat on the back cost nothing and Motivates 40

Questions ???? Good reading : 41 Published in ComputerWorld.co.uk “Say hello to QA and goodbye to testing” “Failure is only a stepping stone to success” “Test data governance” “Test data warehouse” “What does quality look like” “Test data management is the new Automation”