Assurance: the Evolution of Test Management Assurance, Re-Training, Mentoring 11-Nov-18 Assurance: the Evolution of Test Management Paul Gerrard paul@gerrardconsulting.com Twitter: @paul_gerrard Web: gerrardconsulting.com Intelligent Definition and Assurance

Intelligent Definition and Assurance Agenda Defining Assurance Scope and Role of Assurance Is Assurance for Large Projects Only? From Test Management to Assurance Will You get into Assurance? Intelligent Definition and Assurance

Intelligent Definition and Assurance Intro Agile has no clear role for test managers Some test managers reverted to testers Some specialise (test automation, performance or security) Some moved upstream to be business analysts or product owners Some left the business completely Test Manager -> Virtual Test Management Although Test Managers are under threat Assurance is emerging and is a natural progression for delivery-focused professionals. Intelligent Definition and Assurance

Assurance (aka test-, business- or delivery-assurance) A way to improve the relationship with software suppliers, provide support and police them Reports to the board and speaks truth to power Requires coaching and persuasion skills and a deep knowledge of software and organisational dynamics Digital is taking over everything – now is the time for Digital Assurance. Intelligent Definition and Assurance

We need an Assurance vision For twenty years or so, I’ve been involved in Assurance projects Perhaps one meaty project each year Mike Jarred and I are convinced Assurance is an important, emerging discipline The TMF -> ALF rebrand is a natural evolution Assurance is a discipline, a role in business Today, I want to explore and discuss: A definition of Assurance What role it plays in business and software projects How it presents an attractive career progression. Intelligent Definition and Assurance

“Assuring Software Delivery is Successful” Assurance Defined “Assuring Software Delivery is Successful”

Assurance (from dictionary.com)

Missing from those definitions We assure delivery Not about process and box-ticking, it’s about improving (the chances of) successful delivery Independence Requires independence – no responsibility for delivery Advisory Advising projects on approaches, methods, plans, tools, techniques Critical Not here to make friends, we critically evaluate the performance of all activities and participants. Intelligent Definition and Assurance

Missing from those definitions 2 Uber alles Anything that has a bearing on successful delivery is potentially in scope Stakeholder focused We interpret the status and performance of projects and suppliers to inform stakeholders; we pass on stakeholder concerns to projects People Are the right people in place? Appropriate/adequate capabilities? Motivation, incentives, support available? Governance Are the right people involved? Are appropriate checks and balances in place? Tech Requirements, development, test, implementation, cutover approaches. Intelligent Definition and Assurance

Scope, Role and Other Aspects https://atlas.mindmup.com/2018/01/db39fd10f61b11e78ce661868bc109c8/free_mind_map/index.html

Is Assurance for Large Projects Only? No

Intelligent Definition and Assurance Project scale $200m SAP migration, £150m government infrastructure Flash: Possible involvement in a £1.2bn project next week But Assurance is scaleable to smaller and Agile projects Consider the tester’s role in an Agile Scrum project Testers might not do much testing; rather they act on intervention points. Intelligent Definition and Assurance

(Test Strategy as) Agile Interventions I’m using Scrum/Sprint terminology, but you don’t have to of course https://www.youtube.com/embed/Ed6YkYEkCRM

These activities are repeated for each Sprint iteration Interventions No. Activity When? 1 Story Challenge As stories are added to the Product Backlog 2 Story Definition As stories are added to a Sprint Backlog These activities are repeated for each Sprint iteration 3 Daily Stand-Up Once per day during the Sprint 4 Story Refinement Occurs throughout the Sprint as new information emerges 5 Developer Testing Occurs throughout the Sprint as the developer codes the stories 6 Integration (and incremental System) Testing During and at the end of each sprint, including the final sprint 7 System Testing At the end of each sprint, including the final sprint 8 User Acceptance Testing 9 Non-functional Testing and Pre- Production Testing Expected to take place on an as-needs basis. On the following slides, we highlight 8 interventions Some are test phases, but some aren’t Intelligent Definition and Assurance

Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) Story Challenge Suggest ‘what-ifs’ to challenge new stories and define story headlines Sprint Backlog Sprint Backlog Sprint Backlog 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Sprint 1 Sprint 2 Sprint 3 Developed Stories Developed Stories Developed Stories New Code Integration into Existing Code base Automated testing 6. Integration Test 6. Integration Test 6. Integration Test Increasing Scope of Sys. Test and UAT 7. System Test 8. User Test Increasing Scope of Integration, System and Users Testing Complete Tests after Final Sprint Intelligent Definition and Assurance

Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) Story Challenge Suggest ‘what-ifs’ to challenge new stories and define story headlines Sprint Backlog Sprint Backlog Sprint Backlog 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Sprint 1 Sprint 2 Sprint 3 Developed Stories Developed Stories Developed Stories New Code Integration into Existing Code base Automated testing 6. Integration Test 6. Integration Test 6. Integration Test Increasing Scope of Sys. Test and UAT 7. System Test 8. User Test Increasing Scope of Integration, System and Users Testing Complete Tests after Final Sprint Intelligent Definition and Assurance

Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) Story Challenge Suggest ‘what-ifs’ to challenge new stories and define story headlines Sprint Backlog Sprint Backlog Sprint Backlog 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Sprint 1 Sprint 2 Sprint 3 Developed Stories Developed Stories Developed Stories New Code Integration into Existing Code base Automated testing 6. Integration Test 6. Integration Test 6. Integration Test Increasing Scope of Sys. Test and UAT 7. System Test 8. User Test Increasing Scope of Integration, System and Users Testing Complete Tests after Final Sprint Intelligent Definition and Assurance

Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) Story Challenge Suggest ‘what-ifs’ to challenge new stories and define story headlines Sprint Backlog Sprint Backlog Sprint Backlog 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Sprint 1 Sprint 2 Sprint 3 Developed Stories Developed Stories Developed Stories New Code Integration into Existing Code base Automated testing 6. Integration Test 6. Integration Test 6. Integration Test Increasing Scope of Int. Sys. and UAT 7. System Test 8. User Test Increasing Scope of Integration, System and Users Testing Complete Tests after Final Sprint Intelligent Definition and Assurance

Project Level Test Activities (This diagram shows three sprints, but there could be more or fewer) Story Challenge Suggest ‘what-ifs’ to challenge new stories and define story headlines Sprint Backlog Sprint Backlog Sprint Backlog 2. Story Definition Introduce scenarios to enhance the Acceptance Criteria Sprint 1 Sprint 2 Sprint 3 Developed Stories Developed Stories Developed Stories New Code Integration into Existing Code base Automated testing 6. Integration Test 6. Integration Test 6. Integration Test Increasing Scope of Int. Sys. and UAT 7. System Test 8. User Test Increasing Scope of Integration, System and Users Testing Complete Tests after Final Sprint Intelligent Definition and Assurance

Test Activities in the Sprint 3. Daily Stand-Up Report anomalies found, stories tested, amended, created 5) Developer Testing Private ad-hoc tests and build/run automated unit tests Daily Scrum Stand-Up Meeting 24 Hours 4. Story Refinement Refine scenarios to enhance story definition, create system tests as stories, as required 6) Integration/System Testing Incorporate automated unit tests into the CI regime. On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests. 2-4 Weeks Backlog tasks expanded by team Sprint Backlog Potentially Shippable Product increment Product backlog As prioritised by Product Owner Intelligent Definition and Assurance

Test Activities in the Sprint 3. Daily Stand-Up Report anomalies found, stories tested, amended, created 5) Developer Testing Private ad-hoc tests and build/run automated unit tests Daily Scrum Stand-Up Meeting 24 Hours 4. Story Refinement Refine scenarios to enhance story definition, create system tests as stories, as required 6) Integration/System Testing Incorporate automated unit tests into the CI regime. On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests. 2-4 Weeks Backlog tasks expanded by team Sprint Backlog Potentially Shippable Product increment Product backlog As prioritised by Product Owner Intelligent Definition and Assurance

Test Activities in the Sprint 3. Daily Stand-Up Report anomalies found, stories tested, amended, created 5) Developer Testing Private ad-hoc tests and build/run automated unit tests Daily Scrum Stand-Up Meeting 24 Hours 4. Story Refinement Refine scenarios to enhance story definition, create system tests as stories, as required 6) Integration/System Testing Incorporate automated unit tests into the CI regime. On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests. 2-4 Weeks Backlog tasks expanded by team Sprint Backlog Potentially Shippable Product increment Product backlog As prioritised by Product Owner Intelligent Definition and Assurance

Test Activities in the Sprint 3. Daily Stand-Up Report anomalies found, stories tested, amended, created 5) Developer Testing Private ad-hoc tests and build/run automated unit tests Daily Scrum Stand-Up Meeting 24 Hours 4. Story Refinement Refine scenarios to enhance story definition, create system tests as stories, as required 6) Integration/System Testing Incorporate automated unit tests into the CI regime. On weekly basis and at end of Sprint, deploy to System test environment and tester runs system tests. 2-4 Weeks Backlog tasks expanded by team Sprint Backlog Potentially Shippable Product increment Product backlog As prioritised by Product Owner Intelligent Definition and Assurance

The tester’s Assurance contribution Think of testing as interventions, not stages The testing role is redistributed and split The tester doesn’t own testing – think TESTMASTER – ASSURANCE in the small Identify the intervention points and negotiate with your team. Intelligent Definition and Assurance

From Test Management to Assurance

Intelligent Definition and Assurance Test Managers You might have been a tester, then a test lead As a lead you become a coach, mentor to other testers, but still spend time testing As a Test Manager, things change You are accountable for the performance of the team You represent the team to Project and Stakeholder management You have to plan, organise, manage, control Justify, defend testing, timescales, scope, coverage… Intelligent Definition and Assurance

Testing Measures Achievement Although you don’t test, you have to coach, cajole, command the team to test Project managers can push people around and provide graphs of money and time spent Inputs give a rear-view mirror perspective Testing reveals the truth: Achievement: what works and what does not Risk: what is the view of the risk of release. Intelligent Definition and Assurance

Testing is at the heart of Assurance Information on achievement and risks is pretty much what stakeholders need You are responsible, accountable for providing that information Assurance, being independent, depends on the outcomes of testing to a large degree (But it’s not everything of course) Test Managers are well placed to take the role. Intelligent Definition and Assurance

My Challenge to You

Will you get into assurance? In your current project(s), would an assurance role help? There is no test manager, but test management happens Who keeps an eye on the virtual test manager? You might not have all the skills required, but how will you acquire them? First opportunity is likely to be a technical one. Intelligent Definition and Assurance

Assurance: the Evolution of Test Management Assurance, Re-Training, Mentoring 11-Nov-18 Assurance: the Evolution of Test Management Paul Gerrard paul@gerrardconsulting.com Twitter: @paul_gerrard Web: gerrardconsulting.com Intelligent Definition and Assurance