Testing journey in Scaled Agile Framework® Rajiv Bajwala – Manager, Software Development Engineering Mayuresh Deshpande - Director, Quality Assurance Manish Kulkarni - Manager, Quality Assurance Engineering Fiserv India Pvt. Ltd. Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Abstract After the success that the IT industry has experienced in adapting Agile Development Methodologies; it was only a matter of time, to have multiple teams operating in the same methodology to deliver large programs. This setup demanded multiple Agile teams to operate in the same rhythm to ensure synchronized delivery with built-in quality. The Scaled Agile Framework® (SAFe®) has been one of the popular frameworks to handle this situation. SAFe® has also evolved in the past 5+ years (today we have SAFe® v 4.0) and in the process have added to the expectations from the testing activities and their impact on multiple roles prescribed in this framework. The aim has been to make this scaling successful from built-in quality perspective In this presentation we are exploring these additional expectations from different roles (individuals and teams) as compared to isolated agile teams. Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Activities & Roles that shape the Testing expedition As we try and understand the activities spread across multiple levels in SAFe®, we realize that compared to isolated Agile teams, there are several new roles and activities that influence the testing journey. These are needed to maintain the synchronization across multiple Agile teams operating as an Agile Release Train (ART) and to deliver integrated quality. As seen in this adjacent figure, we will explore the key roles and their related activities contributing to quality. We have tried to look at the benefits these activities can bring and challenges that may arise in their absence. Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Activities at Team Level Presentation Body Activities at Team Level Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Delivery Team(s) … During Execution Role Activities Cross-functional team to build and test Solution Value in time-boxes Are responsible to continuously build and improve the quality The teams and ART are inseparable; the whole is greater than the sum of its parts Automated unit testing, dev testing, component testing including service virtualization In-sprint/ ’n+1’ automation Testing interfaces - adjacent components Integrate and demonstrate Business Benefit Thoroughly tested, Reviewed and Release ready features Continuous integration & regression, driving quality throughout the ART Early integration with adjacent components (pipe-cleaning) Consequences of Not Doing Disparate features not meeting acceptance criteria Starved Regression Backlog Surprises when end to end testing activities Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Delivery Team(s) … During Inspection Role Activities Prepare and conduct Team Demo and System Demo Participate during Inspect and Adapt workshop Demo working features to stakeholders & seek feedback Retrospect, Identify and find solution for biggest problems, create improvement backlog Business Benefit Stakeholder feedback on working system features and changes Adapt to technology challenges Consequences of Not Doing Product features not in line with customer vision or market expectation Missed opportunity to implement improvements Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Activities at Program Level Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Product Management * Roles Activities Create and Communicate Program Vision Customer interaction Define and prioritize Requirements * At the Value Stream, Solution Management plays a similar role Orchestrate System/Solution Demo Defining clear acceptance criteria Plan and Perform acceptance testing Participate in PI Planning Business Benefit Confidence on what is getting built, is in sync with the defined requirements Clearly Articulated sequence and dependencies in the Program Board Consequences of Not Doing Missing early feedback on development Late discoveries of dependencies, out of synch development Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Release Train Engineer * Role Activities Orchestrator for ART Facilitator for driving ART level impediments, risk management * At the Value Stream, the Value Stream Engineer plays a similar role Facilitate End to End (E2E), System Integration (SI) and NFR Test Planning Monitor Synchronized Test Execution at program level Test Strategy at Value Stream level and participate in definition of others Business Benefit Seamless product integration at a program level and establish E2E quality Early identification of System/Integration issues Consequences of Not Doing Misaligned features across systems Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

System Arch/Eng * Roles Activities Align Value Stream and Agile Release Train to technological and architectural vision Define systems and sub systems * At the Value Stream, the Solution Architect plays a similar role Establish Critical NFRs for the Solution Envisions Continuous Integration, Test First, and Automated Test Framework Business Benefit Architectural Runway created enables teams to continuously monitor quality Consequences of Not Doing Delivery teams will not be able to run at an optimum velocity in the absence of well founded Architectural runway and CI related benefits Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

System Team Role Activities Special Agile team (including senior testing presence) assisting releases and inter team coordination Build infrastructure, CI and Integrate Assets Test and Evaluate E2E system Test E2E installation process PI System Demo Business Benefit Early identification of system integration and synchronization issues Enablers for early feedback from stakeholders Consequences of Not Doing Missing early feedback on E2E system flow and integration issues Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Component Team(s) Role Activities Focused on Technical Stories, Refactoring and Spikes Build the test automation framework for delivery teams Perform performance testing and testing for other NFR’s Developing UX standards and artefacts Business Benefit Feature teams able to leverage test automation for frequent regression cycles Focus of the Feature teams remains on the Customers requirements Consequences of Not Doing Feature teams would be focused on these activities, thereby reducing their velocity to deliver working features Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Summary Quality is the responsibility of all the roles integral to SAFe®. Thorough orchestration and synchronization of these roles and their activities, with the required skills will help in achieving this goal. This can be achieved through proper Planning & Execution of Test Strategy and Plans supported by Technical Excellence and able Tools at all levels within SAFe® Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Author Abstract The authors have been working on Agile software development programs since 2007. They have rich experience in Transformation Programs and working with a large number of Agile teams as part of these programs. Apart from being CSMs, they are Certified Scaled Agilists and have understanding of SAFe® and its benefits while scaling Agile for a large number of teams. The diverse experience in development and testing in Agile and other methodologies has helped them craft this presentation. Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

Thank You!!! Reproduced with permission from © 2011-2016 Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.