1. Testing journey in Scaled Agile Framework®
Rajiv Bajwala – Manager, Software Development Engineering
Mayuresh Deshpande - Director, Quality Assurance
Manish Kulkarni - Manager, Quality Assurance Engineering
Fiserv India Pvt. Ltd.
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

2. Abstract
After the success that the IT industry has experienced in adapting Agile Development Methodologies; it was only a matter of time, to have multiple teams operating in the same methodology to deliver large programs. This setup demanded multiple Agile teams to operate in the same rhythm to ensure synchronized delivery with built-in quality. The Scaled Agile Framework® (SAFe®) has been one of the popular frameworks to handle this situation.
SAFe® has also evolved in the past 5+ years (today we have SAFe® v 4.0) and in the process have added to the expectations from the testing activities and their impact on multiple roles prescribed in this framework. The aim has been to make this scaling successful from built-in quality perspective
In this presentation we are exploring these additional expectations from different roles (individuals and teams) as compared to isolated agile teams.
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

3. Activities & Roles that shape the Testing expedition
As we try and understand the activities spread across multiple levels in SAFe®, we realize that compared to isolated Agile teams, there are several new roles and activities that influence the testing journey. These are needed to maintain the synchronization across multiple Agile teams operating as an Agile Release Train (ART) and to deliver integrated quality.
As seen in this adjacent figure, we will explore the key roles and their related activities contributing to quality. We have tried to look at the benefits these activities can bring and challenges that may arise in their absence.
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

4. Activities at Team Level
Presentation Body
Activities at Team Level
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

5. Delivery Team(s) … During Execution
Role
Activities
Cross-functional team to build and test Solution Value in time-boxes
Are responsible to continuously build and improve the quality
The teams and ART are inseparable; the whole is greater than the sum of its parts
Automated unit testing, dev testing, component testing including service virtualization
In-sprint/ ’n+1’ automation
Testing interfaces - adjacent components
Integrate and demonstrate
Business Benefit
Thoroughly tested, Reviewed and Release ready features
Continuous integration & regression, driving quality throughout the ART
Early integration with adjacent components (pipe-cleaning)
Consequences of Not Doing
Disparate features not meeting acceptance criteria
Starved Regression Backlog
Surprises when end to end testing activities
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

6. Delivery Team(s) … During Inspection
Role
Activities
Prepare and conduct Team Demo and System Demo
Participate during Inspect and Adapt workshop
Demo working features to stakeholders & seek feedback
Retrospect, Identify and find solution for biggest problems, create improvement backlog
Business Benefit
Stakeholder feedback on working system features and changes
Adapt to technology challenges
Consequences of Not Doing
Product features not in line with customer vision or market expectation
Missed opportunity to implement improvements
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

7. Activities at Program Level
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

8. Product Management * Roles Activities
Create and Communicate Program Vision
Customer interaction
Define and prioritize Requirements
* At the Value Stream, Solution Management plays a similar role
Orchestrate System/Solution Demo
Defining clear acceptance criteria
Plan and Perform acceptance testing
Participate in PI Planning
Business Benefit
Confidence on what is getting built, is in sync with the defined requirements
Clearly Articulated sequence and dependencies in the Program Board
Consequences of Not Doing
Missing early feedback on development
Late discoveries of dependencies, out of synch development
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

9. Release Train Engineer *
Role
Activities
Orchestrator for ART
Facilitator for driving ART level impediments, risk management
* At the Value Stream, the Value Stream Engineer plays a similar role
Facilitate End to End (E2E), System Integration (SI) and NFR Test Planning
Monitor Synchronized Test Execution at program level
Test Strategy at Value Stream level and participate in definition of others
Business Benefit
Seamless product integration at a program level and establish E2E quality
Early identification of System/Integration issues
Consequences of Not Doing
Misaligned features across systems
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

10. System Arch/Eng * Roles Activities
Align Value Stream and Agile Release Train to technological and architectural vision
Define systems and sub systems
* At the Value Stream, the Solution Architect plays a similar role
Establish Critical NFRs for the Solution
Envisions Continuous Integration, Test First, and Automated Test Framework
Business Benefit
Architectural Runway created enables teams to continuously monitor quality
Consequences of Not Doing
Delivery teams will not be able to run at an optimum velocity in the absence of well founded Architectural runway and CI related benefits
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

11. System Team Role Activities
Special Agile team (including senior testing presence) assisting releases and inter team coordination
Build infrastructure, CI and Integrate Assets
Test and Evaluate E2E system
Test E2E installation process
PI System Demo
Business Benefit
Early identification of system integration and synchronization issues
Enablers for early feedback from stakeholders
Consequences of Not Doing
Missing early feedback on E2E system flow and integration issues
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

12. Component Team(s) Role Activities
Focused on Technical Stories, Refactoring and Spikes
Build the test automation framework for delivery teams
Perform performance testing and testing for other NFR’s
Developing UX standards and artefacts
Business Benefit
Feature teams able to leverage test automation for frequent regression cycles
Focus of the Feature teams remains on the Customers requirements
Consequences of Not Doing
Feature teams would be focused on these activities, thereby reducing their velocity to deliver working features
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

13. Summary
Quality is the responsibility of all the roles integral to SAFe®.
Thorough orchestration and synchronization of these roles and their activities, with the required skills will help in achieving this goal.
This can be achieved through proper Planning & Execution of Test Strategy and Plans supported by Technical Excellence and able Tools at all levels within SAFe®
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

14. Author Abstract
The authors have been working on Agile software development programs since They have rich experience in Transformation Programs and working with a large number of Agile teams as part of these programs.
Apart from being CSMs, they are Certified Scaled Agilists and have understanding of SAFe® and its benefits while scaling Agile for a large number of teams.
The diverse experience in development and testing in Agile and other methodologies has helped them craft this presentation.
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.

15. Thank You!!!
Reproduced with permission from © Scaled Agile, Inc. All rights reserved. SAFe and Scaled Agile Framework are registered trademarks of Scaled Agile Inc.