Diego Lo Giudice, Vice President, Principal Analyst Webinar Continuous Testing In Agile Changes The Testing Tools Landscape Diego Lo Giudice, Vice President, Principal Analyst October 21, 2013. Call in at 10:55 a.m. Eastern time

As software applications become more differentiating business assets . . . WHY ? Image source: CompareBusinessProducts.com (http://www.comparebusinessproducts.com/) . . . development and testing tools evolve and the way you use them too . . . !

Agenda New practices of Agile testing improve quality and speed: How? How do testing tools evolve to support the new testing trends? Emerging and renewed testing practices and tools

Agenda New practices of Agile testing improve quality and speed: How? How do testing tools evolve to support the new testing trends? Emerging and renewed testing practices and tools

SPEED !! Faster time-to-market!! Faster and more frequent delivery Image source: Fotolia (http://us.fotolia.com/) Faster time-to-market!! Faster and more frequent delivery

Bugs in production biggest quality issue Wait time for testing Optimized test coverage needs Test environments provisioning . . . Image source: Nanophase Technologies (http://www.nanophase.com/)

The arrival of modern applications won’t make things better! Source: February 13, 2012, “Mobile Is The New Face Of Engagement” Forrester report

One-size AD&D will no longer fit all! Longer AD&D cycle, stable requirements, slow innovation Short AD&D cycle, business feedback, change, innovation Testing practices, process, and tools as we’ve known them won’t work any longer! Agile, continuous delivery Source: February 13, 2012, “Mobile Is The New Face Of Engagement” Forrester report

End-to-end integration For Agile development: Testing needs to be done continuously, early, and fast! End-to-end integration Integration and integration testing start early on; test data fed continuously, performance testing can’t be done only at the end. Source: January 15, 2013, “Consistent Performance In Agile Teams Must Include Testing” Forrester report

Testing shifts left. . . . an old concept comes true. Testing involved early on from sprint 0 Definition of done includes precise requirements for testing ! Functional and nonfunctional test and process automation Performance, test data management, integration Leaner test management with UATD, TDD, and BDD Exploratory testing for remaining manual testing

Less focus on GUI automation Automation beyond the GUI API automation Stronger need for higher levels of automation gets developers more involved in testing. Less focus on GUI automation Automation beyond the GUI API automation Services automation Better design and reuse SDLC to develop automation tests A task for developers Image source: HAAP Media (http://www.sxc.hu/)

In three to five years TCoEs, as we know them, will change: shifting to become practice centers Focus of TCoE shifts to practices for: More automation. Performance-load testing. Provisioning fast/better test environments. Provisioning test data. Provisioning virtualized assets. Provisioning SVT solutions. Shifting testing to the left. Developers pick more testing up. TPCoE: testing practice center of excellence Or: Community of practice Source: January 15, 2013, “Consistent Performance In Agile Teams Must Include Testing” Forrester report

Agenda New practices of Agile testing improve quality and speed: How? How do testing tools evolve to support the new testing trends? Emerging and renewed testing practices and tools

True Agile requires tools that enable the whole dev-test-ops life cycle “Which of the following testing and release management practices does your development team currently use?” (Select all that apply) Unit testing 58% Exploratory testing 20% Performance/load testing 38% Automation/regression testing 30% Continuous integration with multiple weekly builds 32% Base: 698 North American, European, and Asian professional software, internal IT, game developers, and consultants; Source: Forrsights Developer Survey, Q1 2013

And tools must get developers more involved in testing! “How often do you use the following tools when you develop software/applications?” Tech-testing tools Base: 2,038 North American, European, and Asian software developers; Source: Forrsights Developer Survey, Q1 2013

Five “must-haves” on testing tools to support continuous testing Simplicity Continuous testing Automation Image source: Wikibon (http://wikibon.org/); Fotolia (http://us.fotolia.com/) Integration

The testing tool landscape Test management Unit testing frameworks TDD/BDD/ UATDD Test automation tools Service virtualization and testing Performance/load Defect and issue tracking Test data management Technical code quality

The landscape of testing tools is becoming more crowded and dynamic Source: July 18, 2013, “Navigating The Agile Testing Tool Landscape” Forrester report

The landscape of testing tools is becoming more crowded and dynamic (cont.) Source: July 18, 2013, “Navigating The Agile Testing Tool Landscape” Forrester report

The landscape of testing tools is becoming more crowded and dynamic (cont.) Source: July 18, 2013, “Navigating The Agile Testing Tool Landscape” Forrester report

Agenda New practices of Agile testing improve quality and speed: How? Testing tools evolve to support the testing trends. Emerging and revival of testing practices and tools

Testing tool landscape: emerging and revival Test management AUTOMATION Unit testing frameworks TDD/BDD/ UATDD Test automation tools Service virtualization and testing 25% < 10% automation 38% < 20% automation 30% between 30% to 50% 12% above 80 Performance/load Defect and issue tracking Test data management Technical code quality

What is service virtualization? Forrester definition: The practices and tools of simulating interfaces, APIs, services, and behavior of application, data, and infrastructure to help application development teams and testers perform early virtual integration testing and comprehensive functional and nonfunctional testing (e.g., performance testing) and source test data all in an automated way

Integration testing in complex environments is part of the challenge of today’s composite apps Web browser Mobile apps Tablet Internet apps Composite applications Existing infrastructure Applications under test SWIFT SAP/ legacy LDAP SOAP Rest DB JDBC MQ Mainframe Web services Security server Application services Online cloud third-party services INTEGRATION TESTING Unavailable for testing or expensive to use for testing

More testing challenges addressed by SVT . . . Increase levels of automation. Performance testing presents many struggles. Test environments are an issue in many ways. Creating and provisioning test data is ad hoc or not done. Integration testing is hard, just getting harder. Developers develop ad hoc stubs, which don’t necessarily mimic production precision and are not reusable.

Virtualization and testing market Watch out for the Forrester Wave™ on service virtualization and testing tools, coming Q4 2013. Grid-Tools Shunra SmartBear IBM CA HP Parasoft Services Network Data Open source stubs: Mockito, jMock

Testing is the weak link between dev-and-ops but SVT . . . “Le trade-d’union” SVT Dev Dev Ops Ops Testing Testing

Test data management: the Cinderella of testing . . . Test data cranked together ad hoc by developers/testers Test data provisioned in a secure way, sub-setted, masked from production Have a strategy around TDM; don’t make it happen “ad hoc.” Low cost dev Tactical Poor quality High cost dev Skills lack Slow High cost for business Poor quality Higher quality Compliance Tools requirement: Look at the five musts. Source: July 18, 2013, “Navigating The Agile Testing Tool Landscape” Forrester report

Diego Lo Giudice +39 340.771.0775 dlogiudice@forrester.com Twitter: @dlogiudice

Our Agile and Lean Playbook has a wealth of additional information on testing Source: Forrester Research, The Agile And Lean Playbook