Agile Testing and/or Agility in Testing? Juha Itkonen SoberIT Teknillinen Korkeakoulu Test Summit, , Savonlinna

2 Juha Itkonen, SoberIT, TKK, Contents  Overview of agility  Agile principles and testing  Two views of agile testing: eXtreme and Exploratory  Third view and challenges of agile testing  Agile development and testing services

3 Juha Itkonen, SoberIT, TKK, Agile Manifesto Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan How do these values affect testing?

4 Juha Itkonen, SoberIT, TKK, Some agile principles  Satisfy the customer through early and continuous delivery of valuable software.  Working software is the primary measure of progress.  Deliver working software frequently, from a couple of weeks to a couple of months.  Welcome changing requirements, even late in development.  The most efficient and effective method of conveying information to and within a development team is face-to- face conversation.  Business people and developers must work together daily throughout the project.  Simplicity--the art of maximizing the amount of work not done--is essential.

5 Juha Itkonen, SoberIT, TKK, Agile methodologies and testing  eXtreme programming  Scrum  DSDM  FDD  LD  ASD  MS Synch-and-Stabilize  Crystal family  Some define strict disciplined testing practices  Some do not say much about testing approach  E.g. FDD: “… processes used for testing are not the main process issues with which the organisations are struggling … and most organizations already have reasonable testing processes in place” “Agile Testing” is a buzzword that is not yet defined

6 Juha Itkonen, SoberIT, TKK, Two views of agile testing eXtreme Testing  Automated unit testing  Developers write tests  Test first development  Daily builds with unit tests always 100% pass  Functional testing  Customer-owned  Comprehensive  Repeatable  Automatic  Timely  Public Exploratory Testing  Manual testing by professional skilled testers  Freedom, flexibility and fun for testers  Controllability, reliability and high quality for managers  Optimized to find bugs  Continually adjusting plans, re-focusing on the most promising risk areas  Following hunches  Minimizing time spent on documentation Focus on automated verification – enabling agile software development Focus on manual validation – making testing activities agile

7 Juha Itkonen, SoberIT, TKK, Definitions of Exploratory Testing  In scripted testing, tests are first designed and recorded. Then they may be executed at some later time or by a different tester.  In exploratory testing, tests are designed and executed at the same time, and they often are not recorded.  You build a mental model of the product while you test it. This model includes what the product is and how it behaves, and how it’s supposed to behave. James Bach, Rapid Software Testing, 2002  To the extent that the next test we do is influenced by the result of the last test we did, we are doing exploratory testing. James Bach, 2001  Exploratory testing involves simultaneously learning, planning, running tests, troubleshooting and reporting results. Cem Kaner, 2001

8 Juha Itkonen, SoberIT, TKK, Scripted vs. Exploratory Tests mines fixes

9 Juha Itkonen, SoberIT, TKK, Value of testing in agile development  Testing in agile software development should provide the information that stakeholders need to make decisions and steer the development into the right direction  This information must be provided promptly  Testing provides data on the status of the deliverables and generates project intelligence  Project intelligence is knowledge of risks and benefits  Knowledge of risks, benefits, test records and results are more valuable than test documentation and infrastructure  We can increase the value of testing most by  Improved intelligence  Providing intelligence earlier

10 Juha Itkonen, SoberIT, TKK, Third view of agile testing  Embracing the change  Testing practices must not be fixed  Test procedures must be clearly defined and described as atomic modules  Test procedures must be easily selected to suit the changing situations and requirements  Development and testing must collaborate and agree on the test procedures and quality goals for each increment  Not only functionality but also non-functional features  Test requirements  Expected results  Business must set the upper-level quality requirements

11 Juha Itkonen, SoberIT, TKK, Challenges for testing in agile software development  What information is the testing based on?  What to test and what are the expected results?  How to make testing, development and business collaborate?  How to involve customer and business people in testing?  How do we know where we are?  Working software is the primary measure of progress  Testing should provide the data  If it’s not tested it doesn’t exist  How to keep up with the pace of the development?  How to produce and communicate relevant information promptly?  How to test early but not do anticipatory test design?

12 Juha Itkonen, SoberIT, TKK, Agile development and testing services  It is important to understand the challenges of agile practices  Hard to solve even within one company  If an agile company wishes to use testing services (e.g. for reliability, load, stress, compatibility or functional testing) it might be confronted with big problems  Arranging collaboration and communication  Service provider’s ability to respond to change  Service provider’s ability to adopt the pacing  The challenges of testing in agile development have to be solved!

13 Juha Itkonen, SoberIT, TKK, Further reading  Beck, K., Test-Driven Development by Example, Addison- Wesley,  Crispin, L., T. House, Testing Extreme Programming, Addison- Wesley,  Kaner, C., J. Bach, B. Pettichord, Lessons Learned in Software Testing, John Wiley & Sons,       