Graham Thomas Independent Software Testing Consultant Software Testing Secrets That We Dare Not Tell ! #esconfs

Abstract These questions are important because they drive at the very heart of what we are doing in the software testing industry today. 2 I have worked in IT for over 30 years, in software testing for over 20 years, so you might think that I know a lot about IT and in particular software testing. Well, I am going to share some dark secrets about software testing with you, which we dare not tell, and do that in the form of questions. This session has been designed to be a highly interactive discussion which many people might find challenges their basic understandings. I will act as facilitator, give an introduction to each question, then actively moderate the debate and if needed act as arbiter. Come along, expect to be involved, and if you have a view then please share it. Help to drive forward the discussion – and the software testing industry.

A bit about you A feel for the audience today  Current Role Above Program Test Manager Program Test Manager or Test Manager Test Leader Test Analyst or Test Execution New to Testing  Industry Experience 15 years + Over 10 Over 5 Done a year or more Still got the L plates on  Industry exposure Industry body Workgroup Contributor Blog Speaker Author  Conferences First time at EuroSTAR Other conferences and seminars Workshops or meetings 3

Secretive Air Force Space Plane X37B Lands 4

1.What is the purpose of Software Testing? 5

What non-testers think I was explaining this presentation to my doctor the other day. I said “It is questioning the purpose of Software Testing.” He said “ That is easy, it is to find all the bugs.” I said “Not necessarily.” He gave me a funny look, and I realised that this was a longer conversation that we had time for! 6

7

8 2.Just how effective is the way we test – and how do we know? (Trad, V-Model, Structured Testing, agile or any other form of testing for that matter?)

Effectiveness  GBP / USD per bug  Overall cost by requirement / use case /story  Organisational cost – If I do this I can’t then do that  Can I afford to test?  How long will it take  Why is it now taking longer  How do I measure it?  Can’t I have it now?  And what will I get for all that The Time / Cost / Quality Dice  How will I know I have got all of that?  Do I need all that?  What happens if I don’t get all of that? 9

Discuss 10

3.If checking isn't software testing, then why is it that ‘checking’ is what our stakeholders are paying us to do? 11

Drivers for Testing Checking  Prove it works  Show me it working  Coverage  Understand what I will get  Quantify the risk of delivery / non-delivery Testing  Michael Bolton quoting Laurent Bossavit “A Testers Role is to see through the veil of conflicting illusions!”  Agile Testing is ‘Exploratory Learning’  Discovery 12

Discuss 13

4.If software testing is so difficult, demanding and challenging, then why is it that we keep on assigning the least skilled or experienced to perform it? 14

Conundrum And even if we don’t know; what the purpose of software testing is, how effective our software testing is, or agree on how we go about software testing, surely we should at least be able to draw together some attributes for the kind of people we need to perform software testing? 15

Discuss 16

5.Why do software testers spend so much of their time running tests that do not find bugs? 17 ToolsScripting Execution AnalysisResources

 Automation Paradox To successfully automate tests they all have to pass. However what is the point of running tests that don’t find any bugs?  Tools to automate testing If we had tools that could test software we wouldn’t need software testers! Though, how would we know whether the tools were working or not?  Automating testing does not save any time, and in most cases takes longer than manual testing!  The latest trends in test optimisation are to stop running regression suites that take time effort and resource but no longer find bugs! Inefficiencies 18

Discuss 19

To Conclude As an industry, collectively: “We do not know, or agree on the purpose of software testing” “We do not know how efficient or effective our test models are, regardless of type” “We don’t even agree on what it is we should do when we are testing software” “We don’t understand what skills and experience we need to perform software testing” “And finally when we do testing, an awful lot of it doesn’t actually make any difference” I think we should either: “Keep these secrets very, very quiet” “Or tell everyone and set about finding the answers!” 20

Let me think about that for a minute 21

Contact Details Graham Thomas Independent Software Testing Consultant  