WHY IS SOFTWARE TESTING NEEDED? Presented by - Santosh Bethi 22 nd October, 2003

Real Time Software Attacks The Funny The IRS sends out $68 billion tax bills ATMs in Germany give unlimited amounts of money after the Euro conversion The Serious The loss of the Mars Climate Orbiter – Cause: Improper unit conversion AT&T’s telephone network breaks down -- Cause: Untested code path The Tragic The Patriot missile failure – Cause: Cumulative rounding error

Attacks cause Real Problems Software fails just as mechanical devices fail Software is often invisible We often take software for granted Our reliance on software is increasing Our vulnerability to software flaws is increasing

Why Test Software? “Program testing can be used to show the presence of bugs but never to show their absence.” —Edsger W. Dijkstra, 1972

Testing is a State of Mind “If our goal is to show the absence of errors, we will find very few of them.” “If our goal is to show the presence of errors, we will discover a large number of them.” —G. J. Meyers, 1979

Developer’s Mindset A good developer is clever Emphasis on depth of expertise Product Expertise

Tester’s Mindset A good tester is mischievous Emphasis on breadth of expertise Product Expertise

Is Testing a Big Problem? YES Developers don’t test well Misconception that testing is “boring” Requirements are not always clear Must test everything Exhaustive testing is intractable

Where is Testing Useful? ATM Vending Machine Alarm Clock Thermostat Everywhere!

So Why Test? Reliability Security Usability Usefulness Sanity

So Why Test? Make the user happier!

How To Test Software Testing The Design Ambiguity is the mother of all screwups Testing starts before the coding Follow the specification cliché – Design globally – Specify locally

Designing a Test Plan Be complete (thorough, not excessive) Find the “interesting” tests cases Do every test at least twice Update the plans as you go – Add regression tests – Incorporate user feedback

Designing Tests Tests are “attacks” against the software Find vulnerabilities and target them – Black box – White box

Black Box Tests Targeted at the apparent simplicity of the software – Makes assumptions about implementation – Good for testing component interactions Tests the interfaces and behavior InputOutput

White Box Tests Targeted at the underlying complexity of the software – Intimate knowledge of implementation – Good for testing individual functions Tests the implementation and design InputOutput

Types of Tests Unit – Individual classes or types Component – Group of related classes or types Integration – Interaction between classes Unit – Individual classes or types Component – Group of related classes or types Integration – Interaction between classes

References [1] (Nov 1999) [2] n.csl.sri.com [3] ,00.html [4] (Feb 1991) [5] [6] [7] [8] [9]