Software Testing #IRL

Agenda Why Software Testing Software Testing Principles – Exhaustive testing is impossible – Early Testing – Testing shows the Presence of Bugs – Defect Clustering – The Pesticide Paradox – Testing is Context Dependent – Absence of errors fallacy Summary Recommended reading

Other professions in Computer Science Project Lead DeveloperArchitect Technical WriterCustomerIT Ops

How Other professions in Computer Science sees test Project LeadDeveloperArchitect Technical WriterCustomerIT Ops

Agenda Why Software Testing Software Testing Principles – Exhaustive testing is impossible – Early Testing – Testing shows the Presence of Bugs – Defect Clustering – The Pesticide Paradox – Testing is Context Dependent – Absence of errors fallacy Summary Recommended reading

Exhaustive Testing is Impossible

What you Think you are Testing

Developer Forgot «this.»

Off by One

Type Casting

Previous Values

Date and Time

Obey Thermodynamics – it’s also a LAW! 1 year of power of sun is enough energy to flip bits to count to all combinations in a 187 bit number. 187 bit < 24 byte. Google: “schneier dog crypteto”

Early Testing: V-Model Architecture Requirements Specification Design Code Acceptance Test System Test Integration Test Unit Test

Early Testing: W-Model Architecture Requirements Specification Design Code Acceptance Test System Test Integration Test Unit Test Design Acceptance Test Design System Test Design Integration Test Design Unit Test

Cost of Defect

Quality Attributes: ISO/IEC :2001 Functionality - satisfies needs. – Suitability – Accuracy – Interoperability – Security – Functionality Compliance Reliability - maintains operation – Maturity – Fault Tolerance – Recoverability – Reliability Compliance Usability - effort needed for use – Understandability – Learnability – Operability – Attractiveness – Usability Compliance Efficiency - performance given resources – Time Behaviour – Resource Utilisation – Efficiency Compliance Maintainability - make modifications – Analyzability – Changeability – Stability – Testability – Maintainability Compliance Portability - change environment – Adaptability – Installability – Co-Existence – Replaceability – Portability Compliance

Types of Testing Acceptance Test System Test Integration Test Unit Test Functional Efficiency Interopability X X X X X X X

Several Layers Acceptance Test System Test Integration Test Unit Test FunctionalEfficiencySecurity X X X X X X X System Int. TestX

Testing shows the presence of bugs, but can not show that there are no defects 1986: Programming Pearls, binarySearch() bevist og testet i et kapittel 1997: Implementert i Sun java.util.Arrays.binarySearch() 2006: Feil funnet og fikset i Java- biblioteket

Defect Clustering

Bugs Found with Bug Seeding

Pesticide Paradox Age of test Bugs found per day by a test

Regression Testing Acceptance Test System Test Integration Test Unit Test FunctionalEfficiencySecurity X X X X X X X System Int. TestX X Levels Types

Testing is Context Dependent Coverage Cost 0%50%100% Cost of testing 80-85%? Cost of defects

Absence of Errors does not mean the System is OK Acceptance Test System Test Integration Test Unit Test 95% Performance/Stability

Agenda Why Software Testing Software Testing Principles – Exhaustive testing is impossible – Early Testing – Testing shows the Presence of Bugs – Defect Clustering – The Pesticide Paradox – Testing is Context Dependent – Absence of errors fallacy Summary Recommended reading

TL;DL Testing #FTW #YOLO You can’t test everything! Testing is Context Dependent! Get involved early! Testing doesn’t prove software is without bugs Absence of errors does not prove the system is ok Defects cluster Old tests find fewer bugs

Books you should read to learn more about… People and testing Creative Testing Coding Software EngineeringTesting in TrondheimFun Web

Software Testing #IRL

Copyrights Ant Hill: RukaKuusamo.com Money bag: CC AT Book: CC AThttp:// Cables: CC AT SAhttp:// Bokhylle: Sehoon Lee Phote bookshelf-by-sehoon-lee/ bookshelf-by-sehoon-lee/ Hender: CC AT NM NC Angel: CC AT NC Guarantee: CC AT Road: CC AT SA