1. Software Testing #IRL jorgen.austvik@kantega.no

2. Agenda Why Software Testing Software Testing Principles – Exhaustive testing is impossible – Early Testing – Testing shows the Presence of Bugs – Defect Clustering – The Pesticide Paradox – Testing is Context Dependent – Absence of errors fallacy Summary Recommended reading

4. Other professions in Computer Science Project Lead DeveloperArchitect Technical WriterCustomerIT Ops

5. How Other professions in Computer Science sees test Project LeadDeveloperArchitect Technical WriterCustomerIT Ops

7. Agenda Why Software Testing Software Testing Principles – Exhaustive testing is impossible – Early Testing – Testing shows the Presence of Bugs – Defect Clustering – The Pesticide Paradox – Testing is Context Dependent – Absence of errors fallacy Summary Recommended reading

8. Exhaustive Testing is Impossible

9. What you Think you are Testing

10. Developer Forgot «this.»

11. Off by One

12. Type Casting

13. Previous Values

14. Date and Time

15. Obey Thermodynamics – it’s also a LAW! 1 year of power of sun is enough energy to flip bits to count to all combinations in a 187 bit number. 187 bit < 24 byte. Google: “schneier dog crypteto” http://www.schneier.com/blog/archives/2009/09/the_doghouse_cr.html

16. Early Testing: V-Model Architecture Requirements Specification Design Code Acceptance Test System Test Integration Test Unit Test

17. Early Testing: W-Model Architecture Requirements Specification Design Code Acceptance Test System Test Integration Test Unit Test Design Acceptance Test Design System Test Design Integration Test Design Unit Test

18. Cost of Defect http://www.abeacha.com/NIST_press_release_bugs_cost.htm

19. Quality Attributes: ISO/IEC 9126-1:2001 Functionality - satisfies needs. – Suitability – Accuracy – Interoperability – Security – Functionality Compliance Reliability - maintains operation – Maturity – Fault Tolerance – Recoverability – Reliability Compliance Usability - effort needed for use – Understandability – Learnability – Operability – Attractiveness – Usability Compliance Efficiency - performance given resources – Time Behaviour – Resource Utilisation – Efficiency Compliance Maintainability - make modifications – Analyzability – Changeability – Stability – Testability – Maintainability Compliance Portability - change environment – Adaptability – Installability – Co-Existence – Replaceability – Portability Compliance

20. Types of Testing Acceptance Test System Test Integration Test Unit Test Functional Efficiency Interopability X X X X X X X

21. Several Layers Acceptance Test System Test Integration Test Unit Test FunctionalEfficiencySecurity X X X X X X X System Int. TestX

22. Testing shows the presence of bugs, but can not show that there are no defects 1986: Programming Pearls, binarySearch() bevist og testet i et kapittel 1997: Implementert i Sun java.util.Arrays.binarySearch() 2006: Feil funnet og fikset i Java- biblioteket http://googleresearch.blogspot.it/2006/06/extra-extra-read-all-about-it-nearly.html

23. Defect Clustering http://www.slideshare.net/andreas.zeller/myths-in-software-engineering

24. Bugs Found with Bug Seeding http://www.slideshare.net/andreas.zeller/seeding-bugs-to-find-bugs-presentation

25. Pesticide Paradox Age of test Bugs found per day by a test

26. Regression Testing Acceptance Test System Test Integration Test Unit Test FunctionalEfficiencySecurity X X X X X X X System Int. TestX X Levels Types

27. Testing is Context Dependent Coverage Cost 0%50%100% Cost of testing 80-85%? Cost of defects

28. Absence of Errors does not mean the System is OK Acceptance Test System Test Integration Test Unit Test 95% Performance/Stability

29. Agenda Why Software Testing Software Testing Principles – Exhaustive testing is impossible – Early Testing – Testing shows the Presence of Bugs – Defect Clustering – The Pesticide Paradox – Testing is Context Dependent – Absence of errors fallacy Summary Recommended reading

30. TL;DL Testing #FTW #YOLO You can’t test everything! Testing is Context Dependent! Get involved early! Testing doesn’t prove software is without bugs Absence of errors does not prove the system is ok Defects cluster Old tests find fewer bugs

31. Books you should read to learn more about… People and testing Creative Testing Coding Software EngineeringTesting in TrondheimFun Web

32. Software Testing #IRL jorgen.austvik@kantega.no

33. Copyrights Ant Hill: RukaKuusamo.com Money bag: http://www.flickr.com/photos/68751915@N05/ CC AT SAhttp://www.flickr.com/photos/68751915@N05/ Book: http://www.flickr.com/photos/pat00139/ CC AThttp://www.flickr.com/photos/pat00139/ Cables: http://www.flickr.com/photos/alq666/ CC AT SAhttp://www.flickr.com/photos/alq666/ Bokhylle: Sehoon Lee Phote http://design-milk.com/squaring-movable- bookshelf-by-sehoon-lee/http://design-milk.com/squaring-movable- bookshelf-by-sehoon-lee/ Hender: CC AT NM NC http://www.flickr.com/photos/johnkay/http://www.flickr.com/photos/johnkay/ Angel: CC AT NC http://www.flickr.com/photos/jens-h-friedrichhttp://www.flickr.com/photos/jens-h-friedrich Guarantee: CC AT http://www.flickr.com/photos/vectorportalhttp://www.flickr.com/photos/vectorportal Road: CC AT SA http://www.flickr.com/photos/huangjiahui/http://www.flickr.com/photos/huangjiahui/