Software testing techniques TESTING BASED ON ERROR GUESSING Presentation on the seminar Kaunas University of Technology

Testing based on error guessing 2018.02.16 Testing based on error guessing 2 Agenda What is an error? Testing based on error guessing Purpose Used techniques How to improve your error guessing techniques? How to design test cases? Effectiveness Pros and Cons Examples

Testing based on error guessing 2018.02.16 Testing based on error guessing 3 What is an error? The occurrence of an incorrect result produced by a computer A software error is present when the program does not do what its end user reasonably expects it to do.    A human action that produces an incorrect result.  [IEEE 610] 

Testing based on error guessing 2018.02.16 Testing based on error guessing 4 Testing based on error guessing Test design technique where the experience of a tester is used to find the components of software where defects might be present It is the process of using intuition and past experience to fill in the gaps in the test data sets.

Purpose of error guessing (1) 2018.02.16 Testing based on error guessing 5 Purpose of error guessing (1) The purpose of error guessing is to focus the testing activity on areas that have not been handled by the other more formal techniques, such as equivalence partitioning and boundary value analysis Error guessing is as important as equivalence partitioning and boundary value analysis because it is intended to compensate for the inherent incompleteness of those functions. inherent incompleteness - būdingas neišbaigtumas 5

Purpose of error guessing (2) 2018.02.16 Testing based on error guessing 6 Purpose of error guessing (2) As equivalence partitioning and boundary value analysis complement one another, error guessing complements both of these techniques. Example in the next slide illustrates the controlled results from these approaches

Purpose of error guessing (3) 2018.02.16 Testing based on error guessing 7 Purpose of error guessing (3)

Testing based on error guessing 2018.02.16 Testing based on error guessing 8 Used techniques Testers who are effective at error guessing use a range of techniques: Knowledge about the AUT(1), such as the design method or implementation technology Knowledge of the results of any earlier testing phases (particularly important in Regression Testing) Experience of testing similar or related systems (and knowing where defects have arisen previously in those systems) Knowledge of typical implementation errors (such as division by zero errors) General testing rules 1 AUT - Application Under Test

How to improve your error guessing techniques? (1) 2018.02.16 Testing based on error guessing 9 How to improve your error guessing techniques? (1) Improve your memory: List interesting error-types you come across Use existing bugs lists (for example as an appendix in Testing Computer Software, 2nd Edition) Improve your technical understanding: Go into the code, see how things are implemented, understand concepts like buffer overflow, null pointer assignment, array index boundaries, iterators, etc Learn about the technical context in which the software is running, special conditions in your OS, DB or web server

How to improve your error guessing techniques? (2) 2018.02.16 Testing based on error guessing 10 How to improve your error guessing techniques? (2) Remember to look for errors not only in the code. Look for: Errors in requirements Errors in design Errors in coding Errors in build Errors in testing Errors in usage

How to design test cases? 2018.02.16 Testing based on error guessing 11 How to design test cases? There are no specific tools to use Test cases can be designed depending on the situation: It can be drawn when an unexpected/undocumented error is found while testing operations It can be drawn from functional documents

How to design test cases? Historical data about defects that have been found in code that the same developer or developers have written in the past can be used For example, if you know from past experience that developer A sometimes forgets to use "==" instead of "=", and developer B sometimes forgets to check pointers for NULL after allocating memory, then those are defects that you might wish to look for in any code written by those two developers in the future

Testing based on error guessing 2018.02.16 Testing based on error guessing 13 Effectiveness Different people with different experience will show different results Different experiences with different parts of the software will show different results As tester advances in the project and learns more about the system, he/she may become better in Error Guessing

Testing based on error guessing 2018.02.16 Testing based on error guessing 14 Pros and Cons + Highly successful testers are very effective at quickly evaluating a program and running an attack that exposes defects + Can be used to complement other testing approaches + It is a skill that is well worth cultivating since it can make testing much more effective and efficient - Testing experience is essential - Relies on tester’s intuition

Testing based on error guessing 2018.02.16 Testing based on error guessing 15 Example 1 Requirement 1: Contact No. should be numerals and should be not less than 10 characters Application is having Contact No. field These are the Error guessing technique: What will happen if you left Contact No. blank? What will happen if you enter other than numerals? What will happen if you enter less than 10 numerals? What will happen if you enter alphabets in between the digits? …

Testing based on error guessing 2018.02.16 Testing based on error guessing 16 Example 2 Requirement 2. Entered arrival date should be numerals and in valid range. Application has arrival date field These values for testing can be used: 2011.04.31 – April has only 30 days 2011.02.29 – 2011 is not a leap-year …

Testing based on error guessing 2018.02.16 Testing based on error guessing 17 Questions What is testing based on error guessing? What is the purpose of this testing? What techniques are used for this testing and how test cases can be designed?

Testing based on error guessing 2018.02.16 Testing based on error guessing 18 References http://istqb.org/download/attachments/2326555/ISTQB+Glossary+of+Testing+Terms+2+1.pdf Just enough software test automation, Daniel J. Mosley, Bruce A. Posey. Page 64. http://www.managedtesting.net/2010/09/error-guessing-and-exploratory-testing/ http://searchsoftwarequality.techtarget.com/tip/Finding-software-flaws-with-error-guessing-tours http://en.wikipedia.org/wiki/Error_guessing