Module A Fundamentals of Testing by K.N.Sasidhar

Fundamentals of Testing Why is Testing necessary? What is Testing? General Testing Principles Fundamental Test Process The Psychology of Testing Re-testing and Regression testing Expected Results Prioritisation of Tests

Why is testing necessary? Software systems context Define errors/mistake,faults,failures and reliability Errors and how they occur Cost of errors Exhaustive testing is impossible Testing and risk Testing and quality Testing and contractual, legal,regulatory requirements How much testing is enough

Software Systems context Software systems are an increasing part of life, from business applications to healthcare systems to consumer products etc… Software that does not work correctly can lead to many problems including loss of money, time or business reputation, and could even cause injury or death.

Define Errors, Fault,Failures According to BS 7925-1 Error: A human action that produces an incorrect result. Fault: Manifestation of an error in software or in implementation. Failure: Deviation of the software from its expected delivery or service. A human being can make an error(mistake),which produces a defect (fault,bug) in the code, in software system.If a defect in code is executed , the system will fail to do what it should do, causing a failure. Bug – Generic term for defect,fault,failure

Errors and How they occur - Can be introduced at any stage of SDLC We are all prone to making simple human errors Miscommunication, Poor requirements Working under pressures such as tight deadlines, budget restrictions,conflicting priorities Complexity of technologies / infrastructure Poorly documented code Due to environmental conditions

Cost of Errors May vary from nothing at all to large amount of money loss and even loss of life. A single failure can cost nothing or a lot. Software in safety-critical systems can cause death or injury if it fails. Ex: Online trading systems: $100,000 per minute revenue loss

Exhaustive Testing is impossible “ Executing a program with all possible combinations of inputs or values for program variables “ Only exhaustive testing can show a program is free from defects. - It is practically impossible due to time and resources - It is not cost effective It is much better to : - use testing techniques to give confidence - Base the testing on the risks to be contained

Testing and Risk - Testing is a risk reduction process “ The amount of testing performed depends on the risks involved ( Failure / Loss) Risks - As a basis for allocating the test time that is available and for selecting what to test and where to place emphasis. A priority must be assigned to each test so that most important tests are run first. The objective is to always “ minimise risk”

Testing and Quality Testing increases software quality “ Testing identifies faults and whose removal increases software quality by increasing the software’s potential reliability “. Testing is the measurement of software quality “ We measure how closely we have achieved quality by testing the relevant factors such as Correctness,Reliability,Usability, Maintainability etc.”

Quality Control : “ is the activity performed to ensue that a Product is ‘fit for Purpose’ “. Quality Assurance: “is the activity performed to test the process is being followed” Quality Management: Responsible choosing the correct process to ensure that the product is ‘fit for purpose’.

Testing and Contractual,Legal,Regulatory requirements Other factors that may determine the testing performed may be: Legal Contractual Normally defined industry specific standards Ex: FSA, US FAA, OFTEL

How much testing is enough ? Testing is a matter of judging risks against the cost of extra testing efforts. Planning testing effort before you begin and setting completion criteria ensure that testing is properly prioritised.

What is Testing ? Definition Common objectives of testing Purpose of testing at different phases of SDLC

Testing – Definition “ The Process of exercising software to verify that it satisfies specified requirements and to detect errors” BS 7925-1 Testing is : - a Risk reduction process - aimed at discovering faults - an iterative process - the measurement of quality

General Testing Principles Principle 1- Testing shows presence of defects Principle 2- Exhaustive testing is impossible Principle 3- Early testing Principle 4- Defect clustering Principle 5- Pesticide Paradox Principle 6- Testing is context dependent Principle 7- Absence-of-errors fallacy

Fundamental Test Process Test Planning Test Analysis and Design Test implementation and execution Test Recording Checking for test completion(Exit criteria)

The Psychology of Testing Why do we test? Tester-Developer relationships Test Independence

Why do we test ? Primarily to find faults in the software This can be perceived as being a destructive process,not constructive How does this fit with the mind set of the developer and tester ?

Developer characteristics ? Specialized Creative Valued by the organisation Trained Sensitive to criticism

Testers characteristics: Methodical Happy when they are finding faults Under-valued by the organisation Not always intended to be a tester Good communicators Need to be multi-talented Tact & diplomacy are essential

Is this true always ? Do Developers & Testers contribute to their problems ?

Developer – Tester Relationship -Communication is the Key !! - must be constructive not destructive - Developers must inform testers of any changes made to the system - Testers must report problems to developers clearly and concisely. - It is a two-way process !!

Test Independence Independent testing is more effective. The author should not test their own work -Assumptions made are carried into testing -People see what they want to see -There can be emotional attachment with the product - We’re human

Levels of Independence Where test cases are - designed by the person who writes the software under test - designed by another person - designed by people from another department - designed by people from another organisation - not chosen by a person

Re-Testing and Regression Testing Fault-fixing and re-testing Regression testing and automation Selecting regression test cases

Fault-fixing and Re-testing When ever a fault is detected and fixed,then the software should be re-tested to ensure that the original fault has been successfully removed. “The re-run of tests to show that what was not working has been corrected and is now working satisfactorily. - Make your tests repeatable Normally performed when a fault has been fixed. Can occur at any level of testing Should consider testing for similar or related faults

What is Regression testing ? “The re-run of tests with the intention of checking that what was working is still working” i.e modification have not caused unintended adverse side defects in the unchanged software It is performed whenever the software or its environment is changed When a major modification to the software is done, then the entire regression pack is likely to be run. For a minor changes. Test manager must be selective in identifying regression tests Regression tests are good candidates for test automation

Expected Results What are expected results ? -They are expected outcome of a test -The outcome may include output -It is the identification of the desired behavior of a test -They are determined before a test is executed Outcome is everything you expect to happen and output is physical output ( ex:Database changes and table changes) Outcome of a test may include a physical output.

Why do we need expected results ? To ensure that the outcome of a test is as expected To make a judgment as to whether to Pass or Fail a test How do we derive expected results? From documentation: From existing system From specialized user knowledge Also known as the “Oracle assumption” Should never be derived from the delivered system What if we don’t have them? -Testers can make the wrong judgment as to Pass/Fail -Go back to the Oracle

Prioritisation of tests Why do we need to Prioritise? How do we Prioritise? What is the Criteria?

Why do we need to Prioritise ? Time constraints Resource Constraints We need to get the best done in the time available How do we Prioritise? Identification of the Most Important Tests first(MITs)

Prioritisation basis: - Business criticality - Severity of potential failure - Visibility of failure - Likelihood of error - User requirements and priorities - Technical criticality and complexity - contractual requirements / Compliance requirement