CS351 © 2003 Ray S. Babcock Software Testing What is it?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test? Pass? Fail?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test? Pass? Fail? When do you do testing?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test? Pass? Fail? When do you do testing? Who should do testing?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test? Pass? Fail? When do you do testing? Who should do testing? How do you do testing?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test? Pass? Fail? When do you do testing? Who should do testing? How do you do testing? Why do you do testing?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test? Pass? Fail? When do you do testing? Who should do testing? How do you do testing? Why do you do testing? What is the results of doing testing ?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test? Pass? Fail? When do you do testing? Who should do testing? How do you do testing? Why do you do testing? What is the results of doing testing ? Deliverables?

CS351 © 2003 Ray S. Babcock Software Testing What is it? What is a good test? Pass? Fail? When do you do testing? Who should do testing? How do you do testing? Why do you do testing? What is the results of doing testing ? Deliverables? When do you stop testing?

CS351 © 2003 Ray S. Babcock What is testing? “Testing is a process of executing a program with the intent of finding an error.”

CS351 © 2003 Ray S. Babcock What is testing? “Testing is a process of executing a program with the intent of finding an error.” “A good test case is one that has a high probability of finding an as-yet- undiscovered error.”

CS351 © 2003 Ray S. Babcock What is testing? “Testing is a process of executing a program with the intent of finding an error.” “A good test case is one that has a high probability of finding an as-yet- undiscovered error.” “A successful test is one that uncovers an as-yet-undiscovered error.”

CS351 © 2003 Ray S. Babcock What is a good test? What we need here is an attitude adjustment! When we find errors, we often say “oh no!” When we find errors, we should say “oh yes!, we found it earlier rather than later.” There will always be one more bug.

CS351 © 2003 Ray S. Babcock About that term bug The term came from the old machine days when real bugs crawled into machines. Admiral Grace Hopper used the term when referring to a moth that crawled into the Eniac and fouled a relay. When using this term to refer to an error in a program, it implies that it somehow “CRAWLED IN FROM OUTSIDE”

CS351 © 2003 Ray S. Babcock Reality Check! YOU PLACED EVERY SINGLE BUG INTO YOUR CODE YOURSELF!

CS351 © 2003 Ray S. Babcock Reality Check! YOU PLACED EVERY SINGLE BUG INTO YOUR CODE YOURSELF! You designed it. You implemented it. And now you're surprised that you have to find it???

CS351 © 2003 Ray S. Babcock When do you do testing? In many life cycle models, there is a “testing phase”. This implies you don't do anything before this phase and that you are all done when the phase is over. Acutally we now think of testing across the entire life cycle. You are always thinking about testing.

CS351 © 2003 Ray S. Babcock When do you do testing? You can consider testing in every step of the project. ● Test an idea. ● As you analyze, consider testing. ● Consider how you will test each and every requirement. ● Test your design. ● Test your implementation (unit, integration, system, performance, stress) ● Test your documentation. ● Test your usability.

CS351 © 2003 Ray S. Babcock Who should do testing? There are two sides to doing your own testing? ● You know your code best so you can design test data quickly. ● You know your code best and often assume you have done it right. This often causes you to overlook the obvious errors.

CS351 © 2003 Ray S. Babcock Who should do testing? Some suggestions: ● Project Manager ● Software Engineer ● Testing Specialists

CS351 © 2003 Ray S. Babcock How do you do testing? Idea : pass it by others? Analysis : Have you looked at everything? Requirements : How will you verify that this requirement has been met by the software? Design : Design/Requirement matrix. Implementation : This is where we tend to concentrate most. Unit, Integration, System, Performance, Stress, Fail-Safe. Documentation : Review by normal users.

CS351 © 2003 Ray S. Babcock Test Plan A written document detailing how the testing for a product will be done. A dynamic document, constantly updated throughout the project. Used to keep track of adaptive and perfective maintenance after the project is delivered.

CS351 © 2003 Ray S. Babcock Why do you do testing? It is a myth to say “Testing assures that there are no errors in your code.” Testing CANNOT prove a program is error- free. I like the following way to describe it: “Testing raises your confidence that the program is performing correctly.”

CS351 © 2003 Ray S. Babcock Performing Correctly? This normally means MEETS REQUIREMENTS Verification : “Are we building the product right?” Validation : “Are we building the right product?”

CS351 © 2003 Ray S. Babcock What is the results of doing testing? You have more confidence that the program is working correctly. You have documentated all tests. You have developed test data sets and saved all test suites for future regressive testing. You have satisfied your customer by delivering what they wanted.

CS351 © 2003 Ray S. Babcock When do you stop testing? When you are satisfied that it is now completely correct?

CS351 © 2003 Ray S. Babcock When do you stop testing? When you are satisfied that it is now completely correct? NO!

CS351 © 2003 Ray S. Babcock When do you stop testing? When you are satisfied that it is now completely correct? NO! When you run out of time?

CS351 © 2003 Ray S. Babcock When do you stop testing? When you are satisfied that it is now completely correct? NO! When you run out of time? NO!

CS351 © 2003 Ray S. Babcock When do you stop testing? When you are satisfied that it is now completely correct? NO! When you run out of time? NO! When you run out of money?

CS351 © 2003 Ray S. Babcock When do you stop testing? When you are satisfied that it is now completely correct? NO! When you run out of time? NO! When you run out of money? NO!

CS351 © 2003 Ray S. Babcock When do you stop testing? When you are satisfied that it is now completely correct? NO! When you run out of time? NO! When you run out of money? NO! An author suggests the following as an answer: “When the product is retired!”

CS351 © 2003 Ray S. Babcock Footnote 1 (to this testing introduction) You've heard me state Babcock's first rule: “NEVER DEMONSTRATE ANYTHING” I also have a rule related to software testing: “IF IT LOOKS LIKE IT'S WORKING, MAYBE IT IS” The “fire over there, looking over here” story.

CS351 © 2003 Ray S. Babcock Footnote 2 (to this testing introduction) Regression testing is the verification that the rest of the package still works when you change a small part of it. The “MIDAS data base” story.