 CS 5380 Software Engineering Chapter 8 Testing

Validation vs Verification  Validation  Are we building the right product?  Verification  Are we building the product right? Chapter 7 - Design and Implementation2

Purpose of Testing  Demonstration of fulfillment of requirements  Identification of defects Chapter 7 - Design and Implementation3

Meeting Requirements (Validation)  At least one test per requirement  xxxxxxxxx  Often more than one test  Several valid cases  xxxxxxxxx Chapter 7 - Design and Implementation4

Identification of Defects  Identifying undesirable results  System crash  Incorrect computation  Data corruption  May need many tests Chapter 7 - Design and Implementation5

Testing is Never Final  Dijkstra: Testing can show presence of errors, not their absence  “There is always one more bug” Chapter 7 - Design and Implementation6

How Much Testing? - Factors Driving Testing  Software purpose  Safety drives high requirements  User expectations  Some tolerance  Marketing  “We lose money every day the product is not delivered” Chapter 7 - Design and Implementation7

Inspection vs Testing  Both address same issues  Does the software meet requirements?  Does the software fail in some situations?  Advantage of Inspection  Identification of multiple errors at one time possible  Identification of errors in incomplete software  Looks beyond defects  Inefficient code, poor code structure, reuse  Advantage of testing  Can be automated  Detailed scenarios evaluated Chapter 7 - Design and Implementation8

Inspection  Gives insight into the method, and hence potential flaws  Does not give us automation, regression  May be difficult to comprehend all cases  Complements other methods  Good review point is before check-in Chapter 7 - Design and Implementation9

Testing Stages  Development  Done during development, by programmers  Often on development systems  Release  Testing complete system before delivery  Generally on a separate test system  Formal test plans  User  Test by users, on their systems  Real world application  Beta testing Chapter 7 - Design and Implementation10

Development Testing Levels  Unit  Object, function level  Component  Interaction of several objects  Focus on component interactions  System  Entire system  Focus on interactions Chapter 7 - Design and Implementation11

Unit Testing  Object  Automated  Setup  Perform operation  Evaluate Results  Execution  Not UI Chapter 7 - Design and Implementation12

State Diagram Testing  System or object for which you have a state diagram  Test all transitions  State, event Chapter 7 - Design and Implementation13

Activity Diagram Testing  Diagram clearly shows alternatives to be tested. Chapter 7 - Design and Implementation14

Chapter 7 - Design and Implementation15

Component Testing  Testing of composite components  Several objects  Interfaces to consider  Parameter (object)  Procedural (object)  Shared memory  Message passing Chapter 7 - Design and Implementation16

Component Examples  Objects  Fast city grocers  Web browser Chapter 7 - Design and Implementation17

System Testing  Testing all components that come together  Internal objects  External objects – GUI, db  External systems – credit card Chapter 7 - Design and Implementation18

Regression Testing  Testing that past cases should still work  Automation is key to effective regression testing Chapter 7 - Design and Implementation19

Release testing  Similar to system testing, but  Separate team (not development)  Goal: more about meeting requirements than finding bugs  Goal: verify to the customer that the software is ready Chapter 7 - Design and Implementation20

Scenario Testing  Complete scenarios with data  May be followed by user  May be automated. Chapter 7 - Design and Implementation21

Performance Testing  Response times  Concurrency response  Identify degradation issues  Failure due to unexpected combination of events  Stress the system beyond normal use Chapter 7 - Design and Implementation22

User Testing  Alpha – users with developers  Beta –  Users in production mode  Generally to identify bugs  Acceptance testing  Customer acceptance  Acceptance plan needed Chapter 7 - Design and Implementation23

Release Cutoff  Software released when number of bugs/type tolerable  Unusual case causes software to crash  Unexplained loss of data – single record lost  Data corrupt (verified in customer data)  Single minor function (.1% of users use this) fails  Software lacks feature (50% of users want it) Chapter 7 - Design and Implementation24

Extent of Testing  Depends on Purpose  Health/Safety Critical – insulin pump  Credit card retention  Web ordering  Depends on User Expections  Depends on Marketing pressures  New customer delivery needed  Customer testimonials/references  Features for existing customers  Existing customer response to errors Chapter 7 - Design and Implementation25

Testers  Who does test?  Object level?  Programmer other than developer  Component level?  Programmers/ Test bed specialists  Integration level?  User domain / Programmers  Full scenario  Customers / Beta Chapter 7 - Design and Implementation26

Experience with Testers  Search for people that test well  Most people will use a system for 1 hour and say it has 2 bugs  Another tester might use the same software for 10 minutes and find 10 bugs Chapter 7 - Design and Implementation27