CS 5380 Software Engineering Chapter 8 Testing
Validation vs Verification Validation Are we building the right product? Verification Are we building the product right? Chapter 7 - Design and Implementation2
Purpose of Testing Demonstration of fulfillment of requirements Identification of defects Chapter 7 - Design and Implementation3
Meeting Requirements (Validation) At least one test per requirement xxxxxxxxx Often more than one test Several valid cases xxxxxxxxx Chapter 7 - Design and Implementation4
Identification of Defects Identifying undesirable results System crash Incorrect computation Data corruption May need many tests Chapter 7 - Design and Implementation5
Testing is Never Final Dijkstra: Testing can show presence of errors, not their absence “There is always one more bug” Chapter 7 - Design and Implementation6
How Much Testing? - Factors Driving Testing Software purpose Safety drives high requirements User expectations Some tolerance Marketing “We lose money every day the product is not delivered” Chapter 7 - Design and Implementation7
Inspection vs Testing Both address same issues Does the software meet requirements? Does the software fail in some situations? Advantage of Inspection Identification of multiple errors at one time possible Identification of errors in incomplete software Looks beyond defects Inefficient code, poor code structure, reuse Advantage of testing Can be automated Detailed scenarios evaluated Chapter 7 - Design and Implementation8
Inspection Gives insight into the method, and hence potential flaws Does not give us automation, regression May be difficult to comprehend all cases Complements other methods Good review point is before check-in Chapter 7 - Design and Implementation9
Testing Stages Development Done during development, by programmers Often on development systems Release Testing complete system before delivery Generally on a separate test system Formal test plans User Test by users, on their systems Real world application Beta testing Chapter 7 - Design and Implementation10
Development Testing Levels Unit Object, function level Component Interaction of several objects Focus on component interactions System Entire system Focus on interactions Chapter 7 - Design and Implementation11
Unit Testing Object Automated Setup Perform operation Evaluate Results Execution Not UI Chapter 7 - Design and Implementation12
State Diagram Testing System or object for which you have a state diagram Test all transitions State, event Chapter 7 - Design and Implementation13
Activity Diagram Testing Diagram clearly shows alternatives to be tested. Chapter 7 - Design and Implementation14
Chapter 7 - Design and Implementation15
Component Testing Testing of composite components Several objects Interfaces to consider Parameter (object) Procedural (object) Shared memory Message passing Chapter 7 - Design and Implementation16
Component Examples Objects Fast city grocers Web browser Chapter 7 - Design and Implementation17
System Testing Testing all components that come together Internal objects External objects – GUI, db External systems – credit card Chapter 7 - Design and Implementation18
Regression Testing Testing that past cases should still work Automation is key to effective regression testing Chapter 7 - Design and Implementation19
Release testing Similar to system testing, but Separate team (not development) Goal: more about meeting requirements than finding bugs Goal: verify to the customer that the software is ready Chapter 7 - Design and Implementation20
Scenario Testing Complete scenarios with data May be followed by user May be automated. Chapter 7 - Design and Implementation21
Performance Testing Response times Concurrency response Identify degradation issues Failure due to unexpected combination of events Stress the system beyond normal use Chapter 7 - Design and Implementation22
User Testing Alpha – users with developers Beta – Users in production mode Generally to identify bugs Acceptance testing Customer acceptance Acceptance plan needed Chapter 7 - Design and Implementation23
Release Cutoff Software released when number of bugs/type tolerable Unusual case causes software to crash Unexplained loss of data – single record lost Data corrupt (verified in customer data) Single minor function (.1% of users use this) fails Software lacks feature (50% of users want it) Chapter 7 - Design and Implementation24
Extent of Testing Depends on Purpose Health/Safety Critical – insulin pump Credit card retention Web ordering Depends on User Expections Depends on Marketing pressures New customer delivery needed Customer testimonials/references Features for existing customers Existing customer response to errors Chapter 7 - Design and Implementation25
Testers Who does test? Object level? Programmer other than developer Component level? Programmers/ Test bed specialists Integration level? User domain / Programmers Full scenario Customers / Beta Chapter 7 - Design and Implementation26
Experience with Testers Search for people that test well Most people will use a system for 1 hour and say it has 2 bugs Another tester might use the same software for 10 minutes and find 10 bugs Chapter 7 - Design and Implementation27