1. Different Testing Methodology
Inspection and Review (People Intensive)
Mostly applicable documents and non-machine executable material
requires preparation and some training in looking for problem, recording, follow-up, etc.
Formal proof of correctness (Very Skill Intensive)
Mostly applicable --- very small algorithmic solutions
requires formal training in theorem proving
Running code (Machine Intensive & People)
Applies to all executable material
requires a lot of preparation of test cases and some training in test process, test execution, and test tools

2. Test Case Generation How much do you test ?
From code point of view ( what is the goal?)
1 to 100% of executable statements
1 to 100% of distinct paths ( #of predicates + 1)
1 to 100% of program logic (loops can be a bear!)
From data point of view (what is the goal ?)
every input variable is assigned a valid value and executed once
every input variable is assigned a valid and an invalid value and executed once
every input variable is assigned a value outside of its boundary, on the boundary, and inside of its boundary and executed. (boundary value testing)

3. Total # of Paths .vs. Total Executable Statement Coverage1
Since for each binary decision, there are 2 paths and
there are 3 in sequence, there are 23 = 8 total “logical” paths
path1 : S1-C1-S2-C2-C3-S4
path2 : S1-C1-S2-C2-C3-S5
path3 : S1-C1-S2-C2-S3-C3-S4
path4 : S1-C1-S2-C2-S3-C3-S5
path5 : S1-C1-C2-C3-S4
path6 : S1-C1-C2-C3-S5
path7 : S1-C1-C2-S3-C3-S4
path8 : S1-C1-C2-S3-C3-S5
All Statement Coverage: S1, C1, S2, C2, S3, C3, S4, S5
With path1 we covered S1, C1, S2,C2, C3, S4 (S3 & S5 uncovered)
But with path5 we cover S1, C1, S2, C2, S3, C3, S5 C1 2 3 S2 4 C2 5 6 S3 7 C3 9 8 S5 S4

4. When do we Stop Testing ? Most common goal for stopping:
When we have i) executed all of our test cases and when ii) all the failures which came from the testing are fixed.
Less common goal for stopping:
When we have i) executed a fixed percentage of test cases and when ii) all the failures which came from the testing are fixed
When we have i) executed all the test cases and when ii) only a fixed percentage of the failures which came from the testing are fixed
Lesser common goal for stopping:
devising a stopping scheme with a goal and stop when the goal is reached.

5. A Devised Test Stopping Scheme
Seeding: Faults into the product and NOT tell testers where
assumes that (detected seeded errors)/(total seeded errors) is equal to (detected real errors)/(total real errors)
look at a way to utilize this
dse/tse = dre/tre (from above)
tre (total real errors) = (tse * dre)/dse
if the goal is to have x% of the estimated total real errors found prior to stopping, we can look at the ratio of dre/tre.
If the x% is met or exceeded, then we can STOP
if not, we can continue to run more test cases and monitor the results:
dse increases with no increase in dre (good situation: because tre = (tse*dre)/dse tse is fixed thus dse increase will mean lesser tre)
dre increases with no increase in dse (bad situation: dre increase drags an increase in tre because tre = (tse*dre) /dse )
both dse and dre increases (depends on the numbers)
Make sure the all seeded faults are removed prior to release

6. Look at An Example tse = 20 dse = 15 dre = 50
tre = (20 * 50) / 15 => approximately 67 total real errore estimated
dre/tre = 50/67 = appr. 75 %
if you want to have larger dre/tre ratio such as 80% (more faults are detected and fixed) then :
increase dre (this can mean that there are really more tre if there is no comparable increase in dse)
decrease tre via finding more seeded faults (increasing dse) without running into any real fault.
*** another mechanism is to introduce more total seeded faults(tse) and see if dse keeps on increasing without comparable increase in dre ***
*** this method is not fool proof because enough increase in dre will mean more testing is needed with more tse and the cycle continues.

7. Test -Fix Cycle
Testing-Fix cycle involves both the testers and the fixers and therefore must be managed.
Problems found must be described and submitted for fix
Resource must be assigned to diagnose (accept or reject) the failures found
Resource must be assigned to fix the failures
Resources must be set aside to accept the fixes and merge them into the next test “build cycle.”
Resource and schedule must be set to include re-test of the fixes

8. “Possible” Problem Report
Problem report should contain some of these items:
person who found the problem
date of problem submission (problem “open” date)
problem id #
component and version id of the software that was tested
test case(s) used
brief description of the problem symptoms with possibly screen snapshots
tester’s assessment of the problem severity
if a retest of another problem, give the earlier problem id #
Current Status: open; in fix; fixed; closed
closed date if closed

9. Test- Fix Cycle The Testing Side The Fixing Side Any Problem ?
Test the
Software
Submit the
problem
Yes
Software
Test
Library NO
Problem
Report DB
The Testing Side
The Fixing Side
Accept
Problem ?
Send fixed
material
to test library
Update
Problem
Status
Fix
Problem
Yes NO