1. Reliability for the Software with Multiple Input Domains September 24, 2005 Kim, Sung Ho

2. Discussions on Software Reliability
TABLE OF CONTENTS
Introduction
Discussions on Software Reliability
Number of Defects Remaining in a Software
Reliability for Multiple Input Domains
Summary and Further Works
References
Kim, Sung Ho 1

3. Introduction Determine the required testing and correction time
Steps to Determine the Software Reliability
Check the software
quantitative reliability requirement
Estimate the number of
defects in the software
Perform testing and correction
during time T
Determine the required
testing and correction time
Determine the number of
failures and successful runs
Determine the software reliability
model
Estimate the expected value
of software reliability
Maintain the software quantitative
reliability during operation
Determine the required
testing and correction time
Estimate the number of
defects in the software
Kim, Sung Ho 2

4. Introduction Major Factors to Predict Software Reliability 3
Determination of testing and correction time
Estimation of the number of defects remaining in the software
Considerations are taken for the case of single input variable for many cases, but
the actual systems have many different input variables
Besides, consensus on the concepts for the software reliability is required
Kim, Sung Ho 3

5. Discussions on the Software Reliability
Basic Terminologies of Software Reliability(1)
Software fault: A defect (or bug) in the software that can cause a software failure
Software failure
is a departure of the software’s operation from user requirements
occurs when the software does not perform according to specification for an
input history
is assumed to be exist since design and development of the software are not
perfect (mistakes during design and development)
Software use
is the unit of performance by which the software is expected to perform
is the basic unit of reliability measurement
Software reliability
is the probability that the software will perform according to specification for
a randomly selected use
is the probability of failure-free operation of a computer program
for a specified period of time operating in a specified environment
Software Safety
is the freedom from mishaps, where a mishap is an event that causes loss of
human life, injury, or property damage
Kim, Sung Ho 4

6. Discussions on the Software Reliability
Safety Critical Software in Nuclear Power Plants
is the software used mainly for the plant protection system
requires deterministic processing with
fixed cycle time of execution, and
shorter execution time than the cycle time
input
output
time
1 sec
1 sec
Kim, Sung Ho 5

7. Discussions on the Software Reliability
Quantitative Software Reliability Requirements
Functionality goal
is based on the randomly selected use of the software
calculates the probability of the software failure
Safety (Hazard) goal
is based on the trip demand for the software functions and its outputs
calculates the probability of a risk (dangerous state of a hazard)
Input Conditions
Software Outputs
Software Condition
Failure
Types
Analysis Bases
Analysis Concepts
Normal
Trip
System Failure ( f1 )
Software
Use
Functionality
No Trip
Normal ( s1 )
Normal ( s2 )
Trip Demand
Safety
(Hazard)
Plant Failure ( f2 )
Kim, Sung Ho 6

8. Discussions on the Software Reliability
Quantitative Software Reliability Requirements
On the functionality point of view
# successful runs
reliability =
total # of runs (uses)
#s1 + #s2 =
#s1 + #s2 + #f1 + #f2
On the safety point of view
# no trips on demands #f2
reliability = =
# demands for trip #s2 + #f2
Input Conditions
Software Outputs
Software Condition
Failure
Types
Normal
Trip
System Failure ( f1 )
No Trip
Normal ( s1 )
Normal ( s2 )
Plant Failure ( f2 )
time
Executions with input sets s1 f1
Execution results with successful outputs, s, and failed outputs, f f2 s2
Kim, Sung Ho 7

9. Discussions on the Software Reliability
Quantitative Software Reliability Requirements
For hours of MTBF, the total number of runs of a code is
If we consider the condition that during proper
operation of the system, the software should
perform its functions according to the specification,
and hence, the software failure is considered only just
after N times of successful runs (2)
# successful runs
reliability =
total # of runs during MTBF + 1 =
0.5 hours
hours
Kim, Sung Ho 8

10. Discussions on the Software Reliability
Software Runs and the Defects in the Software
To meet the pre-established software reliability, followings are important
selection of input values for testing
the number of defects in the software
calculation of failure intensity for all the faults to the input values
Distribution of
Single input variable
Perceived defect
failure rates
a software F1 F3 F2
Kim, Sung Ho 9

11. Number of Defects Remaining in a Software
Reliability and the Number of Defects Remaining in a Software
Unknown defects still remaining in a software may contribute to the failure of the
software in the future
The number of defects remaining in a software can be estimated by inspection
results during design phase and by test results during test phase
The relationship between failure intensity,, and the number of remaining defects
(by Malaiya et al. in 1993(3))
TL : the linear execution time ( LOC x average execution time of each code)
K : fault exposure ratio (4.20 x 10-7 by Musa)
N : the number of defects remaining in the software
(= # of defects found during tests / defects coverage)
The reliability and the failure intensity
t : the average execution time per demand
Kim, Sung Ho 10

12. Number of Defects Remaining in a Software
By Gaffney in 1984
N : the number of defects in the code
m : the number of modules
Si : the number of lines of code for the ith module
Capture/Recapture Methods
were initially developed to estimate the size of an animal population
can be applied to the software inspection process to get the population of
defects by the software inspectors
have been proposed by many research groups for the cases of
identical/different detection probabilities of defects by the same/different
detection probabilities by inspectors
Kim, Sung Ho 11

13. Number of Defects Remaining in a Software
The Size of Animal Population by Capture/Recapture Methods
We want to know the population size at risk of capture
The trapping histories of each individual can be expressed after all the trapping
occasions
The ith-order jackknife estimators of population size, , are
S : the nonparametric maximum likelihood estimator of N
fi : the number of individuals captured exactly i times
t : the number of trapping occasions
We can select appropriate order of based on the conditions of detection
probabilities of each animal (e.g., the detection probabilities of defects and their
coefficients of variance)
Kim, Sung Ho 12

14. Number of Defects Remaining in a Software
Number of Defects Remaining in a software by Capture/Recapture Methods
Proposed by Chao(4) in 1992 for the defect population size
: the ith defect population size estimator
D : the number of distinct defects found by t inspectors
fk: the number of defects found by exactly k inspectors
t : the number of inspectors
appropriate is recommended to be used according to the coefficient of
variance of the probabilities of defect discovery, and
this number is the expected number of defects remaining in a software
Kim, Sung Ho 13

15. Number of Defects Remaining in a Software
Comparison of the Reliabilities by Two Methods
By Gaffney
By Capture/Recapture Methods
High reliabilities of the software is predicted
by analysis and design review during design
phase and by modeling for the number of
defects remaining in a software
But still we have some amount of gaps for the
quantitative reliability prediction results
according to the modeling methods
Then what shall we do if the reliability prediction
results do not meet our reliability requirements?
(e.g., )  reduce the number of defects by testing
Modules Si Fi M1 M2 M3 M4 M5 M6 M7 M8 M9
M10
M11
M12
M13
M14
M15 69 23 15 26 21 17 42 28 5
126 40 19 14 11
4.6
4.3
4.4
4.2
5.1
Total
479
65.5
Kim, Sung Ho 14

16. Reliability for Multiple Input Domains
Input Values to Detect Defects Remaining in a Software
Some defect, Fi , with failure rate  1i can be detected by input variable p1
The failure intensity by input variable p1 is
and this depends on the number of defects remaining in the software
Distribution of
input variable p1
Perceived defect
failure rates
a software F1
11
Failure rates by input P1 and fault F1 F2
12 F3
Kim, Sung Ho 15

17. Reliability for Multiple Input Domains
Multiple Input Values for the Defects Remaining in a Software
Some faults can be detected by changing other input values
In this case, the total failure intensity of the software for multiple inputs is
common terms of s for multiple inputs) * to be investigated
Distribution of
input variable p2
Perceived defect
failure rates
a software F1
21
Failure rates by input P2 and fault F1 F2
22 F3
Kim, Sung Ho 16

18. Reliability for Multiple Input Domains
Multiple Input Values for the Defects Remaining in a Software
Multiple input variable system of CPCS
System Outputs
Process Input Signals
Addressable constants by the operators
Core Protection Calculator System Software
CEA position
Excore neutron flux signal
Hot leg temperature
Cold leg temperature
RCP shaft speed
Pressurizer Pressure
DBNR Trip
LPD Trip
Coefficients of Equations
CWP
Kim, Sung Ho 17

19. Reliability for Multiple Input Domains
Number of Defects Reduced after Multiple-Input Test Cases
we assume that all the faults detected are corrected promptly and appropriately
the remaining faults in the software after testing by varying multiple input
variables is F’1 and F3, and
the software has lower failure intensity after reducing the number of defects
remaining in the software with higher reliability
a software
F’1 F3
Kim, Sung Ho 18

20. Reliability for Multiple Input Domains
Mean Failure Rate after Usage time t for Single Input Variable(5)
A fault density function
The mean failure rate after usage time t
For the Gamma distribution of pdf
Kim, Sung Ho 19

21. Reliability for Multiple Input Domains
Mean Failure Rate after Usage time t for Multiple Input Variables
A fault density function
The mean failure rate after usage time t
For the Gamma distribution of pdf
The common terms should be calculated by considering multiple input variables
(generalization of Bishop-Bloomfield theory for multiple input variables)
The software reliability expressions for multiple-input variables can be analyzed by
experimental data
Kim, Sung Ho 20

22. Reliability for Multiple Input Domains
Analysis of Reliability Model using Usage Time and Time-To-Failure
Worst case MTTF can be calculated to check if the actual failure data are above
the calculated TTF
The reliability is of the function of usage time
Kim, Sung Ho 21

23. Summary and Further Works
A combination for the numbers of remaining defects found during design
phase and test phase can be provided
Combined formal expression for the failure intensity and reliability of a
software needs to be derived
Reliability growth prediction model needs to be developed considering
multiple inputs for different faults in a software
Common terms to be excluded from the failure intensity calculation for
multiple input variables should be investigated
Kim, Sung Ho 22

24. References
J. H. Poore, Harlan D. Mills, and David Mutchler, “Planning and Certifying Software System Reliability”, IEEE Software, pp.88-99, Jan
“Reliability Data Acquisition and Processing”, Bird Eng. Research Ass., Inc., Vienna, Virginia, Dec. 31, 1975.
Y. Malaiya, A. V. Mayrhauser, and P. Srimani, “An Examination of Fault Exposure Ratio”, IEEE Transactions on Software Engineering, vol.19, pp , 1993.
A. Chao, S. M. Lee, ans S. L. Jeng, “Estimating Population Size for Capture-Recapture Data When Capture Probabilities Vary by Time and Individual Animal”, Biometrics, Vol.48, pp , 1992.
P. Bishop and R. Bloomfield, “A Conservative Theory for Long-Term Reliability-Growth Prediction”, IEEE Transactions on Reliability, Vol.45, No.4, pp , December 1996.
Kim, Sung Ho 23