1 Prediction of Software Reliability Using Neural Network and Fuzzy Logic Professor David Rine Seminar Notes.

Slides:



Advertisements
Similar presentations
On the application of GP for software engineering predictive modeling: A systematic review Expert systems with Applications, Vol. 38 no. 9, 2011 Wasif.
Advertisements

Design of Experiments Lecture I
THE INTELLIGENCE SYSTEM OF SOFTWARE COMPLEXITY AND QUALITY EVALUATION AND PREDICTION Oksana Pomorova, Tetyana Hovorushchenko Khmelnitsky National University.
Support Vector Machines
Early Effort Estimation of Business Data-processing Enhancements CS 689 November 30, 2000 By Kurt Detamore.
Comparative Analysis of Statistical Tools To Identify Recruitment-Environment Relationships and Forecast Recruitment Strength Bernard A. Megrey Yong-Woo.
1 The Role of the Revised IEEE Standard Dictionary of Measures of the Software Aspects of Dependability in Software Acquisition Dr. Norman F. Schneidewind.
Software Quality Ranking: Bringing Order to Software Modules in Testing Fei Xing Michael R. Lyu Ping Guo.
Software Quality Metrics
Fuzzy Logic Based on a system of non-digital (continuous & fuzzy without crisp boundaries) set theory and rules. Developed by Lotfi Zadeh in 1965 Its advantage.
POWER SYSTEM DYNAMIC SECURITY ASSESSMENT – A.H.M.A.Rahim, S.K.Chakravarthy Department of Electrical Engineering K.F. University of Petroleum and Minerals.
I welcome you all to this presentation On: Neural Network Applications Systems Engineering Dept. KFUPM Imran Nadeem & Naveed R. Butt &
SYSTEMS Identification Ali Karimpour Assistant Professor Ferdowsi University of Mashhad Reference: “System Identification Theory For The User” Lennart.
Supervised classification performance (prediction) assessment Dr. Huiru Zheng Dr. Franscisco Azuaje School of Computing and Mathematics Faculty of Engineering.
1 Software Testing and Quality Assurance Lecture 31 – Testing Systems.
Learning From Data Chichang Jou Tamkang University.
Development of Empirical Models From Process Data
1 Simple Linear Regression Chapter Introduction In this chapter we examine the relationship among interval variables via a mathematical equation.
CS Instance Based Learning1 Instance Based Learning.
Introduction to Systems Analysis and Design
Neuro-fuzzy Systems Xinbo Gao School of Electronic Engineering Xidian University 2004,10.
1 NASA OSMA SAS02 Software Reliability Modeling: Traditional and Non-Parametric Dolores R. Wallace Victor Laing SRS Information Services Software Assurance.
1 Forecasting Field Defect Rates Using a Combined Time-based and Metrics-based Approach: a Case Study of OpenBSD Paul Luo Li Jim Herbsleb Mary Shaw Carnegie.
University of Toronto Department of Computer Science © 2001, Steve Easterbrook CSC444 Lec22 1 Lecture 22: Software Measurement Basics of software measurement.
CS527: (Advanced) Topics in Software Engineering Overview of Software Quality Assurance Tao Xie ©D. Marinov, T. Xie.
Attention Deficit Hyperactivity Disorder (ADHD) Student Classification Using Genetic Algorithm and Artificial Neural Network S. Yenaeng 1, S. Saelee 2.
Introduction to Systems Analysis and Design Trisha Cummings.
Chapter 4 Components of the Software Quality Assurance System
A Shaft Sensorless Control for PMSM Using Direct Neural Network Adaptive Observer Authors: Guo Qingding Luo Ruifu Wang Limei IEEE IECON 22 nd International.
Software Software is omnipresent in the lives of billions of human beings. Software is an important component of the emerging knowledge based service.
Chapter 6 : Software Metrics
Chapter 9 Neural Network.
Outline What Neural Networks are and why they are desirable Historical background Applications Strengths neural networks and advantages Status N.N and.
Using Neural Networks to Predict Claim Duration in the Presence of Right Censoring and Covariates David Speights Senior Research Statistician HNC Insurance.
Software Measurement & Metrics
NEURAL NETWORKS FOR DATA MINING
Relationship Between in-situ Information and ex-situ Metrology in Metal Etch Processes Jill Card, An Cao, Wai Chan, Bill Martin, Yi-Min Lai IBEX Process.
INTERACTIVE ANALYSIS OF COMPUTER CRIMES PRESENTED FOR CS-689 ON 10/12/2000 BY NAGAKALYANA ESKALA.
Software Project Management Lecture # 3. Outline Chapter 22- “Metrics for Process & Projects”  Measurement  Measures  Metrics  Software Metrics Process.
Manag ing Software Change CIS 376 Bruce R. Maxim UM-Dearborn.
Zibin Zheng DR 2 : Dynamic Request Routing for Tolerating Latency Variability in Cloud Applications CLOUD 2013 Jieming Zhu, Zibin.
A.N.N.C.R.I.P.S The Artificial Neural Networks for Cancer Research in Prediction & Survival A CSI – VESIT PRESENTATION Presented By Karan Kamdar Amit.
Cmpe 589 Spring 2006 Lecture 2. Software Engineering Definition –A strategy for producing high quality software.
© The McGraw-Hill Companies, Inc., 2002 McGraw-Hill/Irwin Slide Cost Estimation.
Fault Tolerance Benchmarking. 2 Owerview What is Benchmarking? What is Dependability? What is Dependability Benchmarking? What is the relation between.
A Metrics Program. Advantages of Collecting Software Quality Metrics Objective assessments as to whether quality requirements are being met can be made.
Neural Networks Demystified by Louise Francis Francis Analytics and Actuarial Data Mining, Inc.
Introduction to Operations Research. MATH Mathematical Modeling 2 Introduction to Operations Research Operations research/management science –Winston:
Software Quality Assurance and Testing Fazal Rehman Shamil.
Special Challenges With Large Data Mining Projects CAS PREDICTIVE MODELING SEMINAR Beth Fitzgerald ISO October 2006.
Chapter 11  2000 by Prentice Hall System Analysis and Design: Methodologies and Tools Uma Gupta Introduction to Information Systems.
Learning Photographic Global Tonal Adjustment with a Database of Input / Output Image Pairs.
A field of study that encompasses computational techniques for performing tasks that require intelligence when performed by humans. Simulation of human.
Dynamic Neural Network Control (DNNC): A Non-Conventional Neural Network Model Masoud Nikravesh EECS Department, CS Division BISC Program University of.
1 Development of Empirical Models From Process Data In some situations it is not feasible to develop a theoretical (physically-based model) due to: 1.
IEEE AI - BASED POWER SYSTEM TRANSIENT SECURITY ASSESSMENT Dr. Hossam Talaat Dept. of Electrical Power & Machines Faculty of Engineering - Ain Shams.
LOAD FORECASTING. - ELECTRICAL LOAD FORECASTING IS THE ESTIMATION FOR FUTURE LOAD BY AN INDUSTRY OR UTILITY COMPANY - IT HAS MANY APPLICATIONS INCLUDING.
Neural networks (2) Reminder Avoiding overfitting Deep neural network Brief summary of supervised learning methods.
Verification vs. Validation Verification: "Are we building the product right?" The software should conform to its specification.The software should conform.
MANAGEMENT INFORMATION SYSTEM
Business Intelligence and Decision Support Systems (9 th Ed., Prentice Hall) Chapter 6: Artificial Neural Networks for Data Mining.
Software Defects Cmpe 550 Fall 2005
Regression Testing with its types
Table 1. Advantages and Disadvantages of Traditional DM/ML Methods
Introduction to Systems Analysis and Design
Test Case Test case Describes an input Description and an expected output Description. Test case ID Section 1: Before execution Section 2: After execution.
Masoud Nikravesh EECS Department, CS Division BISC Program
Fuzzy Logic Bai Xiao.
Evolutionary Ensembles with Negative Correlation Learning
Fuzzy Logic Based on a system of non-digital (continuous & fuzzy without crisp boundaries) set theory and rules. Developed by Lotfi Zadeh in 1965 Its advantage.
Presentation transcript:

1 Prediction of Software Reliability Using Neural Network and Fuzzy Logic Professor David Rine Seminar Notes

2 Overview Motivation Background and Related Work Guidelines for Modeling Problem Formulation Main Contributions Results Future Work

3 Motivation Software is often a key component of the high technology systems that are so common in modern society. High reliable software is critical both to software producer and users, as well as society in general, because failures of software can cause major disruption to business and can even threaten emergency service.

4 Motivation It is a challenge to be able to enhance the quality a model early enough to prevent problems from fault later in the life cycle because it is much more cost-effective to correct software faults early in the development process than later when they cause failure. This is why building software reliability growth models have gained considered importance in assessing reliability of software products.

5 Background One approach proposed by Brocklehurst et al. (IEEE Trans. On Software Eng. 1990) is to try a set of model and selecting the one that best suit the situation. This is a trial and error procedure. It was claimed that different models have different predictive capabilities at different phases of testing and there is no single model that can be relied on for accurate prediction in all circumstances (Whitely et al. IEEE Tran. On Software Eng. 1992).

6 D. Whitley et al. (IEEE Tran. On Software Eng. 1992) stated that,” the problem of selecting a model can be addressed in two ways: by generalizing the applicability of software reliability growth models by analyzing their predictability across a broad spectrum of representative data sets or by developing adaptive models, Nevertheless, the issue of generalization still remains as an open issue”. Background

7 Previous Research Decades of research results on many model structure for predictive software reliability. Many difficulties were reported that made analysis and design of such model a quite challenging process. For example: Missing data Large number of variables Strong co-linearity between the variables. Complex non-linear relationship between model variables. Outliers and small sizes of the data set.

8 Artificial Neural Networks (ANNs) ANNs approaches found to be useful in many applications. For example: Prediction System Identification and Control Computer Network Design Image Processing and many others.

9 Software Community Software Community also realized these advantages. They used ANN as a tool to solve problems in: Cost Estimation Software Reliability Engineering Strategy Software size estimation Software quality prediction

10 Interesting study An open-issue comparative study for developing predictive models of software metrics was provided (A. Gray and S. MacDonell 1997 in the Information and Software Technology Journal, Elsevier). This study suggested the following techniques: Least square regression Robust regression analysis Neural network Fuzzy systems Rule based systems Case-based reasoning in developing reliability models

11 Goal The goal is to improve accuracy of the Software Reliability Estimate.  Provide a comparison among parametric models and non- parametric software reliability growth models.  Develop better software reliability growth models using NNs and Fuzzy logic.

12 Proposed Methodology Whitley stated that,” The influence of the external parameters and other peculiarities of a model can be eliminated if we have a system that can develop its own model from the past failure history of the software system”. This is the methodology we are adopted in our current research. This why we suggest building software reliability growth models using prior collected fault data.

13 Possible Methodology cont. y(k)=f(y(k-1), y(k-2),…., y(k-m)) Where:y(k) is the accumulated faults at instant k and m is the selected number of delays Model Structure Past Failure Data Predicted Faults

14 Some Guidelines The problem of software reliability modeling can be split to number of sub-problems: Selection of the model structure Estimation of the model parameters Evaluation of the model prediction capabilities.

15 Linear Model Structure Using the provided data sets and according experimental studies, It was found that the best Linear model which can be developed using Least-Square Regression is the Auto-regression model order 4. Linear Model Structure

16 Least square regression Predicted faults Previous faults  We have implemented many model structure to predict faults.  According to experimental results, it was found that a regression model of order 4 is the best in our case.  This why we considered it as a base for comparison.

17 A Previous Model Whitley’s NNs Model A single-input single-output model was proposed. The model input is the execution time t=1,2,…, N. N is the number of tests The model output is the observed faults. Execution time t Observed faults. Hidden Layers

18 A Previous Model Whitley’s NNs Model Whitley’s NNs Model was build with the use of traditional model structure in mind which count on the execution time as a metric for predicting faults.

19 A Possible FF-NN Model Structure Input layer Output layer Hidden Layer

20 RBF-NN It was claimed that RBF trains faster than a FF-NN and that produce a better decision boundaries (Leonard and Kramer 1991). The hidden layer in the RBF is easier to interpret than the hidden layer in an FF-NN (Leonard and Kramer 1991).

21 RBF-NN RBF was realized by the Software Engineering Community as a useful tool for software engineering applications. For example, M. Shin and A. Goel have used RBF in software effort estimation for a well-known software effort dataset from NASA (IEEE Tran. Software Eng. Vol. 26, No. 6, June 2000).

22 A Possible RBF-NN Model Structure Input layer Output layer Hidden Layer

23 Fuzzy Logic Fuzzy Logic approach has been successfully used to solve varieties of problems in modeling and identification of nonlinear systems. For the problem under study, it was found that the relationship between the predictive model input and output is nonlinear.

24 Advantages of fuzzy logic Fuzzy logic is useful when dealing with incomplete sets of data. Fuzzy set membership of data gives greater sensitivity for decision making. Fuzzy logic is useful when dealing with vague (unclear) information.

25 Advantages of fuzzy logic Fuzzy Logic have been used in some preliminary work by the Software Engineering Community as a useful tool for software engineering applications. For example, Using a FF-NN with fuzzy weights to predict software reliability.

26 A Possible Fuzzy Model Structure In our work, we propose a regression fuzzy model structure to predict accumulated faults. Using the set of input-output data We specify the number of clusters We specify the number of rules Then, estimate the fuzzy model parameters.

27 Evaluation (Fitness) Criteria The well known criterion which have been extensively used by Software Engineering Community are: The Average Error and the Average Bias Mean Magnitude of Relative Error Total no. of defects to be detected in the future time Accumulated no. of faults detected at time j

28 An Evaluation Criterion One can use the Sum of the Square of the Error as an evaluation criterion. One can also compute the Correlation Error in some cases.

29 Possible Results FF-NN, RBF NN and Fuzzy logic models out perform the tradition software reliability growth model. FF-NN, RBF NN and Fuzzy logic models were able to provide models with smaller SSE than regression for all studied projects.