1. 27th International Symposium on Software Reliability Engineering
ISSRE’2016
Error Abstraction Accuracy and Fixation during Error-based Requirements Inspections
Supported by National Science Foundation Awards and
Authors:
Vaibhav Anu, Gursimran Walia (North Dakota State University)
Wenhua Hu, Jeffrey C. Carver (University of Alabama)
Gary Bradshaw (Mississippi State University)

2. Human Errors Human Errors Slips Lapses Mistakes
Failure in the execution of plan
Failure in the plan of action
EXECUTION
EXECUTION
PLANNING
e.g., lack of attention.
e.g., inadequate planning, mostly from a lack of knowledge.
e.g., forgetting one or more steps in a process. 2

3. Proposed APPROACH: Error ABSTRACTION AND INSPECTION (EAI)
Error Abstraction (EA) Process
What caused that fault?
List of Human Errors
List of New Faults (found during human error based reinspection)
Reinspection
(human error based)
Requirements
Document (SRS)
List of Faults (found during FC Inspection)
Inspection (FC based)

4. THE Human Error taxonomy (HET)
Requirements Phase Human Errors
STEP 1: Identifying Requirements Phase Human Errors from Software Engineering Literature
STEP 2: Human Error Classification System Identified from Cognitive Psychology Literature
Clerical Errors
SLIPS
LAPSES
MISTAKES
Lack of consistency in Requirement Specifications
Loss of information from stakeholders
Accidentally overlooking requirements
Application errors
Environment errors
Information Management errors
Wrong assumptions
Low understanding of each other’s roles
Mistaken belief that it is impossible to specify non-functional requirements in a verifiable form
Not having a clear demarcation between client and users
Lack of awareness of sources of requirements
Problem-Solution errors
Inadequate Requirements Process
Syntactic errors

5. Purpose of current RESEARCH
The current study examines the impact of Error Abstraction (EA) accuracy on inspectors’ fault detection effectiveness (during human error based reinspection).
Another aspect of this research evaluates the “fixation tendencies” of inspectors during EA.

6. Empirical study design
Training: Inspecting SRS using HET
PRE-TEST
Error Report Form
Re-inspect SRS for remaining faults
New Fault List 3 1
Reflection 2
Abstract and Classify Errors
Inspectors
POST -TEST 3 1 2
New Fault Form
Fault Form
Error Report Form
Dr. Walia, I do not have the original experiment design figure. The one I have is monochromatic. Can you please replace this with the original.
Inspect SRS using Errors
Fault Checklist
Error
Abstraction
Inspectors
Post Study Questionnaire and Discussion with Subjects

7. Results of data analysis
RQ1: Can an inspector’s EA accuracy predict their fault detection effectiveness during an error-based requirements inspection?
Result: For both pre-test and post-test, it was found that EA accuracy was strongly and significantly correlated with the number of additional faults found using abstracted human errors.
Implications for project managers employing error based inspections: it is important to ensure that inspectors are able to correctly perform the EA step, which in turn would help them detect more faults and ensure software quality.
Pre-test (PGCS): EA Accuracy vs Fault Detection Effectiveness (r = 0.79, p = )
Post-test (RIM): EA Accuracy vs Fault Detection Effectiveness (r = 0.61, p = )

8. Results of data analysis
RQ2: Is there a potential for fixation bias during EA?
Results:
Inspectors tend to fixate on certain human error classes.
More formal training on human error identification and improved error abstraction guidelines are required to help inspectors avoid fixation.
Implications: Avoiding fixation is likely to improve EA accuracy, which consequently will improve fault detection effectiveness during error based inspections (as EA accuracy is strongly correlated with fault detection effectiveness).
Effect of EA Fixation
Clerical Errors
Application Errors
Lack of Consistency Errors
Reported number of errors in RIM SRS (by all 16 subjects)
174 58 52
# of errors that were correctly abstracted (out of the reported number) 44 4 2
Top 3 most reported human errors classes (of HET) by subjects:
Clerical errors (Slip)
Application errors (Mistake)
lack of consistency (Slip)
Severe negative impact on EA accuracy

9. conclusion
Results showed that EAI and HET can help provide significant gains in fault detection effectiveness
Subjects provided suggestions for improving both,
the error abstraction process, and
the re-inspection (using abstracted errors) process
More empirical studies under academic and industrial settings are required to generalize the results (particularly the results on human error insights).

10. Ongoing - Human ERROR ABSTRACTION ASSIST (HEAA)

11. 27th International Symposium on Software Reliability Engineering
ISSRE’2016
Error Abstraction Accuracy and Fixation during Error-based Requirements Inspections
Supported by National Science Foundation Awards and
Authors:
Vaibhav Anu, Gursimran Walia (North Dakota State University)
Wenhua Hu, Jeffrey C. Carver (University of Alabama)
Gary Bradshaw (Mississippi State University)