Defect Localization Using Dynamic Call Tree Mining and Matching Anis Yousefi, PhD Candidate Department of Computing and Software McMaster University May.

Slides:



Advertisements
Similar presentations
Test process essentials Riitta Viitamäki,
Advertisements

Lecture 8: Testing, Verification and Validation
Identification of Distributed Features in SOA Anis Yousefi, PhD Candidate Department of Computing and Software McMaster University July 30,
Mahadevan Subramaniam and Bo Guo University of Nebraska at Omaha An Approach for Selecting Tests with Provable Guarantees.
Testing and Debugging CS221 – 2/13/09. Airline Program.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 23 Slide 1 Software testing.
SE 450 Software Processes & Product Metrics Reliability: An Introduction.
Software Quality Metrics
I.1 ii.2 iii.3 iv.4 1+1=. i.1 ii.2 iii.3 iv.4 1+1=
Objectives Understand the basic concepts and definitions relating to testing, like error, fault, failure, test case, test suite, test harness. Understand.
I.1 ii.2 iii.3 iv.4 1+1=. i.1 ii.2 iii.3 iv.4 1+1=
1 Software Testing and Quality Assurance Lecture 5 - Software Testing Techniques.
High Level: Generic Test Process (from chapter 6 of your text and earlier lesson) Test Planning & Preparation Test Execution Goals met? Analysis & Follow-up.
Types and Techniques of Software Testing
Automated Diagnosis of Software Configuration Errors
1. Topics to be discussed Introduction Objectives Testing Life Cycle Verification Vs Validation Testing Methodology Testing Levels 2.
Objectives Understand the basic concepts and definitions relating to testing, like error, fault, failure, test case, test suite, test harness. Explore.
© 2012 IBM Corporation Rational Insight | Back to Basis Series Chao Zhang Unit Testing.
CS 501: Software Engineering Fall 1999 Lecture 16 Verification and Validation.
CMSC 345 Fall 2000 Unit Testing. The testing process.
Testing Basics of Testing Presented by: Vijay.C.G – Glister Tech.
Bug Localization with Machine Learning Techniques Wujie Zheng
Testing Worshop in Software Engineering Project Department of Computer Science Ben-Gurion university 1Worshop in SE ProjectSpring 2013.
AMA INTERNATIONAL UNIVERSITY BHARAIN SOFTWARE ENGINEERING PROJECT - SBI 302 SOFTWARE ENGINEERING PROJECT - SBI 302A SOFTWARE ENGINEERING PROJECT - SBI.
By Ian Jackman Davit Stepanyan.  User executed untested code.  The order in which statements were meant to be executed are different than the order.
ISV Innovation Presented by ISV Innovation Presented by Business Intelligence Fundamentals: Data Cleansing Ola Ekdahl IT Mentors 9/12/08.
Which Configuration Option Should I Change? Sai Zhang, Michael D. Ernst University of Washington Presented by: Kıvanç Muşlu.
Successful Concepts Study Rationale Literature Review Study Design Rationale for Intervention Eligibility Criteria Endpoint Measurement Tools.
CPIS 357 Software Quality & Testing I.Rehab Bahaaddin Ashary Faculty of Computing and Information Technology Information Systems Department Fall 2010.
Chapter 11: Dynamic Analysis Omar Meqdadi SE 3860 Lecture 11 Department of Computer Science and Software Engineering University of Wisconsin-Platteville.
©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 1 Defect testing l Testing programs to establish the presence of system defects.
A Metrics Program. Advantages of Collecting Software Quality Metrics Objective assessments as to whether quality requirements are being met can be made.
CPSC 873 John D. McGregor Session 9 Testing Vocabulary.
Software Engineering1  Verification: The software should conform to its specification  Validation: The software should do what the user really requires.
Approaches to ---Testing Software Some of us “hope” that our software works as opposed to “ensuring” that our software works? Why? Just foolish Lazy Believe.
CPSC 871 John D. McGregor Module 8 Session 1 Testing.
Software Testing Mehwish Shafiq. Testing Testing is carried out to validate and verify the piece developed in order to give user a confidence to use reliable.
TESTING FUNDAMENTALS BY K.KARTHIKEYAN.
HNDIT23082 Lecture 09:Software Testing. Validations and Verification Validation and verification ( V & V ) is the name given to the checking and analysis.
Testing Overview Software Reliability Techniques Testing Concepts CEN 4010 Class 24 – 11/17.
Testing Worshop in Software Engineering Project Department of Computer Science Ben-Gurion university 1Worshop in SE ProjectSpring 2013.
CPSC 372 John D. McGregor Module 8 Session 1 Testing.
Debugging and Testing Hussein Suleman March 2007 UCT Department of Computer Science Computer Science 1015F.
SOFTWARE TESTING TRAINING TOOLS SUPPORT FOR SOFTWARE TESTING Chapter 6 immaculateres 1.
Software Quality Assurance
John D. McGregor Session 9 Testing Vocabulary
Approaches to ---Testing Software
IS301 – Software Engineering V:
Chapter 8 – Software Testing
Verification and Testing
Learning Software Behavior for Automated Diagnosis
Faults, Errors, Failures CS 4501 / 6501 Software Testing
Object oriented system development life cycle
UML Activity Diagram Documents the Flow the of Program
John D. McGregor Session 9 Testing Vocabulary
Critical Systems Validation
Software Quality Engineering
John D. McGregor Session 9 Testing Vocabulary
Strategies For Software Test Documentation
Software testing strategies 2
Lecture 09:Software Testing
UML Activity Diagram Documents the Flow the of Program
Software Engineering Lecture #13.
Software Engineering Lecture #12.
CS 1111 Introduction to Programming Fall 2018
Test Case Test case Describes an input Description and an expected output Description. Test case ID Section 1: Before execution Section 2: After execution.
Dynamic Program Analysis
Automatically Diagnosing and Repairing Error Handling Bugs in C
Automatic Editing with Soft Edits
Presentation transcript:

Defect Localization Using Dynamic Call Tree Mining and Matching Anis Yousefi, PhD Candidate Department of Computing and Software McMaster University May 28,

Scope Defect detection – Test fails – Customer reports a problem – etc. Defect localization: identifying where the bugs are in the source code – Manual – Tool-supported Defect fixing 2

Problem statement Input – Failure setting: scenario + input causing the failure – Failure description: distorted features Output 3 – Methods manifesting the failure – Path for root cause analysis – Root cause of the problem – Set of passing scenarios + features they exercise

Approach Defect Localization using Dynamic Call Tree Mining and Matching – Dynamic analysis – Granularity: method – Call tree affecting defects – Rationale: failure is associated with deviation from correct execution path Detect correct execution paths (call tree mining) Detect deviating points (call tree matching) 4

Step I Pre-processing – Instrument the target system – Collect execution traces – Build dynamic call trees – Reduce dynamic call trees 5

Step II Mining – Mine frequent sub trees from passing test cases 6 On Msg Main Init Rej Orde r On Msg Main Init Ack Fill Orde r Test case 1Test case 2 On Msg Main Init Ack DFD Test case 3 Fill Orde r

Step III Pattern analysis – Rank patterns according to their relevance to the failing feature 7 Failing Feature: Acknowledge Feature-Specific Shared On Msg Main Init Rej Orde r On Msg Main Init Ack Fill Orde r Test case 1Test case 2 On Msg Main Init Ack DFD Test case 3 Fill Orde r irrelevant

Ranking based on relevance 8 Relation between the support set of a pattern and that of a target (failing) feature Method frequency differences between executions that exercise the target feature and the ones that do not P10 P11 P12 P5 P6 P3 P1 P2 P8 P9 P4 P7 Feature-SpecificShared & Irrelevant Ranked patterns:

Step IV Matching – Input: feature-specific patterns – Match highly relevant patterns against the call tree of the failing execution to identify suspicious call tree changes 9 P10 P11 P12 P5 P6

Locating matching roots 10

Approximate matching 11 Diff DiffSize = 1 ParentDiffSize = 1

Step V Providing the defect localization report – Rank the results (patterns/methods) according to their likelihood of being defective – Provide a report 12

Ranking based on likelihood of defectiveness P10 (root = m1) P11 (root = m1) P12 (root = m1) P5 (root = m2) P6 (root = m2) m1’s location x 1/12/321/8NA m1’s location y 1/102/420/0NA m2’s match (root not found) NA 1/1 (RNF) 13 0/01/1 1/81/102/322/42 Ranking Results – Approach #1: ParentDiffSize/DiffSize 0/01/102/421/11/82/321/1 Ranking Results – Approach #2: Feature-Specific Patterns Matches in the failing tree

Defect localization report 0/0 – P12 and m1’s location y – Wrong branch(es) are executed in m1 or its children. – Suggested locations: P10: method m i P11: methods m j, m k 1/1 – P10 and m1’s location x – Missing/Additional call(s) in the execution of m1 or its children – Incorrect branch(es) executed in m1 or its children – Suggested locations: P10: method m l P12 - method m m P11 - methods m n, m o 1/1 (RNF) – P5 (root, m2, not found) – Missing call to method m2! – Prospective callers are … 14

15

Thank You! Anis Yousefi