Tao Xie (North Carolina State University) Nikolai Tillmann, Peli de Halleux, Wolfram Schulte (Microsoft Research)

Slides:



Advertisements
Similar presentations
Thomas Ball Microsoft Research. C# 3.0C# 3.0 Visual Basic 9.0Visual Basic 9.0 OthersOthers.NET Language Integrated Query LINQ to Objects LINQ to DataSets.
Advertisements

Unit Testing Australian Development Centre Brisbane, Australia.
Leonardo de Moura Microsoft Research. Z3 is a new solver developed at Microsoft Research. Development/Research driven by internal customers. Free for.
Tutorial Pex4Fun: Teaching and Learning Computer Science via Social Gaming Nikolai Tillmann, Jonathan de Halleux, Judith Bishop, Michal.
SOFTWARE TESTING. Software Testing Principles Types of software tests Test planning Test Development Test Execution and Reporting Test tools and Methods.
MOdel-based GENeration of Tests for Embedded Systems # FP7-ICT Embedded Systems Design Institute for Software Technology – Graz University.
Using SMT solvers for program analysis Shaz Qadeer Research in Software Engineering Microsoft Research.
Peli de Halleux Senior Research Software Design Engineer Microsoft Research.
Taking a Waterfall Project Agile REF: Paul Geberth GCSS-J Project Manager Establishment of an Agile Project.
Kai Pan, Xintao Wu University of North Carolina at Charlotte Generating Program Inputs for Database Application Testing Tao Xie North Carolina State University.
The Future of Correct Software George Necula. 2 Software Correctness is Important ► Where there is software, there are bugs ► It is estimated that software.
Pexxxx White Box Test Generation for
DSI Division of Integrated Systems Design Functional Verification Environments Development Goals Our main goals are in the field of developing modular.
CBIIT Quality Assurance Process Preston Wood NCI CBIIT Government Quality Representative (GQR) January 2014 RS.
Analysis of Complex Data Structures with PEX Vadim Mutilin Evgeniy Gerlits Vladimir Fedotov Mentor: Patrice Godefroid.
Jason Hallstrom (Clemson), Joan Krone (Denison), Joseph E. Hollingsworth (IU Southeast), and Murali Sitaraman(Clemson) This workshop is funded in part.
Test-Driven Development With Visual Studio 2005 Erno de Weerd Info Support.
CS527: (Advanced) Topics in Software Engineering Overview of Software Quality Assurance Tao Xie ©D. Marinov, T. Xie.
Automating Software Testing Using Program Analysis -Patrice Godefroid, Peli de Halleux, Aditya V. Nori, Sriram K. Rajamani,Wolfram Schulte, and Nikolai.
Separation of Concerns Tao Xie Peking University, China North Carolina State University, USA In collaboration with Nikolai Tillmann, Peli de Halleux, Wolfram.
Tao Xie North Carolina State University Supported by CACC/NSA Related projects supported in part by ARO, NSF, SOSI.
Introduction to Unit Testing Jun-Ru Chang 2012/05/03.
Automated Testing of System Software (Virtual Machine Monitors) Tao Xie Department of Computer Science North Carolina State University
Tao Xie (North Carolina State University) Nikolai Tillmann, Jonathan de Halleux, Wolfram Schulte (Microsoft Research, Redmond WA, USA)
Automated Developer Testing: Achievements and Challenges Tao Xie North Carolina State University contact:
Texas A&M University Department of Computer Science Sameh S. Sharkawi
Validation Metrics. Metrics are Needed to Answer the Following Questions How much time is required to find bugs, fix them, and verify that they are fixed?
Tao Xie Automated Software Engineering Group Department of Computer Science North Carolina State University
© FPT Software Code Review with VS © FPT Software Agenda What is Code review? Run Code analysis in VS 2012 Configuring Code Analysis rule set.
Mathematical Modeling and Formal Specification Languages CIS 376 Bruce R. Maxim UM-Dearborn.
1 Automatic Identification of Common and Special Object-Oriented Unit Tests Dept. of Computer Science & Engineering University of Washington, Seattle Oct.
Teaching and Learning Programming and Software Engineering via Interactive Gaming Tao Xie University of Illinois at Urbana-Champaign In collaboration with.
Improving Software Reliability via Static and Dynamic Analysis Tao Xie, Automated Software Engineering Group Department of Computer Science North Carolina.
Dr. Tom WayCSC Testing and Test-Driven Development CSC 4700 Software Engineering Based on Sommerville slides.
Tao Xie North Carolina State University Nikolai Tillmann, Peli de Halleux, Wolfram Schulte Microsoft Research.
Code Contracts Parameterized Unit Tests Tao Xie. Example Unit Test Case = ? Outputs Expected Outputs Program + Test inputs Test Oracles 2 void addTest()
Parameterized Unit Tests By Nikolai Tillmann and Wolfram Schulte Proc. of ESEC/FSE 2005 Presented by Yunho Kim Provable Software Lab, KAIST TexPoint fonts.
CS527 Topics in Software Engineering (Software Testing and Analysis) Darko Marinov September 22, 2011.
Implementation of a Digital Image Correlation Interface for the Mechanical Testing of Materials By: Nigel Ray Advisors: Professor Chasiotis Mohammed Naraghi.
Tao Xie (North Carolina State University) Peli de Halleux, Nikolai Tillmann, Wolfram Schulte (Microsoft Research)
TEST-1 6. Testing & Refactoring. TEST-2 How we create classes? We think about what a class must do We focus on its implementation We write fields We write.
Nikolai Tillmann, Jonathan de Halleux Tao Xie Microsoft Research Univ. Illinois at Urbana-Champaign.
Quality Assurance.
Verification and Validation Assuring that a software system meets a user's needs.
Cooperative Developer Testing: Tao Xie North Carolina State University In collaboration with Xusheng ASE and Nikolai Tillmann, Peli de
Dave Lloyd Unit Testing in the Real World.
Parameterized Unit Testing in the Open Source Wild Wing Lam (U. Illinois) In collaboration with Siwakorn Srisakaokul, Blake Bassett, Peyman Mahdian and.
Software Tester By:Nelusha Polgolla. Description of software tester Software testing is an investigation conducted to provide stakeholders with information.
1 Exposing Behavioral Differences in Cross-Language API Mapping Relations Hao Zhong Suresh Thummalapenta Tao Xie Institute of Software, CAS, China IBM.
ISBN Prentice-Hall, 2006 Chapter 8 Testing the Programs Copyright 2006 Pearson/Prentice Hall. All rights reserved.
Objects First With Java A Practical Introduction Using BlueJ Well-behaved objects 2.1.
Improving Structural Testing of Object-Oriented Programs via Integrating Evolutionary Testing and Symbolic Execution Kobi Inkumsah Tao Xie Dept. of Computer.
CS223: Software Engineering Lecture 15: Software construction.
Automated Testing in Sakai Testing applications and services in isolation and in context Josh Holtzman, UC Berkeley David Haines, University of Michigan.
Software engineering - 2 Section 8. QUIZ Show how it is possible to determine the height of a tall building with the aid of a barometer.
1 ITiCSE 2012 Report Kazunari Sugiyama WING meeting 3 rd August, 2012.
Benjamin Unit Testing & Test-Driven Development for Mere Mortals.
1 © Agitar Software, 2007 Automated Unit Testing with AgitarOne Presented by Eamon McCormick Senior Solutions Consultant, Agitar Software Inc. Presented.
Automated Software Testing
Integration Testing.
Dynamic Symbolic Execution
Matching Logic An Alternative to Hoare/Floyd Logic
Reverse engineering and debugging .Net applications
Unit Testing & Test-Driven Development for Mere Mortals
Code Contracts and Pex Peli de Halleux, Nikolai Tillmann
Unit Testing & Test-Driven Development for Mere Mortals
Alon Flaisher Alon Gluska Eli Singerman Intel Corporation
PPT and video are due: no later than November 16, 5:00 PM
Unit Testing & Test-Driven Development for Mere Mortals
Presentation transcript:

Tao Xie (North Carolina State University) Nikolai Tillmann, Peli de Halleux, Wolfram Schulte (Microsoft Research)

 Developers test their code as they write it  vs. testing done by a separate QA organization  Benefits  Gain high confidence in the code while being written  Reduce cost of fixing faults while freshly introduced  Popular in practice  Processes: Unit Testing, Test-Driven Development, …  Tools: Microsoft Research Pex, Agitar AgitarOne, …

 Full code coverage != full/high confidence  Symbolic execution + assertions == proof  Ex. DART, CUTE, Pex, …  How do we improve testing tools to deal with barriers in compromising correctness confidence?  How do we measure and report to the developers the level of correctness confidence after testing is conducted?

 Test-specific assertions  general assertions  Parameterized unit tests (algebraic specs)  Code contracts/design by contracts (axiomatic specs)  What are costs and benefits in writing different types of specs?  How could we measure the quality of written specifications for the code under test?

 Unit testing: test individual units in isolation  Mock or simulate environment dependencies  How could we measure the quality of environment models for “faithfully” simulating an environment?  How could we make smooth transition from (dis)integration testing to integration testing?

 Test oracles/specs still need human  Education/training (  Test generation: imperfect tools tools  developers cooperative developer testing  How to effectively explain the challenges faced by tools and seek guidance from developers?  How to effectively help developers write and debug high-quality specifications? 2010 Dagstuhl Seminar on “Practical Software Testing: Tool Automation and Human Factors”

 Correctness Confidence  Specifications  (Dis)integration Testing  Human Factors Acknowledgment: NSF grants CNS , CCF , CCF- CAREER , CCF , CNS , ARO grants