Tao Xie (North Carolina State University) Nikolai Tillmann, Peli de Halleux, Wolfram Schulte (Microsoft Research)

 Developers test their code as they write it  vs. testing done by a separate QA organization  Benefits  Gain high confidence in the code while being written  Reduce cost of fixing faults while freshly introduced  Popular in practice  Processes: Unit Testing, Test-Driven Development, …  Tools: Microsoft Research Pex, Agitar AgitarOne, …

 Full code coverage != full/high confidence  Symbolic execution + assertions == proof  Ex. DART, CUTE, Pex, …  How do we improve testing tools to deal with barriers in compromising correctness confidence?  How do we measure and report to the developers the level of correctness confidence after testing is conducted?

 Test-specific assertions  general assertions  Parameterized unit tests (algebraic specs)  Code contracts/design by contracts (axiomatic specs)  What are costs and benefits in writing different types of specs?  How could we measure the quality of written specifications for the code under test?

 Unit testing: test individual units in isolation  Mock or simulate environment dependencies  How could we measure the quality of environment models for “faithfully” simulating an environment?  How could we make smooth transition from (dis)integration testing to integration testing?

 Test oracles/specs still need human  Education/training (  Test generation: imperfect tools tools  developers cooperative developer testing  How to effectively explain the challenges faced by tools and seek guidance from developers?  How to effectively help developers write and debug high-quality specifications? 2010 Dagstuhl Seminar on “Practical Software Testing: Tool Automation and Human Factors”

 Correctness Confidence  Specifications  (Dis)integration Testing  Human Factors Acknowledgment: NSF grants CNS , CCF , CCF- CAREER , CCF , CNS , ARO grants