Dagstuhl Intro Mike Whalen Program Director University of Minnesota Software Engineering Center

2 Rockwell Collins Inc. Gryphon Tool Family UMN: simulator, fault seeder, coverage measurement tool, TCG RCI: Information Flow Modeling S. Miller, M. Whalen, D. Cofer, Software Model Checking Takes Off, Communications of the ACM, February 2010 M. Whalen, D. Greve, L. Wagner, Model Checking Information Flow, In: Design and Verification of Microprocessor Systems for High-Assurance Applications, D. Hardin, Ed., Springer, March D. Hardin, D.R. Johnson, L. Wagner, and M. Whalen. Development of Security Software: A High-Assurance Methodology, ICFEM 2009, Rio de Janeiro, Brazil, December, 2009.

3 Test Suite Adequacy Can test adequacy metrics defined over the structure of a program be rendered ineffective by restructuring a program? Examined  Statement, Branch, MCDC  vs. random search  Statement, Branch not very useful  MCDC: better but sensitive to program structure Working on: Transformation-Insensitve Coverage Metrics A. Rajan, M. Whalen, and M.P.E. Heimdahl. The Effect of Program and Model Structure on MC/DC Test Adequacy Coverage. ICSE 2008, Leipzig, Germany, May, 2008.

4 Requirements-Based Test Case Generation Conformance Testing Autogenerate Test Cases From Model Commercial Tools Available  (T-VEC, REACTIS) Show Code Conforms to the Model Goal is Structural Coverage (MC/DC) Need Oracles! Requirements-Based Testing  State Requirements as Properties  Use Bounded Model Checker to Generate Test Cases  Goal is to Cover the Requirement Code Generator Create Model Code Model Requirements Create Requirements Based Tests Create Additional Structural Tests Test Case Generator Test Case Generator Properties A.Rajan, M. Whalen, M. Staats, and M.P.E. Heimdahl. Requirements Coverage as an Adequacy Measure for Conformance Testing. (ICFEM 2008), Kitakyushu City, Japan, October, M. Whalen, A. Rajan, and M.P.E. Heimdahl. Coverage Metrics for Requirements-Based Testing. ISSTA 2006, Portland, Maine, July, M. Staats, M. Whalen, A. Rajan, and M. Heimdahl. Coverage Metrics for Requirements-Based Testing: Evaluation of Effectiveness. NFM Washington, D.C., April 13-15, 2010.

5 RequirementsDesign / CodeTestField Completeness and consistency checking Automated proof (model checking) Test oracles for automated unit and integration test Runtime monitors for failure recovery Automated test generation from requirements Subsystem System System of Systems Level of Scale Convincing Users to Write Formal Requirements Partial analysis and static analysis (ASTREE, Coverity)