Software Engineering & Automated Deduction Willem Visser Stellenbosch University With Nikolaj Bjorner (Microsoft Research, Redmond) Natarajan Shankar (SRI Computer Science Lab, Menlo Park)

Stellenbosch?

Automated Deduction Software Engineering Verification

Automated deduction uses computation to perform symbolic logical reasoning * * [101] Shankar. Automated deduction for verification. ACM Computing Surveys Is a logical formula φ valid? or is there a counter-example? in which case !φ is satisfiable φ valid iff !φ unsatisfiable

Is a logical formula φ satisfiable? Is there a Model that satisfies φ? Satisfiability checking procedures are the cornerstone of automated deduction

We tend to want to know about our code … Given that we have a specification of what is expected

φ Program Specification Logic Formula Automated Deduction Tool

LogicsTechniques First-order logic Automated Theorem Proving Propositional logicSAT Solvers Expressiveness Automation From around the early-mid 1990s SAT solvers improved dramatically

Program φ Logic Formula Integer arithmetic Arrays BitVectors Floating point Strings Linear integers Heaps Machine integers Features Floating point Theories Strings + Satisfiability Modulo Theories (SMT) Solvers Fully Automated Satisfiability Checkers Perfect fit for solving Software Engineering problems

Verification Full automation is not always possible Formulas to prove can fall outside of decidable fragment Even if it can be proved it might be too slow or require tricky invariants Requires human interaction Proof Environments

Too much manual effort seL4 microkernel verification took 20 man years using Isabelle/HOL Program Verifiers are more domain specific they verify annotations in the code holds Spec# ESC-Java

Model Checking Explicit stateSymbolic Predicate Abstraction with CEGAR BDDsSAT SLAMBLAST(Nu)SMVCBMC is M a model for φ

Symbolic Execution void test(int x, int y) { if (y == x*10) S0; else S1; if (x > 3 && y > 10) S2; else S3; } [ Y=X*10 ] S0 [ X>3 & 10<Y=X*10] S2 [ true ] test (X,Y) [ Y!=X*10 & !(X>3 & Y>10) ] S3 [ Y!=X*10 ] S1 [ Y=X*10 & !(X>3 & Y>10) ] S3 [ X>3 & 10<Y!=X*10] S2 SMT solvers check feasibility Test cases derived from models

Concolic Dynamic SE void test(int x, int y) { if (y == x*10) S0; else S1; if (x > 3 && y > 10) S2; else S3; } [ Y=X*10 ] S0 [ X>3 & 10<Y=X*10] S2 [ true ] test (0,1) [ Y!=X*10 & !(X>3 & Y>10) ] S3 [ Y!=X*10 ] S1 [ Y=X*10 & !(X>3 & Y>10) ] S3 [ X>3 & 10<Y!=X*10] S2 Pick random inputs Collect PC during execution Negate one of the conditions If feasible derive new inputs [ Y!=X*10 & (X>3 & Y>10) ] => Test(4,11)

(Dynamic) Symbolic Execution is the poster child for the positive effect of AD in SE An idea from the early 1970s that only sprang to life in late 1990s due to the advances in SAT/SMT solving

Automated Deduction Software Engineering In the past currently

SE drivers for AD Heaps Locally finite theories with limited quantification Data structuresMonadic 2 nd Order Logic (Bounded) Software Model Checking Horn Clauses SecurityStrings Reliability and Information Flow Model Counting (#SAT)

Not good enough any more For test case generation we need models High-integrity code need certifiable proofs Fault localization and repair need unsatisfiable cores Reliability analysis need number of solutions

Informal Survey of SE Researchers What would you like to see most from an Automated Deduction tool? Black-box with limited visibility to internals White-box that exposes inner workings Want to know why not just what

Lessons from Automated Deduction Competitions/Benchmarking Interoperability TPTP DIMACS SMT-LIB2 vs CASC (1996) SMT-COMP (2005) vs SATE (2008) SV-COMP and RERS (2012) SyGus-COMP (2014)

Some New Trends in SE Synthesis Education Sketching * harness void doubleSketch(int x){ int t = x * ??; assert t == x + x; } * Learning to Code

Some More Trends Probabilistic Analysis Reliability [ X>3 & 10<Y=X*10] [ X>3 & 10<Y!=X*10] [ Y!=X*10 & !(X>3 & Y>10) ] [ Y=X*10 & !(X>3 & Y>10) ] y=10x x>3 & y>

Automated Deduction Software Engineering In the past currently Black-box with limited visibility to internals White-box that exposes inner workings Want to know why not just what