Program Checking Sampath Kannan University of Pennsylvania

Talk Outline Traditional software reliability paradigms. Program Checking Examples Extensions Spot checking Streams and stream checking Monitoring JAVA programs Future Directions

Traditional Approaches Testing: ad hoc or random test cases. Need to know answers to these cases. No mathematical guarantees. May look inside program. Done during design phase.

Verification: Proves programs correct. Hard to do. Proofs not human-friendly. Looks inside programs. Done during design phase.

Program Checking [BK95] IO I1 I2. Ik Ok. O2 O1 ProgramChecker Coin Tosser

Checker “Correct” if P is correct on all inputs. “Buggy” if P is wrong on input I. Checkers outputs are as above with high probability. Notion of correctness of P assumed to be provided by some rigorous means.

Example – Matrix Multiplication Checker A, B C Coin Tosser vector v A(Bv) = Cv ? Program Freivalds ’79.

Example – Graph Isomorphism G H

Program G, H “Yes”, here is isomorphism “No” Pick G or H at random. Permute to get graph K. Ask the program (G,K). If K obtained from G Expect “Yes”. If K obtained from H Expect “No”. Call program “Buggy” if it does not meet expectation. If Program says “No”: Checker[BK95]

Program Checking Paradigm Checking done at run-time: Overhead to program. Errors detected as they happen. Environment errors also detected. Actual implementation is checked. Checkers are problem specific: Checker design requires ingenuity. Not universal paradigm. Works against all programs for a problem. Don’t need to look at inside of program.

Checking Correctness of Memory Unreliable Memory User Checker Reliable memory checker requests user requests Can check using O(log N)-sized reliable memory that an adversarial memory (RAM, Stack, Queue) of size N is functioning correctly. Techniques used: Special classes of hash functions, cryptographic primitives. BEGKN ’94.

Problems/Areas with Checkers Linear Algebra Group Theory Arithmetic Number Theory Polynomial Algebra Computations with Real Numbers Sorting Data Structures Graph Problems Combinatorial Optimization

Summary of Part 1 Program checking works on mathematically “clean” problems... but it takes a fair amount of effort to come up with a good checker. The definition requires the checker itself to be “simpler” (faster) than the program being checked, but this does not count the cost of additional calls to the program. Challenge: 1) Reduce Overhead 2) Design checkers for more problems So that the technique can be applied to systems w. low resources.

Spot-Checking Question: Can we settle for an “approximate” notion of correctness in order to drastically reduce overhead? Answer: We can in some cases. Using spot checks [EKKRV ’99]

Spot Checking Sorting A program for sorting is approximately correct if (1 –  ) fraction of the output is in ascending order and (1 -  ) fraction of the output elements are input elements. We want a spot-checker that: Says “okay” to correct output Says “Buggy” to output that is not even approximately correct. How do we design such a spot-checker?

Repeat O(1/  ) times Pick random element x from input. Binary search for x in output. If (x not found) report “Buggy” Report “okay”. Method works because: All elements that are successfully found by binary search form an ascending sequence. We are verifying that there is a large ascending sequence and that most input elements are in output. Checker takes O(log n) time!

Streams Deluge of time-dependent data... if we don’t process it soon, it will become irrelevant! or it will be too late! Need to change both What we ask about the data. How we find the answer to our questions. Need good theoretical models to model the constraints under which we operate.

Model Processor Memory could be a CISCO router with netflow software. (size is small relative to stream size.) [FKSV ’99.]

Detecting Anomalies Is this the output of a fair coin? Given streams of data describing internet traffic flow is there an anomalous day? Application to intrusion detection? Can you stand on Times Square, watch the NYSE ticker tape go by twice and decide whether there was a big change in the stock market in between?

Main Results on Streams... (1, 3), (7, 12), (3, 6), (2, -1), (4, -5), (7, 6), (8, 0), (1, -4), (9, 5), (3, -2), (5, -10), (2, 6)... What is the L 1 distance between these two functions? (i.e., ) Our solution: Above distance can be approximated arbitrarily closely using memory that is only logarithmic in length of stream. Stream computing will become increasingly important as will the problem of verifying properties of data streams.

Run-time Monitoring and Checking Sampath Kannan Moonjoo Kim Insup Lee Oleg Sokolsky Mahesh Viswanathan

Objectives Static analysis –abstract model Dynamic behavior checking –consistency between abstract model and implementation To provide a framework for automatic generation of monitors and checkers

Fundamental Issues How does a monitor gather information from a running system? How does the monitor relate to requirements? How do we integrate dynamic monitoring with static analysis? Can monitor be used to steer a system? What mathematical guarantees do monitors provide?

System Spec System Spec Requirement Spec Requirement Spec Formal verification Design System Implementation System Implementation Monitoring Script Monitoring Script Implementation Checker/ Corrector Checker/ Corrector System Filter Communication Run-time Check MAC Architecture Event Handler Event Handler Corrector Checker

Design Issues Filter –passive versus active –when to take snapshot Event Handler –mapping between concrete state and abstract event Checker –inclusion based on trace, ready semantics, bisimulation Corrector –how to provide feedback

Properties Safety –e.g.: The gate is always closed while a train is in crossing. Security –e.g.: detecting denial of service attack. Performance, Real-time –e.g.: QoS - does my car accelerate from 0-60 in 5 sec’s? Resource usage –e.g.: 10 MB of memory.

Future Directions Run-time assurance of correctness is an attractive complement to static analysis. Traditional correctness paradigms are more able to deal with flow-of-control correctness, type checks and the like. The big gap is in ensuring correctness of mathematically sophisticated computation. Program checking fills this gap. As we begin to reason about large programs and large volumes of input and output we need non-traditional models (both in theory and practice) for talking about efficient computation.