1. Designing Programs that Check Their Work
Manuel Blum
Sampath Kannan
by Jeffrey Corbell

2. Overview Introduction to a Program Checker
Other Methods of Determining Correctness
Definition of a Program Checker
Example of a Checker: Graph Isomorphism
Beigel’s Theorem

3. What is a program checker
Program that checks the output of a program to determine if the program is correct or buggy
Formally:
P and C are programs, I is the input
For any I run on P, C is run and determines whether P is correct for I or buggy

4. Other Methods of Determining Correctness
Program verification
Use a proof to prove a program is correct
Very difficult to do
Argued that it doesn't improve confidence in correctness
very complex
may contain errors which would be difficult to detect

5. Other Methods of Determining Correctness
Program testing
Run program on input that you know the correct output for
Compare program output to expected output
Problems
No general way to create test data
No theorems to describe behavior if they do pass tests

6. Differences Between a Checker and Testing
A checker is a program that uses its own algorithm that allows it to check the output
Program testing usually only uses a small amount of predetermined cases for specific input

7. P has a bug if for some instance x of π
Definition of a Bug
Let π represent a decision or search problem
x represents an input to π with π(x) representing the output
P is a deterministic program that supposedly solves π
P has a bug if for some instance x of π
P(x) ≠ π(x)

8. Definition of a Checker
Let Cπ be the checker, k be the number of different cases the checker tries, and I be the group of test inputs
CπP(I,k) is the output of the checker and follows these conditions:
1. If P(x) = π(x), then with probability ≥ 1- 1/2k
CπP(I,k) = CORRECT
2. If P(x) ≠ π(x), then with probability ≥ 1- 1/2k
CπP(I,k) = BUGGY

9. Definition of a Checker
However, if P has bugs but P(I)=π(I) then CπP(I,k) may output either CORRECT or BUGGY

10. Definition of a Checker
Assumed P halts on all inputs
Not always the case
If P(x) exceeds a predetermined bound then the checker should raise a flag, CπP(I,k) = TIME

11. Definition of a Checker
Runtime includes the time it takes to submit input and receive output from P
Does not include the time it takes P to run

12. Definition of a Checker
If a checker is a program, how can you be sure the checker is correct?
You can’t really
Checker must have the little oh property with respect to the runtime of P
Ensures the checker is programmed differently than the original program

13. Graph Isomorphism a 1 f (a) = 1 f (b) = 2 f (c) = 3 f (d) = 4
f (e) = 5 c b 5 4 d e 2 3

14. Graph Isomorphism Checker
Let P be a program that solves graph isomorphism
Input: two graphs G and H
Output: YES if G is isomorphic to H; NO otherwise
CGIP(G, H, k) checks P on input G and H

15. Graph Isomorphism Checker
Compute P(G,H)
If P(G,H)=YES then
Use P to search for an isomorphism from G to H
Check if the resulting correspondence is an isomorphism
If not, return BUGGY; if yes, return CORRECT

16. Graph Isomorphism Checker
If P(G,H)=NO then
Do k times:
Toss a fair coin
If coin = heads then
Generate a random
permutation G’ of G
Compute P(G,G’)
If P(G,G’)=NO then return BUGGY
If coin = tails then
Generate a random permutation H’ of H
Compute P(G,H’)
If P(G,H’)=YES then return BUGGY
Return CORRECT

17. Graph Isomorphism Checker
CGIP runs in polynomial time
If P has no bugs and G is isomorphic to H, then CGIP(G,H,k) creates an isomorphism from G to H and outputs CORRECT
If P has no bugs and G is not isomorphic to H, then CGIP(G,H,k) tosses coins. It discovers P(G,G’)=YES for all G’ and P(G,H’) for all H’ so outputs CORRECT

18. Graph Isomorphism Checker
If P(G,H) is incorrect then there are two cases:
If P(G,H)=YES but G is not isomorphic to H, then CGIP fails to construct an isomorphism and outputs BUGGY
If P(G,H)=NO but G is isomorphic to H, the only way that C will return CORRECT is if P(G,G’)= YES if the coin is heads and P(G,H’)= NO when it is tails. But G and H are permuted randomly to produce G’ and H’. Therefore P correctly distinguishes G’ from H’ only by chance for just 1 of 2k possible sequences

19. Beigel’s Theorem
Let π1 and π2 be two polynomial-time equivalent decision problems. Then from any polynomial time checker for π1 it is possible to construct a polynomial-time checker for π2.

20. Beigel’s Theorem Have a checker Cπ1 for π1 and a program P2 for π2
Also have two way polynomial time transformations f1,2 and f2,1
This gives us a program for π1
P1(x) =P2(f1,2(x))

21. Beigel’s Theorem
To check P2 on an input y, compute P2(y) then transform into an input z for π1 using f2,1
Then use Cπ1 to check z.
Any call Cπ1 makes to P1 is transformed to a call to P2
f1,2 P1 P2
Cπ1
f2,1 z y

22. Beigel’s Theorem
If P2 is correct then P1 will be correct because P1 is defined in terms of P2
Thus if P1 is correct on z then P2 is correct on y
If P2 is wrong on y and P1 is correct on z then there’s a contradiction because P2(y)=P1(z)
If P1 is wrong on z then the checker Cπ1 will catch it

23. Beigel’s Theorem This checker for π2 runs in polynomial time
Running the checker for π1
One transformation of f2,1
Polynomial number of applications of f1,2

24. Bibliography
Designing programs that check their work - M. Blum and S. Kannan
Social Processes and Proofs of Theorems and Programs - R.A. De Millo, R.J. Lipton, and A.J. Perlis.
Introduction to the Theory of Computation – M. Sipser