Today - Limits of computation - Complexity analysis examples

The “Hello” Assignment Suppose I gave you an assignment to write a computer program which returns “Hello!” and stops. (Assume this is a procedure with no parameters). Suppose I promised you full credit for ANY working program which returns “Hello!” and halts, and no credit if it didn’t work.

The “Hello” Assignment Can I write a program that automatically grades your homework?

The “Hello” Assignment Can I write a program that automatically grades your homework? Sure, right? (define (GradeHello P) (let ((out (P))) (if (eq? out ‘Hello!) 100 0)))

The “Hello” Assignment No! What if a student submits: (define (p) (define (q n) (cond ((> n 0) (q n)) (else 'Hello))) (q 10)) Student’s program runs forever! This means the grading program would run forever. It would never tell me that the student should fail… Maybe we can stop the Program after 1 hour and fail the student if the program hasn’t stopped yet.

The “Hello” Assignment Well, in that case lets submit: (define (p) (hanoi );; hanoi with 50 pegs 'Hello) My grading program would say to fail you. But technically you deserve to Pass.

The “Hello” Assignment We keep running in to problems… Could it be that writing a computer program to grade such a simple assignment is impossible? YES! We’ll prove it…

Is there a limit to the power of computers? Q: Is there any problem a computer cannot solve? What is a “problem”? We need some definition to start with: And what is a “computer”?

What is a “computer”? A1: Fortran program running on PDP 11. A3: C++ program running on MAC G4 under OSX at 800MHz. A4: All of the above A2: Java program running on a PC with Pentium V under Windows XP at 3GHz. A5: None of the above

Church-Turing Thesis All “reasonable” models of programming languages and computers are equivalent. Each can emulate the other. There is nothing essential in Java, C+ +, Cobol, Lambda Calculus, Mathematica, (or even Scheme).

Is there a limit to the power of computers? Q: Is there any problem a computer cannot solve? We still have to define what a “problem” means. We will restrict ourselves to well posed problems with a yes/no answer. A solution will be a scheme procedure.

The halting problem The halting problem H: Given a pair of expressions x and y. x is viewed as a Scheme procedure. H(x,y) is true iff x is a valid procedure and it stops on the input y. Example x = (define (f z) (if (= z 1) 1 (+ z (f z)))), y = 1 H(x,y) is true x = (define (f z) (if (= z 1) 1 (+ z (f z)))), y = 2 H(x,y) is false

The halting problem The halting problem is a very natural problem. For the previous example, it is easy to check halting. But in general this problem is hard because the dynamics of programs is very complicated. Halting is a specific requirement of program correctness.

The halting problem is not solvable Proof: By contradiction. Suppose there is a Scheme procedure halt that solves the halting problem. This means that for all x,y (halt x y) always terminates for all x,y (halt x y) returns true if and only if the procedure x stops when applied to argument y. Now consider the following program D: (define (D x) (define (loopy) (loopy)) (if (halt x x) (loopy) #t))

The halting problem is not solvable Now we run D giving it as argument D: (D D) (define (D x) (define (loopy) (loopy)) (if (halt x x) (loopy) #t)) If D stops on D, then by def of halt procedure (halt D D) returns #t. then from the code of D, D enters an infite loop on D, a contradiction. If D does not stop on D, then by def of halt procedure (halt D D) returns #f. then from the code of D, D stops on D, a contradiction. Conclusion: Both possibilities lead to a contradiction. Therefore a procedure halt that solves the halting problem does not exist.

Is there a limit to the power of computers? Q: Is there any problem a computer cannot solve? A: YES. There are well posed problems a computer cannot solve! In fact there are many well posed problems a computer cannot solve! A counting argument: The number of computer programs is countable (since each is a finite string). The number of problems/languages is uncountable. So most problems are not solvable. But we did not just showed existence, but pointed out a specific non-computable problem.

Final Exam ~4 Questions 3 Hours You may bring any written or printed material (no laptops..) You can use every function studied in class \ recitation \ exercise All covered material except for computability (first part of this lecture) and parallel computation (previous lecture) Good Luck and have a nice vacation!

Complexity Analysis Examples

19 Common Recurrences T(n) = T(n-1) +  (1)   (n) T(n) = T(n-1) +  (n)   (n 2 ) T(n) = T(n/2) +  (1)   (logn) T(n) = T(n/2) +  (n)   (n) T(n) = 2T(n-1) +  (1)   (2 n ) T(n) = 2T(n/2) +  (1)   (n) T(n) = 2T(n/2) +  (n)   (nlogn)

20 (filter (lambda (positions) (safe? positions)) (accumulate append null (map (lambda (rest-of-queens) (map (lambda (new-row) (adjoin-position new-row rest-of-queens)) (enumerate-interval 1 board-size))) (queen-cols (- k 1)))))

(define (mean-record ds) (let ((records ds)) (let ((n (length records))) (map (lambda (x) (/ x n)) (accumulate (lambda (x y) (map + x y)) (car records) (cdr records)))))

(define (onto? n m f) (define (rem-dup s) (if (null? s) s (cons (car s) (filter (lambda(x) (not (eq? (car s) x))) (rem-dup (cdr s)))))) (define (image f n) (rem-dup (map f (integers-between 1 n)))) (eq? m (length (image f n)))

23 Mobile weight (define (total-weight mobile) (if (atom? mobile) mobile (+ (total-weight ) (total-weight ) ))) (define (atom? x) (and (not (pair? x)) (not (null? x)))) (branch-structure (left-branch mobile)) (branch-structure (right-branch mobile))

24 balanced? (define (balanced? mobile) (or (atom? mobile) (let ((l (left-branch mobile)) (r (right-branch mobile))) (and (= ) (balanced? ) (balanced? ))))) (* (branch-length l) (total-weight (branch-structure l))) (* (branch-length r) (total-weight (branch-structure r))) (branch-structure l) (branch-structure r)

25 Complexity –Worst case scenario for size n Need to test all rods May depend on mobile structure –Upper bound Apply total-weight on each sub-mobile O(n 2 ) –Lower bound

26 Mobile structures n n-1 n-2 n-3... T(n) = T(n-1) +  (n)(for this family of mobiles) T(n) =  (n 2 )

27 Mobile structures n/2 T(n) = 2T(n/2) +  (n)(for this family of mobiles) T(n) =  (nlogn) n/2 n/4 n/8