1. Reading and Writing Mathematical Proofs Spring 2015 Lecture 4: Beyond Basic Induction

2. Previously on Reading and Writing Mathematical Proofs Proving Correctness of Algorithms

3. Hoare Logic Formal system for logical reasoning about computer programs Hoare triple {P} C {Q}  Hoare logic contains rules to determine if Hoare triple is correct If P holds, then after running C, Q holds  Pre- and postcondition are statements about variables preconditionpostcondition command(s)

4. Hoare Triples Maximum(A, n) // Algorithm that computes sum of integers in A[1..n] 1. {A contains n integers} 2. r = 0 3. {r = 0} 4. for i = 1 to n 5. do {r = sum of elements in A[1..i-1]} 6. r = r + A[i] 7. {r = sum of elements in A[1..i]} 8. {r = sum of elements in A[1..n]} 9. return r  Goal is to prove Hoare triple {P} C {Q} where C is whole program We have inference rules for single commands Must “break down” Hoare triple into components loop invariant

5. While Rule P ⇒ S, {S ⋀ B} C {S}, S ⋀ ¬B ⇒ Q {P} while B do C {Q}  We already know how to prove loops S is the invariant P ⇒ S is the initialization {S ⋀ B} C {S} is the maintenance S ⋀ ¬B ⇒ Q is the termination  It is hard to come up with a good invariant Therefore you must always prove it in Data Structures!

6. Summary Hoare logic Formal system for proving algorithms Basically defines the “rules of the game” Proofs in Data Structures No Hoare logic! (only in the background) Assignments: generally without proof If-statements: prove using case distinction Loops: prove using loop invariant  Always make the distinction between “what the code does” and “what it is supposed to do”! The goal is to prove that these two things are the same

7. Proving Steps Steps of Proving 1. Figure out what needs to be shown When is a proof complete? What are the proof requirements? 2. Come up with the proof Proving techniques: induction, contradiction, etc. How to choose/combine techniques 3. Write down the proof As clearly as possible With the reader in mind Hoare logic No Hoare logic

8. Nested Loops How to prove nested loops? 1. {P 1 } 2. for i = 1 to n 3. do {S 1 } 4. ….. 5. {P 2 } 6. for j = i+1 to n 7. do {S 2 } 8. …. 9. {Q 2 } 10. {Q 1 } {P 1 } for i = 1 to n do OLB {Q 1 } P 1 ⋀ i=1 ⇒ S 1 S 1 ⋀ i=n+1 ⇒ Q 1 {S 1 ⋀ i≤n } OLB {S 1 [i+1/i]} {S 1 ⋀ i≤n } … {P 2 } {P 2 } for j = i+1 to n do … {Q 2 } Q 2 ⇒ S 1 [i+1/i] P 2 ⋀ j=i+1 ⇒ S 2 S 2 ⋀ j=n+1 ⇒ Q 2 {S 2 ⋀ j≤n } … {S 2 [j+1/j]}

9. Beyond Basic Induction Today…

10. Recursion But first…

11. Function Calls How to prove a function call? 1. {P} 2. EpicFunction(x) 3. {Q}  Function should have specification: pre- and postcondition  If precondition is met, then postcondition will follow  Function is proved separately What if the function call is recursive?  Then we need a different proving technique…

12. Recursion EpicFunc(x) 1. {P} 2. if x = 1 3. then return 1 4. else r = EpicFunc(x-1) 5. r = r + EpicFunc( ⌊x/2⌋) 6. {Q} 7. return r  Cannot prove what EpicFunc does using what EpicFunc does Or can we…?  Recursive calls must have “smaller” input We can use strong induction! No idea what this does! Let’s try something else

13. Example BinarySearch(A, i, j, x) // Returns true iff A[i…j] contains x 1. {A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j] } 2. if i + 1 = j 3. then return (A[i] = x) 4. h = (i + j)/2 5. if A[h] ≤ x 6. then return BinarySearch(A, h, j, x) 7. else return BinarySearch(A, i, h, x) Strong induction Base case(s): when no recursive calls are made Induction step: the rest… Induction on what? Must be “smaller” input!

14. Example Theorem If A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j], then BinarySearch(A, i, j, x) returns true iff A[i…j] contains x Proof We use strong induction on |j – i|. Base case (|j – i| = 1): Then, since A[j] > x, only A[i] can contain x. This is correctly checked by the algorithm. Step (|j – i| ≥ 2 ): First note that i < h < j, so 1 ≤ |j – h| < |j – i| and 1 ≤ |h – i| < |j – i|. We consider two cases: Case (1): A[h] ≤ x Since A is sorted, x cannot be in A[i…h-1] and must be in A[h…j]. This is checked by the recursive call. As required, A[h] ≤ x. Case (2): A[h] > x Since A is sorted, x cannot be in A[h+1…j] and must be in A[i…h]….

15. Recursion Notes on recursion  Always make sure recursive calls are valid Input should satisfy requirements Input should be “smaller” than original input Must eventually reach a base case  What does “smaller” input mean? You get to define it! But must satisfy certain requirements…

16. Induction A more general approach…

17. Induction Different types of induction  Mathematical induction P(1) P(n) ⇒ P(n+1)  Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)  Structural induction ?  Well-founded induction ? For natural numbers For other sets

18. Induction Mathematical Induction  Prove something for all positive integers: ∀ n [n ℕ : P(n)]  What if we want to use a different set: ∀ x [x S : P(x)] Cannot use standard induction: P(x) ⇒ P(x+1) What to do? Examples Prove for all strings that … Prove for all rooted binary trees that … Prove for all graphs that … Prove for all polygons that …

19. Set definitions Defining Infinite Sets  Using properties: Set of rationals = {x | ∃ p,q ℤ [q ≠ 0 and qx = p]} Set of primes = {p | ¬∃ d ℤ [1 < d < p and p is multiple of d]} Set of squares = {x 2 | x ℤ }  Inductive (or recursive) definition: Natural numbers ℕ: 1) 1 ℕ 2) If n ℕ, then n + 1 ℕ Full binary trees T: 1) T 2) If x T and y T, then T x y Useful for induction!

20. Examples 1. Set of positive even numbers E 2 E If n E, then n + 2 E 2. Set of (non-empty) binary strings B 0 B, 1 B If X B, then 0X B and 1X B 3. Set of powers of 3: Q 1 Q If p Q, then 3p Q 4. Set of arithmetic expressions A n A for all n ℕ If e 1, e 2 A, then –e 1, (e 1 ), e 1 + e 2, e 1 – e 2, e 1 * e 2, e 1 / e 2 A 5. Set of prime numbers P Don’t know…

21. Structural Induction Full binary trees T 1) T 2) If x T and y T, then T Structural Induction on T  Base case Prove property for a single node  Induction step Prove property for Can use induction hypothesis on x and y x y x y

22. Example Theorem A full binary tree with n nodes has (n+1)/2 leaves Proof We use structural induction on the set of full binary trees with the inductive rule on the previous slide. Base case (rule (1)): For a single node n = 1 and there is (1+1)/2 = 1 leaf. Step (rule (2)): Suppose that the subtrees x and y, with a and b nodes, have (a+1)/2 and (b+1)/2 leaves, respectively (IH). We need to show that the tree T with n nodes formed by adding a root above x and y has (n+1)/2 leaves. A leaf of T is either a leaf of x or a leaf of y. By IH, the number of leaves of T is then (a+1)/2 + (b+1)/2 = (a+b+2)/2. Since n = a + b + 1, we get that (n+1)/2 = (a+b+2)/2, as required.

23. Example Theorem A full binary tree with n nodes has (n+1)/2 leaves Proof We use structural induction on the set of full binary trees with the inductive rule on the previous slide. Base case (rule (1)): For a single node n = 1 and there is (1+1)/2 = 1 leaf. Step (rule (2)): Consider a full binary tree T with subtrees x and y. Let x and y have a and b nodes, respectively, such that n = a + b + 1. A leaf of T is either a leaf of x or a leaf of y. By the IH, we get that T has (a+1)/2 + (b+1)/2 = (a+b+2)/2 = (n+1)/2 leaves. Isn’t this the same as strong induction?

24. Practice Fancy Sequences S 1) x S for all x ℕ 2) If Y, Z S and x ℕ such that x ∉ Y and x ∉ Z, then YxZ S What does this mean? S contains sequences of integers Any sequence of 1 integer is in S If x ℕ is not in sequences Y, Z S, then YxZ is also in S Examples 12345 S 121423 S 12334 ∉ S 12121 ∉ S

25. Practice Fancy Sequences S 1) x S for all x ℕ 2) If Y, Z S and x ℕ such that x ∉ Y and x ∉ Z, then YxZ S Theorem Every non-empty contiguous subsequence of a fancy sequence contains at least one natural number uniquely

26. Practice Fancy Sequences S 1) x S for all x ℕ 2) If Y, Z S and x ℕ such that x ∉ Y and x ∉ Z, then YxZ S Theorem Every non-empty contiguous subsequence of a fancy sequence contains at least one natural number uniquely Proof We use structural induction on fancy sequences with the above rule. Base case (rule (1)): The (sub)sequence contains only one number, so it must be unique. Step (rule (2)): We perform a case distinction based on the subsequence: Case 1 (subsequence contains x): Since x is not in Y or Z, x is unique. Case 2 (subsequence contained in Y): By the IH on Y, the result holds. Case 3 (subsequence contained in Z): By the IH on Z, the result holds.

27. Induction Different types of induction  Mathematical induction P(1) P(n) ⇒ P(n+1)  Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)  Structural induction Induction using inductive definition of set  Well-founded induction ?

28. Binary Search (again) BinarySearch(A, i, j, x) // Returns true iff A[i…j] contains x 1. {A is sorted, i + 1 ≤ j, and A[i] ≤ x < A[j] } 2. if i + 1 = j 3. then return (A[i] = x) 4. h = (i + j)/2 5. if A[h] ≤ x 6. then return BinarySearch(A, h, j, x) 7. else return BinarySearch(A, i, h, x) Smaller input Size of input defined as |j – i| ⇒ strong induction on |j – i| In general: Order on set of possible inputs (A, i, j, x) ≺ (A’, i’, j’, x’) if |j – i| < |j’ – i’| Must be “smaller” input!

29. Partial order Strict partial order relation ≺ on set S Binary relation: x ≺ y for certain pairs x, y S Anti-reflexive: x ⊀ x Anti-symmetric: if x ≺ y, then y ⊀ x Transitive: If x ≺ y and y ≺ z, then x ≺ z Examples For x, y ℤ: x ≺ y iff x < y For sets X, Y: X ≺ Y iff X ⊂ Y ({1,3} ≺ {1,2,3,4}) For strings S 1, S 2 : S 1 ≺ S 2 iff S 1 is substring of S 2 (“ab” ≺ “cab”) For trees T 1, T 2 : T 1 ≺ T 2 iff T 1 is substring of T 2

30. Partial order Does (strong) induction work for any partial order ≺ ? Not exactly… Theorem For all x ℤ it holds that x = x + 1 Proof By induction on x: We apply IH to x – 1, so that x – 1 = x. (Note that x – 1 < x) By adding 1 to both sides we obtain that x = x + 1. We need base cases! The partial order ≺ must have minimal elements Minimal elements are base cases

31. Well-founded Relation Well-founded relation ≺ on S Every non-empty subset X ⊆ S must have a minimal element Minimal element m X: for all x X it holds that x ⊀ m S contains no infinite descending chains: a ≻ b ≻ c ≻ …. Well-founded induction on S First need well-founded (partial) order ≺ on S Base case(s): Minimal elements of S Induction step: If P(x) for all x ≺ y, then P(y) (for all y S )

32. Practice Which of these partial orders are well-founded?  For x, y ℤ: x ≺ y iff x < y  For x, y ℤ: x ≺ y iff |x| < |y|  For x, y ℕ: x ≺ y iff y is a multiple of x and x ≠ y  For rational numbers x, y ℚ : x ≺ y iff x < y  For strings S 1, S 2 : S 1 ≺ S 2 iff S 1 lexicographically before S 2 ½ > ⅓ > ¼ > ⅕ > … “b” ≻ “ab” ≻ “aab” ≻ “aaab” ≻ …

33. Practice Ackermann(m, n) 1. if m = 0 2. then return n+1 3. else if n = 0 4. then return Ackermann(m – 1, 1) 5. else return Ackermann(m – 1, Ackermann(m, n – 1)) A recursive function terminates on all input if and only if there exists a well-founded order ≺ on the set of inputs such that: “input of recursive call” ≺ “original input” Does the Ackermann function terminate on all inputs? (m, n) ≺ (m’, n’) iff m < m’ or m = m’ and n < n’

34. Practice T(1, n) = 1 T(m, 1) = 1 T(m, n) = T(m/2, n) + T(m, n/2) – T(m/2, n/2) + 1 m, n > 1 Theorem T(m, n) = log(m) log(n) + 1

35. Practice Theorem T(m, n) = log(m) log(n) + 1 Proof We use induction on (m, n), where (m, n) ≺ (m’, n’) iff m < m’ or n < n’. Base case (m = 1): T(1, n) = 1 = log(1) log(n) + 1. Base case (n = 1): T(m, 1) = 1 = log(m) log(1) + 1. Step (m, n > 1): T(m, n) = T(m/2, n) + T(m, n/2) – T(m/2, n/2) + 1{definition} T(m, n) = log(m/2) log(n) + log(m) log(n/2) – log(m/2) log(n/2) + 2 {IH} T(m, n) = log(m/2) (log(n) – log(n/2)) + log(m) log(n/2) + 2 T(m, n) = log(m/2) + 1 + log(m) (log(n) – 1) + 1 T(m, n) = log(m) log(n) + 1

36. Well-founded Induction Well-founded induction Very general type of (strong) induction … but also very abstract Main lesson If you can order the elements of a set, you can do induction Induction hypothesis may always be applied to “smaller” elements Recursion First argue that it terminates Then you can use IH on recursive call to argue correctness

37. Induction Different types of induction  Mathematical induction P(1) P(n) ⇒ P(n+1)  Strong induction P(1) P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)  Structural induction Induction using inductive definition of set  Well-founded induction Strong induction for any set Just needs well-founded order…

38. Summary Steps of Proving 1. Figure out what needs to be shown When is a proof complete? What are the proof requirements? 2. Come up with the proof Proving techniques: induction, contradiction, etc. How to choose/combine techniques 3. Write down the proof As clearly as possible With the reader in mind “Rules” of proving “Tools” of proving hard, creative, fun(?) part