Induction II: Inductive Pictures Great Theoretical Ideas In Computer Science Steven Rudich CS 15-251 Spring 2004 Lecture 14 Feb 26, 2004 Carnegie Mellon University Induction II: Inductive Pictures

“Least Counter-example” “All Previous” Induction Inductive Definition: Inductive Proof: “Standard” Induction “Least Counter-example” “All Previous” Induction Inductive Definition: Recurrences Recursive Programming

Theorem? (k¸0) 1+2+4+8+…+2k = 2k+1 -1 Try it out on small examples: 20 = 21 -1 20 + 21 = 22 -1 20 + 21 + 22 = 23 -1

Sk´ “1+2+4+8+…+2k = 2k+1 -1” Use induction to prove k¸0, Sk Establish “Base Case”: S0. We have already check it. Establish “Domino Property”: k¸0, Sk ) Sk+1 “Inductive Hypothesis” Sk: 1+2+4+8+…+2k = 2k+1 -1 Add 2k+1 to both sides: 1+2+4+8+…+2k + 2k+1= 2k+1 +2k+1 -1 1+2+4+8+…+2k + 2k+1= 2k+2 -1

FUNDAMENTAL LEMMA OF THE POWERS OF TWO: The sum of the first n powers of 2, is one less than the next power of 2.

Yet another way of packaging inductive reasoning is to define an “invariant”. Invariant (adj.) Not varying; constant. (mathematics) Unaffected by a designated operation, as a transformation of coordinates.

Yet another way of packaging inductive reasoning is to define an “invariant”. Invariant (adj.) 3. (programming) A rule, such as the ordering an ordered list or heap, that applies throughout the life of a data structure or procedure. Each change to the data structure must maintain the correctness of the invariant.

Invariant Induction Suppose we have a time varying world state: W0, W1, W2, … Each state change is assumed to come from a list of permissible operations. We seek to prove that statement S is true of all future worlds. Argue that S is true of the initial world. Show that if S is true of some world – then S remains true after one permissible operation is performed.

Odd/Even Handshaking Theorem: At any party, at any point in time, define a person’s parity as ODD/EVEN according to the number of hands they have shaken. Statement: The number of people of odd parity must be even. Initial case: Zero hands have been shaken at the start of a party, so zero people have odd parity. If 2 people of different parities shake, then they both swap parities and the odd parity count is unchanged. If 2 people of the same parity shake, they both change. But then the odd parity count changes by 2, and remains even.

Inductive Definition of n! [said n “factorial”] 0! = 1; n! = n*(n-1)!

0! = 1; n! = n*(n-1)! Program for n! ? F:=1; For x = 1 to n do F:=F*x; Return F Program for n! ?

0! = 1; n! = n*(n-1)! Program for n! ? F:=1; For x = 1 to n do F:=F*x; Return F Program for n! ? n=0 returns 1 n=1 returns 1 n=2 returns 2

0! = 1; n! = n*(n-1)! Loop Invariant: F=x! For x = 1 to n do F:=F*x; Return F Loop Invariant: F=x! True for x=0. If true after k times through – true after k+1 times through.

Inductive Definition of T(n) T(n) = 4 T(n/2) + n Notice that T(n) is inductively defined for positive powers of 2, and undefined on other values.

Inductive Definition of T(n) T(n) = 4T(n/2) + n Notice that T(n) is inductively defined for positive powers of 2, and undefined on other values. T(1)=1 T(2)=6 T(4)=28 T(8)=120

Guess a closed form formula for T(n). Guess G(n) G(n) = 2n2 - n Let the domain of G be the powers of two.

Two equivalent functions? G(n) = 2n2 - n Let the domain of G be the powers of two. T(1) = 1 T(n) = 4 T(n/2) + n Domain of T are the powers of two.

Inductive Proof of Equivalence Base: G(1) = 1 and T(1) = 1 Induction Hypothesis: T(x) = G(x) for x < n Hence: T(n/2) = G(n/2) = 2(n/2)2 – n/2 T(n) = 4 T(n/2) + n = 4 G(n/2) + n = 4 [2(n/2)2 – n/2] + n = 2n2 – 2n + n = 2n2 – n = G(n) G(n) = 2n2 - n T(1) = 1 T(n) = 4 T(n/2) + n

We inductively proved the assertion that G(n) =T(n). Giving a formula for T with no sums or recurrences is called solving the recurrence T.

Solving Recurrences Guess and Verify Guess: G(n) = 2n2 – n Verify: G(1) = 1 and G(n) = 4 G(n/2) + n Similarly:T(1) = 1 and T(n) = 4 T(n/2) + n So T(n) = G(n)

Technique 2 Guess Form and Calculate Coefficients Guess: T(n) = an2 + bn + c for some a,b,c Calculate: T(1) = 1  a + b + c = 1 T(n) = 4 T(n/2) + n  an2 + bn + c = 4 [a(n/2)2 + b(n/2) + c] + n = an2 + 2bn + 4c + n  (b+1)n + 3c = 0 Therefore: b=-1 c=0 a=2

A computer scientist not only deals with numbers, but also with Finite Strings of symbols Very visual objects called graphs And especially, especially the special graphs called trees

GRAPHS b root a

Definition: Graphs A graph G = (V,E) consists of a finite set V of vertices (nodes) and a finite set E of edges. Each edge is a set {a, b} of two different vertices. A graph may not have self loops or multiple edges.

Definition: Directed Graphs A graph G = (V,E) consists of a finite set V of vertices (nodes) and a finite set E of edges. Each edge is an ordered pair <a,b> of two different vertices. Unless we say otherwise, our directed graphs will not have multi-edges, or self loops.

Definition: Tree A tree is a directed graph with one special node called the root and the property that each node must a unique path from the root to itself. Child: If <u,v>2E, we sav is a child of u Parent: If <u,v>2E, we say u is the parent of u Leaf: If u has no children, we say u is leaf. Siblings: If u and v have the same parent, they are siblings. Descendants of u: The set of nodes reachable from u (including u). Sub-tree rooted at u: Descendants of u and all the edges between them where u has been designated as a root.

Classic Visualization: Tree Inductive rule: If G is a single node Viz(G) = If G consists of root r with sub-trees T1, T2, …, Tk Vuz(T1) Viz(T2) Viz(Tk) …..

I own 3 beanies and 2 ties. How many beanie/tie combos might I wear to the ball tonight?

Choice Tree A choice tree is a tree with an object called a “choice” associated with each edge and a label on each leaf.

Definition: Labeled Tree A tree node labeled by S is a tree T = <V,E> with an associated function Label1: V to S A tree edge labeled by S is a tree T = <V,E> with an associated function Label2: E to S

was very illuminating. Let’s do something similar to illuminate the nature of T(1)=1; T(n)= 4T(n/2) + n

T(1)=1; T(n)= 4T(n/2) + n For each n (power of 2), we will define a tree W(n) node labeled by Natural numbers. W(n) will give us an incisive picture of T(n).

Inductive Definition Of Labeled Tree W(n) T(n) = n + 4 T(n/2) W(n) = n W(n/2) W(n/2) W(n/2) W(n/2) T(1) = 1 W(1) = 1

Inductive Definition Of Labeled Tree W(n) = 2 1 1 1 1 T(1) = 1 W(1) = 1

Inductive Definition Of Labeled Tree W(n) = 4 2 1 2 1 2 1 2 1

Inductive Definition Of Labeled Tree W(n) = 4 2 1 2 1 2 1 2 1 NOTE: When we sum all the node labels on W(n), we get T(n)

Invariant: LabelSum W(n) = T(n) T(n) = n + 4 T(n/2) W(n) = n W(n/2) W(n/2) W(n/2) W(n/2) T(1) = 1 W(1) = 1

W(n) = n W(n/2) W(n/2) W(n/2) W(n/2)

W(n) = n n/2 W(n/4) W(n/2) W(n/2) W(n/2)

W(n) = n n/2 W(n/4) n/2 W(n/4) n/2 W(n/4) n/2 W(n/4)

W(n) = n n/2 n/2 n/2 n/2 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 11111111111111111111111111111111 . . . . . . 111111111111111111111111111111111

Level i is the sum of 4i copies of n/2i n/2 + n/2 + n/2 + n/2 Level i is the sum of 4i copies of n/2i . . . . . . . . . . . . . . . . . . . . . . . . . . 1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1 1 2 n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 i

= 1n = 2n = 4n = 2in = (n)n n(1+2+4+8+ . . . +n) = n(2n-1) = 2n2-n

Instead of T(1)=1; T(n)= 4T(n/2) + n We could illuminate T(1)=1; T(n) = 2T(n/2) + 2

T(n) = n + 2 T(n/2) W(n) = n W(n/2) W(n/2) T(1) = 1 W(1) = 1

Level i is the sum of 2i copies of n/2i n/2 + n/2 Level i is the sum of 2i copies of n/2i . . . . . . . . . . . . . . . . . . . . . . . . . . 1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1 1 2 i

n . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 i

Has closed form: nlog2(n) where n is a power of 2 T(1)=1; T(n) = 2T(n/2) + n Has closed form: nlog2(n) where n is a power of 2

Representing a recurrence relation as a labeled tree is one of the basics tools you will have to put recurrence relations in closed form.

The Lindenmayer Game  = {a,b} Start word: a SUB(a) = ab SUB(b) = a For each w = w1 w2 … wn NEXT(w) = SUB(w1)SUB(w2)..SUB(wn)

The Lindenmayer Game SUB(a) = ab SUB(b) = a For each w = w1 w2 … wn NEXT(w) = SUB(w1)SUB(w2)..SUB(wn) Time 1: a Time 2: ab Time 3: aba Time 4: abaab Time 5: abaababa

How long are the strings as a function of time? The Lindenmayer Game SUB(a) = ab SUB(b) = a For each w = w1 w2 … wn NEXT(w) = SUB(w1)SUB(w2)..SUB(wn) Time 1: a Time 2: ab Time 3: aba Time 4: abaab Time 5: abaababa How long are the strings as a function of time?

Aristid Lindenmayer (1925-1989) 1968 Invents L-systems in Theoretical Botany Time 1: a Time 2: ab Time 3: aba Time 4: abaab Time 5: abaababa FIBONACCI(n) cells at time n

The Koch Game  = {F,+,-} Start word: F SUB(F) = F+F--F+F SUB(+)=+ SUB(-)=- For each w = w1 w2 … wn NEXT(w) = SUB(w1)SUB(w2)..SUB(wn)

The Koch Game Gen 0:F Gen 1: F+F--F+F Gen 2: F+F--F+F+F+F--F+F--F+F--F+F+F+F--F+F

The Koch Game Picture representation: F draw forward one unit + turn 60 degree left - turn 60 degrees right. Gen 0: F Gen 1: F+F--F+F Gen 2: F+F--F+F+F+F--F+F--F+F--F+F+F+F--F+F

The Koch Game F+F--F+F

The Koch Game F+F--F+F+F+F--F+F--F+F--F+F+F+F--F+F

Koch Curve

Dragon Game SUB(X) =X+YF+ SUB(Y) = -FX-Y                                     Dragon Curve:            

Hilbert Game SUB(L)= +RF-LFL-FR+ SUB(R)= -LF+RFR+FL-                                     Hilbert Curve:             Note: Make 90 degree turns instead of 60 degrees.          

Hilbert’s Space Filling Curve

Peano-Gossamer Curve

Sierpinski Triangle

Lindenmayer 1968 SUB(F) = F[-F]F[+F][F] Interpret the stuff inside brackets as a branch.

Lindenmayer 1968

Inductive Leaf

Here is another kind of inductive definition called a formal grammar.

Formal Grammar: G = (T, V, S, P) A finite alphabet T called the terminal symbols. A disjoint finite alphabet N of non-terminal symbols (or variable symbols). One symbol S in V is called the start symbol. A finite set P of production rules of the form   , where , 2 (N [ T)* and  contains at least one non-terminal symbol.

Example. Terminal Symbols: {a,b,c} Non-terminal symbols: {S, X} Start symbol: S Production rules: S  SS S  a SS  b

Formal Grammar: G = (T, V, S, P) A finite set P of production rules of the form   , where , 2 (N [ T)* and  contains at least one non-terminal symbol. Let x and y be strings over N[T. We say that y is produced in one step from x if there is a rule such that one occurrence of the substring  is replaced with  to obtain the string y. We denote this x 1 y

S 1 SSS 1 bS 1 ba Terminal Symbols: {a,b,c} Non-terminal symbols: {S, X} Start symbol: S Production rules: S  SS S  a SS  b

Definition of x k y We know what x 1 y means. Define x 0 y only when x = y Define x ->k to mean 9 z s.t. x->1z and z->k-1y Define x*y to mean that there exists a k such that xk y .

Language Produced By Formal Grammar G LG = {x 2 T* | S*x } = {x 2 T* | 9 k s.t. Skx } The language of strings of terminal symbols that are eventually produced by the start symbol.

Language = {a,b}+ Terminal Symbols: {a,b,c} Non-terminal symbols: {S, X} Start symbol: S Production rules: S  SS S  a SS  b

Evaluating A Formal Derivation Let G be a formal grammar. A sequence D = S1, S2, …., Sk, of strings is called a formal derivation of  if: For 1· q · k-1 Sq 1 Sq+1

Once you agree on a specific grammar G, it is trivial to verify proofs of membership in the language.

G = Balanced S  () S  SS S  (S) Now let’s show Symbol Dude the following derivation: S, (S), (SS), (SS), (S()), ((())())

Yo, dude! ((())()) is like totally balanced XXh42TEhha

Expressions in Propositional Logic E  True | False  x | y | z   E  (E Ç E)  (E Æ E)

Yo, dude! :(xÆy) is like totally an expression in propositional logic. XXh42TEhha

Meaning: The Part Symbol Dude Ignores E  True | False (constants)  x | y | z (variables)   E1 (not E1)  (E1 Ç E2) (E1 or E2)  (E1 Æ E2) (E1 and E2)

Inductively Associate a MEANING M(E) of any expression E M(T) = T M(F)=F M(: E) = not M(E) M( (E1 Æ E2) = M(E1) and M(E2) M( (E1 Ç E2) = M(E1) or M(E2) not T = F not F =T AND. F and F = F F and T = F T and F – F T and T= T

Expressions in First Order Logic T  x | y | z  f(T1, T2, …, Tk) (functions) T = Terms E  R(T1, T2, …, Tk) (relation on terms)   E1 | (E1 Ç E2) | (E1 Æ E2)  8 x. E1 (universal quantifier)  9 x. E1 (existential quantifier)

8 x. (S(S(x)) < S(x^S(S(0)))) E.g., Number Theory T  x | y | z  0 (the zero function)  S(T1) (successor function)  (T1+T2) | (T1*T2)| (T1^T2) E  (T1 = T2) | (T1 < T2 ) (relations between terms)  8 x. E1 | 9 x. E1 8 x. (S(S(x)) < S(x^S(S(0)))) = 8 x. ((x+2) < (x2 + 1))

What about something even more complex, like Java? You can write down the entire grammar!

Statements in Java S  ;  Expression ;  { S } | { }  if ( Expression ) then S1 else S2  do S while ( Expression ) ;  while ( Expression ) S ;  return ; | return Expression ; …

Expressions in Java Expression  Numeric Expression  Logical Expression  String Expression  …

Numerical Expressions in Java NE  - NE | --NE | ++NE  NE -- | NE ++  NE += NE | NE -= NE  NE + NE | NE - NE  ( NE )  Identifier…

Programs in Java Compilation Unit  Import Statement Type Decleration  Class Decleration | Interface Decleration Class Decleration  class Identifier extends ClassName { FieldDeclerationList }

Now we will expand our notion of a legal move a bit to include things that are very intuitive.

Symbol Templates Fix a finite alphabet . Fix a finite alphabet . Let {#} be a symbol not in  [  Define a template for  to be any finite string over  [  .

Template Example.  = {a, b}  = {X, Y} Templates: aaXXa XYXYabaXbaaa

Template Examples  = {a, b}  = {X, Y, Z} XYX matches aaabababaaaba aaabababaaaba has the form XYX

Algebra Game Templates X+Y = Y+X (X+y)+z = x+(y+z) (x*y)*z=x*(y*z) X*Y =Y*X X*(Y+Z) = (X*Y) + (X*Z) X+0 = X X*0= 0

Algebra Game Allows substitution too!

Proof: S=S, SS=SS, SSS=SSS Example UBL  = { S, =} One Axiom A: { S=S } One Inference Rule: r() = SS Proof: S=S, SS=SS, SSS=SSS

One Inference Rule: r() = SS Proof that SSS=SSS: S=S, SS=SS, SSS=SSS Example UBL  = { S, =} One Axiom A: { S=S } One Inference Rule: r() = SS Proof that SSS=SSS: S=S, SS=SS, SSS=SSS

One Inference Rule: r() = SS Example UBL  = { S, =} One Axiom A: { S=S } One Inference Rule: r() = SS Proof that premise SSS=S implies SSSSS=SSS: SSS=S, SSSS=SS, SSSSS=SSS

One Inference Rule: r() = SS Example UBL  = { S, =} One Axiom A: { S=S } One Inference Rule: r() = SS UBL is completely defined in syntactic terms. UBL is just a game of symbol pushing. Does it have a natural semantics?

UBL can be interpreted as UNARY BALANCE LOGIC

Axioms = all strings of the form     Propositional Logic Axioms = all strings of the form     Inference rules [Hilbert]: r1(  ) =  r2() =    [this is a relation, actually] r3((  )  ) =   (   ) Br(  , :   ) =   

Example: p, : p  q implies q p, p  q, : p  q, q  q , q Inference rules [Hilbert] : r1(  ) =  r2() =    [this is a relation, actually] r3((  )  ) =   (   ) Br(  , :   ) =   

Propositional Logic Propostional logic and proof can be viewed as a symbol game, or view semantically. : means NOT,  means OR. An n-variable formula is a tautology if it evaluates to TRUE under all 2n truth assignments.

Propositional Logic SOUND: Any formula that can be derived by the manipulation rules is indeed a tautology. COMPLETE: All tautologies can be derived by the logic.

First-order Predicates Enhance our syntax to be give us the possibility of a richer semantics.

Inductive Grammar Notation <integer> ::= <natural> | -<natural> <natural> ::= <decimal digit> | <natural> <decimal digit> <decimal digit> ::= 0|1|2|3|4|5|6|7|8|9

Grammar: <operator> ::= + | - | * | / <expression> ::= <natural> | <natural> <operator> <natural> <expression>::= (<expression) | - (<expression>) |(<expression>)<operator>(<expression>)

Computer Language Syntax The syntax of computer languages can be defined by an inductive grammar. In fact, most languages can be defined by a restricted form of grammar, called BNF form.

The Natural Numbers  = { 0, S0, SS0, SSS0, . . .} Inductive definition: 0 is a natural number called zero. Set notation: 0 2  If X is a natural number, then SX is a natural number called successor of X. Set notation: X 2  ) SX 2 

Inductive Definition of +  = { 0, S0, SS0, SSS0, . . .} Inductive definition of addition (+): X, Y 2  ) X “+” 0 = X X “+” SY = S(X”+”Y)

Defining One to Ten “1” = S0; “2”= SS0; “3” = SSS0. “4” = SSSS0. “5” = SSSSS0; “6” = SSSSSS0; “7” = SSSSSSS0. “8” = SSSSSSSS0; “9=SSSSSSSSS0. “10” = SSSSSSSSSS0 Or…. “1” = S0. “2” = S1. “3” = S2. “4” = S5 …

1 + 1 = 2 Proof: 1 + 1 = S0 + S0 = S(S0 + 0) = S(S0) = SS0 = 2 X, Y 2  ) X “+” 0 = X X “+” SY = S(X”+”Y)

Inductive Definition of *  = { 0, S0, SS0, SSS0, . . .} Inductive definition of times (*): X, Y 2  ) X “*” 0 = 0 X “*” SY = (X”*”Y) + X

Inductive Definition of ^  = { 0, S0, SS0, SSS0, . . .} Inductive definition of raised to the (^): X, Y 2  ) X “^” 0 = 1 [or X0 = 1 ] X “^” SY = (X”^”Y) * X [or XSY = XY * X]

Inductive Definition of Base 10 Notation. Base10( 10*X + 0) = Base10(X) “0” Base10( 10*X + 1) = Base10(X) “1” … Base10( 10*X + 1) = Base10(X) “9” Base10(SSSSSSSSSSSS0) = Base10(SSSSSSSSSS0) 2 = 1 2

 = { 0, 1, 2, 3, . . .} Defining < for : 8 x,y 2  “x > y” is TRUE  “y < x” is TRUE “x > y” is TRUE  “y > x” is FALSE “x+1 > 0” is TRUE “x+1 > y+1” is TRUE ) “x > y” is TRUE

 = { 0, 1, 2, 3, . . .} Defining partial minus for : 8 x,y 2  x-0 = x x>y ) (x+1) – (y+1) = x-y

a = [a DIV b]*b + [a mod b] Defining DIV and MOD for : 8 a,b 2  a DIV b = 1 + (a-b) DIV b a MOD b = a – [b*(a DIV b)] The maximum number of times b goes into a without going over. The remainder when a is divided by b.

45 = [45 DIV 10]*10 + [ 45 MOD 10] = 4*10 + 5 Defining DIV and MOD for : 8 a,b 2  a<b ) a DIV b = 0 a¸b>0 ) a DIV b = 1 + (a-b) DIV b a MOD b = a – [b*(a DIV b)]

Giuseppe Peano [1889] Axiom’s For  A1. Sx  0 A2. [Sx = Sy] ) [x=y] A3. For any proposition P(x) where x2 . Mathematical Induction Applies To P: [P(0) and 8 x2 , P(x)) P(Sx)] ) 8 x P(x)

Giuseppe Peano [1889] Axiom’s For  A1. Sx  0 A2. [Sx = Sy] ) [x=y] Let’s prove the Commutativity of addition: x +y = y+ x A3. For any proposition P(x) where x2 . Mathematical Induction Applies To P: [P(0) and 8 x2 , P(x)) P(Sx)] ) 8 x P(x) Inductive Definition of +: x + 0 = x x + Sy = S(x + y)

Lemma: 0 + x = x Let P(x) be the proposition that “0 + x = x” P(0) is “0 + 0 = 0” Assume P(x): “0 + x = x” Show P(Sx): 0+Sx = S(0+x) = S(x) = Sx

Lemma: Sx + y = S(x+y) Let P(y) be the proposition that “8 x, Sx + y = S(x+y)” P(0) is “8 x, Sx + 0 = S(x+0) = Sx” Assume P(y): “8 x, Sx+y = S(x+y)” Show P(Sy): Sx+Sy = S(Sx+y) = S(S(x+y) ) = S(x+Sy)

Theorem: Commutative Property Of Addition: x + y = y + x Let P(y) be the proposition that “8 x, x + y = y + x” P(0) is “8 x, x + 0 = 0 + x” Assume P(y): “8 x, x + y = y + x” Show P(Sy): x+Sy = S(x+y) = S(y+x) = Sy + x

“God Made Induction On The Naturals. Everything Else Is The Work Of Man.” Peano = { 0, 1, 2, 3, . . .} Peano Axioms: Gives us inductive reasoning. Inductively Define: all of arithmetic, high school math, and beyond…

References The Heritage of Thales, by W. S. Anglin and F. Lambek The Book Of Numbers, by J. Conway and R. Guy Programming Pearls, by J. Bentley History of Mathematics, Histories of Problems, by The Inter-IREM Commission