1. Induction II: Inductive Pictures
Great Theoretical Ideas In Computer Science
Steven Rudich
CS Spring 2004
Lecture 14
Feb 26, 2004
Carnegie Mellon University
Induction II: Inductive Pictures

2. “Least Counter-example” “All Previous” Induction Inductive Definition:
Inductive Proof:
“Standard” Induction
“Least Counter-example”
“All Previous” Induction
Inductive Definition:
Recurrences
Recursive Programming

3. Theorem? (k¸0)
…+2k = 2k+1 -1
Try it out on small examples:
= 21 -1
= 22 -1
= 23 -1

4. Sk´ “1+2+4+8+…+2k = 2k+1 -1” Use induction to prove k¸0, Sk
Establish “Base Case”: S0. We have already check it.
Establish “Domino Property”: k¸0, Sk ) Sk+1
“Inductive Hypothesis” Sk: …+2k = 2k+1 -1
Add 2k+1 to both sides:
…+2k + 2k+1= 2k+1 +2k+1 -1
…+2k + 2k+1= 2k+2 -1

5. FUNDAMENTAL LEMMA OF THE POWERS OF TWO:
The sum of the first n powers of 2, is one less than the next power of 2.

6. Yet another way of packaging inductive reasoning is to define an “invariant”.
Invariant (adj.)
Not varying; constant.
(mathematics) Unaffected by a designated operation, as a transformation of coordinates.

7. Yet another way of packaging inductive reasoning is to define an “invariant”.
Invariant (adj.)
3. (programming) A rule, such as the ordering an ordered list or heap, that applies throughout the life of a data structure or procedure. Each change to the data structure must maintain the correctness of the invariant.

8. Invariant Induction Suppose we have a time varying world state: W0, W1, W2, … Each state change is assumed to come from a list of permissible operations. We seek to prove that statement S is true of all future worlds.
Argue that S is true of the initial world.
Show that if S is true of some world – then S remains true after one permissible operation is performed.

9. Odd/Even Handshaking Theorem: At any party, at any point in time, define a person’s parity as ODD/EVEN according to the number of hands they have shaken. Statement: The number of people of odd parity must be even.
Initial case: Zero hands have been shaken at the start of a party, so zero people have odd parity.
If 2 people of different parities shake, then they both swap parities and the odd parity count is unchanged.
If 2 people of the same parity shake, they both change. But then the odd parity count changes by 2, and remains even.

10. Inductive Definition of n! [said n “factorial”]
0! = 1; n! = n*(n-1)!

11. 0! = 1; n! = n*(n-1)! Program for n! ? F:=1; For x = 1 to n do F:=F*x;
Return F
Program for n! ?

12. 0! = 1; n! = n*(n-1)! Program for n! ? F:=1; For x = 1 to n do F:=F*x;
Return F
Program for n! ?
n=0 returns 1
n=1 returns 1
n=2 returns 2

13. 0! = 1; n! = n*(n-1)! Loop Invariant: F=x!
For x = 1 to n do
F:=F*x;
Return F
Loop Invariant: F=x!
True for x=0. If true after k times through – true after k+1 times through.

14. Inductive Definition of T(n)
T(n) = 4 T(n/2) + n
Notice that T(n) is inductively defined for positive powers of 2, and undefined on other values.

15. Inductive Definition of T(n)
T(n) = 4T(n/2) + n
Notice that T(n) is inductively defined for positive powers of 2, and undefined on other values.
T(1)=1 T(2)=6 T(4)=28 T(8)=120

16. Guess a closed form formula for T(n). Guess G(n)
G(n) = 2n2 - n
Let the domain of G be the powers of two.

17. Two equivalent functions?
G(n) = 2n2 - n
Let the domain of G be the powers of two.
T(1) = 1
T(n) = 4 T(n/2) + n
Domain of T are the powers of two.

18. Inductive Proof of Equivalence
Base: G(1) = 1 and T(1) = 1
Induction Hypothesis:
T(x) = G(x) for x < n
Hence: T(n/2) = G(n/2) = 2(n/2)2 – n/2
T(n) = 4 T(n/2) + n
= 4 G(n/2) + n
= 4 [2(n/2)2 – n/2] + n
= 2n2 – 2n + n
= 2n2 – n
= G(n)
G(n) = 2n2 - n
T(1) = 1
T(n) = 4 T(n/2) + n

19. We inductively proved the assertion that G(n) =T(n).
Giving a formula for T with no sums or recurrences is called solving the recurrence T.

20. Solving Recurrences Guess and Verify
Guess: G(n) = 2n2 – n
Verify: G(1) = 1 and G(n) = 4 G(n/2) + n
Similarly:T(1) = 1 and T(n) = 4 T(n/2) + n
So T(n) = G(n)

21. Technique 2 Guess Form and Calculate Coefficients
Guess: T(n) = an2 + bn + c for some a,b,c
Calculate: T(1) = 1  a + b + c = 1
T(n) = 4 T(n/2) + n
 an2 + bn + c = 4 [a(n/2)2 + b(n/2) + c] + n
= an2 + 2bn + 4c + n
 (b+1)n + 3c = 0
Therefore: b= c=0 a=2

22. A computer scientist not only deals with numbers, but also with
Finite Strings of symbols
Very visual objects called graphs
And especially, especially the special graphs called trees

23. GRAPHS b
root a

24. Definition: Graphs
A graph G = (V,E) consists of a finite set V of vertices (nodes) and a finite set E of edges. Each edge is a set {a, b} of two different vertices.
A graph may not have self loops or multiple edges.

25. Definition: Directed Graphs
A graph G = (V,E) consists of a finite set V of vertices (nodes) and a finite set E of edges. Each edge is an ordered pair <a,b> of two different vertices.
Unless we say otherwise, our directed graphs will not have multi-edges, or self loops.

26. Definition: Tree
A tree is a directed graph with one special node called the root and the property that each node must a unique path from the root to itself.
Child: If <u,v>2E, we sav is a child of u
Parent: If <u,v>2E, we say u is the parent of u
Leaf: If u has no children, we say u is leaf.
Siblings: If u and v have the same parent, they are siblings.
Descendants of u: The set of nodes reachable from u (including u).
Sub-tree rooted at u: Descendants of u and all the edges between them where u has been designated as a root.

27. Classic Visualization: Tree
Inductive rule:
If G is a single node
Viz(G) =
If G consists of root r with sub-trees T1, T2, …, Tk
Vuz(T1)
Viz(T2)
Viz(Tk)
…..

28. I own 3 beanies and 2 ties. How many beanie/tie combos might I wear to the ball tonight?

30. Choice Tree
A choice tree is a tree with an object called a “choice” associated with each edge and a label on each leaf.

31. Definition: Labeled Tree
A tree node labeled by S is a tree T = <V,E> with an associated function
Label1: V to S
A tree edge labeled by S is a tree T = <V,E> with an associated function
Label2: E to S

32. was very illuminating.
Let’s do something similar to illuminate the nature of T(1)=1; T(n)= 4T(n/2) + n

33. T(1)=1; T(n)= 4T(n/2) + n
For each n (power of 2),
we will define a tree W(n) node labeled by Natural numbers. W(n) will give us an incisive picture of T(n).

34. Inductive Definition Of Labeled Tree W(n)
T(n) = n + 4 T(n/2)
W(n) = n
W(n/2)
W(n/2)
W(n/2)
W(n/2)
T(1) =
W(1) = 1

35. Inductive Definition Of Labeled Tree W(n)= 2 1 1 1 1
T(1) =
W(1) = 1

36. Inductive Definition Of Labeled Tree W(n)= 4 2 1 2 1 2 1 2 1

37. Inductive Definition Of Labeled Tree W(n)= 4 2 1 2 1 2 1 2 1
NOTE: When we sum all the node labels on W(n), we get T(n)

38. Invariant: LabelSum W(n) = T(n)
T(n) = n + 4 T(n/2)
W(n) = n
W(n/2)
W(n/2)
W(n/2)
W(n/2)
T(1) =
W(1) = 1

39. W(n) = n
W(n/2)
W(n/2)
W(n/2)
W(n/2)

40. W(n) = n
n/2
W(n/4)
W(n/2)
W(n/2)
W(n/2)

41. W(n) = n
n/2
W(n/4)
n/2
W(n/4)
n/2
W(n/4)
n/2
W(n/4)

42. W(n) = n n/2 n/2 n/2 n/2 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4 n/4

43. Level i is the sum of 4i copies of n/2i
n/ n/ n/ n/2
Level i is the sum of 4i copies of n/2i 1 2
n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 + n/4 i

44. = 1n
= 2n
= 4n
= 2in
= (n)n
n( n) = n(2n-1) = n2-n

45. Instead of
T(1)=1; T(n)= 4T(n/2) + n
We could illuminate
T(1)=1; T(n) = 2T(n/2) + 2

46. T(n) = n + 2 T(n/2)
W(n) = n
W(n/2)
W(n/2)
T(1) =
W(1) = 1

47. Level i is the sum of 2i copies of n/2i
n/ n/2
Level i is the sum of 2i copies of n/2i 1 2 i

48. n 1 2 i

49. Has closed form: nlog2(n) where n is a power of 2
T(1)=1; T(n) = 2T(n/2) + n
Has closed form: nlog2(n) where n is a power of 2

50. Representing a recurrence relation as a labeled tree is one of the basics tools you will have to put recurrence relations in closed form.

51. The Lindenmayer Game  = {a,b} Start word: a SUB(a) = ab SUB(b) = a
For each w = w1 w2 … wn
NEXT(w) = SUB(w1)SUB(w2)..SUB(wn)

52. The Lindenmayer Game SUB(a) = ab SUB(b) = a For each w = w1 w2 … wn
NEXT(w) = SUB(w1)SUB(w2)..SUB(wn)
Time 1: a
Time 2: ab
Time 3: aba
Time 4: abaab
Time 5: abaababa

53. How long are the strings as a function of time?
The Lindenmayer Game
SUB(a) = ab SUB(b) = a
For each w = w1 w2 … wn
NEXT(w) = SUB(w1)SUB(w2)..SUB(wn)
Time 1: a
Time 2: ab
Time 3: aba
Time 4: abaab
Time 5: abaababa
How long are the strings as a function of time?

54. Aristid Lindenmayer (1925-1989)
1968 Invents L-systems in Theoretical Botany
Time 1: a
Time 2: ab
Time 3: aba
Time 4: abaab
Time 5: abaababa
FIBONACCI(n) cells at time n

55. The Koch Game  = {F,+,-} Start word: F
SUB(F) = F+F--F+F SUB(+)=+ SUB(-)=-
For each w = w1 w2 … wn
NEXT(w) = SUB(w1)SUB(w2)..SUB(wn)

56. The Koch Game Gen 0:F Gen 1: F+F--F+F
Gen 2: F+F--F+F+F+F--F+F--F+F--F+F+F+F--F+F

57. The Koch Game Picture representation:
F draw forward one unit + turn 60 degree left - turn 60 degrees right.
Gen 0: F
Gen 1: F+F--F+F
Gen 2: F+F--F+F+F+F--F+F--F+F--F+F+F+F--F+F

58. The Koch Game F+F--F+F

59. The Koch Game F+F--F+F+F+F--F+F--F+F--F+F+F+F--F+F

60. Koch Curve

61. Dragon Game
SUB(X) =X+YF+
SUB(Y) = -FX-Y                                    
Dragon Curve:            

62. Hilbert Game
SUB(L)= +RF-LFL-FR+
SUB(R)= -LF+RFR+FL-                                    
Hilbert Curve:            
Note: Make 90 degree turns instead of 60 degrees.          

63. Hilbert’s Space Filling Curve

64. Peano-Gossamer Curve

65. Sierpinski Triangle

66. Lindenmayer 1968 SUB(F) = F[-F]F[+F][F]
Interpret the stuff inside brackets as a branch.

67. Lindenmayer 1968

68. Inductive Leaf

70. Here is another kind of inductive definition called a formal grammar.

71. Formal Grammar: G = (T, V, S, P)
A finite alphabet T called the terminal symbols.
A disjoint finite alphabet N of non-terminal symbols (or variable symbols). One symbol S in V is called the start symbol.
A finite set P of production rules of the form   , where , 2 (N [ T)* and  contains at least one non-terminal symbol.

72. Example. Terminal Symbols: {a,b,c} Non-terminal symbols: {S, X}
Start symbol: S
Production rules:
S  SS
S  a
SS  b

73. Formal Grammar: G = (T, V, S, P)
A finite set P of production rules of the form   , where , 2 (N [ T)* and  contains at least one non-terminal symbol.
Let x and y be strings over N[T.
We say that y is produced in one step from x if there is a rule such that one occurrence of the substring  is replaced with  to obtain the string y. We denote this x 1 y

74. S 1 SSS 1 bS 1 ba Terminal Symbols: {a,b,c}
Non-terminal symbols: {S, X}
Start symbol: S
Production rules:
S  SS
S  a
SS  b

75. Definition of x k y We know what x 1 y means.
Define x 0 y only when x = y
Define x ->k to mean
9 z s.t. x->1z and z->k-1y
Define x*y to mean that there exists a k such that xk y .

76. Language Produced By Formal Grammar G
LG = {x 2 T* | S*x }
= {x 2 T* | 9 k s.t. Skx }
The language of strings of terminal symbols that are eventually produced by the start symbol.

77. Language = {a,b}+ Terminal Symbols: {a,b,c}
Non-terminal symbols: {S, X}
Start symbol: S
Production rules:
S  SS
S  a
SS  b

78. Evaluating A Formal Derivation
Let G be a formal grammar.
A sequence D = S1, S2, …., Sk, of strings is called a formal derivation of  if:
For 1· q · k-1 Sq 1 Sq+1

79. Once you agree on a specific grammar G, it is trivial to verify proofs of membership in the language.

80. G = Balanced S  () S  SS S  (S)
Now let’s show Symbol Dude the following derivation:
S, (S), (SS), (SS), (S()), ((())())

81. Yo, dude! ((())()) is like totally balanced
XXh42TEhha

82. Expressions in Propositional Logic
E  True | False
 x | y | z
  E
 (E Ç E)
 (E Æ E)

83. Yo, dude! :(xÆy) is like totally an expression in propositional logic.
XXh42TEhha

84. Meaning: The Part Symbol Dude Ignores
E  True | False (constants)
 x | y | z (variables)
  E1 (not E1)
 (E1 Ç E2) (E1 or E2)
 (E1 Æ E2) (E1 and E2)

85. Inductively Associate a MEANING M(E) of any expression E
M(T) = T M(F)=F
M(: E) = not M(E)
M( (E1 Æ E2) = M(E1) and M(E2)
M( (E1 Ç E2) = M(E1) or M(E2)
not T = F not F =T
AND.
F and F = F F and T = F T and F – F T and T= T

86. Expressions in First Order Logic
T  x | y | z  f(T1, T2, …, Tk) (functions)
T = Terms
E  R(T1, T2, …, Tk) (relation on terms)
  E1 | (E1 Ç E2) | (E1 Æ E2)
 8 x. E (universal quantifier)
 9 x. E (existential quantifier)

87. 8 x. (S(S(x)) < S(x^S(S(0))))
E.g., Number Theory
T  x | y | z
 (the zero function)
 S(T1) (successor function)
 (T1+T2) | (T1*T2)| (T1^T2)
E  (T1 = T2) | (T1 < T2 ) (relations between terms)
 8 x. E1 | 9 x. E1
8 x. (S(S(x)) < S(x^S(S(0))))
= 8 x. ((x+2) < (x2 + 1))

88. What about something even more complex, like Java?
You can write down the entire grammar!

89. Statements in Java S  ;  Expression ;  { S } | { }
 if ( Expression ) then S1 else S2
 do S while ( Expression ) ;
 while ( Expression ) S ;
 return ; | return Expression ; …

90. Expressions in Java Expression  Numeric Expression
 Logical Expression
 String Expression
 …

91. Numerical Expressions in Java
NE  - NE | --NE | ++NE
 NE -- | NE ++
 NE += NE | NE -= NE
 NE + NE | NE - NE
 ( NE )
 Identifier…

92. Programs in Java Compilation Unit  Import Statement Type Decleration
 Class Decleration | Interface Decleration
Class Decleration
 class Identifier extends ClassName
{ FieldDeclerationList }

93. Now we will expand our notion of a legal move a bit to include things that are very intuitive.

94. Symbol Templates
Fix a finite alphabet .
Fix a finite alphabet .
Let {#} be a symbol not in  [ 
Define a template for  to be any finite string over  [  .

95. Template Example.
 = {a, b}
 = {X, Y}
Templates:
aaXXa XYXYabaXbaaa

96. Template Examples  = {a, b}  = {X, Y, Z} XYX matches aaabababaaaba
aaabababaaaba has the form XYX

97. Algebra Game Templates
X+Y = Y+X
(X+y)+z = x+(y+z)
(x*y)*z=x*(y*z)
X*Y =Y*X
X*(Y+Z) = (X*Y) + (X*Z)
X+0 = X
X*0= 0

98. Algebra Game Allows substitution too!

99. Proof: S=S, SS=SS, SSS=SSS
Example UBL  = { S, =}
One Axiom A: { S=S }
One Inference Rule:
r() = SS
Proof: S=S, SS=SS, SSS=SSS

100. One Inference Rule: r() = SS Proof that SSS=SSS: S=S, SS=SS, SSS=SSS
Example UBL  = { S, =}
One Axiom A: { S=S }
One Inference Rule: r() = SS
Proof that SSS=SSS: S=S, SS=SS, SSS=SSS

101. One Inference Rule: r() = SS
Example UBL  = { S, =}
One Axiom A: { S=S }
One Inference Rule: r() = SS
Proof that premise SSS=S implies SSSSS=SSS: SSS=S, SSSS=SS, SSSSS=SSS

102. One Inference Rule: r() = SS
Example UBL  = { S, =}
One Axiom A: { S=S }
One Inference Rule: r() = SS
UBL is completely defined in syntactic terms. UBL is just a game of symbol pushing. Does it have a natural semantics?

103. UBL can be interpreted as UNARY BALANCE LOGIC

104. Axioms = all strings of the form    
Propositional Logic
Axioms = all strings of the form    
Inference rules [Hilbert]:
r1(  ) = 
r2() =    [this is a relation, actually]
r3((  )  ) =   (   )
Br(  , :   ) =   

105. Example: p, : p  q implies q
p, p  q, : p  q, q  q , q
Inference rules [Hilbert] :
r1(  ) = 
r2() =    [this is a relation, actually]
r3((  )  ) =   (   )
Br(  , :   ) =   

106. Propositional Logic
Propostional logic and proof can be viewed as a symbol game, or view semantically.
: means NOT,  means OR.
An n-variable formula is a tautology if it evaluates to TRUE under all 2n truth assignments.

107. Propositional Logic
SOUND: Any formula that can be derived by the manipulation rules is indeed a tautology.
COMPLETE: All tautologies can be derived by the logic.

108. First-order Predicates
Enhance our syntax to be give us the possibility of a richer semantics.

109. Inductive Grammar Notation
<integer> ::= <natural>
| -<natural>
<natural> ::= <decimal digit>
| <natural> <decimal digit>
<decimal digit> ::= 0|1|2|3|4|5|6|7|8|9

110. Grammar: <operator> ::= + | - | * | /
<expression> ::= <natural> | <natural> <operator> <natural>
<expression>::= (<expression) | - (<expression>) |(<expression>)<operator>(<expression>)

111. Computer Language Syntax
The syntax of computer languages can be defined by an inductive grammar. In fact, most languages can be defined by a restricted form of grammar, called BNF form.

112. The Natural Numbers  = { 0, S0, SS0, SSS0, . . .}
Inductive definition:
0 is a natural number called zero.
Set notation: 0 2 
If X is a natural number, then SX is a natural number called successor of X.
Set notation: X 2  ) SX 2 

113. Inductive Definition of +
 = { 0, S0, SS0, SSS0, . . .}
Inductive definition of addition (+):
X, Y 2  )
X “+” 0 = X
X “+” SY = S(X”+”Y)

114. Defining One to Ten
“1” = S0; “2”= SS0; “3” = SSS0. “4” = SSSS0. “5” = SSSSS0; “6” = SSSSSS0; “7” = SSSSSSS0. “8” = SSSSSSSS0; “9=SSSSSSSSS0. “10” = SSSSSSSSSS0
Or…. “1” = S0. “2” = S1. “3” = S2. “4” = S5 …

115. 1 + 1 = 2 Proof: 1 + 1 = S0 + S0 = S(S0 + 0) = S(S0) = SS0 = 2
X, Y 2  )
X “+” 0 = X
X “+” SY = S(X”+”Y)

116. Inductive Definition of *
 = { 0, S0, SS0, SSS0, . . .}
Inductive definition of times (*):
X, Y 2  )
X “*” 0 = 0
X “*” SY = (X”*”Y) + X

117. Inductive Definition of ^
 = { 0, S0, SS0, SSS0, . . .}
Inductive definition of raised to the (^):
X, Y 2  )
X “^” 0 = [or X0 = 1 ]
X “^” SY = (X”^”Y) * X [or XSY = XY * X]

118. Inductive Definition of Base 10 Notation.
Base10( 10*X + 0) = Base10(X) “0”
Base10( 10*X + 1) = Base10(X) “1” …
Base10( 10*X + 1) = Base10(X) “9”
Base10(SSSSSSSSSSSS0) = Base10(SSSSSSSSSS0) 2 =
1 2

119.  = { 0, 1, 2, 3, . . .} Defining < for : 8 x,y 2 
“x > y” is TRUE  “y < x” is TRUE
“x > y” is TRUE  “y > x” is FALSE
“x+1 > 0” is TRUE
“x+1 > y+1” is TRUE ) “x > y” is TRUE

120.  = { 0, 1, 2, 3, . . .} Defining partial minus for : 8 x,y 2 
x-0 = x
x>y )
(x+1) – (y+1) = x-y

121. a = [a DIV b]*b + [a mod b] Defining DIV and MOD for : 8 a,b 2 
a DIV b = 1 + (a-b) DIV b
a MOD b = a – [b*(a DIV b)]
The maximum number of times b goes into a without going over.
The remainder when a is divided by b.

122. 45 = [45 DIV 10]*10 + [ 45 MOD 10] = 4*10 + 5
Defining DIV and MOD for :
8 a,b 2 
a<b )
a DIV b = 0
a¸b>0 )
a DIV b = 1 + (a-b) DIV b
a MOD b = a – [b*(a DIV b)]

123. Giuseppe Peano [1889] Axiom’s For 
A1. Sx  0
A2. [Sx = Sy] ) [x=y]
A3. For any proposition P(x) where x2 . Mathematical Induction Applies To P:
[P(0) and 8 x2 , P(x)) P(Sx)] ) 8 x P(x)

124. Giuseppe Peano [1889] Axiom’s For 
A1. Sx  0
A2. [Sx = Sy] ) [x=y]
Let’s prove the Commutativity of addition: x +y = y+ x
A3. For any proposition P(x) where x2 . Mathematical Induction Applies To P:
[P(0) and 8 x2 , P(x)) P(Sx)] ) 8 x P(x)
Inductive Definition of +:
x + 0 = x
x + Sy = S(x + y)

125. Lemma: 0 + x = x Let P(x) be the proposition that “0 + x = x”
P(0) is “0 + 0 = 0”
Assume P(x): “0 + x = x”
Show P(Sx):
0+Sx = S(0+x) = S(x) = Sx

126. Lemma: Sx + y = S(x+y)
Let P(y) be the proposition that “8 x, Sx + y = S(x+y)”
P(0) is “8 x, Sx + 0 = S(x+0) = Sx”
Assume P(y): “8 x, Sx+y = S(x+y)”
Show P(Sy):
Sx+Sy = S(Sx+y) = S(S(x+y) ) = S(x+Sy)

127. Theorem: Commutative Property Of Addition: x + y = y + x
Let P(y) be the proposition that “8 x, x + y = y + x”
P(0) is “8 x, x + 0 = 0 + x”
Assume P(y): “8 x, x + y = y + x”
Show P(Sy):
x+Sy = S(x+y) = S(y+x) = Sy + x

128. “God Made Induction On The Naturals. Everything Else Is The Work Of Man.”
Peano
= { 0, 1, 2, 3, . . .}
Peano Axioms: Gives us inductive reasoning.
Inductively Define: all of arithmetic, high school math, and beyond…

129. References The Heritage of Thales, by W. S. Anglin and F. Lambek
The Book Of Numbers, by J. Conway and R. Guy
Programming Pearls, by J. Bentley
History of Mathematics, Histories of Problems, by The Inter-IREM Commission