1. LeongHW, SoC, NUS (UIT2201: AI) Page 1 © Leong Hon Wai, 2003-2008 Integrating Different Ideas Together  Reading Materials:  Ch 3.6 of [SG]  Contents:  Incrementing a Binary Counter  How many subsets, Representation  Printing All Subsets Problem  Exponential Time Algorithms

2. LeongHW, SoC, NUS (UIT2201: AI) Page 2 © Leong Hon Wai, 2003-2008 Four seemingly unrelated problems P1: You are given a bit-array A = (A[n−1], A[n−2],... A[1] A[0]), where each A[k] is 0 or 1. Given an algorithm for incrementing A by 1. P2: Suppose you are given a set S = {s 1, s 2,..., s n } of n objects, and a subset X of S. How can we use the bit-vector A to represent the subset X? P3: Using P1 and P2 above, or otherwise, give a simple algorithm to generate and print all subsets of S. P4: What is the running time of your algorithm? [Note: Recall that there are 2 n subsets altogether. (A proof is provided by P1 and P2 above.)]

3. Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.3 Copyright © 2005-7 by Leong Hon Wai) n-bit Binary Counter xA[4]A[3]A[2]A[1]A[0]Cost 0000000 1000011 2000103 3000114 4001007 5001018 60011010 70011111 80100015 90100116 100101018 110101119

4. Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.4 Copyright © 2005-7 by Leong Hon Wai) Incrementing a Binary Counter  n-bit Binary Counter: A[0..n  1] x = A[n−1] ⋅ 2 (n−1) +... + A[1] ⋅ 2 1 + A[0] ⋅ 2 0 I NCREMENT (A) 1. i  0 2. while i < length[A] and A[i] = 1 3. do A[i]  0 ⊳ reset a bit 4. i  i + 1 5. if i < length[A] 6. then A[i]  1 ⊳ set a bit

5. LeongHW, SoC, NUS (UIT2201: AI) Page 5 © Leong Hon Wai, 2003-2008 P1: You are given a bit-array A = (A[n−1], A[n−2],... A[1] A[0]), where each A[k] is 0 or 1. Given an algorithm for incrementing A by 1. Note: For UIT2201, only need to understand the first 2 slides on the problem P1 (incrementing a binary counter). (The other slides are included for info only.) Four seemingly unrelated problems

6. Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.6 Copyright © 2005-7 by Leong Hon Wai) Worst-case analysis Consider a sequence of n insertions. The worst-case time to execute one insertion is  (k). Therefore, the worst-case time for all the n insertions is n ·  (k) =  (n  k). In fact, WRONG! In fact, the worst-case cost for n insertions is only  (n) ≪  (n  k). Let’s see why.

7. Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.7 Copyright © 2005-7 by Leong Hon Wai) Tighter analysis xA[4]A[3]A[2]A[1]A[0]Cost 0000000 1000011 2000103 3000114 4001007 5001018 60011010 70011111 80100015 90100116 100101018 110101119 A[0] flipped every op n A[1] flipped every 2 ops n/2 A[2] flipped every 4 ops n/2 2 A[3] flipped every 8 ops n/2 3 … … … … … A[i] flipped every 2 i ops n/2 i Total cost of n operations

8. Hon Wai Leong, NUS (CS5234, 28 Aug 2007) Page L3.8 Copyright © 2005-7 by Leong Hon Wai) Tighter analysis (continued) Cost of n increments. Thus, the average cost of each increment operation is  (n)/n =  (1).

9. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 9 Copyright © by Leong Hon Wai P2: Using n-bit-array to represent subsets?

10. LeongHW, SoC, NUS (UIT2201: AI) Page 10 © Leong Hon Wai, 2003-2008 Four seemingly unrelated problems P1: You are given a bit-array A = (A[n−1], A[n−2],... A[1] A[0]), where each A[k] is 0 or 1. Given an algorithm for incrementing A by 1. P2: Suppose you are given a set S = {s 1, s 2,..., s n } of n objects, and a subset X of S. How can we use the bit-vector A to represent the subset X? Note: For UIT2201, only need to understand bit-representation method for problem P2. (The other slides are included for info only.)

11. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 11 Copyright © by Leong Hon Wai PS: (Power-Set Example) (1) Problem: Given a set S with n elements. How many subsets of S are there? Stage 1: Understanding the Problem PQ: What is the unknown? [# subsets of S] PQ: What is the data? [A set S with n elements] PQ: What is the condition? [Subsets of S. Need to count all of them.] PQ: Is it sufficient? [Yes? Can count one-by-one, but tedious]

12. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 12 Copyright © by Leong Hon Wai PS: (Power-Set Example) (2) Stage 2: Devising a Plan PQ: Have you seen the problem before? PQ: Can you try to work out some small instances? PQ: Can you see any pattern? Stage 3: Carrying out the Plan. PQ: Can you prove the result? n S P(S) # 0 ϕ ϕ 1 1 {x 1 } ϕ, {x 1 } 2 2 {x 1, x 2 } ϕ, {x 1 }, { x 2 }, {x 1, x 2 } 4 3 {x 1, x 2, x 3 } ϕ, {x 1 }, { x 2 }, {x 1, x 2 } {x 3 }, { x 1, x 3 },{ x 2, x 3 }, {x 1, x 2, x 3 } 8

13. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 13 Copyright © by Leong Hon Wai PS: (Power-Set Example) (3) Another Approach? PQ: Can you solve it differently? PQ: Introduce suitable notations? Let A n be the # of subsets of a S with n element. Now, consider the set S’ = S U {x} with n+1 elements. Divide subsets of S’ into (P1) those containing element x, and (P2) those that do not contain element x. Those in P2 are exactly all the subsets of S (and we have A n of them). For every subset T of S in P2, there is a corresponding subset that contains x, namely T U {x} in P1. Thus, there is a 1-1 correspondence betw subsets in P1 and those in P2. Therefore, A n+1 = 2  A n for all n ≥ 0. In addition, we know A 0 = 1.

14. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 14 Copyright © by Leong Hon Wai PS: (Power-Set Example) (4) Yet Another Approach? PQ: Draw a figure? c'c' cc'c' ccc'c' cc'c' bb'b aa' Where are the subset?

15. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 15 Copyright © by Leong Hon Wai PS: (Power-Set Example) (5) Yet Another Method? PQ: Look at the unknown. [Subsets! They come in different sizes] IDEA: Let’s count those of the same size! How many of size 0?1 How many of size 1? n How many of size 2? n(n-1)/2 = n C 2... How many of size k? n C k... How many of size n–1?n How many of size n? 1 Total # of subsets is

16. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 16 Copyright © by Leong Hon Wai PS: (Power-Set Example) (6) Yet another method? PQ: How can the subsets be represented? Let the vector (b 1, b 2, …, b n ) represent the subset T where b k =1, if the element x k is in the set T, and 0 otherwise. PQ: How many such bit-strings are there? Subset Bit-Representation ϕ (0,0,…,0,0) {x 1 } (1,0,…,0,0) {x 2 } (0,1,…,0,0) {x 1, x 2 } (1,1,…,0,0) {x 1, x 2,..., x n } (1,1,…,1,1)

17. LeongHW, SoC, NUS (UIT2201: AI) Page 17 © Leong Hon Wai, 2003-2008 Four seemingly unrelated problems P1: You are given a bit-array A = (A[n−1], A[n−2],... A[1] A[0]), where each A[k] is 0 or 1. Given an algorithm for incrementing A by 1. P2: Suppose you are given a set S = {s 1, s 2,..., s n } of n objects, and a subset X of S. How can we use the bit-vector A to represent the subset X? P3: Using P1 and P2 above, or otherwise, give a simple algorithm to generate and print all subsets of S. P4: What is the running time of your algorithm? [Note: Recall that there are 2 n subsets altogether. (A proof is provided byP1 and P2 above.)]

18. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 18 Copyright © by Leong Hon Wai Solution to P3 and P4 Outline of algorithm for P3: Represent subsets using bit vector A 1. Start with A = (0, 0,..., 0, 0) 2. Repeat these steps 3. Increment the bit vector A (use algorithm from P1) 4. Print out the subset corresponding to A 5. until A = (1, 1,..., 1, 1) Analysis (P4): Steps 3,4 done 2 n time; (there are 2 n subsets!)

19. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 19 Copyright © by Leong Hon Wai Solution to P3 and P4 Analysis (P4): Steps 3,4 done 2 n time; (there are 2 n subsets!) Step 3 takes time at most O(n) Step 4 takes time at most O(n) Total time: 2 n * O(n) = O(n 2 n ) This is an exponential time algorithm! Lower Bound: Since there 2 n subsets; Just printing them will take at least 2 n time!

20. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 20 Copyright © by Leong Hon Wai One Final Question ?

21. LeongHW, SoC, NUS (UIT2201: AI) Page 21 © Leong Hon Wai, 2003-2008 Exponential Time Problems…  Intel designs CPU chips for most of todays’ computers (IBM-compatible, Macs, Linux, etc)  If Intel wants to thoroughly verify their multiplication circuit is correct (i.e. produces the correct product for all input combinations), what does it need to do?  Check that for every a, b compute and verify the product (a * b)  Do you think that Intel has actually done that for their 32-bit and 64-bit processors?

22. Hon Wai Leong, SoC, NUS (Creative Problem Solving) Page 22 Copyright © by Leong Hon Wai Thank you!

23. LeongHW, SoC, NUS (UIT2201: AI) Page 23 © Leong Hon Wai, 2003-2008