1. UMass Lowell Computer Science 91.404 Analysis of Algorithms Prof. Karen Daniels Spring, 2001 Lectures 3/4 Chapters 1 & 2 Wed. 2/7/01 and Thursday 2/8/01 or “How to Make an Algorithm Sandwich” adapted from the fall, 2000 talk Finding the Largest Area Axis-Parallel Rectangle in a Polygon in O(n log 2 n) Time given at UMass Lowell MATHEMATICAL SCIENCES COLLOQUIUM

2. Context: Taxonomy of Problems Supporting Apparel Manufacturing OrderedContainment Geometric Restriction Distance-BasedSubdivision MaximumRectangle Limited Gaps MinimalEnclosure Column-Based Layout Two-Phase Layout LatticePacking Containment Maximal Cover

3. A Common (sub)Problem Find a Good Approximation Outer Inner

4. ä Given a 2D polygon that: ä does not intersect itself ä may have holes ä has n vertices ä Find the Largest-Area Axis-Parallel Rectangle ä How “hard” is it? ä How “fast” can we find it? What’s the Problem?

5. ä Explore the problem to gain intuition: ä Describe it: What are the assumptions? (model of computation, etc...) ä Has it already been solved? ä Have similar problems been solved? (more on this later) ä What does best-case input look like? ä What does worst-case input look like? ä Establish worst-case upper bound on the problem using an algorithm ä Design a (simple) algorithm and find an upper bound on its worst-case asymptotic running time; this tells us problem can be solved in a certain amount of time. Algorithms taking more than this amount of time may exist, but won’t help us. ä Establish worst-case lower bound on the problem ä Tighten each bound to form a worst-case “sandwich” Approach increasing worst-case asymptotic running time as a function of n n 1 2n2n2n2n n2n2n2n2 n3n3n3n3 n4n4n4n4 n5n5n5n5

6. Upper Bound First Attempt

7. Designing an Algorithm to Provide Upper Bound Contacts reduce degrees of freedom ä First attempt uses a straightforward approach: ä “brute-force” ä “naïve” ä Will most likely produce a “loose” upper bound ä Characterize rectangle based on how it can “grow”

8. O(n 5 ) LR Algorithm Find_LR(Polygon P) area0 Find_LR_0_RC(P) area0 Find_LR_0_RC(P) area1 Find_LR_1_RC(P) area1 Find_LR_1_RC(P) area2 Find_LR_2_RC(P) area2 Find_LR_2_RC(P) area3 Find_LR_3_RC(P) area3 Find_LR_3_RC(P) area4 Find_LR_4_RC(P) area4 Find_LR_4_RC(P) return maximum(area0, area1, area2, area3, area4) return maximum(area0, area1, area2, area3, area4)Find_LR_0_RC(P) for i 1 to n [for each edge of P] for j 1 to n for j 1 to n for k 1 to n for k 1 to n for l 1 to n for l 1 to n area area of LR for 0-RC determining set for (i,j,k,l) area area of LR for 0-RC determining set for (i,j,k,l) if LR is empty, then update maximum area if LR is empty, then update maximum area return maximum area return maximum area O(n) O(n 5 ) this test has a O(n) loop down inside it

9. First Upper Bound: What can we really conclude? ä Algorithm’s worst-case running time is in O(n 5 ). ä Problem can be solved in O(n 5 ) time, even for worst-case inputs. ä Note: there might exist algorithms for this problem that take more than n 5 time, but they aren’t useful to us!  If a worst-case input exists that causes algorithm to actually use time proportional to n 5, then algorithm’s worst-case running time is also in  (n 5 ).  In this case, we can say algorithm’s worst-case running time is in  (n 5 ). n 1 n2n2n2n2 2n2n2n2n n5n5n5n5 increasing worst-case asymptotic running time as a function of n An inefficient algorithm for the problem might exist that takes this much time, but would not help us.

10. Lower Bound First Attempt n 1 n2n2n2n2 2n2n2n2n n5n5n5n5 increasing worst-case asymptotic running time as a function of n An inefficient algorithm for the problem might exist that takes this much time, but would not help us.

11. First Attempt ä First attempt will most likely produce a “loose” lower bound that can be improved later.   (1) and  (n) are not hard to achieve:  Rationale for  (n) is that we must examine every vertex at least once in order to solve the problem. This holds for every algorithm that solves the problem, so it gives us a lower bound on the problem in an even stronger sense than our upper bound. ä Remember that our upper bound does not guarantee that no algorithm exists for this problem that takes more than O(n 5 ) time, but if one exists it will not help us! n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem An inefficient algorithm for the problem might exist that takes this much time, but that would not help us. No algorithm for the problem exists that can solve it for worst-case inputs in less than linear time.

12. Know the Difference! n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem An inefficient algorithm for the problem might exist that takes this much time, but would not help us. No algorithm for the problem exists that can solve it for worst-case inputs in less than linear time. Strong Bound: This worst-case lower bound on the problem holds for every algorithm that solves the problem and abides by our problem’s assumptions. Weak Bound: This worst-case upper bound on the problem comes from just considering one algorithm. Other, less efficient algorithms that solve this problem might exist, but we don’t care about them! Both the upper and lower bounds are probably loose (i.e. probably can be tightened later on).

13. Upper Bound Tightening It

14. ä Think harder… ä Design an algorithm that takes only O(n 2 ) time for worst-case inputs Approach n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem n2n2n2n2 Now we no longer care about this one! tighter upper bound

15. ä Think even harder… ä Run into brick wall! ä Characterize the brick wall: this type of case ä Go around brick wall by looking at similar problems. ä This requires ability to compare functions! Approach (continued) n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem n2n2n2n2

16. Some Related Problems n 1 n log(n) n log 2 (n) 2n2n2n2n n5n5n5n5 n  (n) log(n) n  (n) This slide and the next are included just to show a variety of functions. Don’t worry about the details!  (n) is the very slowly growing inverse of Ackermann’s function

17. Some Related Problems (continued) n 1 n log(n) n log 2 (n) 2n2n2n2n n5n5n5n5 n  (n) log(n) n  (n)

18. ä Reduce the O(n 2 ) bound to O(n log 2 n): ä Adapt a technique that worked for a similar problem in order to develop a general framework for the problematic case Approach (continued) n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem n2n2n2n2 n log 2 n

19. Lower Bound Tightening It

20. Approach ä Go around lower bound brick wall by: ä examining strong lower bounds for some similar problems ä transforming a similar problem to our problem [this process is similar to what we do when we prove problems NP-complete] n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem n2n2n2n2 n log 2 n

21. Lower Bounds for Related Problems n1 n log(n) n log 2 (n) 2n2n2n2n n5n5n5n5 SmallestOuterRectangle:   (n) SmallestOuterCircle:   (n) LargestInnerRectangle:   (n) LargestInnerCircle:   (n log n) point set, polygon point set polygon LargestInnerRectangle:   (n log 2 (n))polygon worst-case bounds on our problem

22. Lower Bound of  (n log n) by Transforming a ( seemingly unrelated ) Problem MAX-GAP instance: given n real numbers { x 1, x 2,... x n } find the maximum difference between 2 consecutive numbers in the sorted list. O(n) time transformation specialized polygon x2x2x2x2 x4x4x4x4 x3x3x3x3 x1x1x1x1 Rectangle area is a solution to the MAX-GAP instance Rectangle algorithm must take as least as much time as MAX-GAP. MAX-GAP is known to be in  (n log n). Rectangle algorithm must take  (n log n) time for specialized polygons. [Transforming yet another different problem yields bound for unspecialized polygons.]

23. ä First attempt: ä Establish weak (and loose) worst-case upper bound on the problem using an algorithm ä Establish strong (and loose) worst-case lower bound on the problem ä Tighten bounds to make an “algorithm sandwich” ä Establish weak (but tighter) worst-case O(n log 2 n) upper bound on the problem using an algorithm ä Establish strong (and tighter) worst-case lower bound on the problem by transforming a problem with known lower bound Summary n1 n log(n) n log 2 (n) 2n2n2n2n n5n5n5n5 bounds on problem

24. For More Information ä Computational Geometry: ä Graduate CS course in Computational Geometry to be offered at UMass Lowell in Spring ‘01 ä Introductory texts: ä Computational Geometry in C (O’Rourke) ä Computational Geometry: An Introduction (Preparata & Shamos) ä Bibliography: ftp://ftp.cs.usask.ca/pub/geometry/ ä Software:http://www.geom.umn.edu/software/cglist/ ä My research: ä http://www.cs.uml.edu/~kdaniels http://www.cs.uml.edu/~kdaniels ä Journal paper: “Finding the largest area axis-parallel rectangle in a polygon” ä (Computational Geometry: Theory and Applications) ä Prof. Victor Milenkovic: Frequent co-author and former PhD advisor ä http://www.cs.miami.edu/~vjm http://www.cs.miami.edu/~vjm

25. Remainder of 2/8 Lecture Homework 1 Clarifications & Hints

26. HW1 Clarifications & Hints Printed copy is missing some text from problem 6(d). It should read: (d) Can you conclude from your answers to (b) and (c) that Mystery’s worst-case running time is =  (f(n))? Why or why not? Web copy: view it in Page Layout mode so you see arrows in problem 6 pseudocode and other special (graphical) symbols (like square root!)

27. HW1 Clarifications & Hints (continued) ä General Hint:  Remember the definitions for O,  and  ä f(n) is in the set of functions O(g(n)) if (and only if) 0 = n 0 where c and n 0 are positive constants  f(n) is in the set of functions  (g(n)) if (and only if) f(n) >= cg(n) >= 0 for all n >= n 0 where c and n 0 are positive constants  f(n) is in the set of functions  (g(n)) if (and only if) f(n) is in O(g(n)) and also in  (g(n)). This means: ä 0 = n 0 where c 1 and n 0 are positive constants and ä f(n) >= c 2 g(n) >= 0 for all n >= n 0 where c 2 and n 0 are positive constants ä Important notes: ä c is a constant means that c cannot depend on n!  In the  definition, the constants c 1 and c 2 can be different from each other. But, the upper and lower bounds must work for the same n 0 ! ä For each constant, you only need to find 1 value that works!

28. HW1 Clarifications & Hints (continued) ä Problems 1 & 2 & 3:  The definitions for O and  use inequalities. ä This means that: ä If f(n) is in O(g(n)), then f(n) is also in O(any function larger than g(n))  If f(n) is in  (g(n)), then f(n) is also in  (any function smaller than g(n))

29. HW1 Clarifications & Hints (continued) ä Problem 1: ä Be aware of the difference between a statement about a bound on the running time of an algorithm vs. a statement about a bound on the running time of a problem. Make sure you understand slides 5,9,11,12 of this lecture, which talk about this.  You can assume that in Problem 1 the statements “Let R be a problem. Suppose that the worst-case asymptotic time complexity of R is in O(n 2 ) and also in  (n lg n)” mean that: ä The upper bound is a weak bound that means someone has found an algorithm whose running time is quadratic in n ; this shows the problem can be solved in quadratic time. It does not mean that there does not exist a very inefficient algorithm for this problem that takes more than quadratic time. ä The lower bound is a strong bound that means someone has proven that there cannot exist any algorithm for this problem that can handle worst-case inputs in less than time proportional to nlgn.

30. HW1 Clarifications & Hints (continued) ä Problem 1 (continued):  Also, be sure you understand what is necessary to conclude that an algorithm has a matching upper and lower bound so that the  notation can be used (see the discussion on slide 9 and think back to our discussion in class on loops that cannot exit early…)

31. HW1 Clarifications & Hints (continued) ä Problem 3: ä There are 5 functions, so there are 5! possible orders. To find the right one without examining all possible orders, let transitivity help you (see slide 28). ä Here’s an example of how to do this for 3 functions: 2 n n n 3 ä First pick one pair of the functions (we’ll pick 2 n and n 3, but any pair will do)  You need to figure out whether n 3 is in O(2 n ) or n 3 is in  (2 n ) ä That is, you need to solve: n 3 ? c2 n for all n >= n 0 where ? is either >= or = n 0 where ? is either >= or <= ä In some cases it is easy to isolate c and pick a value for c that satisfies the “n” part of the expression whenever n is greater than some value. This current case is a challenge because of the power of n. One approach is to use logarithms. Taking lg of both sides yields: ä 3lgn ? lgc + n3lgn – n ? lgc ä Now, think about the function 3lgn – n and how it behaves for small vs. large n ä For large n, n dominates and makes the function negative. When the function is negative, any value of c >= 1 works. When does it become (and remain) negative? ä Trying n=2,4,8,16 is interesting because we see that 3lgn – n increases from n=2 to n=4 and then decreases. By the time n=16, it is negative and stays negative. ä If we pick c=1 and n 0 =16, then 3lgn <= lgc + n so n 3 <= c2 n and n 3 is in O(2 n ). ä This means that in the ordering of our 3 functions, n 3 must come before 2 n.

32. HW1 Clarifications & Hints (continued) ä Problem 3 (continued): ä Now consider the remaining function: n. We need to know whether n belongs before n 3, between n 3 and 2 n or after 2 n. ä Let’s compare n with n 3. If we can show that n goes before n 3, then, by transitivity, we’ll have the ordering: n n 3 2 n ä So, let’s try to prove that n is in O(n 3 ). To do this, we must find a value of c and a value of n 0 that work so that: n <= c n 3 ä To isolate c, divide both sides by n 3 (n is positive, so this does not change the sense of the inequality). n/ n 3 <= c n 3 1/ n 2 <= c ä Since 1/ n 2 is =1, there are many possibilities for c when n 0 = 1. Let’s pick c=1. Having found c=1 and n 0 = 1 that make n =1, there are many possibilities for c when n 0 = 1. Let’s pick c=1. Having found c=1 and n 0 = 1 that make n <= c n 3 we can conclude that n is in O(n 3 ). ä Because n is in O(n 3 ) and n 3 is in O(2 n ), the ordering is: n n 3 2 n ä Thus, g 1 = ng 2 = n 3 and g 3 = 2 n

33. HW1 Clarifications & Hints (continued) ä Problem 4: ä Your answer should be a value of n that is greater than 0. ä You might want to write a short program that finds the value for you. ä Problem 5: ä To explain why your pseudocode is correct, briefly say why you’re sure that: ä Your pseudocode terminates ä If recursive, your base case must correctly stop the recursion ä If iterative, your loops must stop (no infinite loops)! ä When your pseudocode terminates, it returns the right answer  Your pseudocode has the proper running time:  (lg n) worst-case time in this case. Since you’re justifying upper and lower bounds on the worst-case running time, be sure you say why: ä a worst-case input for this problem won’t cause your algorithm to take more than time proportional to lgn ä there exists a worst-case input for this problem that actually causes your algorithm to take time proportional to lgn

34. HW1 Clarifications & Hints (continued) ä Problem 6: ä You can assume that each print statement in the Mystery pseudocode takes one unit of time = one step.