1. איך עונים על שאילתה, כשהתוצאה גדולה מאד? שרה כהן בית הספר להנדסה ולמדעי המחשב ע"ש רחל וסלים בנין ע"ש רחל וסלים בנין

2. Overview Interesting problems with large output Complexity A generic approach for achieving polynomial delay: Lawler-Murty Other approaches Conclusions

3. Shortest Path: A Classic Question Given an undirected, unweighted graph G, and nodes s and t, find the shortest path from s to t. Algorithm: Breadth-First Search (BFS) Complexity: O(|V| + |E|) Polynomial algorithms also exist for directed and weighted cases

4. Shortest Path in the Modern World Graphs are common and important in many applications, e.g., social networks רחלי אוי ברכה דן נפתלי שי

5. Shortest Path Problem Suppose: Bracha wants to speak to Naftali Bennett about using Alice in high school CS programs… רחלי אוי ברכהדן נפתלי שי Perhaps this way of reaching Bennett is rather tactless!

6. New Problem: All Paths in Increasing Length Order Given an undirected, unweighted graph G, and nodes s and t, return all simple paths from s to t, ordered by length. User can view all paths, until she finds one that is “good” for her purposes

7. All Paths Suppose: Bracha wants to speak to Naftali Bennett about using Alice in high school CS programs… רחלי אוי ברכה דן נפתלי שי

8. Another Example of this Problem

9. Complexity of All-Paths Problem? I have asked this on Masters entrance exams and gotten the answers: a)Of course it’s polynomial! b)I learnt an algorithm for this in my BSc studies at the Technion – now what was the complexity? c)Definitely NP-complete! d)Wow, I have no clue…Its polynomial…no, wait, its NP-complete,…,no polynomial,…

10. How big can the output be? רחלי אוי ברכה דן נפתלי שי

11. Complexity of All-Paths Problem? The output can be exponential in the input! No polynomial algorithm exists, since any algorithm will have to print the output, and that is already not polynomial! –In the classic sense, no efficient algorithm exists Many other examples for interesting problems with potentially large output…

12. Best Perfect Matching Given a bipartite, weighted, graph, a best perfect matching is –A perfect matching (i.e., edges that disjointly cover all nodes) –Such that the sum of edge weights is maximal Efficiently solvable in polynomial time –Hungarian algorithm (1955) –Blossom algorithm (1965) But, in real life, finding a single best perfect matching may not be sufficient…

13. Best Perfect Matching edges have weights (not specified)

14. Best Perfect Matching I’m not going to clean the toilet if he is baking a cake!!! Last time I picked up the toys. Let someone else do it this time!!! edges have weights (not specified)

15. New Problem: All Perfect Matchings in Order of Decreasing Weight Sum Given a bipartite, weighted, graph, find –all perfect matching (i.e., edges that disjointly cover all nodes) –in order of decreasing sum weights User can view all matching, until she finds one that is “good” for her purposes Can this be solved efficiently? How big can the output be?

16. Maximal Clique A maximal clique is a set of nodes are (1)all connected one to another, and (2)no additional node can be added to the set while preserving this property Can be used to find communities in a social network

17. Maximal Clique A maximal clique is a set of nodes are (1)all connected one to another, and (2)no additional node can be added to the set while preserving this property Can be used to find communities in a social network

18. New Problem: All Maximal Cliques in Order of Decreasing Size Given a graph, find –all maximal cliques –in order of decreasing size User can view all cliques, until they cease to be interesting Can this be solved efficiently? How big can the output be?

19. Overview Interesting problems with large output Complexity A generic approach for achieving polynomial delay: Lawler-Murty Other approaches Conclusions

20. Are all three problems equally hard? Three problems: –All paths in increasing length order –All perfect matchings in decreasing weight order –All maximal cliques in decreasing size order None of these can be solved in polynomial time (since the output can be exponential in size) All of these can be solved in exponential time with a naïve algorithm that considers every subgraph So, are these problems equally hard??

21. What is an efficient algorithm? All Paths(G,s,t) 1. P =  2. For every subset U of V(G)-{s,t} 3. For every ordering O of U 4. If O is a path from s to t 5. then add O to P 6. Sort P by increasing length 7. Return P רחלי אוי ברכה דן נפתלי שי

22. What is an efficient algorithm (cont) The previous algorithm is certainly not efficient in any sense: –Runs in exponential time even when the output is small! –Does not return a single answer until exponential time has passed! רחליאויברכהדןנפתלישי time etc…

23. Need new Notions of Efficiency! start time polynomial total time Running time is a polynomial function of input + output start time polynomial delay Delay between successive answers is polynomial (in the input) ⇑

24. Is the AllPaths Algorithm Efficient? Does not run in poly delay! –Exponential time before a single result is returned Does not run in poly total time! –Exponential time even if result is small All Paths(G,s,t) 1. P =  2. For every subset U of V(G)-{s,t} 3. For every ordering O of U 4. If O is a path from s to t 5. then add O to P 6. Sort P by increasing length 7. Return P

25. Are all three problems equally hard? 1) All paths in increasing length order 2) All perfect matchings in decreasing weight order 3) All maximal cliques in decreasing size order No! Problems 1 and 2 can be solved in poly delay! Problem 3 can be solved in poly total time, but not in poly delay (unless P = NP)

26. Overview Interesting problems with large output Complexity A generic approach for achieving polynomial delay: Lawler-Murty Other approaches Conclusions

27. Often, quite simple (not always!) Bottom Line: Lawler-Murty gives a general reduction: Printing all answers in ranked order Finding best answer under constraints Find top answer under inclusion & exclusion constraints if poly. time then poly. delay [Murty, 1968] [Lawler, 1972] [Murty, 1968] [Lawler, 1972]

28. Problem Formulation O = A collection of objects A = score() 2131282717 score( a ) is high  a is a high-quality answer Huge, implicitly described by a constraint over O ’s subsets… Goal:Enumerate all the answers in ranked order Goal: Enumerate all the answers in ranked order … 32 31 28 start Answers a ⊆ O (that is, by decreasing score) input Polynomial delay Required complexity: Polynomial delay Top-k Answers Special case:

29. Example 1: All Paths O = A = … Edges of the graph G Graph G Nodes s, t Graph G Nodes s, t Paths: Sets of edges defining a path from s to t score( a ): ∑ e ∈ ae ∈ a weight(e) 1

30. Example 2: Best Perfect Matchings O = Edges of the graph G Weighted, bipartite graph G Matchings: Sets of edges—pairwise-disjoint & cover all nodes score( a ): A = … ∑ e ∈ ae ∈ a weight(e)

31. What’s the Problem? O = 32 start 1 st (top) answer “Best answer” problem Assumption: Efficiently solvable 31 2 nd answer ?... 17 k th answer ≠ previous (k-1) answers best among remaining answers Conceivably, much more complicated than finding 1 st ? How to handle this constraint? Moreover, k may be very large!

32. Lawler-Murty’s Method A start [K. G. Murty, 1968] [E. L. Lawler, 1972]

33. 1. Find & Print the Top Answer A start In principle, at this point we should find the second-best answer But Instead…

34. 2. Partition the Remaining Answers A start simple constraints Each partition is defined by a distinct set of simple constraints

35. 3. Find the Top of each Set A start

36. 4. Find & Print the Second Answer A start Best among all the top answers in the partitions Next answer: Best among all the top answers in the partitions

37. 5. Further Divide the Chosen Partition A start … and so on …...

38. A Partition is Defined by Constraints Two types of constraints: Inclusion constraint: “Must contain ” Exclusion constraint: “Must not contain ” A partition is defined by a set I ∪ E of inclusion and exclusion constraints Recall: Partition I ∪ E a How to further partition after removing a ? next 

39. EI Partitioning a Partition a = top(partition) ✗✗✗✗✗ ✓✓✓ Partition I ∪ E - { a }

40. EI Partitioning a Partition a = top(partition) ✗✗✗✗✗ ✓✓✓ Partition I ∪ E - { a } P 1 =( I 1, E 1 ) E I ✗✗✗✗✗ ✓✓✓ ✗

41. EI Partitioning a Partition a = top(partition) ✗✗✗✗✗ ✓✓✓ Partition I ∪ E - { a } P 1 =( I 1, E 1 ) E I ✗✗✗✗✗ ✓✓✓ ✗ P 2 =( I 2, E 2 ) E I ✗✗✗✗✗ ✓✓✓✓ ✗

42. EI Partitioning a Partition a = top(partition) ✗✗✗✗✗ ✓✓✓ Partition I ∪ E - { a } P 1 =( I 1, E 1 ) E I ✗✗✗✗✗ ✓✓✓ ✗ P 2 =( I 2, E 2 ) E I ✗✗✗✗✗ ✓✓✓✓ ✗ P 3 =( I 3, E 3 ) E I ✗✗✗✗✗ ✓✓✓✓ ✗ ✓

43. EI Partitioning a Partition a = top(partition) ✗✗✗✗✗ ✓✓✓ Partition I ∪ E - { a } P 1 =( I 1, E 1 ) E I ✗✗✗✗✗ ✓✓✓ ✗ P 2 =( I 2, E 2 ) E I ✗✗✗✗✗ ✓✓✓✓ ✗ P 4 =( I 4, E 4 ) E I ✗✗✗✗✗ ✓✓✓✓✓✓✓ P 3 =( I 3, E 3 ) E I ✗✗✗✗✗ ✓✓✓✓ ✗ ✓

44. EI Partitioning a Partition a = top(partition) ✗✗✗✗✗ ✓✓✓ Partition I ∪ E - { a } P 1 =( I 1, E 1 ) E I ✗✗✗✗✗ ✓✓✓ ✗ P 2 =( I 2, E 2 ) E I ✗✗✗✗✗ ✓✓✓✓ ✗ P 4 =( I 4, E 4 ) E I ✗✗✗✗✗ ✓✓✓✓✓✓✓ P 3 =( I 3, E 3 ) E I ✗✗✗✗✗ ✓✓✓✓ ✗ ✓ P 5 =( I 5, E 5 ) EI ✗✗✗✗✗ ✓✓✓✓✓✓✓ ✗

45. Complementary Details A partitioned is represented as a triple ( I, E, a ) –I and E are sets of inclusion and exclusion constraints, resp. (lists of objects); a is the top answer of the partition Current triples ( I, E, a ) are stored in a priority queue Q, prioritized by score( a ) –Initially, Q = { ( ∅,∅, a opt ) } In each iteration, 1. 1. The top triple ( I t, E t, a t ) is extracted from Q 2. 2. The answer a t is printed 3. 3. The new nonempty sub-partitions ( I i, E i, a i ) are inserted into Q … until Q is empty –Top-k: until k answers have been printed

46. Often, quite simple (not always!) Printing all  Finding best In the bottom line, Lawler-Murty gives a reduction: Printing all answers in ranked order Find best answer under constraints Find top answer under inclusion & exclusion constraints if poly. time then poly. delay Example: Perfect matchings by decreasing weight 

47. Perfect Matchings by Dec. Weight edges have weights (not specified) Top matching: A perfect matching, such that the total sum of edge weights is maximal Efficiently solvable: Hungarian Algorithm (1955), Blossom Algorithm (1965), …

48. Max. Perfect Matching w/ Constrains ✗ ✓ ✓ edges have weights (not specified)

49. Handling Exclusion Constraints ✗ ✓ ✓ edges have weights (not specified) Excluded edges are simply removed!

50. Handling Inclusion Constraints ✓ ✓ edges have weights (not specified) Non-inclusion edges incident to nodes of inclusion edges are removed

51. That’s All! edges have weights (not specified) It is now the original problem (w/o constraints)! So, we can use Hungarian Algorithm, Blossom Algorithm, …  Perfect matchings by decreasing weight  Perfect matchings by decreasing weight

52. Overview Interesting problems with large output Complexity A generic approach for achieving polynomial delay: Lawler-Murty Other approaches Conclusions

53. Often, quite simple (not always!) General Approach 2 Hereditary Properties algorithm gives a reduction: Printing all Maximal Answers Input Restricted Enumeration if poly. time then poly. delay if poly. total time then poly. total time [C, Kimelfeld, Sagiv 2008]

54. Hereditary Properties P is polynomially verifiable if we can check in polynomial time whether a set a satisfies P P is hereditary if a satisfies Pa’ satisfies P,  a’  a  Example: Clique We can verify in polynomial time if a graph is a clique If G is a clique, every subgraph of G is a clique Example: Clique We can verify in polynomial time if a graph is a clique If G is a clique, every subgraph of G is a clique

55. Often, quite simple (not always!) General Approach 3 Technique Performs Reduction: Printing all Answers Decision of Non-emptiness under constraints if poly. time then poly. delay

56. Overview Interesting problems with large output Complexity A generic approach for achieving polynomial delay: Lawler-Murty Other approaches Conclusions

57. Conclusions In real life, finding optimal answers to problems is often not sufficient –better to return all answers in ranked order Different complexity classes are needed to analyze algorithms that return large output Generic algorithms are available that can reduce the problem of returning all answers to simpler problems