1. CS5234 Combinatorial and Graph Algorithms Welcome!

2. CS5234 Overview  Combinatorial & Graph Algorithms http://www.comp.nus.edu.sg/~cs5234/2013/  Instructor: Seth Gilbert Office: COM2-323 Office hours: by appointment

3. Algorithms for Combinatorial Optimization

4. What is an algorithm? –Set of instructions for solving a problem “First, wash the tomatoes.” “Second, peel and cut the carrots.” “Third, mix the olive oil and vinegar.” “Finally, combine everything in a bowl.” –Finite sequence of steps –Unambiguous –English, Chinese, pseudocode, Java, etc. Algorithms

5. “If you need your software to run twice as fast, hire better programmers. But if you need your software to run more than twice as fast, use a better algorithm.” -- Software Lead at Microsoft

6. Goals: Algorithmic 1. Design (problem solving) 2. Analysis (rigorous, deep understanding) 3. Implementation (able to put it to use)

8. Algorithms for Combinatorial Optimization

9. Optimization: Find the minimum/maximum of… Combinatorial Optimization

10. Optimization: Find the minimum/maximum of: Continuous optimization: a function f Discrete optimization: a collection of items Combinatorial Optimization

11. Discrete Optimization: Find the “best” item in a large set of items. Combinatorial Optimization: Find the “best” item in a large set of items generated by a combinatorial process. Combinatorial: finite discrete structure Combinatorial process: counting, combining, enumerating Combinatorial Optimization

12. Discrete Optimization: Find the “best” item in a large set of items. Combinatorial Optimization: Find the “best” item in a large set of items generated by a combinatorial process. Graphs Matroids Similar structures… Combinatorial Optimization

13. Examples: Find the “best” item in a large set of items. Searching a listMaximum flow Shortest pathsMulticommodity flow Minimum spanning treeGraph partitioning Steiner treeFacility location Travelling salesmanQuadratic assignment Scheduling / planningKnapsack problem MatchingMaximum clique Combinatorial Optimization

14. Find the “best” item in a large set of items: ProblemSet of itemsSize Difficulty Searching List of integersLinearEasy Shortest pathsAll paths in a graphExponentialEasy Minimum spanning treeAll spanning treesExponentialEasy Steiner treeAll steiner treesExponentialHard Travelling salesmanAll possible toursExponentialHard MatchingAll possible matchingsExponentialEasy Bipartite vertex coverAll possible coversExponentialEasy Vertex coverAll possible coversExponentialHard Maximum cliqueAll possible subsetsExponentialVery Hard Combinatorial Optimization

15. Find the “best” item in a large set of items: ProblemDifficulty Maintain student recordsEasy Data compressionEasy Program halting problemImpossible VLSI chip layoutHard Exam timetable schedulingHard Job assignment problemEasy Computer deadlock problemEasy Finding patterns in a databaseEasy Combinatorial Optimization

16. Operations Research: How to make better decisions (e.g., maximize profit) Project planning / critical path analysis Facility location: where to open stores / plants Floorplanning: layout of factory or computer chips Supply chain management Berth assignment problem (BAP): port management Assignment problems (e.g., weapon target assignment) Routing / transportation problems: buses, subways, trucking. Airline ticket pricing Combinatorial Optimization

17. What do we do when problems are NP-hard? 1.Find exponential time solutions 2.Average performance 3.Approximate –Algorithm is efficient –Solution is sub-optimal –Provable guarantee: ratio of output to optimal Combinatorial Optimization

18. Five Representative Problems Combinatorial Optimization

19. Input: Set of jobs with start and finish times Output:Maximum cardinality subset of compatible jobs 1. Interval Scheduling Time 01234567891011 f g h e a b c d h e b Jobs don’t overlap

20. Input: Set of weighted jobs with start and finish times Output:Maximum weight subset of compatible jobs 2. Weighted Interval Scheduling Time 01234567891011 20 11 16 13 23 12 20 26

21. Input: Bipartite graph Output:Maximum cardinality matching 3. Bipartite Matching C 1 5 2 A E 3 B D4

22. Input: Graph Output: Maximum cardinality independent set 4. Independent set 6 2 5 1 7 3 4 6 5 1 4 subset of nodes such that no two joined by an edge

23. Input: Graph with weighted nodes Game: Two players alternate in selecting nodes. Cannot select a node if any of its neighbors have been selected. Goal: Select a maximum weight subset of nodes. 5. Competitive facility location 10 15155 151 10 Second player can get 20, but not 25.

24. Variations on a theme: Independent Set Interval scheduling: Greedy O(n log n) Weighted Interval scheduling: Dynamic programming O(n log n) Bipartite matching: O(n k ) max-flow algorithm Independent set: NP-complete Competitive facility location: PSPACE-complete Five problems

25. General combinatorial problem –Given a finite, discrete set S of objects –Minimize/maximize some function f (S) Algorithmic Issues… –Representation of the set S –Efficient manipulation of the set S –Efficient algorithm to compute f (S) Combinatorial Optimization

26. Given a problem P, – Can it be solved? If “Yes”, give an algorithm A for solving P, – Is algorithm A correct ? – How good is algorithm A ? – Can we find a better algorithm A’ ? How do we define good? – How much time it takes. – How much space it uses. Strategy Computability Verification Efficiency Time Complexity Space Complexity

27. Goals of this course: –Advanced design and analysis of algorithms: Efficiency –Time: How long does it take? –Space: How much memory? How much disk? –Others: Energy, power, heat, parallelism, etc. Scalability –Inputs are large : e.g., the internet. –Bigger problems consume more resources. –Solve important (fun!) problems in combinatorial optimization Algorithms for Combinatorial Optimization

28. Target students: –Beginning graduate students –Advanced (4 th year) undergraduates –Anyone planning to do research in algorithmic design Prerequisites: –CS3230 (Analysis of Algorithms), or equivalent –Strong programming skills Algorithms for Combinatorial Optimization

29. You must already know these: Data Structures (with analyses) –Stacks, Queues, Lists, –Binary search trees, balanced trees, –Heaps and priority queues Algorithm Design Paradigms (with Analysis) –Standard sorting and searching algorithms –Graph algorithms: DFS, BFS, –Shortest Path Algorithms, MST Algorithms –Greedy Algorithms, Divide-and-Conquer Algorithms for Combinatorial Optimization

30. You must already know these: Analysis of Algorithms –Expertise with Big-O, ,  notations –Summation of series, Master Theorem –Competent with Algorithmic Analysis: Quicksort, Heapsort, Divide-and-Conquer algorithms DFS, BFS, Shortest Path & MST algorithms Algorithms for Combinatorial Optimization

31. Assumed knowledge: –Data structures and algorithms –Good programming skills –CS3230 (Analysis of Algorithms) Not a course to learn algorithms: –If not, take CS3230 instead. Algorithms for Combinatorial Optimization

32.  Mid-term exam October 8 In class  Final exam November 19 Reading Week Exams will be graded and returned. CS5234 Overview

33.  Grading 40% Problem sets 25% Mid-term exam 35% Final exam  Problem sets –6-7 sets (about every 1-2 weeks) –A few will have programming components (C++). CS5234 Overview

34.  Problem sets released by tomorrow morning PS0: Covers background knowledge. Do not submit. PS1: Routine problems --- easy practice. Do not submit. Standard problems --- to be submitted. Advanced problems --- for a challenge/fun. Do not submit. CS5234 Overview

35.  Problem set grading Distributed grading scheme: Each of you will be responsible for grading one week during the semester. Grading supervised (and verified) by the TA. CS5234 Overview

36.  What to submit: Concise and precise answers: Solutions should be rigorous, containing all necessary detail, but no more. Algorithm descriptions consist of: 1. Summary of results/claims. 2. Description of algorithm in English. 3. Pseudocode, if helpful. 4. Worked example of algorithm. 5. Diagram / picture. 6. Proof of correctness and performance analysis. CS5234 Overview

37.  Policy on plagiarism: Do your work yourself: Your submission should be unique, unlike anything else submitted, on the web, etc. Discuss with other students: 1. Discuss general approach and techniques. 2. Do not take notes. 3. Spend 30 minutes on facebook (or equiv.). 4. Write up solution on your own. 5. List all collaborators. Do not search for solutions on the web: Use web to learn techniques and to review material from class. CS5234 Overview

38.  Policy on plagiarism: Penalized severely: First offense: minimum of one letter grade lost on final grade for class (or referral to SoC disciplinary committee). Second offense: F for the class and/or referral to SoC. Do not copy/compare solutions! CS5234 Overview

39. Introduction to Algorithms –Cormen, Leiserson, Rivest, Stein –Recommended… Textbooks

40. Algorithm Design –Kleinberg and Tardos –Recommended… Textbooks

41.  Topics (tentative, TBD) Introduction to combinatorial optimization Vertex cover, set cover, Steiner tree, TSP Flows and matching Maximum flow, bipartite matching Graph partitioning Heuristics, spectral bisection Linear programming LPs, duality, relaxations, rounding CS5234 Overview