1. CS4234 Optimization Algorithms Welcome!

2. CS4234 Overview  Optimization Algorithms http://www.comp.nus.edu.sg/~gilbert/CS4234  Instructor: Seth Gilbert Office: COM2-323 Office hours: by appointment

3. Optimization: Find the minimum/maximum… Optimization Algorithms

4. Optimization: Find the minimum/maximum: Discrete: a collection of items Optimization Algorithms

5. Optimization: Find the minimum/maximum: Discrete: a collection of items Combinatorial: a collection of items generated by counting, combining, and enumerating. Optimization Algorithms

6. Optimization: Find the minimum/maximum: Discrete: a collection of items Combinatorial: a collection of items generated by counting, combining, and enumerating. Examples: Graphs Matroids Similar structures… Optimization Algorithms

7. Find the “best” item in a large set of items: ProblemSet of itemsSize Difficulty Searching List of integersLinearEasy Shortest pathsAll paths in a graphExponentialEasy Minimum spanning treeAll spanning treesExponentialEasy Steiner treeAll steiner treesExponentialHard Travelling salesmanAll possible toursExponentialHard MatchingAll possible matchingsExponentialEasy Bipartite vertex coverAll possible coversExponentialEasy Vertex coverAll possible coversExponentialHard Maximum cliqueAll possible subsetsExponentialVery Hard Combinatorial Optimization

8. Find the “best” item in a large set of items: ProblemDifficulty Maintain student recordsEasy Data compressionEasy Program halting problemImpossible VLSI chip layoutHard Exam timetable schedulingHard Job assignment problemEasy Computer deadlock problemEasy Finding patterns in a databaseEasy Combinatorial Optimization

9. Operations Research: How to make better decisions (e.g., maximize profit) Project planning / critical path analysis Facility location: where to open stores / plants Floorplanning: layout of factory or computer chips Supply chain management Berth assignment problem (BAP): port management Assignment problems (e.g., weapon target assignment) Routing / transportation problems: buses, subways, trucking. Airline ticket pricing Combinatorial Optimization

10. Optimization: Find the minimum/maximum: Discrete: a collection of items Combinatorial: a collection of items generated by counting, combining, and enumerating. Continuous: given a function f(x), find the vector x that maximizes f(x). Optimization Algorithms

12. Goals: Algorithmic 1. Design (problem solving) 2. Analysis (rigorous, deep understanding) 3. Implementation (able to put it to use)

14. “If you need your software to run twice as fast, hire better programmers. But if you need your software to run more than twice as fast, use a better algorithm.” -- Software Lead at Microsoft

15. “... pleasure has probably been the main goal all along. But I hesitate to admit it, because computer scientists want to maintain their image as hard- working individuals who deserve high salaries... ” -- D. E. Knuth

16. “... pleasure has probably been the main goal all along. But I hesitate to admit it, because computer scientists want to maintain their image as hard- working individuals who deserve high salaries... ” -- D. E. Knuth

17. “ Computer programming is an art, because it applies accumulated knowledge to the world, because it requires skill and ingenuity, and especially because it produces objects of beauty. A programmer who subconsciously views himself as an artist will enjoy what he does and will do it better.” -- D. E. Knuth

18. Brand new class: –We can make this class what we want. –Talk to me about your goals, interests, etc. CS4234 : Optimization Algorithms

19. Brand new class: –We can make this class what we want. –Talk to me about your goals, interests, etc. Where did this class come from? –CS5234: Combinatorial and Graph Algorithms –More general: all sorts of optimization (not only graphs). –More specific: just optimization CS4234 : Optimization Algorithms

20. Target students: –Advanced (3 rd or 4 th year) undergraduates –Interested in algorithms –Interested in tools for solving hard problems Prerequisites: –CS3230 (Analysis of Algorithms) –Mathematical fundamentals Optimization Algorithms

21. You must already know these: Data Structures (with analyses) –Stacks, Queues, Lists, –Binary search trees, balanced trees, –Heaps and priority queues Algorithm Design Paradigms (with Analysis) –Standard sorting and searching algorithms –Graph algorithms: DFS, BFS, –Shortest Path Algorithms, MST Algorithms –Greedy Algorithms, Divide-and-Conquer Combinatorial and Graph Algorithms

22. You must already know these: Analysis of Algorithms –Expertise with Big-O, ,  notations –Summation of series, Master Theorem –Competent with Algorithmic Analysis: Quicksort, Heapsort, Divide-and-Conquer algorithms DFS, BFS, Shortest Path & MST algorithms Combinatorial and Graph Algorithms

23.  Mid-term exam October 6 In class  Final exam November 25 Exams will be graded and returned. CS4234 Overview

24.  Grading 40% Problem sets 25% Mid-term exam 35% Final exam  Problem sets –5-6 sets (about every 1-2 weeks) –Focused on algorithm design and analysis. CS4234 Overview

25.  Mini-Project Small project Idea: put together some of the different ideas we have used in the class. Time scale: last 2-3 weeks of the semester. CS4234 Overview

26.  Released today/tomorrow Survey: On IVLE. What is your background? Not more than 10 minutes. PS1: Released tomorrow. CS4234 Overview

27.  Problem set grading Simple scheme: 3 : excellent, perfect answer 2 : satisfactory, mostly right 1 : many mistakes / poorly written 0 : mostly wrong / not handed in -1 : utter nonsense CS4234 Overview

28.  What to submit: Concise and precise answers: Solutions should be rigorous, containing all necessary detail, but no more. Algorithm descriptions consist of: 1. Summary of results/claims. 2. Description of algorithm in English. 3. Pseudocode, if helpful. 4. Worked example of algorithm. 5. Diagram / picture. 6. Proof of correctness and performance analysis. CS4234 Overview

29.  How to draw pictures? By hand: Either submit hardcopy, or scan, or take a picture with your phone! Or use a tablet / iPad… Digitally: 1. xfig (ugh) 2. OmniGraffle (mac) 3. Powerpoint (hmmm) 4. ??? CS4234 Overview

30.  Policy on plagiarism: Do your work yourself: Your submission should be unique, unlike anything else submitted, on the web, etc. Discuss with other students: 1. Discuss general approach and techniques. 2. Do not take notes. 3. Spend 30 minutes on facebook (or equiv.). 4. Write up solution on your own. 5. List all collaborators. Do not search for solutions on the web: Use web to learn techniques and to review material from class. CS4234 Overview

31.  Policy on plagiarism: Penalized severely: First offense: minimum of one letter grade lost on final grade for class (or referral to SoC disciplinary committee). Second offense: F for the class and/or referral to SoC. Do not copy/compare solutions! CS4234 Overview

32. Introduction to Algorithms –Cormen, Leiserson, Rivest, Stein –Recommended… Textbooks

33. Algorithm Design –Kleinberg and Tardos –Recommended… Textbooks

34.  Topics (tentative, TBD) Introduction to combinatorial optimization Vertex cover, set cover, Steiner tree, TSP Flows and matching Maximum flow, bipartite matching Linear programming LPs, duality, relaxations, rounding Continuous optimization Gradient descent, meta-heuristics CS4234 Overview

35. What do we do when problems are NP-hard? 1.Find exponential time solutions 2.Average performance 3.Approximate –Algorithm is efficient –Solution is sub-optimal –Provable guarantee: ratio of output to optimal Combinatorial Optimization