Analysis of Algorithms CS 477/677

Slides:



Advertisements
Similar presentations
Analysis of Algorithms CS 477/677
Advertisements

Lecture 12: Revision Lecture Dr John Levine Algorithms and Complexity March 27th 2006.
S. J. Shyu Chap. 1 Introduction 1 The Design and Analysis of Algorithms Chapter 1 Introduction S. J. Shyu.
11 Computer Algorithms Lecture 6 Recurrence Ch. 4 (till Master Theorem) Some of these slides are courtesy of D. Plaisted et al, UNC and M. Nicolescu, UNR.
I Advanced Algorithms Analysis. What is Algorithm?  A computer algorithm is a detailed step-by-step method for solving a problem by using a computer.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 5.
Jan Welcome to the Course of Advanced Algorithm Design (ACS-7101/3)
CS421 - Course Information Website Syllabus Schedule The Book:
DAST, Spring © L. Joskowicz 1 Data Structures – LECTURE 1 Introduction Motivation: algorithms and abstract data types Easy problems, hard problems.
Analysis of Algorithms CS 477/677 Midterm Exam Review Instructor: George Bebis.
Analysis of Algorithms CS 477/677
Analysis of Algorithms CS 477/677 Recurrences Instructor: George Bebis (Appendix A, Chapter 4)
DAST, Spring © L. Joskowicz 1 Data Structures – LECTURE 1 Introduction Motivation: algorithms and abstract data types Easy problems, hard problems.
Instructor: Dr. Sahar Shabanah Fall Lectures ST, 9:30 pm-11:00 pm Text book: M. T. Goodrich and R. Tamassia, “Data Structures and Algorithms in.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 15.
Algorithms: Design and Analysis Summer School 2013 at VIASM: Random Structures and Algorithms Lecture 3: Greedy algorithms Phan Th ị Hà D ươ ng 1.
Week 2 CS 361: Advanced Data Structures and Algorithms
Computer Algorithms Lecture 1 Introduction
Mathematics Review and Asymptotic Notation
Algorithm Analysis An algorithm is a clearly specified set of simple instructions to be followed to solve a problem. Three questions for algorithm analysis.
Design and Analysis of Algorithms Introduction. Class Policy Grading –Homeworks and quizzes (20%) Programming assignments –First and Second exams (20%
Analysis of Algorithms
Fundamentals of Algorithms MCS - 2 Lecture # 7
Analysis of Algorithm Efficiency Dr. Yingwu Zhu p5-11, p16-29, p43-53, p93-96.
ECOE 456/556: Algorithms and Computational Complexity Lecture 1 Serdar Taşıran.
Lecture 1: Introduction and Overview CSCI 700 – Algorithms 1.
Algorithms  Al-Khwarizmi, arab mathematician, 8 th century  Wrote a book: al-kitab… from which the word Algebra comes  Oldest algorithm: Euclidian algorithm.
10/20/20151 CS 3343: Analysis of Algorithms Review for final.
CMPT 438 Algorithms. Why Study Algorithms? Necessary in any computer programming problem ▫Improve algorithm efficiency: run faster, process more data,
Analysis of Algorithms CS 477/677
COSC 3101A - Design and Analysis of Algorithms 14 Review.
A Lecture /24/2015 COSC3101A: Design and Analysis of Algorithms Tianying Ji Lecture 1.
Major objective of this course is: Design and analysis of modern algorithms Different variants Accuracy Efficiency Comparing efficiencies Motivation thinking.
1Computer Sciences Department. Book: Introduction to Algorithms, by: Thomas H. Cormen Charles E. Leiserson Ronald L. Rivest Clifford Stein Electronic:
INTRODUCTION. What is an algorithm? What is a Problem?
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 3.
COSC 3101A - Design and Analysis of Algorithms 2 Asymptotic Notations Continued Proof of Correctness: Loop Invariant Designing Algorithms: Divide and Conquer.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 14.
Midterm Review 1. Midterm Exam Thursday, October 15 in classroom 75 minutes Exam structure: –TRUE/FALSE questions –short questions on the topics discussed.
COSC 3101A - Design and Analysis of Algorithms 6 Lower Bounds for Sorting Counting / Radix / Bucket Sort Many of these slides are taken from Monica Nicolescu,
Design and Analysis of Algorithms (09 Credits / 5 hours per week) Sixth Semester: Computer Science & Engineering M.B.Chandak
CSE 340: Review (at last!) Measuring The Complexity Complexity is a function of the size of the input O() Ω() Θ() Complexity Analysis “same order” Order.
Lecture # 1 Introduction Analysis of Algorithm by Qamar Abbas Analysis of Algorithms.
Algorithms Design and Analysis CS Course description / Algorithms Design and Analysis Course name and Number: Algorithms designs and analysis –
BITS Pilani Pilani Campus Data Structure and Algorithms Design Dr. Maheswari Karthikeyan Lecture1.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 1.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 17.
Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 2.
CMPT 438 Algorithms.
Design and Analysis of Algorithms
Analysis of Algorithms
Analysis of Algorithms CS 477/677
CS 3343: Analysis of Algorithms
CS 3343: Analysis of Algorithms
CS 3343: Analysis of Algorithms
CS 3343: Analysis of Algorithms
CS 3343: Analysis of Algorithms
CS 3343: Analysis of Algorithms
CS 3343: Analysis of Algorithms
Advance Analysis of Lecture No 9 Institute of Southern Punjab Multan
CS 3343: Analysis of Algorithms
CMPT 438 Algorithms Instructor: Tina Tian.
CS 201 Fundamental Structures of Computer Science
CS 3343: Analysis of Algorithms
Algorithm Analysis, Asymptotic notations CISC4080 CIS, Fordham Univ.
Algorithms Recurrences.
INTRODUCTION TO ALOGORITHM DESIGN STRATEGIES
Department of Computer Science & Engineering
Analysis of Algorithms CS 477/677
COMP 122 – Design and Analysis of Algorithms
Presentation transcript:

Analysis of Algorithms CS 477/677 Instructor: Monica Nicolescu Lecture 1

General Information Instructor: Dr. Monica Nicolescu E-mail: monica@cse.unr.edu Office hours: Wednesday: 10am-noon, 1-2pm Office: SEM 239 Teaching assistant: TBA Class webpage: http://www.cse.unr.edu/~monica/Courses/CS477-677/index.html CS 477/677 - Lecture 1

Class Policy Grading Late homework 9-10 homeworks (40%) Extra-credit Programming component Mid-term exam (25%) Closed books, closed notes Final exam (30%) Attendance and class participation (5%) Late homework 10% penalty for each day of delay, up to 3 days Introduction to Algorithms, Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein CS 477/677 - Lecture 1

Homework Submission Typed covered page: your name the section you are in: CS477 or CS677 homework number Handwriting is good for the rest of the homework, but write legibly Staple pages together Homework due at the beginning of the class, late after that CS 477/677 - Lecture 1

Do not do this! Academic Dishonesty “Cheating, plagiarism or otherwise obtaining grades under false pretenses” constitute academic dishonesty according to the code of this university. Academic dishonesty will not be tolerated and penalties can include canceling a student’s enrollment without a grade, giving an F for the course or for the assignment. For more details, see the University of Nevada, Reno General Catalog. Do not do this! CS 477/677 - Lecture 1

Why Study Algorithms? Necessary in any computer programming problem Improve algorithm efficiency: run faster, process more data, do something that would otherwise be impossible Solve problems of significantly large size Technology only improves things by a constant factor Compare algorithms Algorithms as a field of study Learn about a standard set of algorithms New discoveries arise Numerous application areas Learn techniques of algorithm design and analysis CS 477/677 - Lecture 1

Applications Multimedia Internet Communication Computers Science CD player, DVD, MP3, JPG, DivX, HDTV Internet Packet routing, data retrieval (Google) Communication Cell-phones, e-commerce Computers Circuit layout, file systems Science Human genome Transportation Airline crew scheduling, UPS deliveries CS 477/677 - Lecture 1

Roadmap Different problems Different design paradigms Sorting Searching String processing Graph problems Geometric problems Numerical problems Different design paradigms Divide-and-conquer Incremental Dynamic programming Greedy algorithms Randomized/probabilistic CS 477/677 - Lecture 1

Analyzing Algorithms Predict the amount of resources required: memory: how much space is needed? computational time: how fast the algorithm runs? FACT: running time grows with the size of the input Input size (number of elements in the input) Size of an array, polynomial degree, # of elements in a matrix, # of bits in the binary representation of the input, vertices and edges in a graph Def: Running time = the number of primitive operations (steps) executed before termination Arithmetic operations (+, -, *), data movement, control, decision making (if, while), comparison CS 477/677 - Lecture 1

Algorithm Efficiency vs. Speed E.g.: sorting n numbers Friend’s computer = 109 instructions/second Friend’s algorithm = 2n2 instructions Your computer = 107 instructions/second Your algorithm = 50nlgn instructions Your friend = You = Sort 106 numbers! 20 times better!! CS 477/677 - Lecture 1

Algorithm Analysis: Example Alg.: MIN (a[1], …, a[n]) m ← a[1]; for i ← 2 to n if a[i] < m then m ← a[i]; Running time: the number of primitive operations (steps) executed before termination T(n) =1 [first step] + (n) [for loop] + (n-1) [if condition] + (n-1) [the assignment in then] = 3n - 1 Order (rate) of growth: The leading term of the formula Expresses the asymptotic behavior of the algorithm CS 477/677 - Lecture 1

Typical Running Time Functions 1 (constant running time): Instructions are executed once or a few times logN (logarithmic) A big problem is solved by cutting the original problem in smaller sizes, by a constant fraction at each step N (linear) A small amount of processing is done on each input element N logN A problem is solved by dividing it into smaller problems, solving them independently and combining the solution CS 477/677 - Lecture 1

Typical Running Time Functions N2 (quadratic) Typical for algorithms that process all pairs of data items (double nested loops) N3 (cubic) Processing of triples of data (triple nested loops) NK (polynomial) 2N (exponential) Few exponential algorithms are appropriate for practical use CS 477/677 - Lecture 1

Why Faster Algorithms? CS 477/677 - Lecture 1

Asymptotic Notations A way to describe behavior of functions in the limit Abstracts away low-order terms and constant factors How we indicate running times of algorithms Describe the running time of an algorithm as n grows to  O notation:  notation:  notation: asymptotic “less than and equal”: f(n) “≤” g(n) asymptotic “greater than and equal”:f(n) “≥” g(n) asymptotic “equality”: f(n) “=” g(n) CS 477/677 - Lecture 1

Asymptotic Notations - Examples n2/2 – n/2 (6n3 + 1)lgn/(n + 1) n vs. n2  notation n3 vs. n2 n vs. logn = (n2) = (n2lgn) n ≠ (n2) O notation 2n2 vs. n3 n2 vs. n2 n3 vs. nlogn n3 = (n2) 2n2 = O(n3) n = (logn) n2 = O(n2) n  (n2) n3  O(nlgn) CS 477/677 - Lecture 1

Mathematical Induction Used to prove a sequence of statements (S(1), S(2), … S(n)) indexed by positive integers. S(n): Proof: Basis step: prove that the statement is true for n = 1 Inductive step: assume that S(n) is true and prove that S(n+1) is true for all n ≥ 1 The key to proving mathematical induction is to find case n “within” case n+1 CS 477/677 - Lecture 1

Recursive Algorithms Binary search: for an ordered array A, finds if x is in the array A[lo…hi] Alg.: BINARY-SEARCH (A, lo, hi, x) if (lo > hi) return FALSE mid  (lo+hi)/2 if x = A[mid] return TRUE if ( x < A[mid] ) BINARY-SEARCH (A, lo, mid-1, x) if ( x > A[mid] ) BINARY-SEARCH (A, mid+1, hi, x) 12 11 10 9 7 5 3 2 1 4 6 8 mid lo hi CS 477/677 - Lecture 1

Recurrences Def.: Recurrence = an equation or inequality that describes a function in terms of its value on smaller inputs, and one or more base cases E.g.: T(n) = T(n-1) + n Useful for analyzing recurrent algorithms Methods for solving recurrences Iteration method Substitution method Recursion tree method Master method CS 477/677 - Lecture 1

Sorting – Analysis of Running Time Iterative methods: Insertion sort Bubble sort Selection sort 2, 3, 4, 5, 6, 7, 8, 9, 10, J, Q, K, A Divide and conquer Merge sort Quicksort Non-comparison methods Counting sort Radix sort Bucket sort CS 477/677 - Lecture 1

Types of Analysis Worst case (e.g. cards reversely ordered) Best case Provides an upper bound on running time An absolute guarantee that the algorithm would not run longer, no matter what the inputs are Best case Input is the one for which the algorithm runs the fastest Average case Provides a prediction about the running time Assumes that the input is random (e.g. cards reversely ordered) (e.g., cards already ordered) (general case) CS 477/677 - Lecture 1

Specialized Data Structures Left child Right child L R parent key data Problem: Keeping track of customer account information at a bank or flight reservations This applications requires fast search, insert/delete, sort Solution: binary search trees If y is in left subtree of x, then key [y] ≤ key [x] If y is in right subtree of x, then key [y] ≥ key [x] Red-black trees, interval trees, OS-trees 3 2 4 6 7 13 15 18 17 20 9 CS 477/677 - Lecture 1

Dynamic Programming An algorithm design technique (like divide and conquer) Richard Bellman, optimizing decision processes Applicable to problems with overlapping subproblems E.g.: Fibonacci numbers: Recurrence: F(n) = F(n-1) + F(n-2) Boundary conditions: F(1) = 0, F(2) = 1 Compute: F(5) = 3, F(3) = 1, F(4) = 2 Solution: store the solutions to subproblems in a table Applications: Assembly line scheduling, matrix chain multiplication, longest common sequence of two strings, 0-1 Knapsack problem CS 477/677 - Lecture 1

Greedy Algorithms Problem Schedule the largest possible set of non-overlapping activities for SEM 234 Start End Activity 1 8:00am 9:15am Numerical methods class 2 8:30am 10:30am Movie presentation (refreshments served) 3 9:20am 11:00am Data structures class 4 10:00am noon Programming club mtg. (Pizza provided) 5 11:30am 1:00pm Computer graphics class 6 1:05pm 2:15pm Analysis of algorithms class 7 2:30pm 3:00pm Computer security class 8 4:00pm Computer games contest (refreshments served) 9 5:30pm Operating systems class       CS 477/677 - Lecture 1

Greedy Algorithms Similar to dynamic programming, but simpler approach Also used for optimization problems Idea: When we have a choice to make, make the one that looks best right now Make a locally optimal choice in hope of getting a globally optimal solution Greedy algorithms don’t always yield an optimal solution Applications: Activity selection, fractional knapsack, Huffman codes CS 477/677 - Lecture 1

Graphs Applications that involve not only a set of items, but also the connections between them Maps Schedules Computer networks Hypertext Circuits CS 477/677 - Lecture 1

Searching in Graphs Graph searching = systematically follow the edges of the graph so as to visit the vertices of the graph Two basic graph methods: Breadth-first search Depth-first search The difference between them is in the order in which they explore the unvisited edges of the graph Graph algorithms are typically elaborations of the basic graph-searching algorithms u v w x y z CS 477/677 - Lecture 1

Strongly Connected Components Read in a 2D image and find regions of pixels that have the same color Original Labeled CS 477/677 - Lecture 1

Minimum Spanning Trees A connected, undirected graph: Vertices = houses, Edges = roads A weight w(u, v) on each edge (u, v)  E Find T  E such that: T connects all vertices w(T) = Σ(u,v)T w(u, v) is minimized a b c d e h g f i 4 8 7 11 1 2 14 9 10 6 Algorithms: Kruskal and Prim CS 477/677 - Lecture 1

Shortest Path Problems Input: Directed graph G = (V, E) Weight function w : E → R Weight of path p = v0, v1, . . . , vk Shortest-path weight from u to v: δ(u, v) = min w(p) : u v if there exists a path from u to v ∞ otherwise 6 3 4 2 1 2 7 5 3 6 p CS 477/677 - Lecture 1

Variants of Shortest Paths Single-source shortest path (Bellman-Ford, DAG shortest paths, Disjkstra) G = (V, E)  find a shortest path from a given source vertex s to each vertex v  V Single-destination shortest path Find a shortest path to a given destination vertex t from each vertex v Reverse the direction of each edge  single-source Single-pair shortest path Find a shortest path from u to v for given vertices u and v Solve the single-source problem All-pairs shortest-paths (Matrix multiplication, Floyd-Warshall) Find a shortest path from u to v for every pair of vertices u and v CS 477/677 - Lecture 1

Number Theoretic Algorithms Secured communication: RSA public-key cryptosystem Easy to find large primes Hard to factor the product of large primes Secure Message eavesdropper Authenticated Message Bob Alice CS 477/677 - Lecture 1

NP-Completeness Not all problems can be solved in polynomial time Some problems cannot be solved by any computer no matter how much time is provided (Turing’s Halting problem) – such problems are called undecidable Some problems can be solved but not in O(nk) Can we tell if a problem can be solved ? NP, NP-complete, NP-hard Approximation algorithms CS 477/677 - Lecture 1

Readings Chapter 1 Appendix A CS 477/677 - Lecture 1