UMass Lowell Computer Science 91.404 Analysis of Algorithms Prof. Karen Daniels Spring, 2002 Lecture 1 Introduction/Overview Text: Chapters 1, 2 Thurs.

Slides:



Advertisements
Similar presentations
MATH 224 – Discrete Mathematics
Advertisements

Analysis of Algorithms
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2005 Lecture 1 (Part 2) “How to Make an Algorithm Sandwich” adapted.
Analysis & Design of Algorithms (CSCE 321)
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Midterm Review Fri. Oct 26.
Jan Welcome to the Course of Advanced Algorithm Design (ACS-7101/3)
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Lecture 1 (Part 1) Introduction/Overview Tuesday, 9/4/01.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2005 Lecture 1 (Part 1) Introduction/Overview Tuesday, 1/25/05.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2009 Lecture 1 Introduction/Overview Text: Chapters 1, 2 Th. 9/3/2009.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2003 Final Review Wed. 12/10 – Fri. 12/12.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2004 Final Review.
Text Chapters 1, 2. Sorting ä Sorting Problem: ä Input: A sequence of n numbers ä Output: A permutation (reordering) of the input sequence such that:
COMP 122 – Design and Analysis of Algorithms Spring 2004 MW 11:00-12:15, SN 014 Instructor:Jack Snoeyink TA: Nathan Fisher SN.
DAST, Spring © L. Joskowicz 1 Data Structures – LECTURE 1 Introduction Motivation: algorithms and abstract data types Easy problems, hard problems.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2006 Lecture 2 Monday, 2/6/06 Design Patterns for Optimization.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2003 Review Lecture Tuesday, 5/6/03.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2006 Text: Chapter 3 Growth of Functions.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2002 Lecture 1 (Part 1) Introduction/Overview Tuesday, 9/3/02.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2009 Final Review.
UMass Lowell Computer Science Foundations of Computer Science Prof. Karen Daniels Fall, 2009 Lecture 1 Introduction/Overview Th. 9/3/2009.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2002 Review Lecture Tuesday, 12/10/02.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Lecture 1 Introduction/Overview Wed. 9/5/01.
UMass Lowell Computer Science Advanced Algorithms Computational Geometry Prof. Karen Daniels Spring, 2007 Lecture 1 Course Introduction.
2 -1 Chapter 2 The Complexity of Algorithms and the Lower Bounds of Problems.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2009 Lecture 1 (Part 2) “How to Make an Algorithm Sandwich” adapted.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Lectures 2, 3 Chapters 1, 2 Fri. 9/7/01 – Mon. 9/10/01.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2005 Lecture 1 Introduction/Overview Text: Chapters 1, 2 Wed. 9/7/05.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2000 Final Review Wed. 12/13.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2004 Lecture 1 (Part 1) Introduction/Overview Wednesday, 9/8/04.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2007 Lecture 1 Introduction/Overview Text: Chapters 1, 2 Wed. 1/24/07.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2001 Lecture 1 Introduction/Overview Wed. 1/31/01.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2000 Lecture 1 Introduction/Overview Wed. 9/6/00.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2002 Lecture 1 (Part 1) Introduction/Overview Tuesday, 1/29/02.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2001 Final Review Mon. 5/14-Wed. 5/16.
Text Chapters 1, 2. Sorting ä Sorting Problem: ä Input: A sequence of n numbers ä Output: A permutation (reordering) of the input sequence such that:
DAST, Spring © L. Joskowicz 1 Data Structures – LECTURE 1 Introduction Motivation: algorithms and abstract data types Easy problems, hard problems.
Instructor: Dr. Sahar Shabanah Fall Lectures ST, 9:30 pm-11:00 pm Text book: M. T. Goodrich and R. Tamassia, “Data Structures and Algorithms in.
CS223 Algorithms D-Term 2013 Instructor: Mohamed Eltabakh WPI, CS Introduction Slide 1.
LeongHW, SoC, NUS (CS Combinatorial and Graph Algorithms) Page 1 About CS5234: Course Overview CS5234: Combinatorial and Graph Algorithms  Level.
CS 3343: Analysis of Algorithms
Design and Analysis of Algorithms - Chapter 21 Analysis of Algorithms b Issues: CorrectnessCorrectness Time efficiencyTime efficiency Space efficiencySpace.
Course Web Page Most information about the course (including the syllabus) will be posted on the course wiki:
10/20/20151 CS 3343: Analysis of Algorithms Review for final.
1 Introduction to Data Structures. 2 Course Name : Data Structure (CSI 221) Course Teacher : Md. Zakir Hossain Lecturer, Dept. of Computer Science Stamford.
A Lecture /24/2015 COSC3101A: Design and Analysis of Algorithms Tianying Ji Lecture 1.
December 4, Algorithms and Data Structures Lecture XV Simonas Šaltenis Aalborg University
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2009 Lecture 1 (Part 1) Introduction/Overview Tuesday, 1/27/09.
1 BIM304: Algorithm Design Time: Friday 9-12am Location: B4 Instructor: Cuneyt Akinlar Grading –2 Midterms – 20% and 30% respectively –Final – 30% –Projects.
Design and Analysis of Algorithms (09 Credits / 5 hours per week) Sixth Semester: Computer Science & Engineering M.B.Chandak
Computer Science/Ch. Algorithmic Foundation of CS 4-1 Chapter 4 Chapter 4 Algorithmic Foundation of Computer Science.
CS 150: Analysis of Algorithms. Goals for this Unit Begin a focus on data structures and algorithms Understand the nature of the performance of algorithms.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “ Introduction to the Design & Analysis of Algorithms, ” 2 nd ed., Ch. 1 Chapter.
CES 512 Theory of Software Systems B. Ravikumar (Ravi) Office: 141 Darwin Hall Course Web site:
CES 512 Theory of Software Systems B. Ravikumar (Ravi) Office: 141 Darwin Hall Course Web site:
CES 592 Theory of Software Systems B. Ravikumar (Ravi) Office: 124 Darwin Hall.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Review Lecture Tuesday, 12/11/01.
Design and Analysis of Algorithms Introduction Instructors:1. B V Kiran Mayee, 2. A Madhavi
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2008 Final Review.
BITS Pilani Pilani Campus Data Structure and Algorithms Design Dr. Maheswari Karthikeyan Lecture1.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Final Review Mon. 12/10-Wed. 12/12.
Welcome to the Course of Advanced Algorithm Design
Lecture 1 (Part 1) Introduction/Overview Tuesday, 9/9/08
Lecture 1 Introduction/Overview Text: Chapters 1, 2 Wed. 1/28/04
CS 3343: Analysis of Algorithms
CS 583 Fall 2006 Analysis of Algorithms
Definition In simple terms, an algorithm is a series of instructions to solve a problem (complete a task) We focus on Deterministic Algorithms Under the.
Foundations II: Data Structures and Algorithms
COSC 320 Advanced Data Structures and Algorithm Analysis
Lecture 2 Introduction/Overview Fri. 9/8/00
COMP 122 – Design and Analysis of Algorithms
Presentation transcript:

UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Spring, 2002 Lecture 1 Introduction/Overview Text: Chapters 1, 2 Thurs. 1/24/02

Web Page Web Page

Nature of the Course ä Core course: required for all CS majors ä Advanced undergraduate level ä Graduate students take separate course (91.503) ä No programming required ä “Pencil-and-paper” exercises ä Lectures supplemented by: ä Programs ä Real-world examples

What’s It All About? ä Algorithm: ä steps for the computer to follow to solve a problem ä well-defined computational procedure that transforms input into output ä Some of our goals: ä recognize structure of some common problems ä understand important characteristics of algorithms to solve common problems ä select appropriate algorithm to solve a problem ä tailor existing algorithms ä create new algorithms

Some Algorithm Application Areas Computer Graphics Geographic Information Systems Robotics Bioinformatics Astrophysics Medical Imaging Telecommunications Design Apply Analyze

Some Typical Problems Sorting Input: Set of items Problem: Arrange items “in order” Median finding Input: Set of numbers or keys Problem: Find item smaller than half of items and bigger than half of items SOURCE: Steve Skiena’s Algorithm Design Manual (for problem descriptions, see graphics gallery at ) (for problem descriptions, see graphics gallery at Minimum Spanning Tree Input: Graph G = (V,E) with weighted edges Problem: Find subset of E of G of minimum weight which forms a tree on V Shortest Path Input: Edge-weighted graph G, with start vertex and end vertex t Problem: Find the shortest path from to t in G

Tools of the Trade ä Algorithm Design Patterns such as: ä binary search ä divide-and-conquer ä Data Structures such as: ä trees, linked lists, hash tables, graphs ä Theoretical Computer Science principles such as: ä NP-completeness, hardness Growth of Functions Summations Recurrences Sets Probability MATH Proofs Logarithms Permutations Combinations

Tools of the Trade: (continued) Algorithm Animation

What are we measuring? ä Some Analysis Criteria: ä Scope ä The problem itself? ä A particular algorithm that solves the problem? ä “Dimension” ä Time Complexity? Space Complexity? ä Type of Bound ä Upper? Lower? Both? ä Type of Input ä Best-Case? Average-Case? Worst-Case? ä Type of Implementation ä Choice of Data Structure

Prerequisites ä Computing I (91.101) ä Computing II (91.102) ä Discrete Math I & II (92.321, ) ä Statistics for Scientists and Engineers (92.386) ä Calculus I-III ( ) Growth of Functions Summations Recurrences Sets Probability MATH Proofs Logarithms Permutations Combinations

Course Structure: 5 Parts ä Foundations ä Analyzing & Designing Algorithms, Growth of Functions, Recurrences, Probability & Randomized Algorithms ä Sorting ä Heapsort, Priority Queues, Quicksort, Sorting in Linear Time, Medians and Order Statistics ä Data Structures ä Stacks and Queues, Linked Lists, Introduction to Trees, Hash Tables, Binary Search Trees, Balancing Trees: Red- Black Trees ä Advanced Techniques ä Dynamic Programming, Greedy Algorithms ä Graph Algorithms ä DFS, BFS, Topological Sort, MST, Shortest paths

Textbook - - Required: ä Introduction to Algorithms ä by T.H. Corman, C.E. Leiserson, R.L. Rivest ä McGraw-Hill ä 2001 ä ISBN ä see course web site (MiscDocuments) for errata (1 st edition) Ordered for UML bookstore New Edition

Syllabus (current plan) - - See course web site

CS Theory Math Review Sheet The Most Relevant Parts... ä p. 1  O, ,  definitions ä Series ä Combinations ä p. 2 Recurrences & Master Method ä p. 3 ä Probability ä Factorial ä Logs ä Stirling’s approx ä p. 4 Matrices ä p. 5 Graph Theory ä p. 6 Calculus ä Product, Quotient rules ä Integration, Differentiation ä Logs ä p. 8 Finite Calculus ä p. 9 Series Math fact sheet (courtesy of Prof. Costello) is on our web site.

Important Dates ä Midterm Exam : Thursday, 3/14 ä Midterm Exam (Chapters 1-7) : Thursday, 3/14 ä Final Exam:TBA

Grading ä ä Homework35% ä ä Midterm (Chapters 1-7) 30% (open book, notes ) ä ä Final Exam35% (open book, notes )

Homework 1 Th, 1/24 Tues, 2/5 Chapters 1, 2 HW# Assigned Due Content

Text Chapters 1, 2

Sorting ä Sorting Problem: ä Input: A sequence of n numbers ä Output: A permutation (reordering) of the input sequence such that: instance ä Algorithm: ä well-defined computational procedure that transforms input into output ä steps for the computer to follow to solve a problem

Insertion Sort Animation Finding a place for item with value 5 in position 1: Finding a place for item with value 5 in position 1: Swap item in position 0 with item in position 1.

Insertion Sort Animation Positions 0 through 1 are now in non-decreasing order.

Insertion Sort Animation Finding a place for item with value 1 in position 2: Finding a place for item with value 1 in position 2: Swap item in position 1 with item in position 2.

Insertion Sort Animation Finding a place for item with value 1: Finding a place for item with value 1: Swap item in position 0 with item in position 1. Positions 0 through 2 are now in non-decreasing order.

Insertion Sort Animation Finding a place for item with value 3 in position 3: Finding a place for item with value 3 in position 3: Swap item in position 2 with item in position 3.

Insertion Sort Animation Finding a place for item with value 3: Finding a place for item with value 3: Swap item in position 1 with item in position 2.

Insertion Sort Animation Positions 0 through 3 are now in non-decreasing order.

Insertion Sort Animation Finding a place for item with value 2 in position 4: Finding a place for item with value 2 in position 4: Swap item in position 3 with item in position 4.

Insertion Sort Animation Finding a place for item with value 2: Finding a place for item with value 2: Swap item in position 2 with item in position 3.

Insertion Sort Animation Finding a place for item with value 2: Finding a place for item with value 2: Swap item in position 1 with item in position 2.

Insertion Sort Animation Positions 0 through 4 are now in non-decreasing order.

Insertion Sort Animation Finding a place for item with value 6 in position 5: Finding a place for item with value 6 in position 5: Swap item in position 4 with item in position 5.

Insertion Sort Animation Positions 0 through 5 are now in non-decreasing order.

Insertion Sort Animation Finding a place for item with value 4 in position 6: Finding a place for item with value 4 in position 6: Swap item in position 5 with item in position 6.

Insertion Sort Animation Finding a place for item with value 4: Finding a place for item with value 4: Swap item in position 4 with item in position 5.

Insertion Sort Animation Positions 0 through 6 are now in non-decreasing order.

Insertion Sort Animation Finding a place for item with value 7 in position 7: Finding a place for item with value 7 in position 7: Swap item in position 6 with item in position 7.

Insertion Sort Animation Positions 0 through 7 are now in non-decreasing order.

Insertion Sort Animation Positions 0 through 7 are now in non-decreasing order.

Insertion Sort Animation Positions 0 through 7 are now in non-decreasing order.

Asymptotic Notation courtesy of Prof. Costello O(g(n)) is a set of functions, so we often say f(n) is in O(g(n)).

Asymptotic Notation (cont.) courtesy of Prof. Costello

Asymptotic Analysis Math fact sheet (courtesy of Prof. Costello) is on our web site.

Function Order of Growth O( ) upper bound  ( ) lower bound  ( ) upper & lower bound n 1 n lg(n) n lg 2 (n) 2n2n2n2n n5n5n5n5 lg(n) lg(n)lglg(n) n2n2n2n2 know how to use asymptotic complexity notation to describe time or space complexity know how to order functions asymptotically (behavior as n becomes large)

Types of Algorithmic Input Best-Case Input: of all possible algorithm inputs of size n, it generates the “best” result for Time Complexity: “best” is smallest running time for Time Complexity: “best” is smallest running time Best-Case Input Produces Best-Case Running Time Best-Case Input Produces Best-Case Running Time provides a lower bound on the algorithm’s asymptotic running time provides a lower bound on the algorithm’s asymptotic running time (subject to any implementation assumptions) (subject to any implementation assumptions) for Space Complexity: “best” is smallest storage for Space Complexity: “best” is smallest storage Average-Case Input Worst-Case Input these are defined similarly Best-Case Time <= Average-Case Time <= Worst-Case Time

Bounding Algorithmic Time (using cases) n 1 n lg(n) n lg 2 (n) 2n2n2n2n n5n5n5n5 lg(n) lg(n)lglg(n) n2n2n2n2 T(n) =  (1) T(n) =  (2 n ) very loose bounds are not very useful! Worst-Case time of T(n) =  (2 n ) tells us that worst-case inputs cause the algorithm to take at most exponential time (i.e. exponential time is sufficient). But, can the algorithm every really take exponential time? (i.e. is exponential time necessary?) If, for arbitrary n, we find a worst-case input that forces the algorithm to use exponential time, then this tightens the lower bound on the worst-case running time. If we can force the lower and upper bounds on the worst-case time to match, then we can say that, for the worst-case running time, T(n) =  ( 2 n ) (i.e. we’ve found the minimum upper bound, so the upper bound is tight.) Using “case” we can discuss lower and/or upper bounds on: best-case running time or average-case running time or worst-case running time

Bounding Algorithmic Time (tightening bounds) n 1 n lg(n) n lg 2 (n) 2n2n2n2n n5n5n5n5 lg(n) lg(n)lglg(n) n2n2n2n2 T B (n) =  (1) T W (n) =  (2 n ) for example... 1st attempt T B (n) =  (n) 1st attempt 2nd attempt T W (n) =  (n 2 ) Here we denote best-case time by T B (n); worst-case time by T W (n) T B (n) =  (n) 2nd attempt 1st attempt T W (n) =  (n 2 ) Algorithm Bounds

Know the Difference!  (n) 1 2n2n2n2n O(n 5) worst-case bounds on problem on problem An inefficient algorithm for the problem might exist that takes this much time, but would not help us. No algorithm for the problem exists that can solve it for worst-case inputs in less than linear time. Strong Bound: A worst- case lower bound on a problem holds for every algorithm that solves the problem and abides by the problem’s assumptions. Weak Bound: A worst-case upper bound on a problem comes from just considering one algorithm. Other, less efficient algorithms that solve this problem might exist, but we don’t care about them! Both the upper and lower bounds could be loose (i.e. perhaps could be tightened later on).