Lecture 11 Algorithm Design

Slides:



Advertisements
Similar presentations
Towers of Hanoi Move n (4) disks from pole A to pole C such that a disk is never put on a smaller disk A BC ABC.
Advertisements

Recursion Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein.
Recursion. Recursion is a powerful technique for thinking about a process It can be used to simulate a loop, or for many other kinds of applications In.
Recursion CSC 220: Data Structure Winter Introduction A programming technique in which a function calls itself. One of the most effective techniques.
Lesson 19 Recursion CS1 -- John Cole1. Recursion 1. (n) The act of cursing again. 2. see recursion 3. The concept of functions which can call themselves.
Scott Grissom, copyright 2004 Chapter 5 Slide 1 Analysis of Algorithms (Ch 5) Chapter 5 focuses on: algorithm analysis searching algorithms sorting algorithms.
1 CSCD 300 Data Structures Recursion. 2 Proof by Induction Introduction only - topic will be covered in detail in CS 320 Prove: N   i = N ( N + 1.
Recursive Algorithms Nelson Padua-Perez Chau-Wen Tseng Department of Computer Science University of Maryland, College Park.
Chapter 15 Recursive Algorithms. 2 Recursion Recursion is a programming technique in which a method can call itself to solve a problem A recursive definition.
Main Index Contents 11 Main Index Contents Selection Sort Selection SortSelection Sort Selection Sort (3 slides) Selection Sort Alg. Selection Sort Alg.Selection.
COMPSCI 105 S Principles of Computer Science Recursion 3.
Recursion.  Identify recursive algorithms  Write simple recursive algorithms  Understand recursive function calling  With reference to the call stack.
JAVA: An Introduction to Problem Solving & Programming, 5 th Ed. By Walter Savitch and Frank Carrano. ISBN © 2008 Pearson Education, Inc., Upper.
1 Recursion Algorithm Analysis Standard Algorithms Chapter 7.
Recursion l Powerful Tool l Useful in simplifying a problem (hides details of a problem) l The ability of a function to call itself l A recursive call.
Chapter 11Java: an Introduction to Computer Science & Programming - Walter Savitch 1 Chapter 11 l Basics of Recursion l Programming with Recursion Recursion.
CHAPTER 02 Recursion Compiled by: Dr. Mohammad Omar Alhawarat.
Analysis of Algorithms
Lecturer: Dr. AJ Bieszczad Chapter 11 COMP 150: Introduction to Object-Oriented Programming 11-1 l Basics of Recursion l Programming with Recursion Recursion.
1 CS 177 Week 16 Recitation Recursion. 2 Objective To understand and be able to program recursively by breaking down a problem into sub problems and joining.
C++ Programming: From Problem Analysis to Program Design, Second Edition Chapter 19: Searching and Sorting.
CSC 221: Recursion. Recursion: Definition Function that solves a problem by relying on itself to compute the correct solution for a smaller version of.
Data Structures R e c u r s i o n. Recursive Thinking Recursion is a problem-solving approach that can be used to generate simple solutions to certain.
By: Lokman Chan Recursive Algorithm Recursion Definition: A function that is define in terms of itself. Goal: Reduce the solution to.
©TheMcGraw-Hill Companies, Inc. Permission required for reproduction or display. Chapter 15 * Recursive Algorithms.
Chapter 11Java: an Introduction to Computer Science & Programming - Walter Savitch 1 Chapter 11 l Basics of Recursion l Programming with Recursion Recursion.
JAVA: An Introduction to Problem Solving & Programming, 7 th Ed. By Walter Savitch ISBN © 2015 Pearson Education, Inc., Upper Saddle River,
CMPT 120 Topic: Searching – Part 2 and Intro to Time Complexity (Algorithm Analysis)
Python Programming, 3/e1 Python Programming: An Introduction to Computer Science Chapter 13 Algorithm Design and Recursion.
JAVA: An Introduction to Problem Solving & Programming, 6 th Ed. By Walter Savitch ISBN © 2012 Pearson Education, Inc., Upper Saddle River,
Recursion Powerful Tool
Algorithm Analysis 1.
Recursion.
Algorithms.
Using recursion for Searching and Sorting
Chapter 13 Algorithm Design and Recursion
Chapter 19: Recursion.
Chapter 15 Recursion.
Recursion A problem solving technique where an algorithm is defined in terms of itself A recursive method is a method that calls itself A recursive algorithm.
Algorithmic complexity: Speed of algorithms
COP 3503 FALL 2012 Shayan Javed Lecture 15
Towers of Hanoi Move n (4) disks from pole A to pole C
Chapter 15 Recursion.
Analysis of Algorithms
CS1010 Programming Methodology
Algorithm Analysis CSE 2011 Winter September 2018.
Teach A level Computing: Algorithms and Data Structures
Recursion "To understand recursion, one must first understand recursion." -Stephen Hawking.
Applied Algorithms (Lecture 17) Recursion Fall-23
CS 3343: Analysis of Algorithms
Algorithm Design and Recursion
Algorithm design and Analysis
Algorithm An algorithm is a finite set of steps required to solve a problem. An algorithm must have following properties: Input: An algorithm must have.
Python Programming: An Introduction to Computer Science
賦得古原草送別 ~白居易 離離原上草,一歲一枯榮。 野火燒不盡,春風吹又生。 遠芳侵古道,晴翠接荒城。 又送王孫去,萋萋滿別情。
Winter 2018 CISC101 12/2/2018 CISC101 Reminders
Programming Thinking and Method (6-7)
Searching: linear & binary
Algorithmic complexity: Speed of algorithms
Recursion Chapter 11.
Basics of Recursion Programming with Recursion
Recursion Chapter 11.
Ch. 2: Getting Started.
Algorithmic complexity: Speed of algorithms
Intro to Computer Science CS1510 Dr. Sarah Diesburg
賦得古原草送別 ~白居易 離離原上草,一歲一枯榮。 野火燒不盡,春風吹又生。 遠芳侵古道,晴翠接荒城。 又送王孫去,萋萋滿別情。
Recursive Algorithms 1 Building a Ruler: drawRuler()
Chapter 3 :Recursion © 2011 Pearson Addison-Wesley. All rights reserved.
Analysis of Algorithms
Presentation transcript:

Lecture 11 Algorithm Design Xiaojuan Cai(蔡小娟) cxj@sjtu.edu.cn Fall, 2015

Objective To understand basic techniques for analyzing algorithms. To understand the algorithms for linear and binary search. To understand sorting and know the algorithms for selection sort and merge sort. To understand some problems are intractable and others are unsolvable.

Roadmap Searching problem Recursion and Divide-and-Conquer Linear and binary search Recursion and Divide-and-Conquer Sorting problem Selection sort and merge sort Intractability and unsolvability

Search problem Searching is a basic, but very important problem. Specification: def search(x, nums): # nums is a list of numbers and x is a number # Returns the index in the list if x occurs # or -1 if x is not in the list.

Built-in search methods Using in: if x in nums: # do something The index method (throws an exception if x I not in the list.): >>> nums = [3, 1, 4, 2, 5] >>> nums.index(4) 2 You can implement search by using index, however, we want to know how index is implemented.

Linear search Pretend you were given a page full of randomly ordered numbers and were asked where 13 was. How would you do it? Scanning downward from the top to the bottom!

Linear search def search(x, nums): for i in range(len(nums)): if nums[i] == x: return i return -1 The Python in and index operations both implement linear searching algorithms.

Binary search If numbers are in ascending order, what would you do? Like finding some page number in a book. You can “throw away” some parts of the list after one comparison.

Binary search def search(x, nums): low = 0 high = len(nums) - 1 while low <= high: mid = (low + high)/2 item = nums[mid] if x == item: return mid elif x < item: high = mid - 1 else: low = mid + 1 return -1

Comparing algorithms Intuitively, we might expect the linear search to work better for small lists, and binary search for longer lists. But how can we be sure? Method#1: Test them! 1. n <= 10, linear search faster 2. 10 < n <= 1000, little difference 3. n > 1000, binary search faster 4. n = 100,000,000, linear 2.5s, binary .0003s

Comparing algorithms Method#1 is dependent on hardware (cpu, etc.) and software (compilers, etc.). Method#2: counting the steps! How many steps are needed to find a value in a list of size n? Linear: the steps required are linearly related to n Binary: the steps required are related to log n

Comparing algorithms Binary search is much faster. But why the built- in functions use linear search? Before carrying out binary search, you need to sort the list! How efficient can we solve sorting problem? Answer is at least nlog n steps for lists sized n.

Roadmap Searching problem Recursion and Divide-and-Conquer Linear and binary search Recursion and Divide-and-Conquer Sorting problem Selection sort and merge sort Intractability and unsolvability

Recursive version of Binary search mid = (low + high) /2 if low > high x is not in nums elsif x < nums[mid] perform binary search in nums[low]…nums[mid-1] else perform binary search in nums[mid+1]…nums[high] A description of something that refers to itself is called a recursive definition. Binary search is a kind of Divide-and-conquer algorithm.

Divide-and-Conquer Divide Step: the input is partitioned into p parts. Conquer Step: q recursive call(s) of the smaller parts. Combine Step: combine the solved parts to obtain the desired output. In binary search: Divide into two parts sized n/2, one recursive call of the smaller part.

Recursion In mathematics, recursion is frequently used: n! = n* (n-1)!, 0! = 1 fib(n) = fib(n-1) + fib(n-2), fib(1) = 1, fib(2) = 1 Two key characteristics: There are one or more base cases for which no recursion is applied. All chains of recursion eventually end up at one of the base cases.

Factorial function def fact(n): if n == 0: return 1 else: return n * fact(n-1)

Factorial calling stack

Examples String reversal Anagrams of a string Fast exponentiation reverse(“Hello“) -> “olleH” Anagrams of a string anagram(“abc”) -> [“abc”,”acb”,…,”cba”] Fast exponentiation

Binary search def recBinSearch(x, nums, low, high): if low > high: return -1 mid = (low + high)/2 item = nums[mid] if item == x: return mid elif x < item: # Look in lower half return recBinSearch(x, nums, low, mid-1) else: # Look in upper half return recBinSearch(x, nums, mid+1, high) def search(x, nums): return recBinSearch(x, nums, 0, len(nums)-1)

Recursion v.s. Iteration Anything that can be done with a loop can be done with a simple recursive function! Some problems that are simple to solve with recursion are quite difficult to solve with loops. Recursion is not always better than iterations. E.g., Fibonacci function:

Recursion v.s. Iteration For Fibonacci number, the recursion version is very inefficient:

Lessons Recursion is another tool in your problem-solving toolbox. Sometimes recursion provides more elegant solution. At other times, when both algorithms are quite similar, use the looping solution on the basis of speed. Avoid the recursive solution if it is terribly inefficient, unless you can’t come up with an iterative solution.

Roadmap Searching problem Recursion and Divide-and-Conquer Linear and binary search Recursion and Divide-and-Conquer Sorting problem Selection sort and merge sort Intractability and unsolvability

Built-in sort functions sorted(lst[, key = func, reverse = True]) Key can be anonymous functions, e.g., key = lambda a: a.getName() lst.sort([key= func, reverse = True])

Naïve sorting Selection sort Loop n-1 times: Find the smallest element, wap it to the correct position def selSort(nums): n = len(nums) for bottom in range(n-1): mp = bottom for i in range(bottom+1, n): if nums[i] < nums[mp]: mp = i nums[bottom], nums[mp] = nums[mp], nums[bottom]

Divide-and-conquer Merge sort Divide array into two halves. Recursively sort each half. Merge two halves. Assume we need to merge two sorted arrays, with M, N elements respectively. Then How many comparisons in best case? How many comparisons in worst case?

Comparing sorts http://www.sorting-algorithms.com/ Selection sort: n + (n-1) + (n-2) + (n-3) + … + 1 Merge sort: n log n

Hard Problems We have efficient algorithms for searching and sorting problems. Divide and conquer and recursion are very powerful techniques for algorithm design. Not all problems have efficient solutions!

Roadmap Searching problem Recursion and Divide-and-Conquer Linear and binary search Recursion and Divide-and-Conquer Sorting problem Selection sort and merge sort Intractability and unsolvability

Towers of Hanoi def moveTower(n, source, dest, temp): if n == 1: print "Move disk from", source, "to", dest+".“ else: moveTower(n-1, source, temp, dest) moveTower(1, source, dest, temp) moveTower(n-1, temp, dest, source) def hanoi(n): moveTower(n, "A", "C", "B")

Analysis of Hanoi To solve a puzzle of size n will require 2^n-1 steps -- an exponential time algorithm. For 64 disks, moving one a second, round the clock, would require 580 billion years to complete (The current age of the universe is estimated to be about 15 billion years). It belongs to a class of problems known as intractable problems

Even harder: The Halting Problem Write a program that looks at other programs to determine whether they have an infinite loop or not. Program Specification: Program: Halting Analyzer Inputs: A Python program file. The input for the program. Outputs: “OK” if the program will eventually stop. “FAULTY” if the program has an infinite loop.

Diagonal proof First, assume we have such program: def terminates(program, inputData): # program and inputData are both strings # Returns true if program would halt # with inputData as its input

Diagonal proof We could write the following program: def main(): lines = [] print "Type in a program (type 'done' to quit)." line = raw_input("") while line != "done": lines.append(line) testProg = string.join(lines, "\n") if terminates(testProg, testProg): while True: pass # a pass statement does nothing

Conclusion Recursion is a very important technique. Linear searching v.s. binary searching Naïve sort v.s. merge sort Good algorithms are better than super computers. Computer Science is more than programming! The most important computer for any computing professional is between their ears.