Introduction to Computer Algorithms

Introduction to Computer Algorithms
Lecture 9: Introduction to Computer Algorithms

What is an Algorithm? An algorithm is a finite sequence of precise instructions for performing a computation or for solving a problem. While there is no generally accepted formal definition of "algorithm," an informal definition could be "a set of rules that precisely defines a sequence of operations.“ For some people, a program is only an algorithm if it stops eventually; for others, a program is only an algorithm if it stops before a given number of calculation steps. A prototypical example of an algorithm is Euclid’s Algorithms to determine the maximum common divisor of two integers as depicted in the flow chart.

Data Structures To understand some algorithms, you need to be familiar with the design and use of basic data structures such as Lists, Stacks, Queues, Trees, and Graphs. List top Stack Queue back front Tree root leaf nodes child node parent Graph edge vertex cycle path

What you Need to Know and Do
You should be able to understand and implement basic methods for searching and sorting. You should be able to determine the worst-case order of complexity (number of operations) required to solve a problem of size N using the best-known algorithm. Suggest an algorithm and give a worst-case order of complexity for each of the following: Search an unordered list of n items to find a particular item x. Search an ordered list of n items to find a particular item x. Find the smallest value in an unordered list of n items. Find the largest value in an ordered list of n items. Determine if an unordered list contains a repeated value. Find the kth largest value in an unordered list of n items (k<=n). Find the kth largest value in an ordered list of n items (k<=n). Sort an unordered list of n items into ascending order (sort by key comparison). Find a pair of items x and y in an unordered list that are the most similar. Find a pair of items x and y in an ordered list that are the most similar. Find a pair of items in an ordered list that are the least similar. Arrange a list so that the sum of the abs. differences between adj. items is a minimum. Arrange a list so that the sum of the abs. differences between adj. items is a maximum. Find the two pairs of points (in R2 ) whose separations are the most similar.

A Common Sort Routine a procedural approach ReadList(filename, n, S)
for i = 1 to n - 1 for j = i + 1 to n if (S( i ) > S( j )) then exchange(S( i ), S( j )) end if end loop WriteList(filename,S) // reads a list of n values from the text file, filename // into the list S. n and S are passed back to the calling routine // if statement implements the key comparison // exchange swaps the position of two values in the list S // writes the sorted list, S back to the hard drive as filename

Recursion Hides most of the Details
function recurse(n) deal with n // there is usually something to do with the incoming value for each f(n) = m if(m is OK) // if no m is OK then this call has reached its termination condition recurse(m) deal with n // sometimes the incoming value is processed after the recursion It is important to understand that each recursive call interrupts the for-each loop until the call has returned Certain values of m may be OK at beginning of for-each loop but may become not OK by the time the loop reaches them Regardless of the number of valid recursive calls in any for-each loop, if each value of N can be accessed only once, the overall runtime complexity will be O(N). Recursion goes bad (exponential run-time complexity) when each value of problem of size N can be in more than one recursive call.

The Hazards of Recursion
There's a hole in the bucket. Plug the hole with the cork. Bring water from the well. The cork is too big. There is no water. Trim the cork with the knife. Wet the stone with water. The knife is too dull. The sharpening stone is too dry. Sharpen the Knife.

For Example... function foob(n) if n>0 return foob(n-1) + foob(n-1)
function feeb(n) if n>0 return feeb(n-1)

Sort Tree Traversals A B C D E F G H I
All traversals are in the same order The difference is when the node is evaluated (i.e. passed to the output) Sort Trees are normally implemented using dynamic memory and pointers Can you think of a practical alternative? A B D E C F H I G pre-order eval node left-child right-child in-order post-order left-child eval node right-child left-child right-child eval node D B E A H F I C G D E B H I F G C A

Binary Tree Embedded in an Indexed List
1 (1) parent = child / 2 (2) left_child = 2 x parent (3) right_child = 2 x parent + 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Graphs

Types of Graphs ring complete graph bipartite graph
directed acyclic graph

Graph Representations
node list edge list A D F C H B E G A - B C D E F B - A C E H C - A B D E H D - A C F G H E - A B C F G F - A D E G H G - D E F H H - B C D F G A B A C A D A E A F B A B C B E B H C A C B C D C E C H D A D C D F D G D H E A E B E C E F E G F A F D F E F G F H G D G E G F G H H B H C H D H F H G node list - lists the nodes connected to each node edge list - lists each of the edges as a pair of nodes undirected edges may be listed twice XY and YX in order to simplify algorithm implementation adjacency matrix - for an n-node graph we build an nxn array with 1's indicating edges and 0's no edge the main diagonal of the matrix is unused unless a node has an edge connected to itself. If graph is weighted, 1's are replaced with edge weight values adjacency matrix A B C D E F G H A B C D E F G H

Graph Breadth-First Traversal
Given a graph G(V,E) and a starting vertex s, perform a breadth-first traversal (BFT) of G. such that each reachable vertex is entered exactly once. If all vertices are reachable, the edges traversed and the set of vertices will represent a spanning tree embedded in the graph G. 1) BFT suggests an iterative process (rather than a recursive one) 2) BFT vertices order of traversal can be maintained using a Queue data structure 3) The preferred representation for the graph is an adjacency matrix 4) We will need a way to keep up with which vertices have been "used" (e.g. a Boolean list) 5) Process begins by placing the starting vertex in the Queue 6) A vertex is taken from the Queue, every unused vertex adj to this vertex is added to the Queue This operation is repeated until the Queue is empty. 8) The output (answer) is returned in the form of a list of vertices in the order they entered the Queue A D F C H B E G

Graph Depth-First Traversal
Given a graph G(V,E) and a starting vertex s, perform a depth-first traversal (BFT) of G. such that each reachable vertex is entered exactly once. If all vertices are reachable, the edges traversed and the set of vertices will represent a spanning tree embedded in the graph G. 1) DFT suggests a recursive process (rather than an iterative one) 2) DFT vertices order of traversal are maintained automatically by the recursion process (as a Stack) 3) The preferred representation for the graph is an adjacency matrix. 4) We will need a way to keep up with which vertices have been "used" (e.g. a Boolean list) 5) Process begins by passing the starting vertex to a recursive function DFT(s) 6) For the current vertex, s DFT(s) calls itself for each adjacent, unused vertex remaining. This operation is completed when all calls to DFT( ) are completed. 8) The output is returned as a of a list of vertices in the order they were passed to DFT( ). A D F C H B E G

Graph Depth-First Traversal (DFT) Algorithm Implementation
text file representation Given a graph G(V,E) and a starting node vstart perform a depth-first traversal. Provide a list of the nodes in the order they are traversed. Graph is defined in a text file of the following format: 8 A B C D E F G H 1st Line - # of nodes, N # of edges, E 2nd Line - List of node labels (you may assume 1 char each) 3rd Line - through N+3 Line an N x N adjacency matrix A D F C H B E G

Graph Depth-First Traversal (DFT) Algorithm Implementation
pseudo-code DFT( node vk ) { // deal with current node for each node, vi if (node_avail(vi)) then DFT(vi) } node_avail( vi ) is a Boolean function that returns true if node vi is available for traversal, which means that vi has not been traversed, and vi is adjacent to vk.

Balanced vs Unbalanced Trees
from Algorithms in a Nutshell Chapter 1 the Gary and Graham story

Generating the Fibonacci Sequence
The Exponential Algorithm

An Improvement on the Exponential Algorithm
f Dynamic Programming Algorithm 1 2 3 4 5 n 1 1 2 3 5 :

Big-"O" Notation

Estimating Order of Complexity using Big "O"

Sample Problems

Coin Changing An optimal solution to the coin changing problem is the minimum number of coins whose total value equals a specified amount. For example what is the minimum number of coins (current U.S. mint) needed to total 83 cents. 25 25 A Greedy Algorithm for Coin Changing 1. Set remval=initial_value 2. Choose largest coin that is less than remval. 3. Add coin to set of coins and set remval:=revamal-coin_value 4. repeat Steps 2 and 3 until remval = 0; = 58 = 33 = 8 = 3 = 2 = 1 = 0 25 25 10 5 5 1 1 1 1

Coin Changing - Another Example
The greedy method works for the U.S. minted coin set, but will it work for any coin set? Consider the coin set below and a change value of 82 cents.. 26 = 56 = 30 = 4 = 3 = 2 = 1 = 0 26 26 26 26 26 10 10 1 1 5 1 10 10 1 1 By the coin-changing algorithm we obtain 3 26 cent pieces and 4 pennies, but we can use 2 26 cent pieces and 3 dimes to total 82 cents. Hence the greedy approach does not work for this coin set.

Prim's Algorithm Given a weighted graph G consisting of a set of vertices V and a set of edges E with weights, where Prepare a vertex set and an edge set to hold elements selected by Prim's Algorithm. C D F E A G B 4 2 3 5 1 1. Choose an arbitrary starting vertex vj 2. Find the smallest edge e incident with a vertex in the vertex set whose inclusion in the edge set does not create a cycle. 3. Include this edge in the edge list and its vertices in the vertex list. 4. Repeat Steps 2 and 3 until all vertices are in the vertex list. 2

QuickSort

QuickSort Algorithm private static void quickSort(int[] list) { quickSort(list,0,list.length-1); } private static void quickSort(int[] list, int first, int last) { if(last>first) { int pivotIndex = partition(list,first,last); quickSort(list,first,pivotIndex-1); quickSort(list,pivotIndex+1,last); } } 4 6 2 5 3 7 8 4 3 1 6 9 2 4 6 5 3 7 8 4 3 1 6 9 2 3 4 3 1 4 6 5 7 8 6 9

Partition Algorithm the first element is the pivot seaching forward
private static int partition(int[] list, int first, int last) { int pivot = list[first]; int low = first +1; int high = last; while (high > low) { while(low <= high && list[low] <= pivot) low++; while(low <= high && list[high]> pivot) high--; if(high>low) { int tmp = list[high]; list[high] = list[low]; list[low] = tmp; } } while(high>first && list[high]>=pivot) high--; if(pivot > list[high]) { list[first] = list[high]; list[high] = pivot; return high; } else return first; } } the first element is the pivot seaching forward searching backward swap two element in the list swap pivot with list[high]

Heap Sort In order to understand Heap Sort we must first have an understanding of what a heap is and how they are constructed. 9 6 5 4 8 2 3 7 1 A heap is a binary tree in which the parent node holds a value at least as large as its child nodes.

Data Representation An efficient algorithm for converting a binary tree into a heap takes advantage of the list data representation of a tree. parent_index = child_index/2 left_child = parent_index * 2 right_child = parent_index*2 + 1 9 6 5 4 8 2 3 7 1 9 6 8 5 4 2 3 7 1 10 11 12 13 14 15 index list

Half the Nodes are Leaf Nodes
It is important to note that in a complete binary tree, half the nodes are leaf nodes (no children). 9 8 7 4 6 5 3 2 complete & full binary tree

Half the Nodes are Leaf Nodes
It is important to note that in a complete binary tree, half the nodes are leaf nodes (no children). 9 8 7 4 6 5 3 complete binary tree

Heapify Algorithm When we convert a generic binary tree into a heap we say that we heapify the binary tree. The heapify algorithm is defined by the following steps, starting with the leaf nodes and working toward the root node. Step 1 - Given the parent node index, get the left-child and the right-child indices. Step 2 - If there is a left-child node, compare it with the parent node and save the largest. Step 3 - If there is a right-child node, compare it with the value from Step 2 and save the largest. Step 4 - If parent node is not the largest, exchange it with the largest node. Repeat for each node in the tree. 7 6 8 5 3 4 9 2

Heapify Algorithm When we convert a generic binary tree into a heap we say that we heapify the binary tree. The heapify algorithm is defined by the following steps, starting with the leaf nodes and working toward the root node. Step 1 - Given the parent node index, get the left-child and the right-child indices. Step 2 - If there is a left-child node, compare it with the parent node and save the largest. Step 3 - If there is a right-child node, compare it with the value from Step 2 and save the largest. Step 4 - If parent node is not the largest, exchange it with the largest node. Repeat for each node in the tree. 9 8 7 4 6 5 3 2 IT'S A HEAP!

Heapify Implemented as a Recursive Function
When heapify is applied to a particular node in a binary tree, the function operates only on that node and the path to the leaf nodes affected by the successive applications of the exchange( ) function. 7 6 8 9 4 5 3 2

7 8 6 9 4 5 3 2

7 8 9 4 5 6 3 2

7 8 9 4 5 6 3 2 nodes involved

Building a Heap Starting at the midpoint of the list (the first node with children) we heapify the tree node-by-node up to the root node. Building a heap the the first step in HeapSort. must be the largest value in the heap... Why? 9 6 5 4 8 2 3 7 1 The heap property: The value of the parent node is at least as large as the values of its children. Converting the tree into a heap does not sort the values in the list. The heap property does guarantee that the root node of the heap is the largest value in the heap. 9 6 8 5 4 2 3 7 1 10 11 12 13 14 15 index list

HeapSort We start with a heap...
Then we exchange the root node value with the value in the farthest leaf node... Now the value in the root node may violate the heap property so we heapify the list shortened by one.... That is, we leave the old root node in place at the end of the list since it is the largest value in the list. This process is repeated n-1 times, each pass leaving the largest value in the remainder of the list in order at the end of the list.

A Partial Example: HeapSort

alphabetical version of Twenty Questions
Player One - Picks a word. Player Two - Asks a series of YES/NO questions, attempting to guess the word in <20 tries. alphabetical version of Twenty Questions

Backtracking Technique
Backtracking is used to solve problems in which a feasible solution is needed rather than an optimal one, such as the solution to a maze or an arrangement of squares in the 15-puzzle. Backtracking problems are typically a sequence of items (or objects) chosen from a set of alternatives that satisfy some criterion. 1 2 3 4 6 10 14 15 5 7 8 9 6 11 12 13 10 14 15

Backtracking Implementation
Backtracking is a modified depth-first search of the solution-space tree. In the case of the maze the start location is the root of a tree, that branches at each point in the maze where there is a choice of direction.

N-Queens Problem The problem of placing N queens on an NxN chessboard in such a way that no two of them are "attacking" each other, is a classic problem used to demonstrate the backtracking method. A simple brute-force method would be to try placing the first queens on the first square, followed by the second queen on the first available square, scanning the chessboard in a row-column manner. A more efficient backtracking approach is to note that each queen must be in its own column and row. This reduces the search from (N2)! to N!.

Hamiltonian Circuits Problem
A Hamiltonian circuit or tour of a graph is a path that starts at a given vertex, visits each vertex in the graph exactly once, and ends at the starting vertex. Some graphs do not contain Hamiltonian circuits. v1 v2 v6 v4 v5 v3 v1 v2 v6 v4 v5 v3 A state space tree for this problem is as follows. Put the starting vertex at level 0 in the tree, call this the zero'th vertex on the path. At level 1, consider each vertex other than the starting vertex as the first vertex after the starting one. At level 2, consider each of these vertices as the second vertex, and so on. You may now backtrack in this state space tree.

Backtracking in a State Space Tree
function ok(i)return boolean j:index isok:boolean begin if i=n-1 and not W(v(n-1),v(0)) then isok:=false elsif i>0 and not W(v(n-1),v(i)) then else isok:=true; j:=1; while j<i and isok loop if v(i)=v(j) then isok:=false; j:=j+1; end loop; end if; end ok; procedure hamiltonian(i:index) j : index; begin if ok(i) then if i=n-1 then display(v(0..n-1)) else for j in 2..n loop v(i+1):=j; hamiltonian(i+1); end loop; end if; end hamiltonian; 1. The ith vertex on the path must be adjacent to the (i-1)st vertex on the path. 2. The (n-1)st vertex must be adjacent to the 0'th vertex. 3. The ith vertex cannot be one of the i-1 vertices.

Traveling Salesperson Problem
a Branch-and-Bound Algorithm The initial tour is A-F-E-D-B-C-A with a total length of 32. The sum of the minimum costs of leaving every node is 29 so we know that a minimal tour cannot have length less than 29. A B C D E F

The live-node list is ordered by bounding value and then by level of completion. That is, if two nodes have the same bounding value we place the one that is closer to completion ahead of the other node. This is because the bounding value of the of the node closer to completion is more likely to be closer to the actual tour length.

The node shown in red were expanded from an active node (F3) with a possible tour length of 29. However the completed tours turned out to be significantly larger.

Expanding D3 we find an actual tour that is shorter than our initial tour. We do not yet know if this is a minimal tour but we can remove nodes from our live-node list that have a bounding value of >= 31. Now we expand D2 to find that it leads to tours of length >= 33.

Expansion of E2 eliminates this path as a candidate for a minimal tour.

When the live-node list is empty we are finished
When the live-node list is empty we are finished. In this example a minimal tour is A-C-E-D-B-F-A with a length of 31 we have found a tour that is shorter than the original candidate tour that was generated by the greedy method.

Summary "Stuff" You Should Already Know (SYSAK)...
Important Data Structures - lists, stacks, queues, trees, and graphs 13 Problems to Consider Reading Data from Text Files Searching and Sorting Methods for Growth-Rate Analysis Recursion (good and bad) Manipulating Lists and Matrices Dealing with Trees Working with Graphs

Introduction to Computer Algorithms

Similar presentations

Presentation on theme: "Introduction to Computer Algorithms"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Introduction to Computer Algorithms

Similar presentations

Presentation on theme: "Introduction to Computer Algorithms"— Presentation transcript:

Similar presentations

About project

Feedback