© Ronaldo Menezes, Florida Tech Fundamentals of Algorithmic Problem Solving  Algorithms are not answers to problems  They are specific instructions for getting answers  This emphasis on defining precise constructive procedures is what makes computer science distinct from other many other disciplines  The design of algorithms typically has several steps  Understanding the problem  Ascertaining the capabilities of a computational device  Choosing between exact and approximate problem solving  Deciding on appropriate data structures  Algorithms + Data Structures = Programs [Wirth 76]

© Ronaldo Menezes, Florida Tech Connectivity Problem  Let's consider the following problem:  Given a sequence of pairs of integers of the form (p- q) write an algorithm that would input a sequence of pairs and tell whether a pair is redundant.  (p-q) means that p is connected to q, and vice-versa (the connection relation is transitive)  A pair (p-q) is said to be redundant iff p is connected to q via some N number of other integers, N>=0  The algorithm must therefore remember the pairs it has seen (or information related to the pairs) so that it can answer if a given pair is redundant.

© Ronaldo Menezes, Florida Tech Sample Output

© Ronaldo Menezes, Florida Tech Connectivity Applications  Networks  Integers may represent machines and our problem is to find whether two machines need to be connected  Same applies to other kids of networks such as Social Networks.  Integers may represent points in a electric network grid and we want to find out whether the pairs (wires) are sufficient to connect all points in the network  VLSI  In the design of integrated circuits integers may represent components and we want to avoid the use of redundant connection between the components.

© Ronaldo Menezes, Florida Tech Connectivity Applications  Programming Languages  Some languages have the concept of aliases. Integers may represent the variable names and the pairs represent the fact the two variable names are equivalent.  Graph Theory  Pairs represent edges and integers represent nodes. We might want to find out the degree of connectivity of the graph.

© Ronaldo Menezes, Florida Tech Large Connectivity Example (from Sedgewick’s book)

© Ronaldo Menezes, Florida Tech Expressing a Algorithm  Our first task is to think about what is required from the problem statement.  That is, we need to know what the algorithm needs to answer.  It is normally the case that the complexity of the problem is directly proportional to the complexity of the algorithm.  Although complex problems may have a very simple algorithm  In the connectivity case, one needs to answer only a binary question: given a pair of integers, are they connected already (yes/no)?  Note that the problem does not require us to demonstrate the path in the case of a positive answer.  In fact the addition of this requirement would make the problem much more complex.

© Ronaldo Menezes, Florida Tech Expressing a Algorithm  One could also ask for “less” information:  Are M connections sufficient to fully connect N objects together?  Note that we're are not asking anything about specific pairs.

© Ronaldo Menezes, Florida Tech Fundamental Operations  When designing an algorithm it is worth spending some time thinking about the fundamental operations that the algorithm needs to perform.  For the connectivity problem we could have that for every given pair  One has to find whether the pair represents a new connection between the two objects  If it does, then we need to incorporate the information about the new connection (in a data structure) so that future searches can be answered correctly  The above can be abstractly represented using a set.  The input represents elements in a set. The problem then is to:  Find whether a set contains a given items in the pair.  Replace the sets containing the two given items with their union set.

© Ronaldo Menezes, Florida Tech Self-test Exercise  Exercise 1 How can we solve the connectivity problem using the abstract operations described in the previous slide? Sketch an algorithm based on the following description:  We can use a set S to represent the connected elements  When we read a pair (p-q) we check whether they (both) are elements of the set  If they are, we don't print the pair (they are already connected).  If they are not we then make: S U {p,q}.

© Ronaldo Menezes, Florida Tech How good is good enough?  As a rule of thumb we strive for finding efficient algorithms. But there various are factors you should consider:  Very important factors:  How often are you going to need the algorithm?  What is the typical problem size you're trying to solve?  Less important factors that should also be considered:  What language are you going use to implement your algorithm?  What is the cost-benefit of an efficient algorithm?  One of the best quotes I know about design (in general) is from Antoine de Saint-Exupéry:  “A designer knows he has arrived at perfection not when there is no longer anything to add, but when there is no longer anything to take away”

© Ronaldo Menezes, Florida Tech Simplicity is Very Desirable  Independently of how good your algorithm needs to be, you should always try to fully understand the problem  A good understanding exercise is the design of a very simple algorithm that does the job.  A simple (and correct) algorithm may also be useful as the basis for testing a more sophisticated algorithm.  A simple approach (that some of you may have tried for the connectivity problem) could be:  Use a data structure to store all (needed) pairs.  For any new pair (p-q), search the data structure and try to build a path from p to q.  BTW. How good is this approach?

© Ronaldo Menezes, Florida Tech Brute Force  Brute force is a straightforward approach to solving a problem usually directly based on the problem's statement and on the definitions of the concepts involved [Levitin 2003]  It is considered a design strategy although the strategy is simply: just do something that works!  As an example consider the exponentiation problem  Compute a n for a given number a and a nonnegative integer n.  The brute force approach consists of:  Can you calculate a n using less then (n-1) multiplications?

© Ronaldo Menezes, Florida Tech Uses of Brute Force  The brute force design technique rarely yields efficient algorithms.  But it does have a few advantages:  Brute force algorithms are normally more general.  In fact it is normally difficult to point out problems that a brute- force approach cannot tackle.  Most elementary algorithmic tasks use brute force solutions.  e.g. Calculating the sum of a list of numbers; finding the max element in a list.  The solutions are normally reasonable for practical instances sizes.  A simple (correct) solution may serve as  the basis for testing more sophisticated solutions.  as a yardstick with which to judge more efficient alternatives for solving the problem – a benchmark.

© Ronaldo Menezes, Florida Tech The Brute-force Approach to The Connectivity Problem  Let’s go back to the brute force approach to the connectivity problem we mentioned earlier  It may not be considered an algorithm because:  How big does the data structure to save all pairs need to be?  No obvious solution for building the path comes to mind.

© Ronaldo Menezes, Florida Tech Union-Find Algorithm  If we want to solve the connectivity problem based on union-find operations we must:  Define a data structure to represent the set(s)  The quick-find solution (as defined by Sedgewick's) uses an array of size N where N is the number of objects in the connectivity problem. Let's called it id  The value of the id[i] object is initialized with the value i.  Define the union operation  Two objects ( p and q ) are said to belong to the same set (therefore connected) iff id[p] == id[q]  Union is defined by changing all entries in the array with value equal to id[p] to id[q]

© Ronaldo Menezes, Florida Tech View of Quick-Find id

© Ronaldo Menezes, Florida Tech Quick-find solution to the connectivity problem public class QuickF { public static void main(String[] args) { int N = Integer.parseInt(args[0]); int id[] = new int[N]; for (int i = 0; i < N ; i++) id[i] = i; for( In.init(); !In.empty(); ) { int p = In.getInt(), q = In.getInt(); int t = id[p]; if (t == id[q]) continue; for (int i = 0; i < N; i++) if (id[i] == t) id[i] = id[q]; Out.println(" " + p + " " + q); } public class QuickF { public static void main(String[] args) { int N = Integer.parseInt(args[0]); int id[] = new int[N]; for (int i = 0; i < N ; i++) id[i] = i; for( In.init(); !In.empty(); ) { int p = In.getInt(), q = In.getInt(); int t = id[p]; if (t == id[q]) continue; for (int i = 0; i < N; i++) if (id[i] == t) id[i] = id[q]; Out.println(" " + p + " " + q); }

© Ronaldo Menezes, Florida Tech How good is Quick-Find?  Property 1.1 The quick-find algorithm executes at least MN instructions, where N is the number of objects and M is the number of unions. For each union operation the internal for-loop has to traverse the entire array  This seems a reasonable algorithm but if M and N are large, the algorithm does not perform that well.  Can we do better?  The algorithm we saw earlier is a union-find algorithm where the find operation is fast but the union is slower.  Let's look at a solution that uses the same abstraction but with a different interpretation for the values stored in the array.