ANALYSIS OF ALGORITHMS Dr inż. ZDZISŁAW PÓLKOWSKI Polkowice, 2015 University of Pitesti Dolnośląska Wyższa Szkoła Przedsiębiorczości i Techniki w Polkowicach Ion Paul Mihai

 What is Algorithm ?  What is Algorithm Design ?  How to Design an Algorithm ?  Graph Algorithm  Flow Chart  Importance of Algorithm Design  Conclusion TOPICS MENU

An algorithm (pronounced AL-go-rith-um) is a procedure or formula for solving a problem. The word derives from the name of the mathematician, Mohammed ibn-Musa al-Khwarizmi, who was part of the royal court in Baghdad and who lived from about 780 to 850. Al-Khwarizmi's work is the likely source for the word algebra as well. A computer program can be viewed as an elaborate algorithm. In mathematics and computer science, an algorithm usually means a small procedure that solves a recurrent problem. What is algorithm ?

What is algorithm-design ? Algorithm design is a creative activity that is not subject to recipes. The existence of many important problems for which no efficient algorithms are known is an evidence of this fact. In practice, the space of choices to develop algorithms is enormous. For this reason, and in opposition to the development from scratch, the task of solving algorithmic problems can be broached applying more abstract design techniques. These techniques offer common solving strategies for different problems. A design technique is often expressed in pseudocode as a template that can be particularized for concrete problems. Let us name this template algorithm schemas.

How to design algorithm ?? There are four FAMOUS techniques of algorithm design: GGreedy Algorithm DDivide and Conquer DDynamic Programming BBack Tracking How to design algorithm ?

Greedy Algorithm A greedy algorithm is a mathematical process that looks for simple, easy- to-implement solutions to complex, multi-step problems by deciding which next step will provide the most obvious benefit. Such algorithms are called greedy because while the optimal solution to each smaller instance will provide an immediate output, the algorithm doesn’t consider the larger problem as a whole. Once a decision has been made, it is never reconsidered. Greedy algorithms work by recursively constructing a set of objects from the smallest possible constituent parts. Recursion is an approach to problem solving in which the solution to a particular problem depends on solutions to smaller instances of the same problem. The advantage to using a greedy algorithm is that solutions to smaller instances of the problem can be straightforward and easy to understand. The disadvantage is that it is entirely possible that the most optimal short- term solutions may lead to the worst possible long-term outcome.

 Image of greedy algorithm

 Divide and Conquer Divide and Conquer Algorithm Divide-and-conquer is a top-down technique for designing algorithms that consists of dividing the problem into smaller subproblems hoping that the solutions of the subproblems are easier to find and then composing the partial solutions into the solution of the original problem. Little more formally, divide-and-conquer paradigm consists of following major phases: -Breaking the problem into several sub-problems that are similar to the original problem but smaller in size, -Solve the sub-problem recursively (successively and independently), -Combine these solutions to subproblems to create a solution to the original problem. Binary Search (simplest application of divide-and-conquer) Binary Search is an extremely well-known instance of divide-and-conquer paradigm. Given an ordered array of n elements, the basic idea of binary search is that for a given element we "probe" the middle element of the array. We continue in either the lower or upper segment of the array, depending on the outcome of the probe until we reached the required (given) element.

 Image of Divide and Conquer

 Dynamic Programming Example: Fibonacci numbers computed by iteration. Warshall ’s algorithm implemented by iterations. Dynamic Programming Algorithm Dynamic programming is a fancy name for using divide-and-conquer technique with a table. As compared to divide-and-conquer, dynamic programming is more powerful and subtle design technique. Let me repeat, it is not a specific algorithm, but it is a meta-technique (like divide-and- conquer). This technique was developed back in the days when "programming" meant "tabular method" (like linear programming). It does not really refer to computer programming. Here in our advanced algorithm course, we'll also think of "programming" as a "tableau method" and certainly not writing code. Dynamic programming is a stage-wise search method suitable for optimization problems whose solutions may be viewed as the result of a sequence of decisions.

 Images of Dynamic Programming Fibonacci_dynamic_programming.svg.png

Example: Eight queens puzzle. Traveling salesman problem. Back Tracking Algorithm Backtracking is a general algorithmic technique that considers searching every possible combination in order to solve an optimization problem. Backtracking is also known as depth-first search or branch and bound. By inserting more knowledge of the problem, the search tree can be pruned to avoid considering cases that don't look promising. While backtracking is useful for hard problems to which we do not know more efficient solutions, it is a poor solution for the everyday problems that other techniques are much better at solving.However, dynamic programming and greedy algorithms can be thought of as optimizations to backtracking, so the general technique behind backtracking is useful for understanding these more advanced concepts. Learning and understanding backtracking techniques first provides a good stepping stone to these more advanced techniques because you won't have to learn several new concepts all at once.

Image of back tracking Backtracking-no-backjumping.svg.png

 Graph Algorithm Graph Algorithm What is a graph? It's an abstract notion, used to represent the idea of some kind of connection between pairs of objects. A graph consists of: -A collection of "vertices", which I'll usually draw as small circles on the blackboard, -A collection of "edges", each connecting some two vertices. I'll usually draw these as curves on the blackboard connecting the given pair of vertices. For this definition it doesn't matter what the vertices or edges represent -- that will be different depending on what application the graph comes from. It also doesn't matter how I draw the graph, the only part that matters is which pairs of vertices are connected with each other. There are two different types of graph that we'll commonly see.

-In the other type of graph, known as a directed graph, an edge goes from one of the vertices, towards the other. In this case I'll draw an arrowhead at the vertex towards which the edge is going. If we drew a graph like the handshake graph described above, but instead connected two people if one had written a letter to the other, the result would be directed: if I've written a letter to you, you may not have written a letter back to me.In one type, known as an undirected graph it doesn't matter which end of the graph is which -- the relation between the two is symmetric. In that case I'll just draw the edges as an undecorated curve. -For instance, suppose we draw a graph with one vertex for every person in the U.S., and one edge connecting any two people who have shaken hands with each other. If I've shaken hands with you, you've shaken hands with me, so each edge is symmetric. It seems to be true that graphs like this have very small {\em diameter}: you can connect any two people with a very short chain of handshakes. -In the other type of graph, known as a directed graph, an edge goes from one of the vertices, towards the other. In this case I'll draw an arrowhead at the vertex towards which the edge is going.

 Image of Graph Algorithm

 Flow chart A flowchart is a type of diagram that represents an algorithm, workflow or process, showing the steps as boxes of various kinds, and their order by connecting them with arrows. Flowcharts are used in analyzing, designing, documenting or managing a process or program in various fields. Flow chart

Symbols of Flow Chart

It is used to store and access large quantities of data efficiently. It is used to solve complex computational problems and to design of good programs It is important to justify an algorithm correctness mathematically It provides clear, simple and unambiguous description Importance of algorithm design

Conclusion Usually a given problem can be solved using various approaches however it is not wise to settle for the first that comes to mind. More often than not, some approaches result in much more efficient solutions than others. Consider again the Fibonacci numbers computed recursively using the decrease-and-conquer approach, and computed by iterations using dynamic programming. In the first case the complexity is O(2n), while in the second case the complexity is O(n). On the other hand, consider sorting based on decrease-and-conquer (insertion sort) and brute force sorting. For almost sorted files insertion sort will give almost linear complexity, while brute force sorting algorithms have quadratic complexity. The basic question here is: How to choose the approach? First, by understanding the problem, and second, by knowing various problems and how they are solved using different approaches.