FURQAN MAJEED ALGORITHMS

A computer algorithm is a detailed step-by-step method for solving a problem by using a computer. An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time. More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output. Algorithm Input output What is Algorithm?

Most basic and popular algorithms are Sorting algorithms Searching algorithms Which algorithm is best? Mainly, it depends upon various factors, for example in case of sorting The number of items to be sorted The extent to which the items are already sorted Possible restrictions on the item values The kind of storage device to be used etc. Popular Algorithms, Factors of Dependence

Problem The statement of the problem specifies, in general terms, the desired input/output relationship. Algorithm The algorithm describes a specific computational procedure for achieving input/output relationship. Example One might need to sort a sequence of numbers into non-decreasing order. Algorithms Various algorithms e.g. merge sort, quick sort, heap sorts, radix sort, counting sort etc. One Problem, Many Algorithms

Brute Force Straightforward, naïve/simple approach Mostly expensive Divide-and-Conquer Divide into smaller sub-problems Iterative Improvement Improve one change at a time Decrease-and-Conquer Decrease instance size Transform-and-Conquer Modify problem first and then solve it Space and Time Tradeoffs Use more space now to save time later Important Designing Techniques

Greedy Approach Locally optimal decisions, can not change once made. Efficient Easy to implement The solution is expected to be optimal Every problem may not have greedy solution Dynamic programming Decompose into sub-problems like divide and conquer Sub-problems are dependant Record results of smaller sub-problems Re-use it for further occurrence Mostly reduces complexity exponential to polynomial Some of the Important Designing Techniques

Problem Strategy Algorithm Input Output Steps Analysis Correctness Time & Space Optimality Implementation Verification Problem Solving Process

Design assumption Level of abstraction which meets our requirements Neither more nor less e.g. [0, 1] infinite continuous interval Analysis independent of the variations in Machine Operating system Programming languages Compiler etc. Low-level details will not be considered Our model will be an abstraction of a standard generic single-processor machine, called a random access machine or RAM. Model of Computation (Assumptions)

A RAM is assumed to be an idealized machine Infinitely large random-access memory Instructions execute sequentially Every instruction is in fact a basic operation on two values in the machines memory which takes unit time. These might be characters or integers. Example of basic operations include Assigning a value to a variable Arithmetic operation (+, -, ×, /) on integers Performing any comparison e.g. a < b Boolean operations Accessing an element of an array. Model of Computation (Assumptions)

In theoretical analysis, computational complexity Estimated in asymptotic sense, i.e. Estimating for large inputs Big O, Omega, Theta etc. notations are used to compute the complexity Asymptotic notations are used because different implementations of algorithm may differ in efficiency Efficiencies of two given algorithm are related By a constant multiplicative factor Called hidden constant. Model of Computation (Assumptions)

First poor assumption We assumed that each basic operation takes constant time, i.e. model allows Adding Multiplying Comparing etc. two numbers of any length in constant time Addition of two numbers takes a unit time! Not good because numbers may be arbitrarily Addition and multiplication both take unit time! Again very bad assumption Drawbacks in Model of Computation

Finally what about Our Model? But with all these weaknesses, our model is not so bad because we have to give the Comparison not the absolute analysis of any algorithm. We have to deal with large inputs not with the small size Model seems to work well describing computational power of modern nonparallel machines Can we do Exact Measure of Efficiency ? Exact, not asymptotic, measure of efficiency can be sometimes computed but it usually requires certain assumptions concerning implementation Model of Computation not so Bad

Analysis will be performed with respect to this computational model for comparison of algorithms We will give asymptotic analysis not detailed comparison i.e. for large inputs We will use generic uniprocessor random- access machine (RAM) in analysis All memory equally expensive to access No concurrent operations All reasonable instructions take unit time, except, of course, function calls Summary : Computational Model

DESIGN AND ANALYSIS In order to understand design and analysis procedure we will do an example

SELECTION PROBLEM Suppose you want to purchase a Laptop You want to pick fastest Laptop But fast Laptops are expensive You cannot decide which one is more important price or speed

SELECTION PROBLEM Definitely do not want a Laptop if there is a another Laptop that is both fast and cheaper We say that fast cheap Laptops “dominates” the slow expensive So given a list of Laptops we want those that are not dominated by the other

CRITERION FOR SELECTION Two criterion for selection: Speed Price

MATHEMATICAL MODEL P is in two dimensional space and its coordinates are P=(p.x, p.y) (x,y) x is speed of Laptop y is negation of price We can say about y High value of y mean cheap Laptop and low y means Laptop is expensive A point p is said to be dominated by a point q if p.x <= q.x and p.y <= q.y In an array of points (Laptops) P={p 1,p 2,p 3,….,p n } a maximal point is not dominated by any other point

EXAMPLE (700,15) (500,20) (200,1) (1000,50) (900,35) (800,30) (1400,70) (1500,80) (1200,60)

EXAMPLE Given a set of point P={p 1,p 2,p 3,….,p n } output the maximal points Those points of P such that p i is not dominated by any other point

BRUTE-FORCE ALGORITHM Maximal ( int n, Point p[1……n]) 1 for i = 1 to n 2 maximal = true 3 for j = 1 to n 4 if( i != j ) and (p[i].x <= p [j]. x) and (p [i].y <= p [j].y) 5 maximal = false; 6 break; 7if (maximal == true) 8then output p [i].x, p[i]. y

RUNNING TIME ANALYSIS We can observe that: The running time depends upon the input size. Different input of the same size may result different time

RUNNING TIME ANALYSIS Count the number of steps of the pseudo code that are executed Or count the number of times an element of p is accessed Or number of comparisons that are performed

ANALYSIS OF SELECTION PROBLEM Input size is n We will count how number of steps

BRUTE-FORCE ALGORITHM Maximal(int n, Point p[1……n]) 1 for i = 1 to n 2 maximal = true 3 for j = 1 to n 4 if( i != j ) and (p[i].x <= p [j]. x) and (p [i].y <= p [j].y) 5 maximal = false; 6 break; 7if (maximal == true) 8 output p [i].x, p[i]. y n times 1 time n times 5 times 2 times Worst Case Analysis

ANALYSIS OF SELECTION PROBLEM The outer loop runs n times For each iteration. The inner loop runs n times in the worst case P is accessed four times in the if statement The output statement accesses p 2 times

Maximal(int n, Point p[1……n]) 1 for i = 1 to n 2 maximal = true 3 for j = 1 to n 4 if( i != j ) and (p[i].x <= p [j]. x) and (p [i].y <= p [j].y) 5 maximal = false; 6 break; 7if (maximal == true) 8 output p [i].x, p[i]. y n times 1 time n times 5 times 2 times

ANALYSIS OF SELECTION PROBLEM In the worst case every point is maximal so every point is output Worst-case time is:

ANALYSIS OF SELECTION PROBLEM

Suppose n=50 and We use a computer whose speed is instructions per second Then this computer requires

COMPLEXITY OF THE ALGORITHMS Algorithm complexity is the work done by the algorithm there are following complexities: a.Worst case time b.Average case time c.Best case time d.Expected time

GROWTH OF FUNCTIONS Input SizeTime of T(n)

IMPORTANT We will almost always work with worst case time. Average case time is most difficult to compute Worst case refer to the upper limit on the running time

ANALYSIS OF SELECTION PROBLEM T(n) is Asymptotically equivalent to n 2

DIVIDE & CONQUER A strategy to solve large number of problems. It has following stages:  Divide : divide the problem into small number of pieces  Conquer : Solve each piece by applying divide and conquer recursively  Combine : Rearrange the pieces  Examples: MergeSort, QuickSort