1. Algorithms
Furqan Majeed

2. What is Algorithm? Algorithm Input output
A computer algorithm is a detailed step-by-step method for solving a problem by using a computer.
An algorithm is a sequence of unambiguous instructions for solving a problem in a finite amount of time.
More generally, an Algorithm is any well defined computational procedure that takes collection of elements as input and produces a collection of elements as output.
Algorithm
Input
output

3. Popular Algorithms, Factors of Dependence
Most basic and popular algorithms are
Sorting algorithms
Searching algorithms
Which algorithm is best?
Mainly, it depends upon various factors, for example in case of sorting
The number of items to be sorted
The extent to which the items are already sorted
Possible restrictions on the item values
The kind of storage device to be used etc.

4. One Problem, Many Algorithms
The statement of the problem specifies, in general terms, the desired input/output relationship.
Algorithm
The algorithm describes a specific computational procedure for achieving input/output relationship.
Example
One might need to sort a sequence of numbers into non-decreasing order.
Algorithms
Various algorithms e.g. merge sort, quick sort, heap sorts, radix sort, counting sort etc.

5. Important Designing Techniques
Brute Force
Straightforward, naïve/simple approach
Mostly expensive
Divide-and-Conquer
Divide into smaller sub-problems
Iterative Improvement
Improve one change at a time
Decrease-and-Conquer
Decrease instance size
Transform-and-Conquer
Modify problem first and then solve it
Space and Time Tradeoffs
Use more space now to save time later

6. Some of the Important Designing Techniques
Greedy Approach
Locally optimal decisions, can not change once made.
Efficient
Easy to implement
The solution is expected to be optimal
Every problem may not have greedy solution
Dynamic programming
Decompose into sub-problems like divide and conquer
Sub-problems are dependant
Record results of smaller sub-problems
Re-use it for further occurrence
Mostly reduces complexity exponential to polynomial

7. Problem Solving Process
Strategy
Algorithm
Input
Output
Steps
Analysis
Correctness
Time & Space
Optimality
Implementation
Verification

8. Model of Computation (Assumptions)
Design assumption
Level of abstraction which meets our requirements
Neither more nor less e.g. [0, 1] infinite continuous interval
Analysis independent of the variations in
Machine
Operating system
Programming languages
Compiler etc.
Low-level details will not be considered
Our model will be an abstraction of a standard generic single-processor machine, called a random access machine or RAM.

9. Model of Computation (Assumptions)
A RAM is assumed to be an idealized machine
Infinitely large random-access memory
Instructions execute sequentially
Every instruction is in fact a basic operation on two values in the machines memory which takes unit time.
These might be characters or integers.
Example of basic operations include
Assigning a value to a variable
Arithmetic operation (+, - , × , /) on integers
Performing any comparison e.g. a < b
Boolean operations
Accessing an element of an array.

10. Model of Computation (Assumptions)
In theoretical analysis, computational complexity
Estimated in asymptotic sense, i.e.
Estimating for large inputs
Big O, Omega, Theta etc. notations are used to compute the complexity
Asymptotic notations are used because different implementations of algorithm may differ in efficiency
Efficiencies of two given algorithm are related
By a constant multiplicative factor
Called hidden constant.

11. Drawbacks in Model of Computation
First poor assumption
We assumed that each basic operation takes constant time, i.e. model allows
Adding
Multiplying
Comparing etc.
two numbers of any length in constant time
Addition of two numbers takes a unit time!
Not good because numbers may be arbitrarily
Addition and multiplication both take unit time!
Again very bad assumption

12. Model of Computation not so Bad
Finally what about Our Model?
But with all these weaknesses, our model is not so bad because we have to give the
Comparison not the absolute analysis of any algorithm.
We have to deal with large inputs not with the small size
Model seems to work well describing computational power of modern nonparallel machines
Can we do Exact Measure of Efficiency ?
Exact, not asymptotic, measure of efficiency can be sometimes computed but it usually requires certain assumptions concerning implementation

13. Summary : Computational Model
Analysis will be performed with respect to this computational model for comparison of algorithms
We will give asymptotic analysis not detailed comparison i.e. for large inputs
We will use generic uniprocessor random-access machine (RAM) in analysis
All memory equally expensive to access
No concurrent operations
All reasonable instructions take unit time, except, of course, function calls

14. Design And Analysis
In order to understand design and analysis procedure we will do an example

15. Selection Problem Suppose you want to purchase a Laptop
You want to pick fastest Laptop
But fast Laptops are expensive
You cannot decide which one is more important price or speed

16. Selection Problem
Definitely do not want a Laptop if there is a another Laptop that is both fast and cheaper
We say that fast cheap Laptops “dominates” the slow expensive
So given a list of Laptops we want those that are not dominated by the other

17. Criterion for selection
Two criterion for selection:
Speed
Price

18. Mathematical Model
P is in two dimensional space and its coordinates are P=(p.x , p.y)
(x,y)
x is speed of Laptop
y is negation of price
We can say about y
High value of y mean cheap Laptop and low y means Laptop is expensive
A point p is said to be dominated by a point q if p.x <= q.x and p.y <= q.y
In an array of points (Laptops) P={p1,p2,p3,….,pn} a maximal point is not dominated by any other point

19. Example
(700,15) (500,20) (200,1)
(1000,50) (900,35) (800,30)
(1400,70) (1500,80) (1200,60)

20. Example
Given a set of point P={p1,p2,p3,….,pn } output the maximal points
Those points of P such that pi is not dominated by any other point

21. Brute-Force Algorithm
Maximal(int n, Point p[1……n])
for i = 1 to n
maximal = true
for j = 1 to n
if( i != j ) and (p[i].x <= p [j]. x) and (p [i] .y <= p [j].y)
maximal = false;
break;
if (maximal == true)
then output p [i].x , p[i]. y

22. Running Time Analysis We can observe that:
The running time depends upon the input size.
Different input of the same size may result different time

23. Running Time Analysis
Count the number of steps of the pseudo code that are executed
Or count the number of times an element of p is accessed
Or number of comparisons that are performed

24. Analysis of selection Problem
Input size is n
We will count how number of steps

25. Brute-Force Algorithm
Maximal(int n, Point p[1……n])
for i = 1 to n
maximal = true
for j = 1 to n
if( i != j ) and (p[i].x <= p [j]. x) and (p [i] .y <= p [j].y)
maximal = false;
break;
if (maximal == true)
output p [i].x , p[i]. y
n times
1 time
Worst Case Analysis
n times
5 times
2 times

26. Analysis of selection Problem
The outer loop runs n times
For each iteration. The inner loop runs n times in the worst case
P is accessed four times in the if statement
The output statement accesses p 2 times

27. Maximal(int n, Point p[1……n]) for i = 1 to n maximal = true
for j = 1 to n
if( i != j ) and (p[i].x <= p [j]. x) and (p [i] .y <= p [j].y)
maximal = false;
break;
if (maximal == true)
output p [i].x , p[i]. y
n times
1 time
n times
5 times
2 times

28. Analysis of selection Problem
In the worst case every point is maximal so every point is output
Worst-case time is:

29. Analysis of selection Problem

30. Analysis of selection Problem
Suppose n=50 and
We use a computer whose speed is instructions per second
Then this computer requires

31. Complexity of the algorithms
Algorithm complexity is the work done by the algorithm there are following complexities:
Worst case time
Average case time
Best case time
Expected time

32. Growth of Functions Input Size Time of T(n) 50 0.01265 100 0.0503 500
1.2515
1000
5.003
5000
10000
500.03
100000

33. Important We will almost always work with worst case time.
Average case time is most difficult to compute
Worst case refer to the upper limit on the running time

34. Analysis of Selection Problem
T(n) is Asymptotically equivalent to n2

35. Divide: divide the problem into small number of pieces
Divide & Conquer
A strategy to solve large number of problems. It has following stages:
Divide: divide the problem into small number of pieces
Conquer: Solve each piece by applying divide and conquer recursively
Combine: Rearrange the pieces
Examples: MergeSort, QuickSort