1. Design and Analysis of Algorithms - Chapter 1
Σχεδίαση Αλγορίθμων
Introduction to the Design and Analysis of Algorithms by Anany Levitin
Design and Analysis of Algorithms - Chapter 1

2. Design and Analysis of Algorithms - Chapter 1
Σχεδίαση Αλγορίθμων
Ημερολόγιο, βιβλιογραφία, ανακοινώσεις
Θα διανεμηθεί το βιβλίο «Αλγόριθμοι» του Παναγιώτη Μποζάνη, Εκδόσεις Τζιόλα, 2005.
Design and Analysis of Algorithms - Chapter 1

3. Design and Analysis of Algorithms - Chapter 1
Why study algorithms?
Wirth 1976: Algorithms + Data Structures = Programs
Theoretical importance
the core of computer science
Practical importance
A practitioner’s toolkit of known algorithms
Framework for designing and analyzing algorithms for new problems
Design and Analysis of Algorithms - Chapter 1

4. Design and Analysis of Algorithms - Chapter 1
An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
Design and Analysis of Algorithms - Chapter 1

5. Design and Analysis of Algorithms - Chapter 1
“Besides merely being a finite set of rules that gives a sequence of operations for solving a specific type of problem, an algorithm has the five important features” [Knuth]
finiteness (otherwise: computational method)
termination
definiteness
precise definition of each step
input (zero or more)
output (one or more)
effectiveness
Its operations must all be sufficiently basic that they can in principle be done exactly and in a finite length of time by someone using pencil and paper
Design and Analysis of Algorithms - Chapter 1

6. Historical Perspective
Euclid’s algorithm for finding the greatest common divisor
Abū ‘Abd Allāh Muḥammad ibn Mūsā al-Khwārizmī (أبو عبد الله محمد بن موسى الخوارزمي) was a Muslim mathematician, astronomer, astrologer, geographer and author. He was born in Persia around 780, and died around 850.
Because of his book on the systematic solution of linear and quadratic equations, al-Kitāb al-mukhtaṣar fī hīsāb al-ğabr wa’l-muqābala, together with Diophantus, he is considered to be the father of algebra. The word algebra is derived from al-ğabr, one of the two operations used to solve quadratic equations, as described in his book. Algoritmi de numero Indorum, the Latin translation of his other major work, on the Indian numerals, introduced the positional number system and the number zero to the Western world in the 12th century. The words algorism and algorithm stem from Algoritmi, the Latinization of his name
Euclid’s algorithm is good for introducing the notion of an algorithm because it
makes a clear separation from a program that implements the algorithm.
It is also one that is familiar to most students.
Al Khowarizmi (many spellings possible...) – “algorism” (originally) and then
later “algorithm” come from his name.
Design and Analysis of Algorithms - Chapter 1

7. Notion of algorithm problem algorithm input output
“computer”
input
output
Algorithmic solution : a way to teach the computer
Design and Analysis of Algorithms - Chapter 1

8. Greatest Common Divisor - Algorithm 1
Step 1 Assign the value of min{m,n} to t
Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, to Step 4
Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4
Step 4 Decrease the value of t by 1. Go to Step 2
Note: m and n are positive integers
Design and Analysis of Algorithms - Chapter 1

9. Greatest Common Divisor - Algorithm 1
Correctness
The algorithm terminates, because t is decreased by 1 each time we go through step 4, 1 divides any integer, and t eventually will reach 1 unless the algorithm stops earlier.
The algorithm is partially correct, because when it returns t as the answer, t is the minimum value that divides both m and n
Design and Analysis of Algorithms - Chapter 1

10. Greatest Common Divisor - Algorithm 2
Step 1 Find the prime factors of m
Step 2 Find the prime factors of n
Step 3 Identify all the common factors in the two prime expansions found in Steps 1 and 2. If p is a common factor repeated i times in m and j times in n, assign to p the multiplicity min{i,j}
Step 4 Compute the product of all the common factors with their multiplicities and return it as the GCD of m and n
Note: as written, the algorithm requires than m and n be integers greater than 1, since 1 is not a prime
Design and Analysis of Algorithms - Chapter 1

11. Greatest Common Divisor - Algorithm 2
In fact “Algorithm 2” is not an algorithm unless we can provide an effective way to find prime factors of a number
The sieve of Eratosthenes is an algorithm that provides such an effective procedure
Πως δουλεύει το κόσκινο του Ερατοσθένη ??
Design and Analysis of Algorithms - Chapter 1

12. Greatest Common Divisor - Algorithm 3
Euclid’s Algorithm
E0 [Ensure n≤ m] If m < n, exchange m with n
E1 [Find Remainder] Divide m by n and let r be the remainder (We will have 0 ≤ r < n)
E2 [Is it zero?] If r=0, the algorithm terminates; n is the answer
E3 [Reduce] Set m to n, n to r, and go back to E1
Design and Analysis of Algorithms - Chapter 1

13. Greatest Common Divisor - Algorithm 3
Termination of Euclid’s Algorithm
The second number of the pair gets smaller with each iteration and cannot become negative: indeed, the new value of n is r = m mod n, which is always smaller than n. Eventually, r becomes zero, and the algorithms stops.
Design and Analysis of Algorithms - Chapter 1

14. Greatest Common Divisor - Algorithm 3
Correctness of Euclid’s Algorithm
After step E1, we have m=qn+r, for some integer q. If r=0, then m is a multiple of n, and clearly in such a case n is the GCD of m and n. If r≠0, note that any number that divides both m and n must divide m–qn=r, and any number that divides both n and r must divide qn+r=m; so the set of divisors of {m,n} is the same as the set of divisors of {n,r}. In particular, the GCD of {m,n} is the same as the GCD of {n,r}. Therefore, E3 does not change the answer to the original problem.
Design and Analysis of Algorithms - Chapter 1

15. Greatest Common Divisor - Algorithm 3
Book Variation (p.4)
Step 1 If n=0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step 2 Divide m by n and assign the value of the remainder to r.
Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.
Note: m and n are nonnegative, not-both-zero integers
Design and Analysis of Algorithms - Chapter 1

16. Greatest Common Divisor - Algorithm 3
Pseudocode for book version
while n <> 0 do
r  m mod n
m  n
n  r
return m
Design and Analysis of Algorithms - Chapter 1

17. Example of computational problem: sorting
Statement of problem:
Input: A sequence of n numbers <a1,a2,…,an>
Output: A reordering of the input sequence <a1,a´2,…,an> so that ai≤aj whenever i<j
Instance: The sequence <5, 3, 2, 8, 3>
Algorithms:
Selection sort
Insertion sort
Merge sort
(many others)
Design and Analysis of Algorithms - Chapter 1

18. Selection Sort Input: array a[1],...,a[n]
Output: array a sorted in non-decreasing order
Algorithm:
for i=1 to n
swap a[i] with smallest of a[i],…a[n]
(In place sorting, stable sorting)
see also pseudocode, section 3.1
The algorithm is given *very* informally here. Show students the pseudocode in
section 3.1.
This is a good opportunity to discuss pseudocode conventions.
Design and Analysis of Algorithms - Chapter 1

19. Some Well-known Computational Problems
Sorting
Searching
String searching
Graph problems (shortest paths, minimum spanning tree, graph-coloring etc)
Combinatorial problems (tsp)
Geometric problems (closest-pair, convex hull)
Numerical problems (equations, integrals)
Other (primality testing, knapsack problem, chess, towers of Hanoi)
1-4 have well known efficient (polynomial-time) solutions
5: primality testing has recently been found to have an efficient solution
This is a great problem to discuss because it has recently been in the news
(see mathworld news at:
or original article:
6(TSP)-9(chess) are all problems for which no efficient solution has been found
it is possible to informally discuss the “try all possibilities” approach that is required
to get exact solutions to such problems
10: Towers of Hanoi is a problem that has only exponential-time solutions (simply
because the output required is so large)
11: Program termination is undecidable
Design and Analysis of Algorithms - Chapter 1

20. Basic Issues Related to Algorithms
Understanding the problem (instances, ranges)
Ascertaining the capabilities of a computational device (RAM and parallel machines, speed and amount of memory)
Choosing between exact and approximate algorithms
Deciding the correct data structure
Algorithm design techniques
Methods of specifying an algorithm (free language, pseudocode, flowchart)
Proving an algorithm’s correctness
Analyzing an algorithm (efficiency, theoretical and empirical analysis, optimality)
Coding an algorithm
Design and Analysis of Algorithms - Chapter 1

21. Algorithm design strategies
Brute force
Divide and conquer
Decrease and conquer
Transform and conquer
Greedy approach
Dynamic programming
Backtracking and Branch and bound
Space and time tradeoffs
Design and Analysis of Algorithms - Chapter 1

22. Analysis of Algorithms
How good is the algorithm?
Correctness
Time efficiency
Space efficiency
Simple (recursive, iterative)
General
Does there exist a better algorithm?
Lower bounds
Optimality
Design and Analysis of Algorithms - Chapter 1