1. Design and Analysis of Algorithms
Mohammad Shahnawaz Nasir
College of Computer Science and Information System
Jazan University , Jazan

2. Textbook Cover What does the cover include?
Puzzles, Notions, and Stories of Algorithms.

3. Text Book and Reference Books
Anany Levitin, Introduction to the Design and Analysis of Algorithms, Tsinghua University Press, 2003, 39RMB.
Reference Books:
M.H. Alsuwaiyel, Algorithms Design Techniques and Analysis, Publishing House of Electronics Industry, 2003, 40RMB.
Thomas H. Cormen etc., Introduction to Algorithms (Second Edition), Higher Education Press & The MIT Press, 68RMB.
Others, search on the website with keyword “Design and Analysis of algorithm”, you will find more...

4. Grading Schemes Total 100 Exam-1: 10% Exam-2: 10%
Assignments and Mini Projects: 20%
Final Exam: 40%
Lab Exam: 20%

5. Why Study this Course?
Donald E. Knuth stated “Computer Science is the study of algorithms”
Cornerstone of computer science. Programs will not exist without algorithms.
Closely related to our lives
Help to guide how others analyze and solve problems
Help to develop the ability of analyzing and solving problems via computers
Very interesting if you can concentrate on this course

6. Course Prerequisite Data Structure Discrete Structure
C, Java or other programming languages
Advanced Mathematics

7. Algorithm: A brief History
Muhammad ibn Musa al-Khwarizmi, one of the most influential mathematicians in 9th century in Baghdad, wrote a textbook in Arabic about adding, multiplying, dividing numbers, and extracting square roots and computing π.
Many centuries later, decimal system was adopted in Europe, and the procedures in Al Khwarizmi’s book were named after him as “Algorithms”.
(image of Al Khwarizmi from

8. Notion: Algorithms problem algorithm “computer” output input
An algorithm is a sequence of unambiguous instructions for solving a computational problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.
“computer”
problem
algorithm
input
output

9. Notion: Algorithm
More precisely, an algorithm is a method or process to solve a problem satisfying the following properties:
Finiteness - terminates after a finite number of steps
Definiteness - Each step must be rigorously and unambiguously specified.
Input - Valid inputs must be clearly specified.
Output - can be proved to produce the correct output given a valid input.
Effectiveness - Steps must be sufficiently simple and basic.

10. Examples Is the following a legitimate algorithm? i 1
While (i <= 10) do
a  i + 1
Print the value of a
End of loop
Stop

11. Examples of Algorithms – Computing the Greatest Common Divisor of Two Integers
The greatest common divisor of two non-negative, not-both-zero integers m and n, denoted gcd(m, n), is defined as the largest integer that divides both m and n evenly, i.e, with a remainder to zero.
Euclid’s algorithm: gcd(m, n) = gcd(n, m mod n)
For example gcd(60, 24) can be computed as follows:
gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12
Gcd(70, 25) = gcd(25, 20) = gcd(20, 5) = gcd (5, 0) = 5
Gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12

12. Computing the Greatest Common Divisor of Two Integers
Here is more structured description of this algorithm:
Euclid’s Algorithm for computing gcd(m, n)
Step1: If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step2: Divide m by n and assign the value of the remainder to r.
Step 3: Assign the value of n to m and the value of r to n. Go to Step 1.

13. Pseudocode of Euclid’s Algorithm
Algorithm Euclid(m, n)
//Computes gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero integers m and n
//Output: Greatest common divisor of m and n
while n ≠ 0 do
r  m mod n
m  n
n  r
return m .

14. Second Try for gcd(m, n)
Consecutive integer checking algorithm for computing gcd(m, n)
Step1: Assign the value of min{m, n} to t.
Step2: Divide m by t. If the remainder of this division is 0, go to Step3;otherwise, go to Step 4.
Step3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step4.
Step4: Decrease the value of t by 1. Go to Step2.
For example, for number 60 and 24, the algorithm will try first 24, then 23, and so on until it reaches 12, where it stops.

15. Third try for gcd(m, n)
Middle-school procedure for computing gcd(m, n)
Step1: Find the prime factors of m.
Step2: Find the prime factors of n.
Step3: Identify all the common factors in the two prime expansions found in Step1 and Step2. (If p is a common factor occurring Pm and Pn times in m and n, respectively, it should be repeated min{Pm, Pn} times.)
Step4: Compute the product of all the common factors and return it as the gcd of the numbers given.
Thus, for the numbers 60 and 24, we get
60 =
24 =
gcd(60,24) = = 12

16. What can we learn from the previous 3 examples?
Each step of an algorithm must be unambiguous.
The same algorithm can be represented in several different ways. (different pseudo-codes)
There might exists more than one algorithm for a certain problem.
Algorithms for the same problem can be based on very different ideas and can solve the problem with dramatically different speeds.

17. Fundamentals of Algorithmic Problem Solving
We can consider algorithms to be a procedural solutions to problems. These solutions are not answers but rather specific instructions for getting answers.
The sequence of steps one typically goes through in designing and analyzing an algorithm:
Understanding the problem: understand completely the problem given, ask questions if you have doubt, do a few examples by hand, think about special cases, etc.
Deciding on:
- you need to ascertain the capabilities of the computational device the algorithm is intended for.
- Decision to choose between solving the problem exactly or solving it approximately.
- Appropriate data structure

18. Fundamentals of Algorithmic Problem Solving
Design an Algorithm:
using a natural language has an obvious appeal.
a pseudo-code is a mixture of natural language and programming language constructs.
Proving an Algorithm’s Correctness: that is, you have to prove that the algorithm yields a required result for every legitimate input in a finite amount of time.
Analyzing an algorithm:
- Time efficiency : how fast the algorithm runs
- Space efficiency: how much extra memory the algorithm needs.
Coding an algorithm: ultimately implemented as computer programs.

19. Algorithm Design and Analysis Process

20. Two main issues related to algorithms
How to design algorithms
How to analyze algorithm efficiency

21. Algorithm Design Techniques/Strategies
Brute force
Divide and conquer
Decrease and conquer
Transform and conquer
Space and time tradeoffs
Greedy approach
Dynamic programming
Backtracking
Branch and bound

22. Analysis of Algorithms
How good is the algorithm?
time efficiency
space efficiency
Does there exist a better algorithm?
lower bounds
optimality

23. Example: Fibonacci Number
Fibonacci’s original question:
Suppose that you are given a newly-born pair of rabbits, one male, one female.
Rabbits are able to mate at the age of one month so that at the end of its second month a female can produce another pair of rabbits.
Suppose that our rabbits never die.
Suppose that the female always produces one new pair (one male, one female) every month.
Question: How many pairs will there be in one year?
1. In the beginning: (1 pair)
2. End of month 1: (1 pair) Rabbits are ready to mate.
3. End of month 2: (2 pairs) A new pair of rabbits are born.
4. End of month 3: (3 pairs) A new pair and two old pairs.
5. End of month 4: (5 pairs) ...
6. End of month 5: (8 pairs) ...
7. after 12 months, there will be 233 rabbits
(image of Leonardo Fibonacci from

24. Recurrence Relation of Fibonacci Number fib(n):
{0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …}

25. Algorithm: Fibonacci
Problem: What is fib(200)? What about fib(n), where n is any positive integer?
Questions that we should ask ourselves.
1. Is the algorithm correct?
2. What is the running time of our algorithm?
3. Can we do better?
Algorithm 1 fib(n)
if n = 0 then
return (0)
if n = 1then
return (1)
return (fib(n − 1) + fib(n − 2))

26. Answer of Questions Is the algorithm correct?
Yes, we simply follow the definition of Fibonacci numbers
How fast is the algorithm?
If we let the run time of fib(n) be T(n), then we can formulate
T(n) = T(n − 1) + T(n − 2) + 3  1.6n
T(200)  2139
The world fastest computer BlueGene/L, which can run 248 instructions per second, will take 291 seconds to compute. (291 seconds = 7.85 × 1019 years )
Can Moose’s law, which predicts that CPU get 1.6 times faster each year, solve our problem?
No, because the time needed to compute fib(n) also have the same “growth” rate
if we can compute fib(100) in exactly a year,
then in the next year, we will still spend a year to compute fib(101)
if we want to compute fib(200) within a year, we need to wait for 100 years.

27. Improvement Can we do better? Algorithm 2: fib(n)
Yes, because many computations in the previous algorithm are repeated.
Algorithm 2: fib(n)
comment: Initially we create an array A[0: n]
A[0] ← 0,A[1] ← 1
for i = 2 to n do
A[i] = A[i − 1] + A[i − 2]
return (A[n])

28. Important problem types
sorting
searching
string processing
graph problems
combinatorial problems
geometric problems
numerical problems

29. Fundamental data structures
list
array
linked list
string
stack
queue
priority queue
graph
tree
set and dictionary

30. Important Problem Types and Fundamental Data Structures
Important problems types
Sorting, searching, string processing, graph problems
Fundamental data structures
Linear data structures
Stacks, queues, and heaps
Graphs
Trees

31. Sorting
Rearrange the items of a given list in ascending/descending order.
Input: A sequence of n numbers <a1, a2, …, an>
Output: A reordering <a´1, a´2, …, a´n> of the input sequence such that a´1≤ a´2 ≤ … ≤ a´n.
Why sorting?
Help searching
Algorithms often use sorting as a key subroutine.
Sorting key
A specially chosen piece of information used to guide sorting. I.e., sort student records by names.

32. Sorting Examples of sorting algorithms
Bubble sort
Selection sort
Insertion sort
Merge sort
Quick sort
Heap sort …
Evaluate sorting algorithm complexity: the number of key comparisons.
Two properties
Stability: A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input.
In place : A sorting algorithm is in place if it does not require extra memory, except, possibly for a few memory units.

33. Searching Find a given value, called a search key, in a given set.
Examples of searching algorithms
Sequential searching
Binary searching…

34. String Processing
A string is a sequence of characters from an alphabet.
Text strings: letters, numbers, and special characters.
String matching: searching for a given word/pattern in a text.

35. Graph Problems Informal definition
A graph is a collection of points called vertices, some of which are connected by line segments called edges.
Graphs can be used for modeling real-life applications
Estimation of web diameter, which is maximum no. of links one needs to follow to reach one web page.
communication networks
Project scheduling
games
Examples of graph algorithms
Graph traversal algorithms
Shortest-path algorithms
Topological sorting

36. Linear Data Structures
Arrays
fixed length (need preliminary reservation of memory)
contiguous memory locations
direct access
Insert/delete
Linked Lists
dynamic length
arbitrary memory locations
access by following links
Arrays
A sequence of n items of the same data type that are stored contiguously in computer memory and made accessible by specifying a value of the array’s index.
Linked List
A sequence of zero or more elements called nodes each containing two kinds of information: some data and one or more links called pointers to other nodes of the linked list.
Singly linked list (next pointer)
Doubly linked list (next + previous pointers)

37. Singly linked list of n elements
Array of n elements
Singly linked list of n elements
Double linked list of n elements

38. Stacks, Queues, and Heaps (1)
A stack of plates
insertion/deletion can be done only at the top.
LIFO/FILO
Two operations (push and pop)
Queues
A queue of customers waiting for services
Insertion/enqueue from the rear and deletion/dequeue from the front.
FIFO
Two operations (enqueue and dequeue)

39. Stacks, Queues, and Heaps (2)
Priority queues (implemented using heaps)
A data structure for maintaining a set of elements, each associated with a key/priority, with the following operations
Finding the element with the highest priority
Deleting the element with the highest priority
Inserting a new element
Scheduling jobs on a shared computer.

40. Graphs Formal definition (a) (b)
A graph G = <V, E> is defined by a pair of two sets: a finite set V of items called vertices and a set E of pairs of vertex called edges.
If these pairs of vertices are unordered, we say that the graph G is undirected otherwise directed graphs (digraph).
A graph with every pair of its vertices connected by an edge is called complete. K|V|
A graph with relatively few possible edges missing is called dense.
A graph with few edges relative to the no. of its vertices is called sparse.
(a)
(b)

41. Graph Representation Adjacency matrix Adjacency linked lists
Graphs for computer algorithm can be represented in two principal ways:
Adjacency matrix
The adjacency matrix of a graph with n vertices is an n-by-n boolean matrix with one row and one column for each of the graph’s vertices.
The element on the ith row and jth column is 1 if there’s an edge from ith vertex to the jth vertex; otherwise 0.
The adjacency matrix of an undirected graph is symmetric.
Adjacency linked lists
A collection of linked lists, one for each vertex, that contain all the vertices adjacent to the list’s vertex.

42. Weighted Graphs
A weighted graph is a graph or digraphs with numbers assigned to its edges. These numbers are called weights or costs. Used to find shortest path between two points.

43. Graph Properties -- Paths and Connectivity
A path from vertex u to v of a graph G can be defined as a sequence of adjacent (connected by an edge) vertices that starts with u and ends with v.
If all edges of a path are distinct, the path is said to be simple.
Path lengths: the number of edges, or the number of vertices – 1.
Connected graphs
A graph is said to be connected if for every pair of its vertices u and v there is a path from u to v.
Connected component
The maximum connected subgraph of a given graph.
Examples of a simple path and a not simple path.
Connected graphs: starting from any vertex, we can always find a path to reach all the other vertices. (Ball-String example.)
From NIST:
Connected graphs:
Definition: An undirected graph that has a path between every pair of vertices.
Strongly connected graphs:
Definition: A directed graph that has a path from each vertex to every other vertex.
Connected component: …
Strongly connected component:
a strongly connected component of a digraph G is a maximal strongly connected subgraph of G.
Graph that is not connected

44. Graph Properties -- Acyclicity
Cycle
A cycle is a simple path of a positive length that starts and ends a the same vertex.
For example, f, h, i, g, f is a cycle in the graph of above figure.
Acyclic graph
A graph with no cycles is said to be acyclic.
DAG (Directed Acyclic Graph)

45. Trees Trees Properties of trees
A tree (or free tree) is a connected acyclic graph.
A graph that has no cycles but is not necessarily connected is called a forest.
Properties of trees
Trees have several important properties other graphs do not have. In particular, the number of edges in a tree is always one less than the number of its vertices: |E| = |V| - 1
For every two vertices in a tree there always exists exactly one simple path from one of these vertices to the other. Why?
Rooted trees: The above property makes it possible to select an arbitrary vertex in a free tree and consider it as the root of the so-called rooted tree.
Levels of rooted tree.

46. A tree and a forest

47. Rooted Trees ancestors descendants parent, child and siblings Leaves
For any vertex v in a tree T, all the vertices on the simple path from the root to that vertex are called ancestors.
descendants
All the vertices for which a vertex v is an ancestor are said to be descendants of v.
parent, child and siblings
If (u, v) is the last edge of the simple path from the root to vertex v (and u  v), u is said to be the parent of v and v is called a child of u.
Vertices that have the same parent are called siblings.
Leaves
A vertex without children is called a leaf.
Subtree
A vertex v with all its descendants is called the subtree of T rooted at v.

48. Transformation of a free tree into a rooted tree

49. Trees
Depth of a vertex
The length of the simple path from the root to the vertex.
Height of a tree
The length of the longest simple path from the root to a leaf.
For example the depth of vertex c of the above tree is 2, and the height of the tree is 3.

50. Ordered Trees Ordered trees Binary trees Binary search trees
An ordered tree is a rooted tree in which all the children of each vertex are ordered.
Binary trees
A binary tree is an ordered tree in which every vertex has no more than two children and each children is designated as either a left child or a right child of its parent.
Binary search trees
Each vertex is assigned a number.
A number assigned to each parental vertex is larger than all the numbers in its left subtree and smaller than all the numbers in its right subtree.
log2n  h  n – 1, where h is the height of a binary tree.

51. A binary tree A binary search tree
Standard implementation of (b)

52. A rooted tree and its first child-next sibling representation
Here, the left pointer will point to the first child of the vertex while the right pointer will point to its next sibling. Accordingly this representation is called the first child-next sibling representation.