1. Basics of Algorithm Analysis
Asymptotic algorithm analysis
Growth rate
Upper bounds of growth rate
Lower bounds of growth rate
 Notation
Space complexity
Tradeoffs of implementations
Analyzing Problems – Optimal solution

2. Asymptotic Analysis
Asymptotic analysis – study of an algorithm as the input size “gets big” or reaches the limit
Advantages and limitations

3. Growth rate
Growth rate - the rate at which the cost of an algorithm grows as the size of the input grows
Growth rate enables us to compare different algorithms that can be used for the same problem

4. Upper bounds of growth rate
Big-Oh notation - O
Definition
T(n) is in the set O(f(n)) if there exists two positive constants c and n0 such that
| T(n)|  c|f(n)| for all n > n0
Meaning
Algorithm has upper bound to its growth rate of f(n)

5. Examples Example 1 – Sequential search algorithm
T(n) = c1(n+1)/2
|c1n/2|  c1|n| for all n > 1
T(n) is in O(n) for n0 = 1 and c = c1
Example 2 – Assignment operation
T(n) = c2
T(n) is in O(c2) – T(n) is in O(1)
Example 3 – T(n) = c3n2 + c4n
|c3n2 + c4n|  |c3n2 + c4n2|  (c3 + c4)|n2| for all n > 1
T(n) is in O(n2) for n0 = 1 and c = c3 + c4

6. Lower bounds of growth rate
Big-Omega notation - 
Definition
T(n) is in the set (f(n)) if there exists two positive constants c and n0 such that
| T(n)|  c|f(n)| for all n > n0
Meaning
Algorithm has lower bound to its growth rate of f(n)

7. Examples Example 1 – Sequential search algorithm
T(n) = c1n/2
|c1n/2|  c1/2|n| for all n > 1
T(n) is in (n) for n0 = 1 and c = c1/2
Example 2 – Assignment operation
T(n) = c2
T(n) is in (c2) – T(n) is in (1)
Example 3 – T(n) = c3n2 + c4n
|c3n2 + c4n|  c3|n2| for all n > 1
T(n) is in (n2) for n0 = 1 and c = c3

8.  analysis Theta notation -  Definition Meaning
An algorithm is said to be  (f(n)) if it is in the set O(f(n)) and it is in the set in the set (f(n))
Meaning
When upper bounds and lower bounds are the same we indicate this using  notation

9. Example
while (n<0) {
if (ODD(n))
n = 3 * n + 1;
else
n = n / 2; }

10. Examples Example 1 – Sequential search algorithm
T(n) is in (n)
T(n) is in O(n)
T(n) is (n)
Example 2 – Assignment operation
T(n) is in (1)
T(n) is in O(1)
T(n) is (1)
Example 3 – T(n) = c3n2 + c4n
T(n) is in (n2)
T(n) is in O(n2)
T(n) is (n2)

11. Simplifying Rules
If f(n) is in O(g(n)) and g(n) is in O(h(n)), then f(n) is in O(h(n)). [Bounds are not unique]
If f(n) is in O(kg(n)) for any constant k >0, then f(n) is in O(g(n)). [No constant]
If f1 (n) is in O(g1 (n)) and f2 (n) is in O(g2 (n)), then (f1 + f2 )(n) is in O(max(g1 (n), g2 (n))). [Drop low order terms]
If f1 (n) is in O(g1 (n)) and f2 (n) is in O(g2 (n)) then f1 (n) * f2 (n) is in O(g1 (n) * g2 (n)). [Multiply number of actions by cost of each action]

12. Multiple parameters for (int i=0; i<C; i++) sumC++;
for (int j=0; j<P; j++)
sumP++;
If we use P as the measure, then time is (P)
If we use C as the measure, then time is (C)
More accurate is (P + C)

13. Space Bounds
The concept of asymptotic analysis of growth rate completely applies to measuring space requirements
Example – one-dimensional array
(n)
Example – two-dimensional array
(n2)

14. Space/Time tradeoffs Space/Time tradeoff principle
One can often achieve a reduction in time is one is willing to sacrifice space, or vice versa
Disk Based Space/Time tradeoff principle:
The smaller you can make your disk storage requirements, the faster your program will run.

15. Analyzing Problems – Optimal solution
Given a problem, there are many possible algorithms to solve the problem. We have a set of solution algorithms. We usually don't have an enumeration of this set. We are interested to find an optimal solution that take the least running time.
By designing and implementing a correct solution to the problem, we constructively prove an upper bound to the running time of an optimal solution. By improving the existing algorithms and designing new ones, we keep push the upper bound down.

16. Analyzing Problems – Optimal solution
On the another hand, we observe some inherent properties of the problem (not the algorithms), and conclude that no solutions to the problem can have a running time less than a particular lower bound of the running time. We, thus, prove a lower bound to the running time of the optimal solution. Theorists keep studying the properties of the problem and pushing the lower bound up.
When an algorithm has a running time on the same order as the lower bound of the solution to the problem, the upper bound and the lower bound meet. We can then declare that we have found an optimal solution. 

17. Fundamental Data Structures List ADT
Implementations of List ADT
Array based lists
Linked lists
Comparisons of two implementations
Freelists

18. List ADT
A list is finite, ordered sequence of data items known as elements
Each element has a position in the list (first, second, etc.)
A list is said to be empty when it contains no elements
The number of elements currently stored is called length of the list
The beginning of the list is called head
The end of the list is called tail

19. Operations on list ADT Create a list Insert an element
Remove an element
Access an element
Clear (remove all elements)
Access an element next or previous to the currently accessed element

20. Specification of List ADT
class List { // list ADT
private:
// list Elements
public:
List(int =LIST_SIZE); // Constructor
~List(); // Destructor
void clear(); // Remove all Elems from list
void insert(const Elem); // Insert Elem at current position
void append(const Elem); // Insert Elem at tail of list
Elem remove(); // Remove and return current Elem
void setFirst(); // Set curr to first position
void prev(); // Move curr to previous position
void next(); // Move curr to next position
int length() const; // Return current length of list
void setPos(int); // Set curr to specified position
void setValue(const Elem); // Set current Elem's value
Elem currValue() const; // Return current Elem's value
bool isEmpty() const; // Return TRUE if list is empty
bool isInList() const; // TRUE if curr is within list
bool find(int); // Find value (from current position) };

21. Implementations of list
Array-based implementation
Uses an array to store list elements
Linked-list based implementation
Uses a linked list to store list elements

22. Array based implementation
Elements stored in an array in continuous positions
Maximum size of the list must be known before the list is created
Head of the list is at position 0

23. Array based list class class List { // Array-based list class private:
int msize; // Maximum size of list
int numinlist; // Actual number of Elems in list
int curr; // Position of "current" Elem
Elem* listarray; // Array holding list Elems
public:
List(int =LIST_SIZE); // Constructor
~List(); // Destructor
void clear(); // Remove all Elems from list
void insert(const Elem); // Insert Elem at current position
void append(const Elem); // Insert Elem at tail of list
Elem remove(); // Remove and return current Elem
void setFirst(); // Set curr to first position
void prev(); // Move curr to previous position
void next(); // Move curr to next position
int length() const; // Return current length of list
void setPos(int); // Set curr to specified position
void setValue(const Elem); // Set current Elem's value
Elem currValue() const; // Return current Elem's value
bool isEmpty() const; // Return TRUE if list is empty
bool isInList() const; // TRUE if curr is within list
bool find(int); // Find value (from current position) };

24. Array based list class constructor
List::List(int sz) // Constructor: initialize {
msize = sz;
numinlist = 0;
curr = 0;
listarray = new Elem[sz]; }

25. Array based list class destructor
List::~List() // Destructor: return array space {
delete [] listarray; }

26. Array based list class clear
void List::clear() // Remove all Elems from list {
numinlist = 0; //reinitialize values
curr = 0; }

27. Array based list class setFirst
// Set curr to first position
void List::setFirst() {
curr = 0; }

28. Array based list class next
// Move curr to next position
void List::next() {
curr++; }

29. Array based list class previous
// Move curr to previous position
void List::prev() {
curr--; }

30. Array based list class setPosition
// Set curr to specified position
void List::setPos(int pos) {
curr = pos; }

31. Array based list class length
// Return current length of list
int List::length() const {
return numinlist; }

32. Array based list class isEmpty
// Return TRUE if list is empty
bool List::isEmpty() const {
return numinlist == 0; }

33. Array based list class isInList
// TRUE if curr is within list
bool List::isInList() const {
return (curr >= 0) && (curr < numinlist); }

34. Array based list class currentValue
// Return current Elem value
Elem List::currValue()const {
assert(isInList()); //Must be at a valid position
return listarray[curr]; }

35. Array based list class setValue
// Set current Elem value
void List::setValue(const Elem val) {
assert(isInList()); //Must be at a valid position
listarray[curr] = val; }

36. Array based list class append
// Insert Elem at tail of list
void List::append(const Elem item) {
assert(numinlist < msize); // List must not be full
listarray[numinlist++] = item; // Increment list size }

37. Array based list class insert
// Insert Elem at current position
void List::insert(const Elem item) {
// Array must not be full and
//curr must be a legal position
assert((numinlist < msize)&&(curr >=0)
&&(curr <= numinlist));
for(int i=numinlist; i>curr; i--) // Shift Elems up listarray[i] = listarray[i-1];
listarray[curr] = item;
numinlist++; // Increment current list size }

38. Array based list class remove
// Remove and return current Elem
Elem List::remove() {
// Must be an Elem to remove
assert(!isEmpty() && isInList());
Elem temp = listarray[curr]; // Store removed Elem
// Shift elements down
for(int i=curr; i<numinlist-1; i++)
listarray[i] = listarray[i+1];
numinlist--; // Decrement current list size
return temp; }

39. Array based list class find
// Find value (starting at curr)
bool List::find(int val) {
while (isInList())
{ // Stop if reach end
if (key(currValue()) == val) return TRUE; //Found it
else next(); }
return FALSE; // Not found

40. Linked-list based implementation
Elements stored in a linked list
A linked list is made up of series of objects called nodes of the list.
Each node contains a pointer to a next node in the list.
A list object contains pointers to
first (head) node
last (tail) node
current node

41. Current pointer Point to the current node
How do we insert at current position?
How do we delete last element in a list?
Point to the node preceding current node
What do we do if the current element is the first element in a list?
Use a header node

42. Class Link class Link { // Singly-linked node public:
Elem element; // Elem value for node
Link *next; // Pointer to next node
Link(const Elem elemval, Link* nextval =NULL)
{ element = elemval; next = nextval; }
Link(Link* nextval =NULL) { next = nextval; } };
Note: element does not have to be public

43. Linked list class - Interface
class List { // Linked list class
private:
Link* head; // Pointer to list header
Link* tail; // Pointer to last Elem
Link* curr; // Pos of "current" Elem
public:
List(); // Constructor
~List(); // Destructor
void clear(); // Remove all Elems
void insert(const Elem); // Insert at current pos
void append(const Elem); // Insert at tail of list
Elem remove(); // Remove/return Elem
void setFirst(); // Set curr to first pos
void prev(); // Move curr to prev pos
void next(); // Move curr to next pos
int length() const; // Return length
void setPos(int); // Set current pos
void setValue(const Elem); // Set current value
Elem currValue() const; // Return current value
bool isEmpty() const; // TRUE if list is empty
bool isInList() const; // TRUE if now in list
bool find(int); // Find value };

44. List class – Implementation constructor
List::List() {
head = new Link;
tail = head;
curr = head; }

45. List class – Implementation destructor
List::~List() {
while(head != NULL) { // Return link nodes
curr = head; //to free store
head = head->next;
delete curr; }

46. List class – Implementation clear
// Remove all Elems from list
void List::clear()
{ // Return link nodes
while (head->next != NULL) { //to free store
curr = head->next; // (keep header node)
head->next = curr->next;
delete curr; }
curr = head; tail = head; // Reinitialize

47. List class – Implementation setFirst
// Set curr to first position
void List::setFirst() {
curr = head; }

48. List class – Implementation next
// Move curr to next position
void List::next() {
if (curr != NULL)
curr = curr->next; }

49. List class – Implementation previous
// Move curr to previous position
void List::prev() {
Link* temp = head;
if ((curr == NULL) || (curr == head)) //No prev
curr = NULL;
return; //so just return }
while ((temp!=NULL) && (temp->next!=curr))
temp=temp->next;
curr = temp;

50. List class – Implementation setPosition
// Set curr to specified position
void List::setPos(int pos) {
curr = head;
for(int i=0; (curr!=NULL) && (i<pos); i++)
curr = curr->next; }

51. List class – Implementation length
// Return current length of list
int List::length() const {
int cnt = 0;
for (Link* temp = head->next; temp != NULL; temp = temp->next)
cnt++; // Count the number
//of Elems
return cnt; }

52. List class – Implementation isEmpty
// Return TRUE if list is empty
bool List::isEmpty() const {
return head->next == NULL; }

53. List class – Implementation isInList
// TRUE if curr is within list
bool List::isInList() const {
return (curr != NULL) && (curr->next != NULL); }

54. List class – Implementation currentValue
// Return value of current Elem
Elem List::currValue() const {
assert(isInList());
return curr->next->element; }

55. List class – Implementation setValue
// Set current Elem's value
void List::setValue(const Elem val) {
assert(isInList());
curr->next->element = val; }

56. List class – Implementation append
// Insert Elem at tail of list
void List::append(const Elem item) {
tail = tail->next = new Link(item, NULL); }

57. List class – Implementation insert
// Insert Elem at current position
void List::insert(const Elem item) {
assert(curr != NULL); // Must be pointing to Elem
curr->next = new Link(item, curr->next);
if (tail == curr) // Appended new Elem
tail = curr->next; }

58. List class – Implementation remove
// Remove/return Elem
Elem List::remove() {
assert(isInList()); // Must be valid pos
Elem temp = curr->next->element; //Remember val
Link* ltemp = curr->next; // Remember link
curr->next = ltemp->next; // Remove from list
if (tail == ltemp) tail = curr; // Set tail
delete ltemp; // Free link
return temp; // Return value }

59. List class – Implementation find
// Find value (starting at current)
bool List::find(int val) {
while (isInList())
if (key(curr->next->element) == val) return true;
else curr = curr->next; }
return false; // Not found

60. Comparison of List Implementations
Array based
Linked list based
Create a list
Destroy a list
Insert an element at current position
Append and element
Iterate thru all elements
Find an element
Remove an element
(1)
(1)
(n)
(1)
(n)
(1)
(1)
(1)
(n)
(n)
(n)
(n)
(n)
(1)