Basics of Algorithm Analysis Asymptotic algorithm analysis Growth rate Upper bounds of growth rate Lower bounds of growth rate  Notation Space complexity Tradeoffs of implementations Analyzing Problems – Optimal solution

Asymptotic Analysis Asymptotic analysis – study of an algorithm as the input size “gets big” or reaches the limit Advantages and limitations

Growth rate Growth rate - the rate at which the cost of an algorithm grows as the size of the input grows Growth rate enables us to compare different algorithms that can be used for the same problem

Upper bounds of growth rate Big-Oh notation - O Definition T(n) is in the set O(f(n)) if there exists two positive constants c and n0 such that | T(n)|  c|f(n)| for all n > n0 Meaning Algorithm has upper bound to its growth rate of f(n)

Examples Example 1 – Sequential search algorithm T(n) = c1(n+1)/2 |c1n/2|  c1|n| for all n > 1 T(n) is in O(n) for n0 = 1 and c = c1 Example 2 – Assignment operation T(n) = c2 T(n) is in O(c2) – T(n) is in O(1) Example 3 – T(n) = c3n2 + c4n |c3n2 + c4n|  |c3n2 + c4n2|  (c3 + c4)|n2| for all n > 1 T(n) is in O(n2) for n0 = 1 and c = c3 + c4

Lower bounds of growth rate Big-Omega notation -  Definition T(n) is in the set (f(n)) if there exists two positive constants c and n0 such that | T(n)|  c|f(n)| for all n > n0 Meaning Algorithm has lower bound to its growth rate of f(n)

Examples Example 1 – Sequential search algorithm T(n) = c1n/2 |c1n/2|  c1/2|n| for all n > 1 T(n) is in (n) for n0 = 1 and c = c1/2 Example 2 – Assignment operation T(n) = c2 T(n) is in (c2) – T(n) is in (1) Example 3 – T(n) = c3n2 + c4n |c3n2 + c4n|  c3|n2| for all n > 1 T(n) is in (n2) for n0 = 1 and c = c3

 analysis Theta notation -  Definition Meaning An algorithm is said to be  (f(n)) if it is in the set O(f(n)) and it is in the set in the set (f(n)) Meaning When upper bounds and lower bounds are the same we indicate this using  notation

Example while (n<0) { if (ODD(n)) n = 3 * n + 1; else n = n / 2; }

Examples Example 1 – Sequential search algorithm T(n) is in (n) T(n) is in O(n) T(n) is (n) Example 2 – Assignment operation T(n) is in (1) T(n) is in O(1) T(n) is (1) Example 3 – T(n) = c3n2 + c4n T(n) is in (n2) T(n) is in O(n2) T(n) is (n2)

Simplifying Rules If f(n) is in O(g(n)) and g(n) is in O(h(n)), then f(n) is in O(h(n)). [Bounds are not unique] If f(n) is in O(kg(n)) for any constant k >0, then f(n) is in O(g(n)). [No constant] If f1 (n) is in O(g1 (n)) and f2 (n) is in O(g2 (n)), then (f1 + f2 )(n) is in O(max(g1 (n), g2 (n))). [Drop low order terms] If f1 (n) is in O(g1 (n)) and f2 (n) is in O(g2 (n)) then f1 (n) * f2 (n) is in O(g1 (n) * g2 (n)). [Multiply number of actions by cost of each action]

Multiple parameters for (int i=0; i<C; i++) sumC++; for (int j=0; j<P; j++) sumP++; If we use P as the measure, then time is (P) If we use C as the measure, then time is (C) More accurate is (P + C)

Space Bounds The concept of asymptotic analysis of growth rate completely applies to measuring space requirements Example – one-dimensional array (n) Example – two-dimensional array (n2)

Space/Time tradeoffs Space/Time tradeoff principle One can often achieve a reduction in time is one is willing to sacrifice space, or vice versa Disk Based Space/Time tradeoff principle: The smaller you can make your disk storage requirements, the faster your program will run.

Analyzing Problems – Optimal solution Given a problem, there are many possible algorithms to solve the problem. We have a set of solution algorithms. We usually don't have an enumeration of this set. We are interested to find an optimal solution that take the least running time.   By designing and implementing a correct solution to the problem, we constructively prove an upper bound to the running time of an optimal solution. By improving the existing algorithms and designing new ones, we keep push the upper bound down.

Analyzing Problems – Optimal solution On the another hand, we observe some inherent properties of the problem (not the algorithms), and conclude that no solutions to the problem can have a running time less than a particular lower bound of the running time. We, thus, prove a lower bound to the running time of the optimal solution. Theorists keep studying the properties of the problem and pushing the lower bound up.   When an algorithm has a running time on the same order as the lower bound of the solution to the problem, the upper bound and the lower bound meet. We can then declare that we have found an optimal solution. 

Fundamental Data Structures List ADT Implementations of List ADT Array based lists Linked lists Comparisons of two implementations Freelists

List ADT A list is finite, ordered sequence of data items known as elements Each element has a position in the list (first, second, etc.) A list is said to be empty when it contains no elements The number of elements currently stored is called length of the list The beginning of the list is called head The end of the list is called tail

Operations on list ADT Create a list Insert an element Remove an element Access an element Clear (remove all elements) Access an element next or previous to the currently accessed element

Specification of List ADT class List { // list ADT private: // list Elements public: List(int =LIST_SIZE); // Constructor ~List(); // Destructor void clear(); // Remove all Elems from list void insert(const Elem); // Insert Elem at current position void append(const Elem); // Insert Elem at tail of list Elem remove(); // Remove and return current Elem void setFirst(); // Set curr to first position void prev(); // Move curr to previous position void next(); // Move curr to next position int length() const; // Return current length of list void setPos(int); // Set curr to specified position void setValue(const Elem); // Set current Elem's value Elem currValue() const; // Return current Elem's value bool isEmpty() const; // Return TRUE if list is empty bool isInList() const; // TRUE if curr is within list bool find(int); // Find value (from current position) };

Implementations of list Array-based implementation Uses an array to store list elements Linked-list based implementation Uses a linked list to store list elements

Array based implementation Elements stored in an array in continuous positions Maximum size of the list must be known before the list is created Head of the list is at position 0

Array based list class class List { // Array-based list class private: int msize; // Maximum size of list int numinlist; // Actual number of Elems in list int curr; // Position of "current" Elem Elem* listarray; // Array holding list Elems public: List(int =LIST_SIZE); // Constructor ~List(); // Destructor void clear(); // Remove all Elems from list void insert(const Elem); // Insert Elem at current position void append(const Elem); // Insert Elem at tail of list Elem remove(); // Remove and return current Elem void setFirst(); // Set curr to first position void prev(); // Move curr to previous position void next(); // Move curr to next position int length() const; // Return current length of list void setPos(int); // Set curr to specified position void setValue(const Elem); // Set current Elem's value Elem currValue() const; // Return current Elem's value bool isEmpty() const; // Return TRUE if list is empty bool isInList() const; // TRUE if curr is within list bool find(int); // Find value (from current position) };

Array based list class constructor List::List(int sz) // Constructor: initialize { msize = sz; numinlist = 0; curr = 0; listarray = new Elem[sz]; }

Array based list class destructor List::~List() // Destructor: return array space { delete [] listarray; }

Array based list class clear void List::clear() // Remove all Elems from list { numinlist = 0; //reinitialize values curr = 0; }

Array based list class setFirst // Set curr to first position void List::setFirst() { curr = 0; }

Array based list class next // Move curr to next position void List::next() { curr++; }

Array based list class previous // Move curr to previous position void List::prev() { curr--; }

Array based list class setPosition // Set curr to specified position void List::setPos(int pos) { curr = pos; }

Array based list class length // Return current length of list int List::length() const { return numinlist; }

Array based list class isEmpty // Return TRUE if list is empty bool List::isEmpty() const { return numinlist == 0; }

Array based list class isInList // TRUE if curr is within list bool List::isInList() const { return (curr >= 0) && (curr < numinlist); }

Array based list class currentValue // Return current Elem value Elem List::currValue()const { assert(isInList()); //Must be at a valid position return listarray[curr]; }

Array based list class setValue // Set current Elem value void List::setValue(const Elem val) { assert(isInList()); //Must be at a valid position listarray[curr] = val; }

Array based list class append // Insert Elem at tail of list void List::append(const Elem item) { assert(numinlist < msize); // List must not be full listarray[numinlist++] = item; // Increment list size }

Array based list class insert // Insert Elem at current position void List::insert(const Elem item) { // Array must not be full and //curr must be a legal position assert((numinlist < msize)&&(curr >=0) &&(curr <= numinlist)); for(int i=numinlist; i>curr; i--) // Shift Elems up listarray[i] = listarray[i-1]; listarray[curr] = item; numinlist++; // Increment current list size }

Array based list class remove // Remove and return current Elem Elem List::remove() { // Must be an Elem to remove assert(!isEmpty() && isInList()); Elem temp = listarray[curr]; // Store removed Elem // Shift elements down for(int i=curr; i<numinlist-1; i++) listarray[i] = listarray[i+1]; numinlist--; // Decrement current list size return temp; }

Array based list class find // Find value (starting at curr) bool List::find(int val) { while (isInList()) { // Stop if reach end if (key(currValue()) == val) return TRUE; //Found it else next(); } return FALSE; // Not found

Linked-list based implementation Elements stored in a linked list A linked list is made up of series of objects called nodes of the list. Each node contains a pointer to a next node in the list. A list object contains pointers to first (head) node last (tail) node current node

Current pointer Point to the current node How do we insert at current position? How do we delete last element in a list? Point to the node preceding current node What do we do if the current element is the first element in a list? Use a header node

Class Link class Link { // Singly-linked node public: Elem element; // Elem value for node Link *next; // Pointer to next node Link(const Elem elemval, Link* nextval =NULL) { element = elemval; next = nextval; } Link(Link* nextval =NULL) { next = nextval; } }; Note: element does not have to be public

Linked list class - Interface class List { // Linked list class private: Link* head; // Pointer to list header Link* tail; // Pointer to last Elem Link* curr; // Pos of "current" Elem public: List(); // Constructor ~List(); // Destructor void clear(); // Remove all Elems void insert(const Elem); // Insert at current pos void append(const Elem); // Insert at tail of list Elem remove(); // Remove/return Elem void setFirst(); // Set curr to first pos void prev(); // Move curr to prev pos void next(); // Move curr to next pos int length() const; // Return length void setPos(int); // Set current pos void setValue(const Elem); // Set current value Elem currValue() const; // Return current value bool isEmpty() const; // TRUE if list is empty bool isInList() const; // TRUE if now in list bool find(int); // Find value };

List class – Implementation constructor List::List() { head = new Link; tail = head; curr = head; }

List class – Implementation destructor List::~List() { while(head != NULL) { // Return link nodes curr = head; //to free store head = head->next; delete curr; }

List class – Implementation clear // Remove all Elems from list void List::clear() { // Return link nodes while (head->next != NULL) { //to free store curr = head->next; // (keep header node) head->next = curr->next; delete curr; } curr = head; tail = head; // Reinitialize

List class – Implementation setFirst // Set curr to first position void List::setFirst() { curr = head; }

List class – Implementation next // Move curr to next position void List::next() { if (curr != NULL) curr = curr->next; }

List class – Implementation previous // Move curr to previous position void List::prev() { Link* temp = head; if ((curr == NULL) || (curr == head)) //No prev curr = NULL; return; //so just return } while ((temp!=NULL) && (temp->next!=curr)) temp=temp->next; curr = temp;

List class – Implementation setPosition // Set curr to specified position void List::setPos(int pos) { curr = head; for(int i=0; (curr!=NULL) && (i<pos); i++) curr = curr->next; }

List class – Implementation length // Return current length of list int List::length() const { int cnt = 0; for (Link* temp = head->next; temp != NULL; temp = temp->next) cnt++; // Count the number //of Elems return cnt; }

List class – Implementation isEmpty // Return TRUE if list is empty bool List::isEmpty() const { return head->next == NULL; }

List class – Implementation isInList // TRUE if curr is within list bool List::isInList() const { return (curr != NULL) && (curr->next != NULL); }

List class – Implementation currentValue // Return value of current Elem Elem List::currValue() const { assert(isInList()); return curr->next->element; }

List class – Implementation setValue // Set current Elem's value void List::setValue(const Elem val) { assert(isInList()); curr->next->element = val; }

List class – Implementation append // Insert Elem at tail of list void List::append(const Elem item) { tail = tail->next = new Link(item, NULL); }

List class – Implementation insert // Insert Elem at current position void List::insert(const Elem item) { assert(curr != NULL); // Must be pointing to Elem curr->next = new Link(item, curr->next); if (tail == curr) // Appended new Elem tail = curr->next; }

List class – Implementation remove // Remove/return Elem Elem List::remove() { assert(isInList()); // Must be valid pos Elem temp = curr->next->element; //Remember val Link* ltemp = curr->next; // Remember link curr->next = ltemp->next; // Remove from list if (tail == ltemp) tail = curr; // Set tail delete ltemp; // Free link return temp; // Return value }

List class – Implementation find // Find value (starting at current) bool List::find(int val) { while (isInList()) if (key(curr->next->element) == val) return true; else curr = curr->next; } return false; // Not found

Comparison of List Implementations Array based Linked list based Create a list Destroy a list Insert an element at current position Append and element Iterate thru all elements Find an element Remove an element (1) (1) (n) (1) (n) (1) (1) (1) (n) (n) (n) (n) (n) (1)