CES 592 Theory of Software Systems B. Ravikumar (Ravi) Office: 124 Darwin Hall

Course Goals: tools for algorithm design graph searching, flow, matching,... linear programming dynamic programming randomization divide and conquer, induction,... tools for analysis recurrence relations, summation etc. reducibility experimental analysis algorithmic modeling formal systems and their applications logic grammars, automata

Goals for today’s lecture Introductions Answer the questionnaire Work on Quiz # 0 Overview of the course Begin review of Discrete Math

Algorithm Design and Analysis Efficient algorithms are a crucial part of software design tinkering with implementation and coding details can’t get us very far. root of the problem may be an inefficient algorithm. optimization problems allow multiple “correct” solutions, but only one correct and optimal solution. correctness of algorithms can be tricky to establish. predicting the behavior of algorithm is an important part of software design.

Some standard approaches to systematic algorithm design identify commonly occurring problems (such as sorting, searching, multiplication, breadth-first search etc.) and design the most efficient algorithms for them. devise general-purpose techniques (induction, divide and conquer, backtracking, dynamic programming etc.) and study when they are effective by looking at many examples. create and study models (such as trees, graphs, probabilistic algorithms) that can be used to represent problems and solutions.

Analysis of algorithms time taken to solve a problem as a function of problem size usually behaves as a well- defined function (logarithmic, linear, quadratic, n log n, exponential, etc.) when the time complexity function is complicated, it can be approximated to a well-defined function. problems can be classified based on the time complexity into easy (polynomial time solvable), provably intractable (not polynomial time solvable) and hard (not known to be polynomial time solvable, and not likely to be so).

Formal Systems and Languages Specification of software describing the syntax of a programming language (or even parts of natural language to a computer) automata applications string processing problems protocol specification and verification machine learning

Example: Arrange numbers 1, 2,..., 15 in a cyclic order so that the sum of every three consecutive numbers is at most 24. Why was 24 chosen? Can we solve this problem with 23 replacing 24? Systematically trying all permutations. Why is this not a good idea? The problem has multiple solutions. So we can try some random sequences. Algorithm for random permutation generation.

Course overview: Mathematical preliminaries induction summation, approximation, estimation inequalities, upper and lower bounds O,  and  notation recurrence relations review of discrete math Sorting and related problems quadratic time sorting algorithms heap sort and merge sort quick sort selection problem non-comparison based algorithms

Searching binary search tree AVL trees application to a geometric problem hashing Design Techniques divide and conquer dynamic programming greedy method probabilistic method backtracking genetic algorithms, neural networks etc.(?)

Graph Problems basic definitions and models DFS, BFS and applications minimum spanning tree shortest path problem matching in a bipartite graph Formal Systems and Applications finite automata context-free languages predicate logic applications

NP-completeness, decision trees etc. Lower-bound - models reductions Miscellaneous Topics string-matching cryptography parallel computers

Insertion Sorting Efficient algorithm for sorting a small number of elements. Insert(A,i) {// pre-condition: A[0.. i-1] is sorted. //post-condition: A[0.. i] is sorted. temp = A[i]; while (A[i-1] > temp) {A[i] = A[i-1]; i--;} A[i] = temp; } InsertionSort(A) { for (j = 2; j<= A.size; ++j) Insert(A,j) }

Our pseudo-code style is a little different from that of the text. (Ours will be a lot closer to Java or C++.) Analysis of InsertionSort: Focus on a specific operation. (key comparison). Text considers every operation. Measured as a function of parameter(s). In this case, it is the size of the array. Cost of the operation is a constant (1). Cases of interest: worst-case best-case average-case

Insert(A,i) {// pre-condition: A[0.. i-1] is sorted. //post-condition: A[0.. i] is sorted. temp = A[i]; while (A[i-1] > temp) {A[i] = A[i-1]; i--;} A[i] = temp; } worst-case: i best-case: 1 average: will do later. InsertionSort(A) { for (j = 1; j<= A.size; ++j) Insert(A,j) } worst-case: n(n-1)/2 best-case: n

Review of background material Mathematical Preliminaries induction - proof technique for assertion of the form “ for all integer n,... ” Example 1: For all n, n 3 + 5n is divisible by 6. Example r + … + r k = (r k+1 – 1)/(r-1) Proof by induction on k.

Inequalities, O,  and  notation For any positive real constants c and d, c n > n d for all sufficiently large n. There are positive constants c 1 and c 2 such that for all n, c 1 n log n <= log n! <= c 2 n log n. Definition: Let f(n) and g(n) be such that f(n) <= c g(n) for all large enough n. Then, we say f(n) = O(g(n)). If =, we have f(n) =  (g(n)). If both hold, f(n) =  (g(n)).

From previous slide, we have: log n! =  (n log n) Exercise: 1) 1 + 1/2+ 1/ /n =  (log n) 2) 1 + 1/ / /n 2 =  (1) 3) Estimate 1/2 + 2/ / n/2 n Summation formulas for arithmetic, geometric and arithmetic-geometric series