CES 512 Theory of Software Systems B. Ravikumar (Ravi) Office: 141 Darwin Hall Course Web site:

Course Goals: develop critical thinking about problem solving (in the context of software design) emphasis on innovation, avoiding dull and routine line of thinking whenever possible. become exposed to interesting problems (some for pure fun, others for purpose). focus on models – understanding the context of a problem: specialization, generalization. tools to create algorithmic solutions, implement and test etc.

Context of the course in Master’s program MS-CES I created this course mainly because I felt that the students who want to do a thesis in Software Design need to know this course material. (More specifically, the thesis topics I would like to supervise will require background similar to this.) The emphasis (creative thinking, awareness of current trends and developments etc.) is based on my belief that one can excel only by consciously demanding high expectation of him/herself. Emphasis on theoretical ideas and models. But at every stage, we will examine the relevance of the model to practical applications.

Precursors to this course at other institutions Algorithms in the Real-World (CMU) Applied Algorithms (University of Washington). Algorithm Engineering. International conferences and workshops in which modeling, theory (mathematical analysis) as well as implementations and engineering issues are given due emphasis

Course Goals: tools for algorithm design divide and conquer, induction graph searching, network flow, matching linear programming dynamic programming scheduling and other optimization problems heuristics and NP-completeness tools for analysis recurrence relations, summation of series approximations, inequalities reductions between problems experimental analysis

algorithmic modeling case studies formal systems and their applications logic, grammars, automata etc.

Goals for today’s lecture Course outline Discuss Course work homework quiz tests, final exam course project Some examples of programming/algorithm design problems Cover Chapters 1-3 of the text.

Algorithm Design and Analysis Efficient algorithms are a crucial part of software design tinkering with implementation and coding details can’t get us very far. root of the problem may be an inefficient algorithm. optimization problems allow multiple “correct” solutions, but only one correct and optimal solution. correctness of algorithms can be tricky to establish. predicting the behavior of algorithm is an important part of software design.

Some standard approaches to systematic algorithm design identify commonly occurring problems (such as sorting, searching, multiplication, breadth-first search etc.) and design the most efficient algorithms for them. devise general-purpose techniques (induction, divide and conquer, backtracking, dynamic programming etc.) and study when they are effective by looking at many examples. create and study models (such as trees, graphs, probabilistic algorithms) that can be used to represent problems and solutions.

Analysis of algorithms time taken to solve a problem as a function of problem size usually behaves as a well-defined function (logarithmic, linear, quadratic, n log n, exponential, etc.) when the time complexity function is complicated, it can be approximated to a well-defined function. problems can be classified based on the time complexity into easy (polynomial time solvable), provably intractable (not polynomial time solvable) and hard (not known to be polynomial time solvable, and not likely to be so).

Sample Problems 1.How to compress a given text? 2.How to enhance the quality of a digital photograph ? 3.How is a program like mapquest implemented? 4.GRE recently introduced a computer test that adaptively chooses questions based on how well the candidate has performed thus far. How should the questions be selected to make the test fair and reliable? 5.Based on the performance of N teams playing against each other, we want to rank the teams. How should we go about this process?

Course overview: Mathematical preliminaries induction summation, approximation, estimation inequalities, upper and lower bounds O,  and  notation recurrence relations review of discrete math Sorting and related problems quadratic time sorting algorithms heap sort and merge sort quick sort selection problem non-comparison based algorithms

Searching binary search tree AVL trees application to a geometric problem hashing Design Techniques divide and conquer dynamic programming greedy method probabilistic method backtracking genetic algorithms, neural networks etc.(?)

Graph Problems basic definitions and models DFS, BFS and applications minimum spanning tree shortest path problem matching in a bipartite graph Network flow Linear Programming simplex method duality applications

Heuristics genetic algorithms simulated annealing Applications cryptography bioinformatics

Insertion Sorting Efficient algorithm for sorting a small number of elements. Insert(A,i) {// pre-condition: A[0.. i-1] is sorted. //post-condition: A[0.. i] is sorted. temp = A[i]; while (A[i-1] > temp) {A[i] = A[i-1]; i--;} A[i] = temp; } InsertionSort(A) { for (j = 2; j<= A.size; ++j) Insert(A,j) }

Our pseudo-code style is a little different from that of the text. (Ours will be a lot closer to Java or C++.) Analysis of InsertionSort: Focus on a specific operation. (key comparison). Text considers every operation. Measured as a function of parameter(s). In this case, it is the size of the array. Cost of the operation is a constant (1). Cases of interest: worst-case best-case average-case

Insert(A,i) {// pre-condition: A[0.. i-1] is sorted. //post-condition: A[0.. i] is sorted. temp = A[i]; while (A[i-1] > temp) {A[i] = A[i-1]; i--;} A[i] = temp; } worst-case: i best-case: 1 average: will do later. InsertionSort(A) { for (j = 1; j<= A.size; ++j) Insert(A,j) } worst-case: n(n-1)/2 best-case: n

Review of background material Mathematical Preliminaries induction - proof technique for assertion of the form “ for all integer n,... ” Example 1: For all n, n 3 + 5n is divisible by 6. Example r + … + r k = (r k+1 – 1)/(r-1) Proofs will be presented in class

Inequalities, O,  and  notation For any positive real constants c and d, c n > n d for all sufficiently large n. There are positive constants c 1 and c 2 such that for all n, c 1 n log n <= log n! <= c 2 n log n. Definition: Let f(n) and g(n) be such that f(n) <= c g(n) for all large enough n. Then, we say f(n) = O(g(n)). If =, we have f(n) =  (g(n)). If both hold, f(n) =  (g(n)).

From previous slide, we have: log n! =  (n log n) Exercise: 1) 1 + 1/2+ 1/ /n =  (log n) 2) 1 + 1/ / /n 2 =  (1) 3) Estimate 1/2 + 2/ / n/2 n Summation formulas for arithmetic, geometric and arithmetic-geometric series