CSE 2010: Algorithms and Data Structures Algorithms Summer 2017

Abstractions In computer science, we're interested in devising efficient methods (i.e., algorithms) to solve problems. Abstraction vs. implementation. Spherical chickens in the vacuum We use abstractions, e.g., algorithms. Why not just use code? Why abstraction? More intuitive for humans than “code” language Help us focus on the problem and not the details of the implementation http://nossotradeoff.wordpress.com/2013/02/14/spherical-chickens-in-the-vacum/ - “Well, I found a solution to your problem with the chickens. But I had to consider them as spherical objects in the vacuum and with a uniform mass distribution…”

Why computer scientists need to study algorithms? To know a standard set of important algorithms To be able to tell how efficient algorithms are To be able to recognize problems that can be solved with these algorithms To help understand problems and their solutions To help develop our analytical skills

What is an algorithm? An algorithm is a finite sequence of unambiguous instructions for solving a problem, i.e., for obtaining the required output for any legitimate input in a finite amount of time. [Levitin 2003] It can also be seen as a transformation that transforms input into output.

Types of algorithms Some try to classify algorithms according the problem-solving approach they use: Iterative Algorithms Direct/Indirect Recursive Algorithms Divide-and-conquer Algorithms Dynamic-Programming Algorithms Randomized Algorithms Brute-Force Algorithms

Selecting an algorithm to solve a problem Someone arriving at the airport and she needs to get to your house. What algorithm could she use? Get-a-Taxi Algorithm Call-You Algorithm Rent-a-Car Algorithm Bus Algorithm How can we compare these approaches? What do they have in common? How do they differ? They have in common that they all produce the same result: The person arrives at the destination They differ on cost and time it takes to get the answer.

Fundamental of Alg. Problem Solving Algorithms are not answers to problems They are a set of specific instructions that produce the answer This emphasis on defining precise constructive procedures is what makes computer science distinct from other many other disciplines

Typical steps to designing algorithms DRAW A Understand the problem Assess the capabilities of a computational device Choose between exact and approximate approach Decide on appropriate data structures Algorithms + Data Structures = Programs [Wirth 76] Implement any solution (it helps fulfill Step 1) Generate various improved solutions Select the most efficient solution This emphasis on defining precise constructive procedures is what makes computer science distinct from other many other disciplines http://www.imagethink.net/imagethink-2/how-to-draw-a-robot-in-5-easy-steps/

Is the algorithm good enough? Very important factors to consider: How often are you going to need the algorithm? What is the typical problem size you are trying to solve? Less important factors that should also be considered: What language are you going use to implement your algorithm? What is the cost-benefit of an efficient algorithm? Rule of thumb: we strive to find efficient algorithms. http://www.nature.com/srep/2012/120210/srep00260/fig_tab/srep00260_F3.html

Practice

Linear/sequential search

insertAt: Insert an element into an array http://www.algolist.net/Data_structures/Dynamic_array/Access_functions

removeAt: Remove an element from an array http://www.algolist.net/Data_structures/Dynamic_array/Access_functions

Our first algorithm to analyze: Insertion sort We start with an empty left hand and the cards face down on the table. We then remove one card at a time from the table and insert it into the correct position in the left hand. To find the correct position for a card, we compare it with each of the cards already in the hand, from right to left. At all times, the cards held in the left hand are sorted, and these cards were originally the top cards of the pile on the table.

Insertion sort Input numbers are sorted “in place”: the numbers are rearranged within the input array, i.e., array A. Array indices appear above the rectangles Values stored in array positions appear inside the rectangles Black rectangle holds the key taken from A[j]. The value of the key is compared with the values in the shaded rectangles to the key’s left-hand side. Shaded arrows show array values moved one position to the right (we need to create one space for the key to be inserted into the left hand). The black arrows indicate the place into which the key will be moved.

Insertion sort Index j indicates the “current card” being inserted into the hand. At the beginning of each iteration of the for loop, which is indexed by j, the subarray consisting of elements A[1 .. j - 1] represents the currently “sorted hand” The remaining subarray A[j+1 .. n] corresponds to the “pile of cards still on the table”.

Practice Index j indicates the “current card” being inserted into the hand. At the beginning of each iteration of the for loop, which is indexed by j, the subarray consisting of elements A[1 .. j - 1] represents the currently “sorted hand” The remaining subarray A[j+1 .. n] corresponds to the “pile of cards still on the table”.

Analysis of insertion sort The sorting time depends on the size of the input sequence. If two equal-size sequences are partially sorted, then sorting time will be different for each sequence. In general, the longer the sequence, the longer the sorting time. For running time, we can simplify the analysis by defining a cost ci to represent the time it takes to complete the instruction i.

Analysis of insertion sort Line 1: Repeat n times. Ask students why not n-1 since it starts from 2 and not from 1? Line 2: n -1, okay. Line 3: Ask students why cost = 0? Line 4: This location is left of the key. Line 5: tj is the number of times the while loop executes

Insertion sort T(n) is the running time of the algorithm on an input of n values. To calculate T(n), multiply each cost by the corresponding number of times the instruction will repeat. Sum the result.

Insertion sort Best case Sequence is already sorted. Ask students: What does the loop in Line 5 do again? Ask students: How many times will Line 5 repeat in this case? Answer: Once. So tj = 1.

Insertion sort Best case

Insertion sort Worst case Useful identities: Array is in reverse sorted order—that is, in decreasing order. We must compare each element with each element in the entire sorted subarray. Useful identities:

Insertion sort Worst case

Worst case vs. Average case Next, we will look at the concept of rate of growth (or order of growth). Average case is often as bad as the worst case. When analyzing algorithms, we will mostly focus on the worst case.

Rate of growth: Example functions that often appear in algorithm analysis Constant » 1 Logarithmic » log n Linear » n N-Log-N » n log n Quadratic » n2 Cubic » n3 Exponential » 2n T(n) = n2 T(n) = n T(n) = log2 n T(n) = 1

Running time of Rate of growth: consider only the leading term A bit more of abstraction… “ignoring” lower-order terms. Running time of