1. CSE 2010: Algorithms and Data Structures Algorithms
Summer 2017

2. Abstractions
In computer science, we're interested in devising efficient methods (i.e., algorithms) to solve problems.
Abstraction vs. implementation.
Spherical chickens in the vacuum
We use abstractions, e.g., algorithms.
Why not just use code? Why abstraction?
More intuitive for humans than “code” language
Help us focus on the problem and not the details of the implementation
- “Well, I found a solution to your problem with the chickens. But I had to consider them as spherical objects in the vacuum and with a uniform mass distribution…”

3. Why computer scientists need to study algorithms?
To know a standard set of important algorithms
To be able to tell how efficient algorithms are
To be able to recognize problems that can be solved with these algorithms
To help understand problems and their solutions
To help develop our analytical skills

4. What is an algorithm?
An algorithm is a finite sequence of unambiguous instructions for solving a problem, i.e., for obtaining the required output for any legitimate input in a finite amount of time.
[Levitin 2003]
It can also be seen as a transformation that transforms input into output.

5. Types of algorithms
Some try to classify algorithms according the problem-solving approach they use:
Iterative Algorithms
Direct/Indirect Recursive Algorithms
Divide-and-conquer Algorithms
Dynamic-Programming Algorithms
Randomized Algorithms
Brute-Force Algorithms

6. Selecting an algorithm to solve a problem
Someone arriving at the airport and she needs to get to your house. What algorithm could she use?
Get-a-Taxi Algorithm
Call-You Algorithm
Rent-a-Car Algorithm
Bus Algorithm
How can we compare these approaches?
What do they have in common?
How do they differ?
They have in common that they all produce the same result: The person arrives at the destination
They differ on cost and time it takes to get the answer.

7. Fundamental of Alg. Problem Solving
Algorithms are not answers to problems
They are a set of specific instructions that produce the answer
This emphasis on defining precise constructive procedures is what makes computer science distinct from other many other disciplines

8. Typical steps to designing algorithms
DRAW A
Understand the problem
Assess the capabilities of a computational device
Choose between exact and approximate approach
Decide on appropriate data structures
Algorithms + Data Structures = Programs [Wirth 76]
Implement any solution (it helps fulfill Step 1)
Generate various improved solutions
Select the most efficient solution
This emphasis on defining precise constructive procedures is what makes computer science distinct from other many other disciplines

9. Is the algorithm good enough?
Very important factors to consider:
How often are you going to need the algorithm?
What is the typical problem size you are trying to solve?
Less important factors that should also be considered:
What language are you going use to implement your algorithm?
What is the cost-benefit of an efficient algorithm?
Rule of thumb: we strive to find efficient algorithms.

10. Practice

11. Linear/sequential search

12. insertAt: Insert an element into an array

13. removeAt: Remove an element from an array

14. Our first algorithm to analyze: Insertion sort
We start with an empty left hand and the cards face down on the table.
We then remove one card at a time from the table and insert it into the correct position in the left hand. To find the correct position for a card, we compare it with each of the cards already in the hand, from right to left.
At all times, the cards held in the left hand are sorted, and these cards were originally the top cards of the pile on the table.

15. Insertion sort
Input numbers are sorted “in place”: the numbers are rearranged within the input array, i.e., array A.
Array indices appear above the rectangles
Values stored in array positions appear inside the rectangles
Black rectangle holds the key taken from A[j].
The value of the key is compared with the values in the shaded rectangles to the key’s left-hand side.
Shaded arrows show array values moved one position to the right (we need to create one space for the key to be inserted into the left hand).
The black arrows indicate the place into which the key will be moved.

16. Insertion sort
Index j indicates the “current card” being inserted into the hand.
At the beginning of each iteration of the for loop, which is indexed by j, the subarray consisting of elements A[1 .. j - 1] represents the currently “sorted hand”
The remaining subarray A[j+1 .. n] corresponds to the “pile of cards still on the table”.

17. Practice
Index j indicates the “current card” being inserted into the hand.
At the beginning of each iteration of the for loop, which is indexed by j, the subarray consisting of elements A[1 .. j - 1] represents the currently “sorted hand”
The remaining subarray A[j+1 .. n] corresponds to the “pile of cards still on the table”.

18. Analysis of insertion sort
The sorting time depends on the size of the input sequence.
If two equal-size sequences are partially sorted, then sorting time will be different for each sequence.
In general, the longer the sequence, the longer the sorting time.
For running time, we can simplify the analysis by defining a cost ci to represent the time it takes to complete the instruction i.

19. Analysis of insertion sort
Line 1: Repeat n times. Ask students why not n-1 since it starts from 2 and not from 1?
Line 2: n -1, okay.
Line 3: Ask students why cost = 0?
Line 4: This location is left of the key.
Line 5: tj is the number of times the while loop executes

20. Insertion sort
T(n) is the running time of the algorithm on an input of n values.
To calculate T(n), multiply each cost by the corresponding number of times the instruction will repeat. Sum the result.

21. Insertion sort Best case Sequence is already sorted.
Ask students: What does the loop in Line 5 do again?
Ask students: How many times will Line 5 repeat in this case?
Answer: Once. So tj = 1.

22. Insertion sort
Best case

23. Insertion sort Worst case Useful identities:
Array is in reverse sorted order—that is, in decreasing order.
We must compare each element with each element in the entire sorted subarray.
Useful identities:

24. Insertion sort
Worst case

25. Worst case vs. Average case
Next, we will look at the concept of rate of growth (or order of growth).
Average case is often as bad as the worst case.
When analyzing algorithms, we will mostly focus on the worst case.

26. Rate of growth: Example functions that often appear in algorithm analysis
Constant » 1 Logarithmic » log n
Linear » n N-Log-N » n log n
Quadratic » n2 Cubic » n3 Exponential » 2n
T(n) = n2
T(n) = n
T(n) = log2 n
T(n) = 1

27. Running time of Rate of growth: consider only the leading term
A bit more of abstraction… “ignoring” lower-order terms.
Running time of