1. Algorithmic Efficency
How fast is that thing?

2. Scaling Up
I ask you to do something involving a list of students, it takes you 3 seconds. How long will the same task take you if I double the length of the list?

3. Scaling Up What if the task is:
Tell me if the list is more than one page long?
Find the highest score on the last assignment?
Sort a pile of student papers by last name?

4. First Question What do we care about? Usually a tradeoff…
Running time?
Memory required?
Programmer time?
Usually a tradeoff…

5. Measure Work
Time is usually most important but depends on lots of factors other than algorithm
Focus on work to handle problem size of n

6. Measuring Work
How many units of work?

7. Measuring Work
How many units of work? 1
4 ∙ 1 4 ∙ 1 10

8. Measuring Work
How many units of work?
4 ∙ 1 4 ∙ 1 8

9. Measuring Work
How many units of work? 1
4 ∙ 1
4 ∙ 1 4 ∙ 1 14

10. 8, 10, 14 all are constant do not depend on x
Measuring Work
How many units of work?
8, 10, 14 all are constant do not depend on x

11. Measuring Work
How many units of work? 1
x ∙ 1 x ∙ 1
2x + 2

12. Measuring Work
How many units of work?
x ∙ 1
x ∙ 1 x ∙ 1 3x

13. 2x + 2 and 3x are linear functions of x
Measuring Work
How many units of work?
2x + 2 and 3x are linear functions of x

14. Linear vs Constant
For large problems, any linear function is more expensive:

15. Big-O
Categorize based on class of largest term

16. Big-O No constants, only largest term Expression Category 5n
n : Linear
3n2
n2 : Quadratic
n2/2 + 6n
log2(n) + 6
log2(n) : Logarithmic 10
1 : Constant

17. Searches

18. Linear Search Linear search of 5 item list Best case: Worst case:
Average case: 12 39 20 24 27

19. Linear Search Linear search of 5 item list Best case: 1 Worst case: 5
Average case: 12 39 20 24 27

20. Linear Search Linear search of 5 item list Best case: 1 Worst case: 5
Average case: ( ) / 5 = 3 12 39 20 24 27

21. Linear Search Linear search of 100 item list Best case: 1
Worst case: 100
Average case: ( … ) / 100 = 50.5

22. Linear Search Linear search general:
Linear search is a linear algorithm

23. Binary Search Efficiency
Each step cuts search space in half

24. Cutting In Half Exponent: multiply this many 2's to get y y = 2x
Logarithm: y is the number of 2's you have to multiply to get x
y = log2x

25. Cutting In Half
Logarithm: y is the number of 2's you have to multiply to get x
y = log2x
log22 = 1
log216 = 4
log24 = 2
log2128 = 7
log28 = 3
log2100 = 6.64

26. Binary Search Efficiency
Binary Search is logarithmic : O(log(n))
Worst case for n items = log2n
Number Items 10 50
100
500
1000
10000
100000
Binary Worst Case 4 6 7 9 14 17

27. Comparison
Not even close…

28. Sorts

29. Selection Sort Selection Sort:

30. Selection Sort
Selection Sort:

31. Selection Sort
Selection Sort:
Comparisons:

32. Sum
What does this add to?

33. Sum
What does this add to?

34. 𝑛−1 2 𝑝𝑎𝑖𝑟𝑠 𝑛 𝑐𝑜𝑚𝑝𝑎𝑟𝑖𝑠𝑜𝑛𝑠 𝑝𝑒𝑟 𝑝𝑎𝑖𝑟 = 𝑛−1 2 𝑛 = 𝑛 2 2 − 𝑛 2
Sum
𝑛−1 2 𝑝𝑎𝑖𝑟𝑠 𝑛 𝑐𝑜𝑚𝑝𝑎𝑟𝑖𝑠𝑜𝑛𝑠 𝑝𝑒𝑟 𝑝𝑎𝑖𝑟 = 𝑛−1 2 𝑛 = 𝑛 2 2 − 𝑛 2

35. 𝑛−1 2 𝑝𝑎𝑖𝑟𝑠 𝑛 𝑐𝑜𝑚𝑝𝑎𝑟𝑖𝑠𝑜𝑛𝑠 𝑝𝑒𝑟 𝑝𝑎𝑖𝑟 = 𝑛−1 2 𝑛 = 𝑛 2 2 − 𝑛 2
Sum
𝑛−1 2 𝑝𝑎𝑖𝑟𝑠 𝑛 𝑐𝑜𝑚𝑝𝑎𝑟𝑖𝑠𝑜𝑛𝑠 𝑝𝑒𝑟 𝑝𝑎𝑖𝑟 = 𝑛−1 2 𝑛 = 𝑛 2 2 − 𝑛 2
Focus on just the biggest term: n2

36. Selection Sort
Selection Sort and Insertion Sort are Quadratic - O(n2)

37. Linear Search vs Binary Search vs Sorting

38. Estimation

39. Estimating
Time taken is proportional to work required:

40. Work and BigO Work depends on problem size and category 1000 items is
1 unit of work with constant algorithm
~10 units of work with logarithmic algorithm
1000 units of work with linear algorithm
1,000,000 units of work with quadratic algorithm

41. Sample
Selection sort on 10,000 takes seconds. How long will it take on 50,000 items

42. Sample
Selection sort on 10,000 takes seconds. How long will it take on 50,000 items
Selection sort does n2 work

43. Sample
Selection sort on 10,000 takes seconds. How long will it take on 50,000 items

44. Sample 2
Linear search on 10,000,000 items takes seconds. How long will binary search take?

45. Sample 2
Linear search on 10,000,000 items takes seconds. How long will binary search take?
Linear Binary search search does does log2(n) work n work

46. Sample 2
Linear search on 10,000,000 items takes seconds. How long will binary search take?