Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Examples of problems with different time complexities Section 1.2: Comparing algorithms

Section 1: Problem Solving AQA Computing A2 © Nelson Thornes 2009 Time complexity In this presentation you will see examples of problems that have different time complexity. The examples will include problems that have: constant complexity linear complexity logarithmic complexity polynomial complexity (including quadratic and cubic complexity) exponential complexity factorial complexity.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Constant time – O(1) An algorithm has constant time complexity – O(1) – if the time taken does not grow no matter how the input n varies. An algorithm that finds out if a given number n is odd or even will have O(1) complexity – no matter how large n gets it will take the same number of operations to find out. In this case only one instruction is needed: If n mod 2 = 1 then n odd=true else odd=false.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Constant time – O(1) Not all algorithms with O(1) time complexity need only 1 instruction. O(1) time means it takes the same amount of time irrespective of how big the input is. Another example of an algorithm with O(1) time complexity is an algorithm that sorts the first 10 items in a list into order. No matter how large the list is, the algorithm will always take the same amount of time – sorting the first 10 items in a list of 10 items will take as long as sorting the first 10 items in a list of items.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Linear time – O(n) Algorithms of linear time complexity – O(n) – are less time efficient than O(1) complexity algorithms. As the input n gets bigger they take more time; the amount of time taken grows at the same rate as the increase in n.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Linear time – O(n) Mowing a lawn is an example of a problem that has linear time complexity. As the size, n, of the lawn increases the amount of time grows correspondingly. If a strip of lawn 10 metres long takes 2 minutes to mow: a strip of lawn 11 metres long would take 2 minutes 12 seconds 20 metres would take 4 minutes.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Linear time – O(n) A linear search algorithm has O(n) time complexity. If there are 10 items in the list it will take at most 10 comparisons to find the desired item in the list (or to find out it is not in the list). This is because a linear search compares each item in the list with the item being searched for and the desired item could be the last item in the list. A list of 11 items would take 11 comparisons at most; n items will take n comparisons at most.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Logarithmic time – O(log n) On sorted lists, there are faster searches than the linear search. An example would be a binary search. In this algorithm you start by looking at the middle item of the list. If this is not the item being searched for then you look at the middle item of the half of the list where the item being searched for will be found (if it is in the list). You keep on doing this until the item is found, or you discover the item is not in the list. The size of the list being searched through halves with each comparison.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Logarithmic time – O(log n) This means that finding an item will, on average, take far less time than finding an item with linear search. A binary search has logarithmic time complexity – O(log n) – as the size of the input n grows (i.e. size of the list) the time taken grows as well, but not as fast as the size of the list grows.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Polynomial time – O(n k ) Algorithms with Polynomial time complexity – O(n k ) – have the time they take increase at a faster rate than the rate of increase in the size of input n. Polynomial time complexity includes: Quadratic time complexity – O(n 2 ) and Cubic time complexity – O(n 3 ).

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Quadratic time – O(n 2 ) Here is an example of a problem that takes Quadratic time: Suppose that there is a group of people in a room who all need to shake hands with each other once. If there are 2 people (A and B) in the room handshake is needed.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Quadratic time – O(n 2 ) If there are 3 people (A, B and C) in the room then 3 handshakes are needed (AB, AC, BC).

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Quadratic time – O(n 2 ) If there are 4 people in the room (A, B, C, D) then 6 handshakes are needed (AB, AC, AD, BC, BD, CD)

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Quadratic time – O(n 2 ) 5 people need 10 handshakes 6 people need 15 handshakes 7 people need 21 handshakes n people need ½ (n 2 – n) handshakes

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Quadratic time – O(n 2 ) As you can see as the number of people increases, the number of handshakes needed increases by larger amounts. The formula ½ (n 2 -n) ≈ n 2 for large values of n; so the time complexity is O(n 2 ). Note: ≈ means approximately equal to.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Quadratic time – O(n 2 ) Bubble Sort is an example of an algorithm that has quadratic time complexity. Animation source: Nuno Nogueira

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Exponential time – O(k n ) Problems with Exponential time complexity – O(k n ) – can only be solved in a reasonable amount of time for small values of the input n. The story of ‘The Chinese Emperor and the Warlord’ is an example of a problem that has Exponential time complexity.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Exponential time – O(k n ) The Chinese Emperor was going to lose his throne to an invading army and asked for help from a neighbouring warlord.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Exponential time – O(k n ) The Warlord agreed to help if the pay was good enough. He said that he would accept either: ¼ of all of China’s rice crop produced over the next five years, or payment based on a chess board.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Exponential time – O(k n ) Payment based on a chess board meant: 1 grain of rice on the 1 st square 2 grains of rice on the 2 nd square 4 grains of rice on the 3 rd square 8 grains of rice on the 4 th square that the grains of rice placed on each square would be double that of the previous square.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Exponential time – O(k n ) The Emperor chose the chess board method of payment. The Warlord defeated the Emperor’s enemies and came to collect his payment. The Emperor started placing grains of rice on the squares of the chess board.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Exponential time – O(k n ) There were 255 grains placed on the first row. There were grains of rice on the second row of the chess board. There were grains of rice on the third row of the chess board. There would be several 100 times more grains of rice on the last square alone than is produced in the whole of China each year.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Exponential time – O(k n ) The Emperor abdicated in light of his huge debt and the Warlord became the new Emperor. Calculating the number of grains of rice in this problem takes Exponential time. As the number of squares increases, the time taken to calculate the number of grains of rice to place on the square grows exponentially.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Exponential problems – O(k n ) Exponential problems cannot be solved in a reasonable amount of time for anything but small inputs. Obtaining results for large inputs would take longer than the Earth will be around for.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Factorial time – O(n!) Problems that need factorial time – O(n!) – are even worse than O(k n ) problems. The algorithm to solve the Travelling Salesman Problem (TSP) using the brute force method (look at all possible routes to find the shortest) is O(n!). Look at the animation available on kerboodle! for more details.

Section 1: Problem solving AQA Computing A2 © Nelson Thornes 2009 Factorial time – O(n!) It is possible to solve the TSP in Exponential time, rather than Factorial time using a different method called Dynamic programming. But this still does not give solutions to the problem in a reasonable amount of time for any except the smallest versions of the problem.

Section 1: Problem Solving AQA Computing A2 © Nelson Thornes 2009 Summary You have now seen examples of problems that have different time complexity. Constant complexity O(1) – takes the same amount of time no matter how big the problem is. Linear complexity O(n) – as the size of the problem increases, the time taken to solve the problem grows at the same rate. Logarithmic complexity O(log n) – as the size of the problem grows, the time taken to solve the problem increases, but at a slower rate.

Section 1: Problem Solving AQA Computing A2 © Nelson Thornes 2009 Summary Polynomial complexity O(n k ) – as the size of the problem grows, the time taken to solve it grows faster, but can be solved in a reasonable amount of time. Exponential complexity O(k n ) – as the size of the problem grows, the time taken to solve it grows by larger and larger amounts. Only small size versions of these problems can be solved in a reasonable amount of time. Factorial complexity O(n!) – even worse than exponential.