1. Computing Science 1P Lecture 21: Wednesday 18 th April Simon Gay Department of Computing Science University of Glasgow 2006/07

2. Computing Science 1P Lecture 21 - Simon Gay2 What's coming up? Wed 18 th April (today):lecture Fri 20 th April:lecture Mon 23 rd April, 12.00-14.00:FPP demo session; PRIZES! Mon 23 rd – Wed 25 th April:labs: exam preparation / lab exam preparation Wed 25 th April:lecture / tutorial Friday 27 th April:lecture: revision / questions with Peter Saffrey Mon 30 th April – Wed 2 nd May:Lab Exam Wed 2 nd May:No lecture Fri 4 th May:No lecture THE END

3. 2006/07Computing Science 1P Lecture 21 - Simon Gay3 Lab Exam 2 As you know, there will be a second lab exam, during the week beginning 30 th April. It is worth 10% of the module mark. The question has been available since Monday afternoon. In the first lab exam, although there were many very good submissions, a worrying number of submissions showed very little evidence of advance preparation. You can prepare for the lab exam in any way you want to, including asking a friend to explain the solution to you. On the day, of course, you are on your own. Trying to solve the problem from scratch in the exam is just making life difficult for yourself.

4. 2006/07Computing Science 1P Lecture 21 - Simon Gay4 Algorithms We have looked at algorithms for sorting; we saw that choosing a better algorithm can have a dramatic effect on the efficiency of a program. Now let's consider searching, another basic computing task. The general problem: find a desired item of data in a collection. Example: in a collection of (name,address) pairs, find a particular person's address.

5. 2006/07Computing Science 1P Lecture 21 - Simon Gay5 Searching in unstructured data Imagine that we have a list of (key,value) pairs and we do not know anything about the order. We can easily define: def find(key,data): for i in data: if i[0] == key: return i[1] raise "KeyNotFound",key

6. 2006/07Computing Science 1P Lecture 21 - Simon Gay6 Searching in unstructured data What can we say about the time taken by find ? Just as for sorting, the relevant measure is the number of comparisons. Clearly it is possible that the key we are looking for is at the end of the list. In that case we have to compare the given key with every key in the list. If we imagine testing find repeatedly with a large number of random lists, on average it will have to search half way along the list. When analysing algorithms, sometimes we talk about the average case and sometimes the worst case. In this situation they are both the same: order n, where n is the length of the list.

7. 2006/07Computing Science 1P Lecture 21 - Simon Gay7 Searching in unstructured data It's obvious that we can't do better than order n for searching in an unstructured list, because we can't avoid the possibility that the desired key is at the end. Remarkably, there is an algorithm for quantum computers which only takes square root of n operations to search in an unstructured list. However, quantum computers of a useful size have not yet been built. To find out more, look up Grover's algorithm. But let's stick to conventional algorithms…

8. 2006/07Computing Science 1P Lecture 21 - Simon Gay8 More efficient search If we can't improve the algorithm for search in an unstructured list, the only alternative is to change the data structure: don't use an unstructured list! The first idea is quite simple: use an ordered list instead. In other words, put the data in the list in such a way that the keys are in order. Often this means alphabetical order or numerical order, but other more complex orders could be defined. Example: in a dictionary, the words are in alphabetical order, and we can take advantage of this to find words quickly.

9. 2006/07Computing Science 1P Lecture 21 - Simon Gay9 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat

10. 2006/07Computing Science 1P Lecture 21 - Simon Gay10 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat It could be anywhere in the list.

11. 2006/07Computing Science 1P Lecture 21 - Simon Gay11 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat It could be anywhere in the list. The list has length 12. Divide it by 2 and look at position 6.

12. 2006/07Computing Science 1P Lecture 21 - Simon Gay12 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat It could be anywhere in the list. The list has length 12. Divide it by 2 and look at position 6. cat < garage

13. 2006/07Computing Science 1P Lecture 21 - Simon Gay13 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Because the list is ordered, we now know that cat must be before garage, i.e. it is in the first half of the list.

14. 2006/07Computing Science 1P Lecture 21 - Simon Gay14 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Because the list is ordered, we now know that cat must be before garage, i.e. it is in the first half of the list.

15. 2006/07Computing Science 1P Lecture 21 - Simon Gay15 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 6.

16. 2006/07Computing Science 1P Lecture 21 - Simon Gay16 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 6. Divide by 2 and look at position 3.

17. 2006/07Computing Science 1P Lecture 21 - Simon Gay17 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 6. Divide by 2 and look at position 3. cat < door

18. 2006/07Computing Science 1P Lecture 21 - Simon Gay18 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be before door.

19. 2006/07Computing Science 1P Lecture 21 - Simon Gay19 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be before door.

20. 2006/07Computing Science 1P Lecture 21 - Simon Gay20 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 3.

21. 2006/07Computing Science 1P Lecture 21 - Simon Gay21 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 3. Divide by 2 and look at position 1.

22. 2006/07Computing Science 1P Lecture 21 - Simon Gay22 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat Now repeat, searching in a list of length 3. Divide by 2 and look at position 1. cat > badger

23. 2006/07Computing Science 1P Lecture 21 - Simon Gay23 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be after badger.

24. 2006/07Computing Science 1P Lecture 21 - Simon Gay24 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be after badger. We have narrowed down the possible position of cat to just one place. And in fact cat is there, so we have found it.

25. 2006/07Computing Science 1P Lecture 21 - Simon Gay25 Binary search android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: cat We now know that cat must be after badger. We have narrowed down the possible position of cat to just one place. And in fact cat is there, so we have found it. If a different word is there, then cat is not in the list.

26. 2006/07Computing Science 1P Lecture 21 - Simon Gay26 Binary search The idea of binary search is very simple, but implementing it correctly requires care: there are many possibilities for "off by one" errors. Searching in a dictionary is often used as an example of binary search, but we don't really use dictionaries in exactly this way. Usually we flick through the pages quickly to find the right letter, then do something similar to binary search. A typical dictionary has extra structure to support this process (e.g. words in the page headers; thumbholes for indexing).

27. 2006/07Computing Science 1P Lecture 21 - Simon Gay27 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle

28. 2006/07Computing Science 1P Lecture 21 - Simon Gay28 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle

29. 2006/07Computing Science 1P Lecture 21 - Simon Gay29 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle

30. 2006/07Computing Science 1P Lecture 21 - Simon Gay30 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle

31. 2006/07Computing Science 1P Lecture 21 - Simon Gay31 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle

32. 2006/07Computing Science 1P Lecture 21 - Simon Gay32 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle We could stop here, but if we follow the algorithm strictly, we continue dividing the region in two

33. 2006/07Computing Science 1P Lecture 21 - Simon Gay33 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle

34. 2006/07Computing Science 1P Lecture 21 - Simon Gay34 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle

35. 2006/07Computing Science 1P Lecture 21 - Simon Gay35 Another example android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 Search for the key: handle Now we can certainly stop

36. 2006/07Computing Science 1P Lecture 21 - Simon Gay36 Analysing binary search Remember that we are interested in the number of comparisons. Suppose that we are searching in a list of length n. We compare the middle item with the search key. The result might tell us we have found the key, but in general it narrows down the region of the list in which we are searching. The possible region of the list is now half the size it was.

37. 2006/07Computing Science 1P Lecture 21 - Simon Gay37 Analysing binary search We keep halving the size of the region, until we narrow it down to a single position in which the key should be found. How many times do we have to halve the size? n = 16: 8, 4, 2, 1 4 comparisons n = 64: 32, 16, 8, 4, 2, 1 6 comparisons It is the logarithm of n to base 2, i.e. the power of 2 which gives n. Binary search is an order log n algorithm.

38. 2006/07Computing Science 1P Lecture 21 - Simon Gay38 Analysing binary search We can compare the efficiency of an order n algorithm with that of an order log n algorithm: nlog ntimen 103 1006 1 00099 microsec1 0001 millisec 10 0001212 microsec10 00010 millisec 100 0001515 microsec100 000100 millisec 1 000 0001818 microsec1 000 0001 sec 10 000 0002121 microsec10 000 00010 sec

39. 2006/07Computing Science 1P Lecture 21 - Simon Gay39 Implementing binary search def find(key,data): lower = 0 upper = len(data)-1 length = upper - lower + 1 while length > 1: midpoint = lower + length/2 if key < data[midpoint]: upper = midpoint - 1 else: lower = midpoint length = upper - lower + 1 if key == data[lower]: return lower else: raise "KeyNotFound",key

40. 2006/07Computing Science 1P Lecture 21 - Simon Gay40 Implementing binary search def find(key,data): lower = 0 upper = len(data)-1 length = upper - lower + 1 while length > 1: midpoint = lower + length/2 if key < data[midpoint]: upper = midpoint - 1 else: lower = midpoint length = upper - lower + 1 if key == data[lower]: return lower else: raise "KeyNotFound",key android badger cat door ending fireman garage handle iguana jumper kestrel lemon 0 1 2 3 4 5 6 7 8 9 10 11 lower upper midpoint

41. 2006/07Computing Science 1P Lecture 21 - Simon Gay41 Termination When writing programs with loops, we have to be sure that they terminate, i.e. eventually stop. In almost all of our previous programs, it has been obvious that loops terminate. In a for loop, the number of iterations is known before we start, e.g. for x in range(10) In a while loop, the condition can be anything, but we have always used a simple structure: i = 0 while i < 10: # code inside the loop, not changing i i = i + 1

42. 2006/07Computing Science 1P Lecture 21 - Simon Gay42 Termination Binary search uses a while loop with a more complex structure: length = upper - lower + 1 while length > 1: midpoint = lower + length/2 if key < data[midpoint]: upper = midpoint - 1 else: lower = midpoint length = upper - lower + 1 Let's prove that eventually length <= 1, so that the loop terminates.

43. 2006/07Computing Science 1P Lecture 21 - Simon Gay43 Proving termination Consider one iteration of the loop. At the beginning we have values lower, upper and length. At the end of the body of the loop we have new values lower', upper' and length'. We will prove that length' < length. Therefore as we go round the loop repeatedly, length gets smaller and smaller. It is always an integer value, so eventually it must reach 1 or less.

44. 2006/07Computing Science 1P Lecture 21 - Simon Gay44 Proving termination At the top of the loop we have values lower, upper. We have length = upper – lower + 1 Assume that length > 1, so that we go into the loop. At the bottom of the loop we have new values lower', upper' and we have a new value length' = upper' – lower' + 1 we also have midpoint = lower + length/2 Now we consider the possible ways of calculating lower', upper'

45. 2006/07Computing Science 1P Lecture 21 - Simon Gay45 Proving termination Case 1: we take the first branch of the if statement. upper' = midpoint – 1 lower' = lower length' = upper' – lower' + 1 = midpoint – 1 – lower + 1 = midpoint – lower = lower + length/2 – lower = length/2 1

46. 2006/07Computing Science 1P Lecture 21 - Simon Gay46 Proving termination Case 2: we take the second branch of the if statement. upper' = upper lower' = midpoint length' = upper' – lower' + 1 = upper – midpoint + 1 = upper – (lower + length/2) + 1 = upper – lower – length/2 + 1 = length – length/2 1

47. 2006/07Computing Science 1P Lecture 21 - Simon Gay47 Proving termination We have proved that whichever path we take through the body of the while loop, length decreases. Therefore the loop must terminate. With further calculation of a similar kind (exercise!) we can prove that when the loop terminates, length = 1 (not < 1), meaning that we really have identified one location in the list where the key should be found if it is present at all.

48. 2006/07Computing Science 1P Lecture 21 - Simon Gay48 Refining binary search It might turn out that when we look at the midpoint of the list, the key we want happens to be there. We might as well take advantage of that case: while length > 1: midpoint = lower + length/2 if key < data[midpoint]: upper = midpoint - 1 elif key > data[midpoint]: lower = midpoint else: return midpoint but notice that we have introduced an extra comparison. What can we do about this?

49. 2006/07Computing Science 1P Lecture 21 - Simon Gay49 The function cmp The problem is that when we compare two values, there are three possible results: equal, first one smaller, second one smaller The comparisons >= == return a boolean result, so they only tell us one of two possible results. To solve this problem, Python provides the function cmp. cmp(a,b) returns 0 if a == b -1 if a < b 1 if a > b

50. 2006/07Computing Science 1P Lecture 21 - Simon Gay50 Using cmp while length > 1: midpoint = lower + length/2 r = cmp(key,data[midpoint]) if r == -1: upper = midpoint - 1 elif r == 1: lower = midpoint else: return midpoint