1. CS1022 Computer Programming & Principles
Lecture 2
Combinatorics

2. Plan of lecture Binomial expansion Rearrangement theorem
Efficiency of algorithms
CS1022

3. Binomial expansion (1)
Numbers C(n, k) arise naturally when we expand expressions of the form (a  b)n algebraically
For example:
(a  b)3  (a  b) (a  b) (a  b)
 aaa  aab  aba  abb  baa  bab  bba  bbb
 a3  3a2b  3ab2  b3
Terms are from multiplying variables from brackets
Three terms (abb, bab, bba) with one a and two b’s
This is because there are C(3, 2) = 3 ways of selecting two b’s from the three brackets
CS1022

4. Binomial expansion (2) Coefficients in simplified expansion are
C(3, 0)  1, C(3, 1)  3, C(3, 2)  3, C(3, 3)  3
To obtain values from C(n, k), we assume 0! = 1
There is only one way to select nothing from a collection
CS1022

5. (a  b)n  C(n,0)an  C(n,1)an – 1b  C(n,2)an – 2b2 ...  C(n, n)bn
Binomial expansion (3)
(a  b)n expansion contains terms (an – kbk)
Multiplying a from (n – k) brackets and b from remaining k brackets (k takes values from 0 up to n)
Since there are C(n, k) ways of selecting k brackets,
There are precisely C(n, k) terms of the form (an – kbk), for k  0, 1, ..., n
Therefore
(a  b)n  C(n,0)an  C(n,1)an – 1b  C(n,2)an – 2b2 ...  C(n, n)bn
This formula is called binomial expansion
C(n, k), in this context, is called binomial coefficient
CS1022

6. Pascal’s triangle (1)
Binomial coefficients arranged as Pascal’s triangle
C(0, 0)
C(1, 0) C(1, 1)
C(2, 0) C(2, 1) C(2, 2)
C(3, 0) C(3, 1) C(3, 2) C(3, 3)
C(4, 0) C(4, 1) C(4, 2) C(4, 3) C(4, 4)
C(5, 0) C(5, 1) C(5, 2) C(5, 3) C(5, 4) C(5, 5)
C(n, 0) C(n, 1) C(n, n – 1) C(n, n)
Entry in row n  1 correspond to coefficients (in order) in the binomial expansion of (a  b)n
CS1022

7. Pascal’s triangle (2) If we calculate numerical values we obtain 1 1 1
...
Since C(n, 0)  C(n, n)  1, the outer edges of Pascal’s triangle are indeed 1
Vertical symmetry also holds as C(n, k)  C(n, n – k)
CS1022

8. Rearrangement theorem (1)
Consider the problem of finding rearrangements of letters in “DEFENDER”
There are 8 letters which can be arranged in 8! Ways
However, the 3 Es are indistinguishable – they can be permuted in 3! ways without changing the arrangement
Similarly, 2 Ds can be permuted without changing the arrangement
Therefore, the number of distinguishable arrangements is just
CS1022

9. Rearrangement theorem (2)
In general, there are
different arrangements of n objects of which
n1 are of type 1
n2 are of type 2
and so on, up to
nr of type r
CS1022

10. Rearrangement theorem (3)
In how many ways can 15 students be divided into 3 tutorial groups with 5 students in each group?
Solution:
There are 15 objects to be arranged into 3 groups of 5
This can be done in
different ways
CS1022

11. Efficiency of algorithms
Issue: design & analysis of efficient algorithms
Two solutions: which one is better?
Is your algorithm the best possible?
What is “better”/“best”?
How can we compare algorithms?
Measure time and space (memory)
Not size of algorithms (fewer lines not always good)
CS1022

12. Towers of Hanoi (1) A “simple” problem/game: the Towers of Hanoi
3 rods and n disks of different sizes
Start: disks in ascending order on one rod
Finish: disks on another rod
Rules:
Only one disk to be moved at a time
Move: take upper disk from a rod & slide it onto another rod, on top of any other disks
No disk may be placed on top of a smaller disk.
CS1022

13. Towers of Hanoi (2)
Animation of solution for 4 disks:
CS1022

14. Towers of Hanoi (3) Many algorithms to solve this problem:
Alternate between smallest and next-smallest disks
For an even number of disks:
make the legal move between pegs A and B
make the legal move between pegs A and C
make the legal move between pegs B and C
repeat until complete
For an odd number of disks:
To move n discs from peg A to peg C:
move n − 1 discs from A to B
move disc n from A to C
move n − 1 discs from B to C
CS1022

15. Towers of Hanoi (4) How can we compare solutions?
What about counting the number of moves?
The fewer moves, the better!
Instead of using a stop-clock, we count how often
The most frequent instruction/move is performed OR
The most expensive instruction/move is performed
We simplify our analysis, so we don’t consider
Computer, operating system, or language we might use
CS1022

16. 584,942,417.36 years Towers of Hanoi (5) 64 disks require 264 moves
That’s 18,446,744,073,709,551,616 moves
If we carry out one move per millisecond, we have
18,446,744,073,709,551,616 ms
18,446,744,073,709, sec
307,445,734,561, min
5,124,095,576, hrs
213,503,982, days
584,942, years
CS1022

17. Estimating time We don’t measure time with a stop-clock!
It’s an algorithm, so it won’t run...
Implement it and then time it (not good either)
Some algorithms take too long (centuries!)
We estimate the time the algorithm takes as a function of the number of values processed
Simplest way: count, for an input of size n, the number of elementary operations carried out
Important: same considerations about time apply to space needed (memory)
CS1022

18. Searching in a dictionary (1)
Search for word X in a dictionary with n words
Simple solution: sequential search
Check if X is the 1st word, if not check if it’s the 2nd, etc.
Worst case: X is not in dictionary (n comparisons)
Another solution: binary search
Check if X is the “middle” word (words are ordered)
If not, decide if X could be on the first or second half
Dictionary always “half the size”
Worst case: 1 + log2n comparisons
CS1022

19. Searching in a dictionary (2)
Performance (in terms of comparisons) n
1 + log2n 8 4 64 7 n
1 + log2n 8 4 n
1 + log2n n
1 + log2n 8 4 64 7
218  250,000 19
log22k= k
CS1022

20. Comparing algorithms (1)
Suppose we have a choice of algorithms
A, B, C, D, E require n, 3n2, 2n2 + 4n, n3, 2n
elementary operations (respectively)
Each operation takes 1 millisecond
Find overall running time for n = 1, 10, 100 & 1000 A B C D E n
3n2
2n2 + 4n n3 2n 1
1ms
3ms
6ms
2ms 10
10ms
300ms
240ms 1s
1.024s
100
100ms
30s
20.4s
0.28h
41017 centuries
1000
1000ms
0.83h
0.56h
11.6 days
10176 centuries A B C D E n
3n2
2n2 + 4n n3 2n 1
1ms
3ms
6ms
2ms 10
10ms
300ms
240ms 1s
1.024s
100
100ms
30s
20.4s
0.28h
41017 centuries
1000
1000ms
0.83h
0.56h
11.6 days
10176 centuries
CS1022

21. Comparing algorithms (2)
Table shows huge difference in times
Formulae with powers of n (polynomial functions)
Formulae with powering by n (exponential functions)
When the same highest power of n is involved, running times are comparable
For instance, B and C in the previous table
If f(n) and g(n) measure efficiency of 2 algorithms
They are called time-complexity functions
f(n) is of order at most g(n), written as O(g(n)),
If there is a positive constant c, |f(n)|  c|g(n)|, n  N
CS1022

22. Comparing algorithms (3)
Suppose two functions f(n) and g(n), such that
|f(n)|  c1|g(n)| and |g(n)|  c2|f(n)|
one is of order at most the other
They have same order of magnitude
Their times are comparable
They perform, when n is big enough, similarly B C
3n2
2n2 + 4n 1
3ms
6ms 10
300ms
240ms
100
30s
20.4s
1000
0.83h
0.56h
CS1022

23. Hierarchy of functions (1)
We can define a hierarchy of functions
Each has greater order of magnitude than predecessors
Example of hierarchy:
Left-to-right: greater order of magnitude
As n increases, the value of latter functions increases more rapidly 1
log n
n n2... nk... 2n 
constant
logarithmic
polynomial
exponential
CS1022

24. Hierarchy of functions (2)
Growth of functions as a graph
CS1022

25. Complexity of functions
We compare algorithms based on the complexity of the function describing their performance
We “simplify” a function to work out which curve (or which class in the hierarchy) “bounds” it
Example: f(n) = 9n + 3n6 + 7 log n
Constants do not affect magnitudes, so
9n is O(n) 3n6 = O(n6) 7 log n = O(log n)
Since n and log n occur earlier than n6 in hierarchy, we say that 9n and 7 log n are in O(n6)
Hence f(n) is O(n6), as the fastest growing term is 3n6
The function increases no faster than function n6
CS1022

26. Further reading
R. Haggarty. “Discrete Mathematics for Computing”. Pearson Education Ltd (Chapter 6)
Wikipedia
Analysis of Wikipedia
100 solutions to the “Tower of Hanoi” problem
CS1022