1. Computational Complexity
Dr. Colin Campbell
Course: EMAT20531

2. Order Notation for Functions
The order notation is used to give a bound on the limiting behaviour of a function.
We write f(x) is O(g(x)) iff

3. Order example f=100x3+50x2+10x+100 g=x3 f-mg m=110 m=120 m=130 m=140

4. The big O Notation for Algorithms
We can represent the complexity of an algorithm A by a function
fA(n)=number of steps (or time) it takes for A to complete given an input of size n.
The size of input may be measured in a number of different ways.

5. Big O Notation f(n)=2n+1 O(f(n))=n A is linear in n A= Do i=1,..,n
Print “hello world”
Clear
Repeat
Print “End”
f(n)=2n+1
O(f(n))=n
A is linear in n

6. Big O Notation: 2 f(n)=2n2+1 O(f(n) is n2 A= Do i=1,…,n Do j=1,…,n
Print “Hello World”
Clear
Repeat
Print “End”
f(n)=2n2+1
O(f(n) is n2

7. The Quicksort Algorithm
Input: a list L=(a1,…,an) of numbers
Choose an number p at random from L
Determine L1 all numbers ai less than p and L2 all numbers ai greater than or equal to p
Recursively sort L1 and L2
Solution=concatenate L1 and L2

8. Quicksort Example Sort the list (1,5,2,7,6,4)
L=(1,5,2,7,6,4) sort p=4 [(1,2)(4,5,6,7)=(1,2,4,5,6,7)]
L1=(1,2) L2=(5,7,6,4)
sort L=(5,7,6,4) p=6 [(4,5)(6,7)=(4,5,6,7)]
L1=(5,4), L2=(7,6)
sort L=(5,4) [(4,5)]
sort L=(7,6) [(6,7)]

9. Quicksort: The Worst Case
Given p partitioning into L1 and L2 is O(n)
The worst possible case would be that every time we partition the list we have L1 length n-1 and L2 length 1
For the i’th partition we have a computation O(n-i)
Now (n-i)=n2. Hence the entire computation is O(n2).

10. Quicksort: On Average
On average the split between L1 and L2 will result in two lists of n/2.
This is based on the assumption that the elements of the list are selected from an interval [x,y] according to a uniform distribution
Consider P(pai), the expected number of elements in L2 is nP(pai)

11. Quicksort: On Average (2)
p is a uniformly distributed random variable taking values in [x,y]
Therefore
ai is also a uniformly distributed random variable from [x,y] with expected value (x+y)/2.
Therefore, the expected value of P(pai) is ½ and the expected number of elements in L2 is n/2

12. Quicksort: On Average (3)
Consider, the number of such partitions before we end up with strings of length 1. i.e. find k for which n/2k=1
Log(n/2k)=0 log(n)=log(2k)=k
Hence, there are on average log(n) partitions each requiring O(n) computations
This gives O(nlog(n))

13. Polynomial Time
An algorithm runs in polynomial time if fA(n) is O(nk) for some positive integer k.
E.g. Quicksort always runs in polynomial time even in the worst case when it is O(n2)
The class P refers to those algorithms which run in polynomial time on a Turing Machine as described in the earlier section.

14. Non-deterministic Turing Machines
A Non-deterministic Turing machine is a Turing machine where for some state qi and symbol S there is more than one instruction beginning qiS.
1:R q2 q1
0:R q3

15. Computation Paths
A deterministic Turing machine follows a computation path.
A Non-Deterministic generates a tree of computations.
We can either think of a NDTM as following all paths simultaneously until it finds an accepting state
Or being a very lucky guesser between when choosing between states

16. Accepting States and Paths
For NDTM we usually think of binary yes no responses from a computation.
We define a set of accepting state AQ.
An accepting path is a path which terminates in an accepting state.
A NDTM accepts an input x if there is an accepting path for that input.

17. NDTM Example
Let TMP be a deterministic Turing machine that given a string of 1s and 0s terminates in state q+ if there are an even number of 1 and q- otherwise. AQ={q+}
Further suppose that the start state of TMP is q3
Given a string of 1s we mean substrings to be a string of the same length where some or all of the 1s have been switched to 0.
Eg 111 has substrings 111,110,101,100, 011,010,001,000

18. NDTM Example: 2 TMP 1111 0:R 1:0 1111 0211 1111 1021 0111 1:R 1113q1 q2
1111
1021
0111
1:R
TMP q3
1113
1103
1011
0111
0011 q- q+
1013
1003
0113
0103
0013
0003
Several deterministic steps q+ q- q+ q- q- q+

19. Non-Deterministic Polynomial
Intuitively, we see that a NDTM is more efficient than a DTM since it can explore many computational paths simultaneously.
NP refer to the class of algorithms which run in polynomial time on a NDTM.
Clearly PNP but does P=NP?

20. NP Complete
NP-Complete are a collection of decision problems for which there is an algorithm in NP but no known algorithm in P.
Formally, a problem in NP is NP-Complete if any other problem in NP can be reduced to it.
Problem C is reducible to NP-Complete problem D if there is an algorithm A which solves C with a subroutine A+ that solves D, such if A+ ran in polynomial time so would A.

21. SAT: An NP Complete Problem
SAT: Given a sentence of Propositional Logic S in CNF, is there an allocation of truth values for which S is true?
There is no known algorithm in P which solves SAT
This is basically because all (known) approaches require us (in the worst case) to check all possible truth value allocations. If S has n propositional variables there are 2n possible allocations of truth values.

22. A NDTP algorithm for SAT
Simplify things by assuming that S is in CNF and each clause has exactly k literals for k>2. (k-SAT)
Notice that for any given allocation of truth values we can check if satisfies S in O(nk) – n is the number of variables in S.
There are 2n(2n-1)..(2n-k+1)/3! Different clauses.
Assume we can check if each is satisfied in 1 step then the computation is O(nk)

23. SAT Algorithm: 2 S=(PR Q)(PRQ) Algorithm
For each variable guess either t or f. [O(n) on NDTM]
Check if allocation of truth values satisfies S [O(nk)] P t f Q Q t f t f R R R R t f t f f t f t
accept
accept
accept
accept
accept
accept
reject
reject

24. Proving NP-Completeness
Usually a problem D is shown to be NP-complete by proving that a known NP-complete problem C reduces to it.
All NP problems
reduce
All NP problems
reduce
NP complete C
reduce D D

25. CLIQUE
In graph theory a CLIQUE is a set of vertices (nodes) where each is connected to the others. 5 6 1 4 5 2 1
Clique of size 3 3 2

26. The CLIQUE Problem
For a given graph decide whether or not there is a clique of size k.
This has application in chip design where for example nodes are wires on a chip and edges signify that two wires can overlap.
CLIQUE tells the designer something about how much space is required on the chip.
We shall show that SAT reduces to CLIQUE
So CLIQUE is NP-Complete.

27. CLIQUE and SAT Let S be a sentence in CNF with k clauses. S=C1C2…Ck
We construct a graph G where
Vertices={Li:LiCi} i.e. every literal from every clause
Edges={<Li,Lj> LiLj and ij} i.e. edges between literals from different clauses provides they do not conflict.
S is satisfiable if and only if G has a clique of size k.

28. CLIQUE and SAT: 2 Example S=(PQ)(PQ)(PR)
L1=P, L2=Q, L3=P, L4=Q, L5=P, L6=R
Graph G L1 L5 L4 L2 L6 L3

29. CLIQUE and SAT: 3
(If) Suppose there is a clique of size k then there are k non-conflicting literals, one from each clause in S.
Since they are non-conflicting there is a truth allocation that makes each true and hence each clause in S true.
(Only If) Suppose there is an allocation of truth values for which S is true.
At least one literal from each clause must be true
These cannot be conflicting, hence they must form a clique of size k in G

30. CLIQUE and SAT: 4 L1 L5 L4 Graph G L1 L4 L5 L4 L2 P=t,Q=t,R={t,f} L6
P={t,f},Q=t,R=t
P=t,Q=t,R={t,f}
P=f,Q=t,R=t

31. Other NP-Complete Problems: The Knapsack Problem
Weight: wi
Utility: ui
Quantity: xi
500g
100
300g
400
Maximize
1kg
200
subject to
750g 80
Decision problem: is there a solution for which
and ?

32. Other NP-Complete Problems: The Travelling Salesman Problem10
Starting at A find the shortest route which visits all towns and then returns to A. A B 8 7 18 4 C D F 12 9 16 25 E