1. Efficient computation
The Chinese University of Hong Kong
Fall 2009
CSC 3130: Automata theory and formal languages
Efficient computation
Andrej Bogdanov

2. Why do we care about undecidability?
TM M accepts some input
TM M accepts input w
CFG G is ambiguous
Post Correspondence Problem
undecidable
decidable
Can you get from point A
to point B in 100 steps?
L = {0n1n: n > 0}
Can you schedule final exams
so that there are no conflicts?
Is P a valid java program?
Can you pair up people so that every pair is happy?
... pretty much everything else

3. Decidable problems L = {0n1n: n > 0} Is P a valid java program?
On input x:
n := |x|
if n is odd reject
for i := 0 to n/2:
if xi ≠ 0 or xn-i ≠ 1 reject
otherwise accept
Option 1:
Try all derivations
Option 2:
CYK algorithm
Option 3:
LR(1) algorithm
Can you get from point A
to point B in 100 steps? 8 6 10 14 3 9 4 2 5 A B
For all paths out of point A
of length at most 100:
Try this path
If path reaches B, accept
Otherwise, reject

4. Scheduling Can you schedule final exams
so that there are no conflicts?
CSC 3230
CSC 2110
CSC 3160
CSC 3130
Say we have n exams, k slots (maybe n = 200, k = 10)
Exams → vertices
Conflicts → edges
...
Slots → colors
Task: Assign one of k colors to the vertices
so that no edge has both endpoints of same color

5. Scheduling algorithm Task: Assign one of k colors to the vertices
so that no edge has both endpoints of same color
For every possible assignment of colors to vertices:
Try this assignment
If every edge has endpoints of different colors, accept
If all colorings failed, reject
CSC 3230
CSC 2110
CSC 3130
CSC 3160
...

6. Matching Can you pair up people so that every pair is happy?
People → vertices
Happy together → edges
Pairing → matching
For every possible pairing:
If each pair is happy, accept
If all pairings failed, reject

7. Shades of decidability
Sure, we can write a computer program...
Can you get from point A
to point B in 100 steps?
L = {0n1n: n > 0}
Can you schedule final exams
so that there are no conflicts?
Is P a valid java program?
Can you pair up people so that every pair is happy?
...but what happens when we run it?

8. How fast? L = {0n1n: n > 0} Is P a valid java program?
On input x:
n := |x|
if n is odd reject
for i := 0 to n/2:
if xi ≠ 0 or xn-i ≠ 1 reject
otherwise accept
Option 1:
Try all derivations
Option 2:
CYK algorithm
Option 3:
LR(1) algorithm
Can you get from point A
to point B in 100 steps? 8 6 10 14 3 9 4 2 5 A B
For all paths out of point A
of length at most 100:
Try this path
If path reaches B, accept
Otherwise, reject

9. How fast? efficient? very inefficient Is P a valid java program?
P = 100 line java program
Option 1:
Try all derivations
the lifetime of the universe
very inefficient
efficient?
Option 2:
1 week
CYK algorithm
Option 3:
LR(1) algorithm
a few milliseconds

10. Finding paths Is there a better way to do this?8 6 10 14 3 9 4 2 5 A B
Is there a better way to do this?
Yes! Dijkstra’s algorithm
Can you get from point A
to point B in 1000 steps?
For all paths out of point A
of length at most 1000:
Try this path
If path reaches B, accept
Otherwise, reject
Edsger Dijkstra
( )

11. Matching Can we do better? Yes! Edmonds’ algorithm
For every possible pairing:
If each pair is happy, accept
If all pairings failed, reject
Can we do better?
Yes! Edmonds’ algorithm
Jack Edmonds

12. Scheduling algorithm Can we do better?
Task: Assign one of k colors to the vertices
so that no edge has both endpoints of same color
For every possible assignment of colors to vertices:
Try this assignment
If every edge has endpoints of different colors, accept
If all colorings failed, reject
CSC 3230
CSC 2110
CSC 3130
CSC 3160
100 vertices, 10 colors
10100 assignments
Can we do better?

13. Some history 1950s UNIVAC first electronic computer
integrated circuits

14. The need for optimization
routing problems
packing problems
how to route calls through
the phone network?
how many trucks do you
need to fit all the boxes?

15. The need for optimization
30 min
45 min
50 min
40 min
50 min
0 min
scheduling problems
constraint satisfaction
how do you schedule jobs
to complete them
as soon as possible?
can you satisfy a system
of local constraints?

16. Fast algorithms
shortest paths
matchings
and a few more…
1956
1965

17. Proving theorems Princeton, 20 March 1956 Dear Mr. von Neumann:
With the greatest sorrow I have learned of your illness. […]
Since you now, as I hear, are feeling stronger, I would like to allow myself to write you about a mathematical problem: […] This would have consequences of the greatest importance. It would obviously mean that in spite of the undecidability of the Entscheidungsproblem, the mental work of a mathematician concerning Yes-or-No questions could be completely replaced by a machine.
Kurt Gödel
J. von Neumann

18. Proving theorems 52 years later, we still don’t know!
We know some true statements are not provable
But let’s forget about those.
Can you get a computer to find the proof reasonably quickly?
Better than “try all possible proofs?”
52 years later, we still don’t know!

19. Optimization versus proofs
If we can find proofs quickly, then we can also optimize quickly!
Question: Can we color G with so there are no conflicts?
Proof: We can, set v1 v2 v3 v4 v1 v2 v3 v4
(v1,R), (v2,Y), (v3,R), (v4,Y)

20. The Cook-Levin Theorem
Constraint satisfaction is no harder than
finding proofs quickly
However,
It is no easier, either
Cook 1971, Levin 1973
Stephen Cook
Leonid Levin

21. Equivalence of optimization problems
constraint
satisfaction
routing
covering
scheduling
packing
theorem-proving
are all as hard as one another!
Q: So are they all easy or all hard?
A: We don’t know, but we suspect all hard
Richard Karp
(1972)