1. Instructor: Aaron Roth aaroth@cis.upenn.edu
CIS 262 Automata, Computability, and Complexity Spring
Instructor: Aaron Roth
Review Lecture: May 1, 2019

2. Final Exam Thursday, May 9, 9:00am-11:00am.
Topics: All lectures (except for bonus lectures)
Closed book; just you and your pencil

3. Problem Classification
CoNP NP P NP
Complete
EXPTIME
Regular
Decidable
Recognizable
All languages

4. DFAs, NFAs, and Regular Expressions
S = {a, b, c }; L = { w | w contains at least one b and no c’s }
DFA M: NFA M’ :
Regular Expression: a* . b . (a U b)*
Need to know: How to construct a DFA/NFA/Reg-exp from a description, and how to understand the constraint expressed by a given DFA/NFA/Reg-exp a
a,b
a, b
a,b b b q0 q1 q0 q1 c c q2
a, b, c

5. Correctness Proof for DFAsq0 q1 a b q3 q2
Correctness Proof for DFAs
Given L described by a mathematical constraint over strings
how to show that a DFA M accepts exactly strings in L
To prove: For all strings w,
d*(q0, w) = q0 if count(w,a) is even and w does not end with b
= q1 if count(w,a) is odd and w does not end with b
= q2 if count(w,a) is even and w ends with b
= q3 if count(w,a) is odd and w ends with b
Prove the claim by induction on string w
Need to know: What is the proof technique, how to use it, and why do we need it ?

6. Lower Bounds and Non-regularity
Given L, how do we show that every DFA accepting L must have at least k states ? Or that L is not regular ?
Two strings u and v are distinguishable w.r.t. L if there exists an extension w such that only one of u.w and v.w is in L
To show DFA for L needs at least k states, find a set S of k pair-wise distinguishable strings
To show L is not regular, find an infinite set S of pair-wise distinguishable strings
Exercise: L = { w | count(w,a) > count(w, b) }
Complete the proof: don’t just give S
Consider all possible cases of pairs of strings in S
Don’t apply proof patterns without careful thought!

7. Closure Properties
What does it mean for regular languages to be closed under operation X?
Know basic closure constructions:
Product (for intersection)
How to complement
Concatenation, Kleene-star
If we define new operation X on strings, then to show that regular languages are closed under X:
Consider a DFA M for L
Show how to construct NFA/DFA M’ which accepts a string w
if and only if w can be obtained by applying X to a string accepted by M
First step in construction: understanding what M’ needs to check

8. Closure Properties: Exercise
Reverse(L) = { w | Reverse(w) is in L }
Prove that each of the classes below is closed under Reverse
1. Regular
2. Recognizable
3. Decidable
4. P
5. NP
For 1, need to define states/transitions of transformed machine precisely
For others, suffices to give high-level description

9. Automata Constructions
How to translate an NFA (or e-NFA) to a DFA ?
Subset construction, what’s its cost?
How to translate a regular expression to e-NFA?
Closure constructions for union, concatenation, Kleene-star

10. Decision Procedures Membership question:
Given a DFA A and input w, does A accept w ?
Non-acceptance :
Given a DFA A and input w, does A reject w ?
Emptiness question:
Given a DFA A, is there a string that A accepts ?
Universality question:
Given a DFA A, does A accept all strings ?
Equivalence question:
Given two DFAs A and B, do they accept exactly same languages ?
How to solve these problems and what’s their complexity ?
What happens if instead of DFA, input is an NFA, or a regular expression?

11. Turing Machines a b b a b b a _q
Definition of basic (deterministic single-tape) model
What’s a configuration ?
Given a configuration, what’s its successor configuration ?
Definition of L(M)
What’s a halting TM ?
Need to be able to construct TM for problems such as
L = { u # v | u and v in {0, 1}* and are unequal }

12. Universality of Turing Machines
The general idea of transforming one machine model to another:
Given a multi-tape TM M, possible to construct single-tape TM M’ that simulates execution of M step-by-step
Different variations (e.g. nondeterministic TM) can be compiled
Church-Turing Thesis: TM can simulate any program/computer
Universal Turing Machine U:
Takes as input description of another TM M and string w,
executes M on w step-by step
L(U) = ATM = { <M, w> | TM M accepts w }
but U is not a halting TM
Turing’s theorem: No halting TM can exist for ATM

13. Decidable vs Recognizable Languages
L is decidable if there exists a halting TM M such that L(M) = L
L is recognizable if there exists a TM M such that L(M) = L
Closure properties for recognizable languages:
Not closed under complement !
Always be aware of possibility of non-termination
Key trick: dove-tailing construction

14. Recognizable-but-undecidable Problems
Typical pattern: Problem can be formulated as
Does there exist k, ranging over a countable set, such that
some decidable check that depends on k
-- ATM: Given M and w, is there k such that M accepts w in k steps?
-- NETM: Given M, are there i and k such that M accepts wi in k steps ?
-- HALTTM : Given M and w, is there k such that M halts on w in k steps
-- Hilbert10: Given a polynomial p(x1, ... xn), are there integers c1, .. cn such that p(c1, … cn) = 0 ?
-- PCP : Post Correspondence Problem
To prove that a language is recognizable-but-undecidable:
1. Show that it is recognizable, by finding a TM that accepts
2. Show that it is undecidable, by reducing a problem such as ATM to it

15. Unrecognizable Problems
Intuitively, even to accept, one needs to try all from an infinite set
(i.e. statement involves a universal quantifier over an infinite set)
-- NATM: Given M and w, for all k, M does not accept w in k steps
-- ETM: Given M, for all w, M does not accept w
-- EQTM : Given M and M’, for all w, M accepts w iff M’ accepts w
-- REGTM : Given M, is there a DFA A such that for all w,
M accepts w if and only if A accepts w
-- ONETM: Given M, is there w such that for all w’ != w,
M accepts w and M does not accept w’
To prove that a language is unrecognizable: Either
1. show that its complement is recognizable-but-undecidable, OR
2. reduce a problem such as NATM or ETM to it

16. Example Reduction
L is prefix-closed if whenever a string w belongs to L, all the prefixes of w also belong to L
A = { <M> | M is a TM and L(M) is prefix-closed }
What’s the complexity of A ?
For all strings w,
if M accepts w
then for all prefixes u of w, M also accepts u
Unrecognizable
Proof by reduction from
NATM = { <M,w> | M does not accept w }

17. Example Reduction
A = { <M> | M is a TM and L(M) is prefix-closed }
NATM = { <M,w> | M does not accept w }
Given input <M,w>, construct M’ such that
L(M’) is prefix-closed exactly when M does not accept w
L(M’) = some set not closed under prefix if M accepts w
= some prefix-closed set otherwise
L(M’) = { a } if M accepts w
= Empty otherwise
M’ should accept a if M accepts w and nothing otherwise

18. Unrecognizability Proof
A = { <M> | M is a TM and L(M) is prefix-closed }
NATM = { <M, w> | M is a TM and M does not accept w }
Consider a TM R such that L(R) = A
Define the following TM T :
Given input <M, w>:
If <M> does not encode a TM, then reject
Construct the description <M’> of the following TM:
“Given input x,
If (x = a) then if M accepts w, then accept”
Execute R on input <M’>
If R accepts, accept
Claim: <M’> is in A if and only if <M, w> is in NATM
Add a few lines explaining this claim based on our reasoning in last slide
But NATM is unrecognizable, so A is unrecognizable

19. Kolmogorov Complexity
The minimal description of x, denoted d(x), is the shortest string <M, w> such that M is a Turing Machine, and when M is run on input w, M halts with x on its tape.
The Kolmogorov Complexity of x is:
K(x) = |d(x)|,
the length of the minimal description of x.
Almost every string x has K(x) > |x|
K(x) is not a computable function.
There is a constant L such that for every x, K(x) > L is not provable!

20. Class P Time complexity of TM/programs Big-oh notation
L is in P if it can be decided by a TM of time complexity O(nk)
To show a problem to be in P:
Give pseudo-code and establish polynomial upper bound on time
Avoid iterating over exponentially many choices !
Example problems
Reachability in graphs
Problems about automata
(but note that NFA  DFA translation is exponential)

21. NP and Co-NP
L is in NP if it can be decided by a NTM of time complexity O(nk)
Equivalently, there exists a (deterministic) poly-time verifier V such that there exists a certificate s.t. V accepts <w,c> if and only if w is in L
To show a problem to be in NP, use one of the above definitions
Example problems
Is there a clique of given size in a graph?
Is there an independent set of given size in a graph?
Is there a Hamiltonian path from s to t in a graph?
Can a graph be colored using given number of colors ?
L is in CoNP if its complement is in NP
Open question: Is P = NP (if yes, then NP = CoNP) ?

22. Poly-time Reductions and NP-completeness
To show A <P B:
given an input w for A, show how to construct in polynomial-time f(w) such that w is in A if and only if f(w) is in B
More generally, give a poly-time algorithm assuming a poly-time algo for B
A is NP-hard if for every language L in NP, L <P A
A is NP-complete if A is in NP and is NP-hard
Cook’s theorem: SAT is NP-complete
To show a problem to be NP-hard, show that a known NP-hard problem poly-time reduces to it
A large collection of NP-complete problems
If one of them turns out to be in P, then so are all in P

23. Boolean Formulas Know definitions:
Built from Boolean variables and logical connectives ~, & , |
Truth assignment
Satsifiability, Validity, Equivalence
Literals, clauses, conjunctive normal form (CNF), 3CNF
Complexity of various decision problems
Reductions involving Boolean formulas
e.g. independent set, vertex cover, 3-coloring

24. Approximation Algorithms
An algorithm A is a c-approximation algorithm for a minimization problem (e.g. vertex cover) if, given any instance G as input, A outputs a solution S such that:
S is a feasible solution to instance G
|S| ≤ c⋅OPT(G)
The simple 2-approximation algorithm for VERTEX-COVER
The simple 2-approximation algorithm for MAX-CUT

25. Key concepts not to be forgotten after May 9…
Finite-state Automata and Regular Languages
Limited expressiveness, but robust and analyzable
Nondeterministic Machines
When it does and does not change expressiveness
3. Turing Machines
Mathematical definition of what computers can do
4. Undecidability
Some problems cannot be solved by computers
Questions about what a program computes are undecidable
5. NP-completeness and problem reduction
Common optimization problems have no known poly-time solutions
A large set of problems are all computationally equivalent
6. Approximation
NP-completeness does not rule out finding pretty good solutions