Instructor: Aaron Roth aaroth@cis.upenn.edu CIS 262 Automata, Computability, and Complexity Spring 2019 http://www.seas.upenn.edu/~cse262/ Instructor: Aaron Roth aaroth@cis.upenn.edu Review Lecture: May 1, 2019

Final Exam Thursday, May 9, 9:00am-11:00am. Topics: All lectures (except for bonus lectures) Closed book; just you and your pencil

Problem Classification CoNP NP P NP Complete EXPTIME Regular Decidable Recognizable All languages

DFAs, NFAs, and Regular Expressions S = {a, b, c }; L = { w | w contains at least one b and no c’s } DFA M: NFA M’ : Regular Expression: a* . b . (a U b)* Need to know: How to construct a DFA/NFA/Reg-exp from a description, and how to understand the constraint expressed by a given DFA/NFA/Reg-exp a a,b a, b a,b b b q0 q1 q0 q1 c c q2 a, b, c

Correctness Proof for DFAs q0 q1 a b q3 q2 Correctness Proof for DFAs Given L described by a mathematical constraint over strings how to show that a DFA M accepts exactly strings in L To prove: For all strings w, d*(q0, w) = q0 if count(w,a) is even and w does not end with b = q1 if count(w,a) is odd and w does not end with b = q2 if count(w,a) is even and w ends with b = q3 if count(w,a) is odd and w ends with b Prove the claim by induction on string w Need to know: What is the proof technique, how to use it, and why do we need it ?

Lower Bounds and Non-regularity Given L, how do we show that every DFA accepting L must have at least k states ? Or that L is not regular ? Two strings u and v are distinguishable w.r.t. L if there exists an extension w such that only one of u.w and v.w is in L To show DFA for L needs at least k states, find a set S of k pair-wise distinguishable strings To show L is not regular, find an infinite set S of pair-wise distinguishable strings Exercise: L = { w | count(w,a) > count(w, b) } Complete the proof: don’t just give S Consider all possible cases of pairs of strings in S Don’t apply proof patterns without careful thought!

Closure Properties What does it mean for regular languages to be closed under operation X? Know basic closure constructions: Product (for intersection) How to complement Concatenation, Kleene-star If we define new operation X on strings, then to show that regular languages are closed under X: Consider a DFA M for L Show how to construct NFA/DFA M’ which accepts a string w if and only if w can be obtained by applying X to a string accepted by M First step in construction: understanding what M’ needs to check

Closure Properties: Exercise Reverse(L) = { w | Reverse(w) is in L } Prove that each of the classes below is closed under Reverse 1. Regular 2. Recognizable 3. Decidable 4. P 5. NP For 1, need to define states/transitions of transformed machine precisely For others, suffices to give high-level description

Automata Constructions How to translate an NFA (or e-NFA) to a DFA ? Subset construction, what’s its cost? How to translate a regular expression to e-NFA? Closure constructions for union, concatenation, Kleene-star

Decision Procedures Membership question: Given a DFA A and input w, does A accept w ? Non-acceptance : Given a DFA A and input w, does A reject w ? Emptiness question: Given a DFA A, is there a string that A accepts ? Universality question: Given a DFA A, does A accept all strings ? Equivalence question: Given two DFAs A and B, do they accept exactly same languages ? How to solve these problems and what’s their complexity ? What happens if instead of DFA, input is an NFA, or a regular expression?

Turing Machines a b b a b b a _ q Definition of basic (deterministic single-tape) model What’s a configuration ? Given a configuration, what’s its successor configuration ? Definition of L(M) What’s a halting TM ? Need to be able to construct TM for problems such as L = { u # v | u and v in {0, 1}* and are unequal }

Universality of Turing Machines The general idea of transforming one machine model to another: Given a multi-tape TM M, possible to construct single-tape TM M’ that simulates execution of M step-by-step Different variations (e.g. nondeterministic TM) can be compiled Church-Turing Thesis: TM can simulate any program/computer Universal Turing Machine U: Takes as input description of another TM M and string w, executes M on w step-by step L(U) = ATM = { <M, w> | TM M accepts w } but U is not a halting TM Turing’s theorem: No halting TM can exist for ATM

Decidable vs Recognizable Languages L is decidable if there exists a halting TM M such that L(M) = L L is recognizable if there exists a TM M such that L(M) = L Closure properties for recognizable languages: Not closed under complement ! Always be aware of possibility of non-termination Key trick: dove-tailing construction

Recognizable-but-undecidable Problems Typical pattern: Problem can be formulated as Does there exist k, ranging over a countable set, such that some decidable check that depends on k -- ATM: Given M and w, is there k such that M accepts w in k steps? -- NETM: Given M, are there i and k such that M accepts wi in k steps ? -- HALTTM : Given M and w, is there k such that M halts on w in k steps -- Hilbert10: Given a polynomial p(x1, ... xn), are there integers c1, .. cn such that p(c1, … cn) = 0 ? -- PCP : Post Correspondence Problem To prove that a language is recognizable-but-undecidable: 1. Show that it is recognizable, by finding a TM that accepts 2. Show that it is undecidable, by reducing a problem such as ATM to it

Unrecognizable Problems Intuitively, even to accept, one needs to try all from an infinite set (i.e. statement involves a universal quantifier over an infinite set) -- NATM: Given M and w, for all k, M does not accept w in k steps -- ETM: Given M, for all w, M does not accept w -- EQTM : Given M and M’, for all w, M accepts w iff M’ accepts w -- REGTM : Given M, is there a DFA A such that for all w, M accepts w if and only if A accepts w -- ONETM: Given M, is there w such that for all w’ != w, M accepts w and M does not accept w’ To prove that a language is unrecognizable: Either 1. show that its complement is recognizable-but-undecidable, OR 2. reduce a problem such as NATM or ETM to it

Example Reduction L is prefix-closed if whenever a string w belongs to L, all the prefixes of w also belong to L A = { <M> | M is a TM and L(M) is prefix-closed } What’s the complexity of A ? For all strings w, if M accepts w then for all prefixes u of w, M also accepts u Unrecognizable Proof by reduction from NATM = { <M,w> | M does not accept w }

Example Reduction A = { <M> | M is a TM and L(M) is prefix-closed } NATM = { <M,w> | M does not accept w } Given input <M,w>, construct M’ such that L(M’) is prefix-closed exactly when M does not accept w L(M’) = some set not closed under prefix if M accepts w = some prefix-closed set otherwise L(M’) = { a } if M accepts w = Empty otherwise M’ should accept a if M accepts w and nothing otherwise

Unrecognizability Proof A = { <M> | M is a TM and L(M) is prefix-closed } NATM = { <M, w> | M is a TM and M does not accept w } Consider a TM R such that L(R) = A Define the following TM T : Given input <M, w>: If <M> does not encode a TM, then reject Construct the description <M’> of the following TM: “Given input x, If (x = a) then if M accepts w, then accept” Execute R on input <M’> If R accepts, accept Claim: <M’> is in A if and only if <M, w> is in NATM Add a few lines explaining this claim based on our reasoning in last slide But NATM is unrecognizable, so A is unrecognizable

Kolmogorov Complexity The minimal description of x, denoted d(x), is the shortest string <M, w> such that M is a Turing Machine, and when M is run on input w, M halts with x on its tape. The Kolmogorov Complexity of x is: K(x) = |d(x)|, the length of the minimal description of x. Almost every string x has K(x) > 0.999999 |x| K(x) is not a computable function. There is a constant L such that for every x, K(x) > L is not provable!

Class P Time complexity of TM/programs Big-oh notation L is in P if it can be decided by a TM of time complexity O(nk) To show a problem to be in P: Give pseudo-code and establish polynomial upper bound on time Avoid iterating over exponentially many choices ! Example problems Reachability in graphs Problems about automata (but note that NFA  DFA translation is exponential)

NP and Co-NP L is in NP if it can be decided by a NTM of time complexity O(nk) Equivalently, there exists a (deterministic) poly-time verifier V such that there exists a certificate s.t. V accepts <w,c> if and only if w is in L To show a problem to be in NP, use one of the above definitions Example problems Is there a clique of given size in a graph? Is there an independent set of given size in a graph? Is there a Hamiltonian path from s to t in a graph? Can a graph be colored using given number of colors ? L is in CoNP if its complement is in NP Open question: Is P = NP (if yes, then NP = CoNP) ?

Poly-time Reductions and NP-completeness To show A <P B: given an input w for A, show how to construct in polynomial-time f(w) such that w is in A if and only if f(w) is in B More generally, give a poly-time algorithm assuming a poly-time algo for B A is NP-hard if for every language L in NP, L <P A A is NP-complete if A is in NP and is NP-hard Cook’s theorem: SAT is NP-complete To show a problem to be NP-hard, show that a known NP-hard problem poly-time reduces to it A large collection of NP-complete problems If one of them turns out to be in P, then so are all in P

Boolean Formulas Know definitions: Built from Boolean variables and logical connectives ~, & , | Truth assignment Satsifiability, Validity, Equivalence Literals, clauses, conjunctive normal form (CNF), 3CNF Complexity of various decision problems Reductions involving Boolean formulas e.g. independent set, vertex cover, 3-coloring

Approximation Algorithms An algorithm A is a c-approximation algorithm for a minimization problem (e.g. vertex cover) if, given any instance G as input, A outputs a solution S such that: S is a feasible solution to instance G |S| ≤ c⋅OPT(G) The simple 2-approximation algorithm for VERTEX-COVER The simple 2-approximation algorithm for MAX-CUT

Key concepts not to be forgotten after May 9… Finite-state Automata and Regular Languages Limited expressiveness, but robust and analyzable Nondeterministic Machines When it does and does not change expressiveness 3. Turing Machines Mathematical definition of what computers can do 4. Undecidability Some problems cannot be solved by computers Questions about what a program computes are undecidable 5. NP-completeness and problem reduction Common optimization problems have no known poly-time solutions A large set of problems are all computationally equivalent 6. Approximation NP-completeness does not rule out finding pretty good solutions