Instructor: Aaron Roth aaroth@cis.upenn.edu CIS 262 Automata, Computability, and Complexity Spring 2019 http://www.seas.upenn.edu/~cse262/ Instructor: Aaron Roth aaroth@cis.upenn.edu Lecture: March 27, 2019

Problem Classification ALL LANGUAGES RECOGNIZABLE Ld DECIDABLE ATM How to classify a given problem L as (1) decidable, (2) recognizable-but-undecidable, or (3) unrecognizable ? Upper bound techniques: (a) To show L is decidable, find a halting TM/program for it (b) To show L is recognizable, find a TM/program that accepts it How to show L is undecidable, or unrecognizable ? Use of closure properties Technique of problem reduction and known results for Ld and ATM

Non-emptiness Problem for TMs NETM = { <M> | M is a TM and L(M) is non-empty } Given a TM, we want to check if there exists a string that it accepts Is this problem: decidable, recognizable-but-undecidable, unrecognizable ?? Claim1: NETM is recognizable Given a TM M, 1. Find a string w 2. Check if M accepts w Infinitely (but countably many) choices for w Execution of M on any given string may not be halting

Dovetailing: Countably Infinite Parallelism Number of steps in execution  1 2 3 4 … w1 w2 w3 w4 w5 Strings  Computational problem: is there a string that M accepts ? Intuitively, execute M on all strings in parallel, and stop if any gets accepted Do there exist numbers i and j such that M accepts string wi in j steps ? Don’t know any bounds on i and j, so must try all combinations systematically

TM for Non-emptiness Problem for TMs Given input <M>, Check the input indeed encodes a Turing machine (if not, reject); k = 1; repeat { for i = 1 to k { Simulate M on wi –the i-th string in S*, for k steps; If M accepts, stop and accept }; k = k+1; } Claim: If above program stops and accepts <M>, then L(M) is non-empty Claim: If L(M) is non-empty, then above program halts and accepts <M> NETM = { <M> | M is a TM and L(M) is non-empty } is recognizable

Non-emptiness Problem for TMs NETM = { <M> | M is a TM and L(M) is non-empty } Given a TM M, is there a string w that M accepts ? We know it is recognizable, but is it decidable ? Seems computationally harder than ATM , which is to check given a TM M and a specific input w, whether M accepts w ? To show undecidability of NETM using known undecidability of ATM, we need to reduce ATM to NETM

Non-emptiness Problem for TMs NETM = { <M> | M is a TM and L(M) is non-empty } ATM = { <M,w> | M is a TM and M accepts w } Goal: Given a halting TM R for NETM, construct a halting TM T for ATM Towards the desired construction : Given TM M and w, we want to construct TM M’ such that L(M’) is non-empty exactly when M accepts w If this is possible, then to check if <M,w> belongs to ATM, T can first construct <M’> from <M> and w, and check if R accepts <M’>

Non-emptiness Problem for TMs Given a TM M and a string w, we want to construct M’ such that M’ accepts some input exactly when M accepts w So what should M’ do ? Ignore its input, and execute M on w Description of M’: Given an input x, Execute M on w; If M accepts w, then accept x For every input x, M’ accepts x exactly when M accepts w Claim: If M accepts w, then L(M’) = S*, else L(M’) = Empty set

Undecidability of Non-emptiness Problem for TMs NETM = { <M> | M is a TM and L(M) is non-empty } ATM = { <M,w> | M is a TM and M accepts w } Consider a halting TM R that solves NETM Define the following TM T : Given input <M, w>: If <M> does not encode a TM, then reject Construct the description <M’> of the following TM: “Given input x, Execute M on w, and if M accepts w, then accept” Execute R on input <M’> If R accepts <M’>, accept, else reject Claim: If R is a halting TM for NETM, T is a halting TM for ATM Conclusion: NETM is undecidable

Comments on Reduction Proof NETM = { <M> | M is a TM and L(M) is non-empty } ATM = { <M,w> | M is a TM and M accepts w } We are reducing one kind of query for machines (is a specific string accepted) to a different kind (does the machine accept some string) Reduction involves constructing a new machine M’ from M and w Description of M and w is “baked” into description of M’ Constructing <M’> from <M,w> can be implemented by a program/TM -- Think of this construction as adding some new states/transitions to the description of M (or adding wrapper code around code of M) -- Constructing <M’> does not involve executing M, or checking any property of L(M)

More on Construction of M’ in the Reduction Given a TM M and a string w, T wants to construct M’ such that L(M’) is non-empty exactly when M accepts w We used the following construction of M’: Given an input x, Execute M on w; If M accepts w, then accept x Claim: If M accepts w, then L(M’) = S*, else L(M’) = Empty set But this is only one possible choice for M’

More on Construction of M’ in the Reduction Given a TM M and a string w, T wants to construct M’ such that L(M’) is non-empty exactly when M accepts w Another possible choice for M’ : Given an input x, If (x = w) then { Execute M on x, and if M accepts, accept } Else reject For this M’: If M accepts w, then L(M) = {w} else L(M’) = Empty set Reduction proof works also with this choice of M’

Emptiness Problem for TMs NETM = { <M>| M is a TM and L(M) is non-empty} recognizable/undecidable Emptiness problem: Given a TM M, check if L(M) is empty ETM = { <M> | M is a TM and L(M) is empty } Claim: ETM is unrecognizable Proof by contradiction: Suppose ETM is recognizable Then so is ~NETM We know that if a language and its negation are both recognizable, then both must be decidable So NETM is decidable, contradiction !

Establishing Unrecognizability How do we show that a language L is unrecognizable ? One technique: Show the complement language, ~L, to be recognizable-but-undecidable Second technique: Show that an unrecognizable problem reduces to L Known unrecognizable problems (so far): ETM = { <M> | M is a TM and L(M) is empty } NATM = { <M,w> | M is a TM and does not accept w } Ld = {wi | wi is not in L(Mi) } (not convenient for reductions)

Equivalence Problem for TMs EQTM = { <M, M’>| M and M’ are TMs and L(M) = L(M’) } Given two TMs, check if they accept exactly same language Decidable ? Recognizable-but-undecidable ? Unrecognizable ? ETM = { <M> | M is a TM and L(M) is empty } is unrecognizable, and EQTM does not seem any simpler Formal proof: by reducing ETM to EQTM

Proving Unrecognizability by Reduction EQTM ={ <M, M’>| M and M’ are TMs and L(M) = L(M’)}: ETM = { <M> | M is a TM and L(M) is empty } Goal: Reduce ETM to EQTM Assume there exists a TM R such that L(R) = EQTM Describe how to construct TM T such that L(T) = ETM Towards the construction: Input to T: a single TM M T wants to check whether L(M) is empty, by calling R What input machines should T give to R so that input machines of R accept same languages if and only if L(M) is empty ?

Unrecognizability Proof by Problem Reduction EQTM = { <M, M’> | M and M’ are TMs and L(M) = L(M’) } ETM = { <M> | M is a TM and L(M) is empty } Consider a TM R such that L(R) = EQTM Define the following TM T : Given input <M>: If <M> does not encode a TM, then reject Construct the description <M’> of the following TM: “Given input x, stop and reject” Execute R on input <M,M’> If R accepts, accept Claim: If <M> is in ETM, <M,M’> is in EQTM, and so, R will halt and accept, and so will T. Thus, L(T) = ETM But ETM is unrecognizable, so R cannot exist, so EQTM is unrecognizable

Singleton Problem for TMs ONETM = { <M>| M is a TM and L(M) is a singleton set } Given a TM, check if it accepts exactly one string Decidable ? Recognizable-but-undecidable ? Unrecognizable ? To solve this, we need to find a string w such that 1. M accepts w 2. M does not accept any string other than w Finding a string w such that 1 holds is recognizable, but condition 2 looks no easier than the non-membership problem (i.e. establishing that a TM does not accept a given string)

Reducing Non-membership to Singleton Problem ONETM = { <M>| M is a TM and L(M) is singleton } NATM = { <M, w> | M is a TM and M does not accept w } Goal: Reduce NATM to ONETM Assume there exists a TM R such that L(R) = ONETM Describe how to construct TM T such that L(T) = NATM Towards the construction: Input to T: a TM M and string w T wants to accept when M does not accept w, and can call R What input machine M’ should T give to R so that M’ accepts exactly one string if and only if M does not accept w ?

Key Step in Reducibility proof Given a TM M and a string w, we want to construct M’ such that L(M’) contains exactly one string if and only if M does not accept w So what should M’ do ? Description of M’: Given an input x, If (x = w), stop and accept Else execute M on w, and if M accepts w, then accept x Claim: If M accepts w, then L(M’) = S* , else L(M’) = { w }

Unrecognizability Proof for Singleton Problem ONETM = { <M> | M is a TM and L(M) is singleton } NATM = { <M, w> | M is a TM and M does not accept w } Consider a TM R such that L(R) = ONETM Define the following TM T : Given input <M, w>: If <M> does not encode a TM, then reject Construct the description <M’> of the following TM: “Given input x, If (x = w) then accept Else execute M on w, and if M accepts w, then accept” Execute R on input <M’> If R accepts, accept Claim: <M’> is in ONETM if and only if <M, w> is in NATM If ONETM is recognizable, then so is NATM But NATM is unrecognizable, so ONETM is unrecognizable

More on Construction of M’ in this proof Given a TM M and a string w, we want to construct M’ such that L(M’) contains exactly one string if and only if M does not accept w Which other M’ will work ? Description of M’: Given an input x, If ( x != w) then reject Else execute M on w; If M accepts w, accept Claim: If M accepts w, then L(M’) = { w } , else L(M’) = Empty set Thus, L(M’) is singleton if and only if M accepts w This does not work for reduction to NATM

More on Construction of T in this proof Consider a TM R such that L(R) = ONETM Define the following TM T : Given input <M, w>: If <M> does not encode a TM, then reject Construct the description <M’> of the following TM: “Given input x, If (x = w) then accept Else execute M on w, and if M accepts w, then accept” Execute R on input <M’> If R accepts, accept Caution: R is only a recognizer: guaranteed to halt only if <M’> is in ONETM This is ok, since T does not need to take any meaningful action when R does not accept Avoid construction of the form “ If R accepts, then …, else … “

Regularity Problem for TMs REGTM = { <M>| M is a TM and L(M) is regular } Given a TM M, check if there is a DFA A such that L(M)=L(A) Decidable ? Recognizable-but-undecidable ? Unrecognizable ? Can we write a computer program to detect this ? Seems no easier than the unrecognizable language ETM, which is to check if the language of a given TM is empty set (a specific regular language) Proof of unrecognizability: Reduction from NATM

Reducing Non-membership to Regularity REGTM = { <M>| M is a TM and L(M) is regular } NATM = { <M, w> | M is a TM and M does not accept w } Goal: Reduce NATM to REGTM Assume there exists a TM R such that L(R) = REGTM Describe how to construct TM T such that L(T) = NATM Given M and w, what input machine M’ should T give to R so that L(M’) is regular if and only if M does not accept w ?

Key Step in Reducibility Proof Given a TM M and string w, we want to construct M’ such that L(M’) is regular if and only if M does not accept w Description of M’: Given an input x, Execute M on w If M accepts, then If x is a palindrome, accept, else reject What is L(M’) ? If M accepts w, then what inputs does M’ accept ? Strings that are palindromes, this is a non-regular language If M does not accept w, then what inputs does M’ accept ? None, L(M’) is the empty set in this case, which is a regular language Claim: L(M’) is regular if and only if M does not accept w

Unrecognizability Proof for Regularity Problem REGTM = { <M> | M is a TM and L(M) is regular } NATM = { <M, w> | M is a TM and M does not accept w } Consider a TM R such that L(R) = REGTM Define the following TM T : Given input <M, w>: If <M> does not encode a TM, then reject Construct the description <M’> of the following TM: “Given input x, Execute M on w; If M accepts w, then if x is a Palindrome, accept else reject” Execute R on input <M’> If R accepts, accept Claim: <M’> is in REGTM if and only if <M, w> is in NATM NATM reduces to REGTM REGTM is unrecognizable

Alternatives for M’ in this proof Given a TM M and string w, we want to construct M’ such that L(M’) is regular if and only if M does not accept w Description of M’: Given an input x, Execute M on w If M accepts, then If x is a palindrome, accept, else reject What other tests will work ? “Count(x, 0) = Count(x, 1)” works The test “ x is in L” works, for any language L that is non-regular, but we also need a machine to check L, so L must be recognizable ??

Recap: Decidable Problems Questions about DFAs, NFAs, Regular expressions such as -- given a DFA M and regular expression w, is L(M) = L(r) ? Algorithms based on constructions on automata -- Computing states that are reachable from initial state -- Closure constructions (e.g. product of automata) -- Translation from regular expressions to NFAs, from NFAs to DFAs Syntactic questions about TMs/programs -- questions about states and transitions Bounded execution of TMs -- Given a TM M, an input w, and bound k, does M accept w in k steps ?

Recap: Recognizable-but-undecidable Problems Typical pattern: Problem can be formulated as Does there exist k, ranging over a countable set, such that some decidable check that depends on k -- ATM: Given M and w, is there k such that M accepts w in k steps? -- NETM: Given M, are there i and k such that M accepts wi in k steps ? -- HALTTM : Given M and w, is there k such that M halts on w in k steps -- Hilbert10: Given a polynomial p(x1, ... xn), are there integers c1, .. cn such that p(c1, … cn) = 0 ? To prove that a language is recognizable-but-undecidable: 1. Show that it is recognizable, by finding a TM that accepts 2. Show that it is undecidable, by reducing a problem such as ATM to it Exercise: { <M> | M accepts at least two strings }

Recap: Unrecognizable Problems Intuitively, even to accept, one needs to try all from an infinite set (i.e. statement involves a universal quantifier over an infinite set) -- NATM: Given M and w, for all k, M does not accept w in k steps -- ETM: Given M, for all w, M does not accept w -- EQTM : Given M and M’, for all w, M accepts w iff M’ accepts w -- REGTM : Given M, is there a DFA A such that for all w, M accepts w if and only if A accepts w -- ONETM: Given M, is there w such that for all w’ != w, M accepts w and M does not accept w’ To prove that a language is unrecognizable: Either 1. show that its complement is recognizable-but-undecidable, OR 2. reduce a problem such as NATM or ETM to it Exercise: { <M> | M accepts at most two strings }