BCS 2143 Theory of Computer Science

BCS 2143 Theory of Computer Science
Part II: Computability Chapter 5: Decidability

Unsolvable problems Computer scientists usually deal with problems which are solvable. We are supposed to find an algorithm to solve a particular problem. Why do we need to know about problems which are unsolvable ? we will not waste time in finding an non-existed algorithm. we will realize that the problem needs to be simplified or altered before we can find an algorithmic solution. unsolvable problems can stimulate our imagination and help us gain an important perspective on computation.

Decidable Languages We have seen in the last chapter an undecidable problem, the 10th problem of Helbert. Now we will show some other problems and consider whether they are decidable by algorithm or not. Use language to represent various computational problems bcs we have terminology to deal with languages.

Decidable Problems concerning Regular Language
Acceptance problem Whether a finite automata accept a string ? ADFA = {<B,w>| B is a DFA that accepts input string w} Whether a nondeterministic finite automata accepts a string ? ANFA = {<B,w>| B is a NFA that accepts input string w} Whether a regular expression generates a given string. AREX = {<R,w>| R is a regular expression that generates string w}

Decidable Problems concerning Regular Lang.
Emptiness problem Whether a language of a finite automata is empty ? EDFA = {<A>| A is a DFA and L(A) = } Equivalent problem Whether two finite automata are equivalent ? EQDFA = {<A,B>| A and B are DFAs and L(A) = L(B)}

ADFA = {<B,w>| B is a DFA that accepts input string w}
Theorem 4.1 ADFA is a decidable language ADFA = {<B,w>| B is a DFA that accepts input string w} Show that the problem of testing whether a given finite automata accepts a given string is decidable Proof idea: Present a TM M that decides ADFA.

Proof M = “on input <B,w>, where B is a DFA and w is a string:
Simulate B on input w. If the simulation ends in an accept state, accept. If it ends in a non-accepting state, reject.” Notes: input of M is an encoded string <B,w> B can be represented reasonably as a list of its five components, Q, , , q0 and F. before M starts its computation, M first checks on whether it properly represents a DFA B and a string w. If not, M rejects.

Proof: Input Tape B Machine M reject
accept reject M keeps track of B’s current state and B’s current position in the input w by writing this information down on its tape.

ANFA = {<B,w>| B is a NFA that accepts input string w}
Theorem 4.2 ANFA is a decidable language ANFA = {<B,w>| B is a NFA that accepts input string w} Proof idea: Present a TM N that decides ANFA. N has M as a subroutine. Because M is designed to work with DFAs, N first converts the NFA B it receives as input to a DFA before passing it to M.

Proof N = “on input <B,w>, where B is a NFA and w is a string:
Convert NFA B to an equivalent DFA C Run TM M from Theorem 4.1 on input <C,w>. If M accepts, accept; otherwise reject.”

Proof: Input Tape C Machine N accept reject Machine M

Theorem 4.3 AREX is a decidable language Proof idea:
AREX = {<R,w>| R is a regular expression that generates string w} Proof idea: Present a TM P that decides AREX.

Proof P = “on input <R,w>, where R is a regular expression and w is a string: Convert regular expression R to an equivalent DFA A. Run TM M on input <A,w>. If M accepts, accept; if M rejects , reject.”

EDFA = {<A>| A is a DFA and L(A) = }
Theorem 4.4 EDFA is a decidable language EDFA = {<A>| A is a DFA and L(A) = } Show that the problem of testing whether a given finite automata accepts any strings at all is decidable. Proof idea: A DFA accepts some string if and only if reaching an accept state from the start state by traveling alone the arrows of the DFA is possible. Present a TM T that uses a marking algorithm.

Proof T = “on input <A>, where A is a DFA:
Mark the start state of A. Repeat until no new states get marked: Mark any state that has a transition coming into it from any state that is already marked. If no accept state is marked, accept; otherwise reject.” A is empty

EQDFA = {<A,B>| A and B are DFAs and L(A) = L(B)}
Theorem 4.5 EQDFA is a decidable language EQDFA = {<A,B>| A and B are DFAs and L(A) = L(B)} Show that the problem of testing whether 2 DFAs recognize the same language is decidable. Proof idea: Construct a new DFA C from A and B, where C accepts only those strings that are accepted by either A or B but not by both. If A and B recognize the same language, C will accept nothing. Test whether L(C) is empty by the theorem 4.4

L(C) Symmetric difference of L(A) and L(B)
L(C) = (L(A)  L(B))  (L(A)  L(B)) L(A) L(B) Symmetric difference of L(A) and L(B) C is also a DFA as the class of regular language is closed under complementation, union and intersection

Proof F = “on input <A,B>, where A and B are DFAs:
Construct DFA C as described. Run TM T from Theorem 4.4 on input <C> If T accepts, accept. If T rejects, reject.”

Proof Machine F Machine T C accept reject

Decidable Problems concerning Context-Free Language
Acceptance problem Whether a CFG accepts a string ? ACFG = {<G,w>| G is a CFG that accepts input string w} Emptiness problem Whether a CFG generates any strings at all ? ECFG = {<G>| G is a CFG and L(G) = } Equivalent problem Whether two CFGs generate the same language ? EQCFG = {<G,H>| G and H are CFGs and L(G) = L(H)}

ACFG = {<G,w>| G is a CFG that generates string w}
Theorem 4.6 ACFG is a decidable language ACFG = {<G,w>| G is a CFG that generates string w} Proof idea: Naïve idea : Go through every derivations to determine whether any is a derivation of w. But this algorithm will not halt if w is not a member of G. Better approach: We need to ensure that the algorithm tries only finitely many derivations, so use CNF form. any derivation of w has 2n-1 steps, where n is the length of w.

Proof S = “on input <G,w>, where G is a CFG and w is a string:
Convert G to an equivalent grammar in Chomsky Normal Form. List all derivations with 2n-1 steps, where n is the length of w. If any of these derivations generate w, accept; if not, reject.”

CFLs and PDAs As we can convert back and forth between CFLs and PDAs.
Hence, everything we say about the decidability of problems concerning CFLs applies well to PDAs.

ECFG = {<G>| G is a CFG and L(G) = }
Theorem 4.7 ECFG is a decidable language ECFG = {<G>| G is a CFG and L(G) = } Proof idea: Naïve idea : to determine whether L(G) =  try going through all possible w’s, one by one. But there are infinitely many w’s to try, so this algorithm could end up running forever. Better approach: We need to test whether the start variable can generate a string of terminals. Determine for each variable whether that variable is capable of generating a string of terminals, the algorithm keeps track of this information by placing a mark on that variable.

Proof R = “on input <G>, where G is a CFG :
Mark all terminal symbols in G. Repeat until no new variables get marked Mark any variable A where G has a rule AU1 U2… Uk and each symbol U1 , …, Uk has already been marked. If the start symbol is not marked, accept; otherwise reject. ”  G is empty

ECFG = {<G,w>| G is a CFG that generates string w}
EQCFG Consider the problem of testing whether 2 CFGs generate the same language. ECFG = {<G,w>| G is a CFG that generates string w} Proof idea: Naïve idea : use the same method as theorem 4.5 But CFL is not closed under complementation or intersections. In fact, ECFG is not decidable. We’ll prove it in the next chapter.

Theorem 4.8 “Every context-free language is decidable.”
We show that every CFL is decidable. Proof idea: Naïve (bad) idea : try to convert a PDA for A directly to a TM. the PDA for A may be nondeterministic, so we get an NTM which we can convert into a deterministic TM. however, PDA can have some branches go on forever, reading and writing the stack without coming to a halt. therefore our TM will not be a decider. Better approach: We prove this theorem with the TM S in Theorem 4.6 to decide ACFG.

Proof Let G be a CFG for A and design a TM MG that decides A.
We build a copy of G into MG. It works as follow. MG = “on input w: Run TM S on input <G,w> If this machine accepts, accept; if it rejects, reject.”

Proof Machine MG Machine S Convert G to CNF form.
accept reject Convert G to CNF form. List all derivations with 2n-1 steps, where n is the length of w. If any of these derivations generate w, accept; if not, reject.

Relationship among classes of Language
Turing-recognizable decidable context-free regular

Non Recursively Enumerator language
We want to find a language that is not Recursively Enumerable This language is not accepted by any Turing Machine It can’t be implemented on computer. example: Verify that the program we wrote performs as specified (that it is correct). Halting problem : Testing whether a TM accepts a given input string (ATM) Final 2013

Theorem 4.9 ATM = {<M,w>|M is a TM and M accepts w}.
ATM is undecidable. Proof: observe that ATM is Turing-recognizable. We try to find a TM (U) that recognizes ATM.

Halting Problem U = “On input <M,w>, where M is a TM and w is a string: Simulate M on input w. If M ever enters its accept state, accept; if M ever enters its reject state, reject.” Note: U loops on input <M,w> if M loops on w, that’s why U doesn’t decide ATM.

Universal Turing Machine
proposed by Turing. a machine is called universal if it is capable to simulate any other Turing machine from the description of that machine. played an important early role in stimulating the development of stored-program computers.

The Diagonalization method
discovered by Georg Cantor in 1873. is the proof of the undecidability of the halting problem. using countable and uncountable set to prove. Idea is : we want to prove that set of Turing-recognizable language is uncountable while set of Turing machine is countable. Final 2012

Cantor’s problem Concerned with the problem of measuring the sizes of infinite sets. If we have 2 infinite sets, how can we tell whether one is larger than the other or whether they are of the same size ? Naïve: try to count the elements of the infinite set, we will never finish!

We can extend this idea to infinite sets.
Cantor’s problem Better approach: Observe that 2 finite sets are of the same size if the elements of the one set can be paired with the element of the other set. We can extend this idea to infinite sets.

Definition: Correspondence
Consider 2 sets, A and B, and a function f from A to B. Say that f is one-to-one if it never maps 2 different elements to the same place that is, f(a)  f(b) whenever a  b. Say that A and B are the same size if there is a one-to-one, onto function f: AB. one-to-one and onto is called Correspondence

Example Let N be the set of Natural numbers {1,2,3,…}  be the set of even natural numbers {2,4,6,…} Using cantor’s method, we can see that N and  have the same size. n f(n) 1 2 4 3 6 . . .

Definition : Countable
A set A is countable if either it is finite or it has the same size as N.

Example Let N be the set of Natural numbers {1,2,3,…} Q be the set of positive rational numbers Q = {m/n |m,n N } Q seems the be bigger than N but actually they have the same size. Using cantor’s method, we list all the elements of Q and then pair each element with the element of N. We have to ensure that every member of Q appears only once on the list.

Correspondence of N and Q
1 2 3 4 5 2 1 3 4 5 3 1 2 4 5 . . . 4 1 2 3 5 5 1 5 2 .

Trick We pair the first element of Q on the list with the number 1 from N. the second element of Q on the list with the number 2 from N. … After seeing the corresponding of N and Q, we can see that as each curve Q has finite number of elements which can be mapped with elements of N, thus both sets have same size.

Theorem 4.14 R is uncountable R is the set of real number.
Proof by the diagonalization method.

Proof by contradiction : suppose that a correspondence f existed between N and R.  means that we must pair all the members of N with all the members of R. show that f fails to work as it should. How to show ?  finding an x in R that is not paired with anything in N.

Proof Finding x by constructing it.
choose each digit of x to make x different from one of the real numbers that is paired with an element of N Sure that x is different from any real number that is paired.

Illustration of the idea
suppose we have a corresponding f: we construct x by let the first digit of x be anything different from the first fractional digit of f(1). let the second digit of x be anything different from the second fractional digit of f(2). … thus we can construct x which is different from f(n) for any n. n f(n) 1 2 3 . . . n f(n) 1 2 3 . . .

Corollary 4.15 Some languages are not Turing-recognizable. Proof idea:
set of Turing-recognizable language is uncountable while set of Turing machine is countable. Because each Turing machine can recognize a single language and there are more languages than TMs, some languages are not recognized by any TM. Thus, there exists some languages which are not Turing-recognizable.

Set of all TMs is countable.
Proof idea: first of all we have to observe that the set of all * is countable, for any alphabet . with only finitely many strings of each length, we may form a list of * by written down all strings of length 0, length 1, length 2 and so on.

Set of all TMs is countable.
Secondly We can encode each TM into a string <M> as the set of all strings is countable therefore the set of all TMs is countable.

Set of all languages is uncountable
Proof idea: first of all we have to observe that the set of all infinite binary sequence is uncountable An infinite binary sequence is an unending sequence of 0s and 1s. we can prove by the diagonalization method. (imagine as the same way as R)

Secondly Let B be the set of all infinite binary sequences. Let L be the set of all languages over alphabet . Show that L is uncountable by giving correspondence with B. the idea is that “as B is uncountable thus L is also uncountable.”

Correspondence of L and B Show that both sets have the same size. Let * = {s1,s2,…}. Each language A L has a unique sequence of B. (one-to-one and onto) Characteristic sequence of A, (A) The ith bit of that sequence is a 1 if si  A and is a 0 if si  A.

Example : Characteristic sequence The function f: L  B, where f(A) equals the Characteristic sequence of A, is one-to-one and onto and hence a correspondence. Therefore, as B is uncountable, L is uncountable as well. * = {  , 1 00 01 10 11 000 001 … } ; A (A)

The halting problem We have proved that there are some languages which are not Turing-recognizable. Now, we want to show that such a TM which can recognize the non Turing-recognizable language doesn’t exist.

Proof of Theorem 4.9 ATM is undecidable. Proof:
Consider  = {a}. The set of all strings is * = {,a, aa, aaa, aaaa…} = {,a1, a2, a3, a4…} the set of all TMs is countable, thus we can order them as M1, M2, M3,… to indicate which language is accepted by what machine, we can again use the characteristic sequence of that language.

Proof of Theorem 4.9 Example:
assume a machine Mi accepts a language L(Mi) L(Mi) = {a2, a4, a6} we can write * = { a1 , a2 a3 a4 a5 a6 a7 a8 ,, … } ; L(Mi) 1

Proof of Theorem 4.9 For each machine, we can write its characteristic sequence

Proof of Theorem 4.9 Consider the language
consists from the 1’s in the diagonal

Proof of Theorem 4.9 Since L is a properly defined language,
we can consider its complement Consider the language consists from the 0’s in the diagonal

Proof of Theorem 4.9 Claim 1: Language is not recursively enumerable
if we can prove that it is not recursively enumerable, we can now find a language which is not Turing recognizable.

Proof: Claim 1 Assume for contradiction that is recursively enumerable
There must exist some machine that accepts

Question:

Answer:

Question: Answer:

Proof: Claim 1 Similarly: for any Because either: or

Proof: Claim 1 Therefore, the machine cannot exist
Therefore, the language is not recursively enumerable End of Proof

(otherwise would be accepted by
Observation: There is no algorithm that describes (otherwise would be accepted by some Turing Machine)

Recursive Recursively Enumerable Non Recursively Enumerable

Testing whether TM accepts a given string : ATM
Try to construct a TM (H) to decide whether another TM (M) accepts a string (w) or not. H is decider on (M,w) accept if M accepts w H(M,w) = reject if M rejects w

If w = the encoded string <M> H is decider on (M,<M>) accept if M accepts <M> H(M,<M>) = reject if M rejects <M>

Now we construct a new TM (D) with H as a subroutine. D does opposite to what its subroutine does. D = “On input <M>, where M is a TM, Run H on input <M,<M>) Output the opposite to what H outputs; that is, if H accepts, reject and if H rejects, accept.”

accept if H rejects <M> D(<M>) = reject if H accepts <M> accept if M rejects <M> reject if M accepts <M>

D(<M>) H (M,<M>) M (<M>) if M accepts <M> then H accepts if M rejects <M> then H rejects accept reject

running M on <M>

output of H

output of D

A Language which is Recursively Enumerable and not Recursive

We want to find a language which
Is recursively enumerable But not recursive There is a Turing Machine that accepts the language The machine doesn’t halt on some input

Is recursively enumerable
Proof We will prove that the language Is recursively enumerable but not recursive

Theorem: Proof: The language is recursively enumerable
We will give a Turing Machine that accepts

Turing Machine that accepts
For any input string Compute , for which Find Turing machine (using the enumeration procedure for Turing Machines) Simulate on input If accepts, then accept End of Proof

Observation: Recursively enumerable Not recursively enumerable
(Thus, also not recursive)

Theorem: Proof: The language is not recursive
Assume for contradiction that is recursive

Proof Then is recursive: Take the Turing Machine that accepts
halts on any input: If accepts then reject If rejects then accept

Proof Therefore: is recursive But we know:
is not recursively enumerable thus, not recursive CONTRADICTION!!!! Therefore, is not recursive End of Proof

Non Recursively Enumerable

Theorem 4.16 A language is decidable (recursive) if and only if it is both Turing-recognizable and co-Turing-recognizable. co-Turing-recognizable language is the the complement of a Turing-recognizable language.

Corollary 4.17 ATM is not Turing-recognizable. Proof:
ATM is Turing-recognizable. if is Turing-recognizable, ATM would be decidable. But ATM is not decidable, thus must not be Turing-recognizable. ATM ATM

BCS 2143 Theory of Computer Science

Similar presentations

Presentation on theme: "BCS 2143 Theory of Computer Science"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

BCS 2143 Theory of Computer Science

Similar presentations

Presentation on theme: "BCS 2143 Theory of Computer Science"— Presentation transcript:

Similar presentations

About project

Feedback