Theory of Computation 1 Theory of Computation Peer Instruction Lecture Slides by Dr. Cynthia Lee, UCSD are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License. Based on a work at Cynthia Lee, UCSDCreative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported Licensewww.peerinstruction4cs.org

What is “Theory of Computation”? The content of many CS courses is perhaps self-explanatory: – Programming in Java – Compilers – Artificial Intelligence For many of you, what exactly will be covered in “Theory of Computability” may be somewhat mysterious… 2

Incredibly Practical Finite Automata – Hardware design, software engineering Regular Expressions – A “must-have” programmer’s tool and resume skills bullet point – Foundational for many programming and scripting tasks (Perl) Context-Free Grammars – Linguistics – Artificial Intelligence – Machine translation (English->Korean) Turing Machines – All modern computers and programming languages! – Computer architecture (CPU/instructions + RAM) 3 What is “Theory of Computation”?

Breathtakingly Beautiful 4 What is “Theory of Computation”?

How is this exploration of Theory of Computation structured? We will examine a series of abstracted models of computation machines – Start with the most limited – Move towards the most versatile* Along the way, – Learn how to use each model as a handy tool in your programming life – Become familiar with the contours of each model’s limitations and powers – Prove things about the limitations, about the powers, and about the relationships between models * Fun fact: this simple-to-complex order is not always the same order in which these were created/discovered 5

What do I do in this course? Be your guide in inducing you to explore concepts Create situations and pose problems that set the scene for your exploration Provide you with feedback Evaluate your progress Answer your questions Not spend lecture reading the textbook to you 6

What do you do in this course? Prepare your brain for maximum in-class learning – Reading, reading quizzes In class: engage with yourself, your neighbors and the class, engage with the ideas—turn them upside down and sideways – iClickers, group discussion Solidify, refine and perfect your skills – Homework Certify what you’ve learned, celebrate victories, redirect as needed – Exams Seek help and seek to help others – In class, forums, office hours, discussion section 7

Rules for what you do in this course Class Participation – Yes: “Click in” and engage actively with your group and with me, even when (or especially when!) you are unsure or struggling – The only way to get through a struggle is to work – If you let me know that you are struggling, by asking a question or by clicker response, I can help—this is a huge favor to me because it helps me steer the lesson in the most helpful direction Treat other class members kindly and be conscious of creating a welcoming, positive, collaborative team environment – No: No clicking in for someone who isn’t here, or otherwise feigning participation I know that everyone has off days, sleepy days, etc, and that’s fine. But please don’t ever allow your behavior to become distracting or rude to anyone else 8

Thought of the Week “To be a good geek you have to have both humility and arrogance in equal measures. The humility is so you’ll admit you don’t know something and get help/read the docs/etc. The arrogance is the bit that says “I don’t know that now… but I can and I will soon.”” --Thomas Beagle, IT/programmer 9

AUTOMATA COMPUTATIONAL MODELS Let’s get started! 10

What Do Automata Do? Input strings, output “accept” or “reject” Every automaton is defined over an alphabet Σ – Ex: Σ = {a,b,c,…,z} or Σ = {0,1} What does a particular automaton do? 1.Test every possible string made from Σ 2.Come up with a set of all the strings that were accepted 11

DETERMINISTIC FINITE AUTOMATA DFA They have almost no memory! 12

DFA 13

Tracing in a DFA a)q0, q1, q2, q2, q3 b)q0, q1, q1, q2, q2, q3 c)q0, q1, q1, q1, q2, q2, q3 d)q0, q1, q1, q1, q1, q2, q2, q3 What is the sequence of states that are followed when running this DFA on the input “111001”? 14

Tracing in a DFA a)q0, q1, q2, q3 b)q0, q1, q2, q3, q3 c)q0, q1, q2, q3, q3, q3 d)q0, q1, q2, q3, q3, q3, q3 What is the sequence of states that are followed when running this DFA on the input “001000”? 15

Quick review of set notation Let’s dissect this (assume Σ = {0,1}): L 1 = { w | length(w) > 3} 16

Quick review of set notation Which choice is NOT a string in language L 1 ? (Σ = {a,b,c}) L 1 = {w | w contains more b’s than a’s} a)“abbcbba” b)“cab” c)“bba” d)“bccccbcccc” e)“bacb” 17

Which is the best description of the language recognized by the DFA M 1 ? (Σ = {a,b}) a)Starts with b and ends with a or b b)Starts with a and ends with a or b c)a‘s followed by b’s M1:M1: 18

Which is the best description of the language recognized by the DFA M 2 ? (Σ = {a,b}) a)Starts with b and ends with b b)Starts with a and ends with b c)b‘s followed by a’s M2:M2: 19

START DESIGNING YOUR OWN DFA Now that you’ve seen how DFAs work: 20

Which states should be in F (the set of “final” or “accept” states) so that DFA M 1 recognizes the language L 1 = {w | b’s never appear after a’s in w} (Σ = {a,b}) a)F = {q2} b)F = {q3} c)F = {q1, q2} d)F = {q1, q3} e)F = {q2, q3} M1 :M1 : 21

Which states should be in F (the set of “final” or “accept” states) so that DFA M 2 recognizes the language L 2 = {w | w contains less than 1 a or more than 1 a}? (Σ = {a,b}) a)F = {q2} b)F = {q3} c)F = {q1, q2} d)F = {q1, q3} e)F = {q2, q3} M2 :M2 : 22

Formal Definition of a DFA Did you know you can specify a DFA (not just its language) without drawing a single circle or arrow? A DFA M 1 is defined as a 5-tuple as follows: M 1 = (Q, Σ, δ, q0, F), where: – Q is a finite set of states – Σ is a finite set of characters, the alphabet – δ: Q x Σ -> Q, the transition function – q0, a member of Q, the start state – F, a subset of Q, the accept state(s) We’ve covered several of these already: Q, Σ, q0, F Let’s take a closer look at δ 23

a) s b) s c) s δ: Q x Σ -> Q Transition Table “b’s never appear after a’s” ab q1q2q1 q2 q3 ab q1q2q1 q2q3 q2q3 ab q1 q2 q3q2 q3 24

a) s b) s c) s δ: Q x Σ -> Q Transition Table {w | |w| is odd} ab q1 q2 q1q2 ab q1q2 q1 ab q2 25

Regular Languages Mathematical objects so far: – Character: a or ‘a’ – String: ababba or “ababbba” – Alphabet (a set of characters): {a,b} – Language (a set of strings): {w | |w| is 5} Now we introduce a particular set of languages: The Regular Languages – A set of sets of strings 26

Regular languages The set of all languages for which there exists a DFA that recognizes it How do we know what is in this set? – What are the boundaries? – (Remember I said that in this course, we would “Become familiar with the contours of each model’s limitations and powers & Prove things about the limitations, about the powers…”) 27

Deterministic Finite Automata: Finiteness Deterministic Finite Automata (DFAs) can only recognize finite languages (a language with 1 string or 2 strings or 10 strings or 1,003,078 strings in it), not infinite languages (infinite number of strings in the language) a)TRUE b)FALSE 28

Deterministic Finite Automata: Uniqueness Each DFA recognizes a unique language, in other words, no two DFAs recognize the same language. a)TRUE b)FALSE 29

Thm. The class of regular languages is closed under the union operation. What does this mean? How can we prove this? 30

Closure Properties/”Closed Under” Which of the of the statements is FALSE? a)Integers are closed under addition b)Integers are closed under division c)Even numbers are closed under addition 31

Closure Properties/”Closed Under” Which of the of the statements is FALSE? a)Real numbers are closed under addition b)Real numbers are closed under division c)Odd numbers are closed under addition 32

Thm. The class of regular languages is closed under the union operation. b's never appear after a’s {w | length(w) is odd} 33 r1 r2

Thm. The class of regular languages is closed under the union operation. Given two regular languages L1, L2, with DFAs M1, M2 – M1 = (Q1,Σ,δ1,q 0 1,F1) – M2 = (Q2,Σ,δ2, q 0 2,F2) We want to construct a DFA M = (Q,Σ,δ, q 0,F), s.t.: – Q = Q1 x Q2 – δ = – q 0 = – F = M recognizes L1 U L2 A DFA recognizes L1 U L2, then L1 U L2 must be regular Our working example: A generalized proof form: ab (r1,q1)(r2,q2)(r2,q1) (r1,q2)(r2,q2)(r2,q3) (r1,q3)(r2,q3) (r2,q1)(r1,q2)(r1,q1) (r2,q2)(r1,q2)(r1,q3) (r2,q3)(r1,q3)

Thm. The class of regular languages is closed under the union operation. Our working example: A generalized proof form: a)q 0 = (r1,q1) b)q 0 = (q 0 1, q 0 2) c)q 0 = {(r1,q1)} ab (r1,q1)(r2,q2)(r2,q1) (r1,q2)(r2,q2)(r2,q3) (r1,q3)(r2,q3) (r2,q1)(r1,q2)(r1,q1) (r2,q2)(r1,q2)(r1,q3) (r2,q3)(r1,q3) Given two regular languages L1, L2, with DFAs M1, M2 – M1 = (Q1,Σ,δ1,q 0 1,F1) – M2 = (Q2,Σ,δ2, q 0 2,F2) We want to construct a DFA M = (Q,Σ,δ, q 0,F), s.t.: – Q = Q1 x Q2 – δ = – q 0 = – F = M recognizes L1 U L2 A DFA recognizes L1 U L2, then L1 U L2 must be regular

Thm. The class of regular languages is closed under the union operation. Our working example: A generalized proof form: a)F = {(r1,q1), (r1,q2), (r2,q1), (r2q2), (r2,q3)} b)F = F1 x F2 c)F = F1 U F2 d)F = {(x,y) in Q | x in F1 or y in F2} e)None of the above ab (r1,q1)(r2,q2)(r2,q1) (r1,q2)(r2,q2)(r2,q3) (r1,q3)(r2,q3) (r2,q1)(r1,q2)(r1,q1) (r2,q2)(r1,q2)(r1,q3) (r2,q3)(r1,q3) Given two regular languages L1, L2, with DFAs M1, M2 – M1 = (Q1,Σ,δ1,q 0 1,F1) – M2 = (Q2,Σ,δ2, q 0 2,F2) We want to construct a DFA M = (Q,Σ,δ, q 0,F), s.t.: – Q = Q1 x Q2 – δ = – q 0 = – F = M recognizes L1 U L2 A DFA recognizes L1 U L2, then L1 U L2 must be regular

Thm. The class of regular languages is closed under the union operation. Given two regular languages L1, L2, with DFAs M1, M2 – M1 = (Q1,Σ,δ1,q 0 1,F1) – M2 = (Q2,Σ,δ2, q 0 2,F2) We want to construct a DFA M = (Q,Σ,δ, q 0,F), s.t.: – Q = Q1 x Q2 – δ = – q 0 = – F = M recognizes L1 U L2 A DFA recognizes L1 U L2, then L1 U L2 must be regular Our working example: A generalized proof form: a)δ: Q1 x Q2  Q b)δ: Q1 x Q2 x Σ  Q c)δ((x,y),c) = (δ1(x,c), δ2(y,c)), for c in Σ and (x,y) in Q d)δ((r1,q1), a) = (r2,q2) ab (r1,q1)(r2,q2)(r2,q1) (r1,q2)(r2,q2)(r2,q3) (r1,q3)(r2,q3) (r2,q1)(r1,q2)(r1,q1) (r2,q2)(r1,q2)(r1,q3) (r2,q3)(r1,q3)

Thm. The class of regular languages is closed under the union operation. Proof: Given: Two regular languages L1, L2. Want to show: L1 U L2 is regular. Because L1 and L2 are regular, we know there exist DFAs M1 = (Q1,Σ,δ1,q01,F1) and M2 = (Q2,Σ,δ2,q02,F2) that recognize L1 and L2. We construct a DFA M = (Q,Σ,δ,q0,F), s.t.: – Q = Q1 x Q2 – δ = – q0 = – F = M recognizes L1 U L2. A DFA recognizes L1 U L2, then L1 U L2 is regular. Q.E.D.

Thm. The class of regular languages is closed under the union operation. Proof: Given: Two regular languages L 1, L 2. Want to show: L 1 U L 2 is regular. Because L 1 and L 2 are regular, we know there exist DFAs M 1 = (Q 1,Σ,δ 1,q 01,F 1 ) and M 2 = (Q 2,Σ,δ 2,q 02,F 2 ) that recognize L 1 and L 2. We construct a DFA M = (Q,Σ,δ,q 0,F), s.t.: – Q = Q 1 x Q 2 – δ((x,y),c) = (δ 1 (x,c), δ 2 (y,c)), for c in Σ and (x,y) in Q – q 0 = (q 01, q 02 ) – F = {(x,y) in Q | x in F 1 or y in F 2 } M recognizes L 1 U L 2. Correctness: ___________________________________ A DFA recognizes L 1 U L 2, so L 1 U L 2 is regular, and the class of regular languages is closed under union. Q.E.D.