1. Lexical Analysis Uses formalism of Regular Languages
Regular Expressions
Deterministic Finite Automata (DFA)
Non-deterministic Finite Automata (NDFA)
RE  NDFA  DFA  minimal DFA
(F)Lex uses RE as input, builds lexor

2. DFAs: Formal Definition
DFA M = (Q, S, d, q0, F)
Q = states finite set
S = alphabet finite set
d = transition function function in Q  S  Q
q0 = initial/starting state q0  Q
F = final states F  Q

3. strings over {a,b} with next-to-last symbol = a
DFAs: Example
strings over {a,b} with next-to-last symbol = a
…aa
…ab a
…ba
…bb e b

4. Nondeterministic Finite Automata
“Nondeterminism” implies having a choice.
Multiple possible transitions from a state on a symbol.
d(q,a) is a set of states
d : Q  S  Pow(Q)
Can be empty, so no need for error/nonsense state.
Acceptance: exist path to a final state?
I.e., try all choices.
Also allow transitions on no input:
d : Q  (S  {e})  Pow(Q)

5. NFAs: Formal Definition
NFA M = (Q, S, d, q0, F)
Q = states a finite set
S = alphabet a finite set
d = transition function a total function in
Q  (S  {e})  Pow(Q)
q0 = initial state q0  Q
F = final states F  Q

6. NFAs: Example strings over {a,b} with next-to-last symbol = a …aS …a …
Loop until we “guess” which is the next-to-last a. a S
…aS …a …

7. NFAs: Example strings over {0,1,2} having
(either 0-or-more 0’s or 0-or-more 1’s)
followed by 0-or-more 2’s e 2s 1 2 0s 1s

8. Regular Expressions Regular expression (over S)  e a where aS r+r’
where r,r’ regular (over S)
Notational shorthand:
r0 = e, ri = rri-1
r+ = rr*

9. RE  NFA Defined inductively on structure of RE.
This construction produces NFA with single final state.
6 cases: , e, a, r’+r’’, r’r’’, r’*

10. Accepts nothing since no edge to final state.
RE  NFA:  q0 qf
Accepts nothing since no edge to final state.

11. RE  NFA: e q0

12. RE  NFA: a q0 a qf

13. e edges guess whether to use r’ or r’’.
RE  NFA: r’+r’’
q’0
q’f
q’’0
q’’f
e edges guess whether to use r’ or r’’. qf q0 e

14. Could conflate q0 with q’0, q’’f with qf.
RE  NFA: r’r’’
q’0
q’f
q’’0
q’’f
Could conflate q0 with q’0, q’’f with qf. e q0 qf

15. Can loop r’ as many times as desired or skip it.
RE  NFA: r’*
Can loop r’ as many times as desired or skip it. q0 e qf
q’0
q’f

16. RE  NFA: Example
(0+01)* e 1 e e

17. RE  NFA: Notes Most constructions produce very large NFAs.
Not optimal for size.
But easy to construct.

18. NFA -> DFA Subset Construction
Complicated but well described in the text
Section (pp ), Algorithm 3.20 (2nd edition)
In section 3.6 (pp ) in 1st edition

19. Minimizing DFA
Partition states of DFA, D, into two sets, final states, and non-final states.
Continue until no more partitions are needed
For each partition, P, split the DFA states of P so that, for each subpartition, all DFA states in that partition have the same transition for each input symbol, x.