1. Regular Expressions and Automata in Language Analysis
Lecture 2
Regular Expressions and Automata in Language Analysis
CS 4705

2. Statistical vs. Symbolic (Knowledge Rich) Techniques
How much linguistic knowledge do our representations and algorithms need to have to do ‘successful’ NLP?
Bill hit John.
John, Bill hit.
80/20 Rule: when do we need to worry about the other 20%?

3. Today
Review some of the simple representations and ask ourselves how we might use them to do interesting and useful things
Regular Expressions
Finite State Automata
Think about the limits of these simple approaches: when do we need more?

4. Uses of Regular Expressions in NLP
As grep, perl: Simple but powerful tools for large corpus analysis and ‘shallow’ processing
What word is most likely to begin a sentence?
What word is most likely to begin a question?
How often do people end sentences with prepositions?
With other unix tools, allow us to
Obtain word frequency and co-occurrence statistics
Build simple interactive applications (e.g. Eliza)
Authorship: Who wrote Shakespeare’s plays? The Federalist papers? The Unibomber letters?

5. A Quick Review A non-blank line Any character /./ /[^A-Z][a-z]*/
Any non-u.c. char
/[^A-Z]/
/[A-Z][a-z]*/
Any u.c. letter
/[A-Z]/
Any l.c. letter
/[a-z]/
/[bckmrs]ite/
Any of these chars
/[bckmsr]/
Any ‘?’
/?/
Possible use
Matches RE
A question
/[a-z]ite/

6. RE
Description
Uses?
/a*/
Zero or more a’s
/(very[ ])*/
/a+/
One or more a’s
/(very[ ])+/
/a?/
Zero or one a’s
/(very[ ])?/
/cat|dog/
‘cat’ or ‘dog’
/[a-z]* (cat|dog)/
/^no$/
A line with only ‘no’ in it
/\bun\B/
Prefixes
Words prefixed by ‘un’

7. RE
E.G.
/pupp(y|ies)/
Morphological variants of ‘puppy’
/ (.+)ier and \1ier /
happier and happier, fuzzier and fuzzier

8. Substitutions (Transductions)
Sed or ‘s’ operator in Perl
s/regexp1/pattern/
s/I am feeling (.+)/You are feeling \1?/
s/I gave (.+) to (.+)/Why would you give \2 \1?/

9. Examples Predictions from a news corpus: Language use:
Which candidate for Governor is mentioned most often in the news? Is going to win?
What stock should you buy?
Which White House advisers have the most power?
Language use:
Which form of comparative is more frequent: ‘Xer’ or ‘more X’?
Which pronouns occur most often in subject position?
How often do sentences end with infinitival ‘to’?
What words most often begin and end sentences?
What are the 20 most common words in your ? In the news? In Shakespeare’s plays?

10. Emotional language: What words indicate what emotions? Happiness Anger
Confidence
Despair
How can we identify emotions automatically?

11. Finite State Automata
FSAs recognize the regular languages represented by regular expressions
SheepTalk: /baa+!/ q0 q4 q1 q2 q3 b a !
Directed graph with labeled nodes and arc transitions
Five states: q0 the start state, q4 the final state, 5 transitions

12. Formally FSA is a 5-tuple consisting of
Q: set of states {q0,q1,q2,q3,q4}
: an alphabet of symbols {a,b,!}
q0: a start state in Q
F: a set of final states in Q {q4}
(q,i): a transition function mapping Q x  to Q q0 q4 q1 q2 q3 b a !

13. a b ! FSA recognizes (accepts) strings of a regular language
baa!
baaa!
baaaa! …
Tape metaphor: will this input be accepted? a b !

14. State Transition Table for SheepTalk
Input b a ! 1 2 3 4

15. Non-Deterministic FSAs for SheepTalkq0 q4 q1 q2 q3 b a ! b a a ! q0 q1 q2 q3 q4 

16. Problems of Non-Determinism
At any choice point, we may follow the wrong arc
Potential solutions:
Save backup states at each choice point
Look-ahead in the input before making choice
Pursue alternatives in parallel
Determinize our NFSAs (and then minimize)
FSAs can be useful tools for recognizing – and generating – subsets of natural language
But they cannot represent all NL phenomena (center embedding: The mouse the cat chased died.)

17. Simple vs. linguistically rich representations….
How do we decide what we need?

18. FSAs as Grammars for Natural Languagedr
the
rev mr
pat l.
robinson q0 q1 q2 q3 q4 q5 q6 ms
hon 
mrs 

19. If we want to extract all the proper names in the news, will this work?
What will it miss?
Will it accept something that is not a proper name?
How would you change it to accept all proper names without false positives?
Precision vs. recall….

20. Summing Up
Regular expressions and FSAs can represent subsets of natural language as well as regular languages
Both representations may be impossible for humans to understand for any real subset of a language
But they are relatively easy to use for small subsets
Can be hard to scale up: when many choices at any point (e.g. surnames)
Next time: Read Ch 3
Class participation opportunity:
Do the experiment at and print a copy of the final page of the experiment to show you’ve done it