1. The Chomsky Hierarchy

2. Sentences The sentence as a string of words E
Sentences The sentence as a string of words E.g I saw the lady with the binoculars string = a b c d e b f

3. The relations of parts of a string to each other may be different I saw the lady with the binoculars is stucturally ambiguous Who has the binoculars?

4. [ I ] saw the lady [ with the binoculars ] = [a] b c d [e b f] I saw [ the lady with the binoculars] = a b [c d e b f]

5. How can we represent the difference
How can we represent the difference? By assigning them different structures. We can represent structures with 'trees'. I read the book

6. a. I saw the lady with the binoculars S NP VP V NP NP PP I saw the lady with the binoculars I saw [the lady with the binoculars]

7. b. I saw the lady with the binoculars S NP VP VP PP I saw the lady with the binoculars I [ saw the lady ] with the binoculars

8. birds fly S NP VP N V birds fly S → NP VP NP → N VP → V
Syntactic rules
Graphs and trees

9. S
NP VP
birds fly
a b
ab = string
Graphs and trees

10. S A B a b ab S → A B A → a B → b
Graphs and trees

11. Rules
Assumption:
natural language grammars are a rule-based systems
What kind of grammars describe natural language phenomena?
What are the formal properties of grammatical rules?

12. Chomsky (1957) Syntactic Struc-tures. The Hague: Mouton
Chomsky, N. and G.A. Miller (1958) Finite-state languages Information and Control 1,
Chomsky (1959) On certain formal properties of languages. Information and Control 2,

13. Rules in Linguistics 1. PHONOLOGY
Rules in Linguistics   1. PHONOLOGY /s/ → [θ]  V ___V   Rewrite /s/ as [θ] when /s/ occurs in context V ____ V   With: V = auxiliary node s, θ = terminal nodes

14. Rules in Linguistics 2. SYNTAX. S. →. NP VP. VP. →. V. NP. →
Rules in Linguistics    2. SYNTAX S → NP VP VP → V NP → N Rewrite S as NP VP in any context With: S, NP, VP = auxiliary nodes V, N = terminal node

15. PHONOLOGY (sound system) Maltese – Word-final devoicing Orthography Pronunciation (spelling) (sound) Sabet sab [sa-bet] [sap] Ħobża ħobż [hob-za] [hops] Vjaġġi vjaġġ [vjağ-ği] [vjačč] voiced [+vd] voiceless [-vd] [b, z, ğ] [p, s, č] [+vd] → [-vd] /____ # (for # = end of word)

16. MORPHOLOGY (word formation) Maltese – Progressive assimilation in 3fsg imprefective (present) Marker for verb in 3rd person feminine singular imperfective t- (3fsgimpf = she) e.g. she breaks = t-kisser I break = n-kisser t-kisser t-ressaq 3fsg-break 3fsg-move she breaks she moves s-sakkar d-dur 3fsg-lock 3fsg-turn she locks she turns *t-sakkar * t-dur t → s,d,etc. /____ [s,d,etc. | [+cor] μ [3fsg] (with μ = morpheme, C = consonant, cor = coronal

17. SYNTAX (phrase/sentence formation) sentence: The boy kissed the girl Subject predicate noun phrase verb phrase art + noun verb + noun phrase S → NP VP VP → V NP NP → ART N

18. SEMANTICS (meaning) The lion attacks the hunter attack (a, b) a λy [attack (y, b)] λz λy [attack (y, z)] b (with a = the lion, b = the hunter)

19. 0. Type 0 (recursively enumerable) languages
Chomsky Hierarchy
0. Type 0 (recursively enumerable) languages
Only restriction on rules: left-hand side cannot be the empty string (* Ø  …….)
1. Context-Sensitive languages - Context-Sensitive (CS) rules
2. Context-Free languages - Context-Free (CF) rules
3. Regular languages - Non-Context-Free (CF) rules
0 ⊇ 1, 1 ⊇ 2, 2 ⊇ 3
a ⊇ b meaning a properly includes b (a is a superset of b),
i.e. b is a proper subset of a or b is in a

20. Generative power 0. Type 0 (recursively enumerable) languages Only restriction on rules: left-hand side cannot be the empty string (* Ø  …….) is the most powerful system 3. Type 3(regular language) is the least powerful

21. Superset/subset relation
S1 S2
a c b d
f g a b

22. Rule Type – 3 Name: Regular Example: Finite State Automata (Markov-process Grammar) Rule type: a) right-linear A  xB or A  x with: A, B = auxiliary nodes and x = terminal node b) or left-linear A  Bx or Generates: ambn with m,n  1 Cannot guarantee that there are as many a’s as b’s; no embedding

23. A regular grammar for natural language sentences S → the A A → cat B A → mouse B A → duck B B → bites C B → sees C B → eats C C → the D D → boy D → girl D → monkey the cat bites the boy the mouse eats the monkey the duck sees the girl

24. Regular grammars
Grammar 1: Grammar 2:
A → a A → a
A → a B A → B a
B → b A B → A b
Grammar 3: Grammar 4:
B → b B → b
Grammar 5: Grammar 6:
S → a A A → A a
S → b B A → B a
A → a S B → b
B → b b S B → A b
S →  A → a

25. Grammars
Grammar 6: Grammar 7:
S → A B A → a
S → b B A → B a
A → a S B → b
B → b b S B → b A
S → 

26. Finite-State Automaton article noun NP NP1 NP2 adjective

27. NP article NP1 adjective NP1 noun NP2 NP → article NP1 NP1 →adjective NP1 NP1 → noun NP2

28. A parse tree S root node NP VP interior nodes N V NP DET N terminal nodes

29. Rule Type – 2 Name: Context Free Example: Phrase Structure Grammars/ Push-Down Automata Rule type: A   with: A = auxiliary node  = any number of terminal or auxiliary nodes Recursiveness allowed: A  A

30. CF Grammar
 A Context Free grammar consists of:
a) a finite terminal vocabulary VT
b) a finite auxiliary vocabulary VA
c) an axiom S  VA
a finite number of context free rules of
form A → γ,
where A  VA
and γ  {VA  VT}*
In natural language syntax S is interpreted as the start symbol for sentence, as in S → NP VP

31. CF Grammars
The following languages cannot be generated by a regular grammar
Language 1: Language 2:
 anbn mirror image
ab abaaba
aabb abbaabba
Context-Free rules:
A → a A a A → a A a
A → a b A → b A b

32. Natural language Is English regular or CF
Natural language Is English regular or CF? If centre embedding is required, then it cannot be regular Centre Embedding: 1. [The cat] [likes tuna fish] a b 2. The cat the dog chased likes tuna fish a a b b 3. The cat the dog the rat bit chased likes tuna fish a a a b b b 4. The cat the dog the rat the elephant admired bit chased likes tuna fish a a a a b b b b ab aabb aaabbb aaaabbbb

33. Centre embedding S
  NP VP
the likes
cat tuna
a b
= ab

34. S
  NP VP
likes
NP S tuna
the b
cat NP VP
a the chased
dog b a
= aabb

35. S NP VP likes NP S tuna the b cat NP VP a chased NP S b the dog NP VP a the bit rat b a = aaabbb

36. Natural language
Is English regular or CF?
If centre embedding is required, then it cannot be regular

37. Centre Embedding 1. [The cat] [likes tuna fish] a b = ab 2
Centre Embedding 1. [The cat] [likes tuna fish] a b = ab 2. [The cat] [the dog] [chased] [likes tuna fish] a a b b = aabb

38. [The cat] [likes tuna fish]
a b
2. [The cat] [the dog] [chased] [likes ...]
a a b b

39. 3. [The cat] [the dog] [the rat] [bit] [chased] [likes ...]
a a a b b b
[The cat] [the dog] [the rat] [the elephant] [admired] [bit] [chased] [likes ....] =
a a a a b b b b
aaabbb
aaaabbbb

40. Natural language 2 More Centre Embedding: 1. If S1, then S2 a a 2
Natural language 2 More Centre Embedding: 1. If S1, then S2 a a 2. Either S3, or S4 b b 3. The man who said S5 is arriving today  4. The man who said S6 is arriving the day after Sentence with embedding: If either the man who said S5 is arriving today or the man who said S5 is arriving tomorrow, then the man who said S6 is arriving the day after abba = abba

41. Natural language 2
More Centre Embedding:
1. If S1, then S2
a a
2. Either S3, or S4
b b
Sentence with embedding:
If either the man is arriving today or the woman is arriving tomorrow, then the child is arriving the day after.
a = [if
b = [either the man is arriving today]
b = [or the woman is arriving tomorrow]]
a = [then the child is arriving the day after]
= abba

42. CS languages The following languages cannot be generated by a CF grammar (by pumping lemma): anbmcndm Swiss German: A string of dative nouns (e.g. aa), followed by a string of accusative nouns (e.g. bbb), followed by a string of dative-taking verbs (cc), followed by a string of accusative-taking verbs (ddd) = aabbbccddd = anbmcndm

43. Swiss German: Jan sait das (Jan says that) … mer em Hans es Huus hälfed aastriiche we Hans/DAT the house/ACC helped paint we helped Hans paint the house abcd NPdat NPdat NPacc NPacc Vdat Vdat Vacc Vacc a a b b c c d d

44. Natural language 3 Inadequacy of phrase structure rules (CF rules) Transformations: Passive NP1 – Aux – V – NP2 → NP2 – Aux + be – V – by + NP1 Transformations are Turing powerful, i.e. can do anything to anything: inversion, deletion

45. Developments in syntax a) do away with or severely constrain: 1
Developments in syntax a) do away with or severely constrain: 1. Phrase Structure rules 2. Transformational Rules b) move away from: derivational/procedural models to: constraint-based/declarative models Head-Driven Phrase Structure Grammar (HPSG) and Optimality Theory (OT) c) development of context-free rules with non-terminals structured as sets of features and values E.gs. N = [+N, -V] V = [-N, +V] sleeps = [-N,+V,-PST,AGR:[+N,-V,+3,-PLU]]

46. Rules in Linguistics Traditional syntactic rules VP → V NP NP → DET N PP → P NP etc. X-bar syntax (' = bar/level one, '' = bar/level two) N'' → DET N’ N' → N V'' → ADV V’ V' → V A'' → DEG A’ A' → A P'' → DET P’ P' → P

47. X-bar rule schema X’’ → X’ X’’ X’ → X │ X’ │ X

48. X-bar Syntax X’’ → (SPEC) X’ X’’ → X’’ MODIFIER X’ → X’ MODIFIER X’ → X (COMPLEMENT) X'' X'' MODIFIER (SPEC) X' X' MODIFIER X' X (COMPLEMENT)

49. the girl often plays the violin S N'' V'' Det N' ADV V' the often N V N'' girl plays Det N' the N violin