1. The Chomsky Hierarchy

2. Sentences The sentence as a string of words E.g I saw the lady with the binoculars string = a b c d e b f

3. The relations of parts of a string to each other may be different I saw the lady with the binoculars is stucturally ambiguous Who has the binoculars?

4. [ I ] saw the lady [ with the binoculars ] = [a] b c d [e b f] I saw [ the lady with the binoculars] = a b [c d e b f]

5. How can we represent the difference? By assigning them different structures. We can represent structures with 'trees'. I read thebook

6. a. I saw the lady with the binoculars S NP VP VNP NPPP Isaw the lady with the binoculars I saw [the lady with the binoculars]

7. b. I saw the lady with the binoculars S NP VP VPPP I saw the lady with the binoculars I [ saw the lady ] with the binoculars

8. Graphs and trees birds fly S NPVP NV birdsfly S →NPVP NP → N VP → V Syntactic rules

9. S NPVP birdsfly a b ab = string Graphs and trees

10. S A B a b S → A B A → a B → b Graphs and trees

11. Rules Assumption: natural language grammars are a rule-based systems What kind of grammars describe natural language phenomena? What are the formal properties of grammatical rules?

12. Chomsky (1957) Syntactic Struc-tures. The Hague: Mouton Chomsky, N. and G.A. Miller (1958) Finite- state languages Information and Control 1, 99- 112 Chomsky (1959) On certain formal properties of languages. Information and Control 2, 137-167

13. Rules in Linguistics 1. PHONOLOGY /s/ → [θ]  V ___V Rewrite /s/ as [θ] when /s/ occurs in context V ____ V With: V =auxiliary node s, θ=terminal nodes

14. Rules in Linguistics 2. SYNTAX S →NP VP VP→V NP→N Rewrite S as NP VP in any context With: S, NP, VP =auxiliary nodes V, N =terminal node

15. PHONOLOGY (sound system) Maltese – Word-final devoicing Orthography Pronunciation (spelling)(sound) Sabetsab[sa-bet][sap] Ħobżaħobż[hob-za][hops] Vjaġġivjaġġ[vjağ-ği][vjačč] voiced [+vd]voiceless [-vd] [b, z, ğ][p, s, č] [+vd]→[-vd]/____ # (for # = end of word)

16. MORPHOLOGY (word formation) Maltese – Progressive assimilation in 3fsg imprefective (present) Marker for verb in 3rd person feminine singular imperfective t- (3fsgimpf = she) e.g.she breaks =t-kisser I break =n-kisser t-kissert-ressaq 3fsg-break3fsg-move she breaksshe moves s-sakkard-dur 3fsg-lock3fsg-turn she locksshe turns *t-sakkar* t-dur t→s,d,etc./____ [s,d,etc. |[+cor] μ [3fsg] (with μ = morpheme, C = consonant, cor = coronal

17. SYNTAX (phrase/sentence formation) SENTENCE : The boykissed the girl S UBJECTPREDICATE NOUN PHRASEVERB PHRASE ART + NOUNVERB + NOUN PHRASE S→NPVP VP→VNP NP→ARTN

18. SEMANTICS (meaning) The lion attacks the hunter ATTACK (a, b) aλy [ ATTACK (y, b)] λz λy [ ATTACK (y, z)]b (with a = the lion, b = the hunter)

19. Chomsky Hierarchy 0.Type 0 (recursively enumerable) languages Only restriction on rules: left-hand side cannot be the empty string (* Ø  …….) 1. Context-Sensitive languages - Context-Sensitive (CS) rules 2. Context-Free languages - Context-Free (CF) rules 3. Regular languages - Non-Context-Free (CF) rules 0 ⊇ 1 ⊇ 2 ⊇ 3 a ⊇ b meaning a properly includes b (a is a superset of b), i.e. b is a proper subset of a or b is in a

20. Generative power 0.Type 0 (recursively enumerable) languages - only restriction on rules: left-hand side cannot be the empty string (* Ø  …….) - is the most powerful system 3. Type 3(regular language) - is the least powerful

21. a b a c b d f g Superset/subset relation S 1 S 2 S 1 is a subset of S 2 ; S 2 is a subset of S 1

22. Rule Type – 3 Name: Regular Example: Finite State Automata (Markov-process Grammar) Rule type: a) right-linear A  xB or A  x with: A, B = auxiliary nodes and x = terminal node b) or left-linear A  Bx or A  x Generates: a m b n with m,n  1 Cannot guarantee that there are as many a’s as b’s; no embedding

23. A regular grammar for natural language sentences S→theA A→catB A→mouseB A→duckB B→bitesC B→seesC B→eatsC C→theD D→boy D→girl D→monkey the cat bites the boy the mouse eats the monkey the duck sees the girl

24. Regular grammars Grammar 1: Grammar 2:A → a A → a BA → B a B → b AB → A b Grammar 3: Grammar 4:A → a A → a BA → B aB → b B → b AB → A b Grammar 5:Grammar 6: S→a AA → A a S→b BA → B a A→a SB → b B→b b SB → A b S→  A → a

25. Grammars: non-regular Grammar 6:Grammar 7: S→A BA → a S→b BA → B a A→a SB → b B→b b SB → b A S→ 

26. Finite-State Automaton articlenoun NP NP1 NP2 adjective

27. NP article NP1 adjectiveNP1 nounNP2 NP → article NP1 NP1 →adjective NP1 NP1 → noun NP2

28. A parse tree S root node NPVPnon- terminal N VNP nodes DETN terminal nodes

29. Rule Type – 2 Name: Context Free Example: Phrase Structure Grammars/ Push-Down Automata Rule type: A   with: A = auxiliary node  = any number of terminal or auxiliary nodes Recursiveness (centre embedding) allowed: A   A 

30. CF Grammar A Context Free grammar consists of: a)a finite terminal vocabulary V T b)a finite auxiliary vocabulary V A c)an axiom S  V A d)a finite number of context free rules of form A → γ, whereA  V A and γ  {V A  V T }* In natural language syntax S is interpreted as the start symbol for sentence, as in S → NP VP

31. CF Grammars The following languages cannot be generated by a regular grammar Language 1:Language 2: a n b n mirror image ababaaba aabbabbaabba Context-Free rules: A→a A a A→a b A →b A b

32. Natural language Is English regular or CF? If centre embedding is required, then it cannot be regular Centre Embedding: 1.[The cat][likes tuna fish] ab 2.The cat the dog chased likes tuna fish a a b b 3.The cat the dog the rat bit chased likes tuna fish a a ab b b 4.The cat the dog the rat the elephant admired bit chased likes tuna fish a a a a b b b b ab aabb aaabbb aaaabbbb

33. Centre embedding S NPVP thelikes cattuna a b = ab

34. S NPVP likes NPStuna the b cat NP VP a the chased dog b a =aabb

35. S NPVP likes NPStuna the b cat NP VP a chased NP S b the dog NP VP a the bit rat b a =aaabbb

36. Natural language Is English regular or CF? If centre embedding is required, then it cannot be regular

37. Centre Embedding 1.[The cat][likes tuna fish] ab =ab 2.[The cat] [the dog] [chased] [likes tuna fish] a a b b =aabb

38. [The cat][likes tuna fish] a b 2.[The cat] [the dog] [chased] [likes...] a a b b

39. 3.[The cat] [the dog] [the rat] [bit] [chased] [likes...] a a ab b b 4.[The cat] [the dog] [the rat] [the elephant] [admired] [bit] [chased] [likes....] = a a a a b b b b aaabbb aaaabbbb

40. Natural language 2 More Centre Embedding: 1.If S 1, then S 2 a 2.Either S 3, or S 4 b 3.The man who said S 5 is arriving today  4. The man who said S 6 is arriving the day after  Sentence with embedding: If either the man who said S 5 is arriving today or the man who said S 5 is arriving tomorrow, then the man who said S 6 is arriving the day after ab  b  a  = abba

41. Natural language 2 More Centre Embedding: 1.If S 1, then S 2 a 2.Either S 3, or S 4 b Sentence with embedding: If either the man is arriving today or the woman is arriving tomorrow, then the child is arriving the day after. a =[if b =[either the man is arriving today] b =[or the woman is arriving tomorrow]] a =[then the child is arriving the day after] = abba

42. CS languages The following languages cannot be generated by a CF grammar (by pumping lemma): a n b m c n d m Swiss German: A string of dative nouns (e.g. aa), followed by a string of accusative nouns (e.g. bbb), followed by a string of dative-taking verbs (cc), followed by a string of accusative-taking verbs (ddd) = aabbbccddd = a n b m c n d m

43. Swiss German: Jan sait das (Jan says that) … merem Hanses Huushälfed aastriiche weHans/DATthe house/ACChelped paint we helped Hans paint the house abcd NPdat NPdat NPacc NPacc Vdat Vdat Vacc Vacc aab b c cd d