Natural Language Processing Lecture 5 : Syntax :The analysis of sentence structure Prepared by Alaa Al Mohammadi

Definition of Syntax Syntax is the study of the rules governing the way words are combined to form sentences in a language. *garden the *Children are *Work in This class: what syntactic structure is and what the rules that determine syntactic structure are like.

Syntax Properties of syntactic knowledge: Humans can understand & produce an infinite number of sentences they never heard before “Some purple gnats are starting to tango on microwave” Our grammar can understand and produce long sentences “Bill said that he thought that the esteemed leader of the house had it in mind to tell the unfortunate vice president that the calls that he made from the office in the White House that he thought was private…..” Determine the grammatical relations in a sentence Mary hired Bill. Vs. Bill hired Mary

Definition of Grammar A formal grammar consists of: a finite set of production rules (left-hand side  right-hand side) where each side consists of a sequence of the following symbols: a finite set of non terminal symbols (indicating that some production rule can yet be applied) a finite set of terminal symbols  (indicating that no production rule can be applied) a start symbol (a distinguished non terminal symbol) A formal grammar defines (or generates) a formal language, which is a (usually infinite) set of finite-length sequences of symbols (i.e. strings) that may be constructed by applying production rules Prepared by Alaa Al Mohammadi

Definition of Grammar A rule may be applied to a sequence of symbols by replacing an occurrence of the symbols on the left-hand side of the rule with those that appear on the right-hand side. A sequence of rule applications is called a derivation. Such a grammar defines the formal language: all words consisting solely of terminal symbols which can be reached by a derivation from the start symbol. Non terminals are often represented by uppercase letters, terminals by lowercase letters, and the start symbol by S . Prepared by Alaa Al Mohammadi

Definition of Grammar  For example, the grammar with terminals {a,b} , non terminals  {S,A,B}, production rules   and start symbol S , defines the language of all words of the form   (i.e.  n copies of a  followed by  copies of b ) Prepared by Alaa Al Mohammadi

Definition of Grammar Prepared by Alaa Al Mohammadi

Generative Grammar Noam Chomsky 1950s Generative = a very explicit system of rules specifying what combinations of basic elements result in well-formed sentences. Defines the syntactic structure of a language.

Generative Grammar “all and only”= all grammatical sentences and only grammatical sentences Finite rules infinite number of well-formed sentences Productivity of language Phrase structure rules Transformational rules

Grammar Hierarchy Type-3 grammars (regular grammars) generate the regular languages. Such a grammar restricts its rules to a single non terminal on the left-hand side and a right-hand side consisting of a single terminal, possibly followed by a single non terminal (right regular) Type-2 grammars (context-free grammars) generate the context-free languages. These are defined by rules of the form   with  A a non terminal and   a string of terminals and non terminals. Type-1 grammars (context-sensitive grammars) generate the context-sensitive languages. These grammars have rules of the form  with  A a non terminal and      strings of terminals and non terminals. The strings   and   may be empty, but   must be nonempty. Prepared by Alaa Al Mohammadi

Grammar Hierarchy Type-0 grammars (unrestricted grammars) include all formal grammars. Prepared by Alaa Al Mohammadi

Grammar Hierarchy Prepared by Alaa Al Mohammadi

Phrase structure rules Some words seem to belong together: {The crazy man} {is jumping off the bridge} Groups of words that belong together are called constituents The component that determines the properties of the constituent is the head, and the constituent can be referred to as a phrase: e.g. noun phrase

Phrase Structure Rules If we look at phrases, some patterns emerge: Det N the instructor = NP Det N a friend = NP some homework = NP two classes = NP

Phrase Structure Rules some more patterns: V Det N call the instructor= VP V Det N meet a friend = VP V Det N do some homework = VP V Det N skip two classes = VP

PSR and yet more patterns: Prep Det N with the instructor= PP from a friend = PP with some homework = PP Prep Det N after two classes = PP

PSR Rules for determining the structure of phrases Generate a lot of sentences from a small number of rules. The structure of a phrase will consist of one or more constituents in a certain order. What does a NP consist of? “noun phrases have a Det and a N” NP Det N

Lexical Rules We need lexical rules to specify which words can be used when we rewrite constituents such as N. PN {Mary, George} N {girl, boy, dog} Art Pro

PSR V Det N V Det N V Det N run a marathon eat the food read the book V Prep Det N V Prep Det N go to the store talk with a teacher V Det N Prep Det N take your sister to the library “Verb phrases have a V, (sometimes) an NP, and (sometimes) a PP” VP -> V (NP) (PP)

The main phrase structure rules 1. S  NP VP 2. NP  {Det N, Pro, PN} 3. VP  V (NP) (PP) (Adv) 4. PP  P NP 5. AP  A (PP)

Phrase Structure Rules & tree diagrams NP  (Det) N PP  P NP the boy in the yard NP The boy (NP) Det N Det N PP The boy P NP Det N The boy in the yard

Phrase Structure Rules VP  V (NP) (PP) S  NP VP took the money from the bank VP took the money (VP) V NP V NP PP Det N Det N P NP took the money Det N took the money from the bank

Example (1) The old tree swayed in the wind S NP VP Det Adj N V PP P

Example (2) The children put the toy in the box S NP VP Det N V NP PP

Example 3

Back to Generative Grammar How superficially different sentences are closely related? How superficially similar sentences are different?

Deep and surface structure The deep structure is an abstract level of structural organization in which all the elements determining structural interpretation are represented. Sentences that have alternative interpretations Sentences that have different surface forms but have the same underlying meaning. Surface structure= how the sentence is actually represented

Deep and surface structure How superficially different sentences are closely related? Charlie broke the window. The window was broken by Charlie Charlie who broke the window. Was the window broken by Charlie? Difference in their surface structure = difference in syntactic forms BUT they have the same ‘deep’ or underlying structure

Structural ambiguity How superficially similar sentences are different? (multiple meanings) E.g. Annie whacked the man with an umbrella Same surface structure but different deep structure The boy saw the man with a telescope The question is: What is the scope of "with the telescope"? Does it modify only "the man" or does it modify "saw the man"?

Structural Ambiguity (1) The boy saw the man with the telescope NP VP Det N V NP PP Det N P NP Det N The boy saw the man with the telescope Meaning: Using the telescope, the boy saw the man

Structural Ambiguity (2) The boy saw the man with the telescope NP VP Det NP N V Det N PP P NP Det N The boy saw the man with the telescope Meaning: The boy saw the man. The man had a telescope.

Recursion Rules can be applied more than once in generating sentences E.g. repeat prepositional phrase more than once The gun was on the table near the window in the bedroom in the pink house Put sentences inside sentences This is the cat that ate the rat that ate the cheese that was sold by the man that lived in the city that was on the river… No end to recursion- produce longer complex sentences

Back to recursion [Mary helped George]. (A sentence) [Cathy knew] that [Mary helped George]. (a sentence within a sentence) [John believed] that [Cathy knew] that [Mary helped George]. The word that introduces the complement phrase

Complement Phrases Cathy knew that Mary helped George That = complementizer (C) introducing complement phrase (CP) The CP comes after the VP S NP VP VP V CP CP C S

Complement Phrases

Transformational Rules Phrase structure rules represent ‘deep’ structure- always generate structures with fixed word order. Mary saw George recently Recently Mary saw George Transformational rules= take a specific part and attach it in another place You will help Cathy Will you help Cathy?

Transformational Rules

Exercises Rewrite the following sentences with Phrase Structure Rules.  Hint: Locate your principal NP and VP before beginning. a)  Miriam swims. b)  The dog is barking. c)  Peter told the truth. d)  The wicked witch spilled the potion. e)  The runner with the best time won the prize.

Exercises Draw a labeled tree diagram for the following English phrases. (Hint: what part of speech is the leader for the phrase?) a.  ancient pyramids b.  in the early evening c.  Drove a car

Exercises Draw phrase structure trees for the following sentences: The puppy found the child The ice melted The hot sun melted the ice. The house on the hill collapsed in the wind. The boat sailed up the river. A girl laughed at the monkey.

Exercises Draw two phrase structure trees representing the two meanings of the sentence: The magician touched the child with the wand.