Why Syntax is Impossible Mike Dowman

Syntax FLanguages have tens of thousands of words FSome combinations of words make valid sentences FOthers don’t FNo one understands the grammar of any language FLanguages have tens of thousands of words FSome combinations of words make valid sentences FOthers don’t FNo one understands the grammar of any language

Syntax is Complicated! I saw Bill with Mary yesterday. You saw WHO with Mary yesterday?! Who did you see with Mary yesterday? I saw Bill with Mary yesterday. You saw WHO with Mary yesterday?! Who did you see with Mary yesterday?

Syntax is Complicated! I saw Bill with Mary yesterday. You saw WHO with Mary yesterday?! Who did you see with Mary yesterday? I saw Bill and Mary yesterday. You saw WHO and Mary yesterday?! I saw Bill with Mary yesterday. You saw WHO with Mary yesterday?! Who did you see with Mary yesterday? I saw Bill and Mary yesterday. You saw WHO and Mary yesterday?!

Syntax is Complicated! I saw Bill with Mary yesterday. You saw WHO with Mary yesterday?! Who did you see with Mary yesterday? I saw Bill and Mary yesterday. You saw WHO and Mary yesterday?! Who did you see and Mary yesterday? I saw Bill with Mary yesterday. You saw WHO with Mary yesterday?! Who did you see with Mary yesterday? I saw Bill and Mary yesterday. You saw WHO and Mary yesterday?! Who did you see and Mary yesterday?

Generative Grammar FAn explicit formal system that defines the set of valid sentences in a language FAnd maybe also explains what each one means FGenerative grammar is the core research topic in linguistics FIncludes strongly nativist theories and theories proposing that languages are primarily learned FAn explicit formal system that defines the set of valid sentences in a language FAnd maybe also explains what each one means FGenerative grammar is the core research topic in linguistics FIncludes strongly nativist theories and theories proposing that languages are primarily learned

Grammar Writing FLinguists take a selection of possible sentences FAnd obtain grammaticality judgments for those sentences FThen they produce a grammar that accounts for all the data FLinguists take a selection of possible sentences FAnd obtain grammaticality judgments for those sentences FThen they produce a grammar that accounts for all the data

Grammar Coverage FLinguists’ grammars only work for selected sentences FThey can’t explain most naturally occurring sentences FThe more data we consider the more surprising quirks of syntax that emerge FLinguists’ grammars only work for selected sentences FThey can’t explain most naturally occurring sentences FThe more data we consider the more surprising quirks of syntax that emerge

Children’s Language Acquisition FKid’s observe a limited number of example sentences FBut quickly internalize a system that correctly characterizes the whole language FKid’s observe a limited number of example sentences FBut quickly internalize a system that correctly characterizes the whole language I-language E-language LAD

How can kids do syntax when linguists can’t? FInnate component of language (provided by genes) FLearned component of language (provided by language data) FInnate component of language (provided by genes) FLearned component of language (provided by language data)

How can kids do syntax when linguists can’t? FInnate component of language (provided by genes) FLearned component of language (provided by language data) FLinguists have to infer both FChildren only the learned component FInnate component of language (provided by genes) FLearned component of language (provided by language data) FLinguists have to infer both FChildren only the learned component

Information Theory FBoth components of language must contain some amount of information FData available to children must provide at least enough information as is in the learned component FThis puts a limit on the complexity of the learned component of language FBoth components of language must contain some amount of information FData available to children must provide at least enough information as is in the learned component FThis puts a limit on the complexity of the learned component of language

Linguists’ Task FLinguists need to have at least as much information as is in the learned and innate components together FCan use data from multiple languages to try to characterize innate components FAnd can use positive and negative data FLinguists need to have at least as much information as is in the learned and innate components together FCan use data from multiple languages to try to characterize innate components FAnd can use positive and negative data

Correspondence to Linguistic Theories Small learned component = parameter setting Large learned component = learned languages Small innate component = general learning mechanism Large innate component = universal grammar Small learned component = parameter setting Large learned component = learned languages Small innate component = general learning mechanism Large innate component = universal grammar

Size of Each Component

Which component is large? FAs we haven’t yet managed to produce a generative grammar, at least one of innate or learned components must be large FChildren learn relatively easily, so the learned component can’t be too big FAs we haven’t yet managed to produce a generative grammar, at least one of innate or learned components must be large FChildren learn relatively easily, so the learned component can’t be too big

Size of Each Component

How big could the innate component be? FGenome contains 3 billion base pairs = 6 billion bits FCell metabolism adds more information FEach base pair can be modified FHuge amount of information! FGenome contains 3 billion base pairs = 6 billion bits FCell metabolism adds more information FEach base pair can be modified FHuge amount of information!

What could be in a huge innate component? FNot words forms - vary from language to language FGrammaticality patterns FRules of syntax would be hugely complex FNot words forms - vary from language to language FGrammaticality patterns FRules of syntax would be hugely complex

Impossibility of Syntax FGrammaticality judgments on average can provide no more than one bit of information each FIf syntax is hugely complex, there will be many grammars that are compatible with any given body of data FBut all but one of these grammars would fail when tested on enough new data FGrammaticality judgments on average can provide no more than one bit of information each FIf syntax is hugely complex, there will be many grammars that are compatible with any given body of data FBut all but one of these grammars would fail when tested on enough new data

A Concrete Example FA multi-agent model FEach agent has: innate component learned component FBoth are bit strings of fixed length FSentences are 100 bit strings FA multi-agent model FEach agent has: innate component learned component FBoth are bit strings of fixed length FSentences are 100 bit strings

Deciding on the Grammaticality of a Sentence 1 FTreat the sentence as a binary number FFind: b i = s mod n i b l = s mod n l b is an index to a bit in the innate ( b i ) or learned ( b l ) component n is the number of bits in the innate ( n i ) or learned ( n l ) component s is the length of the sentences FTreat the sentence as a binary number FFind: b i = s mod n i b l = s mod n l b is an index to a bit in the innate ( b i ) or learned ( b l ) component n is the number of bits in the innate ( n i ) or learned ( n l ) component s is the length of the sentences

Deciding on the Grammaticality of a Sentence 2 FA pseudo-random function maps from the two selected bits plus the sentence to a Boolean grammaticality judgment FIt’s therefore typically necessary to know every bit of the sentence and both the innate and learned bits to predict the grammaticality of the sentence  Every bit counts FUsually about half of sentences are grammatical, half ungrammatical FA pseudo-random function maps from the two selected bits plus the sentence to a Boolean grammaticality judgment FIt’s therefore typically necessary to know every bit of the sentence and both the innate and learned bits to predict the grammaticality of the sentence  Every bit counts FUsually about half of sentences are grammatical, half ungrammatical

4 Kinds of Agent Teacher Innate: Learned: Related Innate: Learned: Unrelated Innate: Learned: Linguist Innate: Learned:

Learning by Related, Unrelated FObserve a sentence from the teacher FWork out if it is grammatical according to current I-language FIf not, invert the relevant bit of the learned component FObserve a sentence from the teacher FWork out if it is grammatical according to current I-language FIf not, invert the relevant bit of the learned component

Grammar Inference by Linguists FChoose random sentences FAsk the teacher if they are grammatical FStore all sentences and grammaticality judgments FSearch for a setting of innate and learned components that assigns the correct grammaticality rating to every sentence FChoose random sentences FAsk the teacher if they are grammatical FStore all sentences and grammaticality judgments FSearch for a setting of innate and learned components that assigns the correct grammaticality rating to every sentence

1,000 Bit Innate and Learned Components

1,000 Bit Innate Component 1,000,000 Bit Learned Component

1,000,000 Bit Innate Component 1,000 Bit Learned Component

Implications of Impossible Syntax FA linguist can write a grammar that will adequately characterize any body of data FBut it will fail when tested on new data FPartial grammars are not a stepping stone to complete generative grammars FA linguist can write a grammar that will adequately characterize any body of data FBut it will fail when tested on new data FPartial grammars are not a stepping stone to complete generative grammars

A Universal Law of Generative Grammar Generative grammar is impossible if: H(learned component) + H(innate component) > H(language data) Unless we can use information from another source (genetic, neuroscientific, psycholinguistic) Generative grammar is impossible if: H(learned component) + H(innate component) > H(language data) Unless we can use information from another source (genetic, neuroscientific, psycholinguistic)

Why do Syntax? FStudying generative grammar may tell us something about the human mind FIt won’t help us build natural language processing systems FIs studying rare and obscure constructions the best way to do syntax? FStudying generative grammar may tell us something about the human mind FIt won’t help us build natural language processing systems FIs studying rare and obscure constructions the best way to do syntax?

Conclusion FThe idea that we can characterize a language by considering enough linguistic data is a hypothesis FIt’s very unlikely that it’s possible to write a complete generative grammar FThe idea that we can characterize a language by considering enough linguistic data is a hypothesis FIt’s very unlikely that it’s possible to write a complete generative grammar