1. Dr. Radhika Mamidi ENG 270 Lecture 2

2. History: 1940-1950’s Major influences on the development of CL -Development of formal language theory (Chomsky, Kleene, Backus) – Formal characterization of classes of grammar (context-free grammar, regular grammar) – Association with relevant automata (finite state automaton) Probability theory: language understanding as decoding through noisy channel (Shannon)

3. 1957-1983 Rule based vs. Statistical based Rule based theory – Use of formal grammars as basis for natural language processing and learning systems. (Chomsky, Harris) Statistical based theory – Probabilistic methods for early speech recognition, OCR 1983-1993: Return of Empiricism [Statistical techniques] Use of statistical techniques for part of speech tagging, parsing, word sense disambiguation, etc.

4. 1993-Present Researchers are interested in both techniques. Emphasis is on machine learning. Advances in software and hardware create NLP needs for search engines, machine translation, spelling and grammar checking, speech recognition and synthesis.

5. CL vs NLP CL and NLP are related with the focus being different. Computational Linguistics aims to model language as people do. Natural Language Processing is processing language from a computational point of view in order to build different applications and tools.

6. Artificial Intelligence the branch of computer science which aims to create the intelligence of machines "the study and design of intelligent agents” "the science and engineering of making intelligent machines”

7. CL and AI One way of finding out machine’s intelligence is by finding out if it understands language. AI and CL are related here. ‘Chatbot’ is a reflection of this: a computer program to simulate an intelligent conversation with one or more human users via auditory or textual methods. a program with artificial intelligence to talk to people through voices or typed words.

8. Turing test: A test to judge the intelligence of a machine. It involves three entities: machine, human, and human judge Judge asks questions of computer and human. -- Machine’s job is to act like a human -- Human’s job is to convince judge that he’s not the machine. Machine is judged “intelligent” if it can fool judge. Judgment of “intelligence” is linked to appropriate answers to questions from the system. Language and Intelligence: Turing test

9. ELIZA, the first chatbot A simple “Rogerian Psychologist” Uses pattern matching to carry on limited form of conversation. It gives a feeling that it is “human” Seems to pass the “Turing Test” It is one of the first chatbots. The answers it gave showed its ‘intelligence’.

10. What’s involved in an “intelligent” Answer? Analysis of answers: Decomposition of the signal (spoken or written) eventually into meaningful units. This involves … Discourse  Sentences  Words  Sounds

11. Levels of Language Processing Phonology Morphology Syntax Semantics Pragmatics Discourse Analysis

12. Memory General Knowledge Lexicon Syntactic Rules Semantic Rules Discourse Rules Lexical Processing INPUTS Syntactic Processing Semantic Processing Discourse Processing OUTPUTS Model of Language Processing To derive meaning you need all kinds of rules – ‘building’, ‘blocks’ Eg: The building blocks are made of plastic. The building blocks the sun.

13. Why is understanding language by a machine so difficult? Human language is: Complex and Ambiguous We use language creatively We don’t mean what we say! Language Understanding needs contextual and general knowledge apart from linguistic knowledge. To know what we mean shared knowledge is necessary. Representing all this knowledge computationally is THE challenge.

14. Ambiguity at different language levels: Pronounce “GHOTI” [one spelling – different sounds] I scream/ice-cream, a nameless man/an aimless man He showed me the mouse - rodent/object The leopard was spotted - verb/adjective She hit the boy with the umbrella I am reading a book on films - [now-a-days/right now] Mary promised Sally(i) to go to her(i) party Mary(i) persuaded Sally to go to her(i) party Human language is ambiguous

15. Human language is complex teach – taught *preach - praught he-his-him *she-shis-shim ring – rang - rung *bring – brang - brung slim chance = fat chance ?slim girl = fat girl No consistency. No regularity.

16. Let’s analyze this spoken sentence: I made her duck. How many meanings does the sentence have?

17. 1. Speech technologies Applications: 1. Speech synthesis tools - Text to Speech conversion 2. Speech recognition tools – Speech to Text conversion Requires knowledge of phonological patterns Text  Speech conversion

18. Uses Text to speech Use: Public announcements – airport, railway stations, blind people, proofreading, when eyes are busy [drivers, writers etc.], speaking clocks etc. Speech Recognition Use: Pronunciation dictionaries, voice commands in pc, voice dialing (e.g., "Call home"), call routing (e.g., "I would like to make a collect call“), simple data entry (e.g., entering a credit card number) etc.

19. Some problems Grapheme to Phoneme and vice-versa conversion Different spellings – same pronunciation Example: reed-read, bear-bare, ear-year, I-eye, peace-piece Same spellings – different pronunciation Example: read, bow, dove, does, minute, number Numbers, Names, Acronyms – pronounced differently 1980 --- uttered differently as year, quantity, currency St. --- street, saint PSU – public sector unit, prince sultan university Give at least five ways of uttering your phone number.

20. 2. Morphological Analysis Inflectional morphology :word variation reflects features like tense, number, degree, gender :grammatical category remains same eg. eat-eats, boy-boys, thin-thinner Derivational morphology :word variation changes grammatical category eg. act-actor, boy-boyish :word variation maintains grammatical category eg. fair-unfair, like-dislike Inflection follows Derivation: act--actor—actors Tools built with morphological knowledge: Morphological analyzer [identifies roots and affixes] and Morphological generator [generates words from roots and affixes]

21. 3. Syntactic Parsing Process of identifying syntactic structure of a valid sentence Represented by trees, rules and networks Syntax Components Phrase Structure Rules Transformational Rules Tools built with syntactic knowledge: - Syntactic Parsers [analyses a sentence automatically] e.g. Augmented Transition Networks

22. Syntax Component Chomsky’s (1965) model of language Phrase Structure rules generate deep structures Deep Structure holds all the syntactic information needed to derive the meaning of a sentence This is fed into the semantic component to obtain acceptable combinations Transformational rules map deep structures to surface structure Surface Structure has words in the right order This is obtained after feeding surface structure into the phonological component

23. Chomsky’s model SYNTAX COMPONENT Surface structures Transformational rules Phrase Structure Rules Deep structures PHONOLOGICAL COMPONENT Phonological rules Selection restriction rules Lexicon SEMANTIC COMPONENT

24. Representation Eg: Riyadh is a beautiful city. 1. Rules: S  NP VP NP  (Art) (Adj) N VP  V NP Lexicon: Art – a Adj – beautiful N – Riyadh, city V – is

25. s1s2s3 NPVP S: s1 s2 s3 articlenoun Empty Adj loop NP: s1 s2 s3 verb NP VP: 2. Representation by Networks

26. S NPVP N V NP Riyadh is art beautiful Adj a Noun place 3. Representation by Trees

27. Example of automatic syntactic analysis by online ‘Link parser’. Sentence given: Riyadh is a beautiful place. Output: (S (NP Riyadh) (VP is (NP a beautiful place)).) http://www.link.cs.cmu.edu/link/submit-sentence-4.html