CCSB354 ARTIFICIAL INTELLIGENCE (AI) CHAPTER 12 NATURAL LANGUAGE PROCESSING (NLP) Textbook (Chapter 13, & especially pages 558 & 588) Instructor: Alicia Tang Y. C.

Language is a complicated phenomenon, involving processes as varied as the recognition of sounds or printed letters, syntactic parsing, high-level semantic inferences,etc.

What is Natural Language Processing (NLP)? Natural language gives computer users the ability to communication with the computer in their native language. This technology allows for conversational type of interface. A general NLP system is not yet possible, especially in recognising and interpreting written sentences.

Types Natural Language Understanding: Investigate methods of allowing the computer to comprehend instructions given in ordinary English. Natural Language Generation: Strives to have computers produce ordinary English language so that people can understand computers easily.

NLP Software Tools  INTELLECT  SPOCK  BBN Parlane  NaturalLink

 Parser  Lexicon  Understander  Knowledge base  Generator Component of NLP  Parser  Lexicon  Understander  Knowledge base  Generator

Stages in Producing an Internal Representation of a Sentence

love kiss ?? lips jungle object agent object Contextual Knowledge Interpretation Expanded representation: possess pet:cheetah experiencer object love Person: tarzan Person:jane agent kiss object instrument lips location jungle location ??

Block Diagram of a Natural Language Understanding Program of the Syntactic-Semantic Analysis Type Input text String Parser Understander Generator Output Lexicon Knowledge base

Block Diagram of a Computer Language Translation System Generalised Intermediate Form (GIF) Rule-based Processor Input Program Parser Rule-based Processor Lexicon Output Program in Target Language

NLP Database Expert Systems: where do they stand?

EASE OF COMMUNICATION WITH DIFFERENT TYPES OF INTEGRATED SYSTEMS Oh no! Well..not bad! Oh yeah!! EASE OF COMMUNICATION WITH DIFFERENT TYPES OF INTEGRATED SYSTEMS

Levels of analysis for natural language Prosody it deals with the rhythm and intonation of language Phonology it examines the sounds that are combined to form language. Morphology it concerns with the components that make up words. E.g. the effect of prefixes (non-, un-) and suffixes (-ing, -ly)

Levels of analysis for natural language Syntax this involves the study of the rule for combining words to form legal sentences Semantics it considers the meaning of words, phrases and sentences Pragmatics this is the study of the ways in which language is used and its effects on the listener World Knowledge this includes knowledge of the ‘physical’ world, the world of our social interaction

Syntax Specification and parsing using Context-free Grammars 1. Sentence  noun-phrase verb-phrase 2. Noun-phrase  noun 3. Noun-phrase  article noun 4. Verb-phrase  verb 5. Verb-phrase  verb noun-phrase 6. Article  a 7. Article  the 8. Noun  man 9. Noun  dog 10. Verb  likes 11. Verb  bites Specification and parsing using Context-free Grammars the rules listed below define a GRAMMAR for simple transitive sentences

Syntax Rules 6. to 11. are terminals 1. Sentence  noun-phrase verb-phrase 2. Noun-phrase  noun 3. Noun-phrase  article noun 4. Verb-phrase  verb 5. Verb-phrase  verb noun-phrase 6. Article  a 7. Article  the 8. Noun  man 9. Noun  dog 10. Verb  likes 11. Verb  bites Rules 6. to 11. are terminals they define a lexicon for the language terms that describe high level linguistic are called nonterminals (sentence, verb, noun-phrase, etc)

Syntax A derivation of the sentence “the man bites the dog” (syntactically correct but semantically wrong, we shall see it later on) String Apply Rule No. sentence 1 noun-phrase verb-phrase 3 article noun verb-phrase 7 the noun verb-phrase 8 the man verb-phrase 5 the man verb noun-phrase 11 the man bites noun-phrase 3 the man bites article noun 7 the man bites the noun 9 the man bites the dog done Parsing algorithms fall into two classes: 1. Bottom-up 2. Top-down

Draw a parse tree for the sentence “the man bites the dog”

Top-down Parsing Parse Tree S Unrecognised I/P the man bites the dog VP A N the man V bites the dog

Bottom-Up Parsing Parse Tree the Unrecognised I/P the man bites the dog man bites the dog bites the dog the dog dog A man N NP bites V the A dog N NP VP S

Transition Network Parsers A transition network parser represents grammars as a set of finite-state machines (i.e. transition networks), like this: Noun-phrase Verb-phrase S S initial final Example: sentence (S)

Transition Network Parsers Example: Noun-phrase Noun Article S S initial final Noun Example: Article Read page 563, text a S S initial final the

The Chomsky Hierarchy and Context-Sensitive Grammars A context free grammar allows rule to have a single nonterminal on their left-hand side. Context-free grammar is not powerful enough to represent rules of natural language syntax the context-sensitive languages form a proper superset of the context-free counterpart

The Chomsky Hierarchy and Context-Sensitive Grammars Here, one or more symbols on the left-hand side of a rule are allowed that makes it possible to define a context in which that rule can be applied. This ensures satisfaction of a global constraints such as number agreement and other semantic checks. The semantic error in earlier example could be detected in context-sensitive grammar if a non- terminal, act_of_biting is added to the grammar, preventing the sentence “man bites dog” to be valid.

The Chomsky Hierarchy To correctly express grammatical structure of a language (e.g. English), rules are needed. We can classify grammars according to the kinds of rules that appear in it. Having done that, we can classify the language into families according to the kinds of rules that are needed to express its grammars. One such means of classifying grammars in this manner is called Chomsky Hierarchy

Grammar Hierarchy The Grammars for the language Type 0 Type 1 Type 2

Grammar Hierarchy Type 0 Name of Grammar: Transformation Grammar Form of Rules: anything  anything Computational Power: General Turing Machine String Characteristics: Any form

Grammar Hierarchy Type 1 Crossing dependencies E.g. Ali1 helps4 Ahmad2 Name of Grammar: Context-sensitive Form of Rules: A B C  A D C Computational Power: Linear Bound Automata String Characteristics: an bn cn Crossing dependencies E.g. Ali1 helps4 Ahmad2 to teach5 Aida3 programming6. a1 a2 a3 b3 b2 b3

E.g. Ali1 who studies in UNITEN2 that offers2 quality programmes Type 2 Name of Grammar: Context-free Form of Rules: A  B C D …. Computational Power: Push Down Stack Automata String Characteristics: an bn 2 2 2 1 1 1 1 1 1 Nested dependencies E.g. Ali1 who studies in UNITEN2 that offers2 quality programmes is1 graduating a1 a2 a3 b3 b2 b1

They have been used for grammars of Morphology Type 3 Name of Grammar: Right Linear Form of Rules: A  x B (x in terminal category) Computational Power: Finite State Automata String Characteristics: a* b* They have been used for grammars of Morphology

NATURAL LANGUAGE PROCESSING Problems with NLP : Ambiguity multiple word meanings the pitcher is angry the pitcher is empty Inaccuracy Incompleteness

How people overcome natural language problems? Imprecision I’ve been waiting for you for a long time. The king ruled the kingdom for a long time. Unclear antecedents Ben hits Bill because the sympathized with mary. How people overcome natural language problems? Context Familiarity Expectations

SPEECH RECOGNITION Advantages: Context Analysing Speech most natural method ease of access speed manual freedom remote access Context Isolated Word Recognition (IWR) Connected Word Recognition (CWR) Continuous Speech Recognition (CSR) Analysing Speech Syllable Phonemes Allophones

Speech Component: A Holistic Look I/O Processing Other data Speech recognition Match with other sound frequencies Freq. spectrum She likes ice-cream Word Sentence Syntactic Analysis Grammar of language (dictionary) Semantic Analysis Meaning of each word (thesaurus) She like ice-cream Sentence Structure x. Likes(x,ice-cream) Partial Meaning Pragmatics Context of utterance likes(siti, ice-cream). Full Sentence Meaning

Exercises:

Question: Consider the grammar defined by the following BNF: S  NP VP NP  <name> | <det> <noun> |PP VP  <verb> | <verb> NP NP | <verb> NP PP | VP NP PP  <prep> NP <name>  “Ben" | “Ann" <noun>  "morning" | “ice-cream" <verb>  "gave" | "saw" <det>  "the" <prep>  "to" | "in" S – SENTENCE NP – NOUN PHRASE VP – VERB PHRASE DET – DETERMINER PP - PREPOSITION

Answers: a) Ben gave Ann the ice-cream. Draw parse trees in this grammar for the following sentences. a) Ben gave Ann the ice-cream. b)   Ann gave the ice-cream to Ben. S  NP VP NP  <name> | <det> <noun> |PP VP  <verb> | <verb> NP NP | <verb> NP PP | VP NP PP  <prep> NP <name>  “Ben" | “Ann" <noun>  "morning" | “ice-cream" <verb>  "gave" | "saw" <det>  "the" <prep>  "to" | "in" Answers: