1. By Rohana Mahmud (NLP week 1-2)
Communication & NL By
Rohana Mahmud
(NLP week 1-2)

2. Study of Language Language:
Written: Long-term record of knowledge from one generation to another
Spoken: primary mean of coordinating day-to-day behavior with others
Natural (eg. Malay, English) vs. Artificial (Java, Prolog, Coding)
Communication
Use sign / natural language/ body language
Sender and Receiver
Studied in several disciplines:
Linguist: structure of language
PsychoLinguist: the process of human language production and comprehension
Philospher: how words can mean anything & how they identify object in the world, what it means to have belief, goals and intention, cognitive capabilities relate to language
CL: to develop a computational theory of Language (using the notions of algorithm & data structure from CS)

3. History Speech & Language Processing: overlapping fields:
Computational Linguistics (Linguistics)
Natural Language Processing (CS)
Speech Recognition (EE)
Computational Psycholinguistics (Psychology)

4. History 1940s & 1950s – Foundational Insights:
the automaton and probabilistic/ information-theoretic models (Turing1936, Shannon 1948)
Formal language theory (chomsky 1956)
Probabilistic algorithm
: 2 Camps- Symbolic and Stochastic
: 4 paradigms – Stochastic, Logic-based, NLU (SHRDLU Winograd, LUNAR Woods), Discourse Modelling (grosz) & Believe-Desire-Intention
Current: The Field comes together – probabilistic & data model, increase in speed and memory of computers & applied to Augmentative and Alternative Communication(AAC) and the rise of Web, need for language-based IR & IE.

5. Goal Scientific goal: Cognitive science – interdisciplinary research
Technological/practical goal - NLP revolutionize the computers are used Computer that understand NL could access all information (human knowledge)
NL interfaces to computers: allow complex system to be accessible to everyone. More flexible and intelligent

6. Speech and Language Processing
Understand human: speech recognition & NLU (lip-reading)
Communicating with human: NLG & Speech Synthesis
Information retrieval, information extraction, inference (draw conclusions based on facts)
Spelling corrections, grammar checking, machine translation
Data processing vs. language processing (knowledge of language – what it means to be a word)

7. Application of NLU
It represents the meaning of sentences in some representation language that can be used later for further processing
Text-based applications
Written text processing (books,newpaper, reports, manual, , sms) = reading-based tasks
Searching/finding from database of text
Extracting information from text
Translating documents
Summarizing texts for certain purpose
Story understanding

8. Application of NLU
Dialogue-based applications – involve human-machine communication (spoken / keyboard/mouse/ recognizer)
Q&A systems, eg. Query database
Automated customer service (phone)
Tutoring systems (interaction with students)
Spoken language control of machine
General cooperative problem-solving system
Speech recognition <> Language understanding system (only identify the word spoken from a given speech signal, not – how words are used to communicate)
Discuss ELIZA system

9. ELIZA system
Mid-1960s, MIT, a Therapist (system) & patient (user), Weizenbaum, 1966
Algorithm:
Has a Dbase of particular words (keywords)
For each keyword -> store an integer, a pattern to match against the input and a specification of the output
Given Sentence(S), find a keyword in S whose pattern matches S
If > 1 keyword, pick the one with highest integer value
Use the output specification that is associated with this keyword to generate next sentence
If there are No keywords, generate an innocuous continuation statement, eg: Tell me more, Go on. (figure 1.2, 1.3 Allen)

10. Representations and Understanding
Computing a representation of the meaning of sentences and texts (Notion of representation)
Why can’t use the sentence itself as a representation of its meaning? Most words have multiple meanings (Senses). eg. Cook, bank, still (verb or noun), I made her duck. I saw a man in the park with a telescope
Thus, ambiguity inhibit system from making the appropriate inferences needed to model understanding (need to resolve or disambiguate: eg. Use Lexical disambiguation: POS, word-sense disambiguation)
A program must explicitly consider each senses of a word to understand a sentence

11. Represent meaning: must have a more precise language
Mathematics & Logic and the use of formally specified representation languages (formal language) – notion of an atomic symbol
Useful representation languages have 2 properties:
Precise and unambiguous
Capture the intuitive structure of the natural language sentences that it represents

12. Models and Algorithm
Toolkit: state machines, formal rule systems, logic, probability theory, machine learning
States, transitions among states and inpur representation
Basic procedural models: deterministic, Non-Deterministic finite-state automata, finite-state transducer -> weighted automata, markov model, HMM
Formal rules: Regular grammar, regular relations, context free grammar (phonology, morphology, syntax)
Involve a search through a state of spaces representing by hypotheses about an input : depth first, best-first and A* search
Logic – first order logic and predicate calculus, semantic network and conceptual dependency – logical representation (semantic, pragmatic and discourse)

13. Bibliography ACL (Association for CL) / EACL
COLING (int conference of CL)
Applied NLP
Workshop on Human Language Technology
Journal: CL & NLE
IEEE ICASSP: Acoustic, Speech and Signal Processing
IEEE Transactions on Pattern Analysis and Machine Intelligence
IJCAI: Int Joint Conference on AI
Journal: AI, Computational Intelligence, Cognitive Science

14. TUTORIAL – WEEK 2 Submit your first week Tutorial
Presents your findings
Definitions (NLP/NLG/NLU) (CL) (LE)
History?
Key works / research done in the area
References

15. Tutorial 2 - task Practical
Hands-on with ELIZA system or similar system
Software?
Algorithm?
Domain?

16. PROJECT / COURSEWORK System Development Human-Computer Interface
Database Retrieval
Expert System Interface
Word Analyzer (morpheme, suffix)
CALL
Dictionary & Senses
Meaning Postulate
Back-end Engine
Discourse Segmentation (Essay – Paragraph – sentence – word)
POS Recognizer (Malay / English)
Information Retrieval (web/ book)
Frequency-checker
Parser

17. Assessment 25 % out of 50% (5% - tutorial, 20% mid-exam/test)
Topic = Open
System = 15 % (Demonstration)
Report = 10% (SD, TE)
Using HLL – Java, Prolog, C & MM softwares
Showing the NL processor tasks and the interface (major/minor)
Submit:
Week 3/4 – topic & proposal
Week 9 – demonstration & report