1. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Lecture 1 Wednesday, January 17, 2001 William H. Hsu Department of Computing and Information Sciences, KSU http://www.cis.ksu.edu/~bhsu Reading for Next Class: Chapter 2, Russell and Norvig Handout, Artificial Intelligence (3 e ), P. H. Winston Handout, Essentials of Artificial Intelligence, M. Ginsberg Artificial Intelligence: Foundations Review

2. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Course Outline Overview: Intelligent Systems and Applications Artificial Intelligence (AI) Software Development Topics –Knowledge representation Logical Probabilistic –Search Problem solving by (heuristic) state space search Game tree search –Planning: classical, universal –Machine learning Models (decision trees, version spaces, ANNs, genetic programming) Applications: pattern recognition, planning, data mining and decision support –Topics in applied AI Computer vision fundamentals Natural language processing (NLP) and language learning survey Practicum (Short Software Implementation Project)

3. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Problem Area –What are intelligent systems and agents? –Why are we interested in developing them? Methodologies –What kind of software is involved? What kind of math? –How do we develop it (software, repertoire of techniques)? –Who uses AI? (Who are practitioners in academia, industry, government?) Artificial Intelligence as A Science –What is AI? –What does it have to do with intelligence? Learning? Problem solving? –What are some interesting problems to which intelligent systems can be applied? –Should I be interested in AI (and if so, why)? Today: Brief Tour of AI History –Study of intelligence (classical age to present), AI systems (1940-present) –Viewpoints: philosophy, math, psychology, engineering, linguistics Questions Addressed

4. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence What is AI? [1] Four Categories of Systemic Definitions –1. Think like humans –2. Act like humans –3. Think rationally –4. Act rationally Thinking Like Humans –Machines with minds (Haugeland, 1985) –Automation of “decision making, problem solving, learning…” (Bellman, 1978) Acting Like Humans –Functions that require intelligence when performed by people (Kurzweil, 1990) –Making computers do things people currently do better (Rich and Knight, 1991) Thinking Rationally –Computational models of mental faculties (Charniak and McDermott, 1985) –Computations that make it possible to perceive, reason, and act (Winston, 1992) Acting Rationally –Explaining, emulating intelligent behavior via computation (Schalkoff, 1990) –Branch of CS concerned with automation of intelligent behavior (Luger and Stubblefield, 1993)

5. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence What is AI? [2] Thinking and Acting Like Humans Concerns: Human Performance (Figure 1.1 R&N, Left-Hand Side) –Top: thought processes and reasoning (learning and inference) –Bottom: behavior (interacting with environment) Machines With Minds –Cognitive modelling Early historical examples: problem solvers (see R&N Section 1.1) Application (and one driving force) of cognitive science –Deeper questions What is intelligence? What is consciousness? Acting Humanly: The Turing Test Approach –Capabilities required Natural language processing Knowledge representation Automated reasoning Machine learning –Turing Test: can a machine appear indistinguishable from a human to an experimenter?

6. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence What is AI? [3] Viewpoints on Defining Intelligence Genuine versus Illusory Intelligence –Can we tell? If so, how? If not, what limitations do we postulate? –The argument from disability (“a machine can never do X”) Turing Test Specification –Objective: develop intelligent system “indistiguishable from human” Blind interrogation scenario (no direct physical interaction – “teletype”) 1 AI system, 1 human subject, 1 interrogator Variant: total Turing Test (perceptual interaction: video, tactile interface) –Is this a reasonable test of intelligence? –Details: Section 26.3, R&N –See also: Loebner Prize page Searle’s Chinese Room –Philosophical issue: is (human) intelligence a pure artifact of symbolic manipulation? –Details: Section 26.4, R&N –See also: consciousness in AI resources

7. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence What is AI? [3] Thinking and Acting Rationally Concerns: Human Performance (Figure 1.1 R&N, Right-Hand Side) –Top: thought processes and reasoning (learning and inference) –Bottom: behavior (interacting with environment) Computational Cognitive Modelling –Rational ideal In this course: rational agents Advanced topics: learning, utility theory, decision theory –Basic mathematical, computational models Decisions: automata (Chomsky hierarchy – FSA, PDA, LBA, Turing machine) Search Concept learning Acting Rationally: The Rational Agent Approach –Rational action: acting to achieve one’s goals, given one’s beliefs –Agent: entity that perceives and acts –Focus of next lecture “Laws of thought” approach to AI: correct inferences (reasoning) Rationality not limited to correct inference

8. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Agent: Definition –Any entity that perceives its environment through sensors and acts upon that environment through effectors –Examples (class discussion): human, robotic, software agents Perception –Signal from environment –May exceed sensory capacity Sensors –Acquires percepts –Possible limitations Action –Attempts to affect environment –Usually exceeds effector capacity Effectors –Transmits actions –Possible limitations Agent Intelligent Agents: Overview Percepts Environment Sensors Effectors Actions ?

9. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Application: Knowledge Discovery in Databases

10. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Rule and Decision Tree Learning Example: Rule Acquisition from Historical Data Data –Customer 103 (visit = 1): Age 23, Previous-Purchase: no, Marital-Status: single, Children: none, Annual-Income: 20000, Purchase-Interests: unknown, Store- Credit-Card: no, Homeowner: unknown –Customer 103 (visit = 2): Age 23, Previous-Purchase: no, Marital-Status: married, Children: none, Annual-Income: 20000: Purchase-Interests: car, Store-Credit- Card: yes, Homeowner: no –Customer 103 (visit = n): Age 24, Previous-Purchase: yes, Marital-Status: married, Children: yes, Annual-Income: 75000, Purchase-Interests: television, Store-Credit- Card: yes, Homeowner: no, Computer-Sales-Target: YES Learned Rule –IF customer has made a previous purchase, AND customer has an annual income over $25000, AND customer is interested in buying home electronics THEN probability of computer sale is 0.5 –Training set: 26/41 = 0.634, test set: 12/20 = 0.600 –Typical application: target marketing

11. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Text Mining: Information Retrieval and Filtering 20 USENET Newsgroups –comp.graphicsmisc.forsalesoc.religion.christiansci.space –comp.os.ms-windows.miscrec.autos talk.politics.gunssci.crypt –comp.sys.ibm.pc.hardwarerec.motorcyclestalk.politics.mideastsci.electronics –comp.sys.mac.hardwarerec.sports.baseballtalk.politics.miscsci.med –comp.windows.xrec.sports.hockeytalk.religion.misc – alt.atheism Problem Definition [Joachims, 1996] –Given: 1000 training documents (posts) from each group –Return: classifier for new documents that identifies the group it belongs to Example: Recent Article from comp.graphics.algorithms Hi all I'm writing an adaptive marching cube algorithm, which must deal with cracks. I got the vertices of the cracks in a list (one list per crack). Does there exist an algorithm to triangulate a concave polygon ? Or how can I bisect the polygon so, that I get a set of connected convex polygons. The cases of occuring polygons are these:... Performance of Newsweeder (Naïve Bayes): 89% Accuracy

12. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Specifying A Learning Problem Learning = Improving with Experience at Some Task –Improve over task T, –with respect to performance measure P, –based on experience E. Example: Learning to Filter Spam Articles –T: analyze USENET newsgroup posts –P: function of classification accuracy (discounted error function) –E: training corpus of labeled news files (e.g., annotated from Deja.com) Refining the Problem Specification: Issues –What experience? –What exactly should be learned? –How shall it be represented? –What specific algorithm to learn it? Defining the Problem Milieu –Performance element: How shall the results of learning be applied? –How shall the performance element be evaluated? The learning system?

13. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Survey of Machine Learning Methodologies Supervised (Focus of CIS730) –What is learned? Classification function; other models –Inputs and outputs? Learning: –How is it learned? Presentation of examples to learner (by teacher) –Projects: MLC++ and NCSA D2K; wrapper, clickstream mining applications Unsupervised (Surveyed in CIS730) –Cluster definition, or vector quantization function (codebook) –Learning: –Formation, segmentation, labeling of clusters based on observations, metric –Projects: NCSA D2K; info retrieval (IR), Bayesian network learning applications Reinforcement (Not Emphasized in CIS730) –Control policy (function from states of the world to actions) –Learning: –(Delayed) feedback of reward values to agent based on actions selected; model updated based on reward, (partially) observable state

14. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Related Online Resources Research –KSU Laboratory for Knowledge Discovery in Databases http://www.kddresearch.org (see especially Group Info, Web Resources) http://www.kddresearch.org –KD Nuggets: http://www.kdnuggets.comhttp://www.kdnuggets.com Courses and Tutorials Online –At KSU CIS798 Machine Learning and Pattern Recognition http://www.kddresearch.org/Courses/Fall-1999/CIS798 http://www.kddresearch.org/Courses/Fall-1999/CIS798 CIS730 Introduction to Artificial Intelligence http:// www.kddresearch.org/Courses/Fall-2000/CIS730 http:// www.kddresearch.org/Courses/Fall-2000/CIS730 CIS690 Implementation of High-Performance Data Mining Systems http:// www.kddresearch.org/Courses/Summer-2000/CIS690 http:// www.kddresearch.org/Courses/Summer-2000/CIS690 –Other courses: see KD Nuggets, www.aaai.org, www.auai.orgwww.aaai.orgwww.auai.org Discussion Forums –Newsgroups: comp.ai.* –Recommended mailing lists: Data Mining, Uncertainty in AI –KSU KDD Lab Discussion Board: http://www.kddresearch.org/Boardhttp://www.kddresearch.org/Board

15. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Terminology Artificial Intelligence (AI) –Operational definition: study / development of systems capable of “thought processes” (reasoning, learning, problem solving) –Constructive definition: expressed in artifacts (design and implementation) Intelligent Agents Topics and Methodologies –Knowledge representation Logical Uncertain (probabilistic) Other (rule-based, fuzzy, neural, genetic) –Search –Machine learning –Planning Applications –Problem solving, optimization, scheduling, design –Decision support, data mining –Natural language processing, conversational and information retrieval agents –Pattern recognition and robot vision

16. Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Summary Points Artificial Intelligence: Conceptual Definitions and Dichotomies –Human cognitive modelling vs. rational inference –Cognition (thought processes) versus behavior (performance) –Some viewpoints on defining intelligence Roles of Knowledge Representation, Search, Learning, Inference in AI –Necessity of KR, problem solving capabilities in intelligent agents –Ability to reason, learn Applications and Automation Case Studies –Search: game-playing systems, problem solvers –Planning, design, scheduling systems –Control and optimization systems –Machine learning: pattern recognition, data mining (business decision support) More Resources Online –Home page for AIMA (R&N) textbook –CMU AI repository –KSU KDD Lab (Hsu): http:// www.kddresearch.orghttp:// www.kddresearch.org –Comp.ai newsgroup (now moderated)