Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Lecture 0 Friday, January.

Slides:



Advertisements
Similar presentations
Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Advertisements

1er. Escuela Red ProTIC - Tandil, de Abril, 2006 Introduction to Machine Learning Alejandro Ceccatto Instituto de Física Rosario CONICET-UNR.
Kansas State University Department of Computing and Information Sciences CIS 798: Intelligent Systems and Machine Learning Tuesday, August 24, 1999 William.
CS/CMPE 535 – Machine Learning Outline. CS Machine Learning (Wi ) - Asim LUMS2 Description A course on the fundamentals of machine.
Machine Learning (Extended) Dr. Ata Kaban
CSCI 3 Introduction to Computer Science. CSCI 3 Course Description: –An overview of the fundamentals of computer science. Topics covered include number.
Machine Learning Group University College Dublin 4.30 Machine Learning Pádraig Cunningham.
A Brief Survey of Machine Learning Used materials from: William H. Hsu Linda Jackson Lex Lane Tom Mitchell Machine Learning, Mc Graw Hill 1997 Allan Moser.
A Brief Survey of Machine Learning
CSE 590ST Statistical Methods in Computer Science Instructor: Pedro Domingos.
CS157A Spring 05 Data Mining Professor Sin-Min Lee.
CS 5941 CS583 – Data Mining and Text Mining Course Web Page 05/cs583.html.
Presented To: Madam Nadia Gul Presented By: Bi Bi Mariam.
CSE 515 Statistical Methods in Computer Science Instructor: Pedro Domingos.
Introduction to machine learning
Data Mining By Andrie Suherman. Agenda Introduction Major Elements Steps/ Processes Tools used for data mining Advantages and Disadvantages.
LLNL-PRES This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
General Information Course Id: COSC6342 Machine Learning Time: MO/WE 2:30-4p Instructor: Christoph F. Eick Classroom:SEC 201
Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence From Data Mining To Knowledge.
Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Lecture 1 Wednesday, January.
Xiaoying Sharon Gao Mengjie Zhang Computer Science Victoria University of Wellington Introduction to Artificial Intelligence COMP 307.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 0 of 41 Wednesday, 18.
Kansas State University Department of Computing and Information Sciences CIS 732: Machine Learning and Pattern Recognition Friday, 12 January 2007 William.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 26 of 41 Friday, 22 October.
General Information Course Id: COSC6342 Machine Learning Time: TU/TH 10a-11:30a Instructor: Christoph F. Eick Classroom:AH123
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 1 Friday 22 August 2003.
Computing & Information Sciences Kansas State University Friday, 18 Jan 2008CIS 732 / 830: Machine Learning / Advanced Topics in AI Lecture 0 of 42 Friday,
Copyright © 2006, The McGraw-Hill Companies, Inc. All rights reserved. Decision Support Systems Chapter 10.
Introduction to machine learning and data mining 1 iCSC2014, Juan López González, University of Oviedo Introduction to machine learning Juan López González.
Kansas State University Department of Computing and Information Sciences CIS 690: Data Mining Systems Lecture 0 Monday, May 15, 2000 William H. Hsu Department.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 25 Wednesday, 20 October.
Text Feature Extraction. Text Classification Text classification has many applications –Spam detection –Automated tagging of streams of news articles,
CS 445/545 Machine Learning Winter, 2012 Course overview: –Instructor Melanie Mitchell –Textbook Machine Learning: An Algorithmic Approach by Stephen Marsland.
Introduction to Science Informatics Lecture 1. What Is Science? a dependence on external verification; an expectation of reproducible results; a focus.
Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Monday, January 22, 2001 William.
Computing & Information Sciences Kansas State University Wednesday, 22 Oct 2008CIS 530 / 730: Artificial Intelligence Lecture 22 of 42 Wednesday, 22 October.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 28 of 41 Friday, 22 October.
1 Machine Learning 1.Where does machine learning fit in computer science? 2.What is machine learning? 3.Where can machine learning be applied? 4.Should.
CSI Topics in Fuzzy Systems : Life Log Management Fall Semester, 2008.
Machine Learning, Decision Trees, Overfitting Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 14,
Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Wednesday, January 24, 2001.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 13 of 41 Monday, 20 September.
Kansas State University Department of Computing and Information Sciences CIS 798: Intelligent Systems and Machine Learning Tuesday, December 7, 1999 William.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 9 of 42 Wednesday, 14.
Kansas State University Department of Computing and Information Sciences CIS 690: Data Mining Systems Lab 0 Monday, May 15, 2000 William H. Hsu Department.
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Friday, 14 November 2003 William.
Machine Learning Introduction. Class Info Office Hours –Monday:11:30 – 1:00 –Wednesday:10:00 – 1:00 –Thursday:11:30 – 1:00 Course Text –Tom Mitchell:
Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Monday, January 24, 2000 William.
Berendt: Advanced databases, winter term 2007/08, 1 Advanced databases – Inferring implicit/new.
Data Mining and Decision Support
Kansas State University Department of Computing and Information Sciences CIS 730: Introduction to Artificial Intelligence Lecture 24 of 41 Monday, 18 October.
Introduction: What is AI? CMSC Introduction to Artificial Intelligence January 3, 2002.
Computing & Information Sciences Kansas State University Wednesday, 04 Oct 2006CIS 490 / 730: Artificial Intelligence Lecture 17 of 42 Wednesday, 04 October.
General Information Course Id: COSC6342 Machine Learning Time: TU/TH 1-2:30p Instructor: Christoph F. Eick Classroom:AH301
Computing & Information Sciences Kansas State University Friday, 13 Oct 2006CIS 490 / 730: Artificial Intelligence Lecture 21 of 42 Friday, 13 October.
FNA/Spring CENG 562 – Machine Learning. FNA/Spring Contact information Instructor: Dr. Ferda N. Alpaslan
Kansas State University Department of Computing and Information Sciences CIS 736: Computer Graphics Friday 23 January 2004 William H. Hsu Department of.
Brief Intro to Machine Learning CS539
Machine Learning for Computer Security
School of Computer Science & Engineering
Spring 2003 Dr. Susan Bridges
Artificial Intelligence and Lisp Lecture 13 Additional Topics in Artificial Intelligence LiU Course TDDC65 Autumn Semester,
Topics Covered in COSC 6368 More general topics:
Special Topics in Data Mining Applications Focus on: Text Mining
Data Mining: Concepts and Techniques Course Outline
Basic Intro Tutorial on Machine Learning and Data Mining
Overview of Machine Learning
2004: Topics Covered in COSC 6368
CSE591: Data Mining by H. Liu
Getting Started with Microsoft Azure Machine Learning
Presentation transcript:

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Lecture 0 Friday, January 14, 2000 William H. Hsu Department of Computing and Information Sciences, KSU Readings: Class Introduction (Handout) Chapters 1, 14, 18, Russell and Norvig A Brief Overview of Knowledge Discovery in Databases

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Lecture Outline Course Information: Format, Exams, Resources, Assignments, Grading Overview –Topics covered –What is knowledge discovery in databases (KDD)? –Applications of data engineering Brief Tour of Advanced Artificial Intelligence (AI) Topics Covered –Analytical learning: combining symbolic and numerical AI –Artificial neural networks (ANNs) for KDD –Uncertain reasoning in decision support –Data mining: KDD applications –Genetic algorithms (GAs) for KDD Brief Tour of Data Engineering Topics Covered –Data engineering for KDD –Knowledge engineering

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Course Information and Administrivia Instructor: William H. Hsu – –Phone: (785) (office), (785) (home) –Office hours: after class; 2-3pm Monday, Wednesday, Friday; by appointment Grading –Assignments (6): 30%, reviews (20): 15%, presentations: 15%, midterm: 15%, project: 25% –Lowest homework score and 5 lowest paper review scores dropped Homework –Six (6) assignments: programming (1), application (2), written (3) –Late policy: due on Fridays; free extension to following Monday (if needed by due date); -10% credit per day after 5:00 PM (1700) Monday –Cheating: don’t do it; see introductory handout for policy Project Option –1-hour project option for graduate students (CIS 798) –Term paper or semester research project –Sign up by February 14, 2000 if interested (see class web page)

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Class Resources Web Page (Required) – –Lecture notes (MS PowerPoint 97, PostScript) –Homeworks (MS Word 97, PostScript) –Exam and homework solutions (MS Word 97, PostScript) –Class announcements (students responsibility to follow) and grade postings Course Notes at Copy Center (Required) Class Web Board (Required) – –Login: Students; password: announced in class –Research announcements (seminars, conferences, calls for papers) –Discussions (instructor and other students) Mailing List (Recommended) –Sign-up sheet (if interested) –Reminders, related research, job announcements

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Course Overview Analytical Learning –Combining symbolic and numerical AI –Role of knowledge in learning systems –Explanations, causal reasoning in data engineering, decision support systems Artificial Neural Networks (ANNs) for KDD –Machine learning using ANNs –Encoding knowledge in ANNs Uncertain Reasoning in Decision Support –Applying probability in data engineering –Bayesian (belief) networks (BBNs) –(Bayesian) statistical inference Data Mining: KDD Applications –Some case studies –Issues: KDD life cycle, tools; wrappers for KDD performance enhancement Genetic Algorithms for KDD –Machine learning using GAs; classifier systems for supervised learning –Encoding knowledge in GAs

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Why Knowledge Discovery in Databases? New Computational Capability –Database mining: converting (technical) records into knowledge –Self-customizing programs: learning news filters, adaptive monitors –Learning to act: robot planning, control optimization, decision support –Applications that are hard to program: automated driving, speech recognition Better Understanding of Human Learning and Teaching –Cognitive science: theories of knowledge acquisition (e.g., through practice) –Performance elements: reasoning (inference) and recommender systems Time is Right –Recent progress in algorithms and theory –Rapidly growing volume of online data from various sources –Available computational power –Growth and interest of learning-based industries (e.g., data mining/KDD)

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence What Are KDD and Data Mining? Two Definitions (FAQ List) –The process of automatically extracting valid, useful, previously unknown, and ultimately comprehensible information from large databases and using it to make crucial business decisions –“Torturing the data until they confess” KDD / Data Mining: An Application of Machine Learning –Guides and integrates learning (model-building) processes Learning methodologies: supervised, unsupervised, reinforcement Includes preprocessing (data cleansing) tasks Extends to pattern recognition (inference or automated reasoning) tasks –Geared toward such applications as: Anomaly detection (fraud, inappropriate practices, intrusions) Crisis monitoring (drought, fire, resource demand) Decision support What Data Mining Is Not –Data Base Management Systems: related but not identical field –“Discovering objectives”: still need to understand performance element

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Stages of KDD

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Rule and Decision Tree Learning Example: Rule Acquisition from Historical Data Data –Patient 103 (time = 1): Age 23, First-Pregnancy: no, Anemia: no, Diabetes: no, Previous-Premature-Birth: no, Ultrasound: unknown, Elective C-Section: unknown, Emergency-C-Section: unknown –Patient 103 (time = 2): Age 23, First-Pregnancy: no, Anemia: no, Diabetes: yes, Previous-Premature-Birth: no, Ultrasound: abnormal, Elective C-Section: no, Emergency-C-Section: unknown –Patient 103 (time = n): Age 23, First-Pregnancy: no, Anemia: no, Diabetes: no, Previous-Premature-Birth: no, Ultrasound: unknown, Elective C-Section: no, Emergency-C-Section: YES Learned Rule –IF no previous vaginal delivery, AND abnormal 2nd trimester ultrasound, AND malpresentation at admission, AND no elective C-Section THEN probability of emergency C-Section is 0.6 –Training set: 26/41 = –Test set: 12/20 = 0.600

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Text Mining: Information Retrieval and Filtering 20 USENET Newsgroups –comp.graphicsmisc.forsalesoc.religion.christiansci.space –comp.os.ms-windows.miscrec.autos talk.politics.gunssci.crypt –comp.sys.ibm.pc.hardwarerec.motorcyclestalk.politics.mideastsci.electronics –comp.sys.mac.hardwarerec.sports.baseballtalk.politics.miscsci.med –comp.windows.xrec.sports.hockeytalk.religion.misc – alt.atheism Problem Definition [Joachims, 1996] –Given: 1000 training documents (posts) from each group –Return: classifier for new documents that identifies the group it belongs to Example: Recent Article from comp.graphics.algorithms Hi all I'm writing an adaptive marching cube algorithm, which must deal with cracks. I got the vertices of the cracks in a list (one list per crack). Does there exist an algorithm to triangulate a concave polygon ? Or how can I bisect the polygon so, that I get a set of connected convex polygons. The cases of occuring polygons are these:... Performance of Newsweeder (Naïve Bayes): 89% Accuracy

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Relevant Disciplines Artificial Intelligence Bayesian Methods Cognitive Science Computational Complexity Theory Control Theory Information Theory Neuroscience Philosophy Psychology Statistics Machine Learning Symbolic Representation Planning/Problem Solving Knowledge-Guided Learning Bayes’s Theorem Missing Data Estimators PAC Formalism Mistake Bounds Language Learning Learning to Reason Optimization Learning Predictors Meta-Learning Entropy Measures MDL Approaches Optimal Codes ANN Models Modular Learning Occam’s Razor Inductive Generalization Power Law of Practice Heuristic Learning Bias/Variance Formalism Confidence Intervals Hypothesis Testing

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Specifying A Learning Problem Learning = Improving with Experience at Some Task –Improve over task T, –with respect to performance measure P, –based on experience E. Example: Learning to Filter Spam Articles –T: analyze USENET newsgroup posts –P: function of classification accuracy (discounted error function) –E: training corpus of labeled news files (e.g., annotated from Deja.com) Refining the Problem Specification: Issues –What experience? –What exactly should be learned? –How shall it be represented? –What specific algorithm to learn it? Defining the Problem Milieu –Performance element: How shall the results of learning be applied? –How shall the performance element be evaluated? The learning system?

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence KDD and Software Engineering Rapid KDD Development Environment

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Design Choices and Issues in KDD Completed Design Determine Target Function Article  Spam?Article  Summary Article  Action Determine Type of Training Experience Reinforcements Labeled Corpus Interactive Knowledge Elicitation Unlabeled Corpus Determine Representation of Learned Function Simple Bayesian classifier Linear function of features Decision tree Artificial neural network Determine Learning Algorithm Gradient descent Linear programming Simulated annealing

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Review of AI and Machine Learning: Basic Topics Analytical Learning: Combining Symbolic and Numerical AI –Inductive learning –Role of knowledge and deduction in integrated inductive and analytical learning Artificial Neural Networks (ANNs) for KDD –Common neural representations: current limitations –Incorporating knowledge into ANN learning Uncertain Reasoning in Decision Support –Probabilistic knowledge representation –Bayesian knowledge and data engineering (KDE): elicitation, causality Data mining: KDD applications –Role of causality and explanations in KDD –Framework for data mining: wrappers for performance enhancement Genetic Algorithms (GAs) for KDD –Evolutionary algorithms (GAs, GP) as optimization wrappers –Introduction to classifier systems

Kansas State University Department of Computing and Information Sciences CIS 830: Advanced Topics in Artificial Intelligence Interesting Industrial Applications Reasoning (Inference, Decision Support) Cartia ThemeScapes news stories from the WWW in 1997 Planning, Control Normal Ignited Engulfed Destroyed Extinguished Fire Alarm Flooding DC-ARM - Database Mining NCSA D2K -