1. Introduction to Machine Learning* Prof. D. Spears COSC 4010/5010, Section 1 Spring 2004 * This material is taken from the textbook, Machine Learning, Volume I, Eds. Michalski, Carbonell, and Mitchell, Tioga, 1983, and from Artificial Intelligence by Russell and Norvig.

2. Definition of Machine Learning Informal definition: Any computer program that improves its performance at some task through experience and/or data. Formal definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E. Wow! Look at how much it learned!

3. Other Disciplines From Which Machine Learning Draws Ideas and Techniques machine learning AI probability & statistics computational complexity theory control theory information theory philosophy psychology neurophysiology ethology decision theory game theory optimization biological evolution statistical mechanics

4. Some Learning Strategies/Techniques Rote learning Inductive inference Stochastic/Bayesian inference Deductive inference Reinforcement learning Neural network learning Evolutionary learning Clustering Analogical learning Learning from human instruction (being told) Learning by discovery Case-based reasoning Speed-up learning Multi-strategy learning is very popular Learning can be passive or active

5. Examples of Types of Knowledge Acquired Via Learning Declarative Knowledge Concepts Preferred values of parameters Grammars Taxonomies Procedural Knowledge Rules Rule strengths Graphs/networks Computer programs Plans Example strategies for acquisition: Inductive inference Evolutionary learning Clustering Analogy Induction Reinforcement learning Evolutionary learning Stochastic learning

6. Example Data Structures Used for Learned Knowledge Decision trees Logical expressions Neural networks Condition-action rules Rule sets Finite-state automata Lisp code C code Type of knowledge: Concepts Behavioral rules Plans Computer programs

7. History of Machine Learning 1950’s: Neural modeling E.g., perceptrons (Rosenblatt, 1958) Groundwork for this work was laid by researchers in mathematical biophysics (Rashevsky, 1948) (McCulloch and Pitts, 1943). Major thrust was on learning tabula rasa. Focus on self-organization and neuron-like learning elements. 1960’s: Pattern recognition and decision-theoretic learning Acquire linear, polynomial, or related forms of a discriminant function from a given set of training examples, e.g., (Nilsson, 1965). Samuel’s checker’s program (Samuel, 1959, 1963). Acquired a master level of performance. Statistical decision theory for pattern recognition, e.g., (Watanabe, 1960) (Duda & Hart, 1973). 1969: Minsky & Papert on theoretical limitations of perceptron learning. 1970s: Adaptive control Self-adjust parameters to maintain stability in spite of disturbances, e.g., (Davies, 1970) (Fu, 1971).

8. History of Machine Learning (cont’d) 1960’s and 70’s: Models of human learning High-level symbolic descriptions of knowledge, e.g., logical expressions or graphs/networks, e.g., (Karpinski & Michalski, 1966) (Simon & Lea, 1974). META-DENDRAL (Buchanan, 1978), for example, acquired task- specific expertise (for mass spectrometry) in the context of an expert system. Winston’s (1975) structural learning system learned logic-based structural descriptions from examples. 1970’s: Genetic algorithms Developed by Holland (1975) 1970’s - present: Knowledge-intensive learning A tabula rasa approach typically fares poorly. “To acquire new knowledge a system must already possess a great deal of initial knowledge.” Lenat’s CYC project is a good example.

9. History of Machine Learning (cont’d) 1970’s - present: Alternative modes of learning (besides examples) Learning from instruction, e.g., (Mostow, 1983) (Gordon & Subramanian, 1993) Learning by analogy, e.g., (Veloso, 1990) Learning from cases, e.g., (Aha, 1991) Discovery (Lenat, 1977) 1991: The first of a series of workshops on Multistrategy Learning (Michalski) 1970’s – present: Meta-learning Heuristics for focusing attention, e.g., (Gordon & Subramanian, 1996) Active selection of examples for learning, e.g., (Angluin, 1987), (Gasarch & Smith, 1988), (Gordon, 1991) Learning how to learn, e.g., (Schmidhuber, 1996)

10. History of Machine Learning (cont’d) 1980 – The First Machine Learning Workshop was held at Carnegie-Mellon University in Pittsburgh. 1980 – Three consecutive issues of the International Journal of Policy Analysis and Information Systems were specially devoted to machine learning. 1981 – A special issue of SIGART Newsletter reviewed current projects in the field of machine learning. 1983 – The Second International Workshop on Machine Learning, in Monticello at the University of Illinois. 1986 – The establishment of the Machine Learning journal. 1987 – The beginning of annual international conferences on machine learning (ICML). 1988 – The beginning of regular workshops on computational learning theory (COLT). 1990’s – Explosive growth in the field of data mining, which involves the application of machine learning techniques.

11. A general model of learning agents environment critic learning element problem generator performance element AGENT feedback learning goals knowledge changes external performance standard sensors effectors

12. Evaluating Learners ACCURACYACCURACY AMOUNT OF TRAINING DATA SEEN on unseen data Learning curves

13. Some Ideas for Projects Multi-agent / swarm reinforcement learning Concept learning using logical, stochastic, neural, or evolutionary representations or hybrids Learning a good representation for learning concepts (meta-learning) Data mining: Discovering patterns in large data sets (medical? consumer?) Modeling the process of scientific discovery Evolving a simple artificial brain Cognitive models of human learning “Safe” learning Learning in artificial life/worlds Learning in soccer-playing agents Unsupervised learning (clustering) to develop taxonomies Learning to predict temporal sequences Training a neural network to recognize objects, faces, etc. Multi-agent learning to cooperate or compete Learning to improve game playing strategies Evolving computer programs (genetic programming) Comparative studies of different learning methods A variant of a study found in a machine learning conference paper Analogical learning (e.g., applying knowledge of one case to a new case) Learning a model of a student for intelligent tutoring