CS 540 - Fall 2016 (Shavlik©), Lecture 2 12/8/2018 Today’s Topics Chapter 2 in One Slide Chapter 18: Machine Learning (ML) Creating an ML Dataset “Fixed-length feature vectors” Relational/graph-based examples HW0 (due in one week) Getting ‘Labeled’ Training Examples Train/Tune/Test Sets N-fold Cross Validation Supervised Learning and Venn Diagrams Read Section 18.8.1 of textbook and Wikipedia articles linked to class home page 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

The Big AI Picture – Chapter 2 The study of ‘agents’ that exist in an environment and perceive, act, and learn 1: Sense Environment 3: Act AI “Agent” 2: Reason 5: Learn 4: Get Feedback 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

What Do You Think Machine Learning Means? Given: Do: Throughout the semester, think of what is missing in current ML, compared to human learning 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

What is Learning? - Herbert Simon “Learning denotes changes in the system that … enable the system to do the same task … more effectively the next time.” - Herbert Simon “Learning is making useful changes in our minds.” - Marvin Minsky But remember, cheese and wine get better over time but don’t learn! 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

Learning from Labeled Examples Positive Examples Negative Examples Category of this example? Concept Solid Red Circle in a (Regular?) Polygon What about? Figures on left side of page Figures drawn before 5pm 2/2/89 <etc> 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

Supervised Machine Learning: Task Overview Real World Feature “Design” (usually done by humans) Feature Space Classifier Construction (done by learning algorithm) Concepts/ Classes/ Decisions 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

Standard Approach for Constructing an ML Dataset for a Task Step 1: Choose a feature space We will use fixed-length feature vectors Choose N features Each feature has Vi possible values Each example is represented by a vector of N feature values (is a point in the feature space) eg <red, 50, round> color weight shape Step 2: Collect examples (“I/O” pairs) Defines a space weight shape color 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

Another View of Std ML Datasets - a Single Table (2D array) Feature 1 Feature 2 . . . Feature N Output Category Example 1 0.0 small red true Example 2 9.3 medium false Example 3 8.2 blue Example M 5.7 green 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

CS 540 - Fall 2016 (Shavlik©), Lecture 2 Standard Feature Types for representing training examples – a source of “domain knowledge” Keep your eye out for places where domain knowledge is (or should be) used in ML Nominal (including Boolean) No ordering among possible values eg, color  {red, blue, green} (vs. color = 1000 Hertz) Linear (or Ordered) Possible values of the feature are totally ordered eg, size  {small, medium, large} ← discrete weight  [0…500] ← continuous Hierarchical (not commonly used) Possible values are partially ordered in an ISA hierarchy eg, shape  closed polygon continuous triangle square circle ellipse 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

CS 540 - Fall 2016 (Shavlik©), Lecture 2 Where We Are Have selected ‘concept’ to learn Have chosen features to rep examples Have created at least 100 labeled examples Next: learn a ‘model’ that can predict output for NEW examples 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2

CS 540 - Fall 2016 (Shavlik©), Lecture 2 Concept Learning Learning systems differ in how they represent concepts Neural Net Backpropagation ID3, C4.5, CART Decision Tree Training Examples AQ, FOIL, Aleph Φ  X  Y Φ  Z Rules . Weighted Sum SVMs If 5x1 + 9x2 – 3x3 > 12 Then + 9/8/16 CS 540 - Fall 2016 (Shavlik©), Lecture 2