1. Machine Learning: Introduction

2. What is learning? A broad general definition from psychology:
the process by which a relatively lasting change in behavior or potential behavior occurs as a result of practice or experience
This suggests
there is some entity which is exhibiting behavior
it can somehow perceive and process experiences
its behavior can change
Normally we also assume that the change is somehow an improvement
“Learning is making useful changes in our minds.” – Marvin Minsky
Instructor Notes: Marvin Minsky was one of the earliest computer scientists to work on artificial intelligence. This definition is very human-centric, however

3. What Is Machine Learning?
“Learning denotes changes in a system that ... enable a system to do the same task more efficiently the next time.” –Herbert Simon
Similar to psychology definition, but more focused on a good result
“Learning is constructing or modifying representations of what is being experienced.” – Ryszard Michalski
The experiences perceived must be captured or represented in some way; learning modifies that representation. This definition focuses on the process, rather than the result.

4. So What is Learning? For our purposes:
changes in a system that modify its behavior or outputs
based on some experiences or inputs
in a way that moves those behaviors closer to some desired result
relatively permanently
Instructor Notes: A system can, of course, be a human being, a computer, a rat, an amoeba.
Interesting discussion: are there systems which change their behavior based on inputs but which don’t learn? Does pressing the gas pedal on a car, which changes its behavior, make it a learning system? Or spurring a horse? (Hint – not permanent).

5. What Does That Mean??
In machine learning, the “system” is a computer and its programs; however, this could apply to biological systems as well.
Another way of looking at machine learning is as a way to get a computer to do things without having to explicitly describe what steps to take, by giving it examples or feedback
The computer then looks for patterns which can explain or predict what happens.
The computer is trained through the examples.

6. Is This Machine Learning?
You set your preferred language to Spanish in Google.
Amazon recommends books for you based on what you have bought in the past.
Your credit card is approved at lunch.
You tell your GPS to avoid a corner because the radio says there has been a crash.
1. No: this is explicitly describing it.
2. Yes. Recommendations are made from patterns of which books are typically bought by the same person.
3. Yes. Credit card companies have developed machine learning algorithms to identify fraud.
4. No. This will not be remembered or affect your GPS’s behavior permanently. (This may depend on the specific GPS, though.)

7. Example: Animals Guessing Game
Animals is a simple machine learning program which plays a guessing game.
The machine gradually learns a set of questions and answers
Some of these questions lead to more questions, some to a guess.
One common misconception is that a computer “can only do what we tell it to”. This game is useful to show that there are many ways of “telling” it.
Run the applet provided and show how it adds questions in response to the users.

8. Animals Behind the Scene
Computer: Think of an animal and I will guess it. Ready?
Human: yes
Computer: Is it a frog?
Is it a frog?
This slide and the next two show how the computer learns, by processing its inputs (answers from the user) to modify its representation of the world (animals and their characteristics) and therefore its behavior (the questions it asks and the guesses it makes.)
Here the computer has a very simple representation, consisting of a single guess.
Paula Matuszek 7

9. Animals Behind the Scene
Computer: Is it a frog?
Human: No
Computer: What is it?
Human: A penguin
Computer: Tell me a question that distinguishes a frog from a penguin.
Human: Is it a bird?Computer: What is the answer for a frog?
Human: no
Is it a bird? No
Yes
Is it a penguin?
Is it a frog?
Here the computer has been expanded by learning a second guess, a question, and the yes-no choices for each guess.
Paula Matuszek 8

10. Animals Behind the Scene
Is it a bird?
Computer: Is it a bird?
Human: No
Computer: Is it a frog?
Computer: What is it?
Human: A mouse
Computer: Tell me a question that distinguishes a frog from a mouse.
Human: Does it have fur?
Computer: What is the answer for a frog?
Human: no
Yes No
Is it a penguin?
Does it have fur?
Yes No
Is it a mouse?
Is it a frog?
Another question, another guess, and two more choices.
Paula Matuszek 9

11. The Architecture of a Machine Learning System
Every machine learning system has four parts:
a representation or model of what is being learned
an actor. This is the part that uses the representation and actually does something.
a critic. The part that provides feedback.
a learner. The part that modifies the representation or model, using the feedback.
Based on Russell and Norvig, Artificial Intelligence: A Modern Approach, Third Edition, Prentice-Hall,

12. Animals Guessing Game Architecture
The Animals Guessing Game has all of the parts of a Machine Learning Architecture:
The Representation is a sequence of questions and pairs of yes/no answers (called a binary decision tree).
The Actor “walks” the tree, interacting with a human; at each question it chooses whether to follow the “yes” branch or the “no” branch.
The Critic is the human player telling the game whether it has guessed correctly.
The Learner elicits new questions and adds questions, guesses and branches to the tree.

13. Examples:
This would be a good place to get some examples from our advisory board or other domains.
Paula Matuszek 11

14. To Review:
We have looked at the four components of a machine learning system for several examples:
Representation
Actor
Critic
Learner
In the following slides we examine each component in more detail.

15. Representation
A learning system must have a representation or model of what is being learned.
This is the component that changes based on experience.
In a machine learning system this may be a mathematical model or formula, a set of rules, a decision tree, or some other form of information.

16. Representing The Problem
Representing the problem to be solved is the first decision to be made in any machine learning application
It’s also the most important.
And the one that depends most on knowing the domain -- the field in which the problem is set.
There are two aspects of representing a problem: the behavior that we want to learn, and the inputs we will learn it from.
The module on inputs discusses this in more detail.

17. Representation: Examples
How do we describe our problem?
Guessing an animal: a tree of questions and answers
Playing checkers: the board and piece positions, sets of rules for choosing moves
Deciding whether an is spam: the frequencies of words used in this and in our entire mailbox.
Using an optical character recognition (OCR) system to decode the amount on a check; 6x10 matrix of light/dark pixels; % light pixels; # straight lines, # curved lines

18. Actor
We are building a machine learning system because we want to do something.
make a prediction
sort into categories
look for similarities
The actor is the part of the system that actually does things.
Once a system has learned, or been trained, this is the component we continue to use.
It may be as simple as a formula to be applied, or it may be a complex program
This is the end result of a machine learning algorithm, and why we are doing it. We want to continue to use the actor to accomplish some task.

19. Actor How do we take action?
Guessing an animal: walk the tree and ask associated questions
Playing checkers: look through the rules to identify a move; choose one; make it.
Identifying spam: examine the set of features (word frequencies) to determine whether this is spam
OCRing a check amount: input the features for a digit, output probability for each of 0 through 9.

20. Critic This component provides the experience we learn from.
Typically, it is a set of examples with the decision that should be reached or action that should be taken
But may be any kind of feedback that gives an indication of how close we are to where we want to be.

21. Critic How do we judge correct actions?
Guessing an animal: human feedback
OCRing digits: Human-categorized training set.
Identifying spam: match to a set of human- categorized test documents.
Playing checkers: who won?
Grouping documents: which are most similar in language or content?
Can be generally categorized as supervised, unsupervised, reinforcement.

22. Learner The learner is the core of a machine learning system. It will
examine the information provided by the critic
use it to modify the representation to move toward a more desirable action the next time.
repeat until the performance is satisfactory, or until it stops improving
Normally, this component will be an existing tool which we can use.
Generally, we will focus on which tools to use, what inputs they need, and what the results mean, rather than on how to implement them.
The learner component is what people typically mean when they refer to a machine learning algorithm or method.

23. Learner What does the learner do?
Guessing an animal: ask the user for a question and add it to the binary tree
OCRing digits: modify the importance of different input features.
Identifying spam: change the set of words likely to be in spam.
Playing checkers: increase the chance of using some rules and decrease the chance for others.
Grouping documents: find clusters of similar documents

24. Why Look at Learning??
Understand and improve efficiency of human learning
Improve methods for teaching and tutoring people
Discover new things or structures that were previously unknown to humans
Example: discover relationships between DNA and diseases
Build software tools that can adapt to their users
Example: learn books we might like based on what we have bought.
Reproduce an important aspect of intelligent behavior
One goal of artificial intelligence research is to reproduce intelligent behavior; learning is an essential component of it.