Lectures 5,6 MACHINE LEARNING EXPERT SYSTEMS

Contents Machine learning Knowledge representation Expert systems

INDUCTION OF DECISION TREES FROM DATA

Sunny Rain Overcast Outlook HumidityWind HighNormalStrongWeak Decision trees Sport=NoSport=Yes

Data from credit history of loan applications

A simplified tree… But how to do it?

The induction algorithm ID3

Partially constructed decision trees STEP 1 STEP 2

A heuristic problem HOW TO SELECT THE BEST PROPERTY?

Approximate trees HighNormal Humidity 85% Outlook 75%100% Sunny Not Sunny

CLASSIFICATION SYSTEMS

A full classification system

Pattern recognition Patterns: –images, personal records, driving habits, etc. Representation: –vector of features (inputs to a neural network) Pattern classification: –Classify a pattern to one of the given classes

> classifier < Marks > classifier < not Marks > classifier < not Marks > classifier < not Marks > classifier < Marks > classifier < not Marks Classifier training

Classifier application > Classifier > Marks Note: The test image does not appear in the training data

LEARNING IN GENERAL

The data and the goals We begin with a collection of positive (and usually negative) examples of a target class (a concept to be learnt) The goal is to infer a general definition that will allow the learner to recognize future instances of the class

Knowledge representation Positive and negative examples can be represented, e.g., in predicate calculus Two positive instances of the concept of “ball” can be expressed as follows: size(obj1,small)  color(obj1,red)  shape(obj1,round) size(obj2,large)  color(obj2,red)  shape(obj2,round) The general concept of “ball” could be defined by: size(X,Y)  color(X,Z)  shape(X,round) where any sentence that unifies with this general definition represents a ball

A general model of the learning process

A set of operations Given a set of training instances, the learner must construct a generalization, heuristic rule or plan that satisfies its goals

The concept space Representation language and the operations define a space of potential concept definitions The learner must search this space to find the desired concept

Heuristic search Learning programs must commit to a direction and order of search, as well as… …to the use of available data and heuristics to search efficiently

PATRICK WINSTON’S PROGRAM ON LEARNING CONCEPTS

Examples and near misses for the concept “arch”

Generalization of descriptions to include multiple examples (I)

Generalization of descriptions to include multiple examples (II)

Specialization of a description to exclude a near miss so that this can’t match Starting with the original we add special constraints

A BRIEF HISTORY OF AI REPRESENTATIONAL SCHEMES

Semantic network developed by Collins & Quillian in their research on human information storage and response times

Network representation of properties of… …snow and ice

…three definitions of the word “plant” Three planes representing…

Intersection path between “cry” and “comfort” (Quillian 1967)

Case frame representation of the sentence “Sarah fixed the chair with glue.”

Conceptual dependency theory of four primitive conceptualizations For example, all actions are assumed to reduce to one or more of the primitive ACTs listed below:

“John ate the egg” “John prevented Mary from giving a book to Bill”

Restaurant script (Schank and Abelson 1977)

Restaurant script (continued)

FRAMES

A frame includes: Frame identification information Its relationship to other frames Descriptors of requirements Procedural information on use of the structure described Frame default information New instance information

Relationship to other frames For instance, the “hotel phone” might be a special instance of “phone”, which might be an instance of a “communication device”

Descriptors of requirements For instance, a chair has its seat between 20 and 40 cm from the floor, its back higher than 60 cm, etc. These requirements may be used to determine when new objects fir the stereotype defined by the frame

Procedural information An important feature of frames is the ability to attach procedural code to a slot

Frame default information These are slot values that are taken to be true when no evidence to the contrary has been found For instance, chairs have four legs, telephones are pushbutton, hotel beds are made by the staff

New instance information Many frame slots may be left unspecified until given a value for a particular instance or when they are needed for some aspect of problem solving For instance, the color of the bedspread may be left unspecified

Part of a frame description of a hotel room “Specialization” indicates a pointer to a superclass

Spatial frame for viewing a cube (Minsky 1975)

CONCEPTUAL GRAPHS: A NETWORK LANGUAGE

Conceptual relations of different arities

“Mary gave John the book” “A dog named emma is brown”

Examples of restriction…

…join, and simplify operations

Inheritance in conceptual graphs

“Tom believes that Jane likes pizza” This example shows the use of a propositional concept

RULE BASED EXPERT SYSTEMS

Architecture of a typical expert system for a particular problem domain

Guidelines to determine whether a problem is appropriate for expert system solution (1) The need for the solution justifies the cost and effort of building an expert system Human expertise is not available in all situations where it is needed The problem may be solved using symbolic reasoning

Guidelines to determine whether a problem is appropriate for expert system solution (2) The problem domain is well structured and does not require commonsense reasoning The problem may not be solved using traditional computing methods Cooperative and articulate experts exist The problem is of proper size and scope

Reasoning with a typical expert system

The role of mental or conceptual models in problem solving

A small system for analysis of automotive problems

The and/or graph searched in the car diagnosis example

The production system at the start of a consultation in the car diagnostic example Imagine that we want to get information about spark plugs

The production system after Rule 1 has fired

The system after Rule 4 has fired Note the stack-based approach to goal reduction

The following dialogue begins with the computer asking the user about the goals present in working memory

CASE BASED REASONING

Case based reasoners Share a common structure For each new problem they –Retrieve appropriate cases from memory –Modify a retrieved case so that it will apply to the current situation –Apply the transformed case –Save the solution, with a record of success or failure, for future use

Possible preference heuristics to help organize the storage and retrieval of cases (1) Goal-directed: Organize cases, at least in part, by goal descriptions; Retrieve cases that have the same goal as the current situation Salient-feature: Prefer cases that match the most important features or those matching the largest number of important features Recency: Prefer cases used most recently

Possible preference heuristics to help organize the storage and retrieval of cases (2) Specify: Look for as exact as possible matches of features before considering more general matches Frequency: Check first the most frequently matched cases Ease of adaptation: Use first cases most easily adapted to the current situation

Transformational analogy

Advantages of rule based approach

Disadvantages

Advantages of case based reasoning

Disadvantages

Combination of rule based and case based systems

3 Robotic Planning

ROBOTIC PLANNING

The blocks world

Predicate based representation

A number of truth relations or rules for robot’s performance

Portion of the search space

Goal state for the blocks world

Descriptor triples Preconditions (P): Conditions the world must meet for an operator to be applied Add List (A): Additions to the state description that are a result of applying the operator Delete List (D): Items that are removed from a state description to create the new state when the operator is applied

Operators as triples of descriptions

A simple tree showing condition action rules

THANK YOU