1. Ch. 19 – Knowledge in Learning
Supplemental slides for CSE 327
Prof. Jeff Heflin

2. Current Best Hypothesis Search
function CURRENT-BEST-LEARNING(examples) returns a hypothesis H  any hypothesis consistent with the first example in examples for each remaining example in examples do if e is false positive for H then H  choose a specialization of H consistent with examples else if e is false negative for H then H  choose a generalization of H consistent with examples if no consistent specialization/generalization can be found then fail return H
Note: here choose is a special operator that allows you to backtrack to a previous choice and select another option when the search fails. An actual implementation would probably use depth-first search instead.
From Figure 19.2, p. 681

3. Example Learning Problem
(current best hypothesis search)
Training Set
Example
Descriptions
Classifications X1
Color(X1,Red)  Size(X1,Large)  Shape(X1,Circle)
Q(X1) X2
Color(X2,Blue)  Size(X2,Large)  Shape(X2,Square)
Q(X2) X3
Color(X3,Red)  Size(X3,Small)  Shape(X3,Square)
Q(X3) X4
Color(X4,Green)  Size(X4,Large)  Shape(X4,Triangle)
Q(X4) X5
Color(X5,Red)  Size(X5,Small)  Shape(X5,Circle)
Q(X5)
Only consider candidate definitions that are positive conjunctive sentences

4. Current-Best Hypothesis Search1
hypothesis
True
example: status
X1: ok
FP = false positive 2
True
FN = false negative
X2: FP 3
Color(x,Red)
Shape(x,Circle)
X3: FP
X3: ? 4
Color(x,Red)  Size(x,Large) 6
Color(x,Red)  Shape(x,Circle)
X4: ok
X4: ok 5
Color(x,Red)  Size(x,Large) 7
Color(x,Red)  Shape(x,Circle)
X5: FN
X5: ok

5. Version Space Learning
function VERSION-SPACE-LEARNING(examples) returns a version space local variables: V, the version space (the set of all hypotheses)
V  the set of all hypotheses for each example e in examples do if V is not empty then V  VERSION-SPACE-UPDATE(V,e) return V
function VERSION-SPACE-UPDATE(V,e) returns an updated version space
V  {h  V: h is consistent with e} return V
From Figure 19.3, p. 683

6. Ordering on Hypothesis Space
P(x)
Q(x)
R(x)
P(x)  Q(x)
P(x)  R(x)
Q(x)  R(x)
P(x)  Q(x)  R(x)

7. Version Space Update Details
function VERSION-SPACE-UPDATE(G,S,e) returns an updated G-set and S-set (version space)
for each g in G if e is a false positive for g G  G – g G  G  {h : h is the most general specialization of g that is consistent with e and h is more general than some member of S} else if e is a false negative for g G  G – g for each s in S if e is a false positive for s S  S – s else if e is a false negative for s S  S – s S  S  {h : h is the most specific generalization of s that is consistent with e and h is more specific than some member of G}
return G,S

8. Example Learning Problem
(version space learning)
Training Set
Descriptions
Classifications
Size(X1,Large)  Shape(X1,Circle)  Color(X1,Red)
Q(X1)
Size(X2,Large)  Shape(X2,Square)  Color(X2,Blue)
Q(X2)
Size(X3,Small)  Shape(X3,Circle)  Color(X3,Red)
Q(X3)
Size(X4,Small)  Shape(X4,Circle)  Color(X4,Blue)
Q(X4)
Size(X5,Large)  Shape(X5,Square)  Color(X5,Red)
Q(X5)
Only consider candidate definitions that are positive conjunctive sentences