1. Semi-Supervised Learning and Active Learning
2007/3/20
Semi-Supervised Learning and Active Learning
Machine Learning Center
Faculty of Mathematics and Computer Science
Hebei University
Bin Wu

2. Labeling the examples in the training set L
2007/3/20
Labeling the examples in the training set L
Time consuming
Tedious
Cost consuming
Error prone process
learn a target concept based on
as few as possible labeled examples

3. semi-supervised learning
2007/3/20
reduce the need for labeled training data
semi-supervised learning 
active learning

4. semi-supervised learning
2007/3/20
semi-supervised learning

5. Semi-supervised Learning
2007/3/20
Labeled data set
Unlabeled data set L h
Select
n (xi, h(xi))
n samples
Labeling

6. Semi-supervised Learning
2007/3/20
Semi-supervised Learning
Loop for N iterations
Let h be the classifier obtained by training L on L
Let MCP be the n examples in U on which h makes the most confident predictions
For each x∈MCP do
Remove x from U
Add <x, h(x)> to L

7. 2007/3/20

8. Active Learning L h Labeled data set Unlabeled data set Select
2007/3/20
Labeled data set
Unlabeled data set L h
Select
(xi, c(xi))
1 samples
Labeling

9. Active Learning LOOP for N iterations Selective Sampling
2007/3/20
Active Learning
Selective Sampling
LOOP for N iterations
let h be the classifier obtained by training L on L
let
remove q from U and ask the user for its label c(q)
add <q, c(q)> to L

10. Common characteristics
2007/3/20
Common characteristics
use an additional unlabeled set U
try to maximize the accuracy

11. boosts the accuracy of a supervised
2007/3/20
Semi-supervised Learning
boosts the accuracy of a supervised
learner based on an additional set of
unlabeled examples
Active Learning
minimizes the amount of labeled data
by asking the user to label only the most
informative examples

12. 2007/3/20
Reference
[1] Learning with labeled and unlabeled data, Matthias Seeger, Institute for Adaptive and Neural ComputationUniversity of Edinburgh, December 19, 2002
[2] Xiaojin Zhu, Semi-Supervised Learning Literature Survey, Computer Sciences, University of Wisconsin-Madiso, 2005,
[3] M. Hasenjäger, Helge Ritter, Active Learning in Neural Networks,
[4] M. Hasenjäger, H. Ritter, Active Learning with Local Models, Neural Processing Letters, 7, , 1998
[5] Ion Alexandru Muslea, ACTIVE LEARNING WITH MULTIPLE VIEWS, the Degree DOCTOR OF PHILOSOPHY, FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA, December 2002