1. GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME
University of Trento
Dept. of Information Engineering and Computer Science
Italy
GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME
Edoardo Pasolli
Farid Melgani
July 28, 2011
IGARSS 2011

2. Introduction Supervised regression approach Pre-processing
Feature extraction
Regression
Image/
Signal
Prediction
Training sample collection
Human expert
Training sample quality/quantity
Impact on prediction errors

3. Introduction Active learning approach for classification problems
Training (labeled) set
Learning (unlabeled) set f2 f2
Training
of classifier
Active learning
method f1 f1
Model of classifier
Selected samples from
learning (unlabeled) set f2
Insertion in training set
Labeling of selected samples
Human expert f1
Selected samples
after labeling

4. Objective
Propose GP-based active learning strategies for biophysical parameter estimation problems

5. Gaussian Processes (GPs)
Predictive distribution
covariance matrix defined by covariance function
noise variance

6. Gaussian Processes (GPs)
Example of predicted function
: training sample
: predicted value
: standard deviation
of predicted value

7. Proposed Strategies GP Regression Insertion in Selection training set
U: Learning set
L: Training set
Insertion in
training set
Selection
Human expert
L’s: Labeled samples
U’s: Selected
unlabeled samples
Labeling

8. Proposed Strategies Minimize covariance measure in feature space (Cov)
: squared exponential
covariance function
signal variance
length-scale
: training sample
: covariance function
with respect to training sample

9. Proposed Strategies Minimize covariance measure in feature space (Cov)
: training sample
: covariance function
with respect to training sample
: covariance measure with
respect to all training samples
: selection of samples with
minimum values of

10. Proposed Strategies Maximize variance of predicted value (Var)
: training sample
: predicted value
: standard deviation
of predicted value

11. Proposed Strategies Maximize variance of predicted value (Var)
: training sample
: variance
: selection of samples with
maximum values of

12. Experimental Results Data set description (MERIS)
Simulated acquisitions
Objective: estimation of chlorophyll concentration in subsurface case I + case II (open and coastal) waters
Sensor: MEdium Resolution Imaging Spectrometer (MERIS)
# channels: 8 ( nm)
Range of chlorophyll concentration: mg/m3

13. Experimental Results Data set description (SeaBAM) Real aquisitions
Objective: estimation of chlorophyll concentration mostly in subsurface case I (open) waters
Sensor: Sea-viewing Wide Field-of-view (SeaWiFS)
# channels: 5 ( nm)
Range of chlorophyll concentration: mg/m3

14. Experimental Results
Mean Squared Error
MERIS
SeaBAM

15. Experimental Results Standard Deviation of Mean Squared Error MERIS
SeaBAM

16. Accuracies on 4000 test samples
Experimental Results
Detailed results
MERIS
Accuracies on 4000 test samples
Method
# training
samples
MSE
σMSE R2
σR2
Full
1000
0.086 -
0.991
Initial 50
1.638
0.869
0.849
0.070
Ran
Cov
Var
150
0.585
0.378
0.184
0.406
0.105
0.054
0.938
0.961
0.980
0.045
0.010
0.005
300
0.237
0.212
0.095
0.084
0.177
0.975
0.977
0.990
0.008
0.018
0.000

17. Accuracies on 4000 test samples
Experimental Results
Detailed results
MERIS
Accuracies on 4000 test samples
Method
# training
samples
MSE
σMSE R2
σR2
Full
1000
0.086 -
0.991
Initial 50
1.638
0.869
0.849
0.070
Ran
Cov
Var
150
0.585
0.378
0.184
0.406
0.105
0.054
0.938
0.961
0.980
0.045
0.010
0.005
300
0.237
0.212
0.095
0.084
0.177
0.975
0.977
0.990
0.008
0.018
0.000

18. Accuracies on 459 test samples
Experimental Results
Detailed results
SeaBAM
Accuracies on 459 test samples
Method
# training
samples
MSE
σMSE R2
σR2
Full
460
1.536 -
0.806
Initial 60
5.221
2.968
0.526
0.215
Ran
Cov
Var
160
2.972
2.210
1.818
1.038
0.074
0.029
0.682
0.745
0.784
0.069
0.007
0.003
310
2.062
1.601
1.573
0.687
0.010
0.753
0.800
0.803
0.066
0.001
0.000

19. Accuracies on 459 test samples
Experimental Results
Detailed results
SeaBAM
Accuracies on 459 test samples
Method
# training
samples
MSE
σMSE R2
σR2
Full
460
1.536 -
0.806
Initial 60
5.221
2.968
0.526
0.215
Ran
Cov
Var
160
2.972
2.210
1.818
1.038
0.074
0.029
0.682
0.745
0.784
0.069
0.007
0.003
310
2.062
1.601
1.573
0.687
0.010
0.753
0.800
0.803
0.066
0.001
0.000

20. Conclusions
In this work, GP-based active learning strategies for regression problems are proposed
Encouraging performances in terms of
convergence speed
stability
Future developments
extension to other regression approaches

21. GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME
University of Trento
Dept. of Information Engineering and Computer Science
Italy
GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME
Edoardo Pasolli
Farid Melgani
July 28, 2011
IGARSS 2011