GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME University of Trento Dept. of Information Engineering and Computer Science Italy GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME Edoardo Pasolli pasolli@disi.unitn.it Farid Melgani melgani@disi.unitn.it July 28, 2011 IGARSS 2011

Introduction Supervised regression approach Pre-processing Feature extraction Regression Image/ Signal Prediction Training sample collection Human expert Training sample quality/quantity Impact on prediction errors

Introduction Active learning approach for classification problems Training (labeled) set Learning (unlabeled) set f2 f2 Training of classifier Active learning method f1 f1 Model of classifier Selected samples from learning (unlabeled) set f2 Insertion in training set Labeling of selected samples Human expert f1 Selected samples after labeling

Objective Propose GP-based active learning strategies for biophysical parameter estimation problems

Gaussian Processes (GPs) Predictive distribution covariance matrix defined by covariance function noise variance

Gaussian Processes (GPs) Example of predicted function : training sample : predicted value : standard deviation of predicted value

Proposed Strategies GP Regression Insertion in Selection training set U: Learning set L: Training set Insertion in training set Selection Human expert L’s: Labeled samples U’s: Selected unlabeled samples Labeling

Proposed Strategies Minimize covariance measure in feature space (Cov) : squared exponential covariance function signal variance length-scale : training sample : covariance function with respect to training sample

Proposed Strategies Minimize covariance measure in feature space (Cov) : training sample : covariance function with respect to training sample : covariance measure with respect to all training samples : selection of samples with minimum values of

Proposed Strategies Maximize variance of predicted value (Var) : training sample : predicted value : standard deviation of predicted value

Proposed Strategies Maximize variance of predicted value (Var) : training sample : variance : selection of samples with maximum values of

Experimental Results Data set description (MERIS) Simulated acquisitions Objective: estimation of chlorophyll concentration in subsurface case I + case II (open and coastal) waters Sensor: MEdium Resolution Imaging Spectrometer (MERIS) # channels: 8 (412-618 nm) Range of chlorophyll concentration: 0.02-54 mg/m3

Experimental Results Data set description (SeaBAM) Real aquisitions Objective: estimation of chlorophyll concentration mostly in subsurface case I (open) waters Sensor: Sea-viewing Wide Field-of-view (SeaWiFS) # channels: 5 (412-555 nm) Range of chlorophyll concentration: 0.02-32.79 mg/m3

Experimental Results Mean Squared Error MERIS SeaBAM

Experimental Results Standard Deviation of Mean Squared Error MERIS SeaBAM

Accuracies on 4000 test samples Experimental Results Detailed results MERIS Accuracies on 4000 test samples Method # training samples MSE σMSE R2 σR2 Full 1000 0.086 - 0.991 Initial 50 1.638 0.869 0.849 0.070 Ran Cov Var 150 0.585 0.378 0.184 0.406 0.105 0.054 0.938 0.961 0.980 0.045 0.010 0.005 300 0.237 0.212 0.095 0.084 0.177 0.975 0.977 0.990 0.008 0.018 0.000

Accuracies on 4000 test samples Experimental Results Detailed results MERIS Accuracies on 4000 test samples Method # training samples MSE σMSE R2 σR2 Full 1000 0.086 - 0.991 Initial 50 1.638 0.869 0.849 0.070 Ran Cov Var 150 0.585 0.378 0.184 0.406 0.105 0.054 0.938 0.961 0.980 0.045 0.010 0.005 300 0.237 0.212 0.095 0.084 0.177 0.975 0.977 0.990 0.008 0.018 0.000

Accuracies on 459 test samples Experimental Results Detailed results SeaBAM Accuracies on 459 test samples Method # training samples MSE σMSE R2 σR2 Full 460 1.536 - 0.806 Initial 60 5.221 2.968 0.526 0.215 Ran Cov Var 160 2.972 2.210 1.818 1.038 0.074 0.029 0.682 0.745 0.784 0.069 0.007 0.003 310 2.062 1.601 1.573 0.687 0.010 0.753 0.800 0.803 0.066 0.001 0.000

Accuracies on 459 test samples Experimental Results Detailed results SeaBAM Accuracies on 459 test samples Method # training samples MSE σMSE R2 σR2 Full 460 1.536 - 0.806 Initial 60 5.221 2.968 0.526 0.215 Ran Cov Var 160 2.972 2.210 1.818 1.038 0.074 0.029 0.682 0.745 0.784 0.069 0.007 0.003 310 2.062 1.601 1.573 0.687 0.010 0.753 0.800 0.803 0.066 0.001 0.000

Conclusions In this work, GP-based active learning strategies for regression problems are proposed Encouraging performances in terms of convergence speed stability Future developments extension to other regression approaches

GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME University of Trento Dept. of Information Engineering and Computer Science Italy GAUSSIAN PROCESS REGRESSION WITHIN AN ACTIVE LEARNING SCHEME Edoardo Pasolli pasolli@disi.unitn.it Farid Melgani melgani@disi.unitn.it July 28, 2011 IGARSS 2011