REMOTE SENSING Multispectral Image Classification Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University

Thematic Maps or Images Multispectral images  Thematic images Themes (also known as classes) Soil Water Vegetation etc. Spectral signatures 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The Classification Process Number and Types of classes Extraction of classification features Classification algorithms (classifiers) Classification accuracy assessment 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Land-use/Land-cover (LULC) Classes 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Feature Extraction Example features Original multispectral bands Subset of bands Derived features Vegetation indices Principal components Textural features 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Feature Space Because of class variability, the “signature” is actually a statistical multivariate distribution of feature vectors. Successful classification requires separated distributions, i.e. minimal overlap. Image classification is essentially a work of feature space partitioning. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Features Selection There may be many features available to us and we need to select “good” features to achieve high classification accuracies. Separation index can be used to measure similarity or dissimilarity among classes. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Types of Image Classification Algorithm Classification processes Supervised classification Unsupervised classification (cluster analysis) Characteristics of classification features Parametric classification Nonparametric classification 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Supervised Classification Collecting training data for each class. Each training pixel is associated with a feature vector. Characterizing the feature pattern for each class from the training data. Delineating class boundaries using mathematical or statistical methods. Assigning pixels to corresponding classes. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Supervised Classification in 2-D Feature Space Satellite image to be classified Feature 2 Training data collection Delineating class boundaries Assigning non-training pixels to certain classes Feature 1 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Statistical Classification Algorithms Let X = (x1, x2,…, xn)T represent an n-dimensional feature vector and 1, 2, …, M be M classes. We now define M decision (or discriminant) functions di(X), i=1, 2,…, M with the property that, if a pixel with feature vector X belongs to class i , then di(X) > dj(X), j=1, 2,…, M, i  j . 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

In the feature space, the decision boundary separating classes i and j is given by values of X which satisfies dij(X) = di(X)  dj(X) = 0 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Minimum Distance Classifier 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The decision boundary is We can also choose Thus, The decision boundary is 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

It does not consider the variance of each class. For minimum distance classifier, each class is characterized by its mean vector in the multidimensional feature space. It does not consider the variance of each class. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Maximum Likelihood Classifier 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

where fi(X) is the probability density function of feature vector of the class i. For 1-dimensional feature space and two classes case, it is equivalent to The ML classifier does not consider the a priori probabilities of individual classes. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Bayes Classifier Let the probability that a particular pixel with feature vector X comes from class i be denoted . If the classifier decides that the pixel comes from j when it actually comes from i, it incurs a loss, denoted Lij. However, the feature vector X may belong to any of the M classes under consideration, thus the average loss incurred in assigning X to j is : 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

From the conditional probability 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The decision for classification is Since p(X) is common for all j(X), we can also choose to use 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Therefore, If we choose 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Thus, the decision criterion for Bayes classifier is The decision function di(X) is 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Bayes Classifier for Gaussian Distribution Consider a 1-dimensional feature space (n = 1) and two classes (M = 2) case. The feature vector X has a normal distribution in each class. For class i, . Thus, 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

If the two classes are equally likely to occur, i. e If the two classes are equally likely to occur, i.e. ,Then, the decision boundary is Xo. pdf pdf of 2 pdf of 1 m1 m2 X Xo Assigning to class 2 Assigning to class 1 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

For an n-dimensional feature space and M-class case, where Ci is the covariance matrix of feature vector X for class i. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

For Gaussian distribution we usually use Therefore, 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

We can also choose The above decision function is a quadratic function and therefore, the decision boundaries are quadratic curves. If the feature vector is truly multivariate Gaussian, the Bayes classifier is the optimal classifier. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Decision function of the minimum distance classifier. If all covariance matrices are equal, Ci = C for i=1,2,…, M, then If C = I (identity matrix) and all classes are equally likely to occur, then Decision function of the minimum distance classifier. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The above equation represents a rotated ellipse centered at . The bivariate normal density (joint density of two normally distributed random variables) can be expressed as Thus, is equivalent to The above equation represents a rotated ellipse centered at . 1 is the eigenvalue of the covariance matrix. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

  11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

If all classes have the same a priori probabilities, For a bivariate (n=2) and 2-class case 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The class boundary can therefore be determined by solving the following equation 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Class parameters 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Assessing the Classification Accuracies The confusion matrix (also known as the error matrix) 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

EC: error of commission EO: error of omission EC: error of commission 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The Kappa analysis The expected accuracy is the accuracy expected based on chance, or the expected accuracy if we randomly assign class values to each pixel. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

1  overall accuracy =1  OA = error by our classification =  1  expected accuracy =1  EA = error by random classification =  11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Classifier Threshold It is implicit in the above classification algorithms that pixels at every point in feature space will be classified into one of the available classes, even if the class likelihoods are very low. Such situation can be dealt with by adopting thresholds to the decision process. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

In practice, thresholds are applied to the discriminant functions and not the probability densities, i.e., 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Independent of the feature vectors of individual pixels. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

A pixel will not be classified if . The decision if for all , and A pixel will not be classified if . 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Unclassified pixels identified by different techniques The chi-square threshold technique The equal-likelihood technique 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Feature Reduction (Ch. 10, Remote Sensing Digital Image Analysis, Richards, 1995) There may be many features (including spectral and textural features) available to users for image classification. Features that do not aid discrimination, by contributing little to the separability of different classes should be discarded. Removal of least effective features is referred to as feature selection, this being one form of feature reduction. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

The other form of feature reduction is to transform the pixel vector into a new set of co-ordinates in which the features that can be removed are made more evident. Feature reduction is performed by checking how separable various classes remain when reduced sets of features are used. For such purpose, a measure of separability of classes is required. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Divergence Divergence is a measure of the separability of a pair of distributions that has its basis in their degree of overlap. It is defined in terms of the likelihood ratio where and are respectively the probability density of classes and at the position x. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Since divergence is never negative it follows therefore that In other words, divergence never decreases as the number of features is increased. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Divergence of a pair of multivariate normal distributions Suppose that and are normal distributions with means and covariances of mi and i and mj and j respectively. It can be shown that 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Other issues Interpretation of the confusion matrix Training data confusion matrix Reference data confusion matrix Proportions of class-specific training data Evaluating uncertainty of classification accuracy by bootstrap resampling Parametric bootstrap resampling (distribution-based stochastic simulation of training samples) Nonparametric bootstrap resampling 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

Unsupervised Classification Unlike the supervised classification, unsupervised classification methods do not require training data. K-means cluster analysis 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

A large number of objects are to be partitioned into m groups based on their scattering in the k-dimensional feature space. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.

As many passes are performed through all the objects as are required, until there are no more changes. 11/23/2018 Laboratory for Remote Sensing Hydrology and Spatial Modeling, Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.