1. Knowledge Mining and Soil Mapping using Maximum Likelihood Classifier with Gaussian Mixture Models
ECE539 final project
Instructor: Yu Hen Hu
Fall 2005
Good morning, everyone. My name is LI Xiao-Bing. I am a Ph.D. candidate from University of Science and Technology of China. Now let me show you my research activities.
Jian Liu
12/13/2005

2. Overview
This study deals with data mining from soil survey maps and soil mapping with mined soil-landscape knowledge.
I’ll give the details of our proposed approaches for redundancy reduction of acoustic model, and my other research work.

3. Soil – landscape models
Soil is a product of the interaction of surrounding environments
“soil-landscape model” (Hudson, 1992)
Soil can be predicated given the environments

4. Environmental variables
Environmental factors affecting soil formation:
Bedrock geology
Elevation (DEM)
Slope gradient
1st derivative along the steepest slope
Profile curvature
2nd derivative along the steepest slope
Planform curvature
2nd derivative perpendicular to contour lines

5. Previous Approaches & Problems
Fuzzy system (Zhu 2001)
Elicits knowledge from a soil scientist and represents it with arbitrary curves
Assumes independence of each environmental variable
ANN (Zhu 2000; Behrens 2005; Scull 2005 )
Black box knowledge representation
High dimensional matrix is hard to comprehend
Decision trees (Bui, 1999; Qi et.al. 2003)
Knowledge extracted is crisp (typical case), no information about gradation

6. Proposal – Knowledge Representation
GMM representation is more suitable because:
Probability representation well captures the physical gradation of the phenomenon
The interactions between environmental variables are taken into account by the multivariate Gaussian distribution
Mixture model gives a great potential of capturing the real distribution
Physically a soil type may have multiple instances.

7. Proposal – Maximum Likelihood Classifier
P(A|Class1) = 0.8
P(A|Class2) = 0.5
A then is classified into class1 based on “Maximum likelihood”
Naturally evaluates the composite effect environmental variables have on the probability of soil formation

8. Algorithm Training procedure: Testing procedure:
Standardize feature dimensions of training set
For each geology group in the training data
For each soil type in the geology group
Fit a GMM using EM algorithm (# of mixtures is preset, k-means is used to initialize the cluster centers)
Testing procedure:
Standardize feature dimensions of testing set
For each sample point
For each class in the corresponding geology group
Calculate the corresponding likelihood based on GMM
The point is classified to the class with the maximum likelihood

9. Case Study … Training set Testing set
elevation
slope gradient
profile curvature
planform curvature
geology
soil map
Testing set
Our objective is to reduce the model size by reducing the redundancy of acoustic model. We proposed the following approaches to reach the goal. The first one is the MCE-based dimensionality reduction to reduce the number of feature dimensions. The second one is the optimal clustering and non-uniform allocation of Gaussian kernels in feature dimension to reduce the number of feature-level parameters. The third one is the state divergence-based determination of the number of Gaussian components of each state to reduce the number of components. …
elevation
soil map
geology

10. Evaluation of the GMM representation
The GMM representations well capture the gradation
of soil on the landscape, which complies well with
expert knowledge
e.g. Council at footslope
e.g. Elbaville at backslope
Let’s go to the first method, the MCE-based dimensionality reduction. Usually the dimensionality reduction is performed by LDA transformation which separates classes through maximizing the ratio of between-class scatter matrix and within-class scatter matrix. But is has little direct relation with the classifier’s target of minimum recognition error rate. So it results in recognition performance degradation. As MCE criterion can adjust the parameters to achieve minimum recognition error, we proposed to adjust the LDA transformation and the classification parameters simultaneously according to the MCE criterion in the DFE framework.

11. Training accuracy & testing accuracy
Overall, 80% classification accuracy against testing data
Increasing number of mixtures leads to higher classification accuracy
at an expense of exponentially increasing storage and computational load
classification accuracy (%)
geology area 1
geology area 2
# of mixtures
training
testing 1
70.04
68.07
79.80
77.13 2
76.66
74.50
78.99
76.84 4
81.51
79.27
80.03
75.55 8
83.17
80.12
84.07
79.23
Let’s go to the first method, the MCE-based dimensionality reduction. Usually the dimensionality reduction is performed by LDA transformation which separates classes through maximizing the ratio of between-class scatter matrix and within-class scatter matrix. But is has little direct relation with the classifier’s target of minimum recognition error rate. So it results in recognition performance degradation. As MCE criterion can adjust the parameters to achieve minimum recognition error, we proposed to adjust the LDA transformation and the classification parameters simultaneously according to the MCE criterion in the DFE framework.

12. Classification Accuracy vs. # of Mixtures
Let’s go to the first method, the MCE-based dimensionality reduction. Usually the dimensionality reduction is performed by LDA transformation which separates classes through maximizing the ratio of between-class scatter matrix and within-class scatter matrix. But is has little direct relation with the classifier’s target of minimum recognition error rate. So it results in recognition performance degradation. As MCE criterion can adjust the parameters to achieve minimum recognition error, we proposed to adjust the LDA transformation and the classification parameters simultaneously according to the MCE criterion in the DFE framework.

13. Mapping accuracy based on field data
64 points are correctly classified out of 83 field sample points (77%), higher than traditional manual based soil survey (usually 60%)
Let’s go to the first method, the MCE-based dimensionality reduction. Usually the dimensionality reduction is performed by LDA transformation which separates classes through maximizing the ratio of between-class scatter matrix and within-class scatter matrix. But is has little direct relation with the classifier’s target of minimum recognition error rate. So it results in recognition performance degradation. As MCE criterion can adjust the parameters to achieve minimum recognition error, we proposed to adjust the LDA transformation and the classification parameters simultaneously according to the MCE criterion in the DFE framework.
Classification result using 8 mixtures
(the dark blue areas are not mapped)

14. More comments
Standardization of feature dimensions is very effective, -- improves mapping accuracy from 55% to 80%
Preprocessing techniques such as data cleaning required by decision tree is not critical to ML because the ML classifier is not as sensitive to training errors as long as they are not of a huge amount.
Let’s go to the first method, the MCE-based dimensionality reduction. Usually the dimensionality reduction is performed by LDA transformation which separates classes through maximizing the ratio of between-class scatter matrix and within-class scatter matrix. But is has little direct relation with the classifier’s target of minimum recognition error rate. So it results in recognition performance degradation. As MCE criterion can adjust the parameters to achieve minimum recognition error, we proposed to adjust the LDA transformation and the classification parameters simultaneously according to the MCE criterion in the DFE framework.

15. Conclusion GMM is suitable to represent soil-landscape knowledge
ML classifier with GMMs is promising for soil knowledge mining and soil mapping
Let’s go to the first method, the MCE-based dimensionality reduction. Usually the dimensionality reduction is performed by LDA transformation which separates classes through maximizing the ratio of between-class scatter matrix and within-class scatter matrix. But is has little direct relation with the classifier’s target of minimum recognition error rate. So it results in recognition performance degradation. As MCE criterion can adjust the parameters to achieve minimum recognition error, we proposed to adjust the LDA transformation and the classification parameters simultaneously according to the MCE criterion in the DFE framework.

16. Future improvement?
Reduce the storage and computational load so that bigger number of mixtures can be used to improve classification accuracy
Use diagonal matrix to replace full covariance matrix (after applying de-correlation to the features)?
Let’s go to the first method, the MCE-based dimensionality reduction. Usually the dimensionality reduction is performed by LDA transformation which separates classes through maximizing the ratio of between-class scatter matrix and within-class scatter matrix. But is has little direct relation with the classifier’s target of minimum recognition error rate. So it results in recognition performance degradation. As MCE criterion can adjust the parameters to achieve minimum recognition error, we proposed to adjust the LDA transformation and the classification parameters simultaneously according to the MCE criterion in the DFE framework.