1. Remote Sensing Classification Accuracy

2. 1. Select Test Areas
Selecte test areas in an image to evaluate the accuracy of a classification
Test areas should be representative categorically and geographically
Sampling methods: uniform wall-to-wall, random, stratified random sampling        
Sample size: pixels each category

5. 2. Error Assessment
A classification is not complete until its accuracy is assessed
Error matrix
KHAT statistics

6. Error Matrix Also called confusion matrix and contingency table
Compares the ground truth and the results of the classification for the test areas
Can be used to evaluate the result of classifying the training set pixels and the results of classifying the actual full-scene

7. Error Matrix Diagonal cells are correctly classified pixels
                
Classified                       Reference Data Data     Water  Sand  Forest   Urban    Corn Hay Row Total
Water        
Sand
Forest              Urban
Corn
Hay
Col Total        68    
Diagonal cells are correctly classified pixels
                            correctly classified pixels
Overall accuracy =  = = 84%                                total pixels evaluated

8. Error Matrix
                
Classified                       Reference Data Data     Water  Sand  Forest   Urban    Corn Hay Row Total
Water        
Sand
Forest              Urban
Corn
Hay
Col Total        68    
In this case, the non-diagonal column cells are omission errors
e.g. omission error for forest = 43/356 = 12%
The non-diagonal row cells are commission errors
e.g. commission error for corn 117/459 = 25%

9. Error Matrix Omission error = 1 (100%) - producer's accuracy
                
Classified                       Reference Data Data     Water  Sand  Forest   Urban    Corn Hay Row Total
Water        
Sand
Forest              Urban
Corn
Hay
Col Total        68    
                                  correctly classified in each category producer's accuracy =                             the total pixels used in the category (col total)
Omission error = 1 (100%) - producer's accuracy

10. Error Matrix Commission error = 1 (100%) - user's accuracy
                
Classified                       Reference Data Data     Water  Sand  Forest   Urban    Corn Hay Row Total
Water        
Sand
Forest              Urban
Corn
Hay
Col Total        68    
                                  correctly classified in each category user's accuracy =                           the total pixels used in the category (row total)
Commission error = 1 (100%) - user's accuracy

13. KHAT Statistics
A measure of the difference between the actual agreement between reference data and the results of classification, and the chance agreement between the reference data and a random classifier

14. KHAT Statistics
^      observed accuracy - chance agreement k  =              chance agreement
The KHAT value usually ranges from 0 to 1
0 indicates the classification is not any better than a random assignment of pixels
1 indicates that the classification is 100% improvement from random assignment

15. KHAT Statistics
            r          r        N × S xii -  S (xi+  ×  x+i) ^         i=1       i=1 k =                     r            N2  -  S (xi+  ×  x+i)                   i=1
r - number of rows in the error matrix
xii - number of obs in row i and column i (the diagonal cells)
xi+ - total obs of row i
x+i - total obs of column i
N - total of obs in the matrix

16. KHAT

17. KHAT Statistics
KHAT considers both omission and commission errors

19. Readings
Chapter 7