1. INTRODUCTION Problem: Damage condition of residential areas are more concerned than that of natural areas in post-hurricane damage assessment. Recognition of residential and natural areas from commonly used low-spatial-resolution hyperspectral images is thus important. Solution: A spatial-feature extraction method based on hierarchical Fourier transform – Co-occurrence matrix is developed. Spatial and spectral features are then combined to a joint feature vector. Best feature combinations are determined by K-fold cross validation. METHOD Lin Cong, Brian Nutter, Daan Liang Wind Science and Engineering Department of Electrical and Computer Engineering Department of Construction Engineering & Engineering Technology Texas Tech University, Lubbock, TX, USA e-mail: {lin.cong, brian.nutter, daan.liang}@ttu.edubrian.nutter, daan.liang}@ttu.edu Joint Solution of Urban Structure Detection from Hyperion Hyperspectral data This material is based upon work supported by the National Science Foundation under Grant No. 0800487 Fourier Transform Co-occurrence matrix Texture measures Spectral correlation Feature selection Bayes Classification Hyperspectral data PCA components K-means clustering PCA transform Flow chart Datasets (a) (b) (d) (c) (h) (e) (g) (f) Figure 1. (a) Original hyperspectral image taken over Lubbock, TX in 01/2003; (b) – (c) The top two significant PCA bands of Lubbock dataset; (d) Spectral correlation against the spectrum of construction asphalt; (e) Original hyperspectral image taken over New Orleans, LA in 04/2005; (f) – (g) The top two significant PCA bands of New Orleans dataset; (h) Spectral correlation against the spectrum of construction asphalt; (a) (d) (c) (b) Figure 2. (a) A sample of residential region; (b) The Fourier transform of the residential region; (c) Plot of the directional energy distribution in Fourier domain; (d) Co-occurrence matrix calculated with the direction of the maximum energy and offset equal to one; (a) (d) (c) (b) Figure 3. (a) Another sample of residential region; (b) The Fourier transform of the residential region; (c) Plot of the directional energy distribution in Fourier domain; (d) Co-occurrence matrix calculated with the direction of the maximum energy and offset equal to one; (a) (d) (c) (b) Figure 4. (a) A sample of natural region; (b) The Fourier transform of the natural region; (c) Plot of the directional energy distribution in Fourier domain; (d) Co-occurrence matrix calculated with the direction of the maximum energy and offset equal to one; (1) Contrast (CON) (2) Dissimilarity (DIS) (3) Homogeneity (HOM) (4) Similarity (SIM) (5) Angular Second Moment (ASM) (6) Maximum Probability (MAX) (7) Entropy (ENT) Texture measures CON DIS HOM SIM ASM MAX ENT Figure 5. Texture measures of Lubbock dataset CON DIS HOMSIM ASMMAXENT Figure 6. Texture measures of New Orleans dataset Feature Selection K-fold cross validation is applied on the training dataset to determine the best combinations of the spectral and spatial features. RankCombinationErrorRankCombinationErrorRankCombinationError 111001110011.88%2111110110002.17%1004000010000012.5% 211111110011.90%2211011000002.18%1005001000011012.7% 311010110011.97%2310101110012.19%1006001000011112.7% 411010110002.01%2410110010012.21%1007000000011112.7% 511010010012.02%2510110110012.21%1008001010011012.8% 611010100012.04%2611111010012.21%1009000000011012.8% 711101110012.05%2710110110002.23%1010000010011012.9% 811011010002.07%2810110100012.23%1011001010010113.0% 911110010012.08%2911111010002.24%1012001000010113.0% 1011110110012.10%3011111100012.25%1013000010010113.1% 1110111000002.10%3110010110012.26%1014000000010113.2% 1211011100002.11%3211111100002.26%1015001100001013.2% 1310011000002.13%3311110010002.27%1016010100000013.7% 1411001110002.13%3411111000012.28%1017000100001014.7% 1511111110002.13%3511110100002.28%1018001000001015.3% 1611011110002.15%3611011010012.29%1019000000001015.8% 1711110100012.15%3711111110102.30%1020001100000016.1% 1811101110002.16%3811010100002.31%1021000100000019.4% 1911011110012.16%3911111110112.31%1022001000000024.4% 2011011000012.17%4011111000002.31%1023010000000027.2% RankCombinationErrorRankCombinationErrorRankCombinationError 111100010015.67%2111111110016.60%1004000010001018.4% 211100110015.77%2211111010006.63%1005110100001018.5% 311101110005.81%2311111100016.66%1006100010001018.7% 411100010005.84%2411111010016.68%1007010100001019.1% 511100100015.84%2511100000016.75%1008000100001021.0% 611101010005.86%2611111000016.80%1009100000001021.6% 711100110005.87%2711111100006.83%1010100010000021.7% 811100100005.89%2811111000006.93%1011100100001022.2% 911101100005.96%2911110000017.12%1012101100000024.0% 1011101010016.01%3011100110107.36%1013001100000024.6% 1111101100016.03%3111101010107.39%1014010000000026.4% 1211101110016.06%3201101110017.43%1015010100000026.5% 1311110110006.19%3311101100107.44%1016000010000026.6% 1411110010006.29%3411100011007.50%1017101000000027.0% 1511110010016.29%3511101110117.52%1018110000000027.0% 1611110110016.34%3611101010117.55%1019000100000027.7% 1711110100006.39%3711101111007.56%1020110100000028.1% 1811111110006.42%3801100110017.58%1021001000000028.9% 1911110100016.50%3911101110107.58%1022100100000031.8% 2011101000016.57%4001100010017.59%1023100000000039.8% Table 1: Performance of a subset of all joint feature combinations for Lubbock dataset. Features are listed in the combinations following the order: PCA1, PCA2, spectral correlation, CON, DIS, HOM, SIM, ASM, MAX, ENT. A “1” means that the feature in the associated position is selected in the combination, and a “0” means that associated feature is not selected. Table 2: Performance of a subset of all joint feature combinations for New Orleans dataset. Bayes Classification (a) Ground truth (d) Joint solution(c) Purely spatial (b) Purely spectral Figure 7. Results of Bayes classification for Lubbock dataset. (a) Manually made ground truth; (b) – (d) Results by using purely spectral features, purely spatial features, joint features, respectively. Blue: residential areas correctly classified; Green: natural areas correctly classified; Red: residential areas misclassifed as natural areas; Pink: natural areas misclassifed as residential areas. (a) Ground truth(d) Joint solution(c) Purely spatial (b) Purely spectral Figure 8. Results of Bayes classification for New Orleans dataset. (a) Manually made ground truth; (b) – (d) Results by using purely spectral features, purely spatial features, joint features, respectively. Blue: residential areas correctly classified; Green: natural areas correctly classified; Red: residential areas misclassifed as natural areas; Pink: natural areas misclassifed as residential areas. Spectral Solution (avg. error: 15.45%) Residential RegionNatural Region Classified as Residential5044312281 Classified as Natural822261734 Error Rate14.20%16.59% Spatial Solution (avg. error: 13.43%) Residential RegionNatural Region Classified as Residential483197479 Classified as Natural1034666536 Error Rate21.26% 10.10% Joint Solution (avg. error: 10.84%) Residential RegionNatural Region Classified as Residential511276848 Classified as Natural753867168 Error Rate12.85%9.25% Fourier transform – Co-occurrence matrix Residential areas display periodic street patterns while the natural areas are universal. Fourier Transform is applied to detect the directions orthogonal to the street patterns. Gray level co-occurrence matrix is calculated between neighboring pixels with an offset of one in the direction orthogonal to the street patterns. Results Table 3: Error rates of the Bayes classification for Lubbock dataset Spectral Solution (avg. error: 17.39%) ResidentialNatural + River Classified as Residential6460915888 Classified as Natural or River310625617 Error Rate4.59%38.28% Spatial Solution (avg. error: 19.34%) ResidentialNatural + River Classified as Residential6270416116 Classified as Natural or River501125389 Error Rate7.40%38.83% Joint Solution (avg. error: 12.99%) ResidentialNatural + River Classified as Residential6522511699 Classified as Natural or River249029806 Error Rate3.68%28.19% Table 4: Error rates of the Bayes classification for New Orleans dataset “Cross” Bayes Classification 1. Training data of New Orleans dataset is used to train the Bayes classifier, and then the Lubbock dataset is classified. (c) Joint solution(b) Purely spatial (a) Purely spectral Figure 9. “Cross” classification results of Lubbock dataset. Blue: residential areas correctly classified; Green: natural areas correctly classified; Red: residential areas misclassifed as natural areas; Pink: natural areas misclassifed as residential areas. Spectral Solution (avg. error: 69.95%) Residential RegionNatural Region Classified as Residential4021469053 Classified as Natural184514962 Error Rate31.35%93.30% Spatial Solution (avg. error: 12.87%) Residential RegionNatural Region Classified as Residential5519213607 Classified as Natural347360408 Error Rate5.92% 18.38% Joint Solution (avg. error: 20.25%) Residential RegionNatural Region Classified as Residential5652424721 Classified as Natural214149294 Error Rate3.65%33.40% Table 5: Error rates of the “cross” classification for Lubbock dataset (c) Joint solution (b) Purely spatial (a) Purely spectral Figure 10. “Cross” classification results of New Orleans dataset. Blue: residential areas correctly classified; Green: natural areas correctly classified; Red: residential areas misclassifed as natural areas; Pink: natural areas misclassifed as residential areas. Spectral Solution (avg. error: 42.07%) ResidentialNatural + River Classified as Residential6188140110 Classified as Natural or River58341395 Error Rate8.62%96.64% Spatial Solution (avg. error: 18.20%) ResidentialNatural + River Classified as Residential538405999 Classified as Natural or River1387535506 Error Rate20.49%14.45% Joint Solution (avg. error: 18.20%) ResidentialNatural + River Classified as Residential539376079 Classified as Natural or River1377835426 Error Rate20.35%14.65% Table 6: Error rates of the “cross” classification for New Orleans dataset 2. Training data of Lubbock dataset is used to train the Bayes classifier, and then the New Orleans dataset is classified. Conclusion 1.Improved accuracy in Bayes classification between residential and natural areas was achieved by using both spectral and macroscopic spatial information. 2.The spatial features extracted by proposed Fourier transform – Co-occurrence matrix method seem to be reliable in “cross” classification, although the purely spectral information between different datasets is so different that it fails the cross classification. Future work 1.More testing and verification on additional datasets are needed in the future. 2.The segmentations of residential and natural areas can be used for model choice in spectral unmixing. 3.The spectral unmixing results at the same position before and after a hurricane can be compared to assess the damage level.