1. Competence Centre on Information Extraction and Image Understanding for Earth Observation 1 High Resolution Complex SAR Image Analysis Using Azimuth Sub-Band & Eigenspace Decompositions Houda Chaabouni-Chouayakh (1) and Mihai Datcu (1,2) 1 German Aerospace Center (DLR), Oberpfaffenhofen, Germany 2 Ecole Nationale Supérieure des Télécommunications (ENST), Paris, France CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Images 29/03/07 Oberpfaffenhofen, Germany

2. Competence Centre on Information Extraction and Image Understanding for Earth Observation 2 Motivation With the increase of SAR sensor resolution, we should expect a more detailed analysis and a finer description of the scene. BUT, In high resolution SAR images, we are confronted by:  High diversity of real man-made targets (buildings, parking, roads, vehicles, vegetation,…)  Very complicated structures (various designs and shapes)  Different behaviors to SAR (according to the sensor angle…)  Complex images (amplitude + phase) In the literature, among the image understanding methods, there are:  Azimuth sub-band decomposition (mainly for relevant scatterers detection in high resolution SAR images)  Eigenspace decomposition (mostly for real valued images for face classification and complex-valued images for very specific military targets classification) Low diversity of military targetsHigh diversity of urban targets ≠

3. Competence Centre on Information Extraction and Image Understanding for Earth Observation 3 Outline  Overview on SAR  Description of the method  Azimuth sub-band decomposition  Eigenspace formalism  Results & Discussion  Conclusions & Perspectives

4. Competence Centre on Information Extraction and Image Understanding for Earth Observation 4 Outline  Overview on SAR  Description of the method  Azimuth sub-band decomposition  Eigenspace formalism  Results & Discussion  Conclusions & Perspectives

5. Competence Centre on Information Extraction and Image Understanding for Earth Observation 5 SAR Data Acquisition and Image Formation Transmitting pulsed signals (chirp) in the range direction, when the platform travels in the azimuth direction Coherently adding the successively reflected and received pulses The platform is moving during the illumination time Doppler Effect

6. Competence Centre on Information Extraction and Image Understanding for Earth Observation 6 Doppler Centroid = Doppler shift of a target positioned in the antenna boresight direction Zero DopplerNon-Zero Doppler

7. Competence Centre on Information Extraction and Image Understanding for Earth Observation 7 Outline  Overview on SAR  Description of the method  Azimuth sub-band decomposition  Eigenspace formalism  Results & Discussion  Conclusions & Perspectives

8. Competence Centre on Information Extraction and Image Understanding for Earth Observation 8 Description of the method = a decomposition of the complex spectrum in the azimuth direction = a selection of an azimuth sub- aperture corresponds to a selection of some viewing angles or sensor positions Azimuth Sub-band Decomposition Could be useful to:  study the variation of the signal from one band to another in order to get a finer description of the targets according to different sensor angles  analyze the behavior of the strong scatterers that may exist in the scene

9. Competence Centre on Information Extraction and Image Understanding for Earth Observation 9 Description of the method Azimuth Sub-band Decomposition Evidence of some details which were not in the original image. Loss or fading of some structures In the sub-band images. Low directivity of the corner reflectors. 1 2 3 Original image Sub-band left Sub-band right

10. Competence Centre on Information Extraction and Image Understanding for Earth Observation 10 Description of the method Steps:  Doppler centroid estimation and compensation of Doppler shift  Unweighting in azimuth  Spectrum division into 2 sub-bands  Centering the obtained sub-images  Zero-padding and hamming weighting of each sub-band Azimuth Sub-band Decomposition

11. Competence Centre on Information Extraction and Image Understanding for Earth Observation 11 Description of the method Eigenspace Decomposition or Covariance Formalism Principal Components Analysis (PCA) Σ=XX *T Covariance Matrix Training Data Vectors C1C1 C2C2 C3C3 X= Eigenspace Decomposition Projection & Comparison Classification

12. Competence Centre on Information Extraction and Image Understanding for Earth Observation 12 Description of the method Eigenspace Decomposition or Covariance Formalism Training Data Vectors C1C1 C2C2 C3C3 X= Projection Eigenspace CkCk Training data of the class C k Averaging WCkWCk Average Projection Vector of the training data of C k Projection ( C k )

13. Competence Centre on Information Extraction and Image Understanding for Earth Observation 13 Description of the method Eigenspace Decomposition or Covariance Formalism Principal Components Analysis (PCA) Σ=XX *T Projection Vector of X T In the Eigenspace Vector X T = Covariance Matrix Training Data Vectors C1C1 C2C2 C3C3 X= Eigenspace Decomposition Projection & Comparison Test Data Classification Average Projection Vector of the training data of C k In the Eigenspace

14. Competence Centre on Information Extraction and Image Understanding for Earth Observation 14 Description of the method Eigenspace Decomposition or Covariance Formalism Principal Components Analysis (PCA) Vector Covariance Matrix Training Data Vectors Eigenspace Decomposition Projection & Comparison Test Data Classification How to use the rich information provided by the azimuth sub-band decomposition in the covariance formalism to better classify the strong scatterers of the high resolution SAR images?

15. Competence Centre on Information Extraction and Image Understanding for Earth Observation 15 Description of the method Azimuth Sub-band Decomposition & Eigenspace Decomposition 2-Azimuth Sub-band Decomposition Matrix to Vector Converter Matrix to Vector Converter Original Image Sub-band 2 Sub-band 1 X 1 = X2 =X2 = & X=X= CLASSIFICATION Eigenspace Decomposition

16. Competence Centre on Information Extraction and Image Understanding for Earth Observation 16 Outline  Overview on SAR  Description of the method  Azimuth sub-band decomposition  Eigenspace formalism  Results & Discussion  Conclusions & Perspectives

17. Competence Centre on Information Extraction and Image Understanding for Earth Observation 17 Results & Discussion Description of the database  SAR image over the city of Dresden in Germany, acquired with the Experimental SAR system (E-SAR) of the German Aerospace Center (DLR)  5 classes (50 images from each class): Big Buildings (BB) Average Buildings (AB) Small Buildings (SB) Vegetation (V) Water (W) W BB AB SB V

18. Competence Centre on Information Extraction and Image Understanding for Earth Observation 18 Results & Discussion How to test the performance of our new classifier? Percentage of Good Classification PGC RC k : number of the correctly Recognized test images of the Class k TC k : Total number of the test images of the Class k 1,2,…,5 correspond respectively to BB, AB, SB, V and W Image size n n n 20<n<60

19. Competence Centre on Information Extraction and Image Understanding for Earth Observation 19 Results & Discussion  The covariance with azimuth sub- band decomposition algorithm outperforms the general covariance algorithm for almost all the image sizes.  In fact, the azimuth decomposition provides finer characterization of the strong scatterers with which the big buildings are mainly constructed (corner reflectors, antennas on the roofs,…). Big Building Classification

20. Competence Centre on Information Extraction and Image Understanding for Earth Observation 20 Results & Discussion  By using only the covariance formalism, the algorithm is not able to recognize the average buildings (less than 30% of good classification in most of the cases).  With the azimuth decomposition, the recognition becomes much better, specially for the large image sizes (more than 40% of well- classified average buildings when n>28). Average Building Classification

21. Competence Centre on Information Extraction and Image Understanding for Earth Observation 21 Results & Discussion  A good classification requires a large image size. The surrounding area in this case seems to react as a relevant characteristic. Indeed, an image of small buildings, includes, in general, different small sub- classes (vegetation, cars, roads, lights,...) which have several backscattering behaviors, and thus requires a sufficiently large number of pixels for a good description. Small Building Classification

22. Competence Centre on Information Extraction and Image Understanding for Earth Observation 22 Results & Discussion  The azimuth decomposition improves advantageously the classification.  The image size is determining (more than 40x40 pixels are needed for a recognition over 70%).  The fact that the vegetation could sometimes be considered as a sub- class for the buildings, results in a kind of confusion between classes for the low image sizes. Vegetation Classification

23. Competence Centre on Information Extraction and Image Understanding for Earth Observation 23 Results & Discussion  The azimuth decomposition has no effective amelioration on water classification.  However, combing it with the covariance formalism provides more flexibility to find the optimal image size. Water Classification

24. Competence Centre on Information Extraction and Image Understanding for Earth Observation 24 Results & Discussion PGC k : Percentage of Good Classification of the class k. 1,2,…,5 correspond respectively to BB, AB, SB, V and W. What is the optimal image size?

25. Competence Centre on Information Extraction and Image Understanding for Earth Observation 25 Results & Discussion BBABSBVW BB54%10%6% AB24%66%20%6% SB22%20%58%12%2% V4%16%78%30% W4%68% Classification results in terms of confusion matrix when n=n opt =48  p: predicted class  a: actual class p a

26. Competence Centre on Information Extraction and Image Understanding for Earth Observation 26 Results & Discussion BBABSBVW BB70%6%2% AB18%60%24% SB12%28%60%8% V6%14%80%22% W12%70% Classification results in terms of confusion matrix when n=n opt =58 p a  p: predicted class  a: actual class

27. Competence Centre on Information Extraction and Image Understanding for Earth Observation 27 Results & Discussion: Training Data Selection Eigenspace & Azimuth Decomposition Training Data Test Data Best Classification? STOP yes no Classification Quality Test TD: Training data Classification Quality function

28. Competence Centre on Information Extraction and Image Understanding for Earth Observation 28 Results & Discussion: Training Data Selection Eigenspace & Azimuth Decomposition Training Data Test Data Best Classification? STOP yes no Steps of the training data selection algorithm Step 0: initialization of the training data: TD=TD 0 Step 1: classification of the whole database using the covariance with azimuth sub-band decomposition algorithm. Step 2: computation of the classification quality function C(TD) and evaluation of the stopping condition of the algorithm: C(TD)>70% If the stopping condition is not yet met, a new training data TD* is chosen by changing randomly the training images of the class k* which has the lowest percentage of good classification. The algorithm is then run again from Step 1.

29. Competence Centre on Information Extraction and Image Understanding for Earth Observation 29 Results & Discussion: Training Data Selection BBABSBVW BB70%6%2% AB18%60%24% SB12%28%60%8% V6%14%80%22% W12%70% Eigenspace & Azimuth Decomposition Training Data Test Data Best Classification? STOP yes no BBABSBVW BB72%6%2% AB18%72%10% SB10%16%74%4% V6%14%88%22% W8%74% p a p a Confusion matrix when n=n opt =58

30. Competence Centre on Information Extraction and Image Understanding for Earth Observation 30 Outline  Overview on SAR  Description of the method  Azimuth sub-band decomposition  Eigenspace formalism  Results & Discussion  Conclusions & Perspectives

31. Competence Centre on Information Extraction and Image Understanding for Earth Observation 31 Conclusions & Perspectives  A preliminary classification of high resolution SAR images has been performed on a five-class database (BB; AB, SB, V and W).  The proposed method aims at combining the rich information provided by the azimuth decomposition with the promising properties of the covariance formalism, to get a superior classification performance.  To evaluate the performance of our classifier, a study on the optimal image size was carried out.  It was demonstrated that:  The covariance with azimuth decomposition algorithm outperforms the general covariance algorithm for all the classes for almost all the image sizes.  The image size is an important and determining parameter for the classification.  The azimuth decomposition provides more flexibility in the choice of the optimal image size.  As perspectives, we can propose:  To test the performance of the proposed method for unsupervised classifications.  To compare the method to some non-linear classification methods (ICA,…)

32. Competence Centre on Information Extraction and Image Understanding for Earth Observation 32 High Resolution Complex SAR Image Analysis Using Azimuth Sub-Band & Eigenspace Decompositions Houda Chaabouni-Chouayakh (1) and Mihai Datcu (1,2) 1 German Aerospace Center (DLR), Oberpfaffenhofen, Germany 2 Ecole Nationale Supérieure des Télécommunications (ENST), Paris, France CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Images 29/03/07 Oberpfaffenhofen, Germany