1. Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues Dept. of Electrical and Computer Engineering University of British Columbia Vancouver, Canada Kaan Ersahin*, Ian Cumming and Rabab K. Ward

2. Motivation Using Ideas from HVS Spectral Graph Partitioning (SGP)  Utilizing patch-based similarity in SGP  Utilizing contour information in SGP Proposed Scheme Results Summary Future Work OUTLINE 2 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

3. Motivation Manual segmentation of SAR data is a common practice  Human experts are often good at visual interpretation 3 Operational use of polarimetric spaceborne systems means:  Daily acquisitions  more data to analyze  Wider spectrum of users with limited or no expertise in SAR Polarimetry Analysis typically involves: Segmentation  e.g., drawing boundaries between agricultural fields, water - ice separation, etc.  Automated segmentation task is very challenging Edge detection followed by linking or region merging methods often do not perform well  Human vision system (HVS) can perform this task easily Identify lines, contours, patterns and regions and make decisions based on global information Automated analysis procedures are needed  To develop better decision making tools that require less analyst (human) interaction Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

4. 4 Local view Convair-580, C-band, color composite Global view © CSA 2004 Importance of using global view Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

5. Motivation – Developing a better method A number of useful analysis techniques have been developed  ML classifier based on Wishart distribution (Lee et. al )  Eigenvalue decomposition  H / A / α-angle (Cloude - Pottier)  Target decomposition based on physical models (Freeman - Durden)  … their combinations and variants 5 These are based on polarimetric attributes of pixels (or averages in a neighborhood)  Not able to capture the information that human observer can pick up Visual aspect of image data can be used to enhance automated segmentation results  Study how humans handle this task  Use the ideas that have led to the state-of-the-art technique in computer vision Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

6. Using Ideas from HVS In computer vision problems (e.g., segmentation, object detection)  The ultimate goal  To reach the performance level of an human expert 6 1 Gestalt: a configuration or pattern of elements so unified as a whole that its properties cannot be derived from a simple summation of its parts. Similarity (e.g., brightness, color, shape) X X X XO X X X X X X XX O X X X X X XX X O X X X X XX X X O Proximity (geometric) X X X Continuity In computer vision, a promising technique that can utilize these ideas has emerged: Spectral Graph Partitioning What does an image mean for humans?  More than the collection of pixels, represents a meaningful organization of objects or patterns In late 1930s, Gestalt Psychologists 1 studied this phenomenon: perceptual organization  Several cues (i.e., factors that contribute to this process) were reported: Closure

7. Spectral Graph Partitioning (SGP) A pair-wise grouping technique: an alternative to central grouping  No assumption on the statistical distribution of the data (e.g., Gaussian)  Avoids the restriction that all the points must be similar to a prototype (i.e., class mean) Enables combination of multiple cues (e.g., different types of features and data sets) Offers flexibility in the definition of affinity functions (i.e., measure of similarity) 7 G = { V, E } is an undirected graph  V  nodes (data points or pixels)  E  edges (connections between node pairs)   ( i, j )  weights (similarity between node i and node j )  W  similarity matrix ; its entries are the weights:  ( i, j )  W G Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

8. Spectral Graph Partitioning Shi and Malik (2000) showed that solving the eigenvalue problem for the Normalized Graph Laplacian: provides a reasonable solution. Yu and Shi (2003) showed that eigenvectors completely characterize all optimal solutions  Space of global optima can be navigated via orthogonal transforms.  Iteratively solve for a discrete solution that is closest to the continuous global optimum using an alternating optimization procedure  Their method is called Multiclass Spectral Clustering (MSC). 8 To divide the graph into two partitions, intuitively:  similarity between the resulting partitions or  cost of removing all the connections between the candidate partitions (i.e., cut) should be minimized A better way: Minimize the Normalized Cut  Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

9. Utilizing Patch-based Similarity in SGP 9 We have used SGP for classification based on patch- based similarity (IGARSS 2006)  Spectral Clustering algorithm is modified to account for the unique properties of SAR data Instead of pixel intensities, the histograms calculated within an edge-aligned window mask are used as attributes. Similarity is measured using the  2 – distance  Form an affinity matrix to account for spatial proximity  Patch-based similarity cues from multiple channels and proximity are combined in an overall affinity matrix (W) Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues Speckle Reduction Form affinity matrix (W) PolSAR Data Multi-looking Spectral Graph Partitioning

10. Utilizing Contour Information in SGP 10 In IGARSS 2007 we used SGP for segmentation based on contour information. The motivation was:  Region-based techniques perform either: Sequential merging of segments based on an appropriate measure (e.g., likelihood ratio test) Optimization of a global objective function Drawback: contour information – a powerful cue for HVS – is not utilized.  Contour-based techniques often start with edge detection, followed by a linking process. Drawback: Only local information is used; decisions on segment boundaries are made prematurely  Leung and Malik addressed this issue by collecting contour information locally (i.e., through orientation energy (OE), but making the decision globally. Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

11. Utilizing Contour Information in SGP Rotated copies of filters will pick up edge contrast at different orientations: 11 Orientation Energy at orientation angle of 0   Orientation energy of a pixel located at (x,y)  Useful properties:  and form a quadrature pair.  Filters are elongated, information is integrated along the edge  Extended contours will stand out as opposed to short ones Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

12. Based on the presence of an extended contour, pixel pairs can be assigned to same or different partitions  OE is strong along l 2  s 1 and s 3 are in different partitions.  OE is weak along l 1  s 1 and s 2 are in the same partition. Pairwise affinity matrix is formed using Eq. 10: Utilizing Contour Information in SGP 12 Dissimilarity of two pixels  Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

13. 13 Form affinity matrix (W) Form affinity matrix for each channel based on OE To account for proximity in the image plane calculate affinities only within a neighborhood. PolSAR Data Multi-looking Perform multi-looking on SLC data set Segmentation of Polarimetric SAR Data Using Contour Information via Spectral Graph Partitioning Perform the steps of Multiclass Spectral Clustering (MSC) algorithm by Yu and Shi. Utilizing Contour Information in SGP Spectral Graph Partitioning

14. Form affinity matrix W and perform SGP  Similarity is defined between segments obtained from the previous step. (  2 – distance between the histograms is used)  Only consider adjacent segments Proposed Scheme 14 PolSAR Data Multi-looking Perform multi-looking on SLC data set Patch-based similarity Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues Proximity Form affinity matrix W for each data channel  To account for proximity in the image plane calculate affinities only within a neighborhood.  Contour information is measured using Orientation Energy (OE) Perform the Spectral Graph Partitioning (SGP) using the Multiclass Spectral Clustering (MSC) algorithm. Contour Information SGP

15. Data Acquisition:  Convair-580, C - band, Sept. 2004 For the regions # 1 and # 2 the reference segmentation was formed by:  Inspection of the field boundaries and crops on the day of the acquisition  Visual interpretation of the image data  Manual Segmentation Data Set: Westham Island, B.C. 15 © CSA 2004 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

16. Data Set: Westham Island, B.C. 16 For region #3 a classification map was formed using:  GPS measurements at the field boundaries  Inspection of the crops in each field on the day of the acquisition  Visual interpretation of the image data Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues © CSA 2004

17. Data Set: Westham Island, B.C. 17 Pumpkin Hay Barley -1 Bare Soil Potatoes Strawberry Turnip Barley – 2 Segmentation of Polarimetric SAR Data Using Contour Information via Spectral Graph Partitioning

18. Results – Region # 1 18 Wishart 6 fields Wishart result contains isolated pixels Proposed Method:  More homogenous  Visually agrees with reference segmentation Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

19. Results – Region # 2 19 8 fields Wishart result contains isolated pixels Proposed Method:  More homogenous  Visually agrees with reference segmentation Wishart Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

20. Results – Region # 3 20 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 13 different fields Problems:  Adjacent fields with same crop type Pumpkin Grass  Concave regions (Similarity calculation using OE suggests there should be two partitions  Non-adjacent fields with same crop type. (To be solved at the level of classification)

21. A new technique for segmentation of polarimetric SAR data is proposed  Motivated by the visual information content that humans utilize  Is based on SGP which was shown to perform well on computer vision problems A pair-wise grouping technique instead of central grouping.  Contour cue and Proximity is used for initial partitioning  Patch-based similarity is used later to merge adjacent partitions Summary 21 Preliminary results are given on image subsets of Convair-580 data (C-band)  Perceptually plausible results: more homogenous, agree with the reference (i.e., manual) segmentation  Resulting classification is better than Wishart This scheme is flexible to allow further improvement using additional information Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

22. Utilize the complete polarimetric information using pairwise similarity of the coherency matrices. Include additional information (e.g., scattering mechanisms) Optimize the cue combination scheme Compare with techniques other than Wishart Validate methodology for  Different datasets (CV-580)  RADARSAT-2 Future Work 22 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues

23. Dept. of Electrical and Computer Engineering University of British Columbia Vancouver, Canada Kaan Ersahin*, Ian Cumming and Rabab K. Ward