May 16-18, 2005MultTemp 2005, Biloxi, MS1 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data James C. Tilton Mail Code 606* NASA GSFC Greenbelt, MD William T. Lawrence Natural Sciences Bowie State University Bowie, MD *Computational & Information Sciences and Technology Office

May 16-18, 2005MultTemp 2005, Biloxi, MS2 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Proposal: To develop tools and methods for automated change detection from remotely sensed imagery utilizing a previously developed approach for creating segmentation hierarchies from imagery data. Step-1 Proposal has been submitted to the ROSES-2005 NRA, Land-Cover/Land-Use Change Element

May 16-18, 2005MultTemp 2005, Biloxi, MS3 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data A set of image segmentations that i.consist of segmentations at different levels of detail, in which ii.the coarser segmentations can be produced from merges of regions from the finer segmentations, and iii.the region boundaries are maintained at the full image spatial resolution. What is a Segmentation Hierarchy?

May 16-18, 2005MultTemp 2005, Biloxi, MS4 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data  Image Analysis is transformed from pixel-by- pixel analysis into object-by-object analysis, allowing the utilization of object shape, texture and context for a more robust and accurate analysis.  A hierarchy of segmentations allows dynamic selection of the appropriate level of segmentation detail for each object of interest. Advantages of a Segmentation Hierarchy

May 16-18, 2005MultTemp 2005, Biloxi, MS5 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data  Collected May 17, 2000 over Baltimore, MD.  Four meter spatial resolution.  Four spectral bands: blue, green, red and nir.  384x384 pixel sub-section.  Twelve-level hierarchical segmentation. Example: Ikonos Data

May 16-18, 2005MultTemp 2005, Biloxi, MS6 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Original Image

May 16-18, 2005MultTemp 2005, Biloxi, MS7 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Region Mean Image with 30 Regions

May 16-18, 2005MultTemp 2005, Biloxi, MS8 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Region Mean Image with 18 Regions

May 16-18, 2005MultTemp 2005, Biloxi, MS9 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Region Mean Image with 11 Regions

May 16-18, 2005MultTemp 2005, Biloxi, MS10 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Region Mean Image with 8 Regions

May 16-18, 2005MultTemp 2005, Biloxi, MS11 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Region Mean Image with 6 Regions

May 16-18, 2005MultTemp 2005, Biloxi, MS12 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Region Mean Image with 4 Regions

May 16-18, 2005MultTemp 2005, Biloxi, MS13 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Original Image

May 16-18, 2005MultTemp 2005, Biloxi, MS14 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Twelve Level Hierarchical Boundaries

May 16-18, 2005MultTemp 2005, Biloxi, MS15 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Hierarchical Segmentations produced by RHSEG:  RHSEG is a hybrid of Hierarchical Step-Wise Optimization* region growing with spectral clustering – controlled by spclust_wght parameter. * J. M. Beaulieu and M. Goldberg, “Hierarchy in picture segmentation: A stepwise optimal approach,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, no. 2, pp , RHSEG and HSEGViewer

May 16-18, 2005MultTemp 2005, Biloxi, MS16 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data  Recursive implementation facilitates a highly efficient parallel implementation – a full Landsat TM scene (6500x6500 by 6 bands) can be processed in under 10 minutes with 256 CPUs.  The HSEGViewer program provides a convenient, user-friendly, tool for visualizing and interacting with the image segmentation hierarchies produced by the RHSEG program. RHSEG and HSEGViewer

May 16-18, 2005MultTemp 2005, Biloxi, MS17 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data  HSEGViewer and demo version of RHSEG are available through  More information on RHSEG available at RHSEG and HSEGViewer

May 16-18, 2005MultTemp 2005, Biloxi, MS18 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Assembled a multi-season, multi-year test data set from MODIS Terra AM1 platform for initial tests:  Bands 1-7 (aggregated to 1KM)  Twelve dates: 31 JAN 2003, 19 APR 2003, 09 AUG 2003, 21 OCT 2003, 28 OCT 2003, 18 NOV 2003, 01 FEB 2004, 20 MAR 2004, 11 JUN 2004, 24 SEP 2004, 29 NOV 2004, 28 FEB  1002x1002 pixels at 1km spatial resolution centered roughly over the Salton Sea.  Southern California fires visible in 28 OCT 2003 scene. Monitoring Change: First Steps

May 16-18, 2005MultTemp 2005, Biloxi, MS19 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 21 OCT 2003 Bands 7, 2 & 1 Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS20 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 28 OCT 2003 Bands 7, 2 & 1 Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS21 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 01 FEB 2004 Bands 7, 2 & 1 Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS22 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 01 FEB 2004 Hierarchical Boundary Map 15 regions

May 16-18, 2005MultTemp 2005, Biloxi, MS23 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 01 FEB 2004 Hierarchical Boundary Map 9 regions

May 16-18, 2005MultTemp 2005, Biloxi, MS24 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 01 FEB 2004 Bands 7, 2 & 1 Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS25 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Data setRegion Label # of Pixels Crit. Value N. Dissim.* vs. HL 0 Hierarchical Levels 31 JAN APR AUG OCT OCT NOV FEB MAR MAR MAR JUN JUN SEP NOV FEB * Normalized Dissimilarity vs. Region Mean at Finest Hierarchical Level.

May 16-18, 2005MultTemp 2005, Biloxi, MS26 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Sum of Masks from Minimum Mean Regions Histogram 0: : : : 872 4: : : : : : : : : 29126

May 16-18, 2005MultTemp 2005, Biloxi, MS27 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Land vs. Water Mask (0-3 designated as land and 4-12 as water.)

May 16-18, 2005MultTemp 2005, Biloxi, MS28 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Region Label # of PixelsCrit. Value N. Dissim. vs. HL 0 Boundary Npix/Area Hierarchical Levels Cloud and Snow Detection/Masking 31 JAN 2003 Data Set (A ) Brightest Region: Region 22 is clearly a cloud at all hierarchical levels (by inspection)

May 16-18, 2005MultTemp 2005, Biloxi, MS29 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Bands 7, 2 & 1 (water masked) Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS30 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Cloud Region (water masked)

May 16-18, 2005MultTemp 2005, Biloxi, MS31 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Cloud and Snow Detection/Masking 31 JAN 2003 Data Set (A ) Selected Region: Region 51//55 is mountain snow at hierarchical levels 0 through 33 (by inspection – and change in normalized dissimilarity) Region Label # of Pixels Crit. Value N. Dissim. vs. HL 0 Boundary Npix/Area Hierarchical Levels

May 16-18, 2005MultTemp 2005, Biloxi, MS32 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Bands 7, 2 & 1 (water masked) Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS33 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Cloud and Snow Regions (water masked)

May 16-18, 2005MultTemp 2005, Biloxi, MS34 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Cloud and Snow Detection/Masking 31 JAN 2003 Data Set (A ) Selected Region: Region 64 is mountain snow through hierarchical level 4 (by inspection – and change in normalized dissimilarity) Region Label # of Pixels Crit. Value N. Dissim. vs. HL 0 Boundary Npix/Area Hierarchica l Levels

May 16-18, 2005MultTemp 2005, Biloxi, MS35 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Bands 7, 2 & 1 (water masked) Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS36 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Cloud and Snow Regions (water masked)

May 16-18, 2005MultTemp 2005, Biloxi, MS37 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data Cloud and Snow Detection/Masking 31 JAN 2003 Data Set (A ) Selected Region: Region 61 is mountain snow through hierarchical level 22 (by inspection – and change in normalized dissimilarity) Region Label # of Pixels Crit. Value N. Dissim. vs. HL 0 Boundary Npix/Area Hierarchica l Levels

May 16-18, 2005MultTemp 2005, Biloxi, MS38 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Bands 7, 2 & 1 (water masked) Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS39 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Cloud and Snow Regions (water masked)

May 16-18, 2005MultTemp 2005, Biloxi, MS40 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Bands 7, 2 & 1 (water, clouds & snow masked) Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS41 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data 31 JAN 2003 Bands 7, 2 & 1 (water masked) Band: bandwidth 1: nm 2: nm 7: nm

May 16-18, 2005MultTemp 2005, Biloxi, MS42 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data  Obtain Cloud Mask data product (MOD 35) for pertinent data set dates and compare with RHSEG results.  If available, obtain Snow Cover data product (MOD 10) and compare with RHSEG results.  Obtain other pertinent MODIS data products (e.g. MOD 12 – Land Cover/Land Cover Change, MOD 14 – Thermal Anomalies, Fires & Biomass Burning, MOD 13 – Gridded Vegetation Indices, …) for analysis and comparison. Monitoring Change: Next Steps

May 16-18, 2005MultTemp 2005, Biloxi, MS43 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data  Develop more flexible tools for analyzing RHSEG segmentation hierarchies, including improvements to HSEGViewer.  Implement other dissimilarity criteria in RHSEG, such as the “Spectral Angle Mapper” criterion.  Implement tools to evaluate various spatial features for use in analyzing the RHSEG segmentation hierarchies, such as convex_area, solidity, and extent, as well as texture and fractal measures.  Implement tools to find and track corresponding regions across multi-temporal data sets. Monitoring Change: Next Steps

May 16-18, 2005MultTemp 2005, Biloxi, MS44 Monitoring Change Through Hierarchical Segmentation of Remotely Sensed Image Data  Automate process to flag areas with intra-data change  Create a rule-based automated classification system to label regions  Create a system to evaluate change as “expected” or “unexpected”  Use a rules-based system to flag areas of change that are not expected Automated evaluation of change would facilitate (human) follow-up for change mediation/intervention Monitoring Change: Future Plans