1. Automated Image Registration Using Morphological Region of Interest Feature Extraction Antonio Plaza University of Extremadura. Caceres, Spain Jacqueline Le Moigne NASA Goddard Space Flight Center, USA Nathan Netanyahu Bar-Ilan University, Israel & University of Maryland, USA

2. Automatic Multiple Source Integration Prediction Models Satellite, Aircraft and Field Data Improved Data Sets Validation & Verification Feedback Design of Future Intelligent Sensor Webs Earth Science Data Integration

3. MultiTemp 2005 Jacqueline Le Moigne, 3 What is Image Registration ? Navigation or Model-Based Systematic Correction –Orbital, Attitude, Platform/Sensor Geometric Relationship, Sensor Characteristics, Earth Model,... Image Registration or Feature-Based Precision Correction –Navigation within a Few Pixels Accuracy –Image Registration Using Selected Features (or Control Points) to Refine Geo-Location Accuracy 2 Approaches: (1) Image Registration as a Post-Processing (Taken here) (2) Navigation and Image Registration in a Closed Loop

4. MultiTemp 2005 Jacqueline Le Moigne, 4 Image Registration Challenges Multi-Resolution / Mono- or Multi-Instrument Multi-temporal data Various spatial resolutions Various spectral resolutions Sub-Pixel Accuracy 1 pixel misregistration=> 50% error in NDVI computation Accuracy Assessment Synthetic data "Ground Truth" (manual registration?) Use down-sampled high-resolution data Consistency ("circular" registrations) studies

5. MultiTemp 2005 Jacqueline Le Moigne, 5 Image to Image Registration Incoming Data Image Characteristics (Features) Extraction Multi-Temporal Image Correlation Landmarking Coregistration Feature Matching Compute Transform

6. MultiTemp 2005 Jacqueline Le Moigne, 6 Image to Map Registration Input Data Map Masking and Feature Extraction Feature Matching Compute Transform

7. MultiTemp 2005 Jacqueline Le Moigne, 7 Multi-Sensor Image Registration ETM/IKONOS Mosaic of Coastal VA Data IKONOS ETM+

8. MultiTemp 2005 Jacqueline Le Moigne, 8 Image Registration Components 0Pre-Processing Cloud Detection, Region of Interest Masking,... 1Feature Extraction (“Control Points”) Edges, Regions, Contours, Wavelet Coefficients,... 2Feature Matching Spatial Transformation (a-priori knowledge) Search Strategy (Global vs Local, Multi-Resolution,...) Choice of Similarity Metrics (Correlation, Optimization Method, Hausdorff Distance,...) 3Resampling, Indexing or Fusion

9. MultiTemp 2005 Jacqueline Le Moigne, 9 Image Registration Subsystem Based on a Chip Database Landmark Chip Database UTM of 4 Scene Corners Known from Systematic Correction Correct UTM of 4 Chip Corners Input Scene (1) Find Chips that Correspond to the Incoming Scene (2) For Each Chip, Extract Window from Scene, Using UTM of: - 4 Approx Scene Corners - 4 Correct Chip Corners (3) Register Each (Chip,Window) Pair and Record Pairs of Registered Chip Corners (4) Compute Global Registration from Multiple Local Ones (5) Compute Correct UTM of 4 Scene Corners of Input Scene

10. MultiTemp 2005 Jacqueline Le Moigne, 10 Image Registration Subsystem Based on Automatic Chip Extraction UTM of 4 Scene Corners Known from Systematic Correction Input Scene (1) Extract Reference Chips and Corresponding Input Windows Using Mathematical Morphology (2) Register Each (Chip,Window) Pair and Record Pairs of Registered Chip Corners (refinement step) (3) Compute Global Registration from Multiple Local Ones (4) Compute Correct UTM of 4 Scene Corners of Input Scene Reference Scene Advantages: Eliminates Need for Chip Database Cloud Detection Can Easily be Included in Process Process Any Size Images Initial Registration Closer to Final Registration => Reduces Computation Time and Increases Accuracy.

11. MultiTemp 2005 Jacqueline Le Moigne, 11 Step 1: Chip-Window Extraction Using Mathematical Morphology Mathematical Morphology (MM) Concept: Nonlinear spatial-based technique that provides a framework. Relies on a partial ordering relation between image pixels. In greyscale imagery, such relation is given by the digital value of image pixels Structuring element Original image Erosion K K Dilation (4-pixel radius Disk SE) Greyscale MM Basic Operations:

12. MultiTemp 2005 Jacqueline Le Moigne, 12 Step 1 (Cont.) Structuring element Binary Erosion Structuring element

13. MultiTemp 2005 Jacqueline Le Moigne, 13 Step 1 (Cont.) Structuring element Binary Dilation Structuring element

14. MultiTemp 2005 Jacqueline Le Moigne, 14 K Greyscale Morphology: Combined Operations e.g., Erosion + Dilation = Opening Step 1 (Cont.)

15. MultiTemp 2005 Jacqueline Le Moigne, 15 Step 1: Chip-Window Extraction Using Mathematical Morphology Scale-Orientation Morphological Profiles (SOMP): From Openings and Closings with SEs=Line Segments of Different Orientations –SOMP = Feature Vector D(x,y) at each Pixel (various scales & orientations) –Entropy of D(x,y) = H(D(x,y)) Algorithm: a. Compute D(x,y) for each (x,y) in reference scene b. Extract reference chip centered around (x’,y’) with Max[H(D(x’,y’))], e.g. 256x256 c.Compute D(X,Y) for each (X,Y) in search area input scene centered (e.g., 1000x1000) around location (x’,y’) d.Compute RMSE(D(X,Y),D(x’,x’)) for all (X,Y) in search area e.Extract input window centered around (X’,Y’) with Min(RMSE) f.Return to step 2. until predefined number of chips is extracted

16. MultiTemp 2005 Jacqueline Le Moigne, 16 Step 1: Chip-Window Extraction Using Mathematical Morphology Results(Landsat-7/ETM+ Data - Central VA) 10 Chips Extracted from Reference Scene (Oct. 7, 1999) 10 Windows Extracted from Input Scene (Nov. 8, 1999)

17. MultiTemp 2005 Jacqueline Le Moigne, 17 Step 2: Chip-Window Refined Registration Using Robust Feature Matching Reference Chip Input Window Wavelet Decomposition Wavelet Decomposition Robust Feature Matching (RFM) Using Hausdorff Distance Maxima Extraction Maxima Extraction Choice of Best Transformation At Each Level of Decomposition { Overcomplete Wavelet-type Decomposition: Simoncelli Steerable Pyramid “Maxima” Extraction: Top 5% of Histogram

18. MultiTemp 2005 Jacqueline Le Moigne, 18 Step 2: Robust Feature Matching Using Hausdorff Distance Search Transformation Space through Hierarchical Spatial Subdivisions Perform Monte Carlo Sampling of Control Points Compute Robust Similarity Measure -k-th smallest squared distance to nearest neighbors, i.e., partial Hausdorff Distance Partial Hausdorff Distance: H k (A, B) = K th a in A min b in B dist (a,b) (1≤ k ≤ |A|; K th is the k th smallest element of set; dist(a,b): Euclidean distance) Prune Search Space by "Range" Similarity Estimates Iterate and Refine on each Level of Wavelet Decomposition

19. MultiTemp 2005 Jacqueline Le Moigne, 19 From each Local Registration, Window-Chip: Corrected Locations of Four corners of Each Window i.e.: for each chip-window i, pair correspondences: –(UL_i_X1,UL_i_Y1) to (UL_i_X2,UL_i_Y2) –(UR_i_X1,UR_i_Y1) to (UR_i_X2,UR_i_Y2) –(LL_i_X1,LL_i_Y1) to (LL_i_X2,LL_i_Y2) –(LR_i_X1,LR_i_Y1) to (LR_i_X2,LR_i_Y2) Use of a Least Mean Square (LMS) Procedure to Compute Global Image Transformation (in pixels) If n chips, 4n points used for the LMS => Step 4: Use Global Transformation to Compute new UTM Coordinates for each of the 4 Corners of the Incoming Scene Step 3: Compute Global Registration from All Local Registrations

20. MultiTemp 2005 Jacqueline Le Moigne, 20 Results of Global Registration On Landsat-7 VA Test Data

21. MultiTemp 2005 Jacqueline Le Moigne, 21 Conclusions Fully Automated System for Registration of Multi-Temporal Landsat Scenes of Any Size, Using Mathematical Morphology and Robust Feature Matching Techniques MM Chip-Window Extractor Can be Used with Any Other Registration Method Eliminates Need of Database Provides Close Initial Match => Follow-up Computations Faster and More Accurate Further Experimentation On-Going