1. Automated Image Registration Using Morphological Region of Interest Feature Extraction
Antonio Plaza
University of Extremadura. Caceres, Spain
Jacqueline Le Moigne
NASA Goddard Space Flight Center, USA
Nathan Netanyahu
Bar-Ilan University, Israel & University of Maryland, USA

2. Earth Science Data Integration
Automatic
Multiple Source
Integration
Prediction Models
Satellite, Aircraft
and Field Data
Improved Data Sets
Validation &
Verification
Feedback
Design of Future
Intelligent
Sensor Webs

3. What is Image Registration ?
Navigation or Model-Based Systematic Correction
Orbital, Attitude, Platform/Sensor Geometric Relationship, Sensor Characteristics, Earth Model, ...
Image Registration or Feature-Based Precision Correction
Navigation within a Few Pixels Accuracy
Image Registration Using Selected Features (or Control Points) to Refine Geo-Location Accuracy
2 Approaches:
(1) Image Registration as a Post-Processing (Taken here)
(2) Navigation and Image Registration in a Closed Loop

4. Image Registration Challenges
Multi-Resolution / Mono- or Multi-Instrument
Multi-temporal data
Various spatial resolutions
Various spectral resolutions
Sub-Pixel Accuracy
1 pixel misregistration=> 50% error in NDVI computation
Accuracy Assessment
Synthetic data
"Ground Truth" (manual registration?)
Use down-sampled high-resolution data
Consistency ("circular" registrations) studies

5. Image to Image Registration
• Multi-Temporal
Image Correlation
• Landmarking
• Coregistration
Image Characteristics
(Features) Extraction
Feature
Matching
Incoming Data
Compute
Transform

6. Image to Map Registration
Input Data
Masking and
Feature Extraction
Feature
Matching
Compute
Transform
Map

7. Multi-Sensor Image Registration
ETM/IKONOS Mosaic of Coastal VA Data
ETM+
IKONOS

8. Image Registration Components
Pre-Processing
Cloud Detection, Region of Interest Masking, ...
Feature Extraction (“Control Points”)
Edges, Regions, Contours, Wavelet Coefficients, ...
Feature Matching
Spatial Transformation (a-priori knowledge)
Search Strategy (Global vs Local, Multi-Resolution, ...)
Choice of Similarity Metrics (Correlation, Optimization Method, Hausdorff Distance, ...)
Resampling, Indexing or Fusion

9. UTM of 4 Scene Corners Known from Systematic Correction
Image Registration Subsystem Based on a Chip Database
UTM of 4 Scene Corners Known
from Systematic Correction
Landmark
Chip
Database
(1) Find Chips that
Correspond to the
Incoming Scene
(2) For Each Chip, Extract
Window from Scene,
Using UTM of:
- 4 Approx Scene Corners
- 4 Correct Chip Corners
(3) Register Each (Chip,Window)
Pair and Record Pairs of
Registered Chip Corners
(4) Compute Global Registration
from Multiple Local Ones
(5) Compute Correct UTM
of 4 Scene Corners of
Input Scene
Correct UTM of
4 Chip Corners
Input Scene

10. UTM of 4 Scene Corners Known from Systematic Correction
Image Registration Subsystem Based on Automatic Chip Extraction
UTM of 4 Scene Corners Known from Systematic Correction
Input Scene
Reference Scene
(1) Extract Reference Chips
and Corresponding Input
Windows Using Mathematical
Morphology
(2) Register Each (Chip,Window)
Pair and Record Pairs of
Registered Chip Corners
(refinement step)
(3) Compute Global Registration
from Multiple Local Ones
(4) Compute Correct UTM
of 4 Scene Corners of
Input Scene
Advantages:
Eliminates Need for Chip Database
Cloud Detection Can Easily be Included in Process
Process Any Size Images
Initial Registration Closer to Final Registration =>
Reduces Computation Time and Increases Accuracy.

11. Step 1: Chip-Window Extraction Using Mathematical Morphology
Mathematical Morphology (MM) Concept:
Nonlinear spatial-based technique that provides a framework.
Relies on a partial ordering relation between image pixels.
In greyscale imagery, such relation is given by the digital value of image pixels
Original image
Greyscale MM Basic Operations: K K
Structuring element
(4-pixel radius Disk SE)
Erosion
Dilation

12. Step 1 (Cont.) Binary Erosion Structuring element Structuring element

13. Step 1 (Cont.) Binary Dilation Structuring element Structuring element

14. Step 1 (Cont.) K Greyscale Morphology: Combined Operations
e.g., Erosion + Dilation = Opening K

15. Step 1: Chip-Window Extraction Using Mathematical Morphology
Scale-Orientation Morphological Profiles (SOMP): From Openings and Closings with SEs=Line Segments of Different Orientations
SOMP = Feature Vector D(x,y) at each Pixel (various scales & orientations)
Entropy of D(x,y) = H(D(x,y))
Algorithm:
a. Compute D(x,y) for each (x,y) in reference scene
b. Extract reference chip centered around (x’,y’) with Max[H(D(x’,y’))], e.g. 256x256
c. Compute D(X,Y) for each (X,Y) in search area input scene centered (e.g., 1000x1000) around location (x’,y’)
Compute RMSE(D(X,Y),D(x’,x’)) for all (X,Y) in search area
Extract input window centered around (X’,Y’) with Min(RMSE)
Return to step 2. until predefined number of chips is extracted

16. Step 1: Chip-Window Extraction Using Mathematical Morphology Results(Landsat-7/ETM+ Data - Central VA)
10 Chips Extracted from Reference Scene (Oct. 7, 1999)
10 Windows Extracted from Input Scene (Nov. 8, 1999)

17. Step 2: Chip-Window Refined Registration Using Robust Feature Matching
Reference
Chip
Input
Window
Wavelet
Decomposition
Robust Feature
Matching (RFM)
Using
Hausdorff Distance
Maxima
Extraction
Choice of
Best
Transformation
At Each
Level of {
Overcomplete Wavelet-type Decomposition: Simoncelli Steerable Pyramid
“Maxima” Extraction: Top 5% of Histogram

18. Step 2: Robust Feature Matching Using Hausdorff Distance
Search Transformation Space through Hierarchical Spatial Subdivisions
Perform Monte Carlo Sampling of Control Points
Compute Robust Similarity Measure
k-th smallest squared distance to nearest neighbors, i.e., partial Hausdorff DistancePartial Hausdorff Distance:
Hk(A, B) = Kth a in A minb in B dist (a,b)
(1≤ k ≤ |A|; Kth is the kth smallest element of set; dist(a,b): Euclidean distance)
Prune Search Space by "Range" Similarity Estimates
Iterate and Refine on each Level of Wavelet Decomposition

19. Step 3: Compute Global Registration from All Local Registrations
From each Local Registration, Window-Chip:
Corrected Locations of Four corners of Each Window
i.e.: for each chip-window i, pair correspondences:
(UL_i_X1,UL_i_Y1) to (UL_i_X2,UL_i_Y2)
(UR_i_X1,UR_i_Y1) to (UR_i_X2,UR_i_Y2)
(LL_i_X1,LL_i_Y1) to (LL_i_X2,LL_i_Y2)
(LR_i_X1,LR_i_Y1) to (LR_i_X2,LR_i_Y2)
Use of a Least Mean Square (LMS) Procedure to Compute Global Image Transformation (in pixels)
If n chips, 4n points used for the LMS
=> Step 4: Use Global Transformation to Compute new UTM Coordinates for each of the 4 Corners of the Incoming Scene

20. Results of Global Registration On Landsat-7 VA Test Data

21. Conclusions
Fully Automated System for Registration of Multi-Temporal Landsat Scenes of Any Size, Using Mathematical Morphology and Robust Feature Matching Techniques
MM Chip-Window Extractor Can be Used with Any Other Registration Method
Eliminates Need of Database
Provides Close Initial Match => Follow-up Computations Faster and More Accurate
Further Experimentation On-Going