1. MIT AI Lab / LIDS Laboatory for Information and Decision Systems & Artificial Intelligence Laboratory Massachusetts Institute of Technology A Unified Multiresolution Framework for Automatic Target Recognition Eric Grimson, Alan Willsky, Paul Viola, Jeremy S. De Bonet, and John Fisher

2. MIT AI Lab / LIDS Outline Review Multiresolution Analysis Models –MAR (Multiresolution Auto-Regressive) –MNP (Multi-scale Nonparametric) Applications of MNP Models –Synthesis and Super-Resolution –Segmentation and Multi-Look Registration –Classification/Recognition Continuing Efforts

3. MIT AI Lab / LIDS V(x,y)= { } coarsefine Parent Vector Multiresolution parent vector

4. MIT AI Lab / LIDS Compare the Distribution of Parent Vectors

5. MIT AI Lab / LIDS Formally... Parent Vector

6. MIT AI Lab / LIDS Freeman and Simoncelli Steerable Pyramids

7. MIT AI Lab / LIDS Oriented Wavelet Pyramid

8. MIT AI Lab / LIDS …for a SAR image

9. MIT AI Lab / LIDS Capturing Structure (Texture Perspective)

10. MIT AI Lab / LIDS Synthesis Results

11. MIT AI Lab / LIDS Synthesis Results

12. MIT AI Lab / LIDS

13. Ergodic/Stationary A texture is assumed to be many samples of a single process –Each sample is almost certainly dependent on the other samples –But actual location of the samples does not matter –(Space invariant process).

14. MIT AI Lab / LIDS Heeger and Bergen

15. MIT AI Lab / LIDS Heeger and Bergen Texture Synthesis Model

16. MIT AI Lab / LIDS original texture patch synthesized texture patch Sampling Procedure Analysis Synthesis

17. MIT AI Lab / LIDS Not quite right...

18. MIT AI Lab / LIDS Wavelet Representation of Edges Wavelet Transform

19. MIT AI Lab / LIDS Pyramid Representation

20. MIT AI Lab / LIDS Conditional Distributions Wavelet Transform

21. MIT AI Lab / LIDS Probabilistic Model Markov Conditionally Independent Successive Conditioning

22. MIT AI Lab / LIDS Estimating Conditional Distributions Non-parametrically

23. MIT AI Lab / LIDS distribution Similarity sample Likelihood distribution condition example image synthesis discrimination registration segmentation denoising super resolution Outline

24. MIT AI Lab / LIDS original texture patch Sampling Procedure Analysis Synthesis synthesized texture patch

25. MIT AI Lab / LIDS Multiresolution progression

26. MIT AI Lab / LIDS

27. Joint feature occurrence across resolution

28. MIT AI Lab / LIDS Joint feature occurrence across resolution

29. MIT AI Lab / LIDS

31. Texture Synthesis Results

32. MIT AI Lab / LIDS

34. Registration pipeline Tie-point determination Multiresolution texture match: flexible histograms Multiresolution alignment search Inputs are first equalized to remove imaging artifacts Tie-point regions, which provide important matching information, are determined A coarse-to-fine alignment search is used to bring the images into registration Quality of registration is measured by comparing the flexible histogram texture match at the landmark regions.

35. MIT AI Lab / LIDS Distinctive regions provide significant constraint on the correct registration, while more recurrent areas provide little or no useful information.  Localized objects (such as structures or vehicles) match only few locations, thus providing strong constraints on registration.  Extended elements (e.g. roads or tree-lines) match a small area, providing a one dimensional constraint.  Common elements (e.g. grass or forest) match large portions of the image, and provide almost no useful information. Using the only the most distinctive regions as tie-points, reduces the computational requirements and increases the performance of most registration algorithms. By determining those regions which have low expected mutual information with other regions in the image, tie-points are found automatically Tie-point determination

36. MIT AI Lab / LIDS Using this metric, automatically determined tie-points correspond to the visual landmarks that a human observer would use. Here, only vehicles provide distinct landmarks. When present, roads and buildings provide useful landmarks as well. Tie-point examples

37. MIT AI Lab / LIDS Coarse Fine At fine resolutions the registration objective function has many local maxima, causing gradient based techniques to be highly sensitive to initial “seeding” conditions. At coarser resolutions there are fewer local maxima; however, the global maximum tends to be less accurate. Coarse to fine alignment

38. MIT AI Lab / LIDS Coarse-to-Fine Registration In practice, actual data does tend conform to our qualitative assumptions. Coarse resolutions lead to smooth, but inaccurate surfaces, while high resolutions are less smooth, but more accurate. Coarse Fine

39. MIT AI Lab / LIDS At each location in the tie-point region a parent vector is extracted. This vector consists of the multiresolution wavelet decomposition at that location. By measuring the frequency with which locations with similar parent structures occur, a flexible histogram is extracted. V(x,y)= { } coarsefine Parent Vector Measuring Visual Structure : Flexible Histogram I

40. MIT AI Lab / LIDS R tie-point R test parent structure B (x,y)= 8 The registration objective function is the difference in visual structure between the tie-points and the corresponding regions, this is measured with the flexible histogram. Measuring Visual Structure : Flexible Histogram II

41. MIT AI Lab / LIDS A difference measure is acquired by comparing the histogram for the test region, measured with respect to the tie-point, to the the histogram for the tie-point measured with respect to itself. R landmark B(, x,y)= 8  2 =  (B-B’) 2 /B B’(x,y)= 3 R tie-point R test Measuring Visual Structure : Flexible Histogram III

42. MIT AI Lab / LIDS Example Registration

43. MIT AI Lab / LIDS Example Registration

44. MIT AI Lab / LIDS Example Registration

45. MIT AI Lab / LIDS Statistical target discrimination distribution image analysis likelihood estimator likelihood / similarity When compared against a threshold value, this measure provides a discrimination function; comparison against the likelihoods of distributions from other model images, provides a classification mechanism. I MODEL I TEST

46. MIT AI Lab / LIDS Flexible histogram I MODEL parent vector By measuring the frequency with which locations with similar parent vectors occur, a flexible histogram is extracted. B (x,y)= 8

47. MIT AI Lab / LIDS I MODEL I TEST Discrimination via histogram comparison B(x,y)= 8  2 =  (B-B’) 2 /B The histogram for the image, measured with respect to the model, is compared to the the histogram for the model measured with respect to itself. B’(x,y)= 3

48. MIT AI Lab / LIDS where Flexible Histogram The frequency of locations in the image which have a parent structure, whose components are each within a threshold of the parent vector of some location in the model, is given by: A difference measure is calculated by taking chi-square difference between each such frequency count in the model and test image which approximates of the Kullbach-Liebler divergence. Similarity can be measured by simply negating the distance.

49. MIT AI Lab / LIDS BMP2-C21 BTR70-C71T72-132 Models Models for target vehicles were generated from example images: generated from vehicles with different numbers from the target vehicles only 10 examples, evenly distributed in heading angle measured at a depression angle of 17 degrees (targets were at 15 degrees)

50. MIT AI Lab / LIDS BMP2-9563BMP2-9566 BTR70-C71 T72-812T72-S7 Target vehicles Five target vehicles were used. Vehicles which differed from the target class were included as confusion targets. There were roughly 200 images in each class.

51. MIT AI Lab / LIDS 2S1 BRDM2D7 T62 ZIL131ZSU23 Confusion vehicles Six additional confusion vehicles were used as well.

52. MIT AI Lab / LIDS BMP2-C21 BTR70-C71T72-132 Flexible Histograms Template Matching

53. MIT AI Lab / LIDS distribution | | condition Prior beliefs about natural images image analysis Image + Noise maximum likelihood sample Blind Image Denoising