2. EADS DS / SDC LTIS Page 1 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Automatic Target Recognition in high resolution Optical Aerial Images 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image Xavier PERROTTONMarc STURZEL Michel ROUX Xavier.perrotton@eads.commarc.sturzel@eads.com michel.roux@enst.fr Image & Signal Processing Laboratory Telecom Paris

3. EADS DS / SDC LTIS Page 2 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen  Objective : make a breakthrough on ATR in visible images  Context –Observation systems (satellites, UAVs, aircraft…) ·Huge volume of data sent back by current and future systems ·Limited number of operators ·Pressure to shorten the loops –Autonomous systems (missiles, UAVs…) ·More intelligence onboard  Strong need in the future for : –Fully automatic processing –Autonomous systems  ATR still unsolved for operational use Why ATR for EADS?

4. EADS DS / SDC LTIS Page 3 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen The problem Challenging problems Lighting, occlusion and background Difficult segmentation Targets size Local descriptors approach Learning appearance characteristics Focusing on discriminative parts of the target

5. EADS DS / SDC LTIS Page 4 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Questions  Can we efficiently use local descriptors?  How to extend application domain by statistical learning?

6. EADS DS / SDC LTIS Page 5 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Local descriptors: method 2 Generating a list of candidate matches 3 Defining an hypothesis 4 Hypothesis propagation Recognized target 1 Selecting and learning keypoints Descriptor : GLOH (Gradient Location orientation Histogram)

7. EADS DS / SDC LTIS Page 6 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Local descriptors: method  Descriptors ·GLOH (Gradient Location orientation Histogram) [1]  How to match Keypoints? ·Looking for the best match on each pixel ·Associating a limited number of matched points for each learned keypoint  How to define an hypothesis ? ·Choosing three points among the best matches ·Evaluating the affine transform  How to propagate an hypothesis? ·Checking for agreement between each candidate point and the geometric model [1] K. Mikolajczykand C. Schmid. A performance evaluation of local descriptors. In Proc. IEEE CVPR, June 2003

8. EADS DS / SDC LTIS Page 7 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Local descriptors: tests on real images (1) Learned target Matched targets

9. EADS DS / SDC LTIS Page 8 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Local descriptors: tests on real images (2) Learned target Matched targets

10. EADS DS / SDC LTIS Page 9 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Local descriptors: tests on real images (3) X Aerial Images difficulties : Few points Not robust to background We must find a way to learn the variability of appearance characteristics

11. EADS DS / SDC LTIS Page 10 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen AdaBoost: a powerful learning concept  Principle : –Iterative learning algorithm introduced by Freund and Schapire [2] –Constructing a “strong” classifier in combining “weak” classifiers –Selecting a “weak” classifier at each iteration  Used for face detection by Viola and Jones [3]  Advantages : –often outperforms most “monolithic” strong classifiers such as Neural Networks –Few parameters to tune [2] Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. 97 [3] Paul Viola and Michael J. Jones. Rapid Object Detection using a Boosted Cascade of Simple Features.IEEE CVPR, 2001

12. EADS DS / SDC LTIS Page 11 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen AdaBoost: algorithm  Adaboost starts with a uniform distribution of “weights” over training samples  We obtain a weak classifier from the weak learning algorithm, h j (x) at each round  We compute  j that measures the confidence assigned to h j (x)  We increase the weights on the training samples that were misclassified  Repeat  At the end, make a weighted linear combination of the weak classifiers obtained at all iterations

13. EADS DS / SDC LTIS Page 12 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Weak classifier Feature X X X X X X X X X X X X X X X X X X Feature output database Database Positive, negative samples Feature + Threshold = weak classifier  A weak classifier is only required to be better than chance  Very simple and computationally inexpensive Haar like features Gabor filters Steerable filters orientation estimation features…

14. EADS DS / SDC LTIS Page 13 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Database  Generation of a representative database with positive and negative samples  The classifier is learned on images of fixed size  Detection is done through a sliding search window  Angle variations : -5° to 5°

15. EADS DS / SDC LTIS Page 14 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Tests on real images Learned different appearance characteristics successfully

16. EADS DS / SDC LTIS Page 15 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Descriptors  Challenge : Finding descriptors less sensitive to background and target texture  Haar like features learn only difference of contrasts –Not enough to discriminate complex textures –But can be very efficient on shadow  Gabor filters, steerable filters, orientation estimation features –More robust to background and target texture

17. EADS DS / SDC LTIS Page 16 7 th CNES/DLR Workshop on Information Extraction and Scene Understanding for Meter Resolution Image – 29/03/07 - Oberpfaffenhofen Conclusion  Local descriptors enable to define an efficient ATR algorithm –Targets can be modelled as a collection of regions –Geometric constraints are efficient to eliminate false alarms  Statistical learning enables to extend the application domain –Selecting the discriminating features –Learning the variability of appearance characteristics –Descriptors -To detect particular oriented edges -To detect different regions