1. Automated Footwear Impression Analysis and Retrieval Based on Periodic Patterns
Adam Kortylewski
Departement of Mathematics and Computer Science
Graphics and Vision Research Group
University of Basel
11th European Meeting for SPTM
21. October 2014

2. Main Goals of the Talk
Present progress of our recent research on automatic footwear impression retrieval
Periodic Pattern Analysis
Close the gap between pattern recognition research and forensic practice
Lack of real case data

3. Project
Core Target: Development of an automated footwear retrieval system
Participation: forensity AG (industry partner)
Several German State Criminal Police Offices (data)
University of Basel, Graphics and Vision Research Group
Duration:
[1] Kortylewski, Albrecht, Vetter; Unsupervised Footwear Impression Analysis and Retrieval from Crime Scene Data; ACCV ’14, Workshop on Robust Local Descriptors
SCPO not officially listed in the project, but delivered a central contribution for the success of the project

4. Synthetic VS Real Case Data
2005 De Chazal DFT 81%
2005 Zhang Edge Histogram 88%
2006 Pavlou SIFT 85%
2006 Ghouti Wavelet Decomposition %
2007 Crookes Phase Only Correlation 82%
2008 Gueham Correlation Filter 94%
2008 Patil Gabor Feature Maps 91%
2008 AlGarni Hu Moments 99%
2008 Pavlou SIFT codebook 92%
2009 Nibouche SIFT + RANSAC 97%
Wei SIFT + Correlation 96%
Luostarinen Review on automated Rec >95%

5. Challenges for Patter Recognition with Real Case Data
Unconstrained, structured noise
Unknown location of the pattern
Unachievable point to point correspondence
Training data is scarce
Unknown rotation and translation
Wooden Piece cannot be modeled easily

6. Periodic Patterns
A pattern is peroidic if it repeats according to a rigid 2D spatial distribution
Out of 1175 references, >50% show periodic patterns
Why concentrating on periodic patterns? -> because the problem is very complex, therefore it makes sense to divide it into chunks
Depending on the definition of periodic pattern (how many repetitions?)

7. Periodic Pattern Examples

8. Pattern Extraction Pipeline
Input Image
Periodicity Extraction
Segmentation
Representation
Lets assume this is the input image
We want to detect the periodic patterns

9. Pattern Extraction Pipeline
Input Image
Periodicity Extraction
Segmentation
Representation
Lets assume this is the input image
We want to detect the periodic patterns

10. Pattern Extraction Pipeline
Input Image
Periodicity Extraction
Segmentation
Representation

11. Periodicity Extraction Results
Region of Interest
State of the Art [2]
Our Result [1]

12. Pattern Extraction Pipeline
Input Image
Periodicity Extraction
Segmentation
Representation

13. Pattern Extraction Pipeline
Input Image
Periodicity Extraction
Segmentation
Representation

14. Segmentation Results
Patterns are well seperated from the structured background

15. Pattern Extraction Pipeline
Input Image
Periodicity Extraction
Segmentation
Representation

16. Query Image
Rank 1
Rank 2
Rank 3
Rank 4
Rank 5
Rank 6
Rank 7
Rank 8
Rank 9

17. Query Image
Rank 1
Rank 2
Rank 3
Rank 4
Rank 5
Rank 6
Rank 7
Rank 8
Rank 9

18. Query Image
Rank 1
Rank 2
Rank 3
Rank 4
Rank 5
Rank 6
Rank 7
Rank 8
Rank 9

19. Overall Performance 133 crime scene impressions and 1175 references
Runtime:
Feature extraction ~20min
Matching < 1 min
No manual preprocessing, except providing the scale on the ruler
Results are promising but just hold data with periodic pattern
Sommer 2015 hinstellen

20. Closing the Gap between Research and Forensic Practice
Research on automation of forensic pattern recognition is not possible without labeled data
e.g. crime scene impression + reference impression
Need for real case data for research purposes
First footwear impression database (by 1st Nov.):
Face Recognition, Fingerprints, Iris Analysis, are today far more in the focus of pattern recognition research
2 reasons:
Of course commercial use in security, from the research point of view it is more complex, because the patterns are more diverse
Fingerprints - NIST SD 27
Know that you all have a lot of work to do

21. Conclusion
Successfully implemented a fully automatic footwear impression matching system based on periodic patterns
Forensic image analysis tasks are at the heart of pattern recognition but unnoticed by the research community
We need to provide as much data as possible for research purposes, to encourage research activity

22. adam.kortylewski@unibas.ch +41 61 267 0551 http://gravis.cs.unibas.ch
Thank You