Automated Footwear Impression Analysis and Retrieval Based on Periodic Patterns Adam Kortylewski Departement of Mathematics and Computer Science Graphics and Vision Research Group University of Basel 11th European Meeting for SPTM 21. October 2014
Main Goals of the Talk Present progress of our recent research on automatic footwear impression retrieval Periodic Pattern Analysis Close the gap between pattern recognition research and forensic practice Lack of real case data
Project Core Target: Development of an automated footwear retrieval system Participation: forensity AG (industry partner) Several German State Criminal Police Offices (data) University of Basel, Graphics and Vision Research Group Duration: 2012 - 2014 [1] Kortylewski, Albrecht, Vetter; Unsupervised Footwear Impression Analysis and Retrieval from Crime Scene Data; ACCV ’14, Workshop on Robust Local Descriptors SCPO not officially listed in the project, but delivered a central contribution for the success of the project
Synthetic VS Real Case Data 2005 De Chazal DFT 81% 2005 Zhang Edge Histogram 88% 2006 Pavlou SIFT 85% 2006 Ghouti Wavelet Decomposition 100% 2007 Crookes Phase Only Correlation 82% 2008 Gueham Correlation Filter 94% 2008 Patil Gabor Feature Maps 91% 2008 AlGarni Hu Moments 99% 2008 Pavlou SIFT codebook 92% 2009 Nibouche SIFT + RANSAC 97% Wei SIFT + Correlation 96% 2014 Luostarinen Review on automated Rec >95%
Challenges for Patter Recognition with Real Case Data Unconstrained, structured noise Unknown location of the pattern Unachievable point to point correspondence Training data is scarce Unknown rotation and translation Wooden Piece cannot be modeled easily
Periodic Patterns A pattern is peroidic if it repeats according to a rigid 2D spatial distribution Out of 1175 references, >50% show periodic patterns Why concentrating on periodic patterns? -> because the problem is very complex, therefore it makes sense to divide it into chunks Depending on the definition of periodic pattern (how many repetitions?)
Periodic Pattern Examples
Pattern Extraction Pipeline Input Image Periodicity Extraction Segmentation Representation Lets assume this is the input image We want to detect the periodic patterns
Pattern Extraction Pipeline Input Image Periodicity Extraction Segmentation Representation Lets assume this is the input image We want to detect the periodic patterns
Pattern Extraction Pipeline Input Image Periodicity Extraction Segmentation Representation
Periodicity Extraction Results Region of Interest State of the Art [2] Our Result [1]
Pattern Extraction Pipeline Input Image Periodicity Extraction Segmentation Representation
Pattern Extraction Pipeline Input Image Periodicity Extraction Segmentation Representation
Segmentation Results Patterns are well seperated from the structured background
Pattern Extraction Pipeline Input Image Periodicity Extraction Segmentation Representation
Query Image Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Rank 6 Rank 7 Rank 8 Rank 9
Query Image Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Rank 6 Rank 7 Rank 8 Rank 9
Query Image Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Rank 6 Rank 7 Rank 8 Rank 9
Overall Performance 133 crime scene impressions and 1175 references Runtime: Feature extraction ~20min Matching < 1 min No manual preprocessing, except providing the scale on the ruler Results are promising but just hold data with periodic pattern Sommer 2015 hinstellen
Closing the Gap between Research and Forensic Practice Research on automation of forensic pattern recognition is not possible without labeled data e.g. crime scene impression + reference impression Need for real case data for research purposes First footwear impression database (by 1st Nov.): http://fid.gravis.cs.unibas.ch Face Recognition, Fingerprints, Iris Analysis, are today far more in the focus of pattern recognition research 2 reasons: Of course commercial use in security, from the research point of view it is more complex, because the patterns are more diverse Fingerprints - NIST SD 27 Know that you all have a lot of work to do
Conclusion Successfully implemented a fully automatic footwear impression matching system based on periodic patterns Forensic image analysis tasks are at the heart of pattern recognition but unnoticed by the research community We need to provide as much data as possible for research purposes, to encourage research activity
adam.kortylewski@unibas.ch +41 61 267 0551 http://gravis.cs.unibas.ch Thank You adam.kortylewski@unibas.ch +41 61 267 0551 http://gravis.cs.unibas.ch