Traffic Sign Recognition Using Discriminative Local Features Andrzej Ruta, Yongmin Li, Xiaohui Liu School of Information Systems, Computing and Mathematics Brunel University, West London

Agenda Problem description Image representation and feature selection Colour discretisation Colour Distance Transform (CDT) Local regions and local dissimilarity Discriminative region selection algorithm Traffic sign recognition System outline Temporal classification Results Conclusions School of Information Systems, Computing and Mathematics, Brunel University, West London

Problem description Input: Output: Points to consider Real-time video stream from a car-mounted, front-looking camera Output: Appropriate visual information, audio signal produced or action taken upon detection and recognition of a sign Points to consider A priori knowledge about the model signs Robustness to noise, varying illumination, uneven motion etc. Real-time performance requirement High cost of false negatives/false positives School of Information Systems, Computing and Mathematics, Brunel University, West London

Colour discretisation Colour discretisation – why bother? Worthwile whenever interest objects contain sparse colours Helps avoid ambiguities Reduces computational burden Scenario 1 – clean template images available Merely changing the physical image representation Proper thresholding in Hue-Saturation-Value space Scenario 2 – real images available Gaussian Mixture model for each distinct colour Supervised training using EM On-line model-driven classification School of Information Systems, Computing and Mathematics, Brunel University, West London

Colour discretisation – example School of Information Systems, Computing and Mathematics, Brunel University, West London

Colour Distance Transform (CDT) The idea: Input: discretised colour image, output: map of distances to the nearest pixel of a given colour Pixels of an interest colour treated as feature pixels, all other pixels treated as non-feature pixels Different distance metrics possible, e.g. (3,4) Chamfer metric Sample output: Original image Black CDT White CDT Red CDT School of Information Systems, Computing and Mathematics, Brunel University, West London

Local regions and local dissimilarity Local image dissimilarity within region rk Average image dissimilarity over region set Weighted average image dissimilarity over region set School of Information Systems, Computing and Mathematics, Brunel University, West London

Discriminative region selection algorithm Assuming a category of targeted object classes and an unknown image , determine the class of by maximising posterior:  indexing variable determining a set of regions to be used  Vector of region relevance In order to learn the best model parameters the following objective function is maximised: School of Information Systems, Computing and Mathematics, Brunel University, West London

Discriminative region selection algorithm – cont. comparison Target class Other classes For each template being compared to the template : Dissimilarity map … STOP when School of Information Systems, Computing and Mathematics, Brunel University, West London

Discriminative region selection algorithm – cont. Determining weights by merging region ranks: Sample output: School of Information Systems, Computing and Mathematics, Brunel University, West London

Traffic sign recognition – system outline School of Information Systems, Computing and Mathematics, Brunel University, West London

Traffic sign recognition – system outline Image preprocessing Intended to highlight characteristic colours and edges Involves colour region clustering to determine RoI-s of certain size Relevant colours enhanced within each RoI to extract colour edges Detection Loy & Barnes’s [2005] equiangular polygon detector Colour edge images used on input Tracking Used only for search region reduction Kalman filter with strong motion assumptions Sign representation Separate discriminative region model trained for each sign Dissimilarity threshold individually tuned for each sign category School of Information Systems, Computing and Mathematics, Brunel University, West London

Temporal classification Single frame Maximum Likelihood approach Maximisation of likelihood equivalent to the minimisation of distance over i Regions and weights denote these learned in the training stage Video sequence Integration of consecutive frame observations instead of individual classifications Classifier’s decision at time t determined from: Observation relevance , dependent on the candidate’s age (and thus size) School of Information Systems, Computing and Mathematics, Brunel University, West London

Results Test data: td RC (55) BC (25) YT (42) BS (13) Overall (135) Real-life video recorded from a moving car 88 clips, 144 signs urban, countryside and motorway scenes td RC (55) BC (25) YT (42) BS (13) Overall (135) detected - 86.4% 100.0% 96.3% 94.4% 95.8% recognised 0.9 90.6% 73.6% 91.2% 85.5% 0.7 88.7% 70.6% 87.7% 0.5 89.5% 96.9% 79.2% 58.3% 80.4% best 93.5% School of Information Systems, Computing and Mathematics, Brunel University, West London

Results School of Information Systems, Computing and Mathematics, Brunel University, West London

Results Classification of signs over time. Ratio of the cumulative distance from the best matching template (upper sign next to each chart) to the cumulative distance from the second best matching template (lower sign next to each chart) is marked with a solid red line. The same but temporally weighted cumulative distance ratio is marked with dashed lines: green (b = 0.8), and blue (b = 0.6). School of Information Systems, Computing and Mathematics, Brunel University, West London

Conclusions Current solution Further work Decent but not sufficient for real-life applications Large gamut of signs recognised Main contribution: road sign representation through discriminative local regions, CDT-based distance metric, region selection method Further work Detection – robustness to noise, occlusions, reduced parametrisation Tracking – probabilistic temporal evolution of pixel/region ”interestingness” Representation – capturing correlations between local regions, alternative definitions of visual saliency Classification – relaxing temporal independence assumption School of Information Systems, Computing and Mathematics, Brunel University, West London

Thank you School of Information Systems, Computing and Mathematics, Brunel University, West London