1. Traffic Sign Recognition Using Discriminative Local Features Andrzej Ruta, Yongmin Li, Xiaohui Liu School of Information Systems, Computing and Mathematics Brunel University, West London

2. Agenda Problem description Image representation and feature selection
Colour discretisation
Colour Distance Transform (CDT)
Local regions and local dissimilarity
Discriminative region selection algorithm
Traffic sign recognition
System outline
Temporal classification
Results
Conclusions
School of Information Systems, Computing and Mathematics, Brunel University, West London

3. Problem description Input: Output: Points to consider
Real-time video stream from a car-mounted, front-looking camera
Output:
Appropriate visual information, audio signal produced or action taken upon detection and recognition of a sign
Points to consider
A priori knowledge about the model signs
Robustness to noise, varying illumination, uneven motion etc.
Real-time performance requirement
High cost of false negatives/false positives
School of Information Systems, Computing and Mathematics, Brunel University, West London

4. Colour discretisation
Colour discretisation – why bother?
Worthwile whenever interest objects contain sparse colours
Helps avoid ambiguities
Reduces computational burden
Scenario 1 – clean template images available
Merely changing the physical image representation
Proper thresholding in Hue-Saturation-Value space
Scenario 2 – real images available
Gaussian Mixture model for each distinct colour
Supervised training using EM
On-line model-driven classification
School of Information Systems, Computing and Mathematics, Brunel University, West London

5. Colour discretisation – example
School of Information Systems, Computing and Mathematics, Brunel University, West London

6. Colour Distance Transform (CDT)
The idea:
Input: discretised colour image, output: map of distances to the nearest pixel of a given colour
Pixels of an interest colour treated as feature pixels, all other pixels treated as non-feature pixels
Different distance metrics possible, e.g. (3,4) Chamfer metric
Sample output:
Original image
Black CDT
White CDT
Red CDT
School of Information Systems, Computing and Mathematics, Brunel University, West London

7. Local regions and local dissimilarity
Local image dissimilarity within region rk
Average image dissimilarity over region set
Weighted average image dissimilarity over region set
School of Information Systems, Computing and Mathematics, Brunel University, West London

8. Discriminative region selection algorithm
Assuming a category of targeted object classes
and an unknown image , determine the class of by maximising posterior:
 indexing variable determining a set of regions to be used
 Vector of region relevance
In order to learn the best model parameters the following objective function is maximised:
School of Information Systems, Computing and Mathematics, Brunel University, West London

9. Discriminative region selection algorithm – cont.
comparison
Target class
Other classes
For each template being compared to the template :
Dissimilarity map …
STOP when
School of Information Systems, Computing and Mathematics, Brunel University, West London

10. Discriminative region selection algorithm – cont.
Determining weights by merging region ranks:
Sample output:
School of Information Systems, Computing and Mathematics, Brunel University, West London

11. Traffic sign recognition – system outline
School of Information Systems, Computing and Mathematics, Brunel University, West London

12. Traffic sign recognition – system outline
Image preprocessing
Intended to highlight characteristic colours and edges
Involves colour region clustering to determine RoI-s of certain size
Relevant colours enhanced within each RoI to extract colour edges
Detection
Loy & Barnes’s [2005] equiangular polygon detector
Colour edge images used on input
Tracking
Used only for search region reduction
Kalman filter with strong motion assumptions
Sign representation
Separate discriminative region model trained for each sign
Dissimilarity threshold individually tuned for each sign category
School of Information Systems, Computing and Mathematics, Brunel University, West London

13. Temporal classification
Single frame
Maximum Likelihood approach
Maximisation of likelihood equivalent to the minimisation of distance over i
Regions and weights denote these learned in the training stage
Video sequence
Integration of consecutive frame observations instead of individual classifications
Classifier’s decision at time t determined from:
Observation relevance , dependent on the candidate’s age (and thus size)
School of Information Systems, Computing and Mathematics, Brunel University, West London

14. Results Test data: td RC (55) BC (25) YT (42) BS (13) Overall (135)
Real-life video recorded from a moving car
88 clips, 144 signs
urban, countryside and motorway scenes td
RC (55)
BC (25)
YT (42)
BS (13)
Overall (135)
detected -
86.4%
100.0%
96.3%
94.4%
95.8%
recognised
0.9
90.6%
73.6%
91.2%
85.5%
0.7
88.7%
70.6%
87.7%
0.5
89.5%
96.9%
79.2%
58.3%
80.4%
best
93.5%
School of Information Systems, Computing and Mathematics, Brunel University, West London

15. Results
School of Information Systems, Computing and Mathematics, Brunel University, West London

16. Results
Classification of signs over time. Ratio of the cumulative distance from the best matching template (upper sign next to each chart) to the cumulative distance from the second best matching template (lower sign next to each chart) is marked with a solid red line. The same but temporally weighted cumulative distance ratio is marked with dashed lines: green (b = 0.8), and blue (b = 0.6).
School of Information Systems, Computing and Mathematics, Brunel University, West London

17. Conclusions Current solution Further work
Decent but not sufficient for real-life applications
Large gamut of signs recognised
Main contribution: road sign representation through discriminative local regions, CDT-based distance metric, region selection method
Further work
Detection – robustness to noise, occlusions, reduced parametrisation
Tracking – probabilistic temporal evolution of pixel/region ”interestingness”
Representation – capturing correlations between local regions,
alternative definitions of visual saliency
Classification – relaxing temporal independence assumption
School of Information Systems, Computing and Mathematics, Brunel University, West London

18. Thank you
School of Information Systems, Computing and Mathematics, Brunel University, West London