1. Advanced Crosswalk Detection for the Bionic Eyeglass
Mihály Radványi
Balázs Varga
Kristóf Karacs
Pázmány Péter Catholic University
Berkeley, 2010.

2. Summary Bionic Camera/Eyeglass The task Difficulties
Algorithmic description
Results
Further plans

3. Bionic Camera/Eyeglass
Visual info audio MULTIMODAL
Three situations
home
banknote, color recognition, etc.
office
pictograms, displays, etc.
street
escalator’s direction
route number-, crosswalk detection,etc.
Autonomous crash avoidance

4. The task Crosswalk recognition / detection Based on road marks
dark – bright
alternating
parallel
Mobile device
navigation  fast decision
Low consumption ( ~1W)
High computing power (TOps/s)
visual microprocessor

5. Difficulties shadows light conditions traffic missing marks others
tricky
disturbing
light conditions
traffic
missing marks
others
categorize!

6. Different approaches Matlab simulation On-chip
Image Processing and MatCNN toolbox
2nd generation
Mean shift color segmentation
On-chip
Bi-i visual processor
128 x 128 pixel
Still image, video
10 fps (1st generation)

7. Manual design - flow Goal: clearest zebra lines Adaptive threshold
Supposed zebra lines
AND
Contrast mask
Color mask
CNN templates
RECALL
Input image
Mean shift

8. Asphalt detection - introduction
foreground-background segmentation
foreground (ROI): asphalt + stripes
mean shift segmentation
Similar to K-means
arbitrary number of clusters!
Iterative, nonparametric
YCbCr color space
Density gradient map with modes ( )
Input image
(Y – Cb – Cr)
Calculate
position of a mode*
(Iterative scheme on a
Probability Density Function
using a moving Gaussian window)
Put pix. with
similar color into
same class
Select i
data points No
Are all
clustered?
Pixels
Yes
Biggest class
results asphalt
*dense region of the feature space

9. Asphalt detection - segmentation
Steps of segmentation
Preprocessing
band-pass color filtering
Foreground-background segmentation
mean shift
Post processing
morphological operations
eliminates false positive/false negative pixels
Input
Result of preprocessing
Classes of mean shift
Result of post processing
Masked foreground

10. Manual design - again Goal: clearest zebra lines Adaptive threshold
Supposed zebra lines
AND
Contrast mask
Color mask
CNN templates
RECALL
Input image
Mean shift

11. Localization of huge patches
Manual design – flow No.2
Space-variant
LOGDIF
Localization of huge patches
EDGE
Decision
Scalar output
Supposed zebra lines A h
600
400
200 h’ A’
where

12. Results - simulation

13. Crosswalk non detected
Results
Enlarged dataset
89.2% performed well
Crosswalk detected
Crosswalk non detected
Crosswalk 33 7
No Crosswalk 2 41
False positive
(2.4%)

14. The prototype
Nokia N95 + Bi-i + WiFi
Eye-Ris

15. Further plans Tracking Online test with blind users(done…)
Correlations between frames
Online test with blind users(done…)
Background estimation through image fusion

16. Thanks for your attention!