1. Towards Night Fog Detection through use of In-Vehicle Multipurpose Cameras Romain Gallen Aurélien Cord Nicolas Hautière Didier Aubert

2. 2 /11 Atmospheric Characterization with in-vehicle Multipurpose Cameras Triple goal –Driving assistance systems (lighting, wipers) –Fiabilize other ADAS based on cameras –Functionality used as input in intelligent speed adaptation systems Previous works on : –Rain detection –Day fog detection Almost nothing about Night fog A. Cord and D. Aubert, Towards Rain Detection through Use of In-Vehicle Multipurpose Cameras, IV’2011 (Poster session on Wednesday) N. Hautière, J.-P. Tarel, J. Lavenant, and D. Aubert, Automatic fog detection and estimation of visibility distance through use of an onboard camera, Machine Vision and Applications, vol. 17, no. 1, pp. 8–20, 2006. [Cord11] [Hautiere06]

3. Two distinct phenomenons that are not observable at the same time (due to the classical camera settings) : 3 /11 Embedded Night Fog Detection Multiple light sources in the environment Road is lit by car own lighting system R. Gallen, A. Cord, N. Hautière et D. Aubert, Procédé et dispositif de détection de brouillard, la nuit, Brevet Français n°1057802, Sept. 2010.

4. 4 /11 Previous works [Leleve06][Kawasaki08] Perceptible back scattered veil J. Leleve, A. Bensrhair,, and J. Rebut, Method for detecting night fog and system implementing said method, Patent EP1 715 456, 10-2006. N. Kawasaki, T. Miyahara, and Y. Tamatsu, Visibility condition determining device for vehicle, Patent 20 080 007 429, 01-2008. Back Scattered Veil Detection (1/5)

5. 5 /11 Experiments conducted in a fog chamber at the LRPC, Clermont-Ferrand, France 30 m deep, 2.7 m high Monitored Fog Back Scattered Veil Detection (2/5) Software simulation Semi Monte Carlo ray tracing Photorealistic scenes PROF-LCPC Software

6. Correlation between real images and synthetic images 6 /11 Synthetic imageImage of in-vehicle camera Correlation mask Back scattered veil detection (3/5) ExperimentsSoftware simulation

7. Meteorological Visibility 7 /11 Back Scattered Veil Detection (4/5) Zero mean Normalized Sum of Squared Differences Correlation

8. 7 /11 Back Scattered Veil Detection (5/5) Raw imageMean image Conclusion : Detection and characterization of night fog. Simple, adaptable, real time. Possibility to use a mean image How to manage in presence of light sources in the environment ?

9. 8 /11 Detection of halos around light sources (1/3) Hypothesis : –halos are present around light sources –intensity decrease as the distance from the source increases –Sources unknown –Automatic camera settings

10. 8 /11 Algorithm for halo detection around light sources Features analysed : - Surface - Gravity center - Compacity (surface/perimeter) - Elongation Detection of halos around light sources (1/3)

11. 8 /11 Regions that appear at intermediate thresholds are not added to the tree Detection of halos around light sources (1/3)

12. 8 /11 Detection of halos around light sources (1/3)

13. 8 /11 Selection is made according to size, shape and lenght of branch criterions Detection of halos around light sources (1/3)

14. 8 /11 Selection of interesting light sources Detection of halos around light sources (1/3)

15. 9 /11 The slope of the profile is relevant regarding the presence of fog Extraction of the intensity profile of light sources Detection of halos around light sources (2/3)

16. Detection based on a single frame ~ 98% of good detection results 10 /11 Detection of halos around light sources (3/3) Fiabilized by a detection based on consecutive frames

17. 11 /11 Conclusion Method allowing for fog detection with a dual algorithm with standard camera using automatic exposure settings Real time implemented and tested algorithms Preserves the usual working state of other camera based ADAS May be combined in order to improve Driver/car orientated ADAS (adaptive lighting systems, future vision enhancement systems in fog) Safety orientated ADAS (Preemptive driver information, Intelligent Speed Adaptation in fog) System orientated ADAS (detection of working state, improvement of camera based ADAS through image restoration)