1. Perceptual Hysteresis Thresholding: Towards Driver Visibility Descriptors Nicolas Hautière, Jean-philippe Tarel, Roland Brémond Laboratoire Central des Ponts et Chaussées, Paris, France

2. Presentation overview 1. Introduction 2. Angular resolution of a camera 3. Human vision system modeling 4. Discrete Cosine Transform 5. Design of a visibility criteria 6. Perceptual hysteresis thresholding 7. Towards road visibility descriptors 8. Conclusion

3. Introduction  The major part of all the information used in driving is visual. Reduced visibility thus leads to accidents.  Reductions in visibility may have a variety of causes (geometry, obstacles, adverse weather/lighting conditions).  Different proposals exist in the literature to mitigate the dangerousness of each of these situations using in-vehicle cameras.  One objective is to inform the driver if he is driving too quickly according to the measured visibility conditions.  Detecting the visible edges in the image is a critical step to assess the driver visibility.  We propose such a technique based on the Contrast Sensitivity Function (CSF) of the Human Visual System (HVS).

4. Angular resolution of a camera ♦ Let’s express the angular resolution of a camera in cpd. ♦ With the notations of Fig. 1, the length d for a visual field = 1° is expressed by: ♦ To have the maximum angular resolution of the camera in cpd, we divide d by the size of two pixels (black and white alternation) of the CCD array: Cycle per degree (cpd): This unit is used to measure how well details of an object can be seen separately without being blurry. This is the number of lines that can be distinguished in a degree of a visual field. Fig. 1

5. ♦ Our ability to discern low contrast patterns varies with the size of the pattern, i.e. its spatial frequency f (cpd). ♦The CTF is a measure of the minimum contrast needed for an object (a sinusoidal grating) to become visible. ♦This CTF is defined as 1/CSF, where CSF is a Contrast Sensitivity Function (see Fig. 2). In this paper, we use Mannos CSF, plotted in Fig. 3 and expressed by: Human vision system modeling Fig. 2 Fig. 3 0 Spatial Frequency [cpd] 45 1 0 Contraste

6. Discrete Cosine Transform  A={a ij } a block of the original image  B={b ij } the corresponding block in the transformed image where c 0 =1/sqrt(2), c i =1 for i=1...n-1.  The maximum frequency of the DCT is obtained for the maximum resolution of the sensor, i.e. r * cpd :  To express the b ij in cpd, we use the following scale factor obtained by computing the ratio between (1) and (4):

7. Design of a visibility criteria DCT vs CTF  We can now plot the DCT coefficients with respect to the CTF curve: Fig. 5: plot of the DCT in the marked blocks with respect to the CTF Fig. 4: Curves of the CSF ( __ ) and of the CTF (---) for the sensor used to grab the images t pix =8.3μm, f =8.5mm.

8. Design of a visibility criteria Visibility Level Definition  Visibility can be related to the contrast C, defined by:  For suprathreshold contrasts, the Visibility Level (VL) of a target can be quantified by the ratio:  As L b is the same for both conditions, then this equation reduces to:  ΔL threshold depends on many parameters and can be estimated using Adrian’s empirical target visibility model (Adrian, 1989).

9. Design of a visibility criteria Visibility Level for Periodic Targets  We propose a new definition of the VL, denoted VL p, valid for periodic targets, i.e. sinusoidal gratings.  We first consider the ratio r ij between a DCT coefficient of the block and the corresponding coefficient of the CTF:  Based on the CSF definition CSF, r ij ≥1 means that the block contains visible edges.  To define VL p, we choose the greatest r ij : Fig. 6: Map of VL p ≥1

10. Perceptual hysteresis thresholding: Edges Detection by Segmentation  The proposed approach may be used with different edge detectors (Canny- Deriche, zero-crossing approach, Sobel)  We propose an alternative method which consists in finding the border F which maximizes the contrast C(s 0 ) between two parts of a block, without adding a threshold on this contrast value. The edges are the pixels on this border:  This approach is based on Köhler’s binarization method and is detailed in [16]. [16] N. Hautière, D. Aubert, and M. Jourlin. Measurement of local contrast in images, application to the measurement of visibility distance through use of an onboard camera. Traitement du Signal, 23(2):145–158, Septembre 2006.

11. Perceptual hysteresis thresholding: Hysteresis Thresholding on the VL p  In the usual hysteresis thresholding, a high threshold and a low threshold of gradient magnitude are set.  We propose to replace these thresholds by thresholds on the VL p (cf. Fig. 7)  Thus, the algorithm is as following: 1.All possible edges are extracted, 2.The edges are selected thanks to its VL p value using low t L and high t H thresholds. Fig. 7: Principle behind thresholding by hysteresis:

12. Perceptual hysteresis thresholding: Results Samples t L =1 ; t H =10 t L =1 ; t H =20 No noisy features are detected whatever are the lighting conditions whereas thresholds are fixed.  The method is thus clearly adaptive.

13. Perceptual hysteresis thresholding: Contrast Detection Threshold of the Human Eye  The value of t L is easy to choose, because it can be related to the HVS.  Setting t L =1 should be appropriate for most applications.  The hysteresis thresholding has now only one parameter !  The value of t H depends on the application.  For lighting engineering, the CIE published some guidelines to set the VL according to the visual task complexity.  VL=7 is a adequate value for night-time driving task.  We can set t L =7 as a starting point. However, a psychophysical validation is necessary.

14. Towards road visibility descriptors  Once visible edges have been extracted, they can be used in the context of an onboard camera to derive driver visibility descriptors: visibility distance estimation…  There are three steps to complete and validate the algorithm from a psychophysical point of view:  An extension to color images may be necessary,  The CSF is valid for a given adaptation level of the HVS. It is interesting to automatically select the properly CSF.  To compare our results with the set of edges which are manually extracted by different people.

15. Conclusion  We present a visible edges selector and use it for in- vehicle applications.  It proposes an alternative to the traditional hysteresis filtering.  We propose to replace the thresholds on the gradient magnitude by visibility levels.  The low threshold can be fixed at 1 in general.  Some guidelines to set the high threshold are proposed.  This algorithm may be used to develop sophisticated driver visibility descriptors.  Thereafter, it can be fused with other visibility descriptors to develop driving assistance systems which takes into account all the visibility conditions.

16. Thank you for your attention ! This work is partly founded by the French ANR project DIVAS (2007-2010) dealing with vehicle-infrastructure cooperative systems.