Obstacle detection using v-disparity image Based on: Global Correlation Based Ground Plane Estimation Using V-Disparity Image, By: J. Zhao, J. Katupitiya and J. Ward, 2007 Obstacle Detection with Stereo Vision for Off-Road Vehicle Navigation, By: A. Broggi, C. Caraffi, R. I. Fedriga, P. Grisleri, 2005 Real Time Obstacle Detection in Stereovision on Non Flat Road Geometry Through ”V-disparity” Representation, By: R. Labayrade, D. Aubert, J. Tarel, 2002 Presented by: Ali Agha March 25, 2009

Motivation Obstacle avoidance using stereo vision Points with Z > 0 The pitch angle between the cameras and the road surface will change. Therefore, we need to compute the pitch angle and disparity of ground pixels dynamically due to static and dynamic factors thus the disparity of ground pixels is changing from time to time.

Related work Disparity map  3D point cloud  Ground plane extraction using plane fitting  obstacle detection Yu et al (2005) Disparity map  Optical flow and ground information  obstacle detection Mascarenhas (2008) Giachetti et al (1998) Disparity map  V-disparity image and ground information  obstacle detection Labayrade (2002) Broggi (2005) Zhao (2007)

V-Disparity image Disparity map IΔ has been computed IvΔ is built by accumulating the pixels of same disparity in IΔ along the v axis u d Recently, V-disparity image has become popular for ground plane estimation [1][9][10][11]. In this image, the abscissa axis (w) plots the offset for which the correlation has been computed; the ordinates axis (v) plots the image row number; the intensity value is settled proportional to the measured correlation, or the number of pixels having the corresponding disparity (w) in a certain row (v). Each planar surface in the field of view is mapped in a segment in the V-disparity image [10]. Vertical surfaces in the 3D world are mapped into vertical line segments, while ground plane in the 3D world are mapped into slanted line segment. This line segment, called ground correlation line in this study, contains the information about the cameras pitch angle at the time of acquisition (mixed with the terrain slope information). v v

The image of a plane in v-disparity image pin-hole camera model Ground correlation line 5

Ideal and real road and obstacle Recently, V-disparity image has become popular for ground plane estimation [1][9][10][11]. In this image, the abscissa axis (w) plots the offset for which the correlation has been computed; the ordinates axis (v) plots the image row number; the intensity value is settled proportional to the measured correlation, or the number of pixels having the corresponding disparity (w) in a certain row (v). Each planar surface in the field of view is mapped in a segment in the V-disparity image [10]. Vertical surfaces in the 3D world are mapped into vertical line segments, while ground plane in the 3D world are mapped into slanted line segment. This line segment, called ground correlation line in this study, contains the information about the cameras pitch angle at the time of acquisition (mixed with the terrain slope information).

Hough Transform Labayrade uses Hough transform to extract the ground correlation line

free space on the road Once the longitudinal profile of the road has been extracted, the disparity values on the road surface are known for each scanning line (it is exactly the disparity value corresponding to the longitudinal profile curve extracted for the scanning line in question). Let Δi denote this value. Thus, it is straightforward to extract the obstacles from the disparity map IΔ already computed: for a given scanning line, pixels whose disparity is equal to Δi are located on the road surface; other pixels belong to potential obstacles. On Fig. 4, note that the vehicle located at almost 80 meters is well perceived as a potential obstacle. Once the obstacle areas have been computed, the free space on the road surface can be extracted (see also [6][7]), using a growing area algorithm (see Fig. 6 Down). The part of the image corresponding to the free space area can be used, for example, for extracting the lane-markings in more suitable conditions.

Robust ground correlation extraction Broggi et al. experimentally, found that the ground correlation line during a pitch variation oscillates, parallel to itself. Zhao et al. investigated this characteristic mathematically and gave the condition for this characteristic to be valid. The behavior of the ground correlation line during a pitch variation is to oscillate, parallel to itself. Experimentally, we found out that the cameras height variation due to oscillation has neglectable effects. Accumulating the Vdisparity image values along each of the candidate lines, it is possible to estimate the ground correlation line (choosing the line that accumulated the greatest value), and then the pitch of the cameras at the time of acquisition.

Condition derived by Zhao et al For the ground correlation lines to be parallel with each other at different pitch angles At different pitch angles shown in Fig. 3, the slope of ground plane in V-disparity image will be different. Suppose gs is the slope of ls using static calibration data when = s, gmax is the slope of lmax when = max, and gmin is the slope of lmin when = min as shown in Fig. 4. Thus for a given 4w and certain pitch oscillation, we need to reduce s. In [10], the tilt angle is 8.5. In [1], the tilt angle is 7.73. Θs= is 7.73.

Result of this condition If this condition is satisfied.

GLOBAL CORRELATION METHOD Exploiting this characteristic  Fast and Robust method (even in lack of distinct features) By accumulating the matching cost (intensity of V-disparity) along each of the candidate lines and choose the one with least (most) accumulated matching cost as ground correlation line.

Verifying equations and assumptions (Road has distinct features) Implementing the Labayrade’s work Disparity map V-D Hough

Verifying equations and assumptions (Road has distinct features) In the sequence of images the position of ground correlation lines is ranging from 25 to 28. The slop(g) of ground correlation lines ranges from 5.5 to 5.5652. After Hough transform, we can get the position(4w) and slope(g) of the ground correlation lines for these image pairs. The result is shown in Fig. 9. The lower plot of this figure shows the position(4w) of ground correlation lines for different image pairs, which is ranging from 25 to 28. This plot tells us there do exist pitch variation in different conditions. The upper plot shows the slop(g) of ground correlation lines, which ranges from 5.5 to 5.5652. From this Figure we can see that the bigger the slope(g) is, the greater 4w. This confirms the match between Eq. 4 and Fig. 4 in section II. Also, the upper plot shows that the change of slope is very small, the difference between the greatest slope and smallest slope is merely 0.0652, which is 1.17%. Thus the assumption that the ground correlation lines under different pitch angles are parallel, is valid.

Results Applying the method on the same images Calculate the matching cost (associated with each candidate ground correlation lines) matching costs associated with candidate lines Same as Hough transform The matching cost of each candidate line is accumulated from the bottom of the image to the level Of horizon GCL

Without distinct features Disparity map V-D Hough

Without distinct features - Comparison Hough transform and global correlation method GCL Hough

Obstacle detection

Conclusion This accuracy is dependant of the image quality and whether or not the ground pixel dominate this area of the image. It seems an appropriate method for detecting obstacles such as vehicles in road in night using structured light (as the headlight of automobile)

Thank you Questions??