- 1 -
中国科技论文在线
Real-time Road Detection with Image Texture
Analysis-based Vanishing Point Estimation#
ZU Zhaozi, XUE Jianru, CUI Dixiao**
(Electronic and Information Engineer School, Xi'an Jiaotong University, Xi'an 710049) 5
Foundations: Specialized Research Fund for the Doctoral Program of Higher Education of China under Grant
(NO. 20090201110029).
Brief author introduction:ZU Zhaozi, (1987-),Male, Postgraduate Student, Image Processing.
Abstract: Visual perception is a key component of an autonomous vehicle system. Road detection is
an essential function of a visual navigation system for an autonomous vehicle. In most cases detecting
lane markings is one feasible and efficient approach to road detection. However, in some situation, for
example, many rural roads have no lane markings. Furthermore, lacking of sharp, smooth edges and
homogeneous surface brings difficulties to road detection. A texture analysis based vanishing point 10
estimation method is used to detect road in this paper. The idea is that consistency in texture leads to
the vanishing point in an image. The method uses the Sobel filter to compute the texture orientations.
Then the vanishing point of the road is voted from the orientation of the texture. Finally, the vanishing
point is used to constrain the searching of two road boundaries since they pass through the vanishing
point. 15
Keywords: Texture orientation; Sobel filter; boundary detection; vanishing point detection
0 Introduction
Auto-driving car is significantly developed in recent years, and the DARPA Grand Challenge
and the Google Auto-driving car have attracted the people’s attention. In China, there is a similar 20
competition like the DARPA Grand Challenge, the Future Challenge Auto-driving Competition.
As we all know, the driving area[1] is very important for a car, but in some situation, the roads
is without the lane marking. Especially, in many rural roads, the road is lack of sharp, smooth
edges and homogeneous surface[2], which brings difficulties for road detection. However, for most
of the roads, they often have strong textures that direct to the road’s vanishing point. This paper 25
describes an algorithm for detecting the ill-structured roads without lane marking. Our method
uses the Sobel filter[3] to compute the texture orientations[4]. The vanishing point of the road is
voted from the orientation of the texture[5,6,7,8]. Finally, the vanishing point is used to constrain the
searching for the road boundaries by the pixels texture of the lines passing through the vanishing
point. 30
The outline of this paper is as follows. In the next section, the Sobel filter that we’ve used
will be explained, and the pixels’ orientations are calculated. In the section 3, the vanishing point
is voted. And the section 4 presents the detection of the two boundaries of the road. Section 5
gives the results and the section 6 concludes this research.
1 Texture orientation calculating 35
For Sobel filter, the operator uses two 3 3× kernels which are convolved with the original
image to calculate the two derivations—one for horizontal change, and one for vertical. I denotes
the source image, and xG and yG are the results of the convolution containing the horizontal and
vertical derivative of each pixel. The computations are as follows:
- 2 -
中国科技论文在线
1 0 1
2 0 2 *
1 0 1
xG I
− +⎡ ⎤⎢ ⎥= − +⎢ ⎥⎢ ⎥− +⎣ ⎦
,
1 2 1
0 0 0 *
1 2 1
yG I
− − −⎡ ⎤⎢ ⎥= ⎢ ⎥⎢ ⎥+ + +⎣ ⎦
(1) 40
The x-coordinate is here defined as increasing in the “right” direction, and the y-coordinate is
defined as increasing in the “down” direction. At each pixel z in the image, the resulting gradient
or the orientation and the gradient magnitude can be calculated as follows:
Magnitude G(z):
2 2( ) ( ) ( )x yG z G z G z= + (2) 45
Orientation ( )zθ :
1
( )
( ) tan ( )
( )
y
x
G z
z
G z
θ −= (3)
In the magnitude image, there are some pixels’ magnitude are smaller than the others’, that
means textures there are very weak that they are not concerned with the vanishing point voting. In
the experiments, we set the magnitude G(z) smaller than *average (magnitude ) as 0, and 50
there texture orientation as 0.
0 ( ) * ( ( ))
( )
( )
if G z average G z
G z
G z other else
⎧ <= ⎨⎩
(4)
0 ( ) * ( ( ))
( )
( )
if G z average G z
z
z other else
θ θ
⎧ <= ⎨⎩
(5)
The images of the magnitude and orientation as figure 1.
55
Figure1. First row: three source road images. Second row: magnitude images with setting the magnitude zero
following the formula (4). The third row: the texture orientation in at pixels from 0o to 180o .
2 Vanishing Point Voting
After having obtained the images of texture orientation and magnitude at each pixel, we start 60
to vote the vanishing point by using the two factors.
Exactly, for a pixel z, its texture orientation is ( )zθ ; the candidate vanishing point O, note
the angle of the line Oz as ( )Oz∠ and the angle between the ( )zθ and the line Oz
- 3 -
中国科技论文在线
is ( , )Oz z∠ . If the ( , )Oz z∠ is smaller than the thresholdγ , vote for the candidate vanishing
point O. And the vanishing point is the one mostly voted. Here we got a problem, as in the figure 2, 65
for a candidate vanishing point 1O and 2O , the voting zone for 1O is larger than that of 2O , that
means that the voting rule is not equal.
Figure2: The voting zones for different candidate vanishing point
70
To deal with this problem, we propose a new voting rule that the vanishing point is with the
largest radio between the total number of the pixels with texture orientation similar to the line
Oz voting for the candidate vanishing point O and the measure of the voting zone S.
( ( ( , ) ))( )
( )
Num z Oz zRatio O
S zone of voting
γ∠ <= (6)
The vanishing point vO is with ( ( ))Max Ratio O . 75
In the experiment, we search the vanishing point between the zone from 10% height to 90%
height, and the threshold of angle is10° .
In the figure3， we can see the images of voting result.
Figure3：The result of the voting image. 80
Then, we got the vanishing point vO .
3 Vanishing Point Voting
After having obtained the vanishing point in section 3，we start to detect the two boundaries
in this section. The correct estimation of the vanishing point in the previous section gives us prior 85
- 4 -
中国科技论文在线
information for the boundaries detection. As noted in the introduction, the two boundaries
intersect at the vanishing point, so we start to search the most dominant line of all the lines
crossing through the vanishing point as the first boundary. We propose a method similar to the
formula (6).
Detection of the first boundary 90
Here, we search the boundaries with the orientation from 15o to 165o considering that the
boundaries of the road are not horizontal in front of the intelligent vehicle.
Figure4：The lines crossing through the vanishing point
95
Just like the show in the figure3，the line vO P is a line crossing through the vanishing
point vO , the orientation of the line ( )vO P∠ , the pixel z on the line vO P with the
orientation ( )zθ . The angle between the ( )zθ and vO P is ( , )vO P z∠ , if it is smaller than the
threshold ( 10 )hereγ o , the pixel z is of the same texture orientation with the line vO P . The first
boundary of the road is the line with the largest ratio between the number of pixels with the 100
similar texture orientations to the line’s orientation and the length of the line.
( ( , ) )( )
( )
v
v
v
Num O P zRatio O P
Length O P
γ∠ <= (7)
The first boundary is with ( ( ))vMax Ratio O P .
Figure5：The first boundary of the road 105
Detection of the second boundary
After having obtained the first boundary of the road, we start to search the second boundary
of the road and update the initial vanishing point by choosing from the pixels on the first
boundary. 110
First, the points on the first boundary of the road are treated as candidate-vanishing points.
- 5 -
中国科技论文在线
Secondly, we search the lines (beside the first boundary FirB) cross the first boundary on
each candidate-vanishing point like the method of detecting the first boundary.
Finally, among all the first boundaries of each candidate-vanishing point, the line with the
largest ratio is the second boundary. 115
Figure6：The pixel z on the first boundary FirB and the lines L crossing the pixel z
In the experiments, we search the second boundary L in the area where the angle difference is
larger than 30o between the first boundary FirB and the second boundary candidates. 120
Figure7: Detection of the second boundary based on the first boundary
Update the vanishing point 125
Once having detected the first and second boundary, we update the vanishing point. The
intersection point of the two boundaries is the new vanishing point.
Figure8: The update vanishing point and the two boundaries of road 130
4 Experimental results
Some of the images that we experiment with are acquired using an onboard camera based on
the Basler scout sca640-74gc. And some are downloaded from the baidu image (a Chinese search
- 6 -
中国科技论文在线
engine). 135
In the experiments, we treat the image at the size of 320╳240; and for the image sequence
acquired by our camera，detection of the vanishing point is limited to a window of 80╳60 pixels,
centered at the vanishing point of the previous frame, which allows a real time application.
5 Conclusion and future development
This algorithm allows a robust way to detect the vanishing point and the boundaries of the 140
road image. This algorithm is suitable for the road that are straight or with a small curvature, but
for the curve road, it can’t detect the vanishing point and the boundaries correctly, so in the future,
we try to detect the trend and the boundaries of the curve road[9].
Acknowledgements
This work was supported by Specialized Research Fund for the Doctoral Program of Higher 145
Education(SRFDP) ，NO. 20090201110029.
And we would like to thank Dr. ZHANG Geng for helping us in doing the experiments.
References
[1] Y. Sha, G. Zhang, and Y. Yang. A road detection algorithm by boosting using feature combination[C]. IEEE 150
Intelligent Vehicles Symp, 2007: 364-368.
[2] Y. He, H. Wang, and B. Zhang. Color-based road detection in urban traffic scenes[J]. Intelligent Transport
System, 2004: 309-318.
[3] Gabor Székely, Vittorio Ferrari, Luc Van Gool. Feature Detection[OL].[data].
155
[4] Chunzhao Guo. Drivable road region detection based on homography estimation with road appearance and
driving state models[C]. ICARA, 2009: 204-209.
[5] Ondrej Miksik. Rapid Vanishing Point Estimation for General Road Detection[C]. IEEE ICRA,2012: 4844-
4849.
[6] C. Rasmussen. Texture-based vanishing point voting for road shape estimation[C]. BMVC, 2004: 55-64 . 160
[7] Hui Kong. Vanishing point detection for road detection[C]. CVPR, 2009: 96-103.
[8] Moghadam, P. Fast Vanishing-Point Detection in Unstructured Environments[J]. IP Trans, 2012: 21-28.
[9] Wang, Y., Teoh, ., Shen[J] D. Lane detection and tracking using B-Snake[J]. Image Vision Computing,
2004: 269-280.
165
基于图像纹理分析的实时消失点估计及道路检测
祖肇梓，薛建儒，崔迪潇
（西安交通大学电子与信息工程学院，西安 710049）
摘要：视觉感知是智能车系统的重要组成部分。对于智能车研究，道路检测具有重要的视觉
导航功能。在大多数情况下，检测车道线是一个可行的和有效的道路检测方法。然而，在某170
些情况下，例如，对于道路与非道路区域边界不明显或道路路面特征不一致的道路，给道路
检测带来了困难。在本文中，基于消失点估计的方法来检测道路。基本方法是利用道路纹理
的一致性确定道路的消失点。然后利用获得的消失点来约束搜索道路的两条边界。
关键词：模式识别；机器视觉；道路检测；消失点检测；纹理特性
中图分类号：TP391 175
