Upload
duongquynh
View
217
Download
0
Embed Size (px)
Citation preview
Rear Lights Vehicle Detection for Collision Avoidance
Evangelos Skodras1, George Siogkas
2, Evangelos Dermatas
2 and Nikolaos Fakotakis
1
1 Artificial Intelligence Group,
2 Pattern Recognition Group
Wire Communications Laboratory, Department of Electrical and Computer Engineering
University of Patras, Patras, Greece {evskodras, fakotaki}@upatras.gr, {siogkas, dermatas}@wcl2.ee.upatras.gr
Abstract— Vehicle detection based on on-board mounted
cameras is an integral component of many driver assistance
systems aiming at alerting the driver about impending
collisions. In this paper an automated algorithm for detection
of preceding vehicles is proposed, based on the detection of
rear vehicle lights. Unlike many systems which make use of
static threshold boundaries for the red color segmentation of
rear lights, our method combines color and radial symmetry
cues while the threshold is dynamically adapted. The extracted
candidate rear lights are morphologically paired in order to
define possible areas where vehicles are present. The
verification of vehicle presence is then carried out through
axial symmetry check. Experimental results that demonstrate
the system’s high detection rates and robustness even in
adverse illumination and weather conditions are finally
presented.
Index Terms— Vehicle detection, rear lights detection, driver
assistance, collision avoidance, computer vision.
I. INTRODUCTION
Over the last decades, many efforts are directed towards
enhancing driving safety, following the dire statistics of
vehicle crashes in terms of expenses and human casualties.
Although vehicle safety improvement has significantly
reduced the death toll in vehicle crashes, accident prediction
and prevention would be the ultimate solution for
maximizing driving safety [1]. On these grounds, pre-crash
sensing has become an area of active research among
automotive manufacturers and universities. The
development of a vehicle-mounted driver assistance system
aiming at alerting the driver about an impending collision
requires reliable detection of preceding vehicles. Up to now,
existing state-of-the-art systems rely on active sensors for
this challenging task [2]. However, the exponential growth
in processing power and the availability of advanced
computer vision techniques render a lot of low-cost methods
for vehicle detection feasible.
Most techniques employed for vehicle detection are using
visual features, motion or appearance. An extensive review
of the most commonly used techniques can be found in [3].
Rear lights have been widely used as a cue for vehicle
detection, especially at night conditions, where other
features are impossible to detect. The most common
approaches use the RGB color space or its components in
order to identify rear vehicle lights. In [4] rear lights are
recognized based on grayscale information and a red
component resulting from the normalized difference of the
R and B channels. In [5] a “red level” of every pixel is
computed in small regions, based on a proportion between
RGB channels, whereas in [6] only the red channel is
utilized. Brake lights are detected in [7] using thresholds in
all RGB channels. However, the RGB color space is not
ideal for color thresholding, as channels are highly
correlated with each other, making it difficult to define and
manipulate color parameters. To overcome this difficulty,
other color spaces have been chosen for this task. Rear
lights are segmented using the YCbCr color space in [8],
using a subjectively chosen threshold in the Cr channel.
O‟Malley et al. in [9] defined thresholds in HSV color space
components derived from the color distribution of rear-lamp
pixels under real-world conditions. Finally, in [10] the
L*a*b* color space is used to detect rear lights, using two
specific thresholds for a* and b* channels.
In this paper we present a vehicle detection algorithm
which can be integrated in driver assistance systems for
forward collision avoidance. It is based upon a combination
of multiple cues present on vehicles, such as the red color of
rear lights, horizontal edges and symmetry. Many existing
systems utilize static thresholds for red color segmentation
of vehicle lights, thus being prone to illumination changes
and environmental conditions. The main feature of our
method lies in the absence of such static thresholds for the
detection of rear lights, making our system resilient to any
changes in illumination conditions, time of day, camera
settings and sensor characteristics. Moreover, the use of
multiple cues is a viable means for improving the reliability
of our system.
The paper is organized as follows. In Section 2 the
proposed algorithm is presented and Section 3 discusses its
performance. Finally, in Section 4, conclusions and future
work are outlined.
II. PROPOSED VEHICLE DETECTION SYSTEM
The proposed vehicle detection system is developed upon
knowledge-based methods and uses some of the vehicle‟s
most prominent features. At first, we extract a binary image
containing candidate rear vehicle lights using color
segmentation and radial symmetry.
Rear lights represent a conspicuous cue for vehicle
detection. Apart from being a common feature among all
vehicles according to legislation, they are also visible under
different illumination, weather conditions and time of day.
Moreover, they can be used to give an advanced warning of
a potential danger, as illuminated rear brake lights indicate
that a vehicle is beginning to slow down.
The second stage of our system involves morphological
pairing of candidate lights and horizontal edge detection, in
order to define possible vehicle areas. Subsequently, a
symmetry check along the vertical bisector is utilized for
vehicle presence verification. For the successfully detected
vehicles, an estimation of their distance is performed. The
architecture of the proposed system is presented in Figure 1.
A. Red Light Detection
In the first processing stage of our system, we seek the
areas with high red chromaticity in the color image. Among
various color spaces, L*a*b* was the most suitable for our
application, as it possesses a number of important features.
L*a*b* color space has the advantage of being a
perceptually uniform color space, mapping equally the
perceived color difference into qualitative distance in the
color space. In L*a*b*, luminance information (L
component) is separated from the chrominance information
(a*, b* components), which we utilize for the red color
segmentation. As a result, illumination changes have a
minimal effect on color information.
For the detection of candidate rear light areas, we utilize the
a* component (red - green) of L*a*b*, and split the positive
from the negative part, in order to acquire only the red
subspace. This subspace image is then scanned for
symmetrical shapes, using the fast radial symmetry
transform presented in [11]. Although there is no constraint
in the shape of rear lights, they generally follow a
symmetrical pattern. A judicious choice of a low radial-
strictness parameter (a=1) gives emphasis to non-radially
symmetric features [11], thus presenting great values at the
positions of rear lights (Figure 2).
The symmetry detection scans for shapes in one or more
ranges N. Normally, in order to blindly detect shapes of any
size, a large size of ranges must be used; however, the
computations are greatly accelerated by choosing a small
sparse set of ranges N, spanning between the extreme sizes
of possible rear lights. The result constitutes a very good
approximation to the output obtained if all the possible
ranges were examined. Using the fast radial transform
approach, the “blooming effect”, caused by the saturation of
bright pixels in CCD cameras with low dynamic range, is
very effectively handled. This is attributed to the fact that
saturated lights appear as bright spots with a red halo
around, thus yielding large radial symmetry values. This
phenomenon is illustrated in Figure 3.
For the final binarization of the image, we utilize the fast
and efficient Otsu‟s thresholding algorithm, which suggests
minimizing the weighted sum of variances of the objects‟
and background pixels to set an optimum threshold,
especially in the case of bimodal images. The resulting
binary image contains the candidate rear vehicle lights.
B. Morphological Lights Pairing
In this stage, a morphological rear lights pairing scheme
is applied to the binary image to determine vehicle
candidates. After connected component labeling, we
compute for each region an ellipse with similar second
moments as the region, in order to calculate its features. The
parameters of the ellipse, i.e., the center coordinates, the
major and minor axis lengths as well as the area are
computed. In order to find pairs of possible lights we
consider all the possible 𝑁!
2!∙ 𝑁−2 ! two-combinations.
However, from all these potential pairs only a few meet the
prerequisites that we impose, regarding the angle between
them and a similarity measure based on their geometrical
properties: Assuming that the target vehicle is in the same
tilt as the observing vehicle, the candidate pair of lights
must be aligned in the horizontal axis (with a permissible
Fig. 1. Overview of the proposed vehicle detection system.
Fig. 3. (a) Original image, (b) pseudo-colored red subspace of the L*a*b* color space where the “blooming effect” is visible and (c)
pseudo-colored fast radial symmetry transform of the red subspace.
Fig. 2. (a) Original image, (b) pseudo-colored red subspace of L*a*b* and
(c) pseudo-colored fast radial symmetry transform of the red subspace.
(c) (b) (a)
(a) (b) (c)
inclination of ±5 degrees). The morphological similarity
measure is based on the normalized difference of their major
axis length, minor axis length and area.
Confining the maximum and minimum allowable
distance of the candidate lights can further narrow down the
number of possible pairs. Even though this can be proven
very useful for speeding up the calculations, we omit it for
purposes of generality.
C. Horizontal Edge Boundaries
Given the candidate rear light pairs, we seek the
horizontal boundaries of the candidate vehicle (the vertical
boundaries are defined by the extreme points of the rear
lights). First, we determine a search region for the upper and
lower horizontal boundaries that is proportional to the width
of the vehicle, which is assigned as the distance between the
extreme points of the rear lights. Figure 4a illustrates a
search region on the original image. The „Canny‟ edge
detector is used to detect the edges in the grayscale image of
the search region (Fig. 4b). The horizontal projection of the
edge map is then computed (Fig. 4c), while the peak values
indicate pronounced horizontal edges. The upper and lower
boundaries of the car are defined as the first and last peak in
the projection graph, with value at least equal to the half of
the largest value. The outcome of this stage is bounding
boxes containing candidate vehicles.
D. Symmetry Check
As one of the main signatures of man-made objects,
symmetry represents a very interesting cue for vehicle
verification. Images of vehicles observed from the rear view
are in general symmetrical in the vertical direction [3]. The
symmetry check is performed by splitting each bounding
box image into two sub-images along the vertical bisector
and comparing them. The comparison of the sub-images is
carried out by utilizing two measures, namely the Mean
Absolute Error (MAE) and the Structural SIMilarity (SSIM)
measure [12]. MAE constitutes a straightforward and
efficient measure, formulated as follows:
𝑀𝐴𝐸 =1
𝑀∙𝑁 𝑥𝑖𝑗 − 𝑦𝑖𝑗
𝑁𝑗=1
𝑀𝑖=1
where M, N are the dimensions of the sub-images x and y
that we compare. The SSIM measure, originally used as an
image quality measure, can be effectively applied in our
system. It searches for similarity using three comparisons,
regarding luminance, contrast and structure. More details
about SSIM measure can be found in [12].
For the verification of vehicle presence, the result from
both measures must lie below thresholds, defined
heuristically, through extensive experimental tests.
E. Distance Estimation
Once the preceding vehicle is successfully detected, the
relative distance is calculated. A precise calculation of the
distance is not feasible, as a single frame cannot contain
enough information. However, a sufficient approximation
for typical sized cars can be achieved: Assuming an average
vehicle width of ~1.7m and given the width of the target
vehicle in the image (as a proportion of the vehicle‟s width
in pixels to the image‟s width in pixels) we are able to
estimate the desired distance. If the camera characteristics
are well known in advance, a more precise estimation can be
computed as in [6].
III. EXPERIMENTAL RESULTS
The performance of the proposed algorithm was tested in
two publicly available databases containing images of cars
from the rear [13] and a publicly available video sequence
of driving in an urban environment [14]. These test sets
contain images of many different cars, under various
illumination conditions, shot with different cameras. We
should clarify that in Caltech DB (Cars 2001) from the 526
images, 22 images were excluded as their red rear lights
were modified beyond legislation [9], or one of the brake
lights was blown. For the video sequence of [14] only
frames that contain a whole visible, preceding vehicle at the
same lane and in distance less than 15m were considered
(2716 out of 11179 frames). Red vehicles, recognized as
large regions in the binary image, were also detected using
the same method. The recognition results are summarized in
Table 1.
Our system scores high detection rates in all test sets (up
to 93.6%), and performs outstandingly in cases when the
preceding vehicle is braking, as can be observed from Table
1. This can be attributed to the intensive, highly
distinguishable color of illuminated brake lights and the
ability of our system to handle the “blooming effect” very
effectively. This specific feature is of key importance, as
accurate recognition at the stage when the preceding vehicle
Fig. 4. (a) Search region on the original image, (b) edge map of the
search region, (c) its horizontal projection and (d) bounding box
containing the candidate vehicle
(1)
TABLE I
DETECTION RATES
Database
Number of
images or
frames
Detection Rate
Detection
Rate when
Braking
Caltech DB
(Cars 1999) 126 92.1% -
Caltech DB
(Cars 2001) 504 93.6% 99.2%
Lara Urban
Sequence 1 2716 92.6% 96.3%
(a) (b) (c) (d)
is braking is very crucial for avoiding an impending
collision. A fruitful comparison can be made with the
system of [15], reporting results at the same databases
(Caltech DBs). Our approach performs better (93.3% versus
92%) on both databases, with the additional advantage of
requiring no training. Some representative detection results
from all data sets used are illustrated in Figure 5. Our
system was also tested on images acquired under adverse
weather conditions, downloaded from the internet. For these
images, although we cannot obtain quantitative results, we
can observe that our system performs sufficiently well,
yielding promising results (Figure 6).
Regarding the cases where vehicles are falsely recognized
(for example, there exist 7 and 46 false positives (FP) for the
Caltech 1999 and 2001, databases respectively), these can
be eliminated in various manners. The FP rate can be
significantly reduced if we impose certain restrictions in the
admissible distance between rear lights, by narrowing down
the region of interest or, most importantly, by taking
advantage of the temporal continuity of the data, as FPs are
not persistent in time (most FPs rarely appear in more than
one frame).
Examining the failed cases of our algorithm, the most
common cause is the presence of other red artifacts near rear
lights, recognized as a unity.
IV. CONCLUSIONS
The development of a robust and reliable vision-based
vehicle detection method is a crucial task for driver
assistance systems. In this paper we have presented an
automatic, resilient to illumination conditions algorithm for
vehicle detection. It makes use of color and radial symmetry
information for the segmentation of rear vehicle lights. After
morphological lights pairing and edge boundaries detection,
symmetry check is performed in the candidate bounding
boxes, in order to verify vehicle presence. Experimental
results report high detection rates even in challenging cases.
The proposed algorithm can be easily extended for vehicle
detection at night, because of its approach of using rear
lights for detection. Future efforts are directed towards
vehicle tracking and combining vehicle detection (and
braking recognition) with driver‟s gaze detection. In this
way, the level of attention of the driver can be correlated
with the potential danger of an impending collision.
REFERENCES
[1] W. Jones, “Keeping Cars from Crashing”, IEEE Spectrum, vol. 38, no
9, pp. 40-45, Sep 2001.
[2] R. Stevenson, “A Driver‟s Sixth Sense”, IEEE Spectrum, vol. 48, no
10, pp. 50-55, Oct 2011.
[3] Z. Sun, G. Bebis, and R. Miller, “On-road Vehicle Detection: A
Review”, IEEE Transactions on Pattern Analysis and Machine
Intelligence, Vol. 28, No. 5, pp. 694–711, May 2006.
[4] Y.-C. Lin, C.-C. Lin, L.-T. Chen and C.-K. Chen, “Adaptive IPM-
Based Lane Filtering for Night Forward Vehicle Detection”, IEEE Conference on Industrial Electronics and Applications (ICIEA),
Beijing, China, pp.1568-1573, June 2011.
[5] L. Gao, C. Li, T. Fang, and Z. Xiong, “Vehicle Detection Based on
Color and Edge Information”, Image Analysis and Recognition, vol.
5112, Lecture Notes in Computer Science , pp. 142–150, 2008.
[6] M. Betke, E. Haritaglu, and L. Davis, “Real-Time Multiple Vehicle
Detection and Tracking from a Moving Vehicle”, Machine Vision and Applications, vol. 12, no. 2, Feb 2000.
[7] P. Thammakaroon and P. Tangamchit, “Predictive Brake Warning at
Night using Taillight Characteristic”, IEEE International Symposium on Industrial Electronics (ISIE), Seoul, Korea, pp. 217–221, Jul 2009.
[8] S. Nagumo, H. Hasegawa and N. Okamoto, “Extraction of Forward Vehicles by Front-mounted Camera Using Brightness Information”,
IEEE Canadian Conference on Electrical and Computer Engineering,
vol. 2, pp. 1243–1246, May 2003.
[9] R. O‟Malley, E. Jones, and M. Glavin, “Rear-Lamp Vehicle
Detection and Tracking in Low-Exposure Color Video for Night Conditions”, IEEE Transactions on Intelligent Transportation
Systems, pp. 453–456, June 2010.
[10] I. Cabani, G. Toulminet, and A. Bensrhair, “Color-based Detection of
Vehicle Lights”, IEEE Intelligent Vehicles Symosium, pp. 278–283,
June 2005.
[11] G. Loy and A. Zelinsky, “Fast Radial Symmetry for Detecting Points
of Interest”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 8, pp. 959-973, Aug 2003.
[12] Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity”,
IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612,
April 2004.
[13] Caltech Image Database, http://www.vision.caltech.edu/html-
files/archive.html [14] Lara Urban Sequence 1, Robotics Centre of Mines ParisTech, 2010,
http://www.lara.prd.fr/benchmarks/trafficlightsrecognition [15] C.-C. Wang and J.-J. Lien, “Automatic Vehicle Detection Using
Local Features–A Statistical Approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 9, no. 1, pp. 83-96, Mar 2008.
Fig. 6. Detected vehicles in adverse weather conditions
Fig. 5. Detection results for the data sets used, (a) Caltech DB (cars
1999), (b) Caltech DB (cars 2001) and (c) Lara Urban Sequence
(a) (b) (c)