7
Robust Grid-Based Road Detection for ADAS and Autonomous Vehicles in Urban Environments Richard Matthaei, Bernd Lichte, Markus Maurer Institute for Control Engineering Technische Universit¨ at Braunschweig Braunschweig, Germany Email: <matthaei,lichte,maurer>@ifr.ing.tu-bs.de Abstract—For future advanced driver assistant systems a detailed knowledge about the road network in the immediate surroundings rises in importance for several reasons. It increases the robustness of scene interpretation especially in urban envi- ronments on the one hand, on the other hand it can be used for map-matching approaches to ensure a lane-accurate matching within an a priori map. In many inner-city side roads there are no lane markings available at all and curbs might be occluded by parking cars. Thus, the only information on the road course can be extracted from the stationary environment. In this paper we present an approach for road course detection in urban environments which is robust against a big variety of urban scenes. Our approach works with sensor data obtained from the road surface as well as from raised buildings. The measurements are obtained from a close-to-production laser-scanner and are accumulated in an occupancy grid. The algorithm runs online in real-time on a standard PC and has been evaluated in real urban environments. I. I NTRODUCTION The development of future advanced driver assistant sys- tems faces new challenges. On the one hand, the degree of automation increases. The degree of automation can be classified according to [1] from today’s assisted driving, to partially automated, highly automated and finally fully auto- mated systems. Fully automated systems were demonstrated during the DARPA challenges, by Google’s autonomous car [2] or by the research project Stadtpilot [3]. This evolution leads to an increasing demand on availability and robustness of the systems. On the other hand, the complexity of the field of application increases. First systems with environment per- ception based applications were introduced in series production for highway scenarios and rural roads. However, intersection assistant systems for urban environments as demonstrated in Intersafe2 1 or GENEVA 2 are still a field of research. Especially the scenes at urban intersections put high de- mands on contextual scene understanding. The constituent parts of such a scene representation are the road network with detailed lane information, the stationary environment, and the traffic participants [4] [5]. In the following, we concentrate on receiving information about the road network in urban scenarios. One source for information about the road network is the use of a priori data like navigation maps. This source is 1 http://cordis.europa.eu/projects/rcn/87267 en.html 2 http://www.geneva-fp7.eu used in several research projects and seems to be trivial to integrate at first sight, assuming that both a highly accurate position of a global navigation satellite system (GNSS) and a highly accurate map are available [6]. But in practice, we have to assume that neither the accuracy of the GNSS-based localization nor the accuracy of the map data will be satisfying in all situations. Especially in urban environments, forests, and tunnels, even the high-end GNSS-based localization methods fail due to missing satellite connections or multi-path effects [7] [8]. These circumstances lead to two possible ways of improv- ing the availability of a complete scene representation even at urban intersections: 1) The position within the map has to be refined using data from the environment perception. 2) The detection of the road network and road course has to be optimized to become independent from a priori maps and provide a sufficiently high detection range. A common approach for localization in a map is to use landmarks (e.g. [9] [10] [11] ). Levinson for example uses a high resolution infrared map of the ground plane for localiza- tion (see [2, p. 5 ff]) which can be interpreted as a dense landmark map. But the area-wide availability of up-to-date landmarks is not yet solved. Currently, a higher availability is given by the road network. Detected lane markings and curbs can be matched into the given maps. This approach was taken during the DARPA Urban Challenge by the Stanford Racing Team to correct the position estimates of the global positioning system (GPS) [2, p. 42 ff]. One possible way to improve the position in the map is to extract the current road course from sensor data and match the extracted road course to the a priori known road course. Thus, for both approaches (improving the position in the map as well as extracting the road course online) the road course has to be derived online from sensor data. The benefit of an online road course estimation is the robustness against outdated map data and GPS issues for the road course within the detection range. That is why we have been following the approach of modeling the environment as independent as possible of a priori map data using close to production sensors, even though the detection range is limited. Optionally, in the case of a successful matching into the map, the detection range can be easily extended by a priori map data in a posterior step. The road course can be estimated mainly on two feature groups: ground features (lane markings and curbs) and raised

Robust Grid-Based Road Detection for ADAS and Autonomous …isif.org/fusion/proceedings/fusion2013/html/pdf/Thursday... · 2014. 10. 2. · sensor data towards a graph-based representation

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Robust Grid-Based Road Detection for ADAS and Autonomous …isif.org/fusion/proceedings/fusion2013/html/pdf/Thursday... · 2014. 10. 2. · sensor data towards a graph-based representation

Robust Grid-Based Road Detection for ADAS andAutonomous Vehicles in Urban Environments

Richard Matthaei, Bernd Lichte, Markus MaurerInstitute for Control Engineering

Technische Universitat BraunschweigBraunschweig, Germany

Email: <matthaei,lichte,maurer>@ifr.ing.tu-bs.de

Abstract—For future advanced driver assistant systems adetailed knowledge about the road network in the immediatesurroundings rises in importance for several reasons. It increasesthe robustness of scene interpretation especially in urban envi-ronments on the one hand, on the other hand it can be used formap-matching approaches to ensure a lane-accurate matchingwithin an a priori map. In many inner-city side roads there areno lane markings available at all and curbs might be occludedby parking cars. Thus, the only information on the road coursecan be extracted from the stationary environment. In this paperwe present an approach for road course detection in urbanenvironments which is robust against a big variety of urbanscenes. Our approach works with sensor data obtained from theroad surface as well as from raised buildings. The measurementsare obtained from a close-to-production laser-scanner and areaccumulated in an occupancy grid. The algorithm runs online inreal-time on a standard PC and has been evaluated in real urbanenvironments.

I. INTRODUCTION

The development of future advanced driver assistant sys-tems faces new challenges. On the one hand, the degreeof automation increases. The degree of automation can beclassified according to [1] from today’s assisted driving, topartially automated, highly automated and finally fully auto-mated systems. Fully automated systems were demonstratedduring the DARPA challenges, by Google’s autonomous car[2] or by the research project Stadtpilot [3]. This evolutionleads to an increasing demand on availability and robustnessof the systems. On the other hand, the complexity of the fieldof application increases. First systems with environment per-ception based applications were introduced in series productionfor highway scenarios and rural roads. However, intersectionassistant systems for urban environments as demonstrated inIntersafe2 1 or GENEVA 2 are still a field of research.

Especially the scenes at urban intersections put high de-mands on contextual scene understanding. The constituentparts of such a scene representation are the road network withdetailed lane information, the stationary environment, and thetraffic participants [4] [5]. In the following, we concentrateon receiving information about the road network in urbanscenarios.

One source for information about the road network is theuse of a priori data like navigation maps. This source is

1http://cordis.europa.eu/projects/rcn/87267 en.html2http://www.geneva-fp7.eu

used in several research projects and seems to be trivial tointegrate at first sight, assuming that both a highly accurateposition of a global navigation satellite system (GNSS) anda highly accurate map are available [6]. But in practice, wehave to assume that neither the accuracy of the GNSS-basedlocalization nor the accuracy of the map data will be satisfyingin all situations. Especially in urban environments, forests, andtunnels, even the high-end GNSS-based localization methodsfail due to missing satellite connections or multi-path effects[7] [8].

These circumstances lead to two possible ways of improv-ing the availability of a complete scene representation even aturban intersections:

1) The position within the map has to be refined using datafrom the environment perception.

2) The detection of the road network and road course has tobe optimized to become independent from a priori mapsand provide a sufficiently high detection range.

A common approach for localization in a map is to uselandmarks (e.g. [9] [10] [11] ). Levinson for example uses ahigh resolution infrared map of the ground plane for localiza-tion (see [2, p. 5 ff]) which can be interpreted as a denselandmark map. But the area-wide availability of up-to-datelandmarks is not yet solved. Currently, a higher availability isgiven by the road network. Detected lane markings and curbscan be matched into the given maps. This approach was takenduring the DARPA Urban Challenge by the Stanford RacingTeam to correct the position estimates of the global positioningsystem (GPS) [2, p. 42 ff].

One possible way to improve the position in the map is toextract the current road course from sensor data and match theextracted road course to the a priori known road course. Thus,for both approaches (improving the position in the map aswell as extracting the road course online) the road course hasto be derived online from sensor data. The benefit of an onlineroad course estimation is the robustness against outdated mapdata and GPS issues for the road course within the detectionrange. That is why we have been following the approach ofmodeling the environment as independent as possible of apriori map data using close to production sensors, even thoughthe detection range is limited. Optionally, in the case of asuccessful matching into the map, the detection range can beeasily extended by a priori map data in a posterior step.

The road course can be estimated mainly on two featuregroups: ground features (lane markings and curbs) and raised

Page 2: Robust Grid-Based Road Detection for ADAS and Autonomous …isif.org/fusion/proceedings/fusion2013/html/pdf/Thursday... · 2014. 10. 2. · sensor data towards a graph-based representation

objects (buildings, walls, bushes, parking cars, light posts,etc.). The availability of the features depends on the current en-vironment (main street or small streets in residential quarters)and on the applied sensors (laser, camera, ultrasonic sensor, ...).The raised objects give only information on the less detailedroad course whereas the ground features additionally give moredetailed information about the lanes.

Our goal is to raise the abstraction level of the incomingsensor data towards a graph-based representation as knownfrom navigation maps or described by Knaup and Homeier [4].In this paper we describe a robust and flexible approach fora simultaneous road course and free space extraction in urbanscenarios which is independent from current motion of the hostvehicle and independent from a priori maps. The algorithmruns in real-time and has been evaluated in several inner-cityscenes with real sensor data derived from a close to productionlaser-scanner.

II. RELATED WORK

Extracting information about the road course is not a newchallenge. This extraction can be done on several abstractionlevels of the incoming sensor data: either on the raw data itself(e.g. laser reflections) or on accumulated data as available ina grid-based representation.

Working on raw data of a laser scanner is well known. It isdone by any object tracking. Depending on the manufacturerof the laser sensor it may already be provided by the sensor(e.g. Ibeo Automotive Systems GmbH provides an object box,a bounding box or an object outline as a polygon for trackedobstacles on their LUX laser scanner).

Bouzouraa presented in [12] an equidistant interval basedfree space model which is used to describe the free space inaddition to the obstacles. A free space hypothesis is generatedin a similar way like an object hypothesis. The dimension ofthe rectangular free space intervals are defined by the lasertargets of each laser scan. The orientation of all free spaceintervals is identical and it is directly linked to the currentorientation of the host vehicle. The lateral position of theintervals can be adjusted based on the predicted trajectory ofthe host vehicle derived from steering angle and yaw rate (seeFig. 6 in [12]).

A histogram-based approach extracting lane markings di-rectly from an ALASCA laser scanner is presented by Diet-mayer [13]. This algorithm is developed for highway scenariosand works in a range up to 30m. The main steps are ego motioncompensation of the incoming laser reflections, detecting lanesby a histogram (assumes the road to be straight) and applyinga linear regression to selected point groups.

A grid-based approach has several advantages in compari-son to an approach working with raw data:

• The number of false positive measurements is reduceddue to Bayesian update logic, so an extraction is morerobust.

• A history is provided so that it is possible to take previ-ously seen but currently occluded objects into account.

• The representation is independent of the current sensor.Thus, the input to the grid-based representation can easilybe exchanged, e.g. from laser to camera.

Weiherer and Bouzouraa presented an approach to effi-ciently represent the environment in front of the host vehicle[14]. The so called ”2D interval map” discretizes the en-vironment in longitudinal direction, whereas it continuouslydescribes the lateral position of obstacles and free spaces.This approach is optimized for assistant systems dealing withlongitudinal traffic.

A driving path detection from a grid-based representationbased on the current pose of the host vehicle is presented byWeiss [15]. It is developed for urban scenarios as well as highway scenarios. The main idea is to look straight ahead in thedirection of the host vehicle up to a certain distance or anoccupied cell and move so called sub lines from this center lineto the sides (see Fig. 8 in [15]). As soon as a sub line reachesoccupied cells, the center point of this sub line is calculated.It can easily be seen that this approach only works as long asthe host vehicle’s orientation is nearly in the direction of theroad and the road is wide or straight. The same approach isused by Konrad to detect the road course [16].

Duchow describes vision based approaches for lane mark-ing detection in urban scenarios [17] [18]. In addition to theintervals presented in [14], the segment’s orientation is adaptedto the orientation of the lane markings. These segments definethe search areas for further lane marking detection.

Our idea is to combine all these approaches to an extendedapproach for road way detection in urban environments work-ing for detection of lane markings as well as road boundaries.

III. MOTION CLASSIFICATION

A robust separation of stationary and movable targets isa basic requirement for a robust road course estimation. Westudied this challenge already in former work [19] [20]. Inthis paper, we will just give a short abstract and present ourprogress and enhancements on this topic.

Due to false detections of the sensor, Bayesian update isused to figure out whether a cell is occupied or not [21]. Thisapproach yields pretty good results in stationary environments.Nevertheless, the timing behavior of a cell updated with Bayesrule does not satisfy the requirements of a predictable motiondetection [22]. Furthermore, classifying incoming data beforeupdating the grid map needs a loop back from the grid itselfand if desired from an object tracking as well. A loop-backfrom the tracking module or occupancy grid leads to anadditional information due to affirmed model assumptions overtime and allows an instantaneous decision of incoming sensordata on the one hand, on the other hand it bears the risk ofself-affirming false-detections.

With our new approach presented in [19] [20] we pursue apredictable dynamic detection (point of view timing behaviorof a cell) which runs in an open-loop mode (see Fig. 1) toavoid self-affirming false-detections. Therefor, we studied theorder of incoming measurements for each cell using a completehistory to figure out which symptoms are indicating motionand which are indicating stationary areas. Obviously, thedistinguishing feature for motion detection is not the differenceto the sum of all incoming measurements (as assumed bythe detection of inconsistencies to the Bayesian update), butit depends on the order of incoming sequences of occupied

Page 3: Robust Grid-Based Road Detection for ADAS and Autonomous …isif.org/fusion/proceedings/fusion2013/html/pdf/Thursday... · 2014. 10. 2. · sensor data towards a graph-based representation

or free updates. The definition of the minimum length of asequence of updates is done in our approach by the relevantcharacteristics of the dynamic environment and the sensorupdate period.

sensor motion

classification

occupancy grid

tracking (pot.) movable targets

stationary targets consistency

analysis

1.

2.

optional feed-back

Fig. 1. Data flow for consistency based motion classification in open-loopmode. Optionally a feed-back from the object tracking can be used to supportthe classification.

In our first implementation of this approach presented in[20] we stored an entire history of the incoming measurementsin each cell in addition to the state of the state machine. Thisapproach helped us determining the indicators for motion at acertain place, but it is expensive in terms of memory usage. Inour reimplementation we have simplified the cell’s state to amodified hit-miss-counter, the state of the state machine, andsome flags. The modification of the hit-miss-counter covers asaturation of both counters as well as an algorithm which resetsthe counters in such a way that the same results from our firstimplementation are achieved. This also includes the handlingof detected false-negative updates as hit updates. With thisstep we use the model assumption about the dynamics of thevehicle’s environment to speed up the detection of stationaryareas. Without this assumption the detection of stationary areaswould be quite slow due to false-negative measurements.

The urban environment provides a wide range of dynamicsand dimensions: dimensions from 0.5 m (pedestrian) to 18 m(truck) or even more (tram) and dynamics from 0 km/h up to70 km/h have to be considered. To make sure the environmentperception is able to detect even the worst case scenarios, wehave to assume the longest vehicles at lower speed to defineour parameters. In [20] we could show that with respect tothe systematic detection errors of a laser sensor the resultingspeed of the host vehicle is strongly limited. To weaken thisconstraint we can follow several approaches:

1) Neglecting slow objects when the host vehicle drives athigher speed:As in our reimplementation of the approach from [20]we only have a modified hit-miss-counter to look forincoming hit or miss sequences, it is quite simple toadapt the cell’s state to the current velocity of the hostvehicle. To come to an adapted interpretation of the hit-miss-counter the thresholds and saturation of the countersare manipulated online depending on the host vehicle’svelocity. This yields a different result of the subsequentstate machine, introduced in [20] and illustrated in Fig 2.A criterion to manipulate the saturation of the hit-countercould be the constraint that within 10 m of ego motion thestationary buildings have to be converged to ”stationary”.So the saturation is directly linked to the velocity of thehost vehicle.

no hits hits only

unknown

significant inconsistency

non-significant inconsistency

hit-counter saturated miss-counter saturated

hit detected

hit detected && miss-counter saturated

miss-counter saturated

hit-counter saturated

Fig. 2. The state-machine defines the states of the grid cells. With the aidof this state machine it is possible to take into account the complete historyof the cell. The state transitions are defined by the sequences of incomingmeasurements (saturations of the hit-/miss-counter). The state ”significantinconsistency” had been introduced for research reasons - we figured out thatit is not constructive for distance measuring sensors.

2) Introduce model assumptions for moving objects:Fitting boxes or L-shapes into clustered laser targetsallows to distinguish between the long and the short sideof an object. Typically, in both cases of real movable andreal stationary objects, the long side of an L-shape hastargets classified as stationary and movable. In the case ofa real stationary object, this effect is caused by systematicfalse negatives of the laser sensor. In the case of a realmovable object at lower speed this effect is caused by thetime of convergence of the grid cells. The distinguishingfeature can be found at the short side of an L-shape.Real movable objects for both, longitudinal and crosstraffic, have only movable (or unknown in the case ofan object driving ahead) targets at the short side whereasreal stationary objects have mainly stationary targets onthat faced side. This approach helps in some cases butmakes the detection also worse in other cases. Hence,this approach can be used to optimize the detection forspecial cases.

3) We can use other sensors like radar to get a measuredrelative velocity directly.

The benefit of our approach in comparison to a Bayesianapproach is the exact knowledge about timing and detectionlatency. However, it may be possible that in some cases thisapproach needs more time to come to a decision.

IV. ROADWAY DETECTION

A detailed knowledge about the course of the road isan essential requirement of some advanced driver assistantsystems. The challenge is to detect the road course online fromsensor data and not to use any a priori data. As mentioned inthe introduction, we have two main sources for road courseinformation: we derive more detailed information from groundinformation (markings and curbs) and less detailed informationfrom raised obstacles. The focus of our new approach lieson the extraction of the less detailed view - the road course.However, this less detailed description can be supported bythe detection of curbs and lane markings even without beingclassified as such.

The algorithm is designed to work in urban environmentsas presented below in the chapter ”Experimental Results”. As

Page 4: Robust Grid-Based Road Detection for ADAS and Autonomous …isif.org/fusion/proceedings/fusion2013/html/pdf/Thursday... · 2014. 10. 2. · sensor data towards a graph-based representation

the urban environment is known for its unstructured manner,we argue that it also works on rural roads and highways as longas there are any structures in parallel to the road. Typically, dueto a lot of occlusion by traffic participants, parking cars etc,long straight lines are not continuously available in inner-citystreets. We have to deal with jumping fronts and interruptionsin the stationary scene from the sensor’s point of view. Theextraction should be independent from motion and steeringangle of the host vehicle so that we have no feedback inautonomous mode in case of false detections.

Beside getting closer to a complete environment model byextracting the road course, we pursue the goal of supportingthe orientation of the host vehicle in a digital map. So we haveto look for an approach which continuously provides enoughfeatures in an urban environment.

Therefor, inspired by the approaches introduced in thechapter ”Related Work”, we have combined the main char-acteristics of these approaches and added some new details.Most of the introduced approaches use a discretization inlongitudinal direction like the 2D-interval map from Weihererand Bouzouraa [14]. So we use it as well and call each interval”road element”. For determination of occupied and free spaceswithin a road element, histograms as used by Dietmayer yieldrobust results. Additionally, we allow the road elements toadjust themselves based on the extracted data. Road elementscan be corrected laterally and rotationally, similar to Duchow[18].

AOIj+1

) ,y,(xRE jjjj

AOIj

) ˆ,y,x(RE 1j1j1j1j

) ,y,(xRE jjjj

AOIj

) ,y,(xRE 1j1j1j1j

prediction step

correction step

d

Fig. 3. Illustration of the predictor-corrector approach for road coursedetection. REj is road element with the number j, the position (x, y), theorientation Ψ and the length d. The symbols x, y, and Ψ are the predictedvalues.

The general idea is based on a predictor-corrector approachin a reference frame fixed in place. The very first road elementis initialized at the host vehicle’s position with the hostvehicle’s orientation, which is the only dependency from thehost vehicle. For all other road elements the prediction step iscalled. Based on the previous road element REj , its position(xj , yj), orientation Ψj and if available an estimated curvatureκ of the road, the center point (xj+1, yj+1) of the followingroad element REj+1, and its orientation Ψj+1 are predicteddepending on the configured length d of the road elements (seealso Fig. 3):

Ψj+1 = Ψj + κ · dxj+1 = xj + d cos(Ψj)

yj+1 = yj + d sin(Ψj)

Fig. 4. This figure visualizes the histograms for each road element. Red barsto the top symbolize occupied spaces, green bars to the bottom symbolizefree spaces. The extracted peaks are shown as small red triangles. The blackmarkers are the reference points of the road elements.

The correction step is more complex. First, we look foroccupied and free sections for each predicted road elementusing histograms (see Fig. 4). All peaks of the occupancyhistogram are extracted and stored to the road element. In thenext step, we look for associations to the peaks of the previousroad element. This is done by analysis of the overlap of thepeaks and a gating defined by the maximum desired curvature.Based on successful associations, a change in orientation andlateral position of the road element depending on the length dof the road elements can be estimated by a Kalman filter. Basedon this result, position and orientation of the road element arecorrected in the last step (see Fig. 5).

This approach tries to catch all relevant information aboutthe general direction of the road. Each link between two roadelements is used to estimate the orientation of the current roadelement, no matter what kind of structure is found (parkingcars, rows of houses, walls or bushes, lane markings or curbs,...). Due to a strict limitation (user defined parameter formaximum curvature) of possible associations and the fact, thatthe roads’ courses are continuous, it is robust against failuresin the association process. The current course of the road isupdated at each correction step of each road element by thedirection of the association between two road elements. Hence,the road course is directly derived from the course of thebuilding lines and can follow even stronger curves.

Notice, this approach runs without any a priori informationand is - once initialized - independent from the motion of thehost vehicle. However, it can be supported by reference linesderived from a digital map or from the history of the hostvehicle’s positions.

In a further step, the detected associations are used toestimate an overall curvature of the road. This is done in asimple way by calculating the mean orientation in the firstand in the second half of the road which are ahead of thehost vehicle. The difference of the orientation at both positionsdivided by their distance result in the curvature. As mentioned

Page 5: Robust Grid-Based Road Detection for ADAS and Autonomous …isif.org/fusion/proceedings/fusion2013/html/pdf/Thursday... · 2014. 10. 2. · sensor data towards a graph-based representation

Fig. 5. Left: Extracted road elements. Green markers symbolize the centerpoint of estimated road elements by Kalman filter. Black markers symbolizepreviously predicted road elements by ego trajectory or curvature. The redlines show the peak associations. Right: The grey polylines are the convertedassociations. In green the simultaneously extracted free spaces are drawn.

above, this curvature can be used to support the prediction stepas well as the correction step proportionately depending on theestimated variance.

V. EXPERIMENTAL RESULTS

The described approaches have been evaluated with realsensor data in urban environments. We use a laser scannerwith 4 scan lines and 145◦ horizontal field of view. One scanline is used to detect the road surface. The sensor is mountedat the front of the vehicle at a height of about 30cm. The testvehicle is equipped with wheel increments as well. Thus, amotion estimation is provided.

Fig. 6. Result of the grid based representation for ground targets. Shownis an inner-city intersection, in white the extracted lane features and in darkmagenta the calculated curvature based on the ground layer extractions.

The ground reflections are accumulated in a grid-basedrepresentation (ground layer) just adding the hit information(no free update) (see Fig. 6). The non-ground laser targets are

classified as described above into stationary, potentially mov-able and unknown targets. All targets (including the groundtargets) lead to a free update up to the target’s distance inthe Bayes occupancy grid (occupancy layer), but only thestationary and unknown targets lead to a hit update.

Both layers of the grid-based representation are abstractedto the three states occupied, free and unknown by user-definedthresholds. This abstracted representation is the input for thedescribed road extraction.

Fig. 7. Detection of moving objects to eliminate them from the occupancygrid. Light blue targets indicate inconsistencies. Thus, these measurementsseem to belong to movable objects.

As shown in Fig. 7, movable objects are eliminated as faras possible without object tracking. The road course extractionruns separately for the ground and the occupancy layer. Forboth sources the curvature is calculated and fused in a posteriorstep. The result of the fused curvature is illustrated in Fig. 8.

Fig. 8. Result of the road extraction and curvature fusion. Grey lines:extraction from occupancy layer. White lines: extraction from ground layer.Light magenta: curvature from occupancy layer. Dark magenta: curvature fromground layer. Cyan: fused curvature result.

The required processing time on an i5 CPU is shown intable I. The two branches and the two extraction steps run inseparated threads.

The data flow is shown in Fig. 9. The data is split atthe very beginning into two branches - one for the groundreflections and one for all other reflections.

For evaluating the road extraction the result of the esti-mated road curvature is compared to the motion of the host

Page 6: Robust Grid-Based Road Detection for ADAS and Autonomous …isif.org/fusion/proceedings/fusion2013/html/pdf/Thursday... · 2014. 10. 2. · sensor data towards a graph-based representation

sensor motion

classification occupancy

grid

stationary targets

consistency analysis

1.

2. ground classification

inverse sensor model bayes

inverse sensor model consistency

occupancy grid

inverse sensor model bayes

(ground)

convert to 3-state

road extraction

curvature estimation

convert to 3-state

road extraction

ground targets

other targets

Fig. 9. The entire data flow from the sensor to the estimation of the curvature.

Working step Processing timeper layer

Extraction 4msConversion from roadelement to polyline < 0.4msEstimation of curvature < 0.2ms

TABLE I. TIMING PERFORMANCE OF THE ROAD EXTRACTION.

vehicle and to the curvature derived from an a priori map. Aslong as the vehicle follows exactly the course of the road,the yaw rate of the vehicle can be used to determine thecurvature of the road. Another possibility is to use a priorimap data. The roads and lanes are represented by polylinesin the map. We obtained the curvature by smoothing andinterpolating the changes in course angle at the support pointsof the corresponding polyline. Fig. 10 shows that the approachis able to follow curves. In this case, the curvature derived fromthe ego motion follows the curvature of the road as well. Wecan also show that the road extraction runs independently ofthe steering angle and, even in cases of lane changes, followsthe original direction of the road (see Fig. 11). Simultaneously,we extract the free space along the reference line (see Fig. 5).

Fig. 10. Red: Curvature estimation based on lane-markings and stationaryenvironment. Blue: Reference from ego motion (right turn). Green: Curvaturefrom the a priori map. It can be seen that the curvature correlates with theyaw rate of the host vehicle as well as the curvature obtained from the a priorimap. The curvature is given in 1/m.

Another way of evaluating the performance of the road

Fig. 11. Red: Curvature estimation based on lane-markings and stationaryenvironment. Blue: Curvature from ego motion (lane change). Green: Curva-ture from the a priori map. It can be seen that the curvature estimation isindependent from ego motion and follows the original direction of the road.The curvature is given in 1/m.

course extraction is to look at the orientations of the extractedline segments. A line segment is the link between the asso-ciated peaks of two road elements. A line segment consistsof two support points and is visualized in gray or white inthe figures above. As mentioned in the introduction, these linesegments can be used to correct the host vehicle’s orientationin a map. Therefor, the orientation of the line segments iscompared to the orientation of the corresponding road coursein a map. It is necessary to know the noise characteristics ofthis matching process to apply the correct filtering mechanismlater on. For this purpose we accumulated all matching resultsbetween the perceived line segments and the road course fromthe map in a histogram. To get the correct orientation in themap a post-processed high-end DGPS-INS-platform is used.The results of this matching process are shown in Fig. 12.

As expected, the result shows a significant peak near 0◦.Also comprehensible is the lower variance of the line segmentsderived from the lane markings (about 10◦) in comparison tothe line segments derived from raised buildings (about 15◦),because the lane markings are better structured than thebuildings. It is noticeable that there are some additionalperiodic peaks at about 6◦ and 11◦. The reason seems to bea discretization issue: We obtained this data with a cell size

Page 7: Robust Grid-Based Road Detection for ADAS and Autonomous …isif.org/fusion/proceedings/fusion2013/html/pdf/Thursday... · 2014. 10. 2. · sensor data towards a graph-based representation

Fig. 12. Separated histograms of the orientation error from numerous linesegments to an a priori map (left: lane markings, right: buildings). X-Axis:Error in degree, Y-Axis: Number of matches. In both histograms there is asignificant peak near 0◦. The variance of the buildings-based errors is a littlehigher than the lane-marking-based errors.

of the grid and width of the histogram bars of 0.2 m anda length of the road elements of 2 m. The resulting angulardiscretization error is arctan(0.2/2) = 5.7◦. This matches thedetected positions of the peaks. We could change these peakschanging the grid resolution and length of the road elements.However, a final analysis of the reason for these additionalpeaks has to be done in future work.

VI. CONCLUSION AND FUTURE WORK

We have shown that it is possible to extract even unstan-dardized road courses in urban scenarios from occupancy grids.Therefor, we augmented existing approaches and applied thealgorithms with real sensor data. The presented results area good initial point for extracting more complex scenes likeintersections. First attempts have shown that it seems to bepossible to extract intersections based on a comparison fromthe extracted free spaces and the occupied background. Due tothe fusion of lane markings and raised buildings this approachyields a high availability and is robust against a big variety ofdifferent environments.

In future work, we plan to use the extracted data for local-ization in digital navigation maps. Due to the robust estimationof the road course, a matching based on the road course to thenavigation map can be done. This yields a higher availabilityand a higher accuracy of the host vehicle’s orientation inthe map. A highly accurate heading is essential for matchingdetected objects in a priori data at higher distances, accordingto the approach of Knaup and Homeier [5] [4].

REFERENCES

[1] T. M. Gasser, Rechtsfolgen zunehmender Fahrzeugautomatisierung, ser.Berichte der Bundesanstalt fur Straßenwesen. Bergisch Gladbach:Wirtschaftsverlag NW, 2012, no. F83.

[2] J. Levinson, “Automatic laser calibration, mapping, and localization forautonomous vehicles,” Ph.D. dissertation, Stanford University, 2011.

[3] J. M. Wille, F. Saust, and M. Maurer, “Stadtpilot: Driving autonomouslyon Braunschweig’s inner ring road,” in Intelligent Vehicles Symposium(IV), San Diego, CA, 2010 IEEE, 2010, p. 506–511.

[4] J. Knaup and K. Homeier, “RoadGraph - graph based environmentalmodelling and function independent situation analysis for driver assis-tance systems,” in 13th International IEEE Conference on IntelligentTransportation Systems (ITSC), Madeira, Portugal, 2010, Sept. 2010,pp. 428 –432.

[5] K. Homeier and L. Wolf, “RoadGraph: high level sensor data fusionbetween objects and street network,” in Intelligent TransportationSystems (ITSC), 2011 14th International IEEE Conference on, Oct.2011, pp. 1380 –1385.

[6] U. Haak, A. Sasse, and P. Hecker, “On the definition of lane accuracy forvehicle positioning systems,” in Proceedings of the 7th IFAC Symposiumon Intelligent Autonomous Vehicles IAV 2010, Lecce, Italy, Sept. 2010.

[7] T. Luettel, M. Himmelsbach, F. Hundelshausen, M. Manz, A. Mueller,and H. J. Wuensche, “Autonomous offroad navigation under poorGPS conditions,” Planning, Perception and Navigation for IntelligentVehicles (PPNIV), p. 56, 2009.

[8] C. Reinholtz, T. Alberi, David Anderson, Andrew Bacha, CherylBauman, Stephen Cacciola, Patrick Currier, Aaron Dalton, Jesse Farmer,Ruel Faruque, Michael Fleming, Scott Frash, Grant Gothing, JesseHurdus, Shawn Kimmel, Chris Sharkey, Andrew Taylor, Chris Terwelp,David Van Covern, Mike Webster, and Al Wicks, “DARPA urbanchallenge technical paper of team VictorTango,” Apr. 2007.

[9] U. Scheunert, H. Cramer, and G. Wanielik, “Precise vehicle localizationusing multiple sensors and natural landmarks,” in Proceedings of theSeventh International Conference on Information Fusion, Stockholm,Sweden, 2004, p. 649–656.

[10] H. Kloeden, D. Schwarz, E. M. Biebl, and R. H. Rasshofer, “Vehi-cle localization using cooperative RF-based landmarks,” in IntelligentVehicles Symposium (IV), 2011 IEEE, 2011, p. 387–392.

[11] C. Brenner, “Vehicle localization using landmarks obtained by a lidarmobile mapping system,” Photogrammetric computer vision, vol. 38, p.139–144, 2010.

[12] M. E. Bouzouraa, “Modellbasierte Freiraumverfolgung fur Fahrerassis-tenzsysteme,” in 8. Workshop Fahrerassistenzsysteme, Walting, 2012.

[13] K. Dietmayer, N. Kampchen, K. Furstenberg, and J. Kibbel, “Roadwaydetection and lane detection using multilayer laserscanner,” in Proceed-ings of Advanced Microsystems for Automotive Applications 2005 ,Berlin, Germany, 2005.

[14] T. Weiherer, E. Bouzouraa, and U. Hofmann, “A generic map basedenvironment representation for driver assistance systems applied todetect convoy tracks,” in 15th International IEEE Conference on In-telligent Transportation Systems (ITSC), 2012, Anchorage, USA, 2012,p. 691–696.

[15] T. Weiss, B. Schiele, and K. Dietmayer, “Robust driving path detectionin urban and highway scenarios using a laser scanner and online occu-pancy grids,” in Intelligent Vehicles Symposium, 2007 IEEE, Istanbul,Turkey, 2007, p. 184–189.

[16] M. Konrad, M. Szczot, and K. Dietmayer, “Road course estimation inoccupancy grids,” in Intelligent Vehicles Symposium (IV), 2010 IEEE,San Diego, CA, 2010, p. 412–417.

[17] C. Duchow and M. Friedl, “Fahrspurerkennung in der Innenstadt mitprobabilistischen und evidenztheoretischen Modellen (Lane Detectionin Urban Environments Using Models of Probability and EvidenceTheory),” tm-Technisches Messen, vol. 75, no. 7-8/2008, p. 464–471,2008.

[18] C. Duchow, Videobasierte Wahrnehmung markierter Kreuzungen mitlokalem Markierungstest und Bayes’scher Modellierung, ser. Schriften-reihe. KIT Scientific Publ., 2010.

[19] R. Matthaei, H. Dyckmanns, B. Lichte, and M. Maurer, “Motion clas-sification for cross traffic in urban environments using laser and radar,”in Proceedings of the 14th International Conference on InformationFusion (FUSION), 2011, Chicago, July 2011, pp. 1 –8.

[20] R. Matthaei, H. Dyckmanns, M. Maurer, and B. Lichte, “Consistency-based motion classification for laser sensors dealing with cross trafficin urban environments,” in Intelligent Vehicles Symposium (IV), 2011IEEE, Baden-Baden, Germany, June 2011, pp. 1136 –1141.

[21] S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics (IntelligentRobotics and Autonomous Agents series), ser. Intelligent robotics andautonomous agents. The MIT Press, Aug. 2005.

[22] J. Canas and V. Matellan, “Dynamic gridmaps: comparing buildingtechniques,” Mathware & soft computing, vol. 13, no. 1, p. 5–22, 2008.