INTELLIGENT AUTONOMOUS SYSTEM FOR CARS

K. Hari Kishore.

08MP1A0229

Abstract—Focusing the safety and uninterrupted

traffic management, this paper presents the

planning for construction and implementation of

intelligent autonomous system in cars. The

method includes implementation of the concepts

like AICC (Autonomous Intelligent Cruise

Control system) which is used to control the

speed of the car automatically, Lane support

system which audibly warns the driver of

unintended lane departure. The lane markings

are detected by processing images from a video

camera. The collision avoidance system predicts

accurately the likelihood of an imminent

collision. If a collision is likely then the system

responds in such a way as to reduce the threat

and the response may involve the automatic

control of the vehicle or simply an appropriate

warning to the driver.

INTRODUCTION

Ideally an intelligent autonomous car would

have an automatic pilot, which can park itself and

guide the vehicle through dense traffic in towns

and at high speeds between towns. At any time the

driver would have the ability to switch between

automatic and driver control. During periods of

manual control the system would act in an

advisory capacity warning the driver of hazards

and giving information about route guidance or

traffic congestion.

Such a vehicle would retain the convenience

and fun of private transport but would take the

V.Chaitanya Prasad, (M-tech).

Assoc.Prof. EEE Dept.

drudgery out of driving during automatic

operation. Such ambitious goals are being

seriously considered by vehicle manufactures.

Fully autonomous systems will not appear for

some time although they may be introduced

sooner if the traffic environment is greatly

constrained. More viable for medium term are

semi autonomous systems where driver maintains

responsibility for the overall control of the car.

Thus they will provide platform for providing the

technology for later fully autonomous systems.

This paper focuses on some of the semi

autonomous systems that are currently being

investigated in Europe, US and Japan which

include:

Autonomous intelligent cruise control

systems.

Lane support systems.

Collision avoidance systems.

AUTONOMOUS INTELLIGENT CRUISE

CONTROL SYSTEM

Traditional cruise control systems have been

in use for many years. They maintain a constant

vehicle speed, set by the driver, thereby improving

comfort in steady traffic conditions. In congested

traffic conditions when speeds vary widely these

systems are no longer effective. The use of cruise

control would be significantly increased if the

vehicle speed could automatically adapt to the

traffic flow.

Figure 1: Architecture of Intelligent

Autonomous Cruise control system

The Autonomous Intelligent Cruise Control

(ACC) systems add $1500 to $3000 to the cost of

a car, use laser beams or radar to measure the

distance from the vehicle they are in to the car

ahead and its speed relative to theirs. If a car

crosses into the lane ahead, say, and the distance

is now less than the preset minimum (typically a

1- or 2-second interval of separation), the system

applies the brakes, slowing the car with a

maximum deceleration of 3.5 m/s2 until it is

following at the desired distance. If the leading car

speeds up or moves out of the lane, the system

opens the throttle until the trailing car has returned

to the cruise control speed set by the driver.

The addition of a radar sensor to the front of

a vehicle would provide the necessary range and

velocity information for this task. Automatic

control of brakes and throttle would allow the

longitudinal controller to maintain a constant time

interval behind the vehicle in front. Such systems

are commonly referred to as Autonomous

Intelligent Cruise Control Systems (AICC) and

aim to improve driver comfort and conveniences.

It is important to remember that these systems are

quite different from the fully autonomous car. The

driver is always responsible for driving and must

deal with emergency situations. The system is

only capable for fine longitudinal control and not

emergency braking. The system is further

enhanced by allowing operation down to zero

speed which would increase comfort during traffic

jams.

Fundamental to any AICC system is a

sensor that can reliably detect obstacles in the

traffic environment in a variety of conditions.

Microwave radar is a method for detecting the

position and velocity of a distant object. A beam

of electromagnetic radiation, with a wavelength

between 30cm and 1mm is transmitted and

reflected back to the transmitter by the object.

Velocity and Range can be derived by measuring

the Doppler frequency shift and time of flight of

the transmission. A major advantage of

microwave radar is that the performance is not

affected by the time of the day, and therefore no

driver adaptation is required for night- time

driving. The performance advantages of radar

over other sensors are enhanced during poor

weather conditions. Systems that rely on visible

light are known to suffer significantly in the very

conditions for which they are relied upon the

most. Experience of microwave radar operation

has shown that reliable results can be obtained,

even in inclement weather conditions.

The operation of microwave radar falls

broadly into two categories: Pulse and Frequency

Modulated Wave (FMCV). Pulse systems rely on

measuring the time of flight of a pulse that is

proportional to range.

Information from the sensors goes to the

Vehicle Application Controller (VAC), the

system's computing and communication center.

The VAC reads the settings the driver has selected

and figures out such things as how fast the car

should go to maintain the proper distance from

cars ahead and when the car should release the

throttle or downshift to slow down. Then it

communicates that information to devices that

control the engine and the transmission.

APPLICATIONS IN CARS

In May 1998, Toyota became the first to

introduce an AICC system on a production vehicle

when it unveiled a laser-based system for its

Progres compact luxury sedan, which it sold in

Japan. Then Nissan followed suit with a radar-

based system, in the company's Cima 41LV-2, a

luxury sedan also sold only in Japan. In

September 1999, Jaguar began offering an ACC

for its XKR coupes and convertibles sold in

Germany and Britain. Mercedes' system is an

option on its C-Class and S-Class models, which

are available in Europe; it was developed by M/A-

Com, Lowell, Mass., and uses radar made by

Filtran Microcircuits Inc., in West Caldwell, N.J.

LANE SUPPORT SYSTEM

A recent US study shows that 25% of all

accidents are caused by unintended lane

departures. Mounting evidence suggests that

drivers falling asleep, although it is difficult to

prove, as many drivers will not admit to falling

asleep, talking to a cell phone, disruption of

children etc. cause many of these accidents.

To address this, Valeo a European maker of

automotive switches and sensors and Iteris U.S.

developer of intelligent transportation system

technologies, has produced AutoVue—an

embedded camera-based lane-marking recognition

and warning system. By monitoring the visual

lane markings on the road and signaling the driver

with an audible or tactile warning, AutoVue alerts

a tired, distracted, or inattentive driver that he or

she is about to leave their lane

The lane support system audibly warns the

driver of unintended lane departure. Audio

messages are issued as the driver crosses a lane

marking without using the indicators and the

balance of the stereo sound system is controlled to

denote the direction of road departure. The lane

markings are detected by processing images from

a video camera. The current system achieves lane-

marking detection at 15 frames per second. Road

edges with no lane-markings are detected

although they are less reliable. Preliminary studies

show that it is possible for positions. The

frequency of oscillation as seen in adjoining figure

appears to be about 1/8 hertz for normal alert

drivers. This information could be used for

determining the state of driver alertness, although

further investigations are necessary before this can

be proved.

More effective than audible warnings, that

could irritate the driver, is the use of haptic

feedback in the steering wheel. The system could

provide position related assistance/resistance in

the steering wheel. The system could provide an

artificial feel of road camber on either side of the

lane. The driver is still expected to steer the

vehicle but experiences the sensation of driving

along the bottom of a bathtub. Again the system is

deactivated by either the use of the indicators or

by the driver exceeding a torque/motion input

threshold at the steering effort, improved steering

stability and safety.

The key components of the lane support

system are shown in the adjoining figure. A video

camera and processor measures the lane markings.

Another sensor measures the torque that the driver

is applying to the steering wheel. The Electronic

Control Unit (ECU) takes this measurement and

drives a motor, which in turn applies torque

through a gearbox to the steering mechanism there

by assisting the driver. The inc The key

components of the lane support system are shown

in the adjoining figure. A video camera and

processor measures the lane markings. Another

sensor measures the torque that the driver is

applying to the steering wheel. The Electronic

Control Unit (ECU) takes this measurement and

drives a motor, which in turn applies torque

through a gearbox to the steering mechanism there

by assisting the driver. The incorporation of also

improves fault tolerance.

OPERATING PRINCIPLE

When AutoVue detects an unplanned lane

departure, a sound is generated to alert the driver.

The sound can be directional so that a left-hand

departure will result in the sound being generated

in a left vehicle speaker and a right hand departure

in a right speaker.

Figure 2: Architecture of Lane Support System

A camera views the road through a "wiped"

section of the windshield, near the vehicle center

line as shown below.

The image that follows shows the outline

of the view processed in the crosshatched area

seen in the previous diagram. The system camera

is mounted on the windshield inside the cabin,

near the center line of the vehicle. The imager is

mounted high on the windshield for better

functioning of the algorithm to track lane

markings and to accommodate viewing over the

hood.

Within the system is a central processor

unit running a program that performs the lane-

tracking and warning algorithm. The CPU runs

this algorithm in conjunction with other embedded

environment tasks (communications, initialization,

and general task management). The lane-tracking

algorithm accepts image input from the system

camera and examines the lane markings. It

determines whether or not a lane departure is

imminent.

Upon such determination, and if all logic

permits (for example, a turn signal is not active,

and no system errors are present), a warning will

be issued in association with the departure. If the

system is also connected to a Controller Area

Network (CAN) a message will be generated

indicating which side of the vehicle the departure

is occurring. This message can be used to create a

warning to the driver through interfaces such as

the infotainment system or steering system.

Audible warnings are designed to be loud enough

to overcome wind noise from an open window, as

well as engine, traffic, and other noises, without

being annoying.

The AutoVue module hosts a complete

lane tracking software solution including image

capture and improvement, lane tracking

algorithms and drivers for the various system

interfaces.

Lane Departure systems have received

positive feedback from commercial vehicle

operators. A driver satisfaction study of Iteris'

Lane Departure system was conducted in 2004

including responses from 140 drivers in the U.S.

and 100 in Europe that have used the Lane

Departure system. A brief summary of the U.S.

data from the study is as follows:

98% believe the system can prevent

accidents

92% believe the system is a valuable

safety feature

71% say the system has made them safer

drivers

80% normally drive with the system

enabled

97% are satisfied or very satisfied with the

system

IMPLEMENTATION IN CARS

Already, Nissan's luxury division, Infiniti, is

offering a system, developed by Iteris and

supplied by Valeo, on its 2005 FX45 sport utility

vehicle, Infiniti's second application of the Valeo

LaneVue system will be on its 2006 M

performance sedan, which goes on sale next

spring. In Europe, Valeo is also supplying the

system to the 2005 C4 and C5 sedans from

Citroen, recently introduced at the Paris auto

show.

COLLISION AVOIDANCE SYSTEM

Every year 50,000 people are killed on the

roads in the European countries. The figure for the

USA is similar. In Britain out of 240,000

accidents in a typical year there are 5,000 fatalities

and 64,000 injuries. Many safety innovations in

the areas of banking systems, airbags, body

structures, steering and suspension have already

had a beneficial effect. However, accident

frequency and severity have still remained

unacceptably high. Accident studies places most

of the blame on drivers. Automatic Collision

Avoidance Systems could have many advantages

over human drivers; they don’t get tired or

distracted, they can simultaneously monitor all

sides of the vehicle, they generally have faster

reaction times and they don’t panic. Apart from

human tragedy these systems could significantly

decrease the economic cost of road accidents and

reduce the inconvenience associated with the

resulting traffic jams.

The main requirement for a collision

avoidance system is to be able to predict

accurately the likelihood of an imminent collision.

If a collision is likely then the system should

respond in such a way as to reduce the threat. The

response could involve the automatic control of

the vehicle or simply an appropriate warning to

the driver. Crucial to the success of any avoidance

system is the ability to reliably identify and locate

different obstacles in the complex and

unconstrained traffic scene, where conditions can

vary from thick fog at night to bright glare from

the sun on a clear day.

COLLISION AVOIDANCE SENSING

Collisions can occur at any point on the car,

therefore sensor coverage should ideally extend to

360 deg. It is probable that no sensor can satisfy

these strict demands. Headway detection systems

have received most attention as they potentially

cover majority of accident situations. However

they are difficult to implement because of the

large maximum range required for driving at

speed. Many false alarms are generated by radar

systems illuminating objects on the bends that are

safely off the vehicle path. False alarm rate should

be very low in a collision avoidance system to

avoid driver irritation. If there is automatic

braking then they should be eliminated.

Calculating collision paths on long ranges and on

bends requires a detection system with wide field

view, a high angular resolution and the detection

of the road geometry.

Figure 3: Architecture of Collision warning

system

Following sensing devices are generally used:

Front sensing system (MMW Radar,

LASER Radar)

Side Sensors

Vision Sensors (CCD Video Camera)

Host Vehicle Sensors

Information from the sensing system flow

to the collision warning processor module,

eventually to the Driver Vehicle Interface (DVI),

this provides the appropriate warning cue to the

driver. Each of the sensing systems receives

information from host vehicle state which

includes:

Differential wheel speed

Vehicle speed

Steering angle

Yawing rate

The date is then send to collision warning

processing module. The data contains information

like lane path, vehicle speed, range etc, relative to

host vehicle. The Collision Warning Processing

Module will then combine data from the active

sensing systems and passive sensing systems to

accomplish object detection, target tracking, in

path target identification and Threat Assessment.

Threat Assessment falls in three categories:

Time To Collision (TTC)

Time To Avoidance (TTA)

Threat or No Threat Decision

If the identified detected target is accessed

as being a potential hazard to the host vehicle then

appropriate warning cues will be issued.

The current Collision Avoidance System

uses a headway video camera and a CCD device.

The radar system is similar to AICC radar system.

The video system detects cars and lane markings

up to a distance of 60m. The low-level image

processing acquires images from a miniature CCD

video camera situated by the rear view mirror.

Edges are extracted using a technique called

Intelligent Thresholding. The edges are then

thinned using morphological techniques. The

thinned edges in the image are then traced by

fitting straight lines to them and then turned into

line vectors, which are then categorized on the

basis of angle. The horizontal lines are used by car

detection algorithm. Range is estimated from the

road to camera geometry by assuming a flat Earth.

As, a result the range data are quite noisy due car

pitching on hilly roads and small possible

variations in the determination of the car position

in the image plane.

A very high computational rate is required

for image processing. Hence microprocessors with

high computational rate are required.

DATA FUSION

Data fusion is a collection of techniques

for combining the measurements from more than

one sensor to provide a more unified result. The

sensors used can be of the same or different type.

This has several benefits:

The overall estimates of parameters can be

more accurate than for individual sensor

estimate, as they reinforce each other.

Any parameter need not depend on one

sensor alone. This has benefits of Fault

Tolerance by allowing redundancy to be

introduced into the system. For example, if

the system had infra red and video based

sensors, it could survive the failure of

either of these and continue to function

although accuracy and performance in

certain conditions can be reduced.

In Lucas system data fusion can gives major

benefits to object detection as the two sensors

could be used to complement each other

extremely well. The video produces good lateral

image but it is not able to provide good estimates

of range. The radar conversely produces good

estimates of range and hence good relative

velocity estimates, but has poor lateral positional

accuracy. Thus by fusing data from these two

sensors the object position can be localized to a

better accuracy by considering the intersection of

the two areas of positional uncertainty generated

by each sensor.

Another possible benefit of data fusion is

that of object identification, by combining the

expected responses of an object in the sensors. For

example, the image processing may confuse a

stationery pedestrian and a traffic sign, especially

at longer ranges, as they both are tall, thin objects.

However fusing data would allow an

unambiguous decision to be reached immediately

as the two objects have radically different radar

responses.

Figure 4: Object Identification by Data Fusion

DECISION STRATEGY

Having detected a hazard the system must

respond in the most appropriate way so as to avoid

the hazard. Evidence suggests that a small

reduction in driver reaction time will dramatically

reduce the number of accidents. This can achieved

either by intervening with the controls of the

vehicle or by warning the driver. Direct

intervention will have the most significant result

but causes the greatest liability concern. In the

worse case the car would contravene the driver’s

wishes by emergency braking just as the driver

decides to brake and overtake. Driver warning is

the safer option but there are still drawbacks. In

critical situation there will be no time to warn the

driver or the warning might even distract the

driver from hazard. If there are too many

warnings, false alarms, or if the driver is already

aware of hazard then warning will be irritating.

Usually, warnings are either visual or

audible. The visual warnings must be in the

driver’s field of view at the time of warning.

Audible warnings are generally more obtrusive

and hence irritating if inappropriate. However also

effective are haptic or tactile warnings. Haptic

feedback in the steering mechanism has already

been described earlier in lane support system, but

the same system can be used to increase the

steering reactance if there is an obstacle in the

driver’s blind spot during a lane change

maneuver. Kinesthetic warnings invoke a very

quick response by jerking the car or by moving

the seat. The driver rapidly decelerating the car

can quickly wake even sleeping passengers. If the

jerk situation is short then there will be little effect

on the car’s speed leaving all options open for the

driver.

Detection of hazards can be calculated by

considering the trajectories of other objects in

relation to your vehicles, its operating envelope

and the state and intentions of the driver. If the

driver is inattentive then action should be taken,

but if the driver is alert then there is no need to

take any action. Hence the decision process

should include state and intentions of the driver.

Monitoring the driver is a hard problem; it is

difficult to distinguish between a genuine lane

change and a lane drift due to drowsiness.

Progress has been made using neural networks to

learn the driver behavior. A backward error

propagation network can predict if the driver is

going to overtake or brake with a success rate of

90% when trained on only 15% of total input data.

In the situation that require direct of the

vehicle, recent work has shown the use of neural

networks for emulating driver brake and throttle

control. Such a system should be particularly

useful for AICC where drivers feel most

comfortable, when the system mimics their own

driving style.

FEATURES

Maintains safe, comfortable distance between

vehicles without driver interventions.

Maintains consistent performance in poor

visibility conditions.

Maintains continuous performance during road

turns and elevation changes.

Alerts drivers by way of automatic braking

BENEFITS

Minimizes speed differentials between

vehicles.

Reduces throttle and brake management.

To be able to adapt - therefore, learning

ability.

To be autonomous - therefore, ability to sense,

model and provide output.

The need for more safety and convenience in

the vehicle.

To overcome the loss of life and financial cost.

CONCLUSION

This paper has described a wide variety of

automotive applications for intelligent

autonomous systems. The fully autonomous car is

probably not viable in the foreseeable future.

Semiautonomous systems as discussed above are

technologically feasible but issues such as driver

acceptance, reliability, safety and product liability

have yet to be resolved. AICC and Lane Support

Systems not only reduce driver comfort but also

reduce the risk of an accident. Collision Warning

System will be useful for alerting a distracted

driver to hazard provided the time to impact is not

too critical.

When considering the benefits, cost is a

major consideration in the automotive market.

Much of the technology tends to be expensive as it

originates from low volume/high cost military

markets where volumes are high such as

microwave satellite receivers and door openers

then cost have become less and markets have been

successfully exploited. The end objective is to be

able to survive in an evolving environment and

changing circumstances, a representation - of real-

world system.

REFERENCES

1. Intelligent Autonomous systems for cars – By

R.H Tribe.

2. Autonomous Intelligent Cruise Control with

automatic braking – By Martin.p, SAE,

Detroit, February 1993.

3. http://www.bharatbook.com/general/

4. http://www.its.com

5. Automotive Design Line

6. http://www.automotivedesignline.com