Copyright Autoliv Inc., All Rights Reserved
En ny generationNight VisionTransportforum 2009
Bo Wass8. January 2009
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Innehåll
�Behovet av Night Vision (NV)
�Översikt IR-teknik för NV
�NV1
�NV2
�Exempel med NV 2
�Slutsats och nästa steg
Copyright Autoliv Inc., All Rights Reserved
Behovet avNight Vision
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
� UMTRI study (2001-33) based on FARS data 1987-1997
� 41597 fatalities (1999) sorted by type and % of occurrence at night.
0%
10%
20%
30%
40%
50%
60%
70%
Other vehicle
in motion
Pedestrian Overturn Tree
Total fatalities
Of which in darkness
Behovet av NV Hur definiera målet ?
Note that the effect of darkness is biased by other important factors such as traffic conditions and human behaviors
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Behovet av NVHur definiera målet ?
�Metod använd av UMTRI för att eliminera andra faktorer änmörker� “Daylight Saving Time” (DST) metoden
� Jämför perioder om 3 veckor före och efter vinter / sommartidsövergång
� Observations tid: 1 timma före och efter dag/natt eller natt/dagövergångar
� Andra parametrar antas i huvudsak konstanta
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Slutsats av UMTRI-studien�Riskökning vid bilkörning i mörker avseende olyckor med
dödlig utgång:� Kollision med fotgängare: 4.14
� Kollision med vilt: 4.60
� Kollision med annan bil i rörelse : 1.33
�Potentiell minskning av dödsolyckor i mörker per år genomatt eliminera effekten av mörker:� Kollision med fotgängare : 76%
*) based on 1999 US statistics
Behovet av NVHur definiera målet ?
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
19000 700052002700Number of fatalities
2/5 > 65 y.o.
1/3 > 75 y.o.
71% away from intersections
47%
14%
2/3 > 65 y.o.
1/2 > 75 y.o.
64% away from crossings (*)
66%
32%
1/5 > 65 y.o.
1/10 > 75 y.o.
Elderly as pedestrian fatalities
80% away from intersections
Location of fatal pedestrian accidents
67%% of pedestrian fatalities at night time
11%% of total traffic fatalities
Fotgängarolyckor med dödlig utgång
Source: IRTAD 2005, www.bast.de
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Day; 36%
Night; 47%
Dawn &
twilight;
17%
Source: CARE Database / ECDate of query: July 2005
Fotgängarolyckor med dödlig utgång i Europa
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
RescueRide DownActivation of Protections
Automatic Braking
�All data indikerar att det första trafiksäkerhetsmålet förNight Vision ska vara att reducera fotgängarolyckor
�Systemets funktionalitet måste inkludera förbättrad“situation awareness” och varning.
Behovet av NV Slutsats
WarningAwareness
enhancement
impact
0
Copyright Autoliv Inc., All Rights Reserved
Infrared (IR) Technology for Night Vision
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
� FIR � “Far Infra Red” or “Passive IR”, (typically 8-12 mikrometer)
� All objects with own temperature above 0°K emit radiation according to Planck´s law (depending on object temperature and emissivity)
� FIR sensors only “listen” to the emitted radiation (Passive)
� Most materials are opaque to FIR (glass, plastic etc.), some specific materials are transparent (Ge, Si etc.)
�NIR � “Near Infra Red” or “Active IR” (typically 0,8 mikrometer)
� Works like visible light: sensors are capturing NIR reflected by objects
� Illumination e.g by modified halogen lamps
FIR och NIR för Night VisionGrundläggande funktion
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
� FIR � “Strange image” from thermal contrast in the scene
� No illumination, hence no disturbance to other cars
� No glare from head lights (FIR is “blocked” by glass and plastics)
� Very long range from low attenuation in air
� Intrinsically fit for detection of living beings
� Moderately affected by adverse weather conditions
�NIR � Conventional image from grey contrasts but living beings are hard to
identify from the scene
� Illumination is a potential problem for oncoming NIR-NV equipped cars
� Glare from head lights (partly removable by image processing)
� Range limited by illuminator range and more limited by attenuation in air
� Strongly affected by adverse weather conditions
FIR och NIR för Night VisionKaraktäristiska egenskaper
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
UMTRI-2004-38 Pedestrian Detection with Near
and Far Infrared Night Vision Enhancement
Distance
40m
80m
120 m
160 m
240 m
200 m
Easier detection using FIR
Younger drivers
Older drivers
FIR och NIR för Night VisionEn jämförelse
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Autoliv FIR Night Vision 1Awareness enhancement
In production since 2005
Copyright Autoliv Inc., All Rights Reserved
Autoliv Night VisionGeneration 2
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Viktiga förutsättningar
� successful experience with FIR NV camera and system
� real world need for early detection of Pedestrians at night
� intrinsic ability of FIR to detect living beings
Autoliv beslutade 2005 att starta produktutvecklingen av NV Generation 2 med fotgängarvarning
� internal development
� 2nd generation camera and ECU
�SOP October 2008
Fotgängardetektering och varningHistorik
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Krav: Varna “soon enough” med hög tillförlitlighet
� “Time to Impact” (TTI) : Interna studier visar att 2-4 s ärönskvärt (reaktionstid + agerande) beroende på hastighet� Ex, varning vid 3,5 s TTI motsvarar ca 88 m detekteringsavstånd vid
konstant 90 km/h
�Systemkrav detekteringsavstånd: 90 m
� “Warning-rate” : > 90 %
� Falska varningar: < 1 per 3 timmar (reference data set)
�Bra bildkvalitet
�HMI (designas av biltillverkaren)
Fotgängardetektering och varningKrav
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Använd kommersiellt tillgänglig sensor och processor
�Detector pixel count and FoV shall provide enough image-pixels to identify a pedestrian at 90m
�Processor & architecture� Fit for running the novel and innovative detection algorithms
� Must meet the high data stream demand
� Automotive grade
� Car specific functions should be well separated from the pedestrian detection software
Fotgängardetektering och varningTeknologibegränsningar
Copyright Autoliv Inc., All Rights Reserved
Fotgängardetektering och varningNV2 System Lay-Out
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Fotgängardetektering och varningFIR Sensing & processing technology choice
Detector:
�Vanadium Oxide Bolometer (FLIR Systems)
�320X240 pixels
�25 µ pixel pitch for reduced cost
�High sensitivity : NETD<100mK
Optics:
�HFoV: 24 deg horizontal
Detection
�Algorithm containing Extractor, Classifier based on latest statistical learning theory, car’s Path Prediction and Tracker
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Fotgängardetektering och varningNV2 FIR Camera
Serviceable Serviceable
Front WindowFront Window
ShutterShutter
FIRFIR
DetectorDetector
Power &Power &
Video BoardsVideo Boards
OpticsOptics
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Fotgängardetektering och varningEstimering av TTI (och position) position
� X/Y Distance and Time-to-Impact estimator
� Examples: 1.8 m and 1.2 m “stationary” pedestrians, car speed 50km/h
� Actual system performance compared with reference sensor data
6 7 8 9 10 11 12 13 14 150
1
2
3
4
5
6
7
8
9TTI (s)
Sequence Time (s)6 7 8 9 10 11 12 13 14 15
-6
-4
-2
0
2
4
6Y distance (m)
Sequence Time (s)
True Estimated
1.8 m
8 9 10 11 12 13 140
1
2
3
4
5
6
7
8
9
10TTI (s)
Sequence Time (s)8 9 10 11 12 13 14
-6
-4
-2
0
2
4
6Y distance (m)
Sequence Time (s)
1.2 m
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Fotgängardetektering och varningVarningstrategi och HMI
The warning is activated when:
� a pedestrian is located in the “static” warning area
� a pedestrian moves into the vehicle path
� size of warning area is speed-dependent
An OEM-specific HMI is used to alert the driver
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Fotgängardetektering och varningVarningstrategi och HMI
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Fotgängardetektering och varningVarningstrategi och HMI
ALV-Wass-Transportforum 2009 Copyright Autoliv Inc., All Rights Reserved
Slutsats och nästa steg
Slutsats
� FIR is confirmed to be the best technology for detecting living beings
�Pedestrian Detection with single FIR camera is a world’s first
�Real World safety benefit expected to be high
�Successful development and SOP
Nästa steg
�Animal Detection & Warning
Copyright Autoliv Inc., All Rights Reserved
Tack!