View
213
Download
0
Category
Preview:
Citation preview
Young Researchers Seminar 2011DTU, Denmark, June 8 - 10, 2011
Young Researchers Seminar 2011DTU, Denmark, 8 - 10 June, 2011
Reliability of In-Vehicle Warning System for RailwayLevel Crossings – a User-Oriented Analysis
Risto Öörni, M.Sc.(Tech)VTT Technical Research Centre of Finland
2
Background• Level crossing accidents are a major safety problem for railways and an existing
traffic safety problem for road users• On average, close to 10 people die and 50 get injured every year in Finland in level
crossing accidents• At least some of the level crossing accidents could be avoided by increasing the
situation-awareness and alertness of the road user• In-vehicle warning system for railway level crossings is currently being tested on a
railway line in Southern Finland• The field test will end in June 2011• Development started in 2006 a small-scale proof-of-concept test
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
3
Objectives• To analyse the reliability of in-vehicle warning system for
railway level crossings from the user point of view• Reliability of the system was analysed from user
perspective mainly for two reasons:To get an overview of the current reliability of the system and factorscontributing to itThe service quality experienced by users has an impact on the potentialthe system has to improve the safety of road
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
4
Functionality of the system• In-vehicle system provides an audible or visual warning when the driver approaches a level
crossing simultaneously with a train• In-vehicle system detects situations in which the car approaches a level crossing• When an approaching situation is detected, the in-vehicle sends a query to the server with the
number of the level crossing, and the server provides the status of the level crossing as a reply tothe in-vehicle system
• The server calculates the status of level crossings on the basis of positioning messages sent bytrains
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
5
Technical architecture
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
AplicomA1 Trax
GPS-receiver GPRS
SemelCL341
GPSreceiver GPRS
SemelMC40
GPSreceiver GPRS
GPS
ADSL
Semel mobile gateway(device management andgateway server for MC40CL341 devices and a web
service interface)
Internet
Level crossing server(VTT)
Windows 2000
MySQL database
Level crossing server application
PHP
In-vehicle system: NokiaNavigator 6710/E52/E75
GPSreceiver GPRS
GPS
Symbian V9.3, Java
Client software
GPRS/3G
GPRS/3G
Internet
Internet
GPRS/3G
Internet
AplicomA1 Trax
GPS-receiver GPRS
AplicomA1 Trax
GPS-receiver GPRS
SemelCL341
GPSreceiver GPRS
SemelCL341
GPSreceiver GPRS
SemelMC40
GPSreceiver GPRS
SemelMC40
GPSreceiver GPRS
GPSGPS
ADSL
Semel mobile gateway(device management andgateway server for MC40CL341 devices and a web
service interface)
InternetInternet
Level crossing server(VTT)
Windows 2000
MySQL database
Level crossing server application
PHP
In-vehicle system: NokiaNavigator 6710/E52/E75
GPSreceiver GPRS
GPSGPS
Symbian V9.3, Java
Client software
GPRS/3GGPRS/3G
GPRS/3GGPRS/3G
InternetInternet
InternetInternet
GPRS/3GGPRS/3G
InternetInternet
6
Definition• Reliability: “the probability of a component,
subsystem of complete system to functioncorrectly over a given period of time under agiven set of operating conditions” (Storey 1996)
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
7
Method
• Analysis of the reliability of the system was based on a frameworkdescribed in earlier research on the reliability of advanced drivingassistance systems (Gietelink 2007)
• The alarms provided by the system were compared against trainswhich passed the level crossing during the observation period
• The empirical data needed to analyse the reliability of the systemwas collected with video monitoring performed at two levelcrossings and with automatic data logging features implemented inthe system
• A criteria for a successful alarm was defined• The system was presented as a reliability block diagram to better
understand the factors contributing to the reliability of the systemReliability of In-Vehicle Warning System for Railway Level
Crossings – a User-Oriented Analysis
8
Prediction matrix and reliabilitymeasures
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Actual dataNegative (safe) Positive (threat
PredictionNegative (safe) TNN FNN
Positive (threat) FPN TPN
Rate DefinitionReal occurrence rate p TPFNFPTNTPFN NNNNNN /
Accuracy accuracyp TPFNFPTNTPTN NNNNNN /
Precision CPp TPFPTP NNN /
True positive rate TPp TPFNTP NNN /
False negative rate FNp TPFNFN NNN /
True negative rate TNp FPTNTN NNN /
False positive rate FPp FPTNFP NNN /
Reliability relpTPFNTPFPTP NNNNN /2
9
Reliability block diagram
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Trainequipment(A1 Trax)
GPRS andpublic
Internet(from
server toIVS)
Levelcrossing
server(VTT)
GPRS andpublic
Internet(from trainto server)
In-vehiclesystem User
Trainequipment(CL341 or
MC40)
Semelmobile
gateway
Publicinternetand webservice
interface
Levelcrossing
server(VTT)
GPRS andpublic
internet(from
server toIVS)
In-vehiclesystem User
GPRS andpublic
Internet(from trainto Semek
mobile gw)
A1 Trax devices installed in Dm12 rail busesand Dv12 diesel engines:
CL341 and MC40 devices installed in Dv12diesel engines
A B C
Trainequipment(A1 Trax)
GPRS andpublic
Internet(from
server toIVS)
Levelcrossing
server(VTT)
GPRS andpublic
Internet(from trainto server)
In-vehiclesystem User
Trainequipment(A1 Trax)
GPRS andpublic
Internet(from
server toIVS)
Levelcrossing
server(VTT)
GPRS andpublic
Internet(from trainto server)
In-vehiclesystem User
Trainequipment(CL341 or
MC40)
Semelmobile
gateway
Publicinternetand webservice
interface
Levelcrossing
server(VTT)
GPRS andpublic
internet(from
server toIVS)
In-vehiclesystem User
GPRS andpublic
Internet(from trainto Semek
mobile gw)
Trainequipment(CL341 or
MC40)
Semelmobile
gateway
Publicinternetand webservice
interface
Levelcrossing
server(VTT)
GPRS andpublic
internet(from
server toIVS)
In-vehiclesystem User
GPRS andpublic
Internet(from trainto Semek
mobile gw)
A1 Trax devices installed in Dm12 rail busesand Dv12 diesel engines:
CL341 and MC40 devices installed in Dv12diesel engines
A B C
10
Equipment user to realise thesystem
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
11
Classification of alarm situations
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Early endingof alarm
Late start
t / s
t / s
t / sCorrectalarm
Latest acceptabletime for starting
alarm:T(arrived) – y
Earliest acceptabletime for starting
alarm:T(arrived) – x
Train arrives atthe levelcrossing:T(arrived)
Latestacceptable time
for end ofalarm:
T(passed) + z
Interruptedalarm
Late start +interruption
t / s
t / s
Train haspassed the level
crossing:T(passed)
Correctalarm t / s
Early endingof alarm
Late start
t / s
t / s
t / sCorrectalarm
Latest acceptabletime for starting
alarm:T(arrived) – y
Earliest acceptabletime for starting
alarm:T(arrived) – x
Train arrives atthe levelcrossing:T(arrived)
Latestacceptable time
for end ofalarm:
T(passed) + z
Interruptedalarm
Late start +interruption
t / s
t / s
Train haspassed the level
crossing:T(passed)
Correctalarm t / s
12
Experimental setup
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Hard disk driverecorder
Videomultiplexer
Levelcrossing
Camera 2
Navigator phone withlevel crossing warning
application
Digital soundrecorder withmicrophone
Camera 1
Hard disk driverecorder
Hard disk driverecorder
Videomultiplexer
Videomultiplexer
Levelcrossing
Camera 2
Navigator phone withlevel crossing warning
application
Digital soundrecorder withmicrophone
Camera 1• Video monitoring was performed in
two level crossings• Two video cameras were used –
one aimed at the in-vehicle deviceand one at the level crossing
• Video streams from the twocameras were combined into onestream with a video multiplexer
• The video stream was recordedwith a hard disk recorder
13
Monitoring equipment• Monitoring equipment was installed
in a metallic box mounted on anelectric pole close to the levelcrossing
• The hard disk and battery had tobe changed every day
• Video data recorded to the harddisk was exported into mpg formatand viewed with Avidemux videoprocessing software
• Relevant events were documentedas a spread sheet
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
14
Test sitesVideo monitoring was carried out in twolevel crossings on a railway linebetween Hanko and Karjaa:1. Lappohjan satama
Shunting activity, rail buses andfreight trainsNo train stop close to the levelcrossing
2. Skogbyn seisakeNo shunting activity, rail buses andfreight trainsLevel crossing located close to atrain stop
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Lappohjan satama level crossing (map from GoogleMaps)
Skogbyn seisake level crossing (map from GoogleMaps)
15
Results of video monitoring
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Events observedTrain observed N
Successful alarm 63
Interrupted alarm 18
Late end 13
Late start 6
Early end 5
Late start + interrupted 1
Early start 1
Late end + interrupted 1
Late start + late end 1
Missed alarm 56
No train
False alarm 13
16
Calculated results
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Actual dataNo train Train observed
PredictionNegative (safe) not applicable 103Positive (threat) 13 63
Rate Definition ResultReal occurrence rate p TPFNFPTNTPFN NNNNNN / 92.7%
Accuracy accuracyp TPFNFPTNTPTN NNNNNN / 35.2%
Precision CPp TPFPTP NNN / 82.9%
True positive rate TPp TPFNTP NNN / 38.0%
False negative rate FNp TPFNFN NNN / 62.0%
True negative rate TNp FPFNTN NNN / 0% (notapplicable)
False positive rate FPp FPTNFP NNN / 1 (notapplicable)
Reliability relpTPFNTPFPTP NNNNN /2 56,1%
17
Causes of unsuccessful alarms
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Causes of missed alarms (nodetection at all)
Cause N
Unequipped train 12
Handover or other shortdisconnection in the dataconnection between train unitand level crossing server ormobile gateway
15
Other 29
total 56
• A preliminary analysis wascarried out to find out thecauses of unsuccessful alarms
• Focus of the analysis was onsituations in which the systemprovided no detection at all (56situations)
• 27 of the 56 missed alarmscould be explained withunequipped trains andhandovers
18
Conclusions• 63 trains of 166 were detected successfully which implies a true positive
rate of 38.0%• If also alarms with quality deviations (47 alarms) are classified as
successful, true positive rate improves to 66.3%The expected functionality has been realised but the current reliability level of thesystem doesn’t meet the reliability required from a commercially available ITSservice
• 63 successful and 13 false alarms were observed; 17.1% of all alarms werefalse alarms
The challenge in the further development seems to be the share of falsenegatives rather than false alarms
• A detailed analysis of the causes of false negatives is needed to betterunderstand the factors contributing to the reliability of the system and toestimate the potential for improvements
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
19
Recommendations for furtherresearch• Potential to improve the performance of the system and to
overcome the limitations of present GPRS data transmission withchanges to algorithms and communication protocols in differentparts of the data processing chain
• Changes needed to the functionality of the system to create asuccessful ITS service
• Impact of new mobile communication technologies such as 3Gtechnologies and LTE on the reliability of the system
Reliability of In-Vehicle Warning System for Railway LevelCrossings – a User-Oriented Analysis
Recommended