1SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
Infrastructure Side Data FusionInfrastructure Side Data Fusion
Tobias Schendzielorz (TUM)
Paul Mathias (Siemens)
SAFESPOTSAFESPOTSAFESPOTSAFESPOT
2SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
Munich University of TechnologyChair of Traffic Engineering and Control
Univ.-Prof. Dr.-Ing. Fritz Busch
Lecturing and Research
in Public and Private Transport
Lecturing and Research
in Public and Private Transport
Munich University of TechnologyChair of Traffic Engineering and ControlArcisstr. 21D-80333 MünchenEmail: [email protected]
www.vt.bv.tum.de
3SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
4SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
Munich University of Technology
Faculty of Civil Engineering and Geodesy
Institute of TransportationChairmen: Univ.-Prof. Dr.-Ing. F. BuschUniv.-Prof. Dr.-Ing. G. Leykauf
Institute of TransportationChairmen: Univ.-Prof. Dr.-Ing. F. BuschUniv.-Prof. Dr.-Ing. G. Leykauf
• Chair of Traffic Engineering and Control• Department of Urban Planning and Development• Institute of Road, Railway and Airfield Construction
Organization Chart
5SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
TUM-VT – Staff
Civil Engineering
Computer Sciences
MathematicsGeography
Transportation EngineeringEconomic Sciences
Disciplines
Civil Engineering
Computer Sciences
MathematicsGeography
Transportation EngineeringEconomic Sciences
Disciplines
6SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
Dinner on Monday Evening
7SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
Meeting Agenda Monday
Monday, 03rd March 2008
Time Topic Presenter
10:30 Welcome Coffee Tobias - TUM
10.45 Update on latest IP issues Angela - MIZAR
11.00Presentation of current status :Outcome of Task Force MeetingExplanation on Data Format & Messages doc
Angela - MIZAR
11:45Presentations of sensor and fusion partners and discussion (max 10min each)
concerned partners
13:00 Lunch
14:00 LDM Issues (LDM Versions / API) LDM Group Member
PARALLEL WORK GROUPS
14:30 Group 1: Sensing systems concerned partners
Group 2: Data fusion and Map providers concerned partners
17:00 Wrap up of the outcome of the WORK GROUPS
17:30 Closure for day
19.30 Dinner together in Munich Organised by TUM
8SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
Meeting Agenda Tuesday
Tuesday, 04th March 2008
Time Topic Presenter
9:00 Wrap up of the yesterdays session Tobias - TUM
9:15 Continue Discussion on Infrastructure Platform: Software Framework / Physical Architecture / Test Site Issues
ALL
11.00 Demonstrations for next Annual Review Angela - MIZAR
12.00 Planning of next deliverable Angela - MIZAR
13:00 Lunch
14.00 Launch of WP5 activities (Evaluation and Testing) LCPC
15:00 New action list and wrap the meeting Angela - MIZAR
15:30 Meeting closed
9SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
20.02.2008
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
CCTV for Visibility(Preprocessing unit)
SP2 RSU System: Functional Architecture
10SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
CCTV for Visibility(Preprocessing unit)
SIE
PTV
CIDAUT
VTT
BME
IBEO
Test Sites???
MIZAR
LCPC
SP5
PTV
VTT
TUM
CSST
SODIT
TUM
LCPC
PTV
TUM
TUM
TUM/MIZAR
under discussion
20.02.2008
SP2 RSU System: Functional Architecture
SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
20.02.2008
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
SAFESPOT RSU System - Motorway
CCTV for Visibility(Preprocessing unit)
SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
20.02.2008
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
SAFESPOT RSU System - Urban
CCTV for Visibility(Preprocessing unit)
SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
20.02.2008
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
SAFESPOT RSU System – Inter-Urban / Rural System
CCTV for Visibility(Preprocessing unit)
SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
20.02.2008
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
SAFESPOT RSU System – Core System
CCTV for Visibility(Preprocessing unit)
15SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
20.02.2008
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Motorway management system
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
SAFESPOT RSU System - Motorway A16 (NL)
CCTV for Visibility(Preprocessing unit)
under discussion
Detected fog + locationDetected slippery conditions + locationDetected road obstruction + location
16SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
20.02.2008
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
SAFESPOT RSU System - Rural Road N629 (NL)
CCTV for Visibility(Preprocessing unit)
under discussion
Dynamic speed advice + location
17SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
20.02.2008
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
SAFESPOT RSU System - Urban Helmond
CCTV for Visibility(Preprocessing unit)
under discussion
SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SP2 - Data Fusion
Data Receiver
LDM
Q- / T-API
SP5 Application
Message Generator
VANET Messages Q- / T-APIQ-API
CCTV for Positioning (Preprocessing Unit)
Gateway to Safety Centre
Gateway to Traffic Control Centre
Thermal Camera f. Living Objects (Preprocessing Unit)
NIR Camera for Ice Detection (Sensor and Preprocessing Unit)
RFID for Ghost Driver Detection (Preprocessing Unit)
Wireless Sensor Network (Preprocessing Unit)
Gatetway to Traffic Light Controller
Message Router (VANET)
Laserscanner (Infrastructure Sensor)
Manoeuvre Estimator
Traffic Information Consolidator
Traffic Data Calculator
ECAID
Situation Refinement
EnvironmentalEvent
Recognition
EnvironmentalConsolidator
Dynamic Black Spot Recognition
Object Refinement
Sensor Level Fusion
Central Level Fusion
Object Matcher
Map Matcher
Cooperative Pre-Data Fusion
IBEO Specific
CCTV for Visibility(Preprocessing unit)
1
2
3
4
5
6
7
8
9
10
11
Data Items in “SF_SP7_Data_format&messages.doc”:1. LaserSensorObjects UDP2. MSGVehicleBeacon UDP3. StatusTrafficLight&PedestrianDetector,
AggregatedDetectorData SOAP4. CameraMovingObjectDetection UDP5. CameraLivingObjectDetection UDP6. CameraIceDetection UDP7. RFIDGhostDriverDetection (std.) UDP8. WirelessSensorNetworkObjects (std.) UDP9. SafetyCenterInformation (std.) UDP10. TrafficControlCenterMotorway (std.) UDP11. CameraMeteoConditionDetection (std.) UDP12. InfAppCoordinatorToDataFusion UDP13. RSU internal (not in SF_SP7_Data_format&messages.doc)
13
12
SIE
PTV
CIDAUT
VTT
BME
IBEO
Test Sites???
MIZAR
LCPC
SP5
PTV
VTT
TUM
CSST
SODIT
TUM
LCPC
PTV
TUM
TUM
TUM/MIZAR
SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
SAFESPOT RSU System (Interface Defintions)20.02.2008
????????????UDPInfAppCoordinatorToDataFusion12
no
???
yes (?)
???
no
no
no
yes
no
yes
yes
Map Matching
CameraMeteoConditionDetection
TrafficControlCenterMotorway
SafetyCenterInformation (std.)
WirelessSensorNetworkObjects
RFIDGhostDriverDetection
CameraIceDetection
CameraLivingObjectDetection
CameraMovingObjectDetection
StatusTrafficLight&PedestrianDetector, AggregatedDetectorData
MSGVehicleBeacon
LaserSensorObjects
Data Item
Detection area is stored in LDM.
noevent
(? 1Hz)VariableUDP6
There will be no link between a certain vehicle in the LDM and the information “on this link is a ghost driver”.
noeventVariableUDP7
?????????UDP8
Reference to links/lane is stored in LDM.
no1 HzfixSOAP3
yes1 Hz (?)VariableUDP4
Detection area is stored in LDM.
noevent
(? 2Hz)VariableUDP5
Detection area is stored in LDM.
noevent
(? 2Hz)variableUDP11
no??????UDP10
noevent (?)VariableUDP9
yes2 HzvariableUDP2
Already matched with vehicle beacon data.
no12,5 HzvariableUDP1
RemarksObject Matching
Freq.# DataObjectsProt.No
SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
Process Timing: Data Fusion / Object Refinement20.02.2008
0 ms 500 ms250 ms 750 ms 0 ms 500 ms250 ms
sensor 1 (1 Hz)
sensor 2 (2 Hz)
sensor 3 (6 Hz)
sensor 4 (event)
VANET (event)
• The OR-ME-block is performed twice a second.• The sensors 1-4 above are assumed to be trajectory related and provide absolute positions of objects: e.g. cameras, laser scanner, etc.• The timing of the transmission of the sensors results should be like this that periodic events lays as close as possible to the left border
of OR, i.e. in one or both of the intervals [0,250] and [500,750].• The VANET events can hardly be influenced concerning the timing of transmission. They are all used no matter when they arrive to
perform the OR. Worst case for the age of a data item at the moment of processing is 500 ms. • Situation refinement or application, that uses in some way positions of objects and/or trajectories, should start either closely after 0
msec or closely after 500 msec.
LD
M
OR ME
LD
M
LD
M
OR ME
LD
M
LD
M
OR ME
LD
M
OR = Object Refinement, ME = Manoeuvre Estimator
Any Situation Refinement Any Situation Refinement Any Situation Refinement
Any Application
described in next slide
SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
R R
R
t1
t1+b
t1
t1-a
Data Fusion: OR Logical Description 20.02.2008
Source 1
Input at t1-a
Source 3Input at t1+b
Source 2Input at t1
Time Alignment &Tracking
Object MatchingData Receipt Map Matching Writing LDM
offset time : aoffset time : blogical reference time : t1
t1
• The alignment computes / calculates the position of an object (could be any other state / attribute of the object as well) to one specific point in time. In the example above are 3 input sources which provide data on the absolute position of objects (WGS 84). These data is sensed at different points in time (t1-a; t1; t1+b). Note: The result can be transmitted at the same point in time (technical point in time vs. logical point in time).
• The alignment “shifts” (either forward or backward) all positions to the same point in time t1. (See column 2 in the picture). This point in time should be the value of the time stamp the object position has in the LDM as well. In order to do this “shifting” the alignment function has to know more about the states of the object (e.g. speed, acceleration, position at the point in time before t 1). I.e. a tracking of the objects is necessary for doing this. And therefore a internal OR memory stack.
• After the alignment the matching of the position of objects provided by the different sources (column 3) is done. • A requirement for this is that all components have synchronized computer clocks. (NTP Server?!?)
22SP2 Plenary Meeting – Data Fusion03 - 04 March 2008, Munich
Data Sources - TODO
1. define UDP data structures,
2. define static description of the sensor (LDM),
3. generate sample data + develop mockup,
4. consideration of NTP (ethernet connection),
5. data transmission synchronisation for trajectory related data ([0,250] and [500,750]),
6. ensure incorporation of accuracy values for the raw data,
7. ensure UDP binary transmission,
8. ensure that UDP time stamp must be the time point of the event (!!)
9. similar data might use the same data structure (classification)