Upload
ramalakshmir
View
229
Download
0
Embed Size (px)
Citation preview
8/2/2019 Multi Sensor Data Fusion Andres Navarro
1/52
Multisensor Data Fusion
Andrs Navarro
8/2/2019 Multi Sensor Data Fusion Andres Navarro
2/52
Contents
Introduction Definition
Sensors and levels
Fusion models Fusion techniques
Distributed Data Fusion
Scenarios
Conclusions
8/2/2019 Multi Sensor Data Fusion Andres Navarro
3/52
Introduction
Motivation Master Thesis
First approach to sensor fusion
Overview of state-of-the-art
Guide for people who do not know about sensorfusion to get introduced to the issue
8/2/2019 Multi Sensor Data Fusion Andres Navarro
4/52
Introduction
Why is sensor fusion needed? Research questions
What is multisensor data fusion? Is there an unanimousdefinition?
What models for multisensor data fusion exist in theliterature? Do they have common descriptions? Do theycontradict each other?
What techniques or methods can be used in multisensordata fusion?
Centralized or decentralized data fusion?
What scenarios can multisensor data fusion be applied?
8/2/2019 Multi Sensor Data Fusion Andres Navarro
5/52
Definition
JDL Data Fusion Lexicom
A. Steinberg and C. Bowman.
Wald
A process dealing with the association, correlation, and combination of dataand information from single and multiple sources to achieve refined positionand identity estimates, and complete and timely assessments of situationsand threats, and their significance. The process is characterized bycontinuous refinements of its estimates and assessments, and theevaluation of the need for additional sources, or modification of the processitself, to achieve improved results.
Data fusion is the process of combining data or information to estimateor predict entity states.
Data fusion is a formal framework in which are expressed means and toolsfor the alliance of data originating from different sources. It aims atobtaining information of greater quality; the exact definition of greaterquality will depend upon the application.
8/2/2019 Multi Sensor Data Fusion Andres Navarro
6/52
Sensors and levels
Commensurate
multisensor data
Non commensurate
multisensor data
Direct Data Fusion
Feature-level Fusion
High-level FusionInformation extraction
8/2/2019 Multi Sensor Data Fusion Andres Navarro
7/52
Fusion models
JDL data fusion model
Dasarathy's functional model
Waterfall fusion process model
Boyd Loop Thomopoulos' Fusion Model
Durrant-Whyte architecture
The Omnibus process model Endsley's Situation Awareness
General Data Fusion Architecture
8/2/2019 Multi Sensor Data Fusion Andres Navarro
8/52
Fusion models
JDL data fusion model Sources
Sourcespreprocessing
Level 1
Level 2
Level 3
Level 4 Database management system
HCI
8/2/2019 Multi Sensor Data Fusion Andres Navarro
9/52
Fusion models
JDL model revisions Drawbacks
Different revisions
New definitions
8/2/2019 Multi Sensor Data Fusion Andres Navarro
10/52
Fusion models
Dasarathy's functional model
Levels of abstraction
Data
Feature
Decision
Categorization of data fusionfunctions in terms of the type ofdata level at input/output.
8/2/2019 Multi Sensor Data Fusion Andres Navarro
11/52
Fusion models
Waterfall fusion process model Fusion process
in stages
Omission of
feedback dataflow is themajor limitation.
8/2/2019 Multi Sensor Data Fusion Andres Navarro
12/52
Fusion models
Boyd Loop OODA cycle:
8/2/2019 Multi Sensor Data Fusion Andres Navarro
13/52
Fusion models
Thomopoulos' Fusion Model
8/2/2019 Multi Sensor Data Fusion Andres Navarro
14/52
Fusion models
Durrant-Whyte achitecture
8/2/2019 Multi Sensor Data Fusion Andres Navarro
15/52
Fusion models
The Omnibus process model
F i d l
8/2/2019 Multi Sensor Data Fusion Andres Navarro
16/52
Fusion models
Endsley's Situation Awareness
F i d l
8/2/2019 Multi Sensor Data Fusion Andres Navarro
17/52
Fusion models
General Data FusionArchitecture
Network based
Levels described asclasses with attributesand functions.
F i d l
8/2/2019 Multi Sensor Data Fusion Andres Navarro
18/52
Fusion models
Models classification
Elmenreich
Abstract Generic Rigid
Durrant-Whyte and Henderson
Architecture: Meta, Algorithmic, Conceptual, Logical andExecution
Centralized - Decentralized
Local Global
Modular Monolithic Heterarchical - Hierarchical
T h i
8/2/2019 Multi Sensor Data Fusion Andres Navarro
19/52
Techniques
Classification Overview
Kalman Filter
Probabilistic Inference
Artificial Neural Networks
Fuzzy Logic
Support Algorithms
Selection
T h i
8/2/2019 Multi Sensor Data Fusion Andres Navarro
20/52
Techniques
Classified by JDL levels
T h i
8/2/2019 Multi Sensor Data Fusion Andres Navarro
21/52
Techniques
Classified by JDL levels
Type of method
Fusion problems
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
22/52
Techniques
Classified by JDL levels
Type of method
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
23/52
Techniques
Classified by JDL levels
Type of method
Fusion problems
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
24/52
Techniques
Classified by
JDL levels
Type of method
Fusion problems
Data Association
Estimation
Identity declaration
Decision-level identity fusion
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
25/52
Techniques
Kalman Filter
Extended Kalman FilterDiscrete Kalman Filter
Assumed noise
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
26/52
Techniques
Kalman Filter with INS
Inertia System:
Good high frequency information
Drift at a slow rate
Other Position System Good data at low frequency, on the average
High frequency noise
The Kalman Filter approach is instead to use the statistical characteristics of the errorsin both the external information and the inertial components to determine this optimalcombination of information. Actually, the filter statistically minimizes the errors in theestimates of the navigation parameters: on an ensemble average basis, no other meansof combining the data will outperform it, assuming the internal model in the filter isadequate.
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
27/52
Techniques
Kalman Filter with INS
Direct implementation
Indirect feedforward
Indirect feedback
Filter fails System Fails
High sample rate CPU load
Erros in the inertial mustremain of small magnitude
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
28/52
Techniques
Probabilistic Inference
Bayesian Inference
It can be used to discriminate between conflictinghypotheses
Initial beliefs are needed before any evidence is evercollected
Sensorfusion:
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
29/52
Techniques
Probabilistic Inference
Bayesian Networks Probabilistic graphical model that represents a set
of variables and their probabilisticinterdependencies
Algorithms to perform inference and learning
Dynamic Bayesian Networks
Extension of Bayesian networks thatallows the representation of temporalinformation Signals
Hidden Markov models Model for Markov process: a stochastic process in which the probability
distribution of the current state is conditionally independent of the path ofpast states
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
30/52
Techniques
Probabilistic Inference
Dempster-Shafer Theory Generalization of Bayesian Theory: Instead of requiring
probabilities for each question, belief functions are used.
Two ideas:
Sensor1 Sensor2
m1(u
0) m
2(u
0)
m(u0)=m
1(u
0)m
2(u
0)
Obtain degrees of belief for one question fromsubjective probabilities fora related question.
Use Dempster's rule for combining these degrees ofbelief:
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
31/52
Techniques
Probabilistic Inference
Dempster-Shafer Theory Generalization of Bayesian Theory: Instead of requiring
probabilities for each question, belief functions are used.
Two ideas:
Sensor1 Sensor2
m1(u
0) m
2(u
2)
m(u0)=m
1(u
0)m
2(u
2)
Obtain degrees of belief for one question from
subjective probabilities fora related question. Use Dempster's rule for combining these degrees of
belief:
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
32/52
Techniques
Probabilistic Inference
Dempster-Shafer Theory Generalization of Bayesian Theory: Instead of requiring
probabilities for each question, belief functions are used.
Two ideas:
Sensor1 Sensor2
m1(u
0) m
2(u
1)
Dempster's rule
Obtain degrees of belief for one question from
subjective probabilities fora related question. Use Dempster's rule for combining these degrees of
belief:
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
33/52
Techniques
Probabilistic Inference
Generalized evidence processing theory
Unifies the Bayesian theory with the D-S, combining theiradvantages and avoiding their disadvantages
Each sensor collects evidence and assigns the evidencevia probability masses; unlike D-S, GEP assigns andcombines probability masses based on the a prioriconditional probability of the hypotheses.
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
34/52
Techniques
Artificial Neural Networks
Computational model of biologicalneural networks:
Densely interconnected set of artificial neurons:simple units as perceptron
Feed-forward / recurrent
Non linear statistical data modelling: They can learn a complex relationship between
inputs and outputs, normally established by theunit weights
Learning:
Units' weights Backpropagation algorithm
Network structure
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
35/52
Techniques
Artificial Neural Networks
Neurons can be trained to represent sensoryinformation and, through associative recall,complex combinations of the neurons can beactivated in response to different sensory stimuli.
The main advantage of neural networks formultisensor fusion is that there is no need of amodel for the sensors or the uncertainties
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
36/52
Techniques
Fuzzy Logic
Fuzzy set theory:
Elements have degrees of membership to thedifferent sets, differing from classical set theory,where elements belong or do not belong to a certain
set.
Rules: IF antecedent THEN consequence
Operators: OR, AND and NOT
Steps: Fuzzyfcation
Rule evaluation
Aggregation
Defuzzyfication
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
37/52
Techniques
Fuzzy Logic
Uncertainty in multisensor fusion can be directlyrepresented in the inference process by allowing eachproposition to be assigned a degree of truth.
Fuzzy Fusion Network:
Input data
Feature Extraction
Feature Level Fusion
Decision Level Fusion
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
38/52
q s
Support Algorithms
Required functions for the fusion system
Library of basic numerical methods
Database management
Man-Machine Interaction Sensor Management
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
39/52
q
Techniques Selection
Goals: Maximum effectiveness: Algorithms should make
inferences with maximum specificity in the presence ofuncertain and missing data, dealing with minimal or noavailable a priori information.
Operational constraints and time constraints must beconsidered.
Resource efficiency in CPU and communications load.
Operational flexibility to account for operational needs
with changing a priori data.
Functional growth.
Techniques
8/2/2019 Multi Sensor Data Fusion Andres Navarro
40/52
q
Techniques Selection
Steps:1.Identifying categories of data-processing techniques or algorithms.
2.Surveying existing prototype and fielded data fusion systems.
3.Analyzing system requirements.
4.Analyzing and defining operational concepts for manual and
automatic processes.5.Identifying preliminary algorithms
6.Performing trade-off analyses of algorithm effectiveness versusrequired system resources.
7.Preparing detailed designs and prototypes of selected algorithms.
8.Refining and tuning the algorithms.
Distributed Data Fusion
8/2/2019 Multi Sensor Data Fusion Andres Navarro
41/52
A collection of processing nodes, connected bycommunication links, in which none of the nodes hasknowledge about the overall network topology.
Requirements
Dynamic Topology Management
Information- Sharing Strategies Algorithms
Scenarios
8/2/2019 Multi Sensor Data Fusion Andres Navarro
42/52
Image Data Fusion
Advantages: Reduction of overall uncertainty and increase of accuracy.
New features in a scene can be perceived
More timely information is available
Scenarios
8/2/2019 Multi Sensor Data Fusion Andres Navarro
43/52
Robot Navigation
Scenarios
8/2/2019 Multi Sensor Data Fusion Andres Navarro
44/52
Ambient Intelligence
Electronic environments that are sensitive and responsive tothe presence of people.
Characteristics:
Embedded
Context-aware
Personalized
Adaptive
Anticipatory
Scenarios
8/2/2019 Multi Sensor Data Fusion Andres Navarro
45/52
Augmented Reality
Integration of virtual content in a real environment in real time.
Alignment between virtualcontent and real world
Fusion of tracking systems:
INS
GPS Ultrasound
Vision-based
Accurate position
Scenarios
8/2/2019 Multi Sensor Data Fusion Andres Navarro
46/52
Augmented Reality
Interaction with otherperceptual systems:
Orienting, Auditory,Haptic, Taste, Smell
Feature extraction
Human Behaviour Sensors: Position, orientation,
body gestures, speech, vitalsigns, eyetracking...
Human behaviour and
experience models andsimulations
Conclusions
8/2/2019 Multi Sensor Data Fusion Andres Navarro
47/52
What is multisensor data fusion? Is there an
unanimous definition?
Most of definitions are restrictive to a certainterminology and applications
A broader definition is needed to cover such awide diversity of sensor fusion applications
Wald's definition is chosen
Discussion will continue
Conclusions
8/2/2019 Multi Sensor Data Fusion Andres Navarro
48/52
What models for multisensor data fusion exist in the
literature? Do they have common descriptions? Dothey contradict each other?
A common point in most of them is the need of divide thefusion process in levels of data abstraction.
More disagreement is found in the idea of a cyclingprocessing.
Relationship between specification and usability
Combine the underlying ideas for the final design.
Conclusions
8/2/2019 Multi Sensor Data Fusion Andres Navarro
49/52
What techniques or methods can be used in
multisensor data fusion?
Data fusion at different levels of abstraction impliesthe use of multiple techniques.
A layout or scheme for the implementation of anykind of sensor fusion application is not feasible.
The design of the fusion algorithms is a lengthy taskwhere multiple fusion techniques can be combined.
Conclusions
8/2/2019 Multi Sensor Data Fusion Andres Navarro
50/52
Centralized or decentralized data fusion?
The decentralized fashion has some advantagesand some disadvantages comparing to thecentralized one.
The implementation of Distributed Data Fusion
requires:
The use of certain fusion models that allowdecentralization.
Specific algorithms
Conclusions
8/2/2019 Multi Sensor Data Fusion Andres Navarro
51/52
Final conclusion
Multisensor Data Fusion is a broad issue due to thewide range of scenarios that it can be applied to.Therefore, to find a definition, a model or an algorithmscheme that is explicit, meaning that it can be followedto implement a real system, and at the same time
usable for any kind of application, is an unfeasibletask. Hence, a view of the different approaches,theories and implementations in the issue of sensorfusion can be presented, intending to be useful as acollection of different ideas that should be combined inthe implementation of a real fusion system.
8/2/2019 Multi Sensor Data Fusion Andres Navarro
52/52
Multisensor Data Fusion
Andrs Navarro