[email protected] Rennes, February, 22th 2018people.irisa.fr/Arnaud.Martin/publi/presentationDGA2018.pdf · 2019. 9. 23. · Information fusion Arnaud Martin [email protected]

Information fusion

Arnaud [email protected]

Université de Rennes 1 - IRISA, Lannion, France

Rennes, February, 22th 2018

Information fusion, A. Martin 22/02/18

1/36

OutlineFusionBeliefApplications

1. Introduction to information fusion

2. Theory of belief functions for information fusion

3. Applications


2/36

What is information?FusionBeliefApplications


3/36

(1/12)

: The traffic is dense on the runway, ready for takeoff.



3/36

(1/12)


Imprecise proposition



3/36

(1/12)



: Ten aircrafts were on the runway, ready for takeoff.



3/36

(1/12)




Precise proposition



3/36

(1/12)




Precise proposition

Imprecision is a kind of imperfection of information



4/36

(2/12)

: A boeing 777 is at the position x, y, z



4/36

(2/12)


Bad weatherUncertain proposition



4/36

(2/12)



Good weatherCertain proposition



4/36

(2/12)



Good weatherCertain proposition

Uncertainty is another kind of imperfection of information

How are the sources?FusionBeliefApplications


5/36

(3/12)

: I am sure that is a fighter plane.A

: That is maybe a A380B

FUSION



5/36

(3/12)



Conflict of information sources hasto be solved

FUSION



5/36

(3/12)



FUSION

A is often mistakenB is never wrong



5/36

(3/12)



FUSION

: That is maybe a A380A is often mistakenB is never wrong



5/36

(3/12)



FUSION

A is often mistakenB is never wrong

Reliability of information sourceshas to be considered

Interest of information fusionFusionBeliefApplications


6/36

(4/12)

Assume all the sources are completly reliable giving totally certain andprecise information

FUSION



6/36

(4/12)


: This is an A380

: This is an A380

: This is an A380

FUSION



6/36

(4/12)


: This is an A380

: This is an A380

: This is an A380

: This is an A380FUSION

ImperfectionsFusionBeliefApplications

Imperfections on information

I Uncertainty: Degree of conformity to the reality

I Imprecision: quantitative default of knowledge on theinformation contain

I Incompleteness: lack of information

Sources of information

I Reliability: to give an indisputable information

I Independence: Sources are not linked (assumption oftenmade)

I Conflicting sources: Sources give information in contradiction


7/36

(5/12)

Information fusionFusionBeliefApplications

Goal: To combine information coming from many imperfectsources in order to improve the decision making taking intoaccount of imprecisions and uncertaintiesTo combine information coming from imperfect sources leadsfatally to conflict

Three actions are possible face to imperfections:

1. We can try to suppress its

2. We can tolerate its and so we need robust algorithms againstthese imperfections

3. We can model its

To model imperfections: uncertainty theories:Probability theory (Bayesian approach) or possibility theory orthe theory of belief functions


8/36

(6/12)

Fusion architecture for classifiers fusionFusionBeliefApplications

s sources S1, S2, ..., Ss that must take a decision on anobservation x in a set of n classes x ∈ Ω = {ω1, ω2, . . . , ωn}classes

ω1 . . . ωi . . . ωnS1...Sj...Ss

M11 (x) . . . M

1i (x) . . . M

1n(x)

.... . .

.... . .

...

M j1 (x) . . . Mji (x) . . . M

jn(x)

.... . .

.... . .

...M s1 (x) . . . M

si (x) . . . M

sn(x)

(1)


9/36

(7/12)

Steps of information fusionFusionBeliefApplications

Problems

I How to combine the information coming from all the sources?

I How to take the decision?

Under-problems depending on the application

I How to model information? i.e. choice of a formalism

I How to estimate the model parameters?

4 steps

1. Modeling

2. Estimation

3. Combination

4. Decision


10/36

(8/12)

Voting processFusionBeliefApplications

Modeling: Indicator functions

M ji (x) =

{1 if Sj : x ∈ ωi0 otherwise

Estimation: αij : reliability of a source for a given classCombination:

MEi (x) =s∑j=1

αijMji (x)

Decision:

x ∈ ωk if MEk (x) = maxiMEi (x) ≥ c.s+ b(x)x ∈ ωn+1 otherwise i.e. no decisionc ∈ [0, 1], b(x) function of MEk (x)


11/36

(9/12)

Voting process: A resultFusionBeliefApplications

I AssumptionsI Statistical independence of sourcesI Same probability of success pI s odd and > 2 (same kind of result if s is even)

I ThenWith PR the probability of success after fusion by votingprocess

I If p > 0.5, PR tends toward 1 with sI If p < 0.5, PR tends toward 0 with sI If p = 0.5, PR = 0.5 for all s

Behind this result: The majority should have right for fusion.


12/36

(10/12)

Probability theory for information fusionFusionBeliefApplications

Modeling: A probability is a positive and additive measure, p isdefined on a σ-algebra of Ω = {ω1, ω2, . . . , ωn} and takes values in[0,1].

It verifies: p(∅) = 0, p(Ω) = 1,∑X∈Ω

p(X) = 1

Estimation: Choice of the distribution, and/or estimation ofparametersCombination: Bayes rule

p(x ∈ ωi/S1, . . . , Ss) =p(S1, . . . , Ss/x ∈ ωi)p(x ∈ ωi)

p(S1, . . . , Ss)(2)

Independence assumption must of the time necessaryDecision: a posteriori maximum, likelihood maximum, meanmaximum, etc.


13/36

(11/12)

Limits of the theory of probabilitiesFusionBeliefApplications

I Difficulties to model the absence of knowledge (ex: Sirius)

I Constraint on the classes (exhaustive and exclusive)

I Constraint on the measures (additivity)

Example of Smets on the matter of additivity

If one symptom f (for fiver) is always true when a patient get aillness A (flu) (p(f |A) = 1), and if we observe this symptom f ,then the probability of the patient having A increases (becausep(A|f) = p(A)/p(f) so p(A|f) ≥ p(A)).The additivity constraint require then that the probability of thepatient having not A decreases:p(A|f) = 1− p(A|f) so p(A|f) ≤ p(A)While there is no reason if the symptom f can be also observe insome other diseases.


14/36

(12/12)




3. Applications


15/36

Bases on Belief functionsFusionBeliefApplications

I Use of functions defined on sub-sets instead of singletons suchas probabilities

I Discernment frame: Ω = {ω1, . . . , ωn}, with ωi are exclusiveand exhaustive classes

I Power set: 2Ω = {∅, {ω1}, {ω2}, {ω1 ∪ ω2}, . . . ,Ω}.I Several functions in one to one correspondence model

uncertainty and imprecision: mass functions, belief functions,plausilibity functions

I Extension of 2Ω to DΩ, hyper power set in order to model theconflicts

I DΩ closed set by union and intersectionoperators

I DΩr : reduced set with constraints(ω2 ∩ ω3 ≡ ∅)


16/36

Modeling (1/13)

Mass functionsFusionBeliefApplications

I The basic belief functions (bba or mass functions) are definedon 2Ω and take values in [0, 1]

I Normalization condition:∑X∈2Ω

m(X) = 1

I A focal element is an element X of 2Ω such as m(X) > 0

I Closed world: m(∅) = 0I We note mj the mass function of the source Sj


17/36

Modeling (2/13)

Mass functionsFusionBeliefApplications

Special cases:

I If only focal elements are ωi then mj is a probability

I mj(Ω) = 1: total ignorance of SjI categorical mass function: mj(X) = 1 (noted mX): Sj has

an imprecise knowledge

I mj(ωi) = 1: Sj has a precise knowledge

I simple mass functions Xw:mj(X) = w and mj(Ω) = 1− w: Sj has an uncertain andimprecise knowledge


18/36

Modeling (3/13)

DiscountingFusionBeliefApplications

From (Shafer, 1976):mαj (X) = αjmj(X), ∀X ∈ 2Ωmαj (Ω) = 1− αj(1−mj(Ω))

αj ∈ [0, 1] discounting coefficient can be seen as the reliability ofthe source SjIf αj = 0 the source are completely unreliable, all the mass istransfered on Ω, the total ignorance

The discounting process increases the intervals [belj ,plj ] (and soreduces the global conflict in the conjunctive combination rule)


19/36

Modeling (4/13)

Go back on fusion architectureFusionBeliefApplications

s sources S1, S2, ..., Ss that must take a decision on anobservation x in a set of n classes x ∈ Ω = {ω1, ω2, . . . , ωn}classes

ω1 . . . ωi . . . ωnS1...Sj...Ss

m11(x) . . . m

1i (x) . . . m

1n(x)

.... . .

.... . .

...

mj1(x) . . . mji (x) . . . m

jn(x)

.... . .

.... . .

...ms1(x) . . . m

si (x) . . . m

sn(x)


20/36

Estimation (5/13)

Distance based model (Denœux 1995)FusionBeliefApplications

Only ωi and Ω are focal elements, n ∗ s sources (experts)I Prototypes case (xi center of ωi). For the observation x

mij(ωi) = αij exp[−γijd2(x,xi)]mij(Ω) = 1− αij exp[−γijd2(x,xi)]

I 0 ≤ αij ≤ 1: discounting coefficient and γij > 0, areparameters to play on the quantity of ignorance and on theform of the mass functions

I The distance allows to give a mass to x higher according tothe proximity to ωi

I belief k-nn: we consider the k-nearest neighbors instead to xiI Then we combine the bbas


21/36

Estimation (6/13)

Probabilistic based modelFusionBeliefApplications

I 2 models proposed by Appriou according to three axioms:

1. the n ∗ s couples [mji , αij ] are distinct information sourceswhere focal elements are: ωi, ωi and Ω

2. If mji (ωi) = 0 and the information is valid (αij = 1) then it iscertain that ωi is not true.

3. Conformity to the Bayesian approach (case where p(Sj |ωj) isexactly the reality (αij = 1) for all i, j) and all the a prioriprobabilities p(ωi) are known)

I Need to estimate p(Sj |ωi)


22/36

Estimation (7/13)

Probabilistic based modelFusionBeliefApplications

Model 1:

mij(ωi) =αijRjp(Sj |ωj)1 +Rjp(Sj |ωj)

mij(ωi) =αij

1 +Rjp(Sj |ωj)mij(Ω) = 1− αij

with Rj ≥ 0 a normalization factor.Model 2:

mij(ωi) = 0

mij(ωi) = αij(1−Rjp(Sj |ωj))mij(Ω) = 1− αij(1−Rjp(Sj |ωj)

with Rj ∈ [0, (maxSj ,i(p(Sj |ωj)))−1]


23/36

Estimation (8/13)

Probabilistic vs DistanceFusionBeliefApplications

Difficulties:

I Appriou: learning the probabilities p(Sj |ωj)I Denœux: choice of the distance d(x,xi)

Easiness:

I p(Sj |ωj) easier to estimate on decisions with the confusionmatrix of the classifiers

I d(x,xi) easier to choose on the numeric outputs of classifiers(ex.: Euclidean distance)


24/36

Estimation (9/13)

Conjunctive rulesFusionBeliefApplications

I Assume: two cognitively independent and reliable sources S1and S2.

I The conjunctive rule is given for m1 and m2 bbas of S1 andS2, for all X ∈ 2Ω, with X 6= ∅ by:

mConj(X) =∑

Y1∩Y2=Xm1(Y1)m2(Y2) (3)

∅ ω1 ω2 ω3 Ωm1 0 0.5 0.1 0 0.4

m2 0 0.2 0 0.5 0.3

m 0.32 0.33 0.03 0.2 0.12


25/36

Combination (10/13)

Dempster’s ruleFusionBeliefApplications

I Dempster’s rule:

mD(X) =1

1− κmConj(X) (4)

where κ =∑

A∩B=∅

m1(A)m2(B) is generally called conflict or

global conflict. That is the sum of the partial conflicts.

I That is not a conflict measure (Liu, 2006).

I Conjunctive rules are not idempotent


26/36

Combination (11/13)

Decision on ΩFusionBeliefApplications

I In general the decision is made on Ω and not on 2Ω

I Pessimist: maxω∈Ω bel(ω)I Optimist: maxω∈Ω pl(ω)I Compromise: maxω∈Ω betP (ω)

Pignistic probability:

betP(ω) =∑

Y ∈2Ω,ω∩Y 6=∅

1

|Y |m(Y )

1−m(∅)(5)


27/36

Decision (12/13)

Decision on the unionsFusionBeliefApplications

I Decision on 2Ω

I The decision functions fd (belief, plausibility, pignisticprobability, etc.) increase by inclusion (Appriou, 2014)

A = argmaxX∈2Ω

(md(X)fd(X)) ,

where

md(X) =

(KdλX|X|r

)with r ∈ [0, 1], is a weighted factor of the wanted precision ofthe decision:r = 1: singleton,r = 0: ignorance


28/36

Decision (13/13)




3. Applications


29/36

Druid scientific challenges and goalsFusionBeliefApplications

Goals

1. To propose well-founded models for social networks andcrowdsourcing management

2. To develop theories for user qualification, in terms ofreliability, data provenance, skills, motivation, independence

3. To implement scalable systems that are proof-of-concepts ofthese models


30/36

(1/6)


Goals


Example

I Declarative, graph-based languages for social network analyticsI Declarative, data oriented workflows for complex crowdsourcing




30/36

(1/6)


Goals





30/36

(1/6)


Goals



Example

I Crowdsourcing quality optimization: consider the right usersI Information fusion from disagreeing users



30/36

(1/6)


Goals





30/36

(1/6)


Goals




Example

I Real-time visualization of social network analyticsI Optimized execution platforms for the crowd


30/36

(1/6)

Druid scientific foundationsFusionBeliefApplications

Users qualification through information fusion

I Combining of information coming from many users

I Taking into account reliability of users

I Managing the conflict

I Modeling of user information

I Computing user qualification

Based on uncertainty theories


31/36

(2/6)

Example 1: Influencers identificationFusionBeliefApplications

Problem: Given a social network, find a set of influencers that areable to trigger a large cascade.


32/36

(3/6)


Solution: Influencers on Twitter (Jendoubi, et al, 2016, 2017)I Define an influence measure based on belief functions by:

I Ω = {I, P} I for influencer, P for passiveI Calculate belief weights on each edge (u, v)I Integrate opinion of tweetI Combine the mass functions

I Compute influence maximisation by CELF algorithm(Leskovec et al. 2007)


33/36

(4/6)



I Ω = {I, P} I for influencer, P for passiveI Calculate belief weights on each edge (u, v)

from numbers of common neighbours, number of tweets whereu mentions v, number of tweets where v retweets from u

I Integrate opinion of tweetI Combine the mass functions



33/36

(4/6)



I Ω = {I, P} I for influencer, P for passiveI Calculate belief weights on each edge (u, v)I Integrate opinion of tweet

I Give a label to each word in the tweet using Stanford POSTagger with the model GATE Twitter part-of-speech tagger,

I Use the SentiWordNet dictionary to get the polarity of eachword in the tweet

I Build a belief function on Θ = {Pos,Neg,Neut}I Combine the mass functions



33/36

(4/6)

Example 2: Community detection: SELPFusionBeliefApplications

Initialize the bba of each node in the network

Select a node nx in VU, find its rx direct neighbors and construct rx bbas

Calculate the fused bba of node nx

Output the bba of each node

Input the labeled nodes in the graph

the maximum of mass assignment of nx is larger than a given threshold

Yes

No

move node nx from set VU to set VL

There is no node in VU

Yes

No

Semi-supervised Evidential Label Propagation algorithm (Zhou et al., 2018)


34/36

(5/6)


Example on Karate Club network

1

2

3

4

5

6

78

9

10

11

12

13

14

15

16

17

18

19

20

2122

23

24

25

26

27

28

29

30

31

32

33

34

Initialization

●

●

Labeled data in ω1Labeled data in ω2Labeled as noisy dataunlabeled data


35/36

(6/6)



1

2

3

4

5

6

78

9

10

11

12

13

14

15

16

17

18

19

20

2122

23

24

25

26

27

28

29

30

31

32

33

34

Iteration 1

●

●



35/36

(6/6)



1

2

3

4

5

6

78

9

10

11

12

13

14

15

16

17

18

19

20

2122

23

24

25

26

27

28

29

30

31

32

33

34

Iteration 2

●

●



35/36

(6/6)



1

2

3

4

5

6

78

9

10

11

12

13

14

15

16

17

18

19

20

2122

23

24

25

26

27

28

29

30

31

32

33

34

Iteration 3

●

●



35/36

(6/6)



1

2

3

4

5

6

78

9

10

11

12

13

14

15

16

17

18

19

20

2122

23

24

25

26

27

28

29

30

31

32

33

34

Iteration 4

●

●



35/36

(6/6)



1

2

3

4

5

6

78

9

10

11

12

13

14

15

16

17

18

19

20

2122

23

24

25

26

27

28

29

30

31

32

33

34

Iteration 5

●

●



35/36

(6/6)

To endFusionBeliefApplications

The theory of belief functions provides an appropriate frameworkfor information fusion.

Druid team

I Social networks

I Crowdsourcing applications

I Sensor fusion

http://www-druid.irisa.fr/


36/36

FusionBeliefApplications

Documents

[email protected] Rennes, February, 22th 2018people.irisa.fr/Arnaud.Martin/publi/presentationDGA2018.pdf · 2019. 9. 23. · Information fusion Arnaud Martin [email protected]