Embed Size (px)
Citation preview
...
1 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Maximum Entropy and Inductive Logic II
Jürgen Landes
Spring School on Inductive Logic
Canterbury, 20.04.2015 - 21.04.2015
Jürgen Landes Maximum Entropy and Inductive Logic II
...
1 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
...
2 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
...
Rational subjective BeliefsFinite propositional language LVariables v1, . . . , vn
Sentences of L, SLNo funny business, self-reference, truth predicates,self-fulfilling
Jürgen Landes Maximum Entropy and Inductive Logic II
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
3 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Probabilities
P : SL→ [0,1]
Set of probability functions PPossible world, states
ω = v1 ∧ v2 ∧ ¬v3 ∧ . . . ∧ vn1 ∧ ¬vn
P(ϕ) =∑ω∈Ωω|=ϕ
P(ω).
A probability function P ∈ P is uniquely determined by itsvalues on possible worlds, 〈P(ω) : ω ∈ Ω〉.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
4 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Inference Processes
Knowledge K leads to E ⊆ P.Formally, an inference process is a map from a set of prob-ability functions (here E) to the set of probability functions.An inference process is a map (or function)Input: EOutput: P+.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
5 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Obviously right
Adopt the function, P†, which solve this optimisation prob-lem
maximise: −∑ω∈Ω
P(ω) log(P(ω))
subject to: P ∈ E .
Shannon Entropy: H(P) = −∑
ω∈Ω P(ω) log(P(ω)).Maximum Entropy Inference Process (MaxEnt):
P+ = arg supP∈E
H(P)
Jürgen Landes Maximum Entropy and Inductive Logic II
...
5 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Obviously right
Adopt the function, P†, which solve this optimisation prob-lem
maximise: −∑ω∈Ω
P(ω) log(P(ω))
subject to: P ∈ E .
Shannon Entropy: H(P) = −∑
ω∈Ω P(ω) log(P(ω)).Maximum Entropy Inference Process (MaxEnt):
P+ = arg supP∈E
H(P)
Jürgen Landes Maximum Entropy and Inductive Logic II
...
5 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Obviously right
Adopt the function, P†, which solve this optimisation prob-lem
maximise: −∑ω∈Ω
P(ω) log(P(ω))
subject to: P ∈ E .
Shannon Entropy: H(P) = −∑
ω∈Ω P(ω) log(P(ω)).Maximum Entropy Inference Process (MaxEnt):
P+ = arg supP∈E
H(P)
Jürgen Landes Maximum Entropy and Inductive Logic II
...
6 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Internal
P+ ∈ Eif E is non-empty, convex and closed.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
6 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Internal
P+ ∈ Eif E is non-empty, convex and closed.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
7 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Open-mindedness
P+(ω) > 0, if there exists a P ∈ E such that P(ω) > 0.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
8 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Language Invariance
L′ generated by v1, v2, . . . , vn, vn+1
and the same knowledge and the same patient? For allϕ ∈ SL
P ′(ϕ) = P+(ϕ) .
Repeat argument for even larger languages.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
8 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Language Invariance
L′ generated by v1, v2, . . . , vn, vn+1
and the same knowledge and the same patient? For allϕ ∈ SL
P ′(ϕ) = P+(ϕ) .
Repeat argument for even larger languages.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
8 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Desideratum: Language Invariance
L′ generated by v1, v2, . . . , vn, vn+1
and the same knowledge and the same patient? For allϕ ∈ SL
P ′(ϕ) = P+(ϕ) .
Repeat argument for even larger languages.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
9 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
10 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
11 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Renaming
Names should not matter.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
12 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Irrelevance
Knowledge entirely irrelevant to the problem in hand can heignored.Languages: L1 with variables v1, . . . , vs, L2 with variablesvs+1, . . . , vn.2 Bodies of Knowledge: K1 formulated within L1, K2 formu-lated with in L2
For all ϕ ∈ L1 (problem at hand)
IP(v1, . . . , vn,K1)(ϕ) = IP(v1, . . . , vn,K1 ∪ K2)(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
12 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Irrelevance
Knowledge entirely irrelevant to the problem in hand can heignored.Languages: L1 with variables v1, . . . , vs, L2 with variablesvs+1, . . . , vn.2 Bodies of Knowledge: K1 formulated within L1, K2 formu-lated with in L2
For all ϕ ∈ L1 (problem at hand)
IP(v1, . . . , vn,K1)(ϕ) = IP(v1, . . . , vn,K1 ∪ K2)(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
12 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Irrelevance
Knowledge entirely irrelevant to the problem in hand can heignored.Languages: L1 with variables v1, . . . , vs, L2 with variablesvs+1, . . . , vn.2 Bodies of Knowledge: K1 formulated within L1, K2 formu-lated with in L2
For all ϕ ∈ L1 (problem at hand)
IP(v1, . . . , vn,K1)(ϕ) = IP(v1, . . . , vn,K1 ∪ K2)(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
12 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Irrelevance
Knowledge entirely irrelevant to the problem in hand can heignored.Languages: L1 with variables v1, . . . , vs, L2 with variablesvs+1, . . . , vn.2 Bodies of Knowledge: K1 formulated within L1, K2 formu-lated with in L2
For all ϕ ∈ L1 (problem at hand)
IP(v1, . . . , vn,K1)(ϕ) = IP(v1, . . . , vn,K1 ∪ K2)(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
13 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
I-AH
Jürgen Landes Maximum Entropy and Inductive Logic II
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
14 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Obstinacy
Learning something one already beliefs should not makeany difference.Given two consistent bodies of knowledge, K1,K2 (on thesame language)If IP(K1) (which is a probability function), is consistent withK2, then
IP(K1) = IP(K1 ∪ K2).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
15 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Relativisation
Suppose you know the probability of ϕ.That is, for all P ∈ E there exists some c ∈ [0,1] such thatP(ϕ) = c.If ω− |= ϕ, then IP(ω−) should not depend on yourknowledge about the ¬ϕ-worlds.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
15 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Relativisation
Suppose you know the probability of ϕ.That is, for all P ∈ E there exists some c ∈ [0,1] such thatP(ϕ) = c.If ω− |= ϕ, then IP(ω−) should not depend on yourknowledge about the ¬ϕ-worlds.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
15 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Relativisation
Suppose you know the probability of ϕ.That is, for all P ∈ E there exists some c ∈ [0,1] such thatP(ϕ) = c.If ω− |= ϕ, then IP(ω−) should not depend on yourknowledge about the ¬ϕ-worlds.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
16 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Independence
If K does not contain any information which makes v1, v2conditionally dependent on v3, then v1, v2 should be condi-tionally independent given v3.If K = P(v3) = γ,P(v1|v3) = α
γ ,P(v2|v3) = βγ (P(γ) > 0),
then
IP(K )(v1 ∧ v2|v3) =α
γ
β
γ.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
16 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Independence
If K does not contain any information which makes v1, v2conditionally dependent on v3, then v1, v2 should be condi-tionally independent given v3.If K = P(v3) = γ,P(v1|v3) = α
γ ,P(v2|v3) = βγ (P(γ) > 0),
then
IP(K )(v1 ∧ v2|v3) =α
γ
β
γ.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
17 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Continuity
If E′ can be obtained from E by moving or deforming E a bit,thenIP(E′) ≈ IP(E).For all sentences ϕ ∈ SL: IP(E′)(ϕ) ≈ IP(E)(ϕ).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
17 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Continuity
If E′ can be obtained from E by moving or deforming E a bit,thenIP(E′) ≈ IP(E).For all sentences ϕ ∈ SL: IP(E′)(ϕ) ≈ IP(E)(ϕ).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
17 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Continuity
If E′ can be obtained from E by moving or deforming E a bit,thenIP(E′) ≈ IP(E).For all sentences ϕ ∈ SL: IP(E′)(ϕ) ≈ IP(E)(ϕ).
Jürgen Landes Maximum Entropy and Inductive Logic II
...
18 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
19 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
DesiderataThe Classical Result
Alena & Jeff
TheoremIf E is closed, convex and non-empty, IP satisfies Renaming,Irrelevance, Obstinacy, Relativisation, Independence andContinuity, then IP is MaxEnt.MaxEnt satisfies language invariance and open-mindednessand is internal.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
20 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
21 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
22 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Continuity - Bristolean
One considers the decision problem of setting degrees ofbeliefand wonders which beliefs are best.A utility function u is used to measure the goodness / bad-ness / utility of a belief function.Determine which beliefs have the “best utility”.The usual caveats for decision making apply: Non-causal,act-state independence, etc.Decision theoretic norm still undetermined.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
23 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
24 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Logarithmic Utility
If ω is the true world, then the utility is u(ω,P+) = log(P+(ω)).Expected utility for P ∈ E is:
∑ω∈Ω P(ω) log(P+(ω)).
If E is closed, convex and non-empty, maximise worst caseexpected utility:
arg supP+∈P
infP∈E
∑ω∈Ω
P(ω) log(P+(ω)) = P† .
The latest rage is to understand u as an accuracy measure.Measure closeness to the truth.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
25 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Decision MakingJustification
Utility Theory
If you do have a utility function and a Decision TheoreticNorm,then you can show that a particular inference process isoptimal with respect to the above.Justifications in terms of common-sense principles hinge onthe common-sensicality of the principles.Justifications in terms of utility functions appear much moreobjective.However, one has to give a story explaining where theutility function and the DTN come from.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
26 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
27 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Ensembles, populations
Consider an entire population M with |M| members.K consists of statements of the form32.2% of patients complain of symptom ϕ
|x ∈ M | x complains about ϕ| ≈ 32.2100|M|
Consider all populations M of fixed large size |M| which sat-isfy the above.Then almost all such populations M satisfy
|x ∈ M | x complains about ϕ||M|
≈ P†(ϕ) .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
28 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Alena & Jeff 2 – one last justification
TheoremFor all δ > 0 there exists some natural number M0 such that forall sentences ϕ ∈ SL and all fixed sizes of populations|M| ≥ M0 the proportion of populations M of fixed size |M| ≥ M0which satisfy∣∣∣ |x ∈ M | x complains about ϕ|
|M|− P†(ϕ)
∣∣∣ < δ
is greater or equal than 1− δ.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
29 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
30 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
31 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Equivocation, come what may
The least opinionated function isP=(ω) = 1
|Ω| , P= = arg supP∈E H(P).
If P= ∈ E, then P† = P=.Entropy strictly decreases along the rays originating fromP=.Entropy maximiser different from P= face P=.If E = P ∈ P | 0 ≤ P(v1) ≤ 0.5, then P†(v1) = 0.5.Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
32 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Donkey
If E = P, then P† = P=.But what if your learn this?Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
32 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Donkey
If E = P, then P† = P=.But what if your learn this?Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
32 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Donkey
If E = P, then P† = P=.But what if your learn this?Can this be right?
Jürgen Landes Maximum Entropy and Inductive Logic II
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
33 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Intuition?!?
Maximising−∑
ω∈Ω P(ω) log(P(ω)) is sooo counter-intuitive.How would you(!) respond to this objection?
If the aim is to reconstruct “rational” human thinking, thenMaxEnt fails; I claim.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
34 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
35 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Dependence
No knowledge, E = P.Possible worlds: red and blue.P†(red) = 1
2 . Okay.Possible worlds: red and light blue and dark blue.P†(red) = 1
3 . Ohho!This phenomenon is called language dependence.It is quite common.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
36 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Invariance
L = 〈v1, v2, . . . , vn〉, L′ = 〈v1, v2, . . . , vn, vn+1〉.Knowledge only concerns v1, . . . , vn.ϕ ∈ SL.
IP(L)(ϕ) = IP(L′)(ϕ)
Centre of Mass is not language invariant! MaxEnt is lan-guage invariant.Ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
37 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Outline
1 Recap2 The Classical Derivation
DesiderataThe Classical Result
3 A Modern ApproachDecision MakingJustification
4 Universes5 Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Jürgen Landes Maximum Entropy and Inductive Logic II
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
...
38 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
MaxEntW
MaxEntW : An agent ought to equivocate (sufficiently) be-tween the basic possibilities that she can express.Language now contains the chance functions P.Density functions:
C1E := f1 : E→ [0,1] :
∫P∈E
f1(P)P dp ∈ P.
“Likelihood of a chance function”MaxEntW : Pick density with greatest entropy
Jürgen Landes Maximum Entropy and Inductive Logic II
...
39 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Level One
Entropy: H(f1) := −∫
P∈E f1(P) · log(f1(P))dp.
Let f †1 be the density in C1E with maximal entropy.
Pick a probability function P+ :∫
P∈E f †1 (P)PdP.P+ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
39 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Level One
Entropy: H(f1) := −∫
P∈E f1(P) · log(f1(P))dp.
Let f †1 be the density in C1E with maximal entropy.
Pick a probability function P+ :∫
P∈E f †1 (P)PdP.P+ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
39 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Level One
Entropy: H(f1) := −∫
P∈E f1(P) · log(f1(P))dp.
Let f †1 be the density in C1E with maximal entropy.
Pick a probability function P+ :∫
P∈E f †1 (P)PdP.P+ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
39 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Level One
Entropy: H(f1) := −∫
P∈E f1(P) · log(f1(P))dp.
Let f †1 be the density in C1E with maximal entropy.
Pick a probability function P+ :∫
P∈E f †1 (P)PdP.P+ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
40 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Levels
Density functions for agents with richer languages
Cn+1E := fn+1 : E→ [0,1] :
∫fn∈Cn
E
fn+1(fn)fn dfn ∈ CnE.
Entropy: H(fn+1) := −∫
fn∈CnE
fn+1(fn) · log(fn+1(fn))dfn.
Let f †n+1 be the density in Cn+1E with maximal entropy (it is
flat).Pick a n density f +
n :∫
f∈CnE
f †n+1(fn)fndfn.
Eventually this determines some P++ ∈ E.P++ = PCoM .
Jürgen Landes Maximum Entropy and Inductive Logic II
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
41 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Density Invariance
Equivocating over a density level (≥ 1) leads to centre ofmass, regardless of the level.Density InvarianceEquivocating over P gives MaxEnt!The level matters.Ay, ay, caramba!
Jürgen Landes Maximum Entropy and Inductive Logic II
...
42 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
References I
Csiszár, I. (1991).Why Least Squares and Maximum Entropy? An AxiomaticApproach to Inference for Linear Inverse Problems.The Annals of Statistics, 19(4):2032–2066.
Grünwald, P. D. and Dawid, A. (2004).Game theory, maximum entropy, minimum discrepancyand robust Bayesian decision theory.Annals of Statistics, 32(4):1367–1433.
Jaynes, E. (1957).Information Theory and Statistical Mechanics.Physical Review, 106(4):620–630.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
43 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
References II
Jaynes, E. T. (2003).Probability Theory: The Logic of Science.Cambridge University Press.
Paris, J. B. (1998).Common Sense and Maximum Entropy.Synthese, 117:75–93.
Paris, J. B. (2006).The Uncertain Reasoner’s Companion: A MathematicalPerspective, volume 39 of Cambridge Tracts in TheoreticalComputer Science.Cambridge University Press, 2 edition.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
44 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
References III
Paris, J. B. and Vencovská, A. (1989).On the applicability of maximum entropy to inexactreasoning.International Journal of Approximate Reasoning,3(1):1–34.
Paris, J. B. and Vencovská, A. (1990).A note on the inevitability of maximum entropy.International Journal of Approximate Reasoning,4(3):183–223.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
45 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
References IV
Paris, J. B. and Vencovská, A. (1997).In Defense of the Maximum Entropy Inference Process.International Journal of Approximate Reasoning,17(1):77–103.
Williamson, J. (2010).In Defence of Objective Bayesianism.Oxford University Press.
Jürgen Landes Maximum Entropy and Inductive Logic II
...
46 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
That’s it. Thank you! Questions? – Progic Tomorrow
Jürgen Landes Maximum Entropy and Inductive Logic II
...
47 / 47
RecapThe Classical Derivation
A Modern ApproachUniverses
Objections, your Honor!
Bug or Feature?Issues of LanguageRisqué
Logarithmic Utility
Intuitive human sensations tend to be logarithmic functionsof the stimulus. – JaynesSavageJon’s L1 – L4
Jürgen Landes Maximum Entropy and Inductive Logic II
